Encoding method, decoding method, encoding device, and decoding device

Abstract

The encoding method includes: encoding first reference information for a first vertex set in a first three-dimensional mesh frame and second reference information for a second vertex set in the first three-dimensional mesh frame into a bitstream (S301); encoding the first vertex set into a bit stream (S302); and encoding the second vertex set in the bit stream (S303), the first reference information indicating a first value when a third vertex set in a second three-dimensional mesh frame temporally different from the first three-dimensional mesh frame is used in encoding the first vertex set, and the second reference information indicating a second value when a fourth vertex set in the first three-dimensional mesh frame is used in encoding the second vertex set.

Description

Technical Field

[0001] The present disclosure relates to encoding methods and the like. Background Art

[0002] Patent Document 1 proposes a method and apparatus for encoding and decoding three-dimensional mesh data.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2006-187015 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] It is desirable to further improve encoding processing, etc. related to three-dimensional data. An object of the present disclosure is to improve encoding processing, etc. related to three-dimensional data.

[0008] Means used to solve problems

[0009] An encoding method related to one embodiment of the present disclosure includes: encoding first reference information for a first vertex set in a first three-dimensional mesh frame and second reference information for a second vertex set in the first three-dimensional mesh frame into a bitstream; encoding the first vertex set into the bitstream; and encoding the second vertex set into the bitstream, wherein when a third vertex set in a second three-dimensional mesh frame that is temporally different from the first three-dimensional mesh frame is used in encoding the first vertex set, the first reference information represents a first value, and when a fourth vertex set in the first three-dimensional mesh frame is used in encoding the second vertex set, the second reference information represents a second value.

[0010] In addition, these general or specific methods can be implemented by systems, devices, methods, integrated circuits, computer programs, or non-temporary recording media such as computer-readable CD-ROMs, or by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0011] Effects of the Invention

[0012] The present disclosure can contribute to improving encoding processing related to three-dimensional data, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a conceptual diagram showing a three-dimensional mesh according to the embodiment.

[0014] Figure 2 This is a conceptual diagram showing basic elements of a three-dimensional mesh according to an embodiment.

[0015] Figure 3 This is a conceptual diagram showing a mapping related to the embodiment.

[0016] Figure 4 This is a block diagram showing a configuration example of a coding / decoding system according to an embodiment.

[0017] Figure 5 This is a block diagram showing a configuration example of an encoding device according to an embodiment.

[0018] Figure 6 This is a block diagram showing another configuration example of the encoding device according to the embodiment.

[0019] Figure 7 This is a block diagram showing a configuration example of a decoding device according to an embodiment.

[0020] Figure 8 This is a block diagram showing another configuration example of a decoding device according to the embodiment.

[0021] Figure 9 This is a conceptual diagram showing an example of the structure of a bit stream related to the embodiment.

[0022] Figure 10 This is a conceptual diagram showing another example of the configuration of a bit stream according to the embodiment.

[0023] Figure 11 This is a conceptual diagram showing yet another example of the configuration of a bit stream according to the embodiment.

[0024] Figure 12 This is a block diagram showing a specific example of a coding and decoding system according to the embodiment.

[0025] Figure 13 This is a conceptual diagram showing a configuration example of point cloud data according to an embodiment.

[0026] Figure 14 This is a conceptual diagram showing an example of a data file of point cloud data according to an embodiment.

[0027] Figure 15 This is a conceptual diagram showing an example of the structure of mesh data according to the embodiment.

[0028] Figure 16 This is a conceptual diagram showing an example of a data file of mesh data according to the embodiment.

[0029] Figure 17 This is a conceptual diagram showing the types of three-dimensional data related to the embodiment.

[0030] Figure 18 This is a block diagram showing a configuration example of a three-dimensional 1-data encoder according to an embodiment.

[0031] Figure 19 This is a block diagram showing a configuration example of a three-dimensional data decoder according to an embodiment.

[0032] Figure 20 This is a block diagram showing another configuration example of the three-dimensional data encoder according to the embodiment.

[0033] Figure 21 This is a block diagram showing another configuration example of a three-dimensional data decoder according to the embodiment.

[0034] Figure 22 This is a conceptual diagram showing a specific example of encoding processing related to the embodiment.

[0035] Figure 23 This is a conceptual diagram showing a specific example of decoding processing related to the embodiment.

[0036] Figure 24 This is a block diagram showing an example of implementation of an encoding device according to the embodiment.

[0037] Figure 25 This is a block diagram showing an implementation example of a decoding device according to the embodiment.

[0038] Figure 26 This is a block diagram showing the architecture of a coding and decoding system according to an embodiment.

[0039] Figure 27 This is a block diagram showing an example of the architecture of an encoding device according to an embodiment.

[0040] Figure 28 This is a block diagram showing another example of the architecture of the encoding device according to the embodiment.

[0041] Figure 29 This is a block diagram showing an example of the architecture of a decoding device according to an embodiment.

[0042] Figure 30 This is a block diagram showing another example of the architecture of a decoding device according to an embodiment.

[0043] Figure 31 This is a block diagram showing yet another example of the architecture of the decoding device according to the embodiment.

[0044] Figure 32 This is a conceptual diagram showing an example of subdivision according to the embodiment.

[0045] Figure 33 This is a block diagram showing yet another example of the architecture of the decoding device according to the embodiment.

[0046] Figure 343D mesh encoding processing according to an embodiment of the present invention is shown in FIG.

[0047] Figure 35 This is a conceptual diagram showing an example of reference information related to the embodiment.

[0048] Figure 36 This is a conceptual diagram showing an example of a three-dimensional area according to the embodiment.

[0049] Figure 37 This is a conceptual diagram showing an example of a vertex set located inside a rectangular parallelepiped according to the embodiment.

[0050] Figure 38 This is a conceptual diagram showing an example of the relationship between the differences between the first vertex set and the third vertex set according to the embodiment.

[0051] Figure 39 This is a conceptual diagram showing an example of the relationship between the differences between the second vertex set and the fourth vertex set according to the embodiment.

[0052] Figure 40 This is a conceptual diagram showing an example of how a plurality of vertices of a first three-dimensional mesh frame according to the embodiment are reconstructed.

[0053] Figure 41 This is a conceptual diagram showing an example of application of connection information related to the embodiment.

[0054] Figure 42 This is a block diagram showing a configuration example of an encoding device that switches between intra prediction and inter prediction for each vertex set according to an embodiment.

[0055] Figure 43 This is a block diagram showing another configuration example of an encoding device that switches between intra prediction and inter prediction for each vertex set according to the embodiment.

[0056] Figure 44 3D mesh decoding processing according to an embodiment of the present invention is shown in FIG.

[0057] Figure 45 This is a block diagram showing a configuration example of a decoding device that switches between intra prediction and inter prediction for each vertex set according to an embodiment.

[0058] Figure 46 This is a block diagram showing another configuration example of a decoding device that switches between intra prediction and inter prediction for each vertex set according to the embodiment.

[0059] Figure 47 This is a conceptual diagram showing a first example of a method for specifying a vertex set for inter-frame prediction according to an embodiment.

[0060] Figure 48This is a syntax diagram showing a syntax structure corresponding to the first example of the method of specifying a vertex set for inter prediction according to the embodiment.

[0061] Figure 49 This is a conceptual diagram showing a second example of a method for specifying a vertex set for inter-frame prediction according to the embodiment.

[0062] Figure 50 This is a syntax diagram showing a syntax structure corresponding to the second example of the method of specifying a vertex set for inter prediction according to the embodiment.

[0063] Figure 51 This is a syntax diagram showing a modified example of the syntax structure corresponding to the second example of the method of specifying a vertex set for inter prediction according to the embodiment.

[0064] Figure 52 This is a conceptual diagram showing a third example of a method for specifying a vertex set for inter-frame prediction according to the embodiment.

[0065] Figure 53 This is a syntax diagram showing a syntax structure corresponding to the third example of the method of specifying a vertex set for inter prediction according to the embodiment.

[0066] Figure 54 This is a conceptual diagram two-dimensionally representing a vertex set specified in the third example of the method of specifying a vertex set for inter prediction according to the embodiment.

[0067] Figure 55 This is a conceptual diagram showing a fourth example of a method for specifying a vertex set for inter-frame prediction according to the embodiment.

[0068] Figure 56 This is a syntax diagram showing a syntax structure corresponding to the fourth example of the method of specifying a vertex set for inter prediction according to the embodiment.

[0069] Figure 57 This is a conceptual diagram two-dimensionally representing a vertex set specified in the fourth example of the method of specifying a vertex set for inter prediction according to the embodiment.

[0070] Figure 58 This is a conceptual diagram showing a fifth example of a method for specifying a vertex set for inter-frame prediction according to the embodiment.

[0071] Figure 59 This is a syntax diagram showing a syntax structure corresponding to the fifth example of the method of specifying a vertex set for inter prediction according to the embodiment.

[0072] Figure 60 This is a flowchart showing an example of basic encoding processing related to the embodiment.

[0073] Figure 61 This is a flowchart showing an example of basic decoding processing related to the embodiment.

[0074] Figure 62 This is a block diagram showing yet another configuration example of the encoding device according to the embodiment.

[0075] Figure 63 This is a block diagram showing yet another configuration example of the decoding device according to the embodiment. DETAILED DESCRIPTION

[0076] <Introduction>

[0077] For example, three-dimensional meshes are used in computer graphics. For example, a computer graphics image can be composed of multiple frames that are different in time, and each frame is represented by a three-dimensional mesh. Frames represented by a three-dimensional mesh are also represented as three-dimensional mesh frames.

[0078] Furthermore, a three-dimensional mesh is composed of vertex information representing the positions of multiple vertices in three-dimensional space, connection information representing the connection relationships between the multiple vertices, and attribute information representing the attributes of each vertex or face. Each face is constructed based on the connection relationships between the multiple vertices. This three-dimensional mesh can be used to represent a variety of computer graphics images. Here, a vertex sometimes refers to the vertex information of that vertex. Furthermore, a vertex set is a collection of one or more vertices and sometimes refers to the vertex information of the one or more vertices.

[0079] Furthermore, efficient encoding and decoding of 3D meshes is desired for transmission and storage. For example, to efficiently encode vertices, an encoding device uses previously encoded vertices to encode the vertices of the encoding target. Specifically, when encoding the vertices of the encoding target, the encoding device uses previously encoded vertices to predict the vertices of the encoding target and encodes the differences between the predicted vertices and the vertices of the encoding target, thereby reducing the amount of code.

[0080] For example, when inter-frame prediction is used for the 3D mesh frame of the encoding target, the vertices of the encoding target are encoded using the vertices in the already encoded 3D mesh frame. Alternatively, when intra-frame prediction is used for the 3D mesh frame of the encoding target, the vertices of the encoding target are encoded using the already encoded vertices in the 3D mesh frame of the encoding target.

[0081] This makes it possible to adaptively switch between inter-frame prediction and intra-frame prediction for each three-dimensional grid frame, thereby reducing the amount of code.

[0082] However, the same three-dimensional mesh frame may contain vertices for which encoding is more efficient using inter-frame prediction and vertices for which encoding is more efficient using intra-frame prediction, and thus the code amount may not be sufficiently reduced.

[0083] Therefore, the encoding method of Example 1 includes: encoding first reference information for a first vertex set in a first three-dimensional mesh frame and second reference information for a second vertex set in the first three-dimensional mesh frame into a bitstream; encoding the first vertex set into the bitstream; and encoding the second vertex set into the bitstream, wherein when a third vertex set in a second three-dimensional mesh frame that is temporally different from the first three-dimensional mesh frame is used in encoding the first vertex set, the first reference information represents a first value, and when a fourth vertex set in the first three-dimensional mesh frame is used in encoding the second vertex set, the second reference information represents a second value.

[0084] Therefore, when encoding the first vertex set and the second vertex set in the same 3D mesh frame, inter-frame prediction may be applied to the first vertex set and intra-frame prediction may be applied to the second vertex set, thereby reducing the amount of code.

[0085] In addition, the encoding method of Example 2 may also be that, in the encoding method of Example 1, when the third vertex set is used in encoding the first vertex set, the first vertex set is encoded using connection information of the three-dimensional mesh in the second three-dimensional mesh frame.

[0086] Therefore, in inter-frame prediction, the connection information can sometimes be used. Therefore, in inter-frame prediction, the encoding of the connection information can sometimes be omitted. Therefore, in inter-frame prediction, the code amount of the connection information can sometimes be reduced.

[0087] In addition, the encoding method of Example 3 may also be that, in the encoding method of Example 1, regardless of whether the third vertex set is used in the encoding of the first vertex set and whether the fourth vertex set is used in the encoding of the second vertex set, the first vertex set and the second vertex set are encoded using the connection information of the three-dimensional mesh in the second three-dimensional mesh frame.

[0088] Therefore, whether inter-frame prediction or intra-frame prediction, the connection information can sometimes be used. Therefore, whether inter-frame prediction or intra-frame prediction, the encoding of the connection information can sometimes be omitted. Therefore, whether inter-frame prediction or intra-frame prediction, the code size of the connection information can sometimes be reduced.

[0089] Furthermore, the encoding method of Example 4 may be any one of the encoding methods of Examples 1 to 3, wherein the first reference information and the second reference information each represent a value for identifying a three-dimensional mesh frame as a reference target.

[0090] This may allow efficient designation of a three-dimensional mesh frame as a reference target.

[0091] Furthermore, the encoding method of Example 5 may be any one of the encoding methods of Examples 1 to 4, wherein the first reference information and the second reference information each indicate, as a value, whether or not to refer to the second three-dimensional mesh frame.

[0092] This makes it possible to efficiently specify whether to use inter-frame prediction.

[0093] Furthermore, the encoding method of Example 6 may be any one of the encoding methods of Examples 1 to 5, wherein the first reference information and the second reference information each indicate, as a value, whether or not to refer to the first three-dimensional mesh frame.

[0094] This makes it possible to efficiently specify whether to use intra-frame prediction.

[0095] Furthermore, the encoding method of Example 7 may be any one of the encoding methods of Examples 1 to 6, wherein the first three-dimensional mesh frame is a three-dimensional mesh frame to be encoded.

[0096] This may allow efficient encoding of each vertex set in the three-dimensional mesh frame to be encoded.

[0097] Furthermore, the encoding method of Example 8 may be any one of the encoding methods of Examples 1 to 7, wherein the second three-dimensional mesh frame is an already encoded three-dimensional mesh frame.

[0098] Therefore, when inter-frame prediction is used for encoding the first vertex set, it may be possible to efficiently encode the first vertex set using an already encoded three-dimensional mesh frame.

[0099] In addition, the decoding method of Example 9 includes: decoding first reference information for a first vertex set in a first three-dimensional mesh frame and second reference information for a second vertex set in the first three-dimensional mesh frame from a bit stream; when the first reference information represents a first value, decoding the first vertex set from the bit stream using a third vertex set in a second three-dimensional mesh frame that is temporally different from the first three-dimensional mesh frame; and when the second reference information represents a second value, decoding the second vertex set from the bit stream using a fourth vertex set in the first three-dimensional mesh frame.

[0100] Thus, when decoding the first vertex set and the second vertex set in the same 3D mesh frame, inter-frame prediction may be applied to the first vertex set and intra-frame prediction may be applied to the second vertex set, thereby sometimes reducing the amount of code.

[0101] Furthermore, the decoding method of Example 10 may be such that, in the decoding method of Example 9, when the first reference information indicates the first value, the first vertex set is decoded using connection information of the three-dimensional mesh in the second three-dimensional mesh frame.

[0102] Therefore, in inter-frame prediction, the connection information can sometimes be used. Therefore, in inter-frame prediction, decoding of the connection information can sometimes be omitted. Therefore, in inter-frame prediction, the code amount of the connection information can sometimes be reduced.

[0103] In addition, the decoding method of Example 11 may also be that, in the decoding method of Example 9, regardless of whether the first reference information represents the first value or the second value, and whether the second reference information represents the first value or the second value, the first vertex set and the second vertex set are decoded using the connection information of the three-dimensional mesh in the second three-dimensional mesh frame.

[0104] Therefore, whether it is inter-frame prediction or intra-frame prediction, the connection information can sometimes be used. Therefore, whether it is inter-frame prediction or intra-frame prediction, the decoding of the connection information can sometimes be omitted. Therefore, whether it is inter-frame prediction or intra-frame prediction, the code size of the connection information can sometimes be reduced.

[0105] Furthermore, the decoding method of Example 12 may be any one of the decoding methods of Examples 9 to 11, wherein the first reference information and the second reference information each represent a value for identifying a three-dimensional mesh frame as a reference target.

[0106] This may allow efficient designation of a three-dimensional mesh frame as a reference target.

[0107] Furthermore, the decoding method of Example 13 may be any one of the decoding methods of Examples 9 to 12, wherein the first reference information and the second reference information each indicate, as a value, whether or not to refer to the second three-dimensional mesh frame.

[0108] This makes it possible to efficiently specify whether to use inter-frame prediction.

[0109] Furthermore, the decoding method of Example 14 may be any one of the decoding methods of Examples 9 to 13, wherein the first reference information and the second reference information each indicate, as a value, whether or not to refer to the first three-dimensional mesh frame.

[0110] This makes it possible to efficiently specify whether to use intra-frame prediction.

[0111] Furthermore, the decoding method of Example 15 may be any one of the decoding methods of Examples 9 to 14, wherein the first three-dimensional mesh frame is a three-dimensional mesh frame to be decoded.

[0112] This may allow each vertex set in the three-dimensional mesh frame to be decoded to be efficiently decoded.

[0113] Furthermore, the decoding method of Example 16 may be any one of the decoding methods of Examples 9 to 15, wherein the second three-dimensional mesh frame is a decoded three-dimensional mesh frame.

[0114] Therefore, when inter-frame prediction is used for decoding the first vertex set, the first vertex set may be efficiently decoded using the decoded three-dimensional mesh frame.

[0115] In addition, the encoding device of Example 17 includes a memory and a circuit capable of accessing the memory, and the circuit, when in operation, performs: encoding first reference information for a first vertex set in a first three-dimensional mesh frame, and second reference information for a second vertex set in the first three-dimensional mesh frame into a bit stream; encoding the first vertex set into the bit stream; and encoding the second vertex set into the bit stream, wherein when a third vertex set in a second three-dimensional mesh frame that is temporally different from the first three-dimensional mesh frame is used in encoding the first vertex set, the first reference information represents a first value, and when a fourth vertex set in the first three-dimensional mesh frame is used in encoding the second vertex set, the second reference information represents a second value.

[0116] Therefore, when encoding the first vertex set and the second vertex set in the same 3D mesh frame, inter-frame prediction may be applied to the first vertex set and intra-frame prediction may be applied to the second vertex set, thereby sometimes reducing the amount of code.

[0117] In addition, the decoding device of Example 18 includes a memory and a circuit capable of accessing the memory, and the circuit performs the following operations: decoding first reference information for a first vertex set in a first three-dimensional mesh frame and second reference information for a second vertex set in the first three-dimensional mesh frame from a bit stream; when the first reference information represents a first value, decoding the first vertex set from the bit stream using a third vertex set in a second three-dimensional mesh frame that is temporally different from the first three-dimensional mesh frame; and when the second reference information represents a second value, decoding the second vertex set from the bit stream using a fourth vertex set in the first three-dimensional mesh frame.

[0118] Therefore, when decoding the first vertex set and the second vertex set in the same 3D mesh frame, inter-frame prediction may be applied to the first vertex set and intra-frame prediction may be applied to the second vertex set, thereby sometimes reducing the amount of code.

[0119] Furthermore, these general or specific methods can be implemented by systems, devices, methods, integrated circuits, computer programs, or non-temporary recording media such as computer-readable CD-ROMs, or by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0120] <Expression and Terminology>

[0121] Here, the following expressions and terms are used.

[0122] (1) 3D grid

[0123] A 3D mesh is a collection of multiple faces, representing, for example, a 3D object. Furthermore, a 3D mesh primarily consists of vertex information, connectivity information, and attribute information. A 3D mesh is sometimes represented as a polygonal mesh or a mesh. Furthermore, a 3D mesh can also vary over time. A 3D mesh can contain metadata related to vertex information, connectivity information, and attribute information, as well as other additional information.

[0124] (2) Vertex information

[0125] Vertex information is information that represents a vertex. For example, vertex information indicates the position of a vertex in three-dimensional space. Furthermore, vertices correspond to the vertices of the faces that make up a three-dimensional mesh. Vertex information is sometimes referred to as "geometric information." Furthermore, vertex information is sometimes referred to as positional information.

[0126] (3) Connection information

[0127] Connectivity information is information that indicates the connections between vertices. For example, connectivity information indicates the connections between faces or edges that form a three-dimensional mesh. Connectivity information is sometimes expressed as "connectivity" or as face information.

[0128] (4) Attribute information

[0129] Attribute information is information that represents the attributes of a vertex or face. For example, attribute information represents attributes such as color, image, and normal vector associated with a vertex or face. Attribute information is sometimes referred to as "texture."

[0130] (5) Surface

[0131] Faces are elements that make up a 3D mesh. Specifically, a face is a polygon on a plane in 3D space. For example, a face can be a triangle in 3D space.

[0132] (6) Plane

[0133] A plane is a two-dimensional surface in a three-dimensional space. For example, a polygon is formed on a plane, and multiple polygons are formed on multiple planes.

[0134] (7) Bitstream

[0135] A bitstream corresponds to the encoded information. A bitstream can also be expressed as a stream, a coded bitstream, a compressed bitstream, or a coded signal.

[0136] (8) Encoding and decoding

[0137] The term "encode" can be replaced by expressions such as "store," "contain," "write," "record," "signal," "transmit," "notify," "save," or "compress," and these expressions can also be replaced with each other. For example, "encode information" can also mean "include the information in a bitstream." Furthermore, "encode information into a bitstream" can also mean "encode the information to generate a bitstream containing the encoded information."

[0138] Furthermore, the term "decode" can be replaced by "read out," "interpret," "read in," "derive," "obtain," "accept," "extract," "restore," "reconstruct," "decompress," or "expand," and these expressions can also be used interchangeably. For example, "decoding information" can also mean "obtaining information from a bitstream." Furthermore, "decoding information from a bitstream" can also mean decoding the bitstream to obtain the information contained in the bitstream.

[0139] (9) Ordinal numbers

[0140] In the description, constituent elements may be assigned ordinal numbers such as 1 and 2. These ordinal numbers may be changed as appropriate. Furthermore, constituent elements may be assigned new ordinal numbers or removed. Furthermore, these ordinal numbers may be assigned to elements for identification purposes and may not correspond to a meaningful order.

[0141] <3D Grid>

[0142] Figure 1 This is a conceptual diagram illustrating a three-dimensional mesh related to this embodiment. A three-dimensional mesh is composed of multiple faces. For example, each face is a triangle. The vertices of these triangles are defined in three-dimensional space. Furthermore, the three-dimensional mesh represents a three-dimensional object. Each face may also have a color or image.

[0143] Figure 2 This is a conceptual diagram showing the basic elements of a three-dimensional mesh in the embodiments. A three-dimensional mesh consists of vertex information, connection information, and attribute information. Vertex information indicates the positions of vertices on a face in three-dimensional space. Connection information indicates the connections between vertices. Faces can be identified using vertex and connection information. In other words, vertex and connection information form a colorless three-dimensional object in three-dimensional space.

[0144] Attribute information can be associated with either vertices or faces. Vertex-associated attribute information is sometimes expressed as "Attribute Per Point." Vertex-associated attribute information can represent both the vertex itself and the faces connected to it.

[0145] For example, color can be associated with a vertex as attribute information. The color associated with a vertex can be the vertex's color or the color of the face connected to the vertex. The face's color can also be the average of multiple colors associated with multiple vertices of the face. Furthermore, a normal vector can be associated with a vertex or face as attribute information. Such a normal vector can represent the front and back sides of a face.

[0146] Alternatively, a two-dimensional image can be associated with a surface as attribute information. A two-dimensional image associated with a surface is also represented as a texture image or an "Attribute Map." Furthermore, information indicating the mapping between the surface and the two-dimensional image can also be associated with the surface as attribute information. This information indicating the mapping is sometimes represented as mapping information, vertex information of a texture image, or "Attribute UV Coordinate."

[0147] Furthermore, information such as color, image, and moving image used as attribute information may be expressed as "Parametric Space: parameter space".

[0148] This attribute information allows texture to be reflected in a three-dimensional object. That is, vertex information, connection information, and attribute information can be used to create a three-dimensional object with color in three-dimensional space.

[0149] In the above description, attribute information is associated with vertices or faces, but may also be associated with edges.

[0150] Figure 3 This is a conceptual diagram illustrating mapping related to this embodiment. For example, a region of a two-dimensional image in a two-dimensional plane can be mapped onto a surface of a three-dimensional mesh in three-dimensional space. Specifically, the coordinate information of the region in the two-dimensional image is associated with the surface of the three-dimensional mesh. This allows the image of the region mapped in the two-dimensional image to be reflected on the surface of the three-dimensional mesh.

[0151] By using mapping, the two-dimensional image used as attribute information can be separated from the three-dimensional mesh. For example, in encoding the three-dimensional mesh, the two-dimensional image can also be encoded using an image encoding method or a video encoding method.

[0152] <System Configuration>

[0153] Figure 4 This is a block diagram showing an example of the configuration of a coding and decoding system according to this embodiment. Figure 4 In the embodiment, the encoding and decoding system includes an encoding device 100 and a decoding device 200.

[0154] For example, encoding device 100 obtains a 3D mesh and encodes it into a bitstream. Encoding device 100 then outputs the bitstream to network 300. For example, the bitstream includes the encoded 3D mesh and control information used to decode the encoded 3D mesh. By encoding the 3D mesh, the 3D mesh information is compressed.

[0155] Network 300 transmits the bitstream from encoding device 100 to decoding device 200. Network 300 may be the Internet, a wide area network (WAN), a local area network (LAN), or a combination thereof. Network 300 is not necessarily limited to bidirectional communication and may also be a unidirectional communication network used for terrestrial digital broadcasting or satellite broadcasting.

[0156] In addition, the network 300 may be replaced by a recording medium such as a DVD (Digital Versatile Disc) or a BD (Blu-Ray Disc (registered trademark)).

[0157] Decoding device 200 obtains a bitstream and decodes a three-dimensional mesh from the bitstream. By decoding the three-dimensional mesh, information about the three-dimensional mesh is expanded. For example, decoding device 200 decodes the three-dimensional mesh using a decoding method corresponding to the encoding method used by encoding device 100 to encode the three-dimensional mesh. In other words, encoding device 100 and decoding device 200 perform encoding and decoding using corresponding encoding and decoding methods.

[0158] In addition, the 3D mesh before encoding can also be expressed as the original 3D mesh. In addition, the 3D mesh after decoding can also be expressed as the reconstructed 3D mesh.

[0159] <Encoding device>

[0160] Figure 5 1 is a block diagram showing a configuration example of the encoding device 100 according to this embodiment. For example, the encoding device 100 includes a vertex information encoder 101 , a connection information encoder 102 , and an attribute information encoder 103 .

[0161] The vertex information encoder 101 is an electrical circuit that encodes vertex information. For example, the vertex information encoder 101 encodes the vertex information into a bit stream in a format specified for the vertex information.

[0162] The connection information encoder 102 is an electrical circuit that encodes the connection information. For example, the connection information encoder 102 encodes the connection information into a bit stream in a format specified for the connection information.

[0163] The attribute information encoder 103 is an electrical circuit that encodes attribute information. For example, the attribute information encoder 103 encodes the attribute information into a bit stream in a format specified for the attribute information.

[0164] In encoding vertex information, connection information, and attribute information, either variable-length coding or fixed-length coding can be used. Variable-length coding can also correspond to Huffman coding or context-adaptive binary arithmetic coding (CABAC).

[0165] The vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be integrated. Alternatively, the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be further subdivided into a plurality of components.

[0166] Figure 6 This is a block diagram showing another configuration example of the encoding device 100 according to this embodiment. Figure 5 In addition to the illustrated configuration, a pre-processor 104 and a post-processor 105 are further provided.

[0167] The pre-processor 104 is an electrical circuit that processes vertex information, connectivity information, and attribute information before encoding. For example, the pre-processor 104 may transform, separate, or multiplex the pre-encoded 3D mesh. More specifically, the pre-processor 104 may separate vertex information, connectivity information, and attribute information from the pre-encoded 3D mesh.

[0168] The post-processor 105 is an electrical circuit that processes the encoded vertex information, connection information, and attribute information. For example, the post-processor 105 may transform, separate, or multiplex the encoded vertex information, connection information, and attribute information. More specifically, the post-processor 105 may multiplex the encoded vertex information, connection information, and attribute information into a bitstream. Furthermore, the post-processor 105 may further perform variable-length coding on the encoded vertex information, connection information, and attribute information.

[0169] <Decoding Device>

[0170] Figure 7 2 is a block diagram showing a configuration example of a decoding device 200 according to this embodiment. For example, the decoding device 200 includes a vertex information decoder 201 , a connection information decoder 202 , and an attribute information decoder 203 .

[0171] The vertex information decoder 201 is an electrical circuit that decodes vertex information. For example, the vertex information decoder 201 decodes vertex information from a bit stream in a format specified for vertex information.

[0172] The connection information decoder 202 is an electrical circuit that decodes the connection information. For example, the connection information decoder 202 decodes the connection information from the bit stream in a format specified for the connection information.

[0173] The attribute information decoder 203 is an electrical circuit that decodes the attribute information. For example, the attribute information decoder 203 decodes the attribute information from the bit stream in a format specified for the attribute information.

[0174] In decoding vertex information, connection information, and attribute information, either variable-length decoding or fixed-length decoding can be used. Variable-length decoding can also correspond to Huffman coding or context-adaptive binary arithmetic coding (CABAC).

[0175] The vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be integrated. Alternatively, the vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be further subdivided into a plurality of components.

[0176] Figure 8 This is a block diagram showing another configuration example of the decoding device 200 according to this embodiment. For example, the decoding device 200 includes Figure 7 In addition to the illustrated configuration, a pre-processor 204 and a post-processor 205 are further provided.

[0177] The pre-processor 204 is an electrical circuit that processes the vertex information, connection information, and attribute information before decoding. For example, the pre-processor 204 may transform, separate, or multiplex the bitstream before decoding the vertex information, connection information, and attribute information.

[0178] More specifically, for example, the pre-processor 204 may separate the sub-bitstream corresponding to the vertex information, the sub-bitstream corresponding to the connection information, and the sub-bitstream corresponding to the attribute information from the bitstream. Furthermore, for example, the pre-processor 204 may perform variable-length decoding on the bitstream before decoding the vertex information, the connection information, and the attribute information.

[0179] The post-processor 205 is an electrical circuit that processes the decoded vertex information, connectivity information, and attribute information. For example, the post-processor 205 may transform, separate, or multiplex the decoded vertex information, connectivity information, and attribute information. More specifically, the post-processor 205 may multiplex the decoded vertex information, connectivity information, and attribute information into a three-dimensional mesh.

[0180] <bitstream>

[0181] Vertex information, connectivity information, and attribute information are encoded and stored in the bitstream. The following shows the relationship between this information and the bitstream.

[0182] Figure 9 This is a conceptual diagram showing an example of the bitstream structure related to this embodiment. In this example, connection information, vertex information, and attribute information are integrated into the bitstream. For example, connection information, vertex information, and attribute information can also be included in a single file.

[0183] Alternatively, multiple parts of this information may be stored sequentially, such as the first part of the connection information, the first part of the vertex information, the first part of the attribute information, the second part of the connection information, the second part of the vertex information, the second part of the attribute information, and so on. These multiple parts may correspond to multiple parts that are different in time, multiple parts that are different in space, or multiple different faces.

[0184] In addition, the storage order of the connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0185] Figure 10 This is a conceptual diagram showing another example of a bitstream structure related to this embodiment. In this example, the bitstream contains multiple files, with connection information, vertex information, and attribute information stored in separate files. While a file containing connection information, a file containing vertex information, and a file containing attribute information are shown here, the storage format is not limited to this example. For example, two of the connection information, vertex information, and attribute information could be contained in one file, while the remaining information could be contained in another file.

[0186] Alternatively, this information can be divided and stored in multiple files. For example, multiple parts of connectivity information can be stored in multiple files, multiple parts of vertex information can be stored in multiple files, or multiple parts of attribute information can be stored in multiple files. These multiple parts can correspond to multiple parts that are different in time, multiple parts that are different in space, or multiple different faces.

[0187] In addition, the storage order of the connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0188] Figure 11 This is a conceptual diagram showing another example of the bitstream configuration according to this embodiment. In this example, the bitstream is composed of multiple separable sub-bitstreams, and connection information, vertex information, and attribute information are stored in different sub-bitstreams.

[0189] Here, a sub-bitstream including connection information, a sub-bitstream including vertex information, and a sub-bitstream including attribute information are shown, but the storage format is not limited to this example.

[0190] For example, two types of information among connectivity information, vertex information, and attribute information may be included in one sub-bitstream, while the remaining type of information may be included in another sub-bitstream. Specifically, attribute information for a two-dimensional image, for example, may be stored separately from the connectivity information and vertex information sub-bitstreams in a sub-bitstream specific to the image coding method.

[0191] Furthermore, each sub-bitstream may include multiple files. For example, multiple parts of connection information may be stored in multiple files, multiple parts of vertex information may be stored in multiple files, or multiple parts of attribute information may be stored in multiple files.

[0192] In addition, the storage order of connection information, vertex information and attribute information is not limited to Figure 9 、 Figure 10 as well as Figure 11 In addition to the above example, a different storage order may be used. For example, vertex information, connection information, and attribute information may be stored in the bitstream in the order described above. Alternatively, any other order may be used, such as the order of connection information, attribute information, and vertex information; the order of vertex information, attribute information, and connection information; the order of attribute information, connection information, and vertex information; or the order of attribute information, vertex information, and connection information.

[0193] Alternatively, the connection information, vertex information, and attribute information may be divided into a plurality of data, and the plurality of data may be stored in a periodic order or a random order in the bit stream.

[0194] <Specific example>

[0195] Figure 12 : is a block diagram showing a specific example of the encoding and decoding system related to this embodiment. Figure 12 In the embodiment, the encoding and decoding system includes a three-dimensional data encoding system 110 , a three-dimensional data decoding system 210 , and an external connector 310 .

[0196] The 3D data encoding system 110 includes a controller 111, an input / output processor 112, a 3D data encoder 113, a 3D data generator 115, and a system multiplexer 114. The 3D data decoding system 210 includes a controller 211, an input / output processor 212, a 3D data decoder 213, a system demultiplexer 214, a prompter 215, and a user interface 216.

[0197] In the three-dimensional data encoding system 110 , sensor data is input from a sensor terminal to a three-dimensional data generator 115 . The three-dimensional data generator 115 generates three-dimensional data such as point cloud data or mesh data based on the sensor data and inputs the data to a three-dimensional data encoder 113 .

[0198] For example, the 3D data generator 115 generates vertex information and generates corresponding connection information and attribute information. The 3D data generator 115 may also process the vertex information when generating the connection information and attribute information. For example, the 3D data generator 115 may reduce the data volume by deleting duplicate vertices or perform transformations (such as position shifting, rotating, or normalizing) on the vertex information. Furthermore, the 3D data generator 115 may also render attribute information.

[0199] In addition, the three-dimensional data generator 115 Figure 12 The 3D data encoding system 110 is a component of the 3D data encoding system 110 , but can also be configured independently from the 3D data encoding system 110 outside.

[0200] The sensor terminal that provides the sensor data used to generate the three-dimensional data may be, for example, a mobile object such as a car, a flying object such as an airplane, a portable terminal, or a camera. Furthermore, distance sensors such as LIDAR, millimeter-wave radar, infrared sensors, or rangefinders, stereo cameras, or a combination of multiple monocular cameras may also be used as the sensor terminal.

[0201] Sensor data can also be the distance (position) of the object, monocular camera image, stereo camera image, color, reflectivity, sensor posture, orientation, gyroscope, sensed position (GPS information or altitude), speed, acceleration, sensing time, temperature, air pressure, humidity or magnetism, etc.

[0202] The three-dimensional data encoder 113 corresponds to Figure 5 For example, the 3D data encoder 113 encodes the 3D data to generate encoded data. Furthermore, the 3D data encoder 113 generates control information during the encoding of the 3D data. The 3D data encoder 113 then inputs the encoded data and the control information to the system multiplexer 114.

[0203] The encoding method of 3D data can be either a geometric encoding method or a video codec encoding method. Here, the geometric encoding method can also be expressed as a geometry-based encoding method. The encoding method using a video codec can also be expressed as a video-based encoding method.

[0204] The system multiplexer 114 multiplexes the encoded data and control information input from the 3D data encoder 113, generating multiplexed data using a predetermined multiplexing method. The system multiplexer 114 may also multiplex other media such as video, audio, subtitles, application data, or document files, or reference time information, along with the encoded data and control information for the 3D data. Furthermore, the system multiplexer 114 may also multiplex attribute information associated with the sensor data or the 3D data.

[0205] For example, the multiplexed data may be in a file format for storage or a packet format for transmission. ISOBMFF or a format based on ISOBMFF may be used as one of these formats. Furthermore, MPEG-DASH, MMT, MPEG-2 TS Systems, or RTP may also be used.

[0206] The input / output processor 112 then outputs the multiplexed data as a transmission signal to the external connector 310. The multiplexed data can be transmitted as a transmission signal via wired or wireless transmission. Alternatively, the multiplexed data can be stored in an internal memory or storage device. The multiplexed data can also be transmitted to a cloud server via the Internet or stored in an external storage device.

[0207] For example, the transmission or storage of multiplexed data is performed by a method corresponding to the medium used for transmission or storage, such as broadcasting or communication. Communication protocols such as http, ftp, TCP, UDP, IP, or a combination thereof may also be used. Furthermore, PULL-type communication methods or PUSH-type communication methods may also be used.

[0208] For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), or coaxial cable can be used. Furthermore, for wireless transmission, 3G / 4G / 5G, wireless LAN, Wi-Fi, Bluetooth, or millimeter waves specified by 3GPP (registered trademark) and IEEE can be used. Furthermore, broadcasting methods such as DVB-T2, DVB-S2, DVB-C2, ATSC 3.0, and ISDB-S3 can be used.

[0209] Alternatively, the sensor data may be input to the three-dimensional data generator 115 or the system multiplexer 114. Furthermore, the three-dimensional data or encoded data may be output directly as a transmission signal to the external connector 310 via the input / output processor 112. The transmission signal output from the three-dimensional data encoding system 110 is input to the three-dimensional data decoding system 210 via the external connector 310.

[0210] In addition, each action of the three-dimensional data encoding system 110 may also be controlled by the controller 111 executing an application program.

[0211] In the 3D data decoding system 210, a transmission signal is input to an input / output processor 212. The input / output processor 212 decodes multiplexed data in a file format or packet format from the transmission signal and inputs the multiplexed data to a system demultiplexer 214. The system demultiplexer 214 extracts coded data and control information from the multiplexed data and inputs them to a 3D data decoder 213. The system demultiplexer 214 may also extract other media or reference time information from the multiplexed data.

[0212] The three-dimensional data decoder 213 corresponds to Figure 7 The decoding device 200 shown in FIG. 3D data decoder 213 decodes 3D data from coded data based on a predetermined coding method. Then, the presenter 215 presents the 3D data to the user.

[0213] Additionally, additional information such as sensor data may be input to the prompter 215. The prompter 215 may also present the three-dimensional data based on the additional information. Furthermore, a user instruction may be input from a user terminal to the user interface 216. Furthermore, the prompter 215 may present the three-dimensional data based on the input instruction.

[0214] In addition, the input / output processor 212 may also obtain three-dimensional data and encoded data from the external connector 310 .

[0215] In addition, each operation of the three-dimensional data decoding system 210 may also be controlled by the controller 211 executing an application program.

[0216] Figure 13 : is a conceptual diagram showing an example of the structure of point cloud data according to this embodiment. Point cloud data is data representing a point cloud of a three-dimensional object.

[0217] Specifically, a point cloud is composed of multiple points and has position information representing the three-dimensional coordinate position of each point and attribute information representing the attributes of each point. Position information is also expressed as geometric information.

[0218] The category of attribute information may be, for example, color or reflectance. Attribute information of one category may be associated with one point, or attribute information of multiple different categories may be associated with one point, or attribute information of the same category having multiple values may be associated with one point.

[0219] Figure 14This is a conceptual diagram showing an example data file for point cloud data related to this embodiment. This example shows a one-to-one correspondence between items of position information and items of attribute information, displaying the position information and attribute information for the N points that comprise the point cloud data. In this example, the position information represents three-dimensional coordinates using the x, y, and z axes, while the attribute information represents color using RGB. PLY files, etc., are representative data files for point cloud data.

[0220] Figure 15 This is a conceptual diagram showing an example of the mesh data structure related to this embodiment. Mesh data is used in CG (Computer Graphics) and other applications. It is a three-dimensional mesh representing the three-dimensional shape of an object using multiple faces. Each face is represented as a polygon, with a polygonal shape such as a triangle or a quadrilateral.

[0221] Specifically, a 3D mesh consists of multiple edges and faces in addition to the multiple points that make up the point cluster. Each point is also represented by a vertex or position. Each edge corresponds to a line segment connecting two vertices. Each face corresponds to an area enclosed by three or more edges.

[0222] Furthermore, a 3D mesh has position information representing the three-dimensional coordinate positions of vertices. This position information is also expressed as vertex information or geometry. Furthermore, a 3D mesh has connectivity information, which indicates the relationships between the multiple vertices that make up an edge or face. This connectivity information is also expressed as connectivity. Furthermore, a 3D mesh has attribute information representing the properties of vertices, edges, or faces. This attribute information is also expressed as texture.

[0223] For example, the attribute information may also represent the color, reflectivity, or normal vector of a vertex, edge, or face. The direction of the normal vector may represent the front or back side of a face.

[0224] As a data file format of mesh data, an object file or the like can be used.

[0225] Figure 16 This is a conceptual diagram showing an example of a data file for mesh data related to this embodiment. In this example, the data file includes positional information G(1) to G(N) for the N vertices that constitute the three-dimensional mesh, and attribute information A1(1) to A1(N) for the N vertices. Furthermore, this example also includes M pieces of attribute information A2(1) to A2(M). Attribute information items do not necessarily correspond one-to-one with vertices or one-to-one with faces. Furthermore, attribute information may not exist at all.

[0226] Connection information is represented by a combination of vertex indices. n[1, 3, 4] represents a triangular face consisting of three vertices: n = 1, n = 3, and n = 4. Furthermore, m[2, 4, 6] represents attribute information corresponding to three vertices: m = 2, m = 4, and m = 6, respectively.

[0227] In addition, the substantial content of the attribute information may be recorded in other files. Furthermore, a pointer to the content may be associated with a vertex or a face. For example, the attribute information representing the image of the face may be stored in a two-dimensional attribute map file. Furthermore, the file name of the attribute map and the two-dimensional coordinate values in the attribute map may be recorded in the attribute information A2(1) to A2(M). The method of specifying attribute information for a face is not limited to these methods, and any method may be used.

[0228] Figure 17 This is a conceptual diagram illustrating the types of three-dimensional data related to this embodiment. Point cloud data and mesh data can represent both static and dynamic objects. Static objects are objects that do not change over time, while dynamic objects do change over time. Static objects can also correspond to three-dimensional data at any point in time.

[0229] For example, point cloud data at any point in time may be represented as a PCC frame. Also, mesh data at any point in time may be represented as a mesh frame. Furthermore, PCC frames and mesh frames may be represented simply as frames.

[0230] Furthermore, the target area can be restricted to a certain range, as with conventional image data, or it can be unlimited, as with map data. Furthermore, the density of points or surfaces can be set in various ways. Sparse point cloud data, sparse grid data, or dense point cloud data or dense grid data can be used.

[0231] Next, the encoding and decoding of point groups or three-dimensional meshes will be described. The apparatus, process, or syntax disclosed herein for encoding and decoding vertex information of three-dimensional meshes can also be applied to encoding and decoding point groups. The apparatus, process, or syntax disclosed herein for encoding and decoding point groups can also be applied to encoding and decoding vertex information of three-dimensional meshes.

[0232] Furthermore, the apparatus, process, or syntax disclosed herein for encoding and decoding attribute information of a point group can also be applied to encoding and decoding connection information or attribute information of a three-dimensional mesh. Furthermore, the apparatus, process, or syntax disclosed herein for encoding and decoding connection information or attribute information of a three-dimensional mesh can also be applied to encoding and decoding attribute information of a point group.

[0233] Furthermore, at least a portion of the processing for encoding and decoding point cloud data and encoding and decoding mesh data can be shared, thereby reducing the scale of circuits and software programs.

[0234] Figure 18 1 is a block diagram showing an example of the configuration of the three-dimensional data encoder 113 according to this embodiment. In this example, the three-dimensional data encoder 113 includes a vertex information encoder 121, an attribute information encoder 122, a metadata encoder 123, and a multiplexer 124. The vertex information encoder 121, the attribute information encoder 122, and the multiplexer 124 may also correspond to Figure 6 Vertex information encoder 101, attribute information encoder 103 and post-processor 105, etc.

[0235] In this example, the 3D data encoder 113 encodes the 3D data using a geometry-based encoding scheme. Geometry-based encoding takes the 3D structure into account. Furthermore, geometry-based encoding uses the composition information obtained from encoding the vertex information to encode attribute information.

[0236] Specifically, vertex information, attribute information, and metadata contained in the three-dimensional data generated from sensor data are first input to vertex information encoder 121, attribute information encoder 122, and metadata encoder 123, respectively. Here, the connection information contained in the three-dimensional data can be treated similarly to the attribute information. Furthermore, in the case of point cloud data, position information can also be treated as vertex information.

[0237] Vertex information encoder 121 encodes the vertex information into compressed vertex information and outputs the compressed vertex information as encoded data to multiplexer 124. Vertex information encoder 121 also generates metadata for the compressed vertex information and outputs it to multiplexer 124. Vertex information encoder 121 also generates composition information and outputs it to attribute information encoder 122.

[0238] The attribute information encoder 122 encodes the attribute information into compressed attribute information using the composition information generated by the vertex information encoder 121 and outputs the compressed attribute information as encoded data to the multiplexer 124 . The attribute information encoder 122 also generates metadata for the compressed attribute information and outputs it to the multiplexer 124 .

[0239] The metadata encoder 123 encodes compressible metadata into compressed metadata, and outputs the compressed metadata as encoded data to the multiplexer 124. The metadata encoded by the metadata encoder 123 may be used to encode vertex information and attribute information.

[0240] The multiplexer 124 multiplexes the compressed vertex information, the metadata of the compressed vertex information, the compressed attribute information, the metadata of the compressed attribute information, and the compressed metadata into a bitstream, and then inputs the bitstream into the system layer.

[0241] Figure 19 This is a block diagram showing an example of the configuration of the three-dimensional data decoder 213 according to this embodiment. In this example, the three-dimensional data decoder 213 includes a vertex information decoder 221, an attribute information decoder 222, a metadata decoder 223, and a demultiplexer 224. The vertex information decoder 221, the attribute information decoder 222, and the demultiplexer 224 may also correspond to Figure 8 Vertex information decoder 201, attribute information decoder 203 and pre-processor 204, etc.

[0242] In this example, the 3D data decoder 213 decodes the 3D data using a geometry-based encoding scheme. Decoding using a geometry-based encoding scheme takes the 3D structure into account. Furthermore, decoding using a geometry-based encoding scheme uses the composition information obtained from decoding the vertex information to decode the attribute information.

[0243] Specifically, the bitstream from the system layer is first input to the demultiplexer 224. The demultiplexer 224 separates the compressed vertex information, the compressed vertex information metadata, the compressed attribute information, the compressed attribute information metadata, and the compressed metadata from the bitstream. The compressed vertex information and the compressed vertex information metadata are input to the vertex information decoder 221. The compressed attribute information and the compressed attribute information metadata are input to the attribute information decoder 222. The metadata is input to the metadata decoder 223.

[0244] Vertex information decoder 221 decodes the compressed vertex information using metadata. Vertex information decoder 221 also generates composition information and outputs it to attribute information decoder 222. Attribute information decoder 222 decodes attribute information from the compressed attribute information using the composition information generated by vertex information decoder 221 and the metadata of the compressed attribute information. Metadata decoder 223 decodes metadata from the compressed metadata. Metadata decoded by metadata decoder 223 can also be used to decode vertex information and attribute information.

[0245] Then, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. Note that this metadata is metadata of the vertex information and attribute information, for example, and can be used in an application.

[0246] Figure 20This is a block diagram showing another configuration example of the three-dimensional data encoder 113 according to this embodiment. In this example, the three-dimensional data encoder 113 includes a vertex image generator 131, an attribute image generator 132, a metadata generator 133, a video encoder 134, a metadata encoder 123, and a multiplexer 124. The vertex image generator 131, the attribute image generator 132, and the video encoder 134 may also correspond to Figure 6 Vertex information encoder 101 and attribute information encoder 103, etc.

[0247] In this example, the 3D data encoder 113 encodes the 3D data using a video-based encoding scheme. In encoding using a video-based encoding scheme, multiple 2D images are generated from the 3D data, and these 2D images are encoded using an image coding scheme. The image coding scheme may also be HEVC (High Efficiency Video Coding) or VVC (Versatile Video Coding).

[0248] Specifically, vertex information and attribute information contained in the three-dimensional data generated from the sensor data are first input to metadata generator 133. Furthermore, the vertex information and attribute information are input to vertex image generator 131 and attribute image generator 132, respectively. Furthermore, metadata contained in the three-dimensional data is input to metadata encoder 123. Here, the connection information contained in the three-dimensional data can be handled in the same way as the attribute information. Furthermore, in the case of point cloud data, position information can also be handled as vertex information.

[0249] The metadata generator 133 generates mapping information of a plurality of two-dimensional images based on the vertex information and the attribute information, and inputs the mapping information to the vertex image generator 131 , the attribute image generator 132 , and the metadata encoder 123 .

[0250] The vertex image generator 131 generates a vertex image based on the vertex information and the mapping information, and inputs the image to the video encoder 134. The attribute image generator 132 generates an attribute image based on the attribute information and the mapping information, and inputs the image to the video encoder 134.

[0251] The video encoder 134 encodes the vertex image and the attribute image into compressed vertex information and compressed attribute information, respectively, according to a video encoding method, and outputs the compressed vertex information and compressed attribute information as encoded data to the multiplexer 124. Furthermore, the video encoder 134 generates metadata for the compressed vertex information and the compressed attribute information, and outputs the metadata to the multiplexer 124.

[0252] The metadata encoder 123 encodes the compressible metadata into compressed metadata and outputs the compressed metadata as encoded data to the multiplexer 124. The compressible metadata includes mapping information. In addition, the metadata encoded by the metadata encoder 123 can also be used to encode vertex information and attribute information.

[0253] The multiplexer 124 multiplexes the compressed vertex information, the metadata of the compressed vertex information, the compressed attribute information, the metadata of the compressed attribute information, and the compressed metadata into a bitstream, and then inputs the bitstream into the system layer.

[0254] Figure 21 This is a block diagram showing another configuration example of the three-dimensional data decoder 213 according to this embodiment. In this example, the three-dimensional data decoder 213 includes a vertex information generator 231, an attribute information generator 232, a video decoder 234, a metadata decoder 223, and a demultiplexer 224. The vertex information generator 231, the attribute information generator 232, and the video decoder 234 may also correspond to Figure 8 Vertex information decoder 201 and attribute information decoder 203, etc.

[0255] In this example, the 3D data decoder 213 decodes the 3D data using a video-based coding scheme. In video-based coding, multiple 2D images are decoded using a video coding scheme to generate 3D data from the multiple 2D images. The video coding scheme may also be HEVC (High Efficiency Video Coding) or VVC (Versatile Video Coding).

[0256] Specifically, the bitstream from the system layer is first input to the demultiplexer 224. The demultiplexer 224 separates the compressed vertex information, the metadata for the compressed vertex information, the compressed attribute information, the metadata for the compressed attribute information, and the compressed metadata from the bitstream. The compressed vertex information, the metadata for the compressed vertex information, the compressed attribute information, and the metadata for the compressed attribute information are input to the image decoder 234. The compressed metadata is input to the metadata decoder 223.

[0257] The video decoder 234 decodes the vertex images according to the video coding method. In this case, the video decoder 234 uses the metadata of the compressed vertex information to decode the vertex images from the compressed vertex information. Furthermore, the video decoder 234 inputs the vertex images to the vertex information generator 231. Furthermore, the video decoder 234 decodes the attribute images according to the video coding method. In this case, the video decoder 234 uses the metadata of the compressed attribute information to decode the attribute images from the compressed attribute information. Furthermore, the video decoder 234 inputs the attribute images to the attribute information generator 232.

[0258] The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 includes mapping information used to generate vertex information and attribute information. Furthermore, the metadata decoded by the metadata decoder 223 can also be used to decode vertex images and attribute images.

[0259] The vertex information generator 231 reproduces vertex information from the vertex image according to the mapping information included in the metadata decoded by the metadata decoder 223. The attribute information generator 232 reproduces attribute information from the attribute image according to the mapping information included in the metadata decoded by the metadata decoder 223.

[0260] Then, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. Note that this metadata is metadata of the vertex information and attribute information, for example, and can be used in an application.

[0261] Figure 22 This is a conceptual diagram showing a specific example of the encoding process related to this embodiment. Figure 22 3D data encoder 113 and description encoder 148 are shown in FIG. In this example, 3D data encoder 113 includes 2D data encoder 141 and mesh data encoder 142. 2D data encoder 141 includes texture encoder 143. Mesh data encoder 142 includes vertex information encoder 144 and connection information encoder 145.

[0262] Vertex information encoder 144, connection information encoder 145 and texture encoder 143 may also correspond to Figure 6 Vertex information encoder 101, connection information encoder 102 and attribute information encoder 103, etc.

[0263] For example, the two-dimensional data encoder 141 operates as the texture encoder 143 and generates a texture file by encoding the texture corresponding to the attribute information as two-dimensional data according to an image encoding method or a video encoding method.

[0264] Furthermore, the mesh data encoder 142 operates as a vertex information encoder 144 and a connection information encoder 145, encoding the vertex information and connection information to generate a mesh file. The mesh data encoder 142 may also encode texture mapping information. Furthermore, the encoded mapping information may be included in the mesh file.

[0265] In addition, the description encoder 148 can also generate a description file by encoding the description corresponding to the metadata such as text data. The description encoder 148 can also encode the description at the system level. For example, the description encoder 148 can also be included in Figure 12 in the system multiplexer 114.

[0266] The above steps generate a bitstream containing texture files, mesh files, and description files. These files can also be reused in the bitstream in the form of glTF (Graphics Language Transmission Format) or USD (Universal Scene Description) files.

[0267] Furthermore, the three-dimensional data encoder 113 may include two mesh data encoders as the mesh data encoder 142. For example, one mesh data encoder may encode vertex information and connection information of a static three-dimensional mesh, while the other mesh data encoder may encode vertex information and connection information of a dynamic three-dimensional mesh.

[0268] Furthermore, correspondingly, two mesh files may be included in the bitstream, for example, one mesh file corresponding to a static three-dimensional mesh and the other mesh file corresponding to a dynamic three-dimensional mesh.

[0269] Furthermore, the static 3D mesh may be a 3D mesh of an intra-frame encoded using intra-frame prediction, and the dynamic 3D mesh may be a 3D mesh of an inter-frame encoded using inter-frame prediction. Furthermore, the dynamic 3D mesh information may include differential information between vertex information or connection information of a 3D mesh of an intra-frame and vertex information or connection information of a 3D mesh of an inter-frame.

[0270] Figure 23 This is a conceptual diagram showing a specific example of the decoding process related to this embodiment. Figure 23 3D data decoder 213, description decoder 248, and prompter 247 are shown. In this example, 3D data decoder 213 includes 2D data decoder 241, mesh data decoder 242, and mesh reconstructor 246. 2D data decoder 241 includes texture decoder 243. Mesh data decoder 242 includes vertex information decoder 244 and connection information decoder 245.

[0271] Vertex information decoder 244, connection information decoder 245, texture decoder 243 and mesh reconstructor 246 may also correspond to Figure 8 Vertex information decoder 201, connection information decoder 202, attribute information decoder 203 and post-processor 205. The prompter 247 may also correspond to Figure 12 Prompt 215, etc.

[0272] For example, the two-dimensional data decoder 241 operates as a texture decoder 243 and decodes the texture corresponding to the attribute information from the texture file into two-dimensional data according to the image coding method or the video coding method.

[0273] The mesh data decoder 242 also operates as a vertex information decoder 244 and a connection information decoder 245 to decode vertex information and connection information from the mesh file. The mesh data decoder 242 may also decode mapping information for textures from the mesh file.

[0274] In addition, the description decoder 248 decodes the description corresponding to the metadata such as text data from the description file. The description decoder 248 can also decode the description in the system layer. For example, the description decoder 248 can also be included in Figure 12 in the system demultiplexer 214.

[0275] The mesh reconstructor 246 reconstructs the 3D mesh from the vertex information, connectivity information, and texture according to the description. The prompter 247 renders the 3D mesh according to the description and outputs it.

[0276] Through the above operations, a three-dimensional mesh is reconstructed and output from the bitstream containing the texture file, the mesh file, and the description file.

[0277] Furthermore, the three-dimensional data decoder 213 may include two mesh data decoders as the mesh data decoders 242. For example, one mesh data decoder decodes vertex information and connection information of a static three-dimensional mesh, while the other mesh data decoder decodes vertex information and connection information of a dynamic three-dimensional mesh.

[0278] Furthermore, two corresponding mesh files may be included in the bitstream, for example, one mesh file corresponding to a static three-dimensional mesh and the other mesh file corresponding to a dynamic three-dimensional mesh.

[0279] Furthermore, the static 3D mesh may be a 3D mesh of an intra-frame encoded using intra-frame prediction, and the dynamic 3D mesh may be a 3D mesh of an inter-frame encoded using inter-frame prediction. Furthermore, the dynamic 3D mesh information may include differential information between vertex information or connection information of a 3D mesh of an intra-frame and vertex information or connection information of a 3D mesh of an inter-frame.

[0280] Dynamic 3D mesh coding is sometimes referred to as DMC (Dynamic Mesh Coding). In addition, dynamic 3D mesh coding based on video is sometimes referred to as V-DMC (Video-based Dynamic Mesh Coding).

[0281] Point cloud encoding is sometimes called PCC (Point Cloud Compression). Furthermore, point cloud encoding based on video is sometimes called V-PCC (Video-based Point Cloud Compression). Furthermore, point cloud encoding based on geometry is sometimes called G-PCC (Geometry-based Point Cloud Compression).

[0282] <Installation Example>

[0283] Figure 24 1 is a block diagram showing an implementation example of the encoding device 100 according to this embodiment. The encoding device 100 includes a circuit 151 and a memory 152. For example, Figure 5 The multiple components of the encoding device 100 shown in FIG. Figure 24 The circuit 151 and memory 152 are shown installed.

[0284] Circuit 151 is a circuit that performs information processing and is capable of accessing memory 152. For example, circuit 151 is a dedicated or general-purpose electrical circuit that encodes a three-dimensional mesh. Circuit 151 may also be a processor such as a CPU. Alternatively, circuit 151 may be a collection of multiple electrical circuits.

[0285] Memory 152 is a dedicated or general-purpose memory that stores information used by circuit 151 to encode the three-dimensional mesh. Memory 152 may be an electrical circuit or connected to circuit 151. Furthermore, memory 152 may be included in circuit 151. Furthermore, memory 152 may be a collection of multiple electrical circuits. Furthermore, memory 152 may be a magnetic disk or optical disk, or may be a storage device or recording medium. Furthermore, memory 152 may be either non-volatile or volatile memory.

[0286] For example, both a three-dimensional mesh and a bit stream may be stored in the memory 152. In addition, the memory 152 may also store a program for the circuit 151 to encode the three-dimensional mesh.

[0287] In addition, in the encoding device 100, it is not necessary to install Figure 5 All of the multiple components shown may not perform all of the multiple processes shown here. Figure 5A portion of the multiple components shown in the figure may be included in other devices, and a portion of the multiple processes shown here may be performed by other devices. In addition, in the encoding device 100, the multiple components of the present disclosure may be arbitrarily combined and installed, and the multiple processes of the present disclosure may be arbitrarily combined and performed.

[0288] Figure 25 2 is a block diagram showing an implementation example of the decoding device 200 according to this embodiment. The decoding device 200 includes a circuit 251 and a memory 252. For example, Figure 7 The multiple components of the decoding device 200 shown in FIG. Figure 25 The circuit 251 and memory 252 are shown installed.

[0289] Circuit 251 is a circuit that performs information processing and is capable of accessing memory 252. For example, circuit 251 is a dedicated or general-purpose electrical circuit that decodes a three-dimensional mesh. Circuit 251 may also be a processor such as a CPU. Alternatively, circuit 251 may be a collection of multiple electrical circuits.

[0290] Memory 252 is a dedicated or general-purpose memory that stores information used by circuit 251 to decode the three-dimensional mesh. Memory 252 can be an electrical circuit or connected to circuit 251. Furthermore, memory 252 can be included in circuit 251. Furthermore, memory 252 can be a collection of multiple electrical circuits. Furthermore, memory 252 can be a magnetic disk or optical disk, or can be a storage device or recording medium. Furthermore, memory 252 can be either non-volatile memory or volatile memory.

[0291] For example, both a three-dimensional mesh and a bit stream may be stored in the memory 252. In addition, the memory 252 may also store a program for the circuit 251 to decode the three-dimensional mesh.

[0292] In addition, the decoding device 200 may not include Figure 7 All of the multiple components shown may not perform all of the multiple processes shown here. Figure 7 A portion of the multiple components shown in the figure may also be included in other devices, and a portion of the multiple processes shown here may also be performed by other devices. In addition, in the decoding device 200, the multiple components of the present disclosure may be arbitrarily combined and installed, and the multiple processes of the present disclosure may be arbitrarily combined and performed.

[0293] The encoding method and decoding method including the steps performed by the components of the encoding device 100 and decoding device 200 of the present disclosure may be executed by any device or system. For example, a computer equipped with a processor, memory, input / output circuits, etc. may execute the encoding method and decoding method in part or in whole. In this case, the encoding method and decoding method may also be executed by the computer executing a program for causing the computer to execute the encoding method and decoding method.

[0294] In addition, a non-transitory computer-readable recording medium such as a CD-ROM may record a program or a bit stream.

[0295] An example of a program can also be a bitstream. For example, a bitstream containing an encoded three-dimensional mesh includes syntax elements for decoding device 200 to decode the three-dimensional mesh. Furthermore, the bitstream causes decoding device 200 to decode the three-dimensional mesh according to the syntax elements contained in the bitstream. Therefore, a bitstream can function similarly to a program.

[0296] The above-mentioned bit stream may be a coded bit stream including the coded three-dimensional mesh, or may be a multiplexed bit stream including the coded three-dimensional mesh and other information.

[0297] Furthermore, the components of encoding device 100 and decoding device 200 may be configured as dedicated hardware, general-purpose hardware that executes the aforementioned programs, or a combination thereof. Furthermore, the general-purpose hardware may be configured as a memory storing a program and a general-purpose processor that reads and executes the program from the memory. The memory may be a semiconductor memory or a hard disk, and the general-purpose processor may be a CPU.

[0298] In addition, dedicated hardware may be composed of a memory and a dedicated processor, etc. For example, a dedicated processor may refer to a memory for recording data to execute the encoding method and the decoding method.

[0299] Furthermore, the components of the encoding device 100 and the decoding device 200 may also be electrical circuits as described above. These electrical circuits may be configured as a single electrical circuit or as separate electrical circuits. Furthermore, these electrical circuits may correspond to dedicated hardware or general-purpose hardware that executes the aforementioned programs, etc. Furthermore, the encoding device 100 and the decoding device 200 may also be implemented as integrated circuits.

[0300] Alternatively, the encoding device 100 may be a transmitting device that transmits a three-dimensional mesh, and the decoding device 200 may be a receiving device that receives a three-dimensional mesh.

[0301] <Encoding and decoding of vertex information>

[0302] A 3D model is a digital representation of an object, allowing users to temporarily render it while exploring the model in all three dimensions using zoom, translation, and rotation. One method for constructing such a representation is to construct a 3D mesh using triangles. The model stores the positions of the triangle vertices, their connectivity, and their associated properties (normals, UV patches, etc.). Storing all this information in an uncompressed form requires a very large storage area, and therefore a very large bandwidth for transmission.

[0303] The triangles that form the mesh often have repeating patterns and similar properties, especially when they are close together in time and space. These repetitions can be used to formulate efficient encoding and decoding methods for storage and transmission.

[0304] Figure 26 : is a block diagram showing the architecture of the encoding and decoding system related to this embodiment. Figure 26 As shown, the architecture of the encoding and decoding system includes an encoding device 100 and a decoding device 200. The encoding and decoding system receives an input 3D mesh frame in the form of vertex geometric coordinates (vertex information), texture coordinates (attribute information) and connectivity data (connection information).

[0305] The encoding device 100 is responsible for encoding all associated information into a bitstream (compressed bitstream). The bitstream can be composed of multiple bitstreams. The bitstream is transmitted to the decoding device 200 via a transmission path. The decoding device 200 decodes the bitstream and uses the decoded vertex geometry coordinates, texture coordinates, and connectivity data to generate a 3D mesh frame.

[0306] Figure 27 1 is a block diagram showing an example of the architecture of the encoding device 100 according to this embodiment. In this example, the encoding device 100 includes a volume capturer 511, a projector 512, a base grid encoder 513, a displacement encoder 514, an attribute encoder 515, and one or more other type encoders 516 as options.

[0307] The volume capturer 511 captures content and transmits it to the projector 512. The projector 512 projects the content onto an input mesh containing vertex geometry coordinates, texture coordinates, and connectivity data. This data is then sent to the base mesh encoder 513, displacement encoder 514, attribute encoder 515, and optionally one or more other encoder types 516. These encoders compress the data into a bitstream.

[0308] Figure 28 5 is a block diagram showing another example of the architecture of the encoding apparatus 100 according to this embodiment. In this example, the encoding apparatus 100 includes a pre-processor 521 and an encoding processor 522 .

[0309] The pre-processor 521 reads the 3D mesh frame, extracts the base mesh, displacement information, and attribute map, and passes them to the encoding processor 522. An example of displacement information is a displacement vector. The encoding processor 522 compresses the base mesh, displacement information, and attribute map separately and combines them to generate a bitstream.

[0310] Figure 29 1 is a block diagram showing an example of the architecture of the decoding device 200 according to this embodiment. In this example, the decoding device 200 includes a base grid decoder 613 , a displacement decoder 614 , an attribute decoder 615 , one or more other type decoders 616 , and a 3D reconstructor 617 .

[0311] The bitstream is sent to a base mesh decoder 613, a displacement decoder 614, an attribute decoder 615, and optionally one or more other type decoders 616. These decoders decode the bitstream, generating decoded data containing vertex geometry coordinates, texture coordinates, and connectivity data. The decoded data is then sent to a 3D reconstructor 617, which reconstructs a 3D mesh frame.

[0312] Figure 30 6 is a block diagram showing another example of the architecture of the decoding apparatus 200 according to this embodiment. In this example, the decoding apparatus 200 includes a decoding processor 622 and a post-processor 623 .

[0313] Decoding processor 622 first reads the bitstream, separates the base mesh, displacement information, and attribute map from the compressed bitstream, decodes each separately, and passes them to post-processor 623. An example of displacement information is a displacement vector. Post-processor 623 processes the base mesh according to the displacement information and attribute map to generate a 3D mesh frame.

[0314] Figure 31 8 is a block diagram showing yet another example of the architecture of the decoding device 200 according to this embodiment. Specifically, the architecture associated with vertex information is shown. In this example, the decoding device 200 includes an inter decoder 811, a vertex buffer 812, and an intra decoder 813.

[0315] When the current frame to be processed is an inter-frame frame, the inter-frame decoder 811 decodes the current vertex to be processed in the current frame with reference to the inter-frame reference vertex in the vertex buffer 812 .

[0316] Here, an inter-frame is a frame in which vertices are processed with reference to vertices in an already processed frame. Vertices processed with reference to vertices in an already processed frame are sometimes expressed as inter-frame vertices. In this case, the referenced frame can be expressed as a reference frame or an inter-frame reference frame. In this case, the referenced vertex can also be expressed as a reference vertex or an inter-frame reference vertex.

[0317] For example, the inter-frame decoder 811 generates an inter-frame prediction vertex using an inter-frame reference vertex. The inter-frame prediction vertex may also be the inter-frame reference vertex itself. Furthermore, the inter-frame decoder 811 decodes the difference between the inter-frame prediction vertex and the current vertex and adds the difference to the inter-frame prediction vertex to decode (reconstruct) the current vertex.

[0318] The current vertex decoded (reconstructed) by the inter-frame decoder 811 is stored as a reconstructed vertex in the vertex buffer 812. The reconstructed vertex stored in the vertex buffer 812 can be referenced as an inter-frame reference vertex when decoding other vertices. Furthermore, the current vertex decoded by the inter-frame decoder 811 constitutes the 3D mesh frame to be output.

[0319] When the current frame to be processed is an intra frame, the intra decoder 813 decodes the current vertex to be processed in the current frame with reference to the intra reference vertex in the current frame.

[0320] For example, the intra decoder 813 generates an intra-prediction vertex using an intra reference vertex. The intra-prediction vertex may also be the intra reference vertex itself. Furthermore, the intra decoder 813 decodes the difference between the intra-prediction vertex and the current vertex and adds the difference to the intra-prediction vertex to decode (reconstruct) the current vertex.

[0321] The current vertex decoded (reconstructed) by the intra decoder 813 is stored as a reconstructed vertex in the vertex buffer 812. The reconstructed vertex stored in the vertex buffer 812 can be referenced as an inter-frame reference vertex when decoding other vertices. Furthermore, the current vertex decoded by the intra decoder 813 constitutes the 3D mesh frame to be output.

[0322] Alternatively, the inter-frame decoder 811 and the intra-frame decoder 813 can decode the current vertex without referencing other vertices. In other words, the inter-frame decoder 811 and the intra-frame decoder 813 can decode the current vertex itself, rather than decoding the difference. This operation corresponds to treating the reference vertex as (0, 0, 0), decoding the difference, and adding it to the reference vertex.

[0323] Furthermore, the decoding apparatus 200 may include a frame header decoder that decodes a frame header for each frame. The frame header may indicate whether the frame is an inter-frame or an intra-frame.

[0324] Figure 32 This is a conceptual diagram showing an example of subdivision according to the present embodiment. For example, a base mesh includes vertices A, B, and C and their connectivity.

[0325] In the first sub-division, new vertices D, E, and F are added between the already connected vertices AB, BC, and CA, and their connectivity is added. These newly added vertices and their connectivity, along with the existing vertices and connectivity, form the first level. In the second sub-division, the same process is repeated, adding vertices G, H, I, J, K, L, M, N, and O and their connectivity to form the second level.

[0326] Figure 33 This is a block diagram showing yet another example of the architecture of decoding device 200 according to this embodiment. Specifically, the architecture associated with displacement information is shown. In this example, decoding device 200 includes a video decoder 631, an image decompressor 632, an inverse quantizer 633, and an inverse wavelet transformer 634.

[0327] The image decoder 631 reads the bitstream and decodes the image data using a frame decompression method, assuming the image in the bitstream consists of two color difference information and one luminance information. The image decompressor 632 extracts the wavelet coefficients associated with each vertex from the decompressed data in image form. Next, the inverse quantizer 633 inverse-quantizes the quantized wavelet coefficients using the three components associated with each vertex. The inverse wavelet transformer 634 inverse-transforms the result to obtain the final displacement information.

[0328] An example of displacement information is a displacement vector, which is used to displace vertices within a three-dimensional mesh frame.

[0329] The following describes a configuration and method for achieving more excellent compression based on temporal redundancy and spatial redundancy in encoding and decoding processes of vertex information.

[0330] <Vertex Information Encoding Processing>

[0331] Figure 34 This is a flowchart illustrating the mesh encoding process according to this embodiment. In this example, first, the connection information of the second 3D mesh frame is applied to the connection information of the first 3D mesh frame (S101). Next, first reference information for the first vertex set in the first 3D mesh frame and second reference information for the second vertex set in the first 3D mesh frame are encoded into a bitstream (S102). The first 3D mesh frame is, for example, the current 3D mesh frame to be encoded.

[0332] Each reference information can be represented by one or more reference parameters. An example of a reference parameter is an index representing an identifier of a 3D mesh frame. An example of an index is a numerical value such as 0, 1, 2, or 3. Another example of an index is a letter such as A, B, or C. Another example of an index is a Roman numeral such as I, II, or III.

[0333] Figure 35 This is a conceptual diagram showing an example of reference information related to this embodiment. In this example, the reference information is represented by a number that serves as an index identifying a 3D mesh frame. Specifically, the reference information is represented by "#3." That is, in this example, the reference information indicates frame #3 among multiple frames #0 to #3. In another example, the reference information can indicate the current frame itself.

[0334] exist Figure 34 In the example of , next, when the first reference information indicates the first value, the first vertex set is encoded using the third vertex set in the second 3D mesh frame (S103). Here, the second 3D mesh frame is a 3D mesh frame that is different in time from the first 3D mesh frame. An example of the second 3D mesh frame is a previously encoded 3D mesh frame.

[0335] An example of the third vertex set is all vertices that enter the three-dimensional region. An example of a three-dimensional region is a cuboid represented by height, width, and depth from a reference point.

[0336] Figure 36 This is a conceptual diagram showing an example of a three-dimensional area related to this embodiment. Figure 36 In one example, a three-dimensional region is represented by a cuboid with a height of 3, a width of 2, and a depth of 3 from the reference point (0, 2, 3). In another example, a three-dimensional region is represented by a sphere with the coordinates of its center and a radius. In another example, a three-dimensional region is represented by a cube with the coordinates of its vertices and the sizes of its sides.

[0337] Figure 37 This is a conceptual diagram illustrating an example of a vertex set located inside a cuboid according to this embodiment. In this example, the third vertex set can be selected based on the fact that some of the vertices M, N, O, P, Q, and R of the three-dimensional mesh are located inside the cuboid. In one example, the third vertex set consists of the vertices within the upper cuboid, namely, M, N, and O. In another example, the third vertex set consists of the vertices within the lower cuboid, namely, P, Q, and R.

[0338] For example, the third vertex set is selected and used as the predicted vertex set for the first vertex set. An example of encoding the first vertex set using the third vertex set is encoding the differences between the third vertex set and the first vertex set. Specifically, the differences between corresponding vertices in the third vertex set and the first vertex set are encoded. Furthermore, when encoding multiple differences, differences between differences may also be encoded.

[0339] Figure 38 This is a conceptual diagram showing an example of the relationship between the differences between the first and third vertex sets in this embodiment. Here, the third vertex set includes vertices M (10, 8, 5), N (14, 3, 3), and O (12, 2, 2). The first vertex set includes vertices A (12, 9, 8), B (17, 4, 4), and C (13, 2, 3). These differences are derived as (2, 1, 3), (3, 1, 1), and (1, 0, 1).

[0340] Alternatively, (2, 1, 3) may be encoded, the difference between (3, 1, 1) and (2, 1, 3), namely (1, 0, -2), may be encoded, and the difference between (1, 0, 1) and (3, 1, 1), namely (-2, 1, 0), may be encoded as the difference between the differences.

[0341] exist Figure 34 In the example of , when the second reference information indicates the second value, the second vertex set is encoded using the fourth vertex set in the first three-dimensional mesh frame ( S104 ).

[0342] An example of encoding the second vertex set using the fourth vertex set is to encode the differences between the vertices in the fourth vertex set and the vertices in the second vertex set.

[0343] For example, the fourth vertex set has one vertex. Furthermore, when encoding the first vertex from the beginning of the second vertex set, the difference between the one vertex in the fourth vertex set and the first vertex from the beginning of the second vertex set is encoded. Furthermore, when encoding the second vertex from the beginning of the second vertex set, the difference between the first vertex from the beginning of the second vertex set and the second vertex from the beginning of the second vertex set is encoded.

[0344] Furthermore, when encoding the third vertex from the beginning of the second vertex set, the difference between the second vertex from the beginning of the second vertex set and the third vertex from the beginning of the second vertex set is encoded. This encoding of the difference can be repeated. Furthermore, the order in which the vertices are encoded can correspond to the order in which the vertices are scanned.

[0345] Figure 39This is a conceptual diagram showing an example of the relationship between the differences between the second and fourth vertex sets in this embodiment. Here, the fourth vertex set includes vertex C (13, 2, 3). The second vertex set includes vertices D (15, 1, 2), E (19, 0, 0), and F (12, 1, 0).

[0346] When encoding vertex D, the difference between vertices C and D, namely (2, -1, -1), can be encoded. Furthermore, when encoding vertex E, the difference between vertices D and E, namely (4, -1, -2), can be encoded. Furthermore, when encoding vertex F, the difference set between vertices E and F, namely (-7, 1, 0), can be encoded.

[0347] Furthermore, the fourth vertex set may be considered to partially overlap with the second vertex set in including vertices C, D, and E. Furthermore, the difference between vertices C and D may be encoded in the encoding of vertex D in the second vertex set, the difference between vertices D and E may be encoded in the encoding of vertex E in the second vertex set, and the difference between vertices E and F may be encoded in the encoding of vertex F in the second vertex set.

[0348] Furthermore, the difference between vertex D and vertex E corresponds to the difference between the difference between vertex C and vertex D and the difference between vertex C and vertex E. In other words, the difference between vertex D and vertex E can be regarded as the difference between differences.

[0349] Figure 39 This example shows how the coordinates of vertices D, E, and F constituting the current frame are encoded using the coordinates of a previously encoded vertex C belonging to the current frame. In this example, the previously encoded vertex C may be one of the vertices constituting the same target as the target constituting the vertices D, E, and F being encoded, and may be a vertex that was encoded immediately before the target vertices D, E, and F in the encoding order. However, the reference vertex is not limited to this example.

[0350] For example, the three vertices D, E, and F to be coded may be coded separately with reference to the three vertices already coded. In this case, the three vertices already coded may be vertices that were predictively coded immediately before the three vertices to be coded in the coding order.

[0351] In addition, for example, the encoded vertex set may be specified by encoding parameters. The encoded vertex set may also belong to the same frame as the encoding target frame. The encoded vertex set may belong to the same object as the encoding target vertex, or to a different object.

[0352] exist Figure 34In the example of , the plurality of vertices of the first three-dimensional mesh frame are then reconstructed (S105). At this time, the first vertex set and the second vertex set are reconstructed. Specifically, if the first vertex set is encoded using the third vertex set, the first vertex set is reconstructed using the third vertex set and the encoded first vertex set. Furthermore, if the second vertex set is encoded using the fourth vertex set, the second vertex set is reconstructed using the fourth vertex set and the encoded second vertex set.

[0353] For example, if the differences between the first and third vertex sets are encoded, the first vertex set can be reconstructed by adding the differences to the third vertex set. Alternatively, if the differences between the second and fourth vertex sets are encoded, the second vertex set can be reconstructed by adding the differences to the fourth vertex set. In this way, multiple vertices of the first 3D mesh frame are reconstructed.

[0354] Figure 40 This is a conceptual diagram illustrating an example of reconstructing multiple vertices of a first 3D mesh frame according to this embodiment. In this example, the multiple vertices of the first 3D mesh frame are reconstructed by combining a first vertex set and a second vertex set. In another example of reconstructing the first and second vertex sets, the multiple vertices of the first and second vertex sets may form a base mesh, and a portion of these vertices may be displaced using a displacement vector encoded in the bitstream.

[0355] exist Figure 34 In the example of , first, the connection information of the second 3D mesh frame is applied to the connection information of the first 3D mesh frame (S101). That is, the connection information of the 3D mesh in the second 3D mesh frame is applied to the connection information of the plurality of vertices reconstructed in the first 3D mesh frame.

[0356] For example, when the first reference information indicates the first value and at least one vertex of the triangle is included in the first vertex set, the connection information of the triangle can be copied from the second 3D mesh frame to the first 3D mesh frame.

[0357] Figure 41 This is a conceptual diagram illustrating an example of the application of connection information related to this embodiment. Here, vertices A, B, and C are included in the first vertex set encoded using the third vertex set of the second 3D mesh frame. Therefore, AB, BC, AC, CE, CF, BD, and BE are copied from the second 3D mesh frame and connected. Furthermore, DE and EF can also be considered connected based on the encoding order.

[0358] Here, the connection information related to the first vertex set encoded using the third vertex set of the second 3D mesh frame is copied from the second 3D mesh frame to the first 3D mesh frame. In other words, the connection information related to the vertex set to which inter-frame prediction is applied is copied from the second 3D mesh frame to the first 3D mesh frame.

[0359] However, regardless of whether the connection information is related to a vertex set to which inter-frame prediction is applied, the connection information may be copied from the second 3D mesh frame to the first 3D mesh frame. In other words, the connection information related to a vertex set to which intra-frame prediction is applied may be copied from the second 3D mesh frame to the first 3D mesh frame.

[0360] In addition, here, the connection information of the second 3D mesh frame is applied to the reconstructed vertex information of the first 3D mesh frame. However, the connection information of the second 3D mesh frame is not limited to the reconstructed vertex information of the first 3D mesh frame, and can also be used as the connection information of the first 3D mesh frame.

[0361] Specifically, the connection information of the second 3D mesh frame can also be used to encode the multiple vertices of the first 3D mesh frame. More specifically, the connection information of the second 3D mesh frame can be used according to the encoding order or prediction order of the multiple vertices of the first 3D mesh frame, and can also be used to determine the reference vertex or the predicted vertex, and can also be used to determine the inter-frame prediction or the intra-frame prediction.

[0362] Figure 42 This is a block diagram showing a configuration example for switching between intra prediction and inter prediction for each vertex set in the encoding apparatus 100 according to this embodiment. In this example, the encoding apparatus 100 includes an inter encoder 711 , a vertex buffer 712 , an intra encoder 713 , and a connection information buffer 714 .

[0363] When the current vertex set to be processed is an inter-frame vertex set, the inter-frame encoder 711 encodes the current vertex to be processed in the current vertex set with reference to the inter-frame reference vertices in the vertex buffer 712. Here, the inter-frame vertex set is a vertex set consisting of inter-frame vertices that are processed with reference to vertices of an already processed frame.

[0364] For example, the inter-frame encoder 711 generates an inter-frame prediction vertex using an inter-frame reference vertex. The inter-frame prediction vertex may also be the inter-frame reference vertex itself. Furthermore, the inter-frame encoder 711 encodes the difference between the inter-frame prediction vertex and the current vertex and adds the difference to the inter-frame prediction vertex to reconstruct the current vertex.

[0365] The current vertex reconstructed by the inter encoder 711 is stored as a reconstructed vertex in the vertex buffer 712. The reconstructed vertex stored in the vertex buffer 712 can be referenced as an inter reference vertex in encoding of other vertices, for example.

[0366] Furthermore, the inter-frame encoder 711 may also use the connectivity information to encode the current vertex. Specifically, the inter-frame encoder 711 may also use the connectivity information to determine an inter-frame reference vertex. Furthermore, the inter-frame encoder 711 may also use the inter-frame reference vertex and the connectivity information to generate an inter-frame prediction vertex. Furthermore, the connectivity information may also be used to determine the current vertex and determine the encoding order.

[0367] Alternatively, the inter-frame encoder 711 may encode the current vertex by referring to the inter-frame reference connection information in the connection information buffer 714. That is, the inter-frame encoder 711 may encode the current vertex using the connection information of the 3D mesh in the reference frame. In this case, the connection information of the 3D mesh in the reference frame may be used as the connection information of the 3D mesh in the current frame to be processed.

[0368] When the current vertex set to be processed is an intra vertex set, the intra encoder 713 encodes the current vertex to be processed in the current vertex set with reference to the intra reference vertex. Here, the intra vertex set is a vertex set consisting of intra vertices processed with reference to vertices of the current frame.

[0369] For example, the intra-frame encoder 713 generates an intra-frame prediction vertex using an intra-frame reference vertex. The intra-frame prediction vertex may also be the intra-frame reference vertex itself. Furthermore, the intra-frame encoder 713 encodes the difference between the intra-frame prediction vertex and the current vertex and adds the difference to the intra-frame prediction vertex to reconstruct the current vertex.

[0370] Furthermore, the intra-frame encoder 713 may also use the connectivity information to encode the current vertex. Specifically, the intra-frame encoder 713 may also use the connectivity information to determine an intra-frame reference vertex. Furthermore, the intra-frame encoder 713 may also use the intra-frame reference vertex and the connectivity information to generate an intra-frame prediction vertex. Furthermore, the connectivity information may also be used to determine the current vertex and determine the encoding order.

[0371] Alternatively, the intra-frame encoder 713 may encode the current vertex by referring to the inter-frame reference connection information in the connection information buffer 714. That is, the intra-frame encoder 713 may encode the current vertex using the connection information of the 3D mesh in the reference frame. In this case, the connection information of the 3D mesh in the reference frame may be used as the connection information of the 3D mesh in the current frame to be processed.

[0372] In addition, the intra-frame encoder 713 encodes the connection information in the current frame and reconstructs the connection information. The connection information reconstructed by the intra-frame encoder 713 is stored as reconstructed connection information in the connection information buffer 714. The reconstructed connection information stored in the connection information buffer 714 can be referenced as inter-frame reference connection information when encoding other frames.

[0373] Furthermore, the inter-frame encoder 711 and the intra-frame encoder 713 can also encode the current vertex without referring to other vertices. That is, the inter-frame encoder 711 and the intra-frame encoder 713 can encode the current vertex itself instead of encoding the difference. This operation corresponds to encoding the difference between the reference vertex and the current vertex, treating the reference vertex as (0, 0, 0).

[0374] Furthermore, the encoding device 100 may include a frame header encoder that encodes a frame header for each frame. The frame header may include reference information indicating whether each vertex set included in the frame is an inter-frame vertex set or an intra-frame vertex set. Furthermore, the encoding device 100 may determine whether each vertex set is an inter-frame vertex set or an intra-frame vertex set based on spatial redundancy and temporal redundancy.

[0375] Figure 43 This is a block diagram showing another example configuration for switching between intra prediction and inter prediction for each vertex set in the encoding device 100 according to this embodiment. In this example, the encoding device 100 includes a vertex encoder 721, an intra reference vertex buffer 722, an intra vertex predictor 723, an inter reference vertex buffer 724, an inter vertex predictor 725, and a switch 726.

[0376] For example, among these components, the inter-frame reference vertex buffer 724 may correspond to Figure 42 In the example of the vertex buffer 712 , the other multiple components may correspond to the inter-frame encoder 711 .

[0377] Vertex encoder 721 obtains an intra-frame predicted vertex or an inter-frame predicted vertex as a predicted vertex via switch 726 and encodes the current vertex in the 3D mesh frame into a bitstream using the predicted vertex. For example, vertex encoder 721 encodes the current vertex by encoding the difference between the predicted vertex and the current vertex. Furthermore, vertex encoder 721 generates a reconstructed vertex by adding the difference to the predicted vertex and stores the reconstructed vertex in intra-frame reference vertex buffer 722 and inter-frame reference vertex buffer 724.

[0378] The intra-frame reference vertex buffer 722 stores reconstructed vertices for the current frame. The reconstructed vertices stored in the intra-frame reference vertex buffer 722 are referenced as intra-frame reference vertices. The inter-frame reference vertex buffer 724 stores not only reconstructed vertices for the current frame but also reconstructed vertices for reference frames encoded previously. The reconstructed vertices stored in the inter-frame reference vertex buffer 724 are referenced as inter-frame reference vertices.

[0379] The intra vertex predictor 723 generates an intra-prediction vertex by referring to an intra reference vertex from the intra reference vertex buffer 722. For example, the intra vertex predictor 723 may generate an intra-prediction vertex by selecting one of one or more intra reference vertices in the intra reference vertex buffer 722 as the intra-prediction vertex. The intra vertex predictor 723 may also generate an intra-prediction vertex using inter-reference connection information, which is connection information of reference frames.

[0380] The inter-frame vertex predictor 725 generates an inter-frame prediction vertex by referring to the inter-frame reference vertex from the inter-frame reference vertex buffer 724. For example, the inter-frame vertex predictor 725 may generate an inter-frame prediction vertex by selecting one of one or more intra-frame reference vertices in the inter-frame reference vertex buffer 724 as the inter-frame prediction vertex. The inter-frame vertex predictor 725 may also generate an inter-frame prediction vertex using inter-frame reference connection information, which is connection information of reference frames.

[0381] The switch 726 provides the intra-predicted vertex obtained by the intra vertex predictor 723 or the inter-predicted vertex obtained by the inter vertex predictor 725 as a predicted vertex to the vertex encoder 721. For example, the switch 726 may switch between the intra-predicted vertex and the inter-predicted vertex based on reference information in units of vertex sets consisting of one or more vertices.

[0382] The encoding apparatus 100 described above can encode a 3D mesh frame including an intra-frame vertex set and an inter-frame vertex set into a bitstream.

[0383] <Vertex Information Decoding Processing>

[0384] Figure 44 This is a flowchart illustrating mesh decoding processing according to this embodiment. In this example, first, the connection information of the second 3D mesh frame is applied to the connection information of the first 3D mesh frame (S201). Next, first reference information for the first vertex set in the first 3D mesh frame and second reference information for the second vertex set in the first 3D mesh frame are decoded from the bitstream (S202). The first 3D mesh frame is, for example, the current 3D mesh frame to be decoded.

[0385] Each reference information may be represented by one or more reference parameters. An example of a reference parameter is an index representing an identifier of a three-dimensional mesh frame. An example of an index is a numerical value such as 0, 1, 2, or 3. Another example of an index is a letter such as A, B, or C. Another example of an index is a Roman numeral such as I, II, or III.

[0386] Figure 35This is a conceptual diagram showing an example of reference information related to this embodiment. In this example, the reference information is represented by a number that serves as an index identifying a 3D mesh frame. Specifically, the reference information is represented by "#3." That is, in this example, the reference information indicates frame #3 among multiple frames #0 to #3. In another example, the reference information can indicate the current frame itself.

[0387] exist Figure 44 In the example of , next, when the first reference information indicates the first value, the first vertex set is decoded using the third vertex set in the second 3D mesh frame (S203). Here, the second 3D mesh frame is a 3D mesh frame that is different in time from the first 3D mesh frame. An example of the second 3D mesh frame is a previously decoded 3D mesh frame.

[0388] An example of the third vertex set is all vertices that enter the three-dimensional region. An example of a three-dimensional region is a cuboid represented by height, width, and depth from a reference point.

[0389] Figure 36 This is a conceptual diagram showing an example of a three-dimensional area related to this embodiment. Figure 36 In one example, a three-dimensional region is represented by a cuboid with a height of 3, a width of 2, and a depth of 3 from the reference point (0, 2, 3). In another example, a three-dimensional region is represented by a sphere with the coordinates of its center and a radius. In another example, a three-dimensional region is represented by a cube with the coordinates of its vertices and the sizes of its sides.

[0390] Figure 37 This is a conceptual diagram illustrating an example of a vertex set located inside a cuboid according to this embodiment. In this example, the third vertex set can be selected based on the fact that some of the vertices M, N, O, P, Q, and R of the three-dimensional mesh are located inside the cuboid. In one example, the third vertex set consists of the vertices within the upper cuboid, namely, M, N, and O. In another example, the third vertex set consists of the vertices within the lower cuboid, namely, P, Q, and R.

[0391] For example, the third vertex set is selected and used as the predicted vertex set for the first vertex set. An example of decoding the first vertex set using the third vertex set is to decode the differences between the third vertex set and the first vertex set and add the differences to the third vertex set to generate the first vertex set. Specifically, the differences between the corresponding vertices in the third vertex set and the first vertex set are decoded. Furthermore, when decoding multiple differences, the differences between the differences may also be decoded.

[0392] Figure 38This is a conceptual diagram showing an example of the relationship between the differences between the first and third vertex sets in this embodiment. Here, the third vertex set includes vertices M(10, 8, 5), N(14, 3, 3), and O(12, 2, 2). For example, (2, 1, 3), (3, 1, 1), and (1, 0, 1) are decoded as differences. In this case, the differences are added to the third vertex set, reconstructing vertices A(12, 9, 8), B(17, 4, 4), and C(13, 2, 3) into the first vertex set.

[0393] It's also possible to decode differences of differences. For example, (2, 1, 3), (1, 0, -2), and (-2, 1, 0) can be decoded as differences of differences. From differences of differences, (2, 1, 3), (2, 1, 3) + (1, 0, -2) = (3, 1, 1), and (3, 1, 1) + (-2, 1, 0) = (1, 0, 1) can also be derived as differences. Based on these differences, vertices A (12, 9, 8), B (17, 4, 4), and C (13, 2, 3) can be reconstructed as the first vertex set.

[0394] exist Figure 44 In the example of , when the second reference information indicates the second value, the second vertex set is decoded using the fourth vertex set in the first three-dimensional mesh frame ( S204 ).

[0395] An example of decoding the second vertex set using the fourth vertex set is to decode the difference between the fourth vertex set and the second vertex set and add the difference to the fourth vertex set to generate the second vertex set.

[0396] For example, the fourth vertex set has one vertex. When decoding the first vertex from the beginning of the second vertex set, the difference between the one vertex in the fourth vertex set and the first vertex from the beginning of the second vertex set is decoded. By adding the difference to the one vertex in the fourth vertex set, the first vertex from the beginning of the second vertex set is reconstructed.

[0397] Furthermore, when decoding the second vertex from the beginning of the second vertex set, the difference between the first vertex from the beginning of the second vertex set and the second vertex from the beginning of the second vertex set is decoded. By adding the difference to the first vertex from the beginning of the second vertex set, the second vertex from the beginning of the second vertex set is reconstructed.

[0398] Furthermore, when decoding the third vertex from the beginning of the second vertex set, the difference between the second vertex from the beginning of the second vertex set and the third vertex from the beginning of the second vertex set is decoded. By adding the difference to the second vertex from the beginning of the second vertex set, the third vertex from the beginning of the second vertex set is reconstructed.

[0399] Such differential decoding may be repeated. In addition, the decoding order of the vertices may correspond to the scanning order of the vertices.

[0400] Figure 39 This is a conceptual diagram showing an example of the relationship between the differences between the second and fourth vertex sets in this embodiment. For example, the fourth vertex set includes vertex C (13, 2, 3). Furthermore, the pairs (2, -1, -1), (4, -1, -2), and (-7, 1, 0) are decoded as differences.

[0401] In this case, by adding (2, -1, -1) to vertex C (13, 2, 3), we reconstruct vertex D (15, 1, 2). By adding (4, -1, -2) to vertex D (15, 1, 2), we reconstruct vertex E (19, 0, 0). Furthermore, by adding (-7, 1, 0) to vertex E (19, 0, 0), we reconstruct vertex F (12, 1, 0).

[0402] Furthermore, the fourth vertex set may be considered to partially overlap with the second vertex set in including vertices C, D, and E. Furthermore, when decoding vertex D in the second vertex set, the difference between vertices C and D may be decoded; when decoding vertex E in the second vertex set, the difference between vertices D and E may be decoded; and when decoding vertex F in the second vertex set, the difference between vertices E and F may be decoded.

[0403] Furthermore, the difference between vertex D and vertex E corresponds to the difference between the difference between vertex C and vertex D and the difference between vertex C and vertex E. In other words, the difference between vertex D and vertex E can be regarded as the difference between differences.

[0404] exist Figure 44 In the example of , then, the plurality of vertices of the first three-dimensional mesh frame are reconstructed ( S205 ). The decoded first vertex set and the decoded second vertex set are included in the plurality of reconstructed vertices of the first three-dimensional mesh frame.

[0405] Figure 40 This is a conceptual diagram illustrating an example of reconstructing multiple vertices of a first 3D mesh frame according to this embodiment. In this example, the multiple vertices of the first 3D mesh frame are reconstructed by combining a first vertex set and a second vertex set. In another example of reconstructing the first and second vertex sets, the multiple vertices of the first and second vertex sets may form a base mesh, and a portion of these vertices may be displaced using displacement vectors decoded from a bitstream.

[0406] exist Figure 44In the example of , first, the connection information of the second 3D mesh frame is applied to the connection information of the first 3D mesh frame (S201). That is, the connection information of the 3D mesh in the second 3D mesh frame is applied to the connection information of the multiple vertices reconstructed in the first 3D mesh frame.

[0407] For example, when the first reference information indicates the first value and at least one vertex of the triangle is included in the first vertex set, the connection information of the triangle can be copied from the second 3D mesh frame to the first 3D mesh frame.

[0408] Figure 41 This is a conceptual diagram illustrating an example of the application of connection information related to this embodiment. Here, vertices A, B, and C are included in the first vertex set decoded using the third vertex set of the second 3D mesh frame. Therefore, AB, BC, AC, CE, CF, BD, and BE are copied from the second 3D mesh frame and connected. Furthermore, DE and EF can also be considered connected based on the decoding order.

[0409] Here, the connection information related to the first vertex set decoded using the third vertex set of the second 3D mesh frame is copied from the second 3D mesh frame to the first 3D mesh frame. In other words, the connection information related to the vertex set to which inter-frame prediction is applied is copied from the second 3D mesh frame to the first 3D mesh frame.

[0410] However, regardless of whether the connection information is related to a vertex set to which inter-frame prediction is applied, the connection information may be copied from the second 3D mesh frame to the first 3D mesh frame. In other words, the connection information related to a vertex set to which intra-frame prediction is applied may be copied from the second 3D mesh frame to the first 3D mesh frame.

[0411] In addition, here, the connection information of the second 3D mesh frame is applied to the reconstructed vertex information of the first 3D mesh frame. However, the connection information of the second 3D mesh frame is not limited to the reconstructed vertex information of the first 3D mesh frame, and can also be used as the connection information of the first 3D mesh frame.

[0412] Specifically, the connection information of the second 3D mesh frame can also be used to decode the multiple vertices of the first 3D mesh frame. More specifically, the connection information of the second 3D mesh frame can be used according to the decoding order or prediction order of the multiple vertices of the first 3D mesh frame, and can also be used to determine the reference vertex or the predicted vertex, and can also be used to determine the inter-frame prediction or the intra-frame prediction.

[0413] Figure 45This is a block diagram showing an example configuration for switching between intra prediction and inter prediction for each vertex set in the decoding apparatus 200 according to this embodiment. In this example, the decoding apparatus 200 includes an inter decoder 811, a vertex buffer 812, an intra decoder 813, and a connection information buffer 814.

[0414] When the current vertex set to be processed is an inter-frame vertex set, the inter-frame decoder 811 decodes the current vertex to be processed in the current vertex set with reference to the inter-frame reference vertices in the vertex buffer 812. Here, the inter-frame vertex set is a vertex set consisting of inter-frame vertices that are processed with reference to vertices of an already processed frame.

[0415] For example, the inter-frame decoder 811 generates an inter-frame prediction vertex using an inter-frame reference vertex. The inter-frame prediction vertex may also be the inter-frame reference vertex itself. Furthermore, the inter-frame decoder 811 decodes the difference between the inter-frame prediction vertex and the current vertex and adds the difference to the inter-frame prediction vertex to reconstruct the current vertex.

[0416] The current vertex reconstructed by the inter decoder 811 is stored as a reconstructed vertex in the vertex buffer 812. The reconstructed vertex stored in the vertex buffer 812 can be referenced as an inter reference vertex in decoding of other vertices, for example.

[0417] Furthermore, the inter-frame decoder 811 may also use the connectivity information to decode the current vertex. Specifically, the inter-frame decoder 811 may also use the connectivity information to determine an inter-frame reference vertex. Furthermore, the inter-frame decoder 811 may also use the inter-frame reference vertex and the connectivity information to generate an inter-frame prediction vertex. Furthermore, the connectivity information may also be used to determine the current vertex and the decoding order.

[0418] Alternatively, the inter-frame decoder 811 may decode the current vertex by referring to the inter-frame reference connection information in the connection information buffer 814. That is, the inter-frame decoder 811 may decode the current vertex by using the connection information of the 3D mesh in the reference frame. In this case, the connection information of the 3D mesh in the reference frame may be used as the connection information of the 3D mesh in the current frame to be processed.

[0419] When the current vertex set to be processed is an intra vertex set, the intra decoder 813 decodes the current vertex to be processed in the current vertex set with reference to the intra reference vertex. Here, the intra vertex set is a vertex set consisting of intra vertices processed with reference to vertices of the current frame.

[0420] For example, the intra decoder 813 generates an intra-prediction vertex using an intra reference vertex. The intra-prediction vertex may also be the intra reference vertex itself. Furthermore, the intra decoder 813 decodes the difference between the intra-prediction vertex and the current vertex and adds the difference to the intra-prediction vertex to reconstruct the current vertex.

[0421] Furthermore, the intra decoder 813 may also use the connectivity information to decode the current vertex. Specifically, the intra decoder 813 may also use the connectivity information to determine an intra reference vertex. Furthermore, the intra decoder 813 may also use the intra reference vertex and the connectivity information to generate an intra prediction vertex. Furthermore, the connectivity information may also be used to determine the current vertex and the decoding order.

[0422] Alternatively, the intra decoder 813 may decode the current vertex by referring to the inter-frame reference connection information in the connection information buffer 814. That is, the intra decoder 813 may decode the current vertex by using the connection information of the 3D mesh in the reference frame. In this case, the connection information of the 3D mesh in the reference frame may be used as the connection information of the 3D mesh in the current frame to be processed.

[0423] The intra-frame decoder 813 decodes the connection information in the current frame and reconstructs the connection information. The connection information reconstructed by the intra-frame decoder 813 is stored as reconstructed connection information in the connection information buffer 814. The reconstructed connection information stored in the connection information buffer 814 can be referenced as inter-frame reference connection information when decoding other frames.

[0424] Inter-frame decoder 811 and intra-frame decoder 813 can also decode the current vertex without referencing other vertices. In other words, inter-frame decoder 811 and intra-frame decoder 813 can also decode the current vertex itself instead of decoding the difference. This operation corresponds to the operation of treating the reference vertex as (0, 0, 0) and decoding the difference between the reference vertex and the current vertex.

[0425] Furthermore, the decoding apparatus 200 may include a frame header decoder that decodes a frame header for each frame. The frame header may include reference information indicating whether each vertex set included in the frame is an inter-frame vertex set or an intra-frame vertex set.

[0426] Figure 46 This is a block diagram showing another example configuration for switching between intra prediction and inter prediction for each vertex set in the decoding device 200 according to this embodiment. In this example, the decoding device 200 includes a vertex decoder 821, an intra reference vertex buffer 822, an intra vertex predictor 823, an inter reference vertex buffer 824, an inter vertex predictor 825, and a switch 826.

[0427] For example, among these components, the inter-frame reference vertex buffer 824 may correspond to Figure 45 In the example of the vertex buffer 812 , the other multiple components may correspond to the inter-frame decoder 811 .

[0428] Vertex decoder 821 obtains an intra-frame predicted vertex or an inter-frame predicted vertex as a predicted vertex via switch 826 and uses the predicted vertex to decode the current vertex in the 3D mesh frame from the bitstream. For example, vertex decoder 821 decodes the current vertex by decoding the difference between the predicted vertex and the current vertex. Furthermore, vertex decoder 821 generates a reconstructed vertex by adding the difference to the predicted vertex and stores the reconstructed vertex in intra-frame reference vertex buffer 822 and inter-frame reference vertex buffer 824.

[0429] The intra-frame reference vertex buffer 822 stores reconstructed vertices for the current frame. The reconstructed vertices stored in the intra-frame reference vertex buffer 822 are referenced as intra-frame reference vertices. The inter-frame reference vertex buffer 824 stores not only reconstructed vertices for the current frame but also reconstructed vertices for previously decoded frames, i.e., reference frames. The reconstructed vertices stored in the inter-frame reference vertex buffer 824 are referenced as inter-frame reference vertices.

[0430] The intra vertex predictor 823 generates an intra-prediction vertex by referring to an intra reference vertex from the intra reference vertex buffer 822. For example, the intra vertex predictor 823 may generate an intra-prediction vertex by selecting one of one or more intra reference vertices in the intra reference vertex buffer 822 as the intra-prediction vertex. The intra vertex predictor 823 may also generate an intra-prediction vertex using inter-reference connection information, which is connection information of reference frames.

[0431] The inter-frame vertex predictor 825 generates an inter-frame prediction vertex by referring to the inter-frame reference vertex from the inter-frame reference vertex buffer 824. For example, the inter-frame vertex predictor 825 may generate an inter-frame prediction vertex by selecting one of one or more intra-frame reference vertices in the inter-frame reference vertex buffer 824 as the inter-frame prediction vertex. The inter-frame vertex predictor 825 may also generate an inter-frame prediction vertex using inter-frame reference connection information, which is connection information of reference frames.

[0432] The switch 826 provides the intra-predicted vertex obtained by the intra vertex predictor 823 or the inter-predicted vertex obtained by the inter vertex predictor 825 as a predicted vertex to the vertex decoder 821. For example, the switch 826 may switch between the intra-predicted vertex and the inter-predicted vertex based on reference information in units of vertex sets consisting of one or more vertices.

[0433] The decoding apparatus 200 described above can decode a 3D mesh frame including an intra-frame vertex set and an inter-frame vertex set from a bitstream.

[0434] <Supplementary information about encoding and decoding of vertex information>

[0435] The encoding and decoding processes of this embodiment can be applied to encoding of point position information in point group compression methods such as V-PCC and G-PCC, for example.

[0436] This embodiment illustrates an example of switching reference vertices based on parameters for each vertex set. For example, in the prediction process for multiple vertex coordinates to be coded, either inter-frame prediction or intra-frame prediction can be selected and applied for each vertex set. Here, inter-frame prediction uses previously coded vertex coordinates from a different frame than the vertices to be coded. Intra-frame prediction uses previously coded vertex coordinates from the same frame as the vertices to be coded.

[0437] However, the present invention is not limited to this example. For each vertex set, one of a prediction process that refers to coded vertex coordinates belonging to different frames and another prediction process that refers to coded vertex coordinates belonging to different frames may be selected and applied. Alternatively, for each vertex set, one of a prediction process that refers to coded vertex coordinates belonging to the same frame and another prediction process that refers to coded vertex coordinates belonging to the same frame may be selected and applied.

[0438] The number of vertices constituting a vertex set may be 1, 2, 3, or a number greater than 3. The number of vertices constituting a vertex set may be fixed or variable.

[0439] For example, multiple vertices that form one or more continuous faces (e.g., triangles and quadrilaterals) can be considered a set. Furthermore, the multiple vertices included in a set can also belong to the same object. Furthermore, the multiple vertices included in a set can also have interconnected connectivity. Furthermore, the vertices included in a set can also form the same mesh.

[0440] Alternatively, inter prediction or intra prediction may be switched for each vertex according to the prediction accuracy. Furthermore, one or more vertices continuously using inter prediction or one or more vertices continuously using intra prediction may constitute one set.

[0441] In addition, in the present disclosure, multiple vertex sets are included in a single frame, allowing intra-frame coded vertex sets and inter-frame coded vertex sets to be mixed in the same frame. Furthermore, it is possible to allow multiple vertex sets coded only intra-frame or multiple vertex sets coded only inter-frame to be included in a single frame. Furthermore, it is also possible to allow a single vertex set coded only intra-frame or a single vertex set coded only inter-frame to be included in a single frame.

[0442] Furthermore, in the inter-frame coded vertex set, inter-frame coding and intra-frame coding can be switched for each vertex.

[0443] Alternatively, reference information may be encoded for each vertex set. The reference information may also include information for switching between inter-frame coding and intra-frame coding. Specifically, the reference information may include identification information indicating a reference frame. Furthermore, the reference information may include a reference list indicating a list of frames that can be referenced.

[0444] Furthermore, the reference information may include mode information indicating whether inter-frame coding or intra-frame coding is used. That is, the reference information may include mode information indicating the mode of the application object among multiple modes including inter-frame coding and intra-frame coding. The mode information may also indicate the mode of the application object as a value.

[0445] Furthermore, the reference information may indicate a vertex set of a reference object. The reference information may indicate a region including the vertex set of the reference object. The reference information may indicate the number of vertices constituting the vertex set. The number of vertices constituting the vertex set may be fixed and encoded as parameter information different from the reference information encoded for each vertex set.

[0446] The reference information may be encoded in the header of the bitstream or as part of the vertex information. Furthermore, the reference information may be determined based on spatial redundancy and temporal redundancy or based on coding efficiency.

[0447] Figure 47 This is a conceptual diagram showing a first example of a method for specifying a vertex set for inter-frame prediction. In this example, reference information includes a reference frame index, the number of reference vertices, and one or more reference vertex indices.

[0448] Specifically, in this example, the reference information has values of "3, 2, 1, 2." These values correspond to the reference frame index (#Frame), the reference vertex number (#number), and the reference vertex index (#index1, #index2, #indexN, ...). The first value "3" indicates the previously decoded frame #3, the second value "2" indicates the reference vertex number, and the third and fourth values "1" and "2" indicate the reference vertex indexes, respectively. In other words, as a whole, "3, 2, 1, 2" refers to A (6, 8, 9) and B (10, 6, 7) of the previously decoded frame #3, and these vertices are used for reference.

[0449] Figure 48 is a syntax diagram showing the syntax structure corresponding to the first example of the method of specifying a vertex set for inter-frame prediction. Figure 48 The syntax structure shown signals the reference information.

[0450] Figure 49 This is a conceptual diagram showing a second example of a method for specifying a vertex set for inter-frame prediction. In this example, reference information includes a frame index, a start index, and an end index.

[0451] Specifically, in this example, the reference information has a value of "3 (1, 3)". This value corresponds to the reference frame index (#Frame), the first index of the reference vertex (start index), and the last index of the reference vertex (end index).

[0452] The initial value "3" indicates the previously decoded frame #3. Furthermore, "(1, 3)" indicates the start and end indices for obtaining vertices within the previously decoded frame. In other words, "3(1, 3)" as a whole refers to A(6, 8, 9), B(10, 6, 7), and C(14, 8, 9) of the previously decoded frame #3. In this example, the start and end indices are specified, and the indices between them are not specified.

[0453] Figure 50 is a syntax diagram showing the syntax structure corresponding to the second example of the method of specifying a vertex set for inter-frame prediction. Figure 50 The syntax structure shown signals the reference information.

[0454] Figure 51 This is a syntax diagram showing a modified example of the syntax structure corresponding to the second example of the method of specifying a vertex set for inter-frame prediction. Figure 51 The syntax structure shown here signals reference information. Specifically, during the period from the start index to the end index, vertices not included in the reference vertex set can be designated as exceptions. Vertices designated as exceptions may not be used as reference vertices.

[0455] Figure 52 This is a conceptual diagram showing a third example of a method for specifying a vertex set for inter-frame prediction. In this example, the reference information includes a frame index, a reference position (x, y, z), and a reference size (height, width, depth).

[0456] Specifically, in this example, the reference information has the value "3, (5, 4, 7), (5, 4, 2)." This value specifies the vertex at a height of 5, a width of 4, and a depth of 2 from the reference position (5, 4, 7) in the previously decoded frame #3. Here, the reference position, height, width, and depth form a rectangular parallelepiped reference area.

[0457] In this example, vertices within the cuboid are selected in the reference frame. In reference frame #3, there are two vertices, A and B, within a range of height 5, width 4, and depth 2 from the reference position. Therefore, these two vertices (A (6, 8, 9) and B (10, 6, 7)) are used as reference vertices.

[0458] In another example, the reference position can be derived using previously decoded vertices and may not be signaled in the bitstream. In another example, the height, width, and depth can be predetermined or signaled in the header.

[0459] Figure 53 is a syntax diagram showing the syntax structure corresponding to the third example of the method of specifying a vertex set for inter-frame prediction. Figure 53 The syntax structure shown will signal the reference information.

[0460] Figure 54 This is a conceptual diagram showing in two dimensions the vertex set specified in the third example of the method of specifying a vertex set for inter-frame prediction. Figure 54 Indicates about Figure 52 The specified method shown (assuming no depth information) selects a set of vertices in two dimensions. In this example, vertices C1, C2, and C3 are decoded. Next, vertex C4 is decoded. In addition, decoding apparatus 200 decodes reference information from the bitstream.

[0461] For example, the reference information indicates a reference position (1, 3), a height of 6, and a width of 3, representing a rectangular area in two-dimensional space (a cuboid in three-dimensional space). Vertices R2, R5, and R6 are located within the rectangle with a height of 6 and a width of 3 from the reference position (111, 3). Since the three vertices R2, R5, and R6 exist within the rectangle, the three vertices C4, C5, and C6 are subsequently decoded in the current frame using reference vertices R2, R5, and R6, respectively.

[0462] Figure 55 This is a conceptual diagram showing a fourth example of a method for specifying a vertex set for inter-frame prediction. In this example, reference information includes a frame index, a reference position (x, y, z), and a radius.

[0463] Specifically, in this example, the reference information has a value of "3, (6, 4, 5), 4." This value specifies the vertex within a radius of "4" from the reference position (6, 4, 5) in the previously decoded frame #3. The reference position and radius form a reference area for the sphere.

[0464] In this example, vertices within the sphere are selected in the reference frame. In reference frame #3, there are two vertices, A (6, 8, 9) and B (10, 6, 7), within a radius of "4" from the reference position (6, 4, 5). Therefore, these two vertices, A (6, 8, 9) and B (10, 6, 7), are used as reference vertices.

[0465] Figure 56 is a syntax diagram showing the syntax structure corresponding to the fourth example of the method of specifying a vertex set for inter-frame prediction. Figure 56 The syntax structure shown will signal the reference information.

[0466] Figure 57 This is a conceptual diagram showing in two dimensions the vertex set specified in the fourth example of the method of specifying a vertex set for inter-frame prediction. Figure 57 Indicates about Figure 55 The specified method shown (assuming no depth information) selects a set of vertices in two dimensions. In this example, vertices C1, C2, and C3 are decoded. Next, vertex C4 is decoded. In addition, decoding apparatus 200 decodes reference information from the bitstream.

[0467] For example, the reference information indicates a reference position (2, 5) and a radius of 2, representing the area of a circle in two-dimensional space (a sphere in three-dimensional space). Vertices R2, R5, and R6 are located within the circle with a radius of 2 from the reference position (2, 5). Since the three vertices R2, R5, and R6 are within the circle, the three vertices C4, C5, and C6 are subsequently decoded in the current frame using reference vertices R2, R5, and R6, respectively.

[0468] Figure 58 This is a conceptual diagram showing a fifth example of a method for specifying a vertex set for inter-frame prediction. In this example, the reference information includes a frame index, the size of each vertex set, and an index of a vertex set to be referenced.

[0469] Specifically, in this example, the reference information has a value of "3, 2, 2." This value specifies a vertex in vertex set #2 of previously decoded frame #3. In this example, vertex set #2 of reference frame #3 contains two vertices, C(14, 8, 9) and D(10, 10, 11). Therefore, these two vertices, C(14, 8, 9) and D(10, 10, 11), are used as reference vertices.

[0470] Figure 59 is a syntax diagram showing the syntax structure corresponding to the fifth example of the method of specifying a vertex set for inter-frame prediction. Figure 59 The syntax structure shown will signal the reference information.

[0471] <Representative Examples>

[0472] Figure 60 This is a flowchart showing an example of basic encoding processing related to this embodiment. For example, Figure 24 The circuit 151 of the encoding device 100 shown in FIG. 1 is in operation. Figure 60 The encoding process shown.

[0473] Specifically, the circuit 151 encodes first reference information for a first vertex set in a first 3D mesh frame and second reference information for a second vertex set in the first 3D mesh frame into a bitstream (S301). Furthermore, the circuit 151 encodes the first vertex set into a bitstream (S302). Furthermore, the circuit 151 encodes the second vertex set into a bitstream (S303).

[0474] Here, when a third vertex set in a second 3D mesh frame that is temporally different from the first 3D mesh frame is used to encode the first vertex set, the first reference information indicates a first value. When a fourth vertex set in the first 3D mesh frame is used to encode the second vertex set, the second reference information indicates a second value.

[0475] Therefore, when encoding the first vertex set and the second vertex set in the same 3D mesh frame, inter-frame prediction may be applied to the first vertex set and intra-frame prediction may be applied to the second vertex set, thereby sometimes reducing the amount of code.

[0476] For example, when the third vertex set is used in encoding the first vertex set, circuit 151 may also encode the first vertex set using the connection information of the three-dimensional mesh in the second three-dimensional mesh frame. This allows the connection information to be used in inter-frame prediction. Consequently, encoding of the connection information may be omitted in inter-frame prediction. Consequently, the code size of the connection information may be reduced in inter-frame prediction.

[0477] Furthermore, for example, the circuit 151 may encode the first vertex set using the connection information of the 3D mesh in the second 3D mesh frame, regardless of whether the third vertex set is used in encoding the first vertex set. Furthermore, the circuit 151 may encode the second vertex set using the connection information of the 3D mesh in the second 3D mesh frame, regardless of whether the fourth vertex set is used in encoding the second vertex set.

[0478] Therefore, whether inter-frame prediction or intra-frame prediction, the connection information can sometimes be used. Therefore, whether inter-frame prediction or intra-frame prediction, the encoding of the connection information can sometimes be omitted. Therefore, whether inter-frame prediction or intra-frame prediction, the code size of the connection information can sometimes be reduced.

[0479] Alternatively, for example, each reference information may indicate a value for identifying a 3D mesh frame of a reference object. This may allow efficient specification of the 3D mesh frame of the reference object. Furthermore, the first value may be a value for identifying a second 3D mesh frame, and the second value may be a value for identifying the first 3D mesh frame.

[0480] In addition, for example, each reference information may indicate whether to refer to the second three-dimensional mesh frame as a value. This may allow efficient designation of whether to use inter-frame prediction. Furthermore, the first value may also be a value indicating whether to refer to the second three-dimensional mesh frame.

[0481] In addition, for example, each reference information may indicate whether to refer to the first three-dimensional mesh frame as a value. This may allow efficient designation of whether to use intra-frame prediction. Furthermore, the second value may also be a value indicating whether to refer to the first three-dimensional mesh frame.

[0482] Furthermore, for example, the first 3D mesh frame may be the 3D mesh frame to be encoded. This may allow efficient encoding of each vertex set in the 3D mesh frame to be encoded.

[0483] Furthermore, for example, the second 3D mesh frame may be an already encoded 3D mesh frame. Thus, when inter-frame prediction is used to encode the first vertex set, the already encoded 3D mesh frame may be used to efficiently encode the first vertex set.

[0484] Figure 61 This is a flowchart showing an example of basic encoding processing related to this embodiment. For example, Figure 25 The circuit 251 of the decoding device 200 shown in FIG. Figure 61 The decoding process shown.

[0485] Specifically, the circuit 251 decodes first reference information for a first vertex set in a first 3D mesh frame and second reference information for a second vertex set in the first 3D mesh frame from a bitstream ( S401 ).

[0486] Furthermore, when the first reference information indicates a first value, the circuit 251 decodes the first vertex set from the bitstream using a third vertex set in a second 3D mesh frame that is temporally different from the first 3D mesh frame (S402). Furthermore, when the second reference information indicates a second value, the circuit 251 decodes the second vertex set from the bitstream using a fourth vertex set in the first 3D mesh frame (S403).

[0487] Thus, when decoding the first vertex set and the second vertex set in the same 3D mesh frame, inter-frame prediction may be applied to the first vertex set and intra-frame prediction may be applied to the second vertex set, thereby sometimes reducing the amount of code.

[0488] For example, when the first reference information indicates the first value, circuit 251 may decode the first vertex set using the connection information of the 3D mesh in the second 3D mesh frame. This allows the connection information to be used in inter-frame prediction. Consequently, decoding of the connection information may be omitted in inter-frame prediction. Consequently, the code size of the connection information may be reduced in inter-frame prediction.

[0489] Furthermore, for example, the circuit 251 may decode the first vertex set using the connection information of the three-dimensional mesh in the second three-dimensional mesh frame, regardless of whether the first reference information indicates the first value or the second value. Furthermore, the circuit 251 may decode the second vertex set using the connection information of the three-dimensional mesh in the second three-dimensional mesh frame, regardless of whether the second reference information indicates the first value or the second value.

[0490] Therefore, whether it is inter-frame prediction or intra-frame prediction, the connection information can sometimes be used. Therefore, whether it is inter-frame prediction or intra-frame prediction, the decoding of the connection information can sometimes be omitted. Therefore, whether it is inter-frame prediction or intra-frame prediction, the code amount of the connection information can sometimes be reduced.

[0491] Alternatively, for example, each reference information may indicate a value for identifying a 3D mesh frame of a reference object. This may allow efficient specification of the 3D mesh frame of the reference object. Furthermore, the first value may be a value for identifying a second 3D mesh frame, and the second value may be a value for identifying the first 3D mesh frame.

[0492] In addition, for example, each reference information may also indicate whether to refer to the second three-dimensional mesh frame as a value. This may allow efficient designation of whether to use inter-frame prediction. Furthermore, the first value may also be a value indicating whether to refer to the second three-dimensional mesh frame.

[0493] In addition, for example, each reference information may indicate whether to refer to the first three-dimensional mesh frame as a value. This may allow efficient designation of whether to use intra-frame prediction. Furthermore, the second value may also be a value indicating whether to refer to the first three-dimensional mesh frame.

[0494] Furthermore, for example, the first 3D mesh frame may also be the 3D mesh frame to be decoded. This may allow each vertex set in the 3D mesh frame to be decoded to be efficiently decoded.

[0495] Alternatively, for example, the second 3D mesh frame may be a decoded 3D mesh frame. Thus, when inter-frame prediction is used in decoding the first vertex set, the decoded 3D mesh frame may be used to efficiently decode the first vertex set.

[0496] Figure 6273 is a block diagram showing still another configuration example of the encoding device 100 according to this embodiment. In this example, the encoding device 100 includes a reference information encoder 731 and a vertex set encoder 732 .

[0497] The reference information encoder 731 is, for example, an electric circuit and may correspond to the pre-processor 104 and the post-processor 105 described above, or may be implemented by the circuit 151 and the memory 152 described above.

[0498] The vertex set encoder 732 is, for example, an electrical circuit and may correspond to the aforementioned vertex information encoder 101 , inter-frame encoder 711 , intra-frame encoder 713 , and vertex encoder 721 , and may also be implemented by the aforementioned circuit 151 and memory 152 .

[0499] For example, the reference information encoder 731 encodes first reference information for a first vertex set in a first 3D mesh frame and second reference information for a second vertex set in the first 3D mesh frame into a bitstream.

[0500] Furthermore, when the first reference information indicates a first value, the vertex set encoder 732 encodes the first vertex set into the bitstream using a third vertex set in a second three-dimensional mesh frame that is temporally different from the first three-dimensional mesh frame. Furthermore, when the second reference information indicates a second value, the vertex set encoder 732 encodes the second vertex set into the bitstream using a fourth vertex set in the first three-dimensional mesh frame.

[0501] Thus, when encoding the first vertex set and the second vertex set in the same 3D mesh frame, inter-frame prediction can sometimes be applied to the first vertex set, while intra-frame prediction can sometimes be applied to the second vertex set. This can sometimes reduce the amount of code. Furthermore, encoding device 100 may also include other encoders for encoding other information.

[0502] Figure 63 83 is a block diagram showing still another configuration example of the decoding device 200 according to this embodiment. In this example, the decoding device 200 includes a reference information decoder 831 and a vertex set decoder 832 .

[0503] The reference information decoder 831 is, for example, an electric circuit and may correspond to the pre-processor 204 and post-processor 205 described above, or may be implemented by the circuit 251 and memory 252 described above.

[0504] The vertex set decoder 832 is, for example, an electrical circuit and may correspond to the aforementioned vertex information decoder 201 , inter decoder 811 , intra decoder 813 , and vertex decoder 821 , and may also be implemented by the aforementioned circuit 251 and memory 252 .

[0505] For example, the reference information decoder 831 decodes first reference information for a first vertex set in a first 3D mesh frame and second reference information for a second vertex set in the first 3D mesh frame from the bitstream.

[0506] Furthermore, when the first reference information indicates the first value, the vertex set decoder 832 decodes the first vertex set from the bitstream using the third vertex set in the second three-dimensional mesh frame that is temporally different from the first three-dimensional mesh frame. Furthermore, when the second reference information indicates the second value, the vertex set decoder 832 decodes the second vertex set from the bitstream using the fourth vertex set in the first three-dimensional mesh frame.

[0507] Thus, when decoding the first vertex set and the second vertex set in the same 3D mesh frame, inter-frame prediction can sometimes be applied to the first vertex set, while intra-frame prediction can sometimes be applied to the second vertex set. This can sometimes reduce the amount of code. Furthermore, decoding apparatus 200 may also include other decoders for decoding other information.

[0508] <Other examples>

[0509] While the encoding device 100 and the decoding device 200 have been described above according to the embodiments, the encoding device 100 and the decoding device 200 are not limited to the embodiments. The embodiments may be modified as conceived by those skilled in the art, and multiple components of the embodiments may be arbitrarily combined.

[0510] For example, a process executed by a specific component in the embodiment may be executed by another component instead of the specific component. In addition, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0511] In the above description, the second three-dimensional mesh frame refers to any three-dimensional mesh frame different from the first three-dimensional mesh frame, and may not necessarily refer to a specific three-dimensional mesh frame.

[0512] The "vertex set" in the present disclosure corresponds to, for example, a plurality of vertices of a unit for encoding a displacement vector. The "vertex set" may also be referred to by other expressions. For example, the "vertex set" may also be referred to as a "vertex group". In addition, the "vertex set" is associated with a plurality of displacement vectors corresponding to a plurality of vertices, and therefore may be referred to as a "displacement vector group", "vector group" or "motion group". Furthermore, other names may also be used. The "vertex set" described in the present disclosure may also be replaced by any one of them.

[0513] In addition, the “vertex set” in the present disclosure may be a set of divided data obtained by dividing arbitrary data other than displacement vectors corresponding to a plurality of vertices into a plurality of data when encoding or decoding.

[0514] Alternatively, the unit for dividing the vertices that make up a mesh may be referred to as a "vertex set" or "vertex group," and the unit for dividing the data of the vertices that make up the mesh may be referred to as a "data set" or "data group," thereby distinguishing the terms. In this case, the term "vertex set" in this disclosure may be replaced by any of these terms, depending on the object referred to by each term.

[0515] Furthermore, the number of vertices or data corresponding to the vertices constituting a "vertex set" in the present disclosure may be fixed or variable within any coding unit, such as a mesh or a frame. Furthermore, the number of vertices or data corresponding to the vertices constituting a "vertex set" may also be derived based on other coding units, such as sub-meshes.

[0516] Furthermore, as described above, at least a portion of the various components of the present disclosure can be implemented as an integrated circuit. At least a portion of the various processes of the present disclosure can also be utilized as an encoding method or a decoding method. A program for causing a computer to execute the encoding method or the decoding method can also be utilized. Furthermore, a non-transitory computer-readable recording medium containing the program can also be utilized. Furthermore, a bitstream for causing the decoding device 200 to perform a decoding process can also be utilized.

[0517] Furthermore, at least a portion of the various configurations and processes disclosed herein may be utilized as a transmitting device, a receiving device, a transmitting method, and a receiving method. A program for causing a computer to execute the transmitting method or the receiving method may also be utilized. Furthermore, a non-transitory computer-readable recording medium having the program recorded thereon may also be utilized.

[0518] Industrial applicability

[0519] The present disclosure is useful, for example, for encoding devices, decoding devices, transmitting devices, and receiving devices related to three-dimensional meshes, and can be applied to computer graphics systems and three-dimensional data display systems.

[0520] Description of labels

[0521] 100 encoding device

[0522] 101, 121, 144 vertex information encoder

[0523] 102, 145 Connection Information Encoder

[0524] 103, 122 Attribute Information Encoder

[0525] 104, 204, 521 preprocessors

[0526] 105, 205, 623 postprocessors

[0527] 110 Three-dimensional data encoding system

[0528] 111, 211 controllers

[0529] 112, 212 input and output processors

[0530] 113 3D Data Encoder

[0531] 114 System Multiplexer

[0532] 115 3D Data Generator

[0533] 123 Metadata Encoder

[0534] 124 Multiplexer

[0535] 131 Vertex Image Generator

[0536] 132 Attribute Image Generator

[0537] 133 Metadata Generator

[0538] 134 Image Encoder

[0539] 141 Two-dimensional data encoder

[0540] 142 Grid Data Encoder

[0541] 143 Texture Encoder

[0542] 148 Describing the Encoder

[0543] 151, 251 circuits

[0544] 152, 252 memory

[0545] 200 Decoding Device

[0546] 201, 221, 244 vertex information decoder

[0547] 202, 245 Connection Information Decoder

[0548] 203, 222 Attribute Information Decoder

[0549] 210 3D data decoding system

[0550] 213 3D Data Decoder

[0551] 214 System Demultiplexer

[0552] 215, 247 Prompt

[0553] 216 User Interface

[0554] 223 Metadata Decoder

[0555] 224 Demultiplexer

[0556] 231 Vertex Information Generator

[0557] 232 Attribute Information Generator

[0558] 234, 631 video decoder

[0559] 241 2D Data Decoder

[0560] 242 Grid Data Decoder

[0561] 243 Texture Decoder

[0562] 246 Grid Reconstructor

[0563] 248 Describe Decoder

[0564] 300 Network

[0565] 310 External Connector

[0566] 511 Volume Capture

[0567] 512 Projector

[0568] 513 Basic Grid Encoder

[0569] 514 displacement encoder

[0570] 515 Attribute Encoder

[0571] 516 Other types of encoders

[0572] 522 encoding processor

[0573] 613 Basic Grid Decoder

[0574] 614 displacement decoder

[0575] 615 Attribute Decoder

[0576] 616 Other types of decoders

[0577] 617 3D Reconstructor

[0578] 622 decoding processor

[0579] 632 Image Decompressor

[0580] 633 Inverse Quantizer

[0581] 634 Inverse Wavelet Transformer

[0582] 711 Interframe Encoder

[0583] 712, 812 vertex buffer

[0584] 713 Intraframe Encoder

[0585] 714, 814 Connection Information Buffer

[0586] 721 Vertex Encoder

[0587] 722, 822 Intra-frame reference vertex buffer

[0588] 723, 823 Intra-frame vertex predictor

[0589] 724, 824 inter-frame reference vertex buffer

[0590] 725, 825 inter-frame vertex predictor

[0591] 726, 826 switcher

[0592] 731 Reference Information Encoder

[0593] 732 Vertex Set Encoder

[0594] 811 Decoder

[0595] 813 Intraframe Decoder

[0596] 821 Vertex Decoder

[0597] 831 Reference Information Decoder

[0598] 832 Vertex Set Decoder

Claims

1. A coding method, wherein: include: encoding first reference information for a first vertex set in a first 3D mesh frame and second reference information for a second vertex set in the first 3D mesh frame into a bitstream; encoding the first vertex set into the bitstream; as well as encoding the second vertex set into the bitstream, When a third vertex set in a second 3D mesh frame that is temporally different from the first 3D mesh frame is used in encoding the first vertex set, the first reference information indicates a first value. When the fourth vertex set in the first three-dimensional mesh frame is used to encode the second vertex set, the second reference information indicates a second value.

2. The encoding method according to claim 1, wherein: When the third vertex set is used in encoding the first vertex set, the first vertex set is encoded using connection information of the three-dimensional mesh in the second three-dimensional mesh frame.

3. The encoding method according to claim 1, wherein: Regardless of whether the third vertex set is used in encoding the first vertex set and whether the fourth vertex set is used in encoding the second vertex set, the first vertex set and the second vertex set are encoded using connection information of the 3D mesh in the second 3D mesh frame.

4. The encoding method according to any one of claims 1 to 3, wherein: The first reference information and the second reference information each indicate a value for identifying a three-dimensional mesh frame as a reference target.

5. The encoding method according to any one of claims 1 to 3, wherein: The first reference information and the second reference information each indicate, as a value, whether to refer to the second three-dimensional mesh frame.

6. The encoding method according to any one of claims 1 to 3, wherein: The first reference information and the second reference information each indicate, as a value, whether to refer to the first three-dimensional mesh frame.

7. The encoding method according to any one of claims 1 to 3, wherein: The first three-dimensional grid frame is a three-dimensional grid frame to be encoded.

8. The encoding method according to any one of claims 1 to 3, wherein: The second three-dimensional grid frame is an encoded three-dimensional grid frame.

9. A decoding method, wherein: include: decoding first reference information for a first vertex set in a first 3D mesh frame and second reference information for a second vertex set in the first 3D mesh frame from a bitstream; When the first reference information indicates a first value, decoding the first vertex set from the bitstream using a third vertex set in a second 3D mesh frame that is temporally different from the first 3D mesh frame; as well as When the second reference information indicates a second value, the second vertex set is decoded from the bitstream using a fourth vertex set in the first three-dimensional mesh frame.

10. The decoding method according to claim 9, wherein: When the first reference information indicates the first value, the first vertex set is decoded using connection information of the three-dimensional mesh in the second three-dimensional mesh frame.

11. The decoding method according to claim 9, wherein: Regardless of whether the first reference information indicates the first value or the second value, and regardless of whether the second reference information indicates the first value or the second value, the first vertex set and the second vertex set are decoded using connection information of the 3D mesh in the second 3D mesh frame.

12. The decoding method according to any one of claims 9 to 11, wherein: The first reference information and the second reference information each indicate a value for identifying a three-dimensional mesh frame as a reference target.

13. The decoding method according to any one of claims 9 to 11, wherein: The first reference information and the second reference information each indicate, as a value, whether to refer to the second three-dimensional mesh frame.

14. The decoding method according to any one of claims 9 to 11, wherein: The first reference information and the second reference information each indicate, as a value, whether to refer to the first three-dimensional mesh frame.

15. The decoding method according to any one of claims 9 to 11, wherein: The first three-dimensional grid frame is a three-dimensional grid frame to be decoded.

16. The decoding method according to any one of claims 9 to 11, wherein: The second three-dimensional grid frame is a decoded three-dimensional grid frame.

17. An encoding device, wherein: have: Memory; and circuitry capable of accessing said memory, The circuit is in action: encoding first reference information for a first vertex set in a first 3D mesh frame and second reference information for a second vertex set in the first 3D mesh frame into a bitstream; encoding the first vertex set into the bitstream; as well as encoding the second vertex set into the bitstream, When a third vertex set in a second 3D mesh frame that is temporally different from the first 3D mesh frame is used in encoding the first vertex set, the first reference information indicates a first value. When the fourth vertex set in the first three-dimensional mesh frame is used to encode the second vertex set, the second reference information indicates a second value.

18. A decoding device, wherein: have: Memory; and circuitry capable of accessing said memory, The circuit is in action: decoding first reference information for a first vertex set in a first 3D mesh frame and second reference information for a second vertex set in the first 3D mesh frame from a bitstream; When the first reference information indicates a first value, decoding the first vertex set from the bitstream using a third vertex set in a second 3D mesh frame that is temporally different from the first 3D mesh frame; as well as When the second reference information indicates a second value, the second vertex set is decoded from the bitstream using a fourth vertex set in the first three-dimensional mesh frame.

Images