Encoding device, decoding device, encoding method, and decoding method

Abstract

The encoding device (100) is provided with a memory (152) and a circuit (151). When a first triangle satisfies a condition during operation, a circuit (151) selects a mode for encoding a vertex to be encoded of a second triangle from a first mode set including a normal mode for encoding the vertex to be encoded using the first triangle; if the first triangle does not satisfy the condition, selecting a mode from a second mode set including an exceptional mode in which a vertex to be encoded is encoded using a third triangle different from both the first triangle and the second triangle; and coding the coding object vertex according to the mode.

Description

Technical Field

[0001] The present disclosure relates to encoding devices and the like. Background Art

[0002] Patent Document 1 proposes a method and apparatus for encoding and decoding three-dimensional mesh data.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2006-187015 Summary of the invention

[0006] Problems to be solved by the invention

[0007] Further improvement is desired with respect to the encoding process of three-dimensional data, etc. The present disclosure aims to improve the encoding process of three-dimensional data, etc.

[0008] Means used to solve problems

[0009] A coding device according to a technical solution of the present disclosure comprises: a memory; and a circuit capable of accessing the memory; when the circuit is in operation, if the first triangle satisfies a condition, the circuit selects a mode for encoding the coding object vertex of the second triangle from a first mode set, the first mode set including a normal mode for encoding the coding object vertex using the first triangle; if the first triangle does not satisfy the condition, the circuit selects the mode from a second mode set, the second mode set including an exceptional mode for encoding the coding object vertex using a third triangle different from both the first triangle and the second triangle; and the coding object vertex is encoded according to the mode.

[0010] Effects of the Invention

[0011] The present disclosure can contribute to improvements in encoding processing of three-dimensional data and the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a conceptual diagram showing a three-dimensional mesh according to an embodiment of the present invention.

[0013] Figure 2 This is a conceptual diagram showing basic elements of a three-dimensional mesh according to an embodiment of the present invention.

[0014] Figure 3 This is a conceptual diagram showing mapping related to the implementation method.

[0015] Figure 4 This is a block diagram showing a configuration example of a coding and decoding system according to an embodiment of the present invention.

[0016] Figure 5 This is a block diagram showing a configuration example of an encoding device according to the embodiment.

[0017] Figure 6 This is a block diagram showing another configuration example of the encoding device according to the embodiment.

[0018] Figure 7 This is a block diagram showing a configuration example of a decoding device according to an embodiment.

[0019] Figure 8 This is a block diagram showing another configuration example of a decoding device according to the embodiment.

[0020] Fig. 9 This is a conceptual diagram showing an example of the structure of a bit stream related to the embodiment.

[0021] Fig.10 This is a conceptual diagram showing another example of the configuration of a bit stream according to the embodiment.

[0022] Fig.11 This is a conceptual diagram showing yet another example of the configuration of a bit stream according to the embodiment.

[0023] Fig.12 It is a block diagram showing a specific example of a coding and decoding system related to the embodiment.

[0024] Fig.13 It is a conceptual diagram showing a configuration example of point cloud data according to an embodiment.

[0025] Fig.14 This is a conceptual diagram showing an example of a data file of point cloud data according to an embodiment.

[0026] Fig.15 This is a conceptual diagram showing a configuration example of mesh data according to the embodiment.

[0027] Fig.16 This is a conceptual diagram showing an example of a data file of mesh data according to the embodiment.

[0028] Fig.17 This is a conceptual diagram showing the types of three-dimensional data related to the embodiment.

[0029] Fig.18 This is a block diagram showing an example of the configuration of a three-dimensional 1 data encoder according to the embodiment.

[0030] Fig.19 This is a block diagram showing a configuration example of a three-dimensional data decoder according to the embodiment.

[0031] Fig. 20 This is a block diagram showing another configuration example of the three-dimensional data encoder according to the embodiment.

[0032] Fig.21 This is a block diagram showing another configuration example of a three-dimensional data decoder according to the embodiment.

[0033] Fig. 22 It is a conceptual diagram showing a specific example of encoding processing related to the implementation mode.

[0034] Fig.23 It is a conceptual diagram showing a specific example of the decoding process related to the implementation mode.

[0035] Fig.24 This is a block diagram showing an example of installation of an encoding device related to the embodiment.

[0036] Fig.25 This is a block diagram showing an implementation example of a decoding device related to the embodiment.

[0037] Fig.26 This is a conceptual diagram showing a parallelogram prediction method related to a reference example.

[0038] Fig. 27 This is a conceptual diagram showing a polynomial prediction method related to a reference example.

[0039] Fig.28 This is a flowchart showing an example of encoding processing related to the first aspect.

[0040] Fig.29 This is a flowchart showing a specific example of the encoding process related to the first aspect.

[0041] Fig.30 This is a flowchart showing another specific example of the encoding process related to the first aspect.

[0042] Fig.31 This is a conceptual diagram showing examples of three angles related to the first form.

[0043] Fig.32 This is a conceptual diagram showing another example of three angles related to the first aspect.

[0044] Fig.33 This is a conceptual diagram showing still another example of three angles related to the first aspect.

[0045] Fig.34 This is a syntax diagram showing an example of the syntax structure of vertex information for a plurality of vertices according to the first aspect.

[0046] Fig.35 This is a syntax diagram showing an example of the syntax structure of vertex information for one vertex in the first form.

[0047] Fig.36This is a syntax diagram showing another syntax structure example of vertex information for one vertex related to the first form.

[0048] Fig.37 This is a table diagram showing an example of a table used for angle prediction.

[0049] Fig.38 This is a table diagram showing another example of a table used for angle prediction.

[0050] Fig.39 This is a conceptual diagram showing an example of prediction using the angles of the previous triangle vertices.

[0051] Fig.40 This is a conceptual diagram showing an example of using the prediction of the angles with respect to the vertices of the previous three-dimensional mesh.

[0052] Fig.41 This is a block diagram showing an example of the configuration of an encoding device related to the first aspect.

[0053] Fig.42 This is a flowchart showing an example of decoding processing related to the first aspect.

[0054] Fig.43 This is a flowchart showing a specific example of the decoding process related to the first aspect.

[0055] Fig.44 This is a flowchart showing another specific example of the decoding process related to the first aspect.

[0056] Fig.45 This is a table diagram showing an example of a lookup table for a sine function and a cosine function.

[0057] Fig.46 This is a block diagram showing a configuration example of a decoding device related to the first aspect.

[0058] Fig.47 This is a flowchart showing an example of encoding processing related to the second aspect.

[0059] Fig.48 This is a conceptual diagram showing an example of a mesh related to the second form.

[0060] Fig.49 is a conceptual diagram showing an example of a fragmented mesh.

[0061] Fig.50 This is a conceptual diagram showing an example of a mesh with repeated vertices.

[0062] Fig.51 This is a conceptual diagram showing an example of exception processing using a vertex set that has been encoded in the past.

[0063] Fig.52 This is a flowchart showing a specific example of the encoding process related to the second aspect.

[0064] Fig.53 This is a block diagram showing an example of the configuration of an encoding device related to the second aspect.

[0065] Fig.54 This is a flowchart showing an example of decoding processing related to the second aspect.

[0066] Fig.55 This is a flowchart showing a specific example of the decoding process related to the second aspect.

[0067] Fig.56 This is a block diagram showing a configuration example of a decoding device related to the second aspect.

[0068] Fig.57 This is a flowchart showing an example of exception processing related to the second aspect.

[0069] Fig.58 This is a flowchart showing another example of the encoding process related to the second aspect.

[0070] Fig.59 This is a flowchart showing another example of the decoding process related to the second aspect.

[0071] Fig.60 This is a conceptual diagram showing an example of the position of parameters in a bitstream.

[0072] Fig.61 This is a conceptual diagram showing another example of the position of parameters in a bitstream.

[0073] Fig.62 This is a flowchart showing an example of basic encoding processing related to the embodiment.

[0074] Fig.63 This is a flowchart showing an example of basic decoding processing related to the embodiment.

[0075] Fig.64 This is a block diagram showing still another configuration example of the encoding device according to the embodiment.

[0076] Fig.65 This is a block diagram showing still another configuration example of the decoding device according to the embodiment. DETAILED DESCRIPTION

[0077] <Introduction>

[0078] For example, three-dimensional meshes are used in computer graphics. A three-dimensional mesh is composed of vertex information indicating the positions of multiple vertices in three-dimensional space, connection information indicating the connection relationship of multiple vertices, and attribute information indicating the attributes of each vertex or each face. Each face is constructed according to the connection relationship of multiple vertices. Through such a three-dimensional mesh, a variety of computer graphics can be expressed.

[0079] In addition, efficient encoding and decoding of three-dimensional meshes are desired for transmission and storage of three-dimensional meshes. For example, in order to efficiently encode vertex information, an encoding device uses position information of coded vertices to predict position information of vertices of an encoding target, and encodes the difference with respect to the prediction, thereby reducing the amount of code. More specifically, for example, the encoding device reduces the amount of code by encoding the vertices of a second triangle adjacent to the first triangle with reference to the first triangle as a reference triangle.

[0080] However, the first triangle may not be suitable as a reference triangle. Specifically, it is possible that the vertices of the first triangle are not encoded or the sides of the first triangle are not connected. Even if the vertices of the second triangle are encoded with reference to such a first triangle, it is possible that the decoding of the vertices of the second triangle becomes difficult or the amount of code is not reduced.

[0081] Therefore, the encoding device of Example 1 comprises: a memory; and a circuit capable of accessing the above-mentioned memory; when the above-mentioned circuit is in operation, when the first triangle satisfies the conditions, the mode for encoding the encoding object vertices of the second triangle is selected from the first mode set, and the above-mentioned first mode set includes a normal mode for encoding the above-mentioned encoding object vertices using the above-mentioned first triangle; when the above-mentioned first triangle does not satisfy the above-mentioned conditions, the above-mentioned mode is selected from the second mode set, and the above-mentioned second mode set includes an exceptional mode for encoding the above-mentioned encoding object vertices using a third triangle different from both the above-mentioned first triangle and the above-mentioned second triangle; the above-mentioned encoding object vertices are encoded according to the above-mentioned mode.

[0082] Thus, when the first triangle does not satisfy the condition, the vertices of the second triangle may be encoded using a third triangle different from the first triangle. Therefore, the vertices of the second triangle may be encoded using a reference triangle suitable for encoding.

[0083] In addition, the encoding device of Example 2 may also be the encoding device of Example 1, and the third triangle is a triangle that shares sides with the first triangle.

[0084] Thus, a third triangle near the first triangle used in the normal mode may be used in the exception mode. Therefore, the difference between the normal mode and the exception mode may be reduced. Therefore, the variation in the code amount may be suppressed.

[0085] In addition, the encoding device of Example 3 can also be the encoding device of Example 1 or Example 2, and the above-mentioned third triangle is a triangle that shares sides with the above-mentioned second triangle.

[0086] Thus, the third triangle near the second triangle may be used in the exception mode. Therefore, the encoding target vertex may be efficiently encoded using the third triangle near the encoding target vertex.

[0087] Furthermore, the encoding device of Example 4 may be the encoding device of any one of Examples 1 to 3, and the third triangle may share vertices with the first triangle.

[0088] Thus, a third triangle near the first triangle used in the normal mode may be used in the exception mode. Therefore, the difference between the normal mode and the exception mode may be reduced. Therefore, the variation in the code amount may be suppressed.

[0089] Furthermore, the encoding device of Example 5 may be the encoding device of any one of Examples 1 to 4, and the third triangle is a triangle whose vertices have been encoded.

[0090] As a result, the encoding target vertex may be encoded using the third triangle reflecting the characteristics of the three-dimensional mesh. Therefore, the encoding target vertex may be efficiently encoded.

[0091] Furthermore, the encoding device of Example 6 may be the encoding device of any one of Examples 1 to 5, and the third triangle is a triangle whose sides are connected.

[0092] As a result, the encoding target vertex may be encoded using the third triangle reflecting the characteristics of the three-dimensional mesh. Therefore, the encoding target vertex may be efficiently encoded.

[0093] Furthermore, the encoding device of Example 7 may be the encoding device of any one of Examples 1 to 4, and the third triangle is a triangle having pseudo vertices.

[0094] This allows the encoding target vertex to be encoded using the third triangle having any vertex. Therefore, the encoding target vertex can be encoded flexibly.

[0095] In addition, the encoding device of Example 8 can also be the encoding device of any one of Examples 1 to 7, and the above-mentioned exception mode includes multiple sub-modes; the above-mentioned circuit further selects a sub-mode for encoding the above-mentioned encoding object vertex from the above-mentioned multiple sub-modes when the above-mentioned exception mode is selected as the above-mentioned mode.

[0096] Thus, the coding target vertex may be coded according to the sub-mode selected from the plurality of sub-modes in the exception mode. Therefore, the coding target vertex may be coded according to the exception processing method adaptively selected from the plurality of exception processing methods.

[0097] Furthermore, the encoding device of Example 9 may also be the encoding device of Example 8, and the above-mentioned circuit encodes the parameters representing the above-mentioned sub-mode.

[0098] Thus, the information of the sub-mode for encoding the encoding target vertex may be shared between the encoding device and the decoding device according to the parameters. Therefore, the same sub-mode may be applied to encoding and decoding according to the parameters.

[0099] In addition, the encoding device of Example 10 can also be the encoding device of Example 8 or Example 9, and the above-mentioned circuit determines the above-mentioned third triangle according to the above-mentioned sub-mode.

[0100] Thus, by selecting a sub-mode from a plurality of sub-modes, a third triangle suitable for encoding may be selected. Therefore, the encoding target vertex may be efficiently encoded using the third triangle suitable for encoding.

[0101] Furthermore, the encoding device of Example 11 may be the encoding device of any one of Examples 8 to 10, and the circuit determines the vertices of the third triangle according to the sub-pattern.

[0102] Thus, by selecting a sub-mode from a plurality of sub-modes, a third triangle suitable for encoding may be selected. Therefore, the encoding target vertex may be efficiently encoded using the third triangle suitable for encoding.

[0103] Furthermore, the encoding device of Example 12 may be the encoding device of any one of Examples 1 to 11, and the above-mentioned condition includes that each of the vertices of the above-mentioned first triangle has been encoded.

[0104] This can prevent a situation where a vertex other than the already encoded first triangle is used in encoding the vertices of the second triangle, and can also prevent a situation where a mode suitable for encoding the vertices of the second triangle is used.

[0105] In addition, the encoding device of Example 13 may be the encoding device of any one of Examples 1 to 12, and the above-mentioned condition includes that the sides of the above-mentioned first triangle have connectivity.

[0106] This can prevent the first triangle whose side does not have connectivity from being used in encoding the vertices of the second triangle, and can also prevent the first triangle from being used in encoding the vertices of the second triangle.

[0107] In addition, the encoding device of Example 14 can also be any one of the encoding devices of Examples 1 to 13, and the above-mentioned circuit uses the above-mentioned third triangle to encode the angle representing the above-mentioned encoding object vertex when encoding the above-mentioned encoding object vertex according to the above-mentioned exception mode.

[0108] This can prevent the amount of code from becoming excessively large. In other words, it can prevent the worst-case amount of code.

[0109] In addition, the decoding device of Example 15 comprises: a memory; and a circuit capable of accessing the above-mentioned memory; when the above-mentioned circuit is in operation, when the first triangle satisfies the conditions, the mode for decoding the decoding object vertex of the second triangle is selected from the first mode set, and the above-mentioned first mode set includes a normal mode for decoding the above-mentioned decoding object vertex using the above-mentioned first triangle; when the above-mentioned first triangle does not satisfy the above-mentioned conditions, the above-mentioned mode is selected from the second mode set, and the above-mentioned second mode set includes an exceptional mode for decoding the above-mentioned decoding object vertex using a third triangle different from both the above-mentioned first triangle and the above-mentioned second triangle; and the above-mentioned decoding object vertex is decoded according to the above-mentioned mode.

[0110] Thus, when the first triangle does not satisfy the condition, the vertices of the second triangle may be decoded using a third triangle different from the first triangle. Therefore, the vertices of the second triangle may be decoded using a reference triangle suitable for decoding.

[0111] In addition, the decoding device of Example 16 may also be the decoding device of Example 15, and the third triangle is a triangle that shares a side with the first triangle.

[0112] Thus, a third triangle near the first triangle used in the normal mode may be used in the exception mode. Therefore, the difference between the normal mode and the exception mode may be reduced. Therefore, the variation in the code amount may be suppressed.

[0113] In addition, the decoding device of Example 17 may also be the decoding device of Example 15 or Example 16, and the third triangle is a triangle that shares a side with the second triangle.

[0114] As a result, the third triangle near the second triangle may be used in the exception mode. Therefore, the decoding target vertex may be efficiently decoded using the third triangle near the decoding target vertex.

[0115] Furthermore, the decoding device of Example 18 may be the decoding device of any one of Examples 15 to 17, and the third triangle is a triangle that shares vertices with the first triangle.

[0116] Thus, a third triangle near the first triangle used in the normal mode may be used in the exception mode. Therefore, the difference between the normal mode and the exception mode may be reduced. Therefore, the variation in the code amount may be suppressed.

[0117] Furthermore, the decoding device of Example 19 may be the decoding device of any one of Examples 15 to 18, and the third triangle is a triangle whose vertices have been decoded.

[0118] As a result, the decoding target vertex may be decoded using the third triangle reflecting the characteristics of the three-dimensional mesh. Therefore, the decoding target vertex may be efficiently decoded.

[0119] Furthermore, the decoding device of Example 20 may be the decoding device of any one of Examples 15 to 19, and the third triangle is a triangle whose sides have connectivity.

[0120] As a result, the decoding target vertex may be decoded using the third triangle reflecting the characteristics of the three-dimensional mesh. Therefore, the decoding target vertex may be efficiently decoded.

[0121] Furthermore, the decoding device of Example 21 may be the decoding device of any one of Examples 15 to 17, and the third triangle is a triangle having pseudo vertices.

[0122] This makes it possible to decode the decoding target vertex using the third triangle having any vertex. Therefore, it is possible to flexibly decode the decoding target vertex.

[0123] In addition, the decoding device of Example 22 can also be the decoding device of any one of Examples 15 to 21, and the above-mentioned exception mode includes multiple sub-modes; the above-mentioned circuit further selects a sub-mode for decoding the above-mentioned decoding object vertex from the above-mentioned multiple sub-modes when the above-mentioned exception mode is selected as the above-mentioned mode.

[0124] Thus, the decoding target vertex may be decoded according to the sub-mode selected from the plurality of sub-modes in the exception mode. Therefore, the decoding target vertex may be decoded according to the exception processing method adaptively selected from the plurality of exception processing methods.

[0125] Furthermore, the decoding device of Example 23 may be the decoding device of Example 22, and the above-mentioned circuit decodes the parameter representing the above-mentioned sub-mode.

[0126] Thus, the information of the sub-mode for decoding the decoding target vertex may be shared between the encoding device and the decoding device according to the parameter. Therefore, the same sub-mode may be applied to encoding and decoding according to the parameter.

[0127] In addition, the decoding device of Example 24 can also be the decoding device of Example 22 or Example 23, and the above-mentioned circuit determines the above-mentioned third triangle according to the above-mentioned sub-mode.

[0128] Thus, by selecting a sub-mode from a plurality of sub-modes, a third triangle suitable for decoding may be selected. Therefore, the decoding target vertex may be efficiently decoded using the third triangle suitable for decoding.

[0129] Furthermore, the decoding device of Example 25 may be the decoding device of any one of Examples 22 to 24, and the circuit determines the vertices of the third triangle according to the sub-pattern.

[0130] Thus, by selecting a sub-pattern from a plurality of sub-patterns, it is possible to select vertices of the third triangle suitable for decoding. Therefore, it is possible to efficiently decode the decoding target vertices using the third triangle suitable for decoding.

[0131] Furthermore, the decoding device of Example 26 may be the decoding device of any one of Examples 15 to 25, and the above-mentioned condition includes that each of the vertices of the above-mentioned first triangle has been decoded.

[0132] This can prevent the use of any vertex other than the decoded first triangle in decoding the vertices of the second triangle, and can also prevent the use of a mode suitable for decoding the vertices of the second triangle.

[0133] In addition, the decoding device of Example 27 may be the decoding device of any one of Examples 15 to 26, and the above-mentioned condition includes that the sides of the above-mentioned first triangle have connectivity.

[0134] This can prevent the first triangle whose side has no connectivity from being used in decoding the vertices of the second triangle, and can also prevent the first triangle from being used in decoding the vertices of the second triangle.

[0135] In addition, the decoding device of Example 28 may be any one of the decoding devices of Examples 15 to 27, and the circuit may decode the angle representing the vertex of the decoding object using the third triangle when decoding the vertex of the decoding object according to the exception mode.

[0136] This can prevent the amount of code from becoming excessively large. In other words, it can prevent the worst-case amount of code.

[0137] In addition, the encoding method of Example 29 includes: when the first triangle satisfies the conditions, the step of selecting a mode for encoding the encoding object vertices of the second triangle from the first mode set, the first mode set including a normal mode for encoding the encoding object vertices using the first triangle; when the first triangle does not satisfy the conditions, the step of selecting the mode from the second mode set, the second mode set including an exceptional mode for encoding the encoding object vertices using a third triangle different from both the first triangle and the second triangle; and the step of encoding the encoding object vertices according to the mode.

[0138] Thus, when the first triangle does not satisfy the condition, the vertices of the second triangle may be encoded using a third triangle different from the first triangle. Therefore, the vertices of the second triangle may be encoded using a reference triangle suitable for encoding.

[0139] In addition, the decoding method of Example 30 includes: when the first triangle satisfies the conditions, the step of selecting a mode for decoding the decoding object vertex of the second triangle from the first mode set, the first mode set including a normal mode for decoding the decoding object vertex using the first triangle; when the first triangle does not satisfy the conditions, the step of selecting the mode from the second mode set, the second mode set including an exceptional mode for decoding the decoding object vertex using a third triangle different from both the first triangle and the second triangle; and the step of decoding the decoding object vertex according to the mode.

[0140] Thus, when the first triangle does not satisfy the condition, the vertices of the second triangle may be decoded using a third triangle different from the first triangle. Therefore, the vertices of the second triangle may be decoded using a reference triangle suitable for decoding.

[0141] Furthermore, these inclusive or specific technical solutions may also be implemented by systems, devices, methods, integrated circuits, computer programs, or non-temporary recording media such as computer-readable CD-ROMs, or by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0142] <Expression and terminology>

[0143] Here, the following expressions and terms are used.

[0144] (1) 3D grid

[0145] A 3D mesh is a collection of multiple faces, for example, representing a 3D object. In addition, a 3D mesh is mainly composed of vertex information, connection information, and attribute information. A 3D mesh is sometimes represented as a polygonal mesh or a mesh. In addition, a 3D mesh may also have temporal changes. A 3D mesh may contain metadata related to vertex information, connection information, and attribute information, and may also contain other additional information.

[0146] (2) Vertex information

[0147] Vertex information is information that represents a vertex. For example, vertex information represents the position of a vertex in a three-dimensional space. In addition, a vertex corresponds to a vertex of a face that constitutes a three-dimensional mesh. Vertex information is sometimes expressed as "Geometory". In addition, vertex information is sometimes expressed as position information.

[0148] (3) Connection information

[0149] The connection information is information indicating the connection between vertices. For example, the connection information indicates the connection of faces or edges used to form a three-dimensional mesh. The connection information is sometimes expressed as "Connectivity". In addition, the connection information is sometimes expressed as face information.

[0150] (4) Attribute information

[0151] Attribute information is information indicating the attributes of a vertex or a face. For example, attribute information indicates attributes such as color, image, and normal vector associated with a vertex or a face. Attribute information is sometimes expressed as "Texture".

[0152] (5) Surface

[0153] A face is an element that makes up a 3D mesh. Specifically, a face is a polygon on a plane in a 3D space. For example, a face can be set to a triangle in a 3D space.

[0154] (6) Plane

[0155] A plane is a two-dimensional surface in a three-dimensional space. For example, a polygon is formed on a plane, and multiple polygons are formed on multiple planes.

[0156] (7) Bitstream

[0157] A bitstream corresponds to the encoded information. A bitstream can also be expressed as a stream, a coded bitstream, a compressed bitstream, or a coded signal.

[0158] (8) Encoding and decoding

[0159] The expression "encode" may be replaced by expressions such as "store", "include", "write", "record", "signal", "send", "notify", "store" or "compress", and these expressions may also be replaced with each other. For example, encoding information may also mean including the information in a bitstream. In addition, encoding information into a bitstream may also mean encoding the information and generating a bitstream containing the encoded information.

[0160] In addition, the expression decoding can be replaced by expressions such as reading out, interpreting, reading, reading in, deriving, obtaining, obtaining, extracting, restoring, reconstructing, decompressing or expanding, and these expressions can also be replaced with each other. For example, decoding information can also be obtaining information from a bit stream. In addition, decoding information from a bit stream can also mean decoding a bit stream and obtaining information contained in the bit stream.

[0161] (9) Ordinal

[0162] In the description, ordinals such as the first and second are sometimes assigned to constituent elements. These ordinals may be assigned in a different order as appropriate. In addition, ordinals may be newly assigned to constituent elements or removed. In addition, these ordinals are sometimes assigned to elements for identification purposes and sometimes do not correspond to a meaningful order.

[0163] <3D Grid>

[0164] Figure 1 3D mesh is a conceptual diagram showing a three-dimensional mesh related to the present embodiment. The three-dimensional mesh is composed of a plurality of faces. For example, each face is a triangle. The vertices of these triangles are set in a three-dimensional space. Furthermore, the three-dimensional mesh represents a three-dimensional object. Each face may also have a color or an image.

[0165] Figure 2 3D mesh is a conceptual diagram showing basic elements of the present embodiment. The 3D mesh is composed of vertex information, connection information and attribute information. The vertex information indicates the position of the vertices of the face in the 3D space. The connection information indicates the connection between the vertices. The face can be determined by the vertex information and the connection information. That is, a 3D object without color is formed in the 3D space by the vertex information and the connection information.

[0166] Attribute information can be associated with either a vertex or a face. Attribute information associated with a vertex is sometimes expressed as "Attribute Per Point". Attribute information associated with a vertex can represent the attributes of the vertex itself or the attributes of the face connected to the vertex.

[0167] For example, a color may be associated with a vertex as attribute information. The color associated with a vertex may be the color of the vertex or the color of a face connected to the vertex. The color of a face may be the average of a plurality of colors associated with a plurality of vertices of the face. In addition, a normal vector may be associated with a vertex or a face as attribute information. Such a normal vector may represent the front side and the back side of a face.

[0168] In addition, a two-dimensional image may be associated with a surface as attribute information. A two-dimensional image associated with a surface is also expressed as a texture image or "Attribute Map". In addition, information indicating a mapping between a surface and a two-dimensional image may be associated with a surface as attribute information. Information indicating such a mapping is sometimes expressed as mapping information, vertex information of a texture image, or "Attribute UV Coordinate".

[0169] Furthermore, information such as color, image, and moving image used as attribute information may be expressed as "ParametricSpace".

[0170] By using such attribute information, texture can be reflected on a three-dimensional object. That is, a three-dimensional object with color is formed in a three-dimensional space by using vertex information, connection information, and attribute information.

[0171] In the above description, the attribute information is associated with the vertices or faces, but may be associated with the edges.

[0172] Figure 3 This is a conceptual diagram showing the mapping related to this embodiment. For example, a region of a two-dimensional image in a two-dimensional plane can be mapped to a surface of a three-dimensional grid in a three-dimensional space. Specifically, the coordinate information of the region in the two-dimensional image is associated with the surface of the three-dimensional grid. Thus, the image of the region mapped in the two-dimensional image is reflected on the surface of the three-dimensional grid.

[0173] By using the mapping, the two-dimensional image used as attribute information can be separated from the three-dimensional mesh. For example, in the encoding of the three-dimensional mesh, the two-dimensional image can also be encoded by the image encoding method or the video encoding method.

[0174] <System Configuration>

[0175] Figure 4 : is a block diagram showing an example of the configuration of a coding and decoding system according to the present embodiment. Figure 4 In the embodiment, the encoding and decoding system includes an encoding device 100 and a decoding device 200.

[0176] For example, the encoding device 100 obtains a three-dimensional mesh and encodes the three-dimensional mesh into a bit stream. Furthermore, the encoding device 100 outputs the bit stream to the network 300. For example, the bit stream includes the encoded three-dimensional mesh and control information for decoding the encoded three-dimensional mesh. By encoding the three-dimensional mesh, the information of the three-dimensional mesh is compressed.

[0177] The network 300 transmits the bit stream from the encoding device 100 to the decoding device 200. The network 300 may be the Internet, a wide area network (WAN), a small-scale network (LAN), or a combination thereof. The network 300 is not necessarily limited to two-way communication, and may be a one-way communication network used for terrestrial digital broadcasting or satellite broadcasting.

[0178] In addition, the network 300 may be replaced with a recording medium such as a DVD (Digital Versatile Disc) or a BD (Blu-Ray Disc (registered trademark)).

[0179] The decoding device 200 obtains a bit stream and decodes the three-dimensional mesh from the bit stream. By decoding the three-dimensional mesh, the information of the three-dimensional mesh is expanded. For example, the decoding device 200 decodes the three-dimensional mesh according to a decoding method corresponding to the encoding method used by the encoding device 100 to encode the three-dimensional mesh. That is, the encoding device 100 and the decoding device 200 perform encoding and decoding according to the encoding method and decoding method corresponding to each other.

[0180] In addition, the 3D mesh before encoding can also be expressed as the original 3D mesh. In addition, the 3D mesh after decoding can also be expressed as the reconstructed 3D mesh.

[0181] <Encoding device>

[0182] Figure 5 1 is a block diagram showing a configuration example of the encoding device 100 according to the present embodiment. For example, the encoding device 100 includes a vertex information encoder 101 , a connection information encoder 102 , and an attribute information encoder 103 .

[0183] The vertex information encoder 101 is an electrical circuit that encodes vertex information. For example, the vertex information encoder 101 encodes the vertex information into a bit stream in a format specified for the vertex information.

[0184] The connection information encoder 102 is an electrical circuit that encodes the connection information. For example, the connection information encoder 102 encodes the connection information into a bit stream in a format specified for the connection information.

[0185] The attribute information encoder 103 is an electrical circuit that encodes the attribute information. For example, the attribute information encoder 103 encodes the attribute information into a bit stream in a format specified for the attribute information.

[0186] In encoding vertex information, connection information, and attribute information, either variable length coding or fixed length coding may be used. Variable length coding may also correspond to Huffman coding or context adaptive binary arithmetic coding (CABAC).

[0187] The vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be integrated. Alternatively, the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be further subdivided into a plurality of components.

[0188] Figure 6 is a block diagram showing another configuration example of the encoding device 100 according to the present embodiment. Figure 5 In addition to the illustrated configuration, a pre-processor 104 and a post-processor 105 are further provided.

[0189] The pre-processor 104 is an electrical circuit that processes vertex information, connection information, and attribute information before encoding. For example, the pre-processor 104 may also perform transformation processing, separation processing, or multiplexing processing on the three-dimensional mesh before encoding. More specifically, for example, the pre-processor 104 may also separate vertex information, connection information, and attribute information from the three-dimensional mesh before encoding.

[0190] The post-processor 105 is an electrical circuit that processes the vertex information, connection information, and attribute information after encoding. For example, the post-processor 105 may also perform transformation processing, separation processing, or multiplexing processing on the encoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 105 may also multiplex the encoded vertex information, connection information, and attribute information into a bit stream. In addition, for example, the post-processor 105 may further perform variable length encoding on the encoded vertex information, connection information, and attribute information.

[0191] <Decoding device>

[0192] Figure 7 2 is a block diagram showing a configuration example of a decoding device 200 according to the present embodiment. For example, the decoding device 200 includes a vertex information decoder 201 , a connection information decoder 202 , and an attribute information decoder 203 .

[0193] The vertex information decoder 201 is an electrical circuit that decodes vertex information. For example, the vertex information decoder 201 decodes vertex information from a bit stream in a format specified for the vertex information.

[0194] The connection information decoder 202 is an electrical circuit that decodes the connection information. For example, the connection information decoder 202 decodes the connection information from the bit stream in a format specified for the connection information.

[0195] The attribute information decoder 203 is an electrical circuit that decodes the attribute information. For example, the attribute information decoder 203 decodes the attribute information from the bit stream in a format specified for the attribute information.

[0196] In decoding vertex information, connection information, and attribute information, variable length decoding or fixed length decoding may be used. Variable length decoding may also correspond to Huffman coding or context adaptive binary arithmetic coding (CABAC).

[0197] The vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be integrated. Alternatively, the vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be further subdivided into a plurality of components.

[0198] Figure 8 is a block diagram showing another configuration example of the decoding device 200 according to the present embodiment. Figure 7 In addition to the illustrated configuration, a pre-processor 204 and a post-processor 205 are further provided.

[0199] The pre-processor 204 is an electrical circuit that processes the vertex information, connection information, and attribute information before decoding. For example, the pre-processor 204 may transform, separate, or multiplex the bit stream before decoding the vertex information, connection information, and attribute information.

[0200] More specifically, for example, the pre-processor 204 may separate the sub-bitstream corresponding to the vertex information, the sub-bitstream corresponding to the connection information, and the sub-bitstream corresponding to the attribute information from the bitstream. In addition, for example, the pre-processor 204 may also perform variable-length decoding on the bitstream before decoding the vertex information, the connection information, and the attribute information.

[0201] The post-processor 205 is an electrical circuit that processes the vertex information, connection information, and attribute information after decoding. For example, the post-processor 205 may also transform, separate, or reuse the decoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 205 may also reuse the decoded vertex information, connection information, and attribute information into a three-dimensional mesh.

[0202] <bitstream>

[0203] Vertex information, connection information, and attribute information are encoded and stored in the bitstream. The following shows the relationship between this information and the bitstream.

[0204] Fig. 9 2 is a conceptual diagram showing an example of the structure of a bitstream related to the present embodiment. In this example, vertex information, connection information, and attribute information are integrated in the bitstream. For example, vertex information, connection information, and attribute information may also be included in one file.

[0205] Furthermore, a plurality of parts of this information may be stored in sequence, such as the first part of vertex information, the first part of connection information, the first part of attribute information, the second part of vertex information, the second part of connection information, the second part of attribute information, ... These plurality of parts may correspond to a plurality of parts that are different in time, a plurality of parts that are different in space, or a plurality of different faces.

[0206] In addition, the storage order of vertex information, connection information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0207] Fig.10 : is a conceptual diagram showing another example of the bitstream structure related to the present embodiment. In this example, the bitstream includes multiple files, and the vertex information, connection information, and attribute information are stored in different files. Here, a file containing vertex information, a file containing connection information, and a file containing attribute information are shown, but the storage format is not limited to such an example. For example, two types of information among vertex information, connection information, and attribute information may be included in one file, and a different type of information may be included in another file.

[0208] Alternatively, the information may be divided and stored in more files. For example, multiple parts of vertex information may be stored in multiple files, multiple parts of connection information may be stored in multiple files, or multiple parts of attribute information may be stored in multiple files. These multiple parts may correspond to multiple parts that are different in time, multiple parts that are different in space, or multiple different faces.

[0209] In addition, the storage order of vertex information, connection information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0210] Fig.11 2 is a conceptual diagram showing another example of bitstream configuration according to the present embodiment. In this example, the bitstream is composed of a plurality of separable sub-bitstreams, and vertex information, connection information, and attribute information are stored in different sub-bitstreams.

[0211] Here, a bitstream including vertex information, a sub-bitstream including connection information, and a sub-bitstream including attribute information are shown, but the storage format is not limited to this example.

[0212] For example, two types of information among vertex information, connection information, and attribute information may be included in one sub-bitstream, and the remaining type of information may be included in another sub-bitstream. Specifically, the sub-bitstreams of attribute information of a two-dimensional image and vertex information and connection information may be separately stored in a sub-bitstream according to an image coding method.

[0213] Furthermore, each sub-bitstream may include multiple files, and multiple parts of vertex information may be stored in multiple files, multiple parts of connection information may be stored in multiple files, or multiple parts of attribute information may be stored in multiple files.

[0214] <Specific example>

[0215] Fig.12 1 is a block diagram showing a specific example of the encoding and decoding system related to the present embodiment. Fig.12 In the embodiment, the encoding and decoding system includes a three-dimensional data encoding system 110 , a three-dimensional data decoding system 210 , and an external connector 310 .

[0216] The three-dimensional data encoding system 110 includes a controller 111, an input / output processor 112, a three-dimensional data encoder 113, a three-dimensional data generator 115, and a system multiplexer 114. The three-dimensional data decoding system 210 includes a controller 211, an input / output processor 212, a three-dimensional data decoder 213, a system demultiplexer 214, a prompter 215, and a user interface 216.

[0217] In the three-dimensional data encoding system 110 , sensor data is input from a sensor terminal to a three-dimensional data generator 115 . The three-dimensional data generator 115 generates three-dimensional data such as point cloud data or mesh data based on the sensor data and inputs it to a three-dimensional data encoder 113 .

[0218] For example, the three-dimensional data generator 115 generates vertex information, and generates connection information and attribute information corresponding to the vertex information. The three-dimensional data generator 115 may also process the vertex information when generating the connection information and attribute information. For example, the three-dimensional data generator 115 may also reduce the amount of data by deleting duplicate vertices, or transform the vertex information (position shift, rotation, or standardization, etc.). In addition, the three-dimensional data generator 115 may also draw attribute information.

[0219] In addition, the three-dimensional data generator 115 Fig.12It is a component of the three-dimensional data encoding system 110, but can also be configured independently from the three-dimensional data encoding system 110 outside.

[0220] The sensor terminal that provides sensor data for generating three-dimensional data may be, for example, a mobile object such as a car, a flying object such as an airplane, a portable terminal, or a camera. In addition, a distance sensor such as a LIDAR, a millimeter wave radar, an infrared sensor or a rangefinder, a stereo camera, or a combination of multiple monocular cameras may be used as the sensor terminal.

[0221] Sensor data may also include the distance (position) of an object, a monocular camera image, a stereo camera image, color, reflectivity, sensor posture, orientation, gyroscope, sensing position (GPS information or altitude), speed, acceleration, sensing time, temperature, air pressure, humidity or magnetism, etc.

[0222] The three-dimensional data encoder 113 corresponds to Figure 5 The coding device 100 shown in the figure is shown in FIG. For example, the three-dimensional data encoder 113 encodes the three-dimensional data to generate the encoded data. In addition, the three-dimensional data encoder 113 generates control information in the encoding of the three-dimensional data. And, the three-dimensional data encoder 113 inputs the encoded data and the control information to the system multiplexer 114.

[0223] The encoding method of the three-dimensional data may be either a geometric encoding method or a video codec encoding method. Here, the geometric encoding method may also be expressed as a geometric-based encoding method. The video codec encoding method may also be expressed as a video-based encoding method.

[0224] The system multiplexer 114 multiplexes the coded data and control information input from the three-dimensional data encoder 113, and generates multiplexed data using a predetermined multiplexing method. The system multiplexer 114 may also multiplex other media such as images, sounds, subtitles, application data, or document files, or reference time information, together with the coded data and control information of the three-dimensional data. Furthermore, the system multiplexer 114 may also multiplex attribute information associated with the sensor data or the three-dimensional data.

[0225] For example, the multiplexed data has a file format for storage or a packet format for transmission. As these formats, ISOBMFF or a format based on ISOBMFF may be used. In addition, MPEG-DASH, MMT, MPEG-2TSSystems, or RTP may also be used.

[0226] Furthermore, the input / output processor 112 outputs the multiplexed data as a transmission signal to the external connector 310. The multiplexed data as a transmission signal can be transmitted by wire or by wireless. Alternatively, the multiplexed data is stored in an internal memory or storage device. The multiplexed data can also be transmitted to a cloud server via the Internet or stored in an external storage device.

[0227] For example, the transmission or storage of the multiplexed data is performed by a method corresponding to the medium used for transmission or storage, such as broadcasting or communication. As a communication protocol, http, ftp, TCP, UDP, IP or a combination thereof may also be used. In addition, a PULL type communication method may also be used, and a PUSH type communication method may also be used.

[0228] For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), or coaxial cable can be used. In addition, for wireless transmission, 3G / 4G / 5G specified by 3GPP (registered trademark) and IEEE, wireless LAN, Wi-Fi, Bluetooth, or millimeter waves can be used. In addition, as a broadcasting method, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 can be used.

[0229] In addition, the sensor data may be input to the three-dimensional data generator 115 or the system multiplexer 114. In addition, the three-dimensional data or the encoded data may be output as a transmission signal to the external connector 310 via the input / output processor 112. The transmission signal output from the three-dimensional data encoding system 110 is input to the three-dimensional data decoding system 210 via the external connector 310.

[0230] In addition, each action of the three-dimensional data encoding system 110 may also be controlled by the controller 111 executing the application program.

[0231] In the three-dimensional data decoding system 210, the transmission signal is input to the input / output processor 212. The input / output processor 212 decodes the multiplexed data in the file format or the packet format from the transmission signal, and inputs the multiplexed data to the system demultiplexer 214. The system demultiplexer 214 obtains the coded data and control information from the multiplexed data, and inputs them to the three-dimensional data decoder 213. The system demultiplexer 214 may also obtain other media or reference time information from the multiplexed data.

[0232] The three-dimensional data decoder 213 corresponds to Figure 7 The decoding device 200 shown in the figure is as follows. For example, the three-dimensional data decoder 213 decodes the three-dimensional data from the coded data based on a predetermined coding method. Then, the presenter 215 presents the three-dimensional data to the user.

[0233] In addition, additional information such as sensor data may be input to the prompter 215. The prompter 215 may also prompt the three-dimensional data based on the additional information. In addition, a user's instruction may be input from the user terminal to the user interface 216. Furthermore, the prompter 215 may also prompt the three-dimensional data based on the input instruction.

[0234] In addition, the input / output processor 212 may also obtain three-dimensional data and encoded data from the external connector 310 .

[0235] In addition, each action of the three-dimensional data decoding system 210 may also be controlled by the controller 211 executing the application program.

[0236] Fig.13 1 is a conceptual diagram showing an example of the configuration of point group data according to the present embodiment. The point group data is data representing a point group of a three-dimensional object.

[0237] Specifically, a point cloud is composed of a plurality of points, and has position information indicating the three-dimensional coordinate position of each point and attribute information indicating the attribute of each point. Position information is also expressed as geometry information.

[0238] The category of attribute information may be, for example, color or reflectance, etc. Attribute information of one category may be associated with one point, attribute information of multiple different categories may be associated with one point, or attribute information of the same category having multiple values ​​may be associated with one point.

[0239] Fig.14 : is a conceptual diagram showing an example of a data file of point group data related to the present embodiment. In this example, it is an example of a case where the items of position information and the items of attribute information correspond to each other one by one, and the position information and attribute information of N points constituting the point group data are shown. In this example, the position information is information representing the three-dimensional coordinate position using the three axes of x, y and z, and the attribute information is information representing the color using RGB. As a representative data file of point group data, a PLY file or the like can be used.

[0240] Fig.15 This is a conceptual diagram showing an example of the composition of mesh data related to the present embodiment. Mesh data is data used in CG (Computer Graphics) and the like, and is data of a three-dimensional mesh that represents the three-dimensional shape of an object using multiple faces. Each face is also represented as a polygon, and has a polygonal shape such as a triangle or a quadrilateral.

[0241] Specifically, a 3D mesh consists of multiple edges and faces in addition to the multiple points that make up the point group. Each point is also represented by a vertex or position. Each edge corresponds to a line segment connected by two vertices. Each face corresponds to an area surrounded by more than three edges.

[0242] In addition, the three-dimensional mesh has position information indicating the three-dimensional coordinate position of the vertex. The position information is also expressed as vertex information or geometry information. In addition, the three-dimensional mesh has connection information, which indicates the relationship between multiple vertices constituting an edge or a face. The connection information is also expressed as connectivity. In addition, the three-dimensional mesh has attribute information indicating the attributes of the vertex, edge or face. The attribute information of the three-dimensional mesh is also expressed as texture.

[0243] For example, the attribute information may also indicate the color, reflectivity, or normal vector of a vertex, edge, or face. The direction of the normal vector may indicate the front side or the back side of a face.

[0244] As a data file format of the mesh data, an object file or the like can be used.

[0245] Fig.16 : is a conceptual diagram showing an example of a data file of mesh data related to the present embodiment. In this example, the data file includes position information G(1) to G(N) of N vertices constituting a three-dimensional mesh and attribute information A1(1) to A1(N) of the N vertices. In addition, in this example, M attribute information A2(1) to A2(M) are included. The items of attribute information may not correspond one-to-one with the vertices or one-to-one with the faces. In addition, attribute information may not exist.

[0246] The connection information is represented by a combination of vertex indices. n[1, 3, 4] represents a triangular face consisting of three vertices of n=1, n=3, and n=4. In addition, m[2, 4, 6] represents attribute information corresponding to three vertices of m=2, m=4, and m=6, respectively.

[0247] In addition, the substantial content of the attribute information may be recorded in other files. Furthermore, a pointer to the content may be associated with a vertex or a face. For example, the attribute information representing the image of the face may be stored in a two-dimensional attribute map file. Furthermore, the file name of the attribute map and the two-dimensional coordinate values ​​in the attribute map may be recorded in the attribute information A2(1) to A2(M). The method of specifying attribute information for a face is not limited to these methods, and any method may be used.

[0248] Fig.17: is a conceptual diagram showing the types of three-dimensional data related to this embodiment. Point group data and mesh data can represent both static objects and dynamic objects. Static objects are objects that do not change over time, and dynamic objects are objects that change over time. Static objects can also correspond to three-dimensional data for any point in time.

[0249] For example, point cloud data at an arbitrary time point may be represented as a PCC frame. Also, mesh data at an arbitrary time point may be represented as a mesh frame. Furthermore, PCC frames and mesh frames may be represented as just frames.

[0250] In addition, the target area can be limited to a certain range like conventional image data, or can be unlimited like map data. In addition, the density of points or surfaces can be set in various ways. Sparse point group data or sparse grid data can be used, or dense point group data or dense grid data can be used.

[0251] Next, the encoding and decoding of point groups or three-dimensional meshes are described. The apparatus, process or syntax for encoding and decoding vertex information of three-dimensional meshes disclosed in the present invention can also be applied to encoding and decoding of point groups. The apparatus, process or syntax for encoding and decoding of point groups disclosed in the present invention can also be applied to encoding and decoding of vertex information of three-dimensional meshes.

[0252] In addition, the device, process or syntax for encoding and decoding the attribute information of a point group disclosed in the present invention can also be applied to the encoding and decoding of the connection information or attribute information of a three-dimensional mesh. In addition, the device, process or syntax for encoding and decoding the connection information or attribute information of a three-dimensional mesh disclosed in the present invention can also be applied to the encoding and decoding of the attribute information of a point group.

[0253] Furthermore, at least a portion of the processing may be shared between the encoding and decoding of the point cloud data and the encoding and decoding of the mesh data, thereby reducing the scale of the circuit and the software program.

[0254] Fig.18 1 is a block diagram showing an example of the configuration of the three-dimensional data encoder 113 according to the present embodiment. In this example, the three-dimensional data encoder 113 includes a vertex information encoder 121, an attribute information encoder 122, a metadata encoder 123, and a multiplexer 124. The vertex information encoder 121, the attribute information encoder 122, and the multiplexer 124 may also correspond to Figure 6 The vertex information encoder 101, attribute information encoder 103 and post-processor 105, etc.

[0255] In this example, the three-dimensional data encoder 113 encodes the three-dimensional data according to a geometry-based encoding method. In the encoding according to the geometry-based encoding method, the three-dimensional structure is considered. In the encoding according to the geometry-based encoding method, the attribute information is encoded using the composition information obtained in the encoding of the vertex information.

[0256] Specifically, first, vertex information, attribute information, and metadata contained in the three-dimensional data generated from the sensor data are input to the vertex information encoder 121, the attribute information encoder 122, and the metadata encoder 123, respectively. Here, the connection information contained in the three-dimensional data can also be treated in the same way as the attribute information. In addition, in the case of point group data, the position information can also be treated as vertex information.

[0257] Vertex information encoder 121 encodes vertex information into compressed vertex information and outputs the compressed vertex information as encoded data to multiplexer 124. Vertex information encoder 121 also generates metadata of compressed vertex information and outputs it to multiplexer 124. Vertex information encoder 121 also generates composition information and outputs it to attribute information encoder 122.

[0258] The attribute information encoder 122 encodes the attribute information into compressed attribute information using the composition information generated by the vertex information encoder 121 , and outputs the compressed attribute information as encoded data to the multiplexer 124 . The attribute information encoder 122 also generates metadata of the compressed attribute information and outputs it to the multiplexer 124 .

[0259] The metadata encoder 123 encodes compressible metadata into compressed metadata, and outputs the compressed metadata as encoded data to the multiplexer 124. The metadata encoded by the metadata encoder 123 may be used for encoding vertex information and encoding attribute information.

[0260] The multiplexer 124 multiplexes the compressed vertex information, the metadata of the compressed vertex information, the compressed attribute information, the metadata of the compressed attribute information, and the compressed metadata into the bitstream. Furthermore, the multiplexer 124 inputs the bitstream to the system layer.

[0261] Fig.19 2 is a block diagram showing an example of the configuration of the three-dimensional data decoder 213 according to the present embodiment. In this example, the three-dimensional data decoder 213 includes a vertex information decoder 221, an attribute information decoder 222, a metadata decoder 223, and a demultiplexer 224. The vertex information decoder 221, the attribute information decoder 222, and the demultiplexer 224 may also correspond to Figure 8 The vertex information decoder 201, attribute information decoder 203 and pre-processor 204, etc.

[0262] In this example, the three-dimensional data decoder 213 decodes the three-dimensional data according to the geometry-based coding method. The three-dimensional structure is considered in the decoding according to the geometry-based coding method. In the decoding according to the geometry-based coding method, the attribute information is decoded using the composition information obtained in the decoding of the vertex information.

[0263] Specifically, first, the bitstream is input from the system layer to the demultiplexer 224. The demultiplexer 224 separates the compressed vertex information, the metadata of the compressed vertex information, the compressed attribute information, the metadata of the compressed attribute information, and the compressed metadata from the bitstream. The compressed vertex information and the metadata of the compressed vertex information are input to the vertex information decoder 221. The compressed attribute information and the metadata of the compressed attribute information are input to the attribute information decoder 222. The metadata is input to the metadata decoder 223.

[0264] The vertex information decoder 221 decodes the vertex information from the compressed vertex information using metadata of the compressed vertex information. In addition, the vertex information decoder 221 generates composition information and outputs it to the attribute information decoder 222. The attribute information decoder 222 decodes attribute information from the compressed attribute information using the composition information generated by the vertex information decoder 221 and metadata of the compressed attribute information. The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 can also be used for decoding the vertex information and decoding the attribute information.

[0265] Then, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. In addition, for example, the metadata is metadata of the vertex information and attribute information, and can be used in an application.

[0266] Fig. 20 1 is a block diagram showing another configuration example of the three-dimensional data encoder 113 according to the present embodiment. In this example, the three-dimensional data encoder 113 includes a vertex image generator 131, an attribute image generator 132, a metadata generator 133, a video encoder 134, a metadata encoder 123, and a multiplexer 124. The vertex image generator 131, the attribute image generator 132, and the video encoder 134 may also correspond to Figure 6 Vertex information encoder 101 and attribute information encoder 103, etc.

[0267] In addition, in this example, the three-dimensional data encoder 113 encodes the three-dimensional data according to a video-based encoding method. In the encoding according to the video-based encoding method, a plurality of two-dimensional images are generated from the three-dimensional data, and the plurality of two-dimensional images are encoded according to an image encoding method. Here, the image encoding method may also be HEVC (High Efficiency Video Coding) or VVC (Versatile Video Coding) or the like.

[0268] Specifically, first, vertex information and attribute information included in the three-dimensional data generated from the sensor data are input to the metadata generator 133. In addition, the vertex information and attribute information are respectively input to the vertex image generator 131 and the attribute image generator 132. In addition, the metadata included in the three-dimensional data is input to the metadata encoder 123. Here, the connection information included in the three-dimensional data can also be treated in the same way as the attribute information. In addition, in the case of point group data, the position information can also be treated as vertex information.

[0269] The metadata generator 133 generates mapping information of a plurality of two-dimensional images from the vertex information and the attribute information, and inputs the mapping information to the vertex image generator 131 , the attribute image generator 132 , and the metadata encoder 123 .

[0270] The vertex image generator 131 generates a vertex image based on the vertex information and the mapping information, and inputs the image to the video encoder 134. The attribute image generator 132 generates an attribute image based on the attribute information and the mapping information, and inputs the image to the video encoder 134.

[0271] The image encoder 134 encodes the vertex image and the attribute image into compressed vertex information and compressed attribute information respectively according to the image encoding method, and outputs the compressed vertex information and compressed attribute information as encoded data to the multiplexer 124. In addition, the image encoder 134 generates metadata of the compressed vertex information and metadata of the compressed attribute information, and outputs them to the multiplexer 124.

[0272] The metadata encoder 123 encodes the compressible metadata into compressed metadata, and outputs the compressed metadata as encoded data to the multiplexer 124. The compressible metadata includes mapping information. In addition, the metadata encoded by the metadata encoder 123 can also be used for encoding vertex information and encoding attribute information.

[0273] The multiplexer 124 multiplexes the compressed vertex information, the metadata of the compressed vertex information, the compressed attribute information, the metadata of the compressed attribute information, and the compressed metadata into the bitstream. Furthermore, the multiplexer 124 inputs the bitstream to the system layer.

[0274] Fig.212 is a block diagram showing another configuration example of the three-dimensional data decoder 213 according to the present embodiment. In this example, the three-dimensional data decoder 213 includes a vertex information generator 231, an attribute information generator 232, a video decoder 234, a metadata decoder 223, and a demultiplexer 224. The vertex information generator 231, the attribute information generator 232, and the video decoder 234 may also correspond to Figure 8 Vertex information decoder 201 and attribute information decoder 203, etc.

[0275] In addition, in this example, the 3D data decoder 213 decodes the 3D data according to a video-based coding method. In the decoding according to the video-based coding method, a plurality of 2D images are decoded according to an image coding method, and 3D data is generated from the plurality of 2D images. Here, the image coding method may also be HEVC (High Efficiency Video Coding) or VVC (Versatile Video Coding), etc.

[0276] Specifically, first, the bitstream is input from the system layer to the demultiplexer 224. The demultiplexer 224 separates the compressed vertex information, the metadata of the compressed vertex information, the compressed attribute information, the metadata of the compressed attribute information, and the compressed metadata from the bitstream. The compressed vertex information and the metadata of the compressed vertex information, the compressed attribute information, and the metadata of the compressed attribute information are input to the image decoder 234. The compressed metadata is input to the metadata decoder 223.

[0277] The image decoder 234 decodes the vertex image according to the image coding method. At this time, the image decoder 234 uses the metadata of the compressed vertex information to decode the vertex image from the compressed vertex information. And the image decoder 234 inputs the vertex image to the vertex information generator 231. In addition, the image decoder 234 decodes the attribute image according to the image coding method. At this time, the image decoder 234 uses the metadata of the compressed attribute information to decode the attribute image from the compressed attribute information. And the image decoder 234 inputs the attribute image to the attribute information generator 232.

[0278] The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 includes mapping information used in generating vertex information and generating attribute information. In addition, the metadata decoded by the metadata decoder 223 can also be used for decoding vertex images and decoding attribute images.

[0279] The vertex information generator 231 reproduces vertex information from the vertex image according to the mapping information included in the metadata decoded by the metadata decoder 223. The attribute information generator 232 reproduces attribute information from the attribute image according to the mapping information included in the metadata decoded by the metadata decoder 223.

[0280] Then, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. In addition, for example, the metadata is metadata of the vertex information and attribute information, and can be used in an application.

[0281] Fig. 22 : is a conceptual diagram showing a specific example of the encoding process related to this embodiment. Fig. 22 3D data encoder 113 and description encoder 148 are shown in FIG. 3D data encoder 113 includes 2D data encoder 141 and mesh data encoder 142. 2D data encoder 141 includes texture encoder 143. Mesh data encoder 142 includes vertex information encoder 144 and connection information encoder 145.

[0282] The vertex information encoder 144, the connection information encoder 145 and the texture encoder 143 may also correspond to Figure 6 The vertex information encoder 101, the connection information encoder 102 and the attribute information encoder 103 etc.

[0283] For example, the two-dimensional data encoder 141 operates as the texture encoder 143 and generates a texture file by encoding the texture corresponding to the attribute information as two-dimensional data according to the image encoding method or the video encoding method.

[0284] In addition, the mesh data encoder 142 operates as a vertex information encoder 144 and a connection information encoder 145, and generates a mesh file by encoding the vertex information and the connection information. The mesh data encoder 142 may also encode the mapping information for the texture. Furthermore, the encoded mapping information may be included in the mesh file.

[0285] In addition, the description encoder 148 may also generate a description file by encoding the description corresponding to the metadata such as text data. The description encoder 148 may also encode the description at the system level. For example, the description encoder 148 may also include Fig.12 in the system multiplexer 114.

[0286] Through the above operations, a bitstream containing texture files, mesh files and description files is generated. These files can also be reused in the bitstream in the form of glTF (Graphics Language Transmission Format) or USD (Universal Scene Description) files.

[0287] Furthermore, the three-dimensional data encoder 113 may include two mesh data encoders as the mesh data encoder 142. For example, one mesh data encoder encodes vertex information and connection information of a static three-dimensional mesh, and the other mesh data encoder encodes vertex information and connection information of a dynamic three-dimensional mesh.

[0288] Furthermore, correspondingly, two mesh files may be included in the bitstream. For example, one mesh file corresponds to a static three-dimensional mesh, and the other mesh file corresponds to a dynamic three-dimensional mesh.

[0289] In addition, the static 3D mesh may be a 3D mesh within a frame using intra-frame prediction coding, and the dynamic 3D mesh may be a 3D mesh between frames using inter-frame prediction coding. In addition, as the information of the dynamic 3D mesh, differential information between vertex information or connection information of a 3D mesh within a frame and vertex information or connection information of a 3D mesh between frames may be used.

[0290] Fig.23 : is a conceptual diagram showing a specific example of the decoding process related to this embodiment. Fig.23 2 shows a three-dimensional data decoder 213, a description decoder 248, and a prompter 247. In this example, the three-dimensional data decoder 213 includes a two-dimensional data decoder 241, a mesh data decoder 242, and a mesh reconstructor 246. The two-dimensional data decoder 241 includes a texture decoder 243. The mesh data decoder 242 includes a vertex information decoder 244 and a connection information decoder 245.

[0291] The vertex information decoder 244, the connection information decoder 245, the texture decoder 243 and the mesh reconstruction 246 may also correspond to Figure 8 The vertex information decoder 201, the connection information decoder 202, the attribute information decoder 203 and the post-processor 205 etc. The prompter 247 may also correspond to Fig.12 Prompt 215, etc.

[0292] For example, the two-dimensional data decoder 241 operates as a texture decoder 243 and decodes the texture corresponding to the attribute information from the texture file into two-dimensional data according to the image coding method or the video coding method.

[0293] The mesh data decoder 242 also operates as a vertex information decoder 244 and a connection information decoder 245 to decode vertex information and connection information from the mesh file. The mesh data decoder 242 may also decode mapping information for textures from the mesh file.

[0294] In addition, the description decoder 248 decodes the description corresponding to the metadata such as text data from the description file. The description decoder 248 may also decode the description in the system layer. For example, the description decoder 248 may also be included in Fig.12 in the system demultiplexer 214.

[0295] The mesh reconstructor 246 reconstructs the three-dimensional mesh from the vertex information, connection information and texture according to the description. The prompter 247 draws the three-dimensional mesh according to the description and outputs it.

[0296] Through the above-mentioned actions, a three-dimensional mesh is reconstructed and output from a bit stream including a texture file, a mesh file and a description file.

[0297] Furthermore, the three-dimensional data decoder 213 may include two mesh data decoders as the mesh data decoders 242. For example, one mesh data decoder decodes vertex information and connection information of a static three-dimensional mesh, and the other mesh data decoder decodes vertex information and connection information of a dynamic three-dimensional mesh.

[0298] Furthermore, two mesh files may be included in the bitstream corresponding to this. For example, one mesh file corresponds to a static three-dimensional mesh, and the other mesh file corresponds to a dynamic three-dimensional mesh.

[0299] In addition, the static 3D mesh may be a 3D mesh within a frame encoded using intra-frame prediction, and the dynamic 3D mesh may be a 3D mesh between frames encoded using inter-frame prediction. In addition, as the information of the dynamic 3D mesh, differential information between vertex information or connection information of a 3D mesh within a frame and vertex information or connection information of a 3D mesh between frames may be used.

[0300] The dynamic 3D mesh coding method is sometimes referred to as DMC (Dynamic Mesh Coding). In addition, the dynamic 3D mesh coding method based on video is sometimes referred to as V-DMC (Video-based Dynamic Mesh Coding).

[0301] The point group coding method is sometimes called PCC (Point Cloud Compression). In addition, the video-based coding method of the point group is sometimes called V-PCC (Video-based Point Cloud Compression). In addition, the geometry-based coding method of the point group is sometimes called G-PCC (Geometry-based Point Cloud Compression).

[0302] <Installation example>

[0303] Fig.24 1 is a block diagram showing an implementation example of the coding device 100 according to the present embodiment. The coding device 100 includes a circuit 151 and a memory 152. For example, Figure 5 The multiple components of the encoding device 100 shown in FIG. Fig.24 The circuit 151 and memory 152 are shown installed.

[0304] The circuit 151 is a circuit for performing information processing and is a circuit capable of accessing the memory 152. For example, the circuit 151 is a dedicated or general-purpose electrical circuit for encoding a three-dimensional grid. The circuit 151 may also be a processor such as a CPU. In addition, the circuit 151 may also be a collection of multiple electrical circuits.

[0305] The memory 152 is a dedicated or general memory for storing information used by the circuit 151 to encode the three-dimensional grid. The memory 152 may also be an electrical circuit, or may be connected to the circuit 151. In addition, the memory 152 may also be included in the circuit 151. In addition, the memory 152 may also be a collection of multiple electrical circuits. In addition, the memory 152 may also be a magnetic disk or an optical disk, or may be expressed as a storage device or a recording medium. In addition, the memory 152 may be either a non-volatile memory or a volatile memory.

[0306] For example, the memory 152 may store both the three-dimensional mesh and the bit stream. In addition, the memory 152 may store a program for the circuit 151 to encode the three-dimensional mesh.

[0307] In addition, in the encoding device 100, it is not necessary to install Figure 5 All of the multiple constituent elements shown may not perform all of the multiple processes shown here. Figure 5A portion of the multiple components shown in the above figure may also be included in other devices, and a portion of the multiple processes shown here may also be performed by other devices. In addition, in the encoding device 100, the multiple components of the present disclosure may be arbitrarily combined and installed, and the multiple processes of the present disclosure may be arbitrarily combined and performed.

[0308] Fig.25 2 is a block diagram showing an implementation example of the decoding device 200 according to the present embodiment. The decoding device 200 includes a circuit 251 and a memory 252. For example, Figure 7 The multiple components of the decoding device 200 shown in FIG. Fig.25 The circuit 251 and memory 252 are shown installed.

[0309] The circuit 251 is a circuit for performing information processing and is a circuit capable of accessing the memory 252. For example, the circuit 251 is a dedicated or general-purpose electrical circuit for decoding a three-dimensional mesh. The circuit 251 may also be a processor such as a CPU. In addition, the circuit 251 may also be a collection of multiple electrical circuits.

[0310] The memory 252 is a dedicated or general memory for storing information used by the circuit 251 to decode the three-dimensional mesh. The memory 252 can be an electrical circuit or connected to the circuit 251. In addition, the memory 252 can also be included in the circuit 251. In addition, the memory 252 can also be a collection of multiple electrical circuits. In addition, the memory 252 can be a magnetic disk or an optical disk, etc., and can also be expressed as a storage device or a recording medium. In addition, the memory 252 can be a non-volatile memory or a volatile memory.

[0311] For example, the memory 252 may store both the three-dimensional mesh and the bit stream. In addition, the memory 252 may store a program for the circuit 251 to decode the three-dimensional mesh.

[0312] In addition, in the decoding device 200, it is also possible not to install Figure 7 All of the multiple constituent elements shown may not perform all of the multiple processes shown here. Figure 7 A portion of the multiple components shown in the above-mentioned figure may also be included in other devices, and a portion of the multiple processes shown here may also be performed by other devices. In addition, in the decoding device 200, multiple components of the present disclosure may be arbitrarily combined and installed, and multiple processes of the present disclosure may be arbitrarily combined and performed.

[0313] The encoding method and decoding method including the steps performed by the components of the encoding device 100 and the decoding device 200 of the present disclosure may also be executed by any device or system. For example, a part or all of the encoding method and the decoding method may be executed by a computer having a processor, a memory, and an input / output circuit. In this case, the encoding method and the decoding method may also be executed by executing a program for causing the computer to execute the encoding method and the decoding method.

[0314] In addition, a non-transitory computer-readable recording medium such as a CD-ROM may record a program or a bit stream.

[0315] An example of a program may also be a bitstream. For example, a bitstream containing an encoded three-dimensional mesh contains syntax elements for causing the decoding device 200 to decode the three-dimensional mesh. Furthermore, the bitstream causes the decoding device 200 to decode the three-dimensional mesh according to the syntax elements contained in the bitstream. Therefore, the bitstream can play the same role as a program.

[0316] The above-mentioned bit stream may be a coded bit stream including a coded three-dimensional mesh, or may be a multiplexed bit stream including a coded three-dimensional mesh and other information.

[0317] In addition, each component of the encoding device 100 and the decoding device 200 may be composed of dedicated hardware, general-purpose hardware that executes the above-mentioned program, etc., or a combination thereof. In addition, the general-purpose hardware may also be composed of a memory that records the program and a general-purpose processor that reads the program from the memory and executes it. Here, the memory may also be a semiconductor memory or a hard disk, etc., and the general-purpose processor may also be a CPU, etc.

[0318] In addition, the dedicated hardware may also be constituted by a memory and a dedicated processor, etc. For example, the dedicated processor may refer to the memory for recording data to execute the encoding method and the decoding method.

[0319] In addition, each component of the encoding device 100 and the decoding device 200 may also be an electrical circuit as described above. These electrical circuits may be constituted as one electrical circuit as a whole, or may be different electrical circuits. In addition, these electrical circuits may correspond to dedicated hardware or general hardware that executes the above-mentioned program, etc. In addition, the encoding device 100 and the decoding device 200 may also be installed as an integrated circuit.

[0320] In addition, the encoding device 100 may also be a transmitting device that transmits a three-dimensional mesh, and the decoding device 200 may also be a receiving device that receives a three-dimensional mesh.

[0321] <Reference Example Regarding Encoding and Decoding of Vertex Information>

[0322] For example, in encoding and decoding of vertex information, a parallelogram prediction method for predicting the positions of vertices may be used.

[0323] Fig.26 This is a conceptual diagram showing a parallelogram prediction method related to a reference example. For example, after encoding the vertex information of the first vertex A, the second vertex B, and the third vertex C of the first triangle, the vertex information of the fourth vertex D of the second triangle having the same second vertex B and third vertex C as the first triangle is encoded. At this time, the parallelogram prediction method is used to predict the position of the fourth vertex D based on the first vertex A, the second vertex B, and the third vertex C, and the error with respect to the prediction is encoded.

[0324] Specifically, first, based on the first triangle ABC, the vertex P of the imaginary parallelogram ABCP having the diagonal lines BC and AP is calculated as a prediction point. The coordinate value of the vertex P is derived using the coordinate values ​​of the first vertex A, the second vertex B, and the third vertex C of the first triangle ABC (P = B + C - A). In addition, an error vector representing the difference between the vertex D and the vertex P in the three-dimensional space is encoded.

[0325] In a more complex version, multiple triangles are used. In this approach, although the compression process is slower, better prediction is achieved.

[0326] In the encoding and decoding of vertex information, a polynomial prediction method for predicting the position of a vertex may also be used. In the polynomial prediction method, the position of a new vertex is predicted using a local configuration. Specifically, the coordinate values ​​of several vertices in the local vicinity are provided to a polynomial function with pre-set coefficients, and the position of the new vertex is predicted according to the polynomial function. The vertices used in the prediction of the new vertex are selected according to specific conditions.

[0327] Fig. 27 is a conceptual diagram showing a polynomial prediction method related to a reference example. Fig. 27 In , the four encoded vertices are used to derive the predicted point of the next vertex. Fig. 27 In , the coordinate values ​​of vertex #1, vertex #2, vertex #3, and vertex #4 are used to predict vertex #5. As in the parallelogram prediction method, an error vector indicating an error with respect to the prediction is encoded.

[0328] In the reference example, the vertex information is encoded using error vectors calculated using prediction methods such as the parallelogram prediction method or the polynomial prediction method. Regardless of the complexity of these prediction methods, it is difficult to ensure that the magnitude of these errors is within a specific range. For example, in an obtuse triangle, there is a tendency for error vectors to be larger than those in an equilateral triangle. In addition, the smoothness of the shape and the size of the surface have an impact on the magnitude of the error. Therefore, it is possible to use more bits in the signal transmission of the vertex information.

[0329] <First Form of Vertex Information Encoding and Decoding>

[0330] In this embodiment, three angles, namely, the dihedral angle and two other angles, are used to encode the non-shared vertices of the adjacent triangles relative to the reference triangle. That is, when the vertex information is encoded using the spatial relationship between adjacent triangles, only three angles are encoded. This makes it possible to reduce the number of bits.

[0331] The sign of the angle can also be used to indicate the direction of the angle relative to the side. A predetermined rule can also be used: a positive value indicates a clockwise direction relative to the reference, and a negative value indicates a counterclockwise direction relative to the reference.

[0332] For example, the two-dimensional plane of the second triangle can be found by the dihedral angle formed by the first triangle as the reference triangle and the second triangle as the processing target triangle. Therefore, the complexity of three dimensions can be reduced to two dimensions to which simpler rules on a plane can be applied.

[0333] The dihedral angle between adjacent faces can also be calculated using the face normal. Furthermore, a corresponding triangle is constructed on the plane of the first triangle as a one-to-one imaginary congruent triangle relative to the second triangle. The first triangle and the corresponding triangle have a common side and are located on opposite sides relative to the common side. The vertices of the corresponding triangle are determined using two angles. Therefore, with respect to the vertices of the second triangle, the vertices of the second triangle are determined by the dihedral angle formed by the first triangle and the second triangle, and the two angles used to determine the vertices of the corresponding triangle that is congruent with the second triangle.

[0334] Therefore, the vertex information of the second triangle can be encoded based on the three angles of the dihedral angle and the other two angles. And, the value of the encoding object can be suppressed from becoming too large according to the expression range of the angle. Therefore, the worst-case code amount can be suppressed regardless of the properties and quality of the three-dimensional mesh.

[0335] Furthermore, the values ​​of these angles tend to follow a specific distribution. Therefore, the compression rate can be improved by utilizing such a trend. For example, the distribution of angles can be investigated and the difference relative to the average value of the distribution can be encoded.

[0336] <Example of Encoding Processing in the First Aspect>

[0337] Fig.28 1 is a flowchart showing an example of encoding processing related to this form. First, a first vertex, a second vertex, and a third vertex are derived from a three-dimensional mesh (S101). Here, the first vertex, the second vertex, and the third vertex form a first triangle on a first plane of the three-dimensional mesh. In addition, for example, the first vertex, the second vertex, and the third vertex are three vertices that have been encoded.

[0338] Next, the first angle, the second angle, and the third angle for deriving the fourth vertex of the second triangle formed on the second plane of the three-dimensional mesh using the side shared with the first triangle are derived (S102).

[0339] Next, the first angle, the second angle, and the third angle are encoded into a bit stream (S103).

[0340] Fig.29 1 is a flowchart showing a specific example of the encoding process related to this aspect. First, the coordinate values ​​of each vertex of the initial face are encoded based on the three-dimensional mesh (S111). The initial face can be randomly selected.

[0341] Next, the process traverses to the next face (S112), and a traversal symbol indicating the traversal instruction and the processing mode of the vertex is written into the bit stream.

[0342] Here, if the vertex of the face has been encoded ("Yes" in S113), the encoding of the vertex is skipped. And, if not all faces have been traversed, the process traverses to the next face. In addition, if all faces have been traversed, the process can be terminated.

[0343] On the other hand, when the vertex has not been encoded (No in S113), the processing mode of the vertex is designated according to the written traversal symbol.

[0344] Vertices are encoded using coordinate values ​​or angles. For example, when the traversal symbol represents coordinates (coordinates in S114), the vertex coordinate values ​​(x, y, z) are directly encoded (S115). This method can also be selected at the beginning of encoding when no vertex has been encoded yet. In addition, for example, when the traversal symbol represents angles (angles in S114), the vertex is encoded using three angles (S116).

[0345] The additional residual can be calculated or signaled separately after the angle is signaled. In the example where the floating point value of the angle is fully represented for reversible coding, the decimal part of the floating point value of the angle can be encoded as the residual value after the integer part of the floating point value of the angle is encoded. The traversal process is continued until the entire three-dimensional grid is traversed (S119).

[0346] A traversal symbol indicating one of skip, end, and execute may also be encoded. Skip corresponds to a vertex that has been processed. End corresponds to all vertices that have been processed. Execute corresponds to a vertex that should be processed. Furthermore, when a traversal symbol indicating execution is encoded, a processing mode indicating a coordinate and an angle may also be encoded separately from the traversal symbol. The processing mode may also be expressed as a mode, an action mode, a prediction mode, an encoding mode, or a decoding mode.

[0347] Fig.30 is a flowchart showing another specific example of the encoding process related to this embodiment. Fig.29 The encoding process of the reference example is integrated. First, the coordinate values ​​of each vertex of the initial face are encoded according to the three-dimensional mesh (S111).

[0348] Next, the process traverses to the next face (S112), and a traversal symbol indicating the traversal instruction and the processing mode of the vertex is written into the bit stream.

[0349] Here, if the vertex of the face has been encoded ("Yes" in S113), the encoding of the vertex is skipped. And, if not all faces have been traversed, the process traverses to the next face. In addition, if all faces have been traversed, the process can be terminated.

[0350] On the other hand, when the vertex has not been encoded (No in S113), the processing mode of the vertex is designated according to the written traversal symbol.

[0351] Vertices are encoded using coordinates, angles, parallelograms, or polynomials. For example, in the case where the traversal symbol represents coordinates (coordinates in S114), the vertex coordinate values ​​(x, y, z) are directly encoded (S115). This method can also be selected at the beginning of encoding when no vertex has been encoded yet. In addition, for example, in the case where the traversal symbol represents angles (angles in S114), the vertex is encoded using three angles (S116).

[0352] Furthermore, for example, when the ergodic symbol represents a parallelogram (parallelogram in S114), the vertices are encoded using the parallelogram (S117). Furthermore, for example, when the ergodic symbol represents a polynomial (polynomial in S114), the vertices are encoded using the polynomial (S118).

[0353] The optimal processing mode may also be determined based on the minimum cost of multiple costs calculated based on the coding rate and distortion for multiple processing modes. The cost of the coding rate and distortion may be calculated based on the number of bits transmitted by the signal and the error between the actual value and the predicted value. In addition, the optimal processing mode corresponding to the minimum error may also be selected based only on the error between the actual value and the predicted value.

[0354] The traversal process is continued until the entire three-dimensional grid is traversed ( S119 ).

[0355] A traversal symbol indicating one of skip, end, and execution may be encoded. Furthermore, when a traversal symbol indicating execution is encoded, a processing mode indicating one of coordinates, angles, parallelograms, and polygons may be encoded separately from the traversal symbol. Furthermore, when a processing mode is preset, the processing mode may not be encoded, and the processing may not be switched.

[0356] For example, Fig.31 , Fig.32 and Fig.33 As shown, the first triangle has a first vertex A, a second vertex B and a third vertex C, the second triangle has a second vertex B, a third vertex C and a fourth vertex D, and the corresponding triangle has a second vertex B, a third vertex C and a fifth vertex E.

[0357] Fig.31 : is a conceptual diagram showing an example of three angles related to this form. In this example, two interior angles of the corresponding triangle are used as the second angle β and the third angle γ. In addition, in this example, the two interior angles of the corresponding triangle are in contact with the common side. And, in this example, the second angle β and the third angle γ are calculated by the following formula (1).

[0358] [Formula 1]

[0359]

[0360] Fig.32 : is a conceptual diagram showing another example of three angles related to this form. In this example, the angle in contact with line AE is used as the second angle β, and the inner angle that is an inner angle of the corresponding triangle and in contact with the common side BC is used as the third angle γ. In this example, the second angle β and the third angle γ are calculated by the following formula (2).

[0361] [Formula 2]

[0362]

[0363] Fig.33 This is a conceptual diagram showing another example of three angles related to this form. In this example, one interior angle of the corresponding triangle is used as the second angle β, and the angle formed by AB and AQ is used as the third angle γ. Point Q is the positive projection of the fifth vertex E on the common side BC. In this example, the second angle β and the third angle γ are calculated by the following formula (3).

[0364] [Formula 3]

[0365]

[0366] exist Fig.31 , Fig.32 and Fig.33 , the dihedral angle between the planes of the first triangle and the second triangle is used as the first angle θ. The first angle θ can be calculated using their face normals. For example, the normal vectors of the first triangle and the second triangle are n1 = [x1, y1, z1] and n2 = [x2, y2, z2], respectively, and are assumed to be normalized. Under this assumption, the first angle θ between the planes of the first triangle and the second triangle is calculated, for example, by θ = π-cos －1 (n1n2)(or θ=cos －1 (n1n2)) calculation.

[0367] Fig.34 This is a syntax diagram showing an example of the syntax structure of vertex information for multiple vertices in this form. In this example, after encoding three vertices, a traversal code is encoded for each of the other vertices. In addition, whether to encode a vertex is controlled according to the traversal code.

[0368] Fig.35 This is a syntax diagram showing an example of the syntax structure of vertex information for one vertex in this aspect. In this example, when prediction_mode is equal to only_angles_mode, the derived angles for the first angle, the second angle, and the third angle are directly encoded.

[0369] Fig.36 This is a syntax diagram showing another example of the syntax structure of vertex information for one vertex related to this form. In this example, another method is used to derive or predict three angles without directly encoding them according to is_init_angles. An example of a method for deriving or predicting three angles is a method using a history list of previously encoded angles. Specifically, when the three angles of the encoding object are not the initial three angles, the three angles are derived or predicted using a history list of previously encoded angles.

[0370] Fig.37 1 is a table diagram showing an example of a table used in the prediction of an angle. In this example, previously encoded angles are associated with indices and stored in the table. The angle stored in the table can also be calculated by averaging the three previously encoded angles. The angles stored in the table can be controlled by first-in-first-out, frequency of occurrence of values, or a combination of both. The index can also be a fixed value.

[0371] Then, one of the multiple angles in the table is used as the predicted angle. When writing the syntax element, only the index value corresponding to the predicted angle and the delta which is the difference between the actual angle and the predicted angle are written into the bitstream.

[0372] Fig.38 This is a table diagram showing another example of a table used in the prediction of angles. In this example, the first angle, the second angle, and the third angle that have been previously encoded are associated with indexes and stored in the table. The angles stored in the table can be controlled by first-in first-out, the frequency of occurrence of values, or a combination of both. The index can also be a fixed value.

[0373] Then, one of the plurality of combinations in the table is used as the combination of prediction angles. When writing the syntax element, only the index value corresponding to the combination of prediction angles and the delta which is the difference between the actual angle and the prediction angle are written into the bitstream.

[0374] Fig.39 : is a conceptual diagram showing an example of using the prediction of the angles of the vertices of the previous triangle. In this example, the three angles of the vertices of the previous triangle are used to derive or predict the three angles of the vertices of the current triangle. The previous triangle is a triangle that was traversed before the current triangle. The previous triangle can be either the triangle immediately before the current triangle or a triangle adjacent to the current triangle.

[0375] Specifically, in this example, the three angles (10°, 30°, 60°) for the vertices of the previous triangle are used to determine the syntactic values ​​corresponding to the three angles for the vertices of the current triangle. For example, when the first angle for the vertex of the current triangle is 9°, 10°-9°=1° is written as the syntactic value.

[0376] Fig.40 2 is a conceptual diagram showing an example of prediction using angles for vertices of a previous three-dimensional mesh. In this example, three angles are derived or predicted using multiple angles for multiple vertices of a previous three-dimensional mesh.

[0377] For example, the previous 3D grid is a 3D grid that is different in space from the current 3D grid and is a 3D grid that was processed earlier than the current 3D grid. Or, for example, the previous 3D grid is a 3D grid that is different in time from the current 3D grid and is a 3D grid that was processed earlier than the current 3D grid.

[0378] Alternatively, the first angles, second angles, and third angles of the vertices of the previous three-dimensional mesh may be averaged and signaled in the current three-dimensional mesh. For example, as a syntactic value, the first delta = first angle x - average first angle is determined and written. For example, this method may be used in a dynamic mesh having spatial redundancy in multiple adjacent frames.

[0379] Fig.41 1 is a block diagram showing a configuration example of a coding device 100 according to the present embodiment. The coding device 100 codes a three-dimensional mesh into a bit stream. Fig.41 As shown, the encoding device 100 may also include a traverser 161, a switch 162, an angle deriver 163, an angle encoder 164, a coordinate encoder 165 and an entropy encoder 166.

[0380] The traverser 161 obtains a three-dimensional mesh. The traverser 161 determines the vertices of the encoding target in the three-dimensional mesh, and outputs the vertices to either the angle deriver 163 or the coordinate encoder 165 via the switch 162. In one example, the traverser 161 selects the coordinate encoder 165 for the first three vertices of the three-dimensional mesh, and selects the angle deriver 163 for the other vertices.

[0381] For example, the coordinate encoder 165 takes in three vertices forming the first triangle on the first plane and encodes the three vertices. Delta encoding can be used for encoding the three vertices. Specifically, the coordinate value of the first vertex A can be directly encoded, and the difference between the coordinate value of the first vertex A and the coordinate value of the second vertex B, and the difference between the coordinate value of the first vertex A and the coordinate value of the third vertex C can be encoded.

[0382] The angle deriver 163 takes in the fourth vertex D of the second triangle formed on the second plane using the second vertex B and the third vertex C, derives three angles for determining the fourth vertex D, and outputs the derived three angles to the angle encoder 164. The second plane may be different from the first plane or may be the same as the first plane. As described above, Fig.31 , Fig.32 and Fig.33 Different examples about three angles are shown.

[0383] The angle encoder 164 encodes the three angles. It is also possible to calculate an additional residual and transmit the signal separately after the angle signal is transmitted. In the example where the floating point decimal value of the angle is fully expressed for reversible encoding, the angle encoder 164 may encode the decimal part of the floating point decimal value of the angle as a residual value after encoding the integer part of the floating point decimal value of the angle.

[0384] The coded information obtained by the coordinate encoder 165 and the coded information obtained by the angle encoder 164 are sent to the entropy encoder 166 and also fed back to the traverser 161. The entropy encoder 166 compresses the coded information and outputs a bit stream. The traverser 161 uses the coded information to determine the next traversal.

[0385] Examples of the encoding method based on entropy encoding may also include Huffman coding, arithmetic coding, range coding, ANS (Asymmetric Numeral Systems) or CABAC (Context-Adaptive Binary Arithmetic Coding).

[0386] The encoding device 100 encodes the three-dimensional mesh according to the above-described configuration and processing. This may suppress the code amount of the three-dimensional mesh.

[0387] In the above description, the corresponding triangle is a triangle that is congruent to the second triangle, but it may also be a triangle that is similar to the second triangle. Even if the second triangle is similar to the corresponding triangle, as long as the correspondence between the second triangle and the corresponding triangle is known, the fourth vertex D can be derived from the fifth vertex E according to the correspondence between the second triangle and the corresponding triangle. In addition, the corresponding triangle may also be the positive projection of the second triangle onto the first plane. In this case, the fourth vertex D can also be derived from the fifth vertex E according to the correspondence between the second triangle and the corresponding triangle.

[0388] <Example of Decoding Processing in the First Aspect>

[0389] Fig.42 1 is a flowchart showing an example of decoding processing related to this form. First, a first vertex, a second vertex, and a third vertex are derived from a three-dimensional mesh (S201). Here, the first vertex, the second vertex, and the third vertex form a first triangle on a first plane of the three-dimensional mesh. In addition, for example, the first vertex, the second vertex, and the third vertex are three vertices that have been decoded.

[0390] Next, the first angle, the second angle, and the third angle used to derive the fourth vertex of the second triangle formed on the second plane of the three-dimensional mesh using the edge shared with the first triangle are decoded from the bitstream (S202). Next, the fourth vertex is derived using the first angle, the second angle, and the third angle (S203).

[0391] Fig.43 2 is a flowchart showing a specific example of the decoding process of this form. The decoding process starts with decoding the coordinate values ​​of each vertex of the first encoded initial face (S211). Next, the traversal symbol of the current face is decoded (S212). And, it is determined whether the traversal symbol represents coordinates, angles, skip and end (S213).

[0392] For example, when the traversal symbol indicates coordinates (coordinates in S213), the vertex coordinate values ​​(x, y, z) are directly decoded (S214). In addition, when the traversal symbol indicates angles (angles in S213), the vertices are decoded using three angles (S215). In addition, when the traversal symbol indicates skip (skip in S213), the process is repeated for the next face. In addition, when the traversal symbol indicates end (end in S213), the process ends.

[0393] A traversal symbol indicating one of skip, end, and execution may also be decoded. Skip corresponds to a vertex that has been processed. End corresponds to all vertices that have been processed. Execute corresponds to a vertex that should be processed. Furthermore, when a traversal symbol indicating execution is decoded, a processing mode indicating one of coordinates and angles may also be decoded separately from the traversal symbol. The processing mode may also be expressed as a mode, an action mode, a prediction mode, an encoding mode, or a decoding mode.

[0394] Fig.44 is a flowchart showing another specific example of the decoding process related to this embodiment. Fig.43 The decoding process of the reference example is integrated with the decoding process of the reference example. The decoding process starts with decoding the coordinate values ​​of each vertex of the initial face that was initially encoded (S211). Next, the traversal symbol of the current face is decoded (S212). And, it is determined whether the traversal symbol represents coordinates, angles, skip and end (S213).

[0395] For example, when the traversal symbol represents coordinates (coordinates in S213), the vertex coordinate values ​​(x, y, z) are directly decoded (S214). In addition, when the traversal symbol represents angles (angles in S213), the vertices are decoded using three angles (S215). In addition, when the traversal symbol represents a parallelogram (parallelogram in S213), the vertex is decoded using the parallelogram (S216). In addition, when the traversal symbol represents a polynomial (polynomial in S213), with respect to a vertex, the next vertex is decoded using the polynomial.

[0396] In addition, when the traversal symbol indicates skip (skip in S213), the process is repeated for the next face. In addition, when the traversal symbol indicates end (end in S213), the process ends.

[0397] It is also possible to decode a traversal symbol indicating one of skip, end, and execution. Furthermore, when a traversal symbol indicating execution is decoded, a processing mode indicating one of coordinates, angles, parallelograms, and polygons may be decoded separately from the traversal symbol. In addition, when a processing mode is preset, the processing mode may not be decoded, and the processing may not be switched.

[0398] Fig.34 This is a syntax diagram showing an example of the syntax structure of vertex information for multiple vertices in this form. In this example, after three vertices are decoded, the traversal code is decoded for each of the other vertices. And, whether to decode the vertex is controlled according to the traversal code.

[0399] Fig.35 This is a syntax diagram showing an example of the syntax structure of vertex information for one vertex in this aspect. In this example, when prediction_mode is equal to only_angles_mode, the derived angles for the first angle, the second angle, and the third angle are directly decoded.

[0400] Fig.36 This is a syntax diagram showing another example of the syntax structure of vertex information for one vertex related to this form. In this example, another method is used to derive or predict three angles without directly decoding them according to is_init_angles. An example of a method for deriving or predicting three angles is a method using a history list of previously decoded angles. Specifically, when the three angles of the decoding object are not the initial three angles, the three angles are derived or predicted using a history list of previously decoded angles.

[0401] Fig.371 is a table diagram showing an example of a table used in the prediction of an angle. In this example, previously decoded angles are associated with indices and stored in the table. The angle stored in the table may also be calculated by averaging three previously decoded angles. The angles stored in the table may be controlled by first-in-first-out, frequency of occurrence of values, or a combination thereof. The index may also be a fixed value.

[0402] Then, one of the multiple angles in the table is used as the prediction angle. In decoding the syntax element, only the index value corresponding to the prediction angle and the delta which is the difference between the actual angle and the prediction angle are decoded from the bit stream.

[0403] Fig.38 1 is a table diagram showing another example of a table used in the prediction of angles. In this example, the first angle, the second angle, and the third angle previously decoded are associated with indexes and stored in the table. The angles stored in the table may be controlled according to first-in first-out, the frequency of occurrence of values, or a combination of both. The index may also be a fixed value.

[0404] Then, one of the plurality of combinations in the table is used as the combination of prediction angles. In decoding the syntax element, only the index value corresponding to the combination of prediction angles and the delta which is the difference between the actual angle and the prediction angle are decoded from the bit stream.

[0405] Fig.39 : is a conceptual diagram showing an example of using the prediction of the angles of the vertices of the previous triangle. In this example, the three angles of the vertices of the previous triangle are used to derive or predict the three angles of the vertices of the current triangle. The previous triangle is a triangle that was traversed before the current triangle. The previous triangle can be either the triangle immediately before the current triangle or a triangle adjacent to the current triangle.

[0406] Specifically, in this example, the three angles (10°, 30°, 60°) for the vertices of the previous triangle are used to determine the syntactic values ​​corresponding to the three angles for the vertices of the current triangle. For example, when the first delta for the vertices of the current triangle is 1°, 10°-9°=1° is derived as the first angle.

[0407] Fig.40 2 is a conceptual diagram showing an example of prediction using angles for vertices of a previous three-dimensional mesh. In this example, three angles are derived or predicted using multiple angles for multiple vertices of a previous three-dimensional mesh.

[0408] For example, the previous 3D grid is a 3D grid that is different in space from the current 3D grid and is a 3D grid that was processed earlier than the current 3D grid. Or, for example, the previous 3D grid is a 3D grid that is different in time from the current 3D grid and is a 3D grid that was processed earlier than the current 3D grid.

[0409] Alternatively, the first angles, second angles, and third angles of the vertices of the previous three-dimensional mesh may be averaged and signaled in the current three-dimensional mesh. For example, the first angle x = average first angle + first delta may be derived from the syntax value. For example, this method may be used in a dynamic mesh having spatial redundancy in multiple adjacent frames.

[0410] The additional residual can also be decoded after the decoding of the three angles. In one example, the value decoded as the angle can also represent the integer part of the floating point decimal value of the angle, and the value decoded as the residual can also represent the decimal part of the floating point decimal value of the angle. By adding the decimal part to the integer part, the actual angle is obtained.

[0411] For example, Fig.31 , Fig.32 and Fig.33 As shown, the first triangle has a first vertex A, a second vertex B and a third vertex C, the second triangle has a second vertex B, a third vertex C and a fourth vertex D, and the corresponding triangle has a second vertex B, a third vertex C and a fifth vertex E.

[0412] In deriving the fourth vertex D of the second triangle, first, the fifth vertex E, which is the projection of the fourth vertex D on the first plane including the first triangle, is calculated using the second angle and the third angle. Then, the fourth vertex D is calculated from the fifth vertex E using the first angle, which is the dihedral angle between the first triangle and the second triangle.

[0413] Fig.31 This is a conceptual diagram showing an example of three angles related to this form. In this example, the two internal angles of the corresponding triangle are used as the second angle β and the third angle γ. That is, in this example, the two internal angles of the corresponding triangle are decoded from the bit stream. By applying the sine rule to the triangle CBE, the following formula (4) is obtained.

[0414] [Formula 4]

[0415]

[0416] When the second angle β, the third angle γ, and the side CB are given, the coordinate value of the fifth vertex E is derived from the above-mentioned formula (4).

[0417] Fig.32This is a conceptual diagram showing another example of three angles related to this form. In this example, the angle in contact with line AE is used as the second angle β, and an internal angle that is an internal angle of the corresponding triangle and in contact with the common side BC is used as the third angle γ. In this example, after decoding the first vertex A, the second vertex B, and the third vertex C from the bit stream, the fourth angle δ formed by BA and BC is derived by the following formula (5).

[0418] [Formula 5]

[0419]

[0420] Next, by applying the sine law to triangle ABE, the following equation (6) is obtained.

[0421] [Formula 6]

[0422]

[0423] The magnitude of BE is derived by the above equation (6). Therefore, the direction and magnitude of BE and the plane where the fifth vertex E is located are known, and the coordinate value of the fifth vertex E can be derived.

[0424] Fig.33 This is a conceptual diagram showing another example of three angles related to this form. In this example, one interior angle of the corresponding triangle is used as the second angle β, and the angle formed by AB and AQ is used as the third angle γ. Point Q is the positive projection of the fifth vertex E on the common side BC. By applying the sine rule to triangle AQB, the following formula (7) is obtained.

[0425] [Formula 7]

[0426]

[0427] The size of QB can be obtained by the above equation (7). Therefore, the coordinate value of point Q can be derived. Next, the following equation (8) is obtained by the right triangle rule of triangle CEQ.

[0428] [Formula 8]

[0429]

[0430] The magnitude of CE is derived by the above equation (8). Therefore, the direction and magnitude of CE and the plane where the fifth vertex E is located are known, and the coordinate value of the fifth vertex E can be derived.

[0431] In use Fig.31 , Fig.32 and Fig.33In the processing of each of the multiple derivation methods described, the fifth vertex E can be derived. And the fourth vertex D can be derived from the fifth vertex E. For example, the second triangle CBD and the corresponding triangle CBE are one-to-one identical. In addition, the fourth vertex D and the fifth vertex E are on different planes that share the common side CB. In this case, the fourth vertex D can be derived by rotating the fifth vertex E around the side BC by π-θ (or θ). Here, θ is the first angle.

[0432] Fig.45 is a table diagram showing an example of a lookup table for the sine function and the cosine function. In the calculation of the above sine function and the cosine function, Fig.45 The lookup table shown.

[0433] Fig.46 2 is a block diagram showing a configuration example of a decoding device 200 according to the present embodiment. The decoding device 200 decodes a three-dimensional mesh from a bit stream. Fig.46 As shown, the decoding device 200 includes a traverser 261, a switch 262, an angle decoder 263, a coordinate exporter 264, a coordinate decoder 265 and an entropy decoder 266.

[0434] The entropy decoder 266 obtains the bit stream. The entropy decoder 266 decompresses the bit stream, obtains decompressed information, and outputs the decompressed information. Examples of the encoding method based on entropy decoding may also be Huffman coding, arithmetic coding, range coding, ANS (Asymmetric Numeral Systems) or CABAC (Context-Adaptive Binary Arithmetic Coding).

[0435] The decompressed information is input to the traverser 261 . The traverser 261 interprets the decompressed information and outputs the decompressed information to either the angle decoder 263 or the coordinate decoder 265 via the switch 262 .

[0436] The coordinate decoder 265 decodes the three vertices forming the first triangle on the first plane based on the decompressed information. As an example of decoding the three vertices, the coordinate value and two differential values ​​of the first vertex A are decoded based on the decompressed information. Then, by adding the two differential values ​​to the coordinate value of the first vertex A, the coordinate value of the second vertex B and the coordinate value of the third vertex C are derived.

[0437] The angle decoder 263 decodes the three angles according to the decompressed information and outputs the three angles to the coordinate exporter 264 .

[0438] The coordinate deriver 264 uses the three angles to derive the vertices of the second triangle having the same side as the first triangle. The second triangle may be located on a plane different from the first plane or on the same plane as the first plane. Specifically, the coordinate deriver 264 uses the three angles to derive the vertices of the second triangle having the same side as the first triangle. Fig.31 , Fig.32 and Fig.33 In the derivation method described above, the vertices of the second triangle are derived based on three angles.

[0439] The vertex information indicating the vertices obtained by the coordinate deriver 264 or the coordinate decoder 265 is output as vertex information of the three-dimensional mesh. In addition, the vertex information is fed back to the traverser 261 for use in determining the next traversal.

[0440] The decoding device 200 decodes the three-dimensional mesh according to the above-mentioned configuration and processing. This may suppress the code amount of the three-dimensional mesh.

[0441] In the above description, the corresponding triangle is a triangle that is congruent to the second triangle, but it may also be a triangle that is similar to the second triangle. Even if the second triangle is similar to the corresponding triangle, as long as the correspondence between the second triangle and the corresponding triangle is known, the fourth vertex D can be derived from the fifth vertex E according to the correspondence between the second triangle and the corresponding triangle. In addition, the corresponding triangle may also be the positive projection of the second triangle onto the first plane. In this case, the fourth vertex D can also be derived from the fifth vertex E according to the correspondence between the second triangle and the corresponding triangle.

[0442] <Supplementary information about encoding and decoding of vertex information>

[0443] The encoding process of vertex information of the present disclosure can be applied to encoding of point position information in point cloud compression methods such as V-PCC (Point Cloud Compression) and G-PCC (Geometry-based Point Cloud Compression), for example.

[0444] The encoding device 100 encodes information for determining the fourth vertex D of the second triangle based on the first reference triangle or its three vertices, namely, the first vertex A, the second vertex B, and the third vertex C. The decoding device 200 decodes information for determining the fourth vertex D of the second triangle based on the first reference triangle or its three vertices, namely, the first vertex A, the second vertex B, and the third vertex C.

[0445] The information for determining the fourth vertex D includes dihedral angle information indicating a dihedral angle and determination information for determining the fifth vertex E. The dihedral angle indicated by the dihedral angle information is a dihedral angle formed by a first plane including the first triangle and a second plane including the second triangle. The fifth vertex E determined by the determination information is a vertex on the first plane and is a vertex of a corresponding triangle corresponding to the second triangle.

[0446] Since the fifth vertex E is a vertex on the first plane, it can be determined relatively easily based on the first triangle on the first plane. In addition, the fourth vertex D can be derived from the fifth vertex E according to the corresponding relationship between the second triangle and the corresponding triangle and the dihedral angle formed by the first plane and the second plane. Since the dihedral angle is, for example, in the range of -180 degrees to 180 degrees, the increase in the code amount is suppressed. Therefore, through the above-mentioned configuration and processing, the fourth vertex D can be effectively encoded and decoded.

[0447] In addition, the information used to determine the 4th vertex D may also include "(1) position information of the vertex (5th vertex E) of the polygon (corresponding triangle BEC)" or "(2) information related to the position and shape of the polygon (corresponding triangle BEC)" as the above-mentioned determination information or instead of the above-mentioned determination information.

[0448] In addition, the information used to determine the fourth vertex D may also include "(3) information indicating the distance and direction between the position of the vertex (fifth vertex E) of the polygon (corresponding triangle BEC) and the position of the processing object vertex (fourth vertex D)" as the above-mentioned dihedral angle information or instead of the above-mentioned dihedral angle information. In addition, the information used to determine the fourth vertex D may also include "(4) information indicating the difference between the plane containing the polygon (corresponding triangle BEC) and the plane containing the polygon (second triangle BDC) having the processing object vertex (fourth vertex D)" as the above-mentioned dihedral angle information or instead of the above-mentioned dihedral angle information.

[0449] In addition, one of the above information may be represented by an angle. This may reduce the amount of code and improve the flexibility and accuracy of prediction. In addition, information indicating which of the above information is used may be encoded.

[0450] Furthermore, for example, the following examples may be added to the above-described configuration and processing, or at least a part of the above-described configuration and processing may be replaced with the following examples.

[0451] For example, "(1) position information of the vertices of the polygon" may also indicate the coordinate values ​​of the second vertex B, the third vertex C, and the fifth vertex E of the corresponding triangle BEC. When the information on the position and shape of the first reference triangle or the position information of the first vertex A, the second vertex B, and the third vertex C is determined, the position information of the vertices of the polygon may also indicate only the coordinate value of the fifth vertex E.

[0452] When it is predetermined or determined that the fifth vertex E existing in the same plane as the plane including the first triangle ABC is used in deriving the position information of the fourth vertex D to be processed, the coordinate value of the fifth vertex E may be expressed by a two-dimensional coordinate value. The coordinate value may be expressed by a difference with other coordinate values, or the coordinate value of the fifth vertex E may be expressed by a vector indicating a difference with respect to the coordinate value of the first vertex A, the second vertex B, or the third vertex C.

[0453] Furthermore, for example, the polygon may be a quadrilateral other than a triangle.

[0454] For example, "(2) information related to the position and shape of the polygon" may also be information that specifies the position and shape of the corresponding triangle BEC. When information related to the position and shape of the first reference triangle ABC, or position information of the first vertex A, the second vertex B, and the third vertex C is specified, the information that specifies the position and shape of the corresponding triangle BEC may also be information indicating the side BC, the angle between the side BC and the side CE, and the angle between the side BC and the side BE. Here, the information indicating the side BC corresponds to the information indicating the use of the corresponding triangle BEC.

[0455] Instead of the information indicating the side BC, information indicating the first vertex A may be encoded. The information indicating the first vertex A corresponds to the information indicating that the fifth vertex E is located on the opposite side of the first vertex A across the side BC.

[0456] When it is predetermined or determined that the corresponding triangle BEC on the same plane as the first triangle ABC is used in deriving the position information of the fourth vertex D of the processing object, only the above two angles may be encoded with respect to the fifth vertex E. Thus, the position and shape of the corresponding triangle BEC, that is, the position information of the fifth vertex E, can be derived only from the two angles, and the amount of code can be reduced compared with the case where the coordinate value of the fifth vertex E is encoded.

[0457] As the two angles, the angles themselves may be encoded, or an index that determines one of a plurality of predetermined angles may be encoded. A plurality of combinations of two angles may be predetermined. Furthermore, an index that determines one of a plurality of combinations may be encoded.

[0458] Instead of the two angles, information indicating one angle and the length of one side may be encoded. For example, the angle formed by side BC and side BE and the length of side BE may be encoded. In addition, instead of the two angles, information indicating one angle and the length of one perpendicular line may be encoded. For example, the angle formed by side BC and side BE and the length of the perpendicular line drawn from the fifth vertex E to side BC may be encoded.

[0459] In addition, instead of the two angles, information indicating the lengths of two sides may be encoded. For example, the lengths of side BE and side CE may be encoded.

[0460] For example, "(3) information indicating the distance and direction between the position of the vertices of the polygon and the position of the vertex to be processed" may also indicate a differential vector indicating the distance and direction between the fifth vertex E of the corresponding triangle BEC and the fourth vertex D of the processing object. In the case where the position of the fourth vertex D coincides with the position of the fifth vertex E or the fourth vertex D is located on the same plane as the fifth vertex E, this information may be omitted.

[0461] For example, "(4) information indicating the difference between the plane containing the polygon and the plane containing the polygon having the processing object vertex" may also indicate the angle between the plane containing the corresponding triangle BEC and the plane containing the second triangle BDC. As the angle, for example, the angle between the side BE and the side BD or the angle between the side CE and the side CD may also be encoded.

[0462] As the angle between the side BE and the side BD or the angle between the side CE and the side CD, the angle itself may be encoded, or an index that specifies one of the predetermined multiple angles may be encoded. The predetermined multiple angles may also be used together when deriving the angle between the side BC and the side CE and the angle between the side BC and the side BE.

[0463] In addition, it is also possible to predefine indices representing multiple combinations of the angle between side BE and side BD, the angle between side CE and side CD, the angle between side BC and side CE, and the angle between side BC and side BE. This can further reduce the amount of code compared to encoding the angle itself.

[0464] When the position of the fourth vertex D coincides with the position of the fifth vertex E or the fourth vertex D is located on the same plane as the fifth vertex E, the angle may be omitted. In addition, when the angle is represented by an index, when the angle is represented with integer precision, or when the coordinate value is represented with integer precision, the differential vector may be encoded in addition to the angle. Thus, the derived coordinate value of the fourth vertex D can be corrected, and the encoding accuracy can be improved.

[0465] Furthermore, the fourth vertex D may be used as the first vertex A, the second vertex B, or the third vertex C relative to other vertices.

[0466] In addition, multiple processing methods may be switched by selecting a processing method such as an encoding method or a decoding method for each processing object vertex. In this case, information indicating the processing method used to encode the processing object vertex may also be encoded. Alternatively, multiple processing methods may be switched by selecting a processing method in three-dimensional mesh units, and information indicating the processing method may also be encoded in three-dimensional mesh units.

[0467] This makes it possible to select a processing method that matches the characteristics of the three-dimensional mesh, and to promote improvement in subjective image quality and reduction in the amount of code.

[0468] In addition, the difference described in the present disclosure may also be referred to as deviation, residual, or delta, etc. In addition, the vertex described in the present disclosure may refer to the position of the vertex.

[0469] In addition, the corresponding triangle in the above description may also be expressed as a predicted triangle or as a hypothetical corresponding triangle. In addition, the fifth vertex E of the corresponding triangle may also be expressed as a corresponding vertex, a corresponding point, a predicted vertex or a predicted point.

[0470] <Second Form of Vertex Information Encoding and Decoding>

[0471] The second form of encoding and decoding of vertex information is a form that can be combined with the first form of encoding and decoding of vertex information.

[0472] Regardless of the prediction method used in mesh encoding, incompleteness of the input 3D mesh can cause exceptions that require special manipulation tools. Exceptions can exist in various components of the polygonal mesh such as connectivity, geometric coordinates or properties. Common defects in 3D models are degenerate faces, duplicate vertices and unconnected parts.

[0473] In the 3D graphics data captured by various hardware in reality, there are often irregularities such as degenerate faces, duplicate vertices, and unconnected parts. Degenerate faces occur when at least two of the multiple vertices constituting the face have the same vertex index, so the surface area becomes zero and the surface normal becomes uncertain.

[0474] Duplicate vertices have exactly the same coordinates but are represented by two or more different indices. Therefore, the same point is represented by different indices in different faces. Therefore, when traversing the topology of the mesh, these vertices are considered different, which leads to flaws in encoding techniques that rely on using vertices that have been encoded in the past. In addition, since there may be unconnected parts, i.e. incomplete parts, in the polygon mesh, it may be difficult to traverse the entire model.

[0475] In order to solve the exceptional situation, a virtual reference triangle is generated to enable the implementation of the angle-based 3D graphics coding technology. The virtual reference triangle can be constructed using a combination of previously encoded vertices and pseudo vertices selected according to a specified condition. The exceptional handling of the 3D model can also be indicated from the encoding device 100 to the decoding device 200 using a mode setting parameter.

[0476] In this aspect, when an exception occurs, a new vertex can be encoded using a virtual reference triangle.

[0477] <Example of Encoding Processing in Second Aspect>

[0478] Fig.47 1 is a flowchart showing an example of encoding processing related to this aspect. First, connection information of the first vertex, the second vertex, and the third vertex is derived from the three-dimensional mesh (S121). Here, the first vertex, the second vertex, and the third vertex form a first triangle.

[0479] Next, determine whether the derived connection information satisfies a condition (S122). Here, there may be the following two conditions. The first condition is that the first vertex, the second vertex, and the third vertex are connected to each other. The second condition is that the first vertex, the second vertex, and the third vertex have been encoded. In addition, not only the connection information but also the vertex information may be derived and used for these determinations.

[0480] Fig.48 This is a conceptual diagram showing an example of a mesh related to this form. A1, B1, C1, D1, E1, and F1 are vertices in the geometric map.

[0481] Fig.49 is an example of a fragmented mesh, and is a conceptual diagram showing an example where some of the three vertices that make up a triangle are not connected. A2, B2, C2, D2, E2, and F2 are vertices in the connectivity map. Fig.49 In the figure, solid lines indicate connections, and dashed lines indicate disconnections. In addition, parts without lines are also disconnected. In the code of vertex D1, since there is no connectivity between A2 and C2 and between A2 and B2, vertex A1 of triangle A1B1C1 is not suitable for use.

[0482] In addition, the lack of connectivity between two vertices means that the two vertices are not connected, which is also expressed as the lack of connectivity of the edge between the two vertices.

[0483] Fig.50 This is a conceptual diagram showing an example of a mesh with duplicate vertices, showing an example where the vertices of the reference triangle are not encoded before the encoding of the encoding target vertex. Vertex C and vertex F are two different points in the mesh and exist on different faces in the mesh. Vertex C and vertex F have the same coordinates, so they are displayed as duplicate points.

[0484] First, vertices A, B, and C are encoded in traversal order. Next, vertex D is encoded using the angle method using reference triangle ABC. Next, when vertex E is encoded using reference triangle BDF, vertex F has not yet been encoded.

[0485] Furthermore, in order to effectively express a three-dimensional image, virtual vertices and virtual triangles may be used in a three-dimensional mesh. In such a case, there is a possibility that the vertices are not encoded or there is no connectivity between the vertices.

[0486] exist Fig.47 In the case where it is determined that the derived connection information satisfies the condition ("Yes" in S122), the first vertex, the second vertex and the third vertex are derived as three vertices (S123). Here, the second vertex, the third vertex and the fourth vertex form a second triangle on the three-dimensional mesh, and the fourth vertex is on the opposite side of the first vertex.

[0487] Fig.31 In this example of representing the positions of three vertices, B and C are the second and third vertices, A is the first vertex, and D is the fourth vertex. The reference triangle formed by ABC can be used to encode vertex D.

[0488] exist Fig.47 In the case where it is determined that the derived connection information does not satisfy the condition ("No" in S122), three vertices including at least the fifth vertex are derived (S124). The fifth vertex is different from the first vertex, the second vertex, and the third vertex. In addition, the fifth vertex in this form is a vertex used to replace one of the first vertex, the second vertex, and the third vertex, and has a different meaning from the fifth vertex in the first form.

[0489] Fig.51 This is a conceptual diagram showing an example of exception processing using a vertex set that was previously encoded. This example is an example of a case where vertex B does not meet the conditions (does not exist or has not been encoded yet). The reference triangle BCD cannot be used for encoding vertex E. Vertex B is replaced by vertex A (the fifth vertex) that was previously encoded, forming a new virtual reference triangle ACD that can be used for encoding vertex E.

[0490] In another example, the fifth vertex may be made as a pseudo vertex with predefined coordinates. The imaginary reference triangle may also be formed by including pseudo vertices. The encoding device 100 may also calculate pseudo vertices. In addition, pseudo vertices may be predefined as fixed coordinates (1, 0, 0), (0, 1, 0) and (0, 0, 1), and these pseudo vertices may function as the first vertex, the second vertex and the third vertex.

[0491] The coordinates of the pseudo-vertices may also be selected by a method inherent to the model. In one example, the encoding device 100 may calculate the size of the 3D bounding box of the object and use these values ​​to create pseudo-vertices. The created pseudo-vertices may also be signaled using a bitstream. In addition, in the case where there are multiple vertices that do not meet the conditions, the same number of pseudo-vertices or vertices that have been encoded in the past as the number of the multiple vertices that do not meet the conditions may also be used.

[0492] exist Fig.47 In step S125, a plurality of angles for determining a fourth vertex are derived from the triangle formed by the derived three vertices. Here, the fourth vertex forms a second triangle using the edges shared by the triangle formed by the derived three vertices. The number of the derived plurality of angles depends on the prediction method used for encoding the fourth vertex. Fig.31 , Fig.32 and Fig.33 A different example of deriving at least a plurality of angles using the fourth vertex and the derived three vertices A, B, and C is shown.

[0493] The plurality of angles are encoded into a bitstream (S126). An example of encoding the plurality of angles is using entropy coding. Examples of encoding methods based on entropy coding may also be Huffman coding, arithmetic coding, range coding, asymmetric digital system (ANS) or context adaptive binary arithmetic coding (CABAC).

[0494] Fig.52 1 is a flowchart showing a specific example of the encoding process related to this embodiment. Fig.52 In the flowchart shown, Fig.30 Compared with the flowchart shown in FIG. 1 , a process (S131) ​​for encoding vertices using exceptions is added. This process (S131) ​​corresponds to Fig.47 The processes shown (S124, S125, S126) are as follows. That is, the process of encoding the vertices using exceptions is one of the encoding modes used in the compression of the mesh data.

[0495] Fig.53 1 is a block diagram showing a configuration example of a coding device 100 according to the present embodiment. The coding device 100 codes a three-dimensional mesh into a bit stream. Fig.53As shown, the encoding device 100 may also include a traverser 161, a determiner 167, an exception handler 168, an angle deriver 163 and an entropy encoder 166.

[0496] The traverser 161 obtains the three-dimensional mesh. In addition, the traverser 161 outputs the connection information of the first vertex, the second vertex, and the third vertex derived from the three-dimensional mesh to the determiner 167. In addition, the traverser 161 determines the fourth vertex of the encoding object in the three-dimensional mesh. Here, the first vertex, the second vertex, and the third vertex form a first triangle, which has a common side with the second triangle including the fourth vertex.

[0497] The determiner 167 checks whether the connection information satisfies the condition. If it is determined that the connection information satisfies the condition, the angle deriver 163 performs processing using the first vertex, the second vertex, and the third vertex.

[0498] When it is determined that the connection information does not satisfy the condition, the exception handler 168 derives three vertices including at least the fifth vertex. The angle deriving unit 163 performs processing using the three vertices including at least the fifth vertex.

[0499] The angle deriver 163 uses the three vertices and the fourth vertex to derive a plurality of angles. In the example where the vertices represent geometric coordinates, three angles are derived. The number of the derived angles depends on the prediction method used in encoding the fourth vertex.

[0500] The entropy encoder 166 encodes the multiple angles derived by the angle deriver 163 into a bitstream.

[0501] <Example of Decoding Processing in Second Aspect>

[0502] Fig.54 2 is a flowchart showing an example of decoding processing related to this aspect. First, connection information of the first vertex, the second vertex, and the third vertex is derived from the bit stream (S221). Here, the first vertex, the second vertex, and the third vertex form a first triangle.

[0503] Next, determine whether the derived connection information satisfies a condition (S222). Here, there may be the following two conditions. The first condition is that the first vertex, the second vertex, and the third vertex are connected to each other. The second condition is that the first vertex, the second vertex, and the third vertex have been decoded. In addition, not only the connection information but also the vertex information may be derived and used for these determinations.

[0504] Fig.48 This is a conceptual diagram showing an example of a mesh related to this form. A1, B1, C1, D1, E1, and F1 are vertices in the geometric map.

[0505] Fig.49is an example of a fragmented mesh, and is a conceptual diagram showing an example where some of the three vertices that make up a triangle are not connected. A2, B2, C2, D2, E2, and F2 are vertices in the connectivity map. Fig.49 In the figure, solid lines indicate connections, and dashed lines indicate disconnections. In addition, parts without lines are also disconnected. In the decoding of vertex D1, since there is no connectivity between A2 and C2 and between A2 and B2, vertex A1 of triangle A1B1C1 is not suitable for use.

[0506] In addition, the lack of connectivity between two vertices means that the two vertices are not connected, which is also expressed as the lack of connectivity of the edge between the two vertices.

[0507] Fig.50 This is a conceptual diagram showing an example of a mesh with duplicate vertices, showing an example where the vertices of the reference triangle are not decoded before the decoding of the target vertices. Vertex C and vertex F are two different points in the mesh and exist on different faces in the mesh. Vertex C and vertex F have the same coordinates, so they are displayed as duplicate points.

[0508] First, vertices A, B, and C are decoded in traversal order. Next, vertex D is decoded using the angle method using reference triangle ABC. Next, when vertex E is decoded using reference triangle BDF, vertex F has not yet been decoded.

[0509] Furthermore, in order to effectively express a three-dimensional image, virtual vertices and virtual triangles may be used in a three-dimensional mesh. In such a case, there is a possibility that the vertices are not decoded or there is no connectivity between the vertices.

[0510] exist Fig.54 In the case where it is determined that the derived connection information satisfies the condition ("Yes" in S222), the first vertex, the second vertex and the third vertex are derived as three vertices (S223). Here, the second vertex, the third vertex and the fourth vertex form a second triangle on the three-dimensional mesh, and the fourth vertex is on the opposite side of the first vertex.

[0511] Fig.31 In this example, the positions of three vertices are shown. B and C are the second and third vertices, A is the first vertex, and D is the fourth vertex. The reference triangle formed by ABC can be used to decode vertex D.

[0512] exist Fig.54In the case where it is determined that the derived connection information does not satisfy the condition ("No" in S222), three vertices including at least the fifth vertex are derived (S224). The fifth vertex is different from the first vertex, the second vertex, and the third vertex. In addition, the fifth vertex in this form is a vertex used to replace one of the first vertex, the second vertex, and the third vertex, and has a different meaning from the fifth vertex in the first form.

[0513] Fig.51 This is a conceptual diagram showing an example of exception processing using a vertex set that was decoded in the past. This example is an example of a case where vertex B does not meet the conditions (does not exist or has not been decoded yet). The reference triangle of BCD cannot be used for decoding vertex E. Vertex B is replaced by vertex A (the fifth vertex) that was decoded previously, forming a new virtual reference triangle ACD that can be used for decoding vertex E.

[0514] In another example, the fifth vertex may be made into a pseudo vertex having predefined coordinates. The imaginary reference triangle may also be formed by including pseudo vertices. The decoding device 200 may also calculate pseudo vertices similarly to the encoding device 100. In addition, pseudo vertices may be predefined as fixed coordinates (1, 0, 0), (0, 1, 0) and (0, 0, 1), and these pseudo vertices may function as the first vertex, the second vertex and the third vertex.

[0515] The coordinates of the pseudo-vertices may also be selected by a method inherent to the model. In one example, the decoding device 200 may also calculate the size of the 3D bounding box of the object and use these values ​​to create pseudo-vertices. In addition, the pseudo-vertices created by the encoding device 100 may also be signaled using a bit stream and decoded by the decoding device 200. In addition, in the case where there are multiple vertices that do not meet the conditions, the same number of pseudo-vertices as the number of the multiple vertices that do not meet the conditions or vertices that have been decoded in the past may also be used.

[0516] exist Fig.54 In step S225, a plurality of angles are decoded from the bitstream. An example of decoding the plurality of angles is using entropy decoding. An example of a coding method based on entropy decoding may also be Huffman coding, arithmetic coding, range coding, asymmetric digital system (ANS) or context adaptive binary arithmetic coding (CABAC). The number of the plurality of angles decoded depends on the prediction method used in the decoding of the fourth vertex.

[0517] The fourth vertex is derived using a plurality of angles (S226). Here, the fourth vertex forms the second triangle using a side common to the triangle formed by the derived three vertices. Fig.31 , Fig.32 and Fig.33 Different examples of deriving a fourth vertex using the derived three vertices A, B, and C and a plurality of angles are shown.

[0518] Fig.55 1 is a flowchart showing a specific example of the decoding process related to this embodiment. Fig.55 In the flowchart shown, Fig.44 Compared with the flowchart shown in FIG. 1 , a process (S231) for decoding the vertex using an exception is added. This process (S231) corresponds to Fig.54 The processes shown (S224, S225, S226) are as follows. That is, the process of decoding the vertices using the exception is one of the decoding modes used for the expansion of the bit stream.

[0519] Fig.56 2 is a block diagram showing a configuration example of a decoding device 200 according to the present embodiment. The decoding device 200 decodes a three-dimensional mesh from a bit stream. Fig.56 As shown, the decoding device 200 includes an entropy decoder 266 , a traverser 261 , a determiner 267 , an exception handler 268 , and a coordinate deriver 264 .

[0520] The entropy decoder 266 obtains the bit stream. In addition, the entropy decoder 266 decodes connection information and a plurality of angles of the first vertex, the second vertex, and the third vertex from the bit stream. Here, the first vertex, the second vertex, and the third vertex form a first triangle.

[0521] The traverser 261 outputs the connection information decoded by the entropy decoder 266 to the determiner 267 .

[0522] The determiner 267 checks whether the connection information satisfies the condition. If it is determined that the connection information satisfies the condition, the coordinate deriver 264 performs processing using the first vertex, the second vertex, and the third vertex.

[0523] When determining that the connection information does not satisfy the condition, the exception handler 268 extracts three vertices including at least the fifth vertex.

[0524] The coordinate deriver 264 derives and outputs the fourth vertex in the three-dimensional mesh using the three vertices and a plurality of angles. Here, the fourth vertex forms the second triangle using the side shared by the triangle formed by the three vertices.

[0525] In this aspect, when a reference triangle used for encoding or decoding a vertex includes an unconnected edge or an unencoded or undecoded vertex, a new virtual reference triangle is formed using new vertices, thereby efficiently encoding or decoding the vertex.

[0526] <Supplementary Information Regarding the Second Form of Encoding and Decoding of Vertex Information>

[0527] In this form, Fig.47 or Fig.54As shown, whether to perform the exception process can be selected based on the result of the determination of whether the information of the reference triangle satisfies the condition. The condition may be, for example, the following condition.

[0528] (i) The sides of the triangle have connectivity (see Fig.49 ). For example, among the vertices of the reference triangle, a vertex located opposite to the processing target vertex is connected to another vertex of the reference triangle via an edge shared by the triangle including the processing target vertex and the reference triangle.

[0529] (ii) The vertices of the reference triangle have been encoded or decoded.

[0530] In addition, the above-mentioned conditions (i) and (ii) are examples, and only one of the conditions may be used for determination, different conditions may be added, or at least one of the conditions may be replaced by a different condition. In other words, in a structure capable of selecting or switching between the first process and the second process, the information of the reference triangle may be used as a condition for selection or switching, wherein the first process uses the reference triangle to process (encode or decode) the vertices, and the second process uses a triangle different from the reference triangle to process the vertices of the object.

[0531] The reference triangle may be defined as a triangle that shares vertices or edges with the triangle having the processing object vertex. A triangle different from the reference triangle may be a triangle that does not share at least one of vertices and edges with the triangle having the processing object vertex. Alternatively, any triangle may be a polygon different from the triangle.

[0532] In determining the selection or switching between the first process and the second process, one or any combination of the position information of the vertices of the reference triangle, the connection information, and the information on whether the vertices can be used (whether the vertices exist or whether the vertices have been encoded or decoded) may be used. In addition, information on other triangles or information on the vertices of the triangle having the vertices to be processed may also be used.

[0533] Instead of selecting or switching the first process and the second process, the process may be selected or switched from a plurality of processes including the first process and the second process. That is, the plurality of processes may include a process different from the first process and the second process.

[0534] Furthermore, as described above, a reference triangle is used in the first process, and a triangle different from the reference triangle is used in the second process, but the first process and the second process are not limited to these. That is, the same triangle may be used in the first process and the second process.

[0535] For example, the first processing may be processing using a reference triangle, and the second processing may be other processing using the reference triangle. Alternatively, the first processing may be processing using a triangle different from the reference triangle, and the second processing may be other processing using a triangle different from the reference triangle.

[0536] Furthermore, for example, the first process may be performed using a reference triangle, and the second process may be performed using another triangle in which at least one side has no connectivity.

[0537] In this case, a process using another triangle having connected sides may be defined as the third process. Also, a process may be selected or switched from a plurality of processes including the first process, the second process, and the third process.

[0538] In this case, the selectable exception processing may be different based on the determination result of the condition. The combination of the determination result of the condition and the exception processing may be, for example, the following configuration.

[0539] As an example, in association with the above-mentioned condition (i), when the vertices of the reference triangle have been processed (encoded or decoded) but one of the sides of the reference triangle has no connectivity, a process may be selected from a plurality of processes including the first process and the third process.

[0540] Here, when the first process is executed, the reference triangle is used regardless of the presence or absence of connectivity, so it is possible to suppress an increase in the amount of processing. When the third process is executed, it is possible to use triangles that constitute the same object as the object constituted by the triangle having the processing target vertex, and it is possible to more efficiently represent the processing target vertex than the first process.

[0541] In addition, as another example, in association with the above-mentioned condition (ii), when the vertex of the reference triangle has not been processed (encoded or decoded), a process may be selected from a plurality of processes including the second process and the third process. In other words, regardless of connectivity, a triangle different from the reference triangle may be used for processing. In the case of the third process, it is possible to perform the processing more efficiently than the second process.

[0542] Furthermore, as another example, when at least one of the conditions (i) and (ii) is not satisfied, the third process described above may always be selected. In other words, when at least one of the conditions (i) and (ii) is not satisfied, the reference triangle may be set to be unusable, and other triangles whose vertices have been processed and whose edges have connectivity may be used for processing. With this structure, it is possible to perform processing more efficiently.

[0543] In addition, the position information of the vertices in the present disclosure is not limited to the information of geometric coordinates, and may be other position information different from geometry.

[0544] For example, in the event of an exception, the vertices that cannot be used for encoding or decoding are replaced with vertices that have already been encoded or decoded. An example of vertices that have already been encoded or decoded is the last three vertices in the portion of the mesh that is being encoded or decoded.

[0545] One or more additional vertices may be used to form a new reference triangle. An index indicating which vertex is used to form a reference triangle may be transmitted from the encoding device 100 to the decoding device 200. Such signal transmission is applicable not only in exceptional cases but also in non-exceptional cases. In addition, such signal transmission is applied not only to the "angle" prediction mode but also to other prediction modes that use a reference triangle such as the parallelogram prediction mode.

[0546] For example, in the case of an exception, a pseudo reference triangle is generated using predefined vertices. The predefined vertices may also be (1, 0, 0), (0, 1, 0), and (0, 0, 1). In addition, a new reference triangle may be created using predefined vertices. In addition, a new reference triangle may be used for vertex processing.

[0547] Fig.57 This is a flowchart showing an example of exception processing related to this aspect. For example, the vertices of the reference triangle are set using predefined values ​​(S301). Then, the vertices of the new triangle are encoded or decoded using the reference triangle.

[0548] Fig.58 1 is a flowchart showing another example of encoding processing related to this embodiment. First, the encoding device 100 derives the coordinates of the fourth vertex as the encoding object from the mesh (S151). Here, the triangle having the fourth vertex is defined as the second triangle, and the triangle sharing the side with the second triangle is defined as the reference triangle.

[0549] The encoding device 100 derives the connection information of the reference triangle from the mesh (S152). In the following steps, the encoding device 100 determines whether the connection information of the reference triangle satisfies a condition, and performs a vertex coordinate encoding process based on the determination result. Here, the following two are exemplified as determination processes.

[0550] (i) Whether the sides of the reference triangle are connected (S153) For example, whether the vertices at both ends of the side shared by the reference triangle and the second triangle are connected to the remaining vertices of the reference triangle.

[0551] (ii) Check whether each vertex of the reference triangle has been encoded (S154).

[0552] When the result of determination in at least one of (i) and (ii) above is negative (No in S153 or No in S154), the encoding device 100 encodes the information for determining the coordinates of the fourth vertex based on the exception processing exemplified in the present disclosure (S155, S156). On the other hand, when both (i) and (ii) above are satisfied (Yes in S153 and Yes in S154), the encoding device 100 encodes the information for determining the coordinates of the fourth vertex based on the reference triangle. Here, an example of using angle information in encoding is shown (S156).

[0553] In addition, the exception processing when (i) is not satisfied and the exception processing when (ii) is not satisfied may be different processing. In addition, not only the connection information but also the vertex information may be derived and used for determination.

[0554] Fig.59 25 is a flowchart showing another example of the decoding process related to this form. First, the decoding device 200 obtains information for determining the coordinates of the fourth vertex of the decoding object from the bit stream. Here, an example of decoding angle information is shown (S251). Here, the triangle having the fourth vertex is defined as the second triangle, and the triangle sharing the side with the second triangle is defined as the reference triangle.

[0555] The decoding device 200 derives the connection information of the reference triangle from the bitstream (S252). In the following steps, the decoding device 200 determines whether the connection information of the reference triangle satisfies a condition, and performs a decoding process of the vertex coordinates based on the determination result. Here, the following two determination processes are exemplified.

[0556] (i) Whether the sides of the reference triangle are connected (S253) For example, whether the vertices at both ends of the side shared by the reference triangle and the second triangle are connected to the remaining vertices of the reference triangle.

[0557] (ii) Check whether each vertex of the reference triangle has been decoded (S254).

[0558] When the result of determination in at least one of the above (i) and (ii) is negative (No in S253 or No in S254), the decoding device 200 derives (decodes) the coordinates of the fourth vertex based on the exception processing exemplified in the present disclosure (S255, S256). On the other hand, when both (i) and (ii) are satisfied (Yes in S253 and Yes in S254), the decoding device 200 derives (decodes) the coordinates of the fourth vertex based on the reference triangle. Here, an example of using angle information in derivation (decoding) is shown (S256).

[0559] In addition, the exception processing when (i) is not satisfied and the exception processing when (ii) is not satisfied may be different processing. In addition, not only the connection information but also the vertex information may be derived and used for determination.

[0560] Fig.60 is a conceptual diagram showing an example of the location of parameters in a bitstream. Information for exception handling may also be as follows Fig.60 The shown is signaled as a parameter in the bitstream.

[0561] Fig.61 is a conceptual diagram showing another example of the location of parameters in a bitstream. Information for exception handling may also be as follows Fig.61 The shown is signaled as a parameter in the header of the bitstream.

[0562] In addition, a parameter may be used to indicate the method of exception handling. In one example, the parameter is an index with a value of 0 or 1. The value may indicate whether to use a predefined pseudo-vertex or a signaled pseudo-vertex. In the case of using a signaled pseudo-vertex, the pseudo-vertex may also be signaled after the parameter.

[0563] <Representative example>

[0564] Fig.62 is a flowchart showing an example of basic encoding processing related to this embodiment. For example, Fig.24 The circuit 151 of the encoding device 100 shown in FIG. 1 is in operation Fig.62 The encoding process shown.

[0565] Specifically, when the first triangle satisfies the condition ("Yes" in S161), the circuit 151 selects a mode for encoding the encoding target vertex of the second triangle from the first mode set, wherein the first mode set includes a normal mode for encoding the encoding target vertex using the first triangle (S162). On the other hand, when the first triangle does not satisfy the condition ("No" in S161), the circuit 151 selects a mode from the second mode set, wherein the second mode set includes an exception mode for encoding the encoding target vertex using a third triangle different from both the first triangle and the second triangle (S163).

[0566] Next, the circuit 151 encodes the encoding target vertex according to the mode (S164).

[0567] Thus, when the first triangle does not satisfy the condition, the vertices of the second triangle may be encoded using a third triangle different from the first triangle. Therefore, the vertices of the second triangle may be encoded using a reference triangle suitable for encoding.

[0568] For example, the third triangle may be a triangle that shares a side with the first triangle. Thus, a third triangle near the first triangle used in the normal mode can be used in the exception mode. Thus, the difference between the normal mode and the exception mode can be reduced. Therefore, the deviation of the code amount can be suppressed.

[0569] In addition, for example, the third triangle may be a triangle that shares a side with the second triangle. Thus, there is a case where the third triangle near the second triangle can be used in the exception mode. Therefore, there is a case where the encoding target vertex can be effectively encoded using the third triangle near the encoding target vertex.

[0570] In addition, for example, the third triangle may be a triangle that shares vertices with the first triangle. Thus, a third triangle near the first triangle used in the normal mode can be used in the exception mode. Thus, the difference between the normal mode and the exception mode can be reduced. Therefore, the deviation of the code amount can be suppressed.

[0571] In addition, for example, the third triangle may be a triangle whose vertices have been encoded. Thus, it is possible to encode the encoding target vertex using the third triangle that reflects the characteristics of the three-dimensional mesh. Therefore, it is possible to efficiently encode the encoding target vertex.

[0572] In addition, for example, the third triangle may be a triangle whose sides have connectivity. Thus, it is possible to encode the encoding target vertex using the third triangle reflecting the characteristics of the three-dimensional mesh. Therefore, it is possible to efficiently encode the encoding target vertex.

[0573] In addition, for example, the third triangle may be a triangle having a dummy vertex. Thus, there is a case where the encoding target vertex can be encoded using the third triangle having an arbitrary vertex. Therefore, there is a case where the encoding target vertex can be flexibly encoded.

[0574] In addition, for example, the exception mode may include a plurality of sub-modes. Furthermore, the circuit 151 may select a sub-mode for encoding the encoding target vertex from the plurality of sub-modes when the exception mode is selected as the mode. Thus, the encoding target vertex may be encoded according to the sub-mode selected from the plurality of sub-modes in the exception mode. Therefore, the encoding target vertex may be encoded according to the exception processing method adaptively selected from the plurality of exception processing methods.

[0575] In addition, for example, the circuit 151 may encode a parameter indicating a sub-mode. Thus, information on a sub-mode for encoding a vertex to be encoded may be shared between the encoding device 100 and the decoding device 200 according to the parameter. Therefore, the same sub-mode may be applied to encoding and decoding according to the parameter.

[0576] In addition, for example, the circuit 151 may determine the third triangle according to the sub-mode. Thus, by selecting a sub-mode from a plurality of sub-modes, a third triangle suitable for encoding may be selected. Thus, the encoding target vertex may be efficiently encoded using the third triangle suitable for encoding.

[0577] In addition, for example, the circuit 151 may determine the vertices of the third triangle according to the sub-pattern. Thus, there is a case where the vertices of the third triangle suitable for encoding can be selected by selecting a sub-pattern from a plurality of sub-patterns. Therefore, there is a case where the encoding target vertices can be efficiently encoded using the third triangle suitable for encoding.

[0578] In addition, for example, the condition may include that each vertex of the first triangle has been encoded. This can prevent the use of a vertex of the first triangle that is not encoded in the encoding of the vertices of the second triangle. In addition, a mode suitable for encoding the vertices of the second triangle can be used.

[0579] In addition, for example, the condition may include that the sides of the first triangle have connectivity. This can prevent the use of the first triangle whose sides do not have connectivity in the coding of the vertices of the second triangle. In addition, a mode suitable for coding the vertices of the second triangle can be used.

[0580] Furthermore, for example, when the coding target vertex is coded in the exception mode, the circuit 151 may code the angle representing the coding target vertex using the third triangle.

[0581] This can prevent the amount of code from becoming excessively large. In other words, it can prevent the worst-case amount of code.

[0582] In addition, for example, the first triangle and the second triangle may share a side. In addition, the side shared by the first triangle and the second triangle may correspond to the opposite side of the encoding target vertex of the second triangle.

[0583] In addition, for example, the third triangle may have a side shared by the first triangle and the second triangle. That is, the first triangle, the second triangle, and the third triangle may share a side. In this case, the third triangle shares a side with the first triangle, shares a side with the second triangle, shares a vertex with the first triangle, and shares a vertex with the second triangle. However, the third triangle is not limited to such a triangle, and may have, for example, any side or any vertex of the first triangle.

[0584] Furthermore, for example, the normal mode may be a mode in which the coding target vertex is coded using the first triangle and one or more angles, and the exceptional mode may be a mode in which the coding target vertex is coded using the third triangle and one or more angles.

[0585] For example, the first pattern set and the second pattern set are different from each other. Specifically, the normal mode may not be included in the second pattern set but only in the first pattern set. Also, the exception mode may not be included in the first pattern set but only in the second pattern set.

[0586] In addition, for example, the first pattern set may also include at least one of a pattern for encoding the encoding target vertex using coordinates, a pattern for encoding the encoding target vertex using a parallelogram, and a pattern for encoding the encoding target vertex using a polynomial. Here, the parallelogram may also be based on the first triangle. Furthermore, the normal pattern may also be a pattern for encoding the encoding target vertex using a parallelogram based on the first triangle.

[0587] In addition, for example, the second mode set may also include at least one of a mode for encoding the encoding target vertex using coordinates, a mode for encoding the encoding target vertex using a parallelogram, and a mode for encoding the encoding target vertex using a polynomial. Here, the parallelogram may also be based on a third triangle. Furthermore, the exception mode may also be a mode for encoding the encoding target vertex using a parallelogram based on a third triangle.

[0588] In addition, for example, the first mode set may include only the normal mode. Furthermore, the circuit 151 may select the normal mode when the first triangle satisfies the condition. That is, the circuit 151 may encode the encoding target vertex according to the normal mode when the first triangle satisfies the condition.

[0589] In addition, for example, the second mode set may include only the exception mode. Furthermore, the circuit 151 may select the exception mode when the first triangle does not satisfy the condition. That is, the circuit 151 may encode the encoding target vertex according to the exception mode when the first triangle does not satisfy the condition.

[0590] In addition, for example, the circuit 151 may select a mode from a mode set including a normal mode and an exception mode. In this case, the mode set may correspond to both the first mode set and the second mode set. Furthermore, the circuit 151 may select the normal mode when the first triangle satisfies the condition and encode the encoding target vertex according to the normal mode. Furthermore, the circuit 151 may select the exception mode when the first triangle does not satisfy the condition and encode the encoding target vertex according to the exception mode.

[0591] In addition, for example, multiple sub-modes may include a sub-mode for encoding the vertices of the encoding object using the third triangle with pseudo vertices. Multiple sub-modes may also include a sub-mode for encoding the vertices of the encoding object using the third triangle whose vertices have been encoded. Multiple sub-modes may also include a sub-mode for encoding the vertices of the encoding object using the third triangle whose edges have connectivity. The sub-mode may correspond to the condition of the third triangle only or to the condition of the vertices of the third triangle.

[0592] That is, the sub-mode may correspond to the method of selecting the third triangle or the vertices of the third triangle.

[0593] In addition, for example, the normal mode may include a plurality of sub-modes. Furthermore, the circuit 151 may select a sub-mode for encoding the encoding target vertex from the plurality of sub-modes when the normal mode is selected as the mode. The circuit 151 may determine the first triangle or the vertex of the first triangle according to the sub-mode. That is, the sub-mode may correspond to the method of selecting the first triangle or the vertex of the first triangle.

[0594] Furthermore, for example, a sub-pattern included in an exception pattern may be expressed as an exception sub-pattern, and a sub-pattern included in a normal pattern may be expressed as a normal sub-pattern.

[0595] In addition, for example, a mode may be expressed as a process, and a normal mode, an exception mode, and a sub-mode may be expressed as a normal process, an exception process, and a sub-process.

[0596] In addition, for example, the circuit 151 may encode the encoding target vertex by entropy coding. Specifically, the circuit 151 may encode the angle representing the encoding target vertex by entropy coding. In addition, for example, the circuit 151 may encode 1, 2, or 3 angles used to determine the encoding target vertex.

[0597] Furthermore, for example, the first triangle may be a reference triangle that is set as a triangle closer to the encoding target vertex according to a reference rule.

[0598] In addition, for example, each of the above-mentioned vertices can also be expressed by geometric coordinates in a three-dimensional mesh.

[0599] Fig.63 This is a flowchart of an example of basic decoding processing related to this embodiment. For example, Fig.25 The circuit 251 of the decoding device 200 shown in FIG. Fig.63 The decoding process shown.

[0600] Specifically, when the first triangle satisfies the condition ("Yes" in S261), the circuit 251 selects a mode for decoding the decoding target vertex of the second triangle from the first mode set, wherein the first mode set includes a normal mode for decoding the decoding target vertex using the first triangle (S262). On the other hand, when the first triangle does not satisfy the condition ("No" in S261), the circuit 251 selects a mode from the second mode set, wherein the second mode set includes an exception mode for decoding the decoding target vertex using a third triangle different from both the first triangle and the second triangle (S263).

[0601] Next, the circuit 251 decodes the decoding target vertex according to the mode ( S264 ).

[0602] Thus, when the first triangle does not satisfy the condition, the vertices of the second triangle may be decoded using a third triangle different from the first triangle. Therefore, the vertices of the second triangle may be decoded using a reference triangle suitable for decoding.

[0603] For example, the third triangle may be a triangle that shares a side with the first triangle. Thus, a third triangle near the first triangle used in the normal mode can be used in the exception mode. Thus, the difference between the normal mode and the exception mode can be reduced. Therefore, the deviation of the code amount can be suppressed.

[0604] In addition, for example, the third triangle may be a triangle that shares a side with the second triangle. Thus, a third triangle near the second triangle may be used in the exception mode. Thus, a third triangle near the decoding target vertex may be used to effectively decode the decoding target vertex.

[0605] In addition, for example, the third triangle may be a triangle that shares vertices with the first triangle. Thus, a third triangle near the first triangle used in the normal mode can be used in the exception mode. Thus, the difference between the normal mode and the exception mode can be reduced. Thus, the deviation of the code amount can be suppressed.

[0606] In addition, for example, the third triangle may be a triangle whose vertices have been decoded. Thus, the decoding target vertex may be decoded using the third triangle reflecting the characteristics of the three-dimensional mesh. Therefore, the decoding target vertex may be decoded efficiently.

[0607] In addition, for example, the third triangle may be a triangle whose sides have connectivity. Thus, the decoding target vertex may be decoded using the third triangle reflecting the characteristics of the three-dimensional mesh. Therefore, the decoding target vertex may be decoded efficiently.

[0608] In addition, for example, the third triangle may be a triangle having a dummy vertex. Thus, the decoding target vertex may be decoded using the third triangle having an arbitrary vertex. Therefore, the decoding target vertex may be decoded flexibly.

[0609] In addition, for example, the exception mode may include a plurality of sub-modes. Furthermore, the circuit 251 may select a sub-mode for decoding the decoding target vertex from the plurality of sub-modes when the exception mode is selected as the mode. Thus, the decoding target vertex may be decoded according to the sub-mode selected from the plurality of sub-modes in the exception mode. Therefore, the decoding target vertex may be decoded according to the exception processing method adaptively selected from the plurality of exception processing methods.

[0610] In addition, for example, the circuit 251 may decode a parameter indicating a sub-mode. Thus, information on a sub-mode for decoding a vertex to be decoded may be shared between the encoding device 100 and the decoding device 200 according to the parameter. Therefore, the same sub-mode may be applied to encoding and decoding according to the parameter.

[0611] In addition, for example, the circuit 251 may determine the third triangle according to the sub-mode. Thus, by selecting a sub-mode from a plurality of sub-modes, a third triangle suitable for decoding may be selected. Thus, the decoding target vertex may be efficiently decoded using the third triangle suitable for decoding.

[0612] In addition, for example, the circuit 251 may determine the vertices of the third triangle according to the sub-pattern. Thus, by selecting a sub-pattern from a plurality of sub-patterns, it is possible to select the vertices of the third triangle suitable for decoding. Thus, it is possible to efficiently decode the decoding target vertices using the third triangle suitable for decoding.

[0613] In addition, for example, the condition may include that each vertex of the first triangle has been decoded. Thus, it is possible to suppress the use of a vertex of the first triangle that has not been decoded in decoding the vertices of the second triangle. In addition, it is possible to use a mode suitable for decoding the vertices of the second triangle.

[0614] In addition, for example, the condition may include that the sides of the first triangle have connectivity. This can prevent the use of the first triangle whose sides do not have connectivity in decoding the vertices of the second triangle. In addition, a mode suitable for decoding the vertices of the second triangle can be used.

[0615] In addition, for example, the circuit 251 may decode the angle representing the decoding target vertex using the third triangle when decoding the decoding target vertex in the exception mode. This can prevent the amount of code from becoming too large. In other words, it can prevent the worst-case amount of code.

[0616] Furthermore, for example, the first triangle and the second triangle may share a common side. Furthermore, the common side between the first triangle and the second triangle may correspond to the opposite side of the decoding target vertex in the second triangle.

[0617] In addition, for example, the third triangle may also have a side shared by the first triangle and the second triangle. That is, the first triangle, the second triangle, and the third triangle may share a side. In this case, the third triangle shares a side with the first triangle, shares a side with the second triangle, shares a vertex with the first triangle, and shares a vertex with the second triangle. However, the third triangle is not limited to such a triangle, and for example, may also have any side or any vertex of the first triangle.

[0618] Furthermore, for example, the normal mode may be a mode in which the first triangle and one or more angles are used to decode the decoding target vertex, and the exceptional mode may be a mode in which the third triangle and one or more angles are used to decode the decoding target vertex.

[0619] In addition, for example, the first pattern set and the second pattern set are different from each other. Specifically, the normal mode may not be included in the second pattern set but only in the first pattern set. In addition, the exception mode may not be included in the first pattern set but only in the second pattern set.

[0620] In addition, for example, the first pattern set may also include at least one of a pattern for decoding the decoding object vertex using coordinates, a pattern for decoding the decoding object vertex using a parallelogram, and a pattern for decoding the decoding object vertex using a polynomial. Here, the parallelogram may also be based on the first triangle. Furthermore, the normal pattern may also be a pattern for decoding the decoding object vertex using a parallelogram based on the first triangle.

[0621] In addition, for example, the second mode set may also include at least one of a mode for decoding the decoding target vertex using coordinates, a mode for decoding the decoding target vertex using a parallelogram, and a mode for decoding the decoding target vertex using a polynomial. Here, the parallelogram may also be based on a third triangle. Furthermore, the normal mode may also be a mode for decoding the decoding target vertex using a parallelogram based on a third triangle.

[0622] In addition, for example, the first mode set may include only the normal mode. Furthermore, the circuit 251 may select the normal mode when the first triangle satisfies the condition. That is, the circuit 251 may decode the decoding target vertex according to the normal mode when the first triangle satisfies the condition.

[0623] In addition, for example, the second mode set may include only the exception mode. Furthermore, the circuit 251 may select the exception mode when the first triangle does not satisfy the condition. That is, the circuit 251 may decode the decoding target vertex according to the exception mode when the first triangle does not satisfy the condition.

[0624] In addition, for example, the circuit 251 may select a mode from a mode set including a normal mode and an exception mode. In this case, the mode set may correspond to both the first mode set and the second mode set. Furthermore, the circuit 251 may select the normal mode when the first triangle satisfies the condition and decode the decoding object vertex according to the normal mode. Furthermore, the circuit 251 may select the exception mode when the first triangle does not satisfy the condition and decode the decoding object vertex according to the exception mode.

[0625] In addition, for example, multiple sub-modes may also include a sub-mode for decoding the vertices of the decoding object using the third triangle with pseudo vertices. Multiple sub-modes may also include a sub-mode for decoding the vertices of the decoding object using the third triangle whose vertices have been decoded. Multiple sub-modes may also include a sub-mode for decoding the vertices of the decoding object using the third triangle whose edges have connectivity. The sub-mode may correspond to the condition of the third triangle or the condition of the vertices of the third triangle.

[0626] That is, the sub-mode may correspond to the method of selecting the third triangle or the vertices of the third triangle.

[0627] In addition, for example, the normal mode may include a plurality of sub-modes. Furthermore, the circuit 251 may select a sub-mode for decoding the decoding target vertex from the plurality of sub-modes when the normal mode is selected as the mode. The circuit 251 may determine the first triangle or the vertex of the first triangle according to the sub-mode. That is, the sub-mode may correspond to the method of selecting the first triangle or the vertex of the first triangle.

[0628] Furthermore, for example, a sub-pattern included in an exception pattern may be expressed as an exception sub-pattern, and a sub-pattern included in a normal pattern may be expressed as a normal sub-pattern.

[0629] In addition, for example, a mode may be expressed as a process, and a normal mode, an exception mode, and a sub-mode may be expressed as a normal process, an exception process, and a sub-process.

[0630] In addition, for example, the circuit 251 may also decode the decoding object vertex by entropy decoding. Specifically, the circuit 251 may also decode the angle representing the decoding object vertex by entropy decoding. In addition, for example, the circuit 251 may also decode one, two or three angles used to determine the decoding object vertex.

[0631] Furthermore, for example, the first triangle may be a reference triangle that is set as a triangle closer to the decoding target vertex according to a reference rule.

[0632] In addition, for example, each of the above-mentioned vertices can also be expressed by geometric coordinates in a three-dimensional mesh.

[0633] Fig.64 17 is a block diagram showing still another configuration example of the encoding device 100 according to the present embodiment. In this example, the encoding device 100 includes a selector 171 and an encoding processor 172 .

[0634] The selector 171 is, for example, a circuit. The selector 171 may correspond to the above-mentioned traverser 161, the determiner 167, the exception handler 168, etc., or may be implemented using the above-mentioned circuit 151 and the memory 152.

[0635] The encoding processor 172 is, for example, a circuit. The encoding processor 172 may correspond to the angle derivation device 163 and the entropy encoder 166 described above, or may be implemented using the circuit 151 and the memory 152 described above.

[0636] For example, the selector 171 selects a mode from a first mode set including normal modes when the first triangle satisfies the condition, and selects a mode from a second mode set including exceptional modes when the first triangle does not satisfy the condition. The encoding processor 172 encodes the encoding target vertex according to the mode.

[0637] Thus, when the first triangle does not satisfy the condition, the vertices of the second triangle may be encoded using a third triangle different from the first triangle. Therefore, the vertices of the second triangle may be encoded using a reference triangle suitable for encoding.

[0638] Fig.65 2 is a block diagram showing still another configuration example of the decoding device 200 according to the present embodiment. In this example, the decoding device 200 includes a selector 271 and a decoding processor 272 .

[0639] The selector 271 is, for example, a circuit. The selector 271 may correspond to the above-mentioned traverser 261, the determiner 267, the exception handler 268, etc., or may be implemented using the above-mentioned circuit 251 and the memory 252.

[0640] The decoding processor 272 is, for example, a circuit. The decoding processor 272 may correspond to the coordinate derivation unit 264 and the entropy decoder 266 described above, or may be implemented using the circuit 251 and the memory 252 described above.

[0641] For example, the selector 271 selects a mode from a first mode set including a normal mode when the first triangle satisfies the condition, and selects a mode from a second mode set including an exception mode when the first triangle does not satisfy the condition. Then, the decoding processor 272 decodes the decoding target vertex according to the mode.

[0642] Thus, when the first triangle does not satisfy the condition, the vertices of the second triangle may be decoded using a third triangle different from the first triangle. Therefore, the vertices of the second triangle may be decoded using a reference triangle suitable for decoding.

[0643] <Other examples>

[0644] The above describes the forms of the encoding device 100 and the decoding device 200 according to the embodiments, but the forms of the encoding device 100 and the decoding device 200 are not limited to the embodiments. The embodiments may be modified as conceived by those skilled in the art, and multiple components of the embodiments may be arbitrarily combined.

[0645] For example, the processing executed by a specific component in the embodiment may be executed by another component instead of the specific component. In addition, the order of a plurality of processing may be changed, or a plurality of processing may be executed in parallel.

[0646] In addition, the encoding and decoding of the present disclosure can be applied to the encoding and decoding of vertex information representing the position of the vertex. In addition, the encoding and decoding of the present disclosure is not limited to the vertices of the faces in the three-dimensional mesh, and can also be applied to the encoding and decoding of vertex information representing the position of other vertices. In addition, each process of the present disclosure can also be performed as one of multiple processes that can be selected.

[0647] In addition, as described above, at least a portion of the multiple configurations of the present disclosure can be installed as an integrated circuit. At least a portion of the multiple processes of the present disclosure can also be used as an encoding method or a decoding method. A program for causing a computer to execute the encoding method or the decoding method can also be used. In addition, a non-temporary computer-readable recording medium having the program recorded thereon can also be used. In addition, a bit stream for causing the decoding device 200 to perform a decoding process can also be used.

[0648] In addition, at least a part of the multiple configurations and multiple processes of the present disclosure may be used as a sending device, a receiving device, a sending method, and a receiving method. A program for causing a computer to execute the sending method or the receiving method may also be used. In addition, a non-temporary computer-readable recording medium having the program recorded thereon may also be used.

[0649] Industrial Applicability

[0650] The present disclosure is useful, for example, for a coding device, a decoding device, a transmitting device, a receiving device, and the like regarding a three-dimensional mesh, and can be applied to a computer graphics system, a three-dimensional data display system, and the like.

[0651] Description of symbols

[0652] 100 Encoding device

[0653] 101, 121, 144 vertex information encoder

[0654] 102, 145 Connection information encoder

[0655] 103, 122 Attribute Information Encoder

[0656] 104, 204 Preprocessor

[0657] 105, 205 Post-processor

[0658] 110 Three-dimensional data encoding system

[0659] 111, 211 controller

[0660] 112, 212 Input and Output Processor

[0661] 113 3D Data Encoder

[0662] 114 System Multiplexer

[0663] 115 3D Data Generator

[0664] 123 Metadata Encoder

[0665] 124 Multiplexer

[0666] 131 Vertex Image Generator

[0667] 132 Attribute Image Generator

[0668] 133 Metadata Generator

[0669] 134 Image Encoder

[0670] 141 Two-dimensional data encoder

[0671] 142 Grid Data Encoder

[0672] 143 Texture Encoder

[0673] 148 Describe encoder

[0674] 151, 251 circuits

[0675] 152, 252 memory

[0676] 161, 261 Traverser

[0677] 162, 262 switch

[0678] 163 Angle Exporter

[0679] 164 Angle encoder

[0680] 165 Coordinate Encoder

[0681] 166 Entropy Encoder

[0682] 167, 267 Determinator

[0683] 168, 268 Exception Handler

[0684] 171, 271 selector

[0685] 172 Encoding Processor

[0686] 200 Decoding device

[0687] 201, 221, 244 Vertex information decoder

[0688] 202, 245 Connection Information Decoder

[0689] 203, 222 Attribute Information Decoder

[0690] 210 3D data decoding system

[0691] 213 3D Data Decoder

[0692] 214 System Demultiplexer

[0693] 215, 247 Prompt

[0694] 216 User Interface

[0695] 223 Metadata Decoder

[0696] 224 Demultiplexer

[0697] 231 Vertex Information Generator

[0698] 232 Attribute Information Generator

[0699] 234 Video Decoder

[0700] 241 2D Data Decoder

[0701] 242 Grid Data Decoder

[0702] 243 Texture Decoder

[0703] 246 Grid Reconstructor

[0704] 248 Describe decoder

[0705] 263 Angle Decoder

[0706] 264 Coordinate Exporter

[0707] 265 coordinate decoder

[0708] 266 Entropy Decoder

[0709] 272 Decoder Processor

[0710] 300 Network

[0711] 310 External connector

Claims

1. A coding device, wherein: have: Memory; and circuitry capable of accessing said memory, The above circuit is in action. When the first triangle satisfies the condition, a mode for encoding the encoding target vertex of the second triangle is selected from the first mode set, the first mode set including a normal mode for encoding the encoding target vertex using the first triangle, When the first triangle does not satisfy the above condition, the mode is selected from a second mode set, wherein the second mode set includes an exception mode for encoding the encoding object vertex using a third triangle different from both the first triangle and the second triangle. The above-mentioned encoding object vertex is encoded according to the above-mentioned mode.

2. The encoding device according to claim 1, wherein: The third triangle is a triangle that shares a side with the first triangle.

3. The encoding device according to claim 1 or 2, wherein: The third triangle is a triangle that shares a side with the second triangle.

4. The encoding device according to claim 1 or 2, wherein: The third triangle is a triangle that shares vertices with the first triangle.

5. The encoding device according to claim 1 or 2, wherein: The third triangle mentioned above is a triangle whose vertices have been encoded.

6. The encoding device according to claim 1 or 2, wherein: The third triangle mentioned above is a triangle whose sides are connected.

7. The encoding device according to claim 1 or 2, wherein: The third triangle mentioned above is a triangle having a pseudo vertex.

8. The encoding device according to claim 1 or 2, wherein: The above exception mode includes multiple sub-modes. The circuit further selects a sub-mode for encoding the encoding target vertex from among the plurality of sub-modes when the exception mode is selected as the mode.

9. The encoding device according to claim 8, wherein: The above circuit encodes the parameters representing the above sub-mode.

10. The encoding device according to claim 8, wherein: The above-mentioned circuit determines the above-mentioned third triangle according to the above-mentioned sub-mode.

11. The encoding device according to claim 8, wherein: The circuit determines the vertices of the third triangle according to the sub-pattern.

12. The encoding device according to claim 1 or 2, wherein: The above-mentioned condition includes that each vertex of the above-mentioned first triangle has been encoded.

13. The encoding device according to claim 1 or 2, wherein: The above conditions include that the sides of the above-mentioned first triangle are connected.

14. The encoding device according to claim 1 or 2, wherein: When encoding the encoding target vertex according to the exception mode, the circuit encodes the angle representing the encoding target vertex using the third triangle.

15. A decoding device, wherein: have: Memory; and circuitry capable of accessing said memory, The above circuit is in action. When the first triangle satisfies the condition, a mode for decoding the decoding target vertex of the second triangle is selected from a first mode set, wherein the first mode set includes a normal mode for decoding the decoding target vertex using the first triangle. When the first triangle does not satisfy the condition, the mode is selected from a second mode set, wherein the second mode set includes an exception mode for decoding the decoding target vertex using a third triangle different from both the first triangle and the second triangle. The above-mentioned decoding target vertex is decoded according to the above-mentioned mode.

16. The decoding device according to claim 15, wherein: The third triangle is a triangle that shares a side with the first triangle.

17. The decoding device according to claim 15 or 16, wherein: The third triangle is a triangle that shares a side with the second triangle.

18. The decoding device according to claim 15 or 16, wherein: The third triangle is a triangle that shares vertices with the first triangle.

19. The decoding device according to claim 15 or 16, wherein: The third triangle mentioned above is a triangle whose vertices have been decoded.

20. The decoding device according to claim 15 or 16, wherein: The third triangle mentioned above is a triangle whose sides are connected.

21. The decoding device according to claim 15 or 16, wherein: The third triangle mentioned above is a triangle having a pseudo vertex.

22. The decoding device according to claim 15 or 16, wherein: The above exception mode includes multiple sub-modes. The circuit further selects a sub-mode for decoding the decoding target vertex from among the plurality of sub-modes when the exception mode is selected as the mode.

23. The decoding device according to claim 22, wherein: The circuit decodes the parameters representing the sub-mode.

24. The decoding device according to claim 22, wherein: The above-mentioned circuit determines the above-mentioned third triangle according to the above-mentioned sub-mode.

25. The decoding device according to claim 22, wherein: The circuit determines the vertices of the third triangle according to the sub-pattern.

26. The decoding device according to claim 15 or 16, wherein: The above-mentioned condition includes that each vertex of the above-mentioned first triangle has been decoded.

27. The decoding device according to claim 15 or 16, wherein: The above conditions include that the sides of the above-mentioned first triangle are connected.

28. The decoding device according to claim 15 or 16, wherein: When the above-mentioned circuit decodes the above-mentioned decoding target vertex according to the above-mentioned exception mode, it decodes the angle indicating the above-mentioned decoding target vertex using the above-mentioned third triangle.

29. A coding method, wherein: include: When the first triangle satisfies the condition, a step of selecting a mode for encoding the encoding target vertex of the second triangle from a first mode set, wherein the first mode set includes a normal mode for encoding the encoding target vertex using the first triangle; In a case where the first triangle does not satisfy the condition, the step of selecting the mode from a second mode set, wherein the second mode set includes an exception mode for encoding the encoding target vertex using a third triangle different from both the first triangle and the second triangle; and The step of encoding the above-mentioned encoding object vertex according to the above-mentioned mode.

30. A decoding method, wherein: include: When the first triangle satisfies the condition, a step of selecting a mode for decoding the decoding target vertex of the second triangle from a first mode set, wherein the first mode set includes a normal mode for decoding the decoding target vertex using the first triangle; In a case where the first triangle does not satisfy the condition, the step of selecting the mode from a second mode set, wherein the second mode set includes an exception mode for decoding the decoding target vertex using a third triangle different from both the first triangle and the second triangle; and The step of decoding the above decoding object vertex according to the above mode.

Images