Encoding device, decoding device, encoding method, and decoding method

Abstract

An encoding device (100) comprises a memory (152) and a circuit (151) capable of accessing the memory (152). The circuit (151), in operation, encodes, for each of a plurality of faces constituting a three-dimensional mesh, a connection type relating to a connection relationship between the face to be processed and an unprocessed face, in the reverse order to a processing order in which the connection type is defined (S2201), and encodes related information for detecting a separation of connected components for a plurality of connection types, each of which is encoded in reverse order as a connection type (S2202). Each connected component corresponds to at least a portion of the three-dimensional mesh.

Description

Encoding device, decoding device, encoding method, and decoding method

[0001] The present disclosure relates to an encoding device and the like.

[0002] Patent Document 1 proposes a method and apparatus for encoding and decoding three-dimensional mesh data. Non-Patent Document 1 discloses a technique related to encoding and decoding three-dimensional mesh data.

[0003] Japanese Patent Application Laid-Open No. 2006-187015

[0004] Jarek Rossignac et al., "3D Compression Made Simple: Edgebreaker on a Corner-Table," [online], [retrieved January 27, 2024], <URL: https: / / www.cs.cmu.edu / ~alla / edgebreaker_simple.pdf>

[0005] Further improvements are desired in encoding processes etc. relating to three-dimensional data. An object of the present disclosure is to improve encoding processes etc. relating to three-dimensional data.

[0006] An encoding device according to one aspect of the present disclosure includes a memory and a circuit that can access the memory, and in operation, the circuit encodes, for each of a plurality of faces that constitute a three-dimensional mesh, a connection type relating to the connection relationship between the face to be processed and unprocessed faces in a reverse order to the processing order in which the connection type is defined, and encodes related information for detecting boundaries of connected components for the plurality of connection types that are each encoded in the reverse order as the connection type, and the connected components correspond to at least a portion of the three-dimensional mesh.

[0007] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0008] The present disclosure may contribute to improvements in encoding processes and the like related to three-dimensional data.

[0009] 1 is a conceptual diagram showing a three-dimensional mesh according to an embodiment. FIG. 2 is a conceptual diagram showing basic elements of a three-dimensional mesh according to an embodiment. FIG. 3 is a conceptual diagram showing mapping according to an embodiment. FIG. 4 is a block diagram showing a configuration example of an encoding / decoding system according to an embodiment. FIG. 5 is a block diagram showing a configuration example of an encoding device according to an embodiment. FIG. 6 is a block diagram showing another configuration example of an encoding device according to an embodiment. FIG. 7 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 8 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 9 is a block diagram showing another configuration example of a decoding device according to an embodiment. FIG. 10 is a conceptual diagram showing another configuration example of a bit stream according to an embodiment. FIG. 11 is a conceptual diagram showing yet another configuration example of a bit stream according to an embodiment. FIG. 12 is a block diagram showing a specific example of an encoding / decoding system according to an embodiment. FIG. 13 is a conceptual diagram showing an example configuration of point cloud data according to an embodiment. FIG. 14 is a conceptual diagram showing an example data file of point cloud data according to an embodiment. FIG. 15 is a conceptual diagram showing an example configuration of mesh data according to an embodiment. FIG. 16 is a conceptual diagram showing an example data file of mesh data according to an embodiment. FIG. 17 is a conceptual diagram showing types of three-dimensional data according to an embodiment. FIG. 18 is a block diagram showing an example configuration of a three-dimensional data encoder according to an embodiment. FIG. 19 is a block diagram showing an example configuration of a three-dimensional data decoder according to an embodiment. FIG. 19 is a block diagram showing another configuration example of a three-dimensional data encoder according to an embodiment. FIG. 19 is a block diagram showing another configuration example of a three-dimensional data decoder according to an embodiment. FIG. 1 is a conceptual diagram showing a specific example of encoding processing according to an embodiment. FIG. 2 is a conceptual diagram showing a specific example of decoding processing according to an embodiment. FIG. 3 is a block diagram showing an implementation example of an encoding device according to an embodiment. FIG. 4 is a block diagram showing an implementation example of a decoding device according to an embodiment. FIG. 5 is a block diagram showing another configuration example of an encoding / decoding system according to an embodiment. FIG. 6 is a block diagram showing yet another configuration example of an encoding device according to an embodiment. FIG. 7 is a block diagram showing yet another configuration example of a decoding device according to an embodiment. FIG. 8 is a conceptual diagram showing five types of connection types according to an embodiment. FIG. 9 is a conceptual diagram showing an example of sequentially determined connection types according to an embodiment. FIG. 10 is a flowchart showing examples of encoding processing and decoding processing of connection types according to an embodiment. FIG. 11 is a flowchart showing a first specific example of encoding of connection types according to an embodiment.1 is a flowchart showing a first specific example of connection type decoding according to an embodiment. FIG. 2 is a flowchart showing a second specific example of connection type encoding according to an embodiment. FIG. 3 is a flowchart showing a third specific example of connection type encoding according to an embodiment. FIG. 4 is a flowchart showing a fourth specific example of connection type encoding according to an embodiment. FIG. 5 is a flowchart showing a fourth specific example of connection type decoding according to an embodiment. FIG. 6 is a flowchart showing a fourth specific example of connection type decoding according to an embodiment. FIG. 7 is a flowchart showing a fourth specific example of connection type encoding according to an embodiment. FIG. 8 is a flowchart showing a fourth specific example of connection type decoding according to an embodiment. FIG. 9 is a flowchart showing another example of a correspondence relationship between a second previous connection type and a value used to determine a context according to an embodiment. FIG. 10 is a flowchart showing yet another example of connection type encoding and decoding according to an embodiment. FIG. 11 is a diagram showing a correspondence relationship between at least one of a second previous connection type and a consecutive number of type C according to an embodiment, and a value used to determine a context. FIG. 12 is a flowchart showing a fifth specific example of connection type encoding according to an embodiment. FIG. 13 is a flowchart showing a fifth specific example of connection type decoding according to an embodiment. FIG. 14 is a conceptual diagram showing an example of transition of stack occupation amount according to an embodiment. FIG. 1 is a syntax diagram showing a first syntax example of a symbol sequence including connected components according to an embodiment. FIG. 2 is a flowchart showing a first decoding example of a symbol sequence including connected components according to an embodiment. FIG. 3 is a flowchart showing a process of restoring a mesh structure according to symbols according to an embodiment. FIG. 4 is a conceptual diagram showing a process of restoring a mesh structure according to type C symbols according to an embodiment. FIG. 5 is a conceptual diagram showing a process of restoring a mesh structure according to type L symbols according to an embodiment. FIG. 6 is a conceptual diagram showing a process of restoring a mesh structure according to type S symbols according to an embodiment. FIG. 7 is a conceptual diagram showing a process of restoring a mesh structure according to type R symbols according to an embodiment.FIG. 1 is a conceptual diagram showing a process of restoring a mesh structure according to type E symbols according to an embodiment. FIG. 2 is a syntax diagram showing a second syntax example of a symbol sequence including connected components according to an embodiment. FIG. 3 is a flowchart showing a second decoding example of a symbol sequence including connected components according to an embodiment. FIG. 4 is a flowchart showing a sixth specific example of connection-type encoding according to an embodiment. FIG. 5 is a flowchart showing a sixth specific example of connection-type decoding according to an embodiment. FIG. 6 is a flowchart showing an example of basic encoding processing according to an embodiment. FIG. 7 is a flowchart showing an example of basic decoding processing according to an embodiment.

[0010] Introduction Three-dimensional (3D) meshes are used in computer graphics images, for example. For example, a computer graphics image may be composed of multiple temporally distinct frames, and each frame may be represented by a 3D mesh.

[0011] A 3D mesh is composed of vertex information indicating the positions of each of the vertices in 3D space, connectivity information indicating the connections between the vertices, and attribute information indicating the attributes of each vertex or face. Each face is constructed according to the connectivity between the vertices. Various computer graphics images can be expressed using such 3D meshes.

[0012] Furthermore, the three-dimensional mesh is coded for transmission and storage. In coding the three-dimensional mesh, a connection type relating to the connection relationship between the processing target surface and unprocessed surfaces may be coded for each of the multiple surfaces constituting the three-dimensional mesh. The connection type of each surface is effective as information for expressing the relationship with other surfaces, and is also effective as information for sequentially processing multiple surfaces.

[0013] However, a 3D mesh may be divided into multiple connected components. In such a case, if the boundaries of the connected components cannot be properly detected for multiple connection types of the faces in the 3D mesh, it may be difficult to efficiently identify the connectivity of the faces. Furthermore, if the connectivity of the faces cannot be efficiently identified, processing delays may increase.

[0014] Therefore, the encoding device of Example 1 includes a memory and a circuit that can access the memory, and in operation, the circuit encodes, for each of a plurality of faces that constitute a three-dimensional mesh, a connection type relating to the connection relationship between the face to be processed and unprocessed faces in a reverse order to the processing order in which the connection type is defined, and encodes related information for detecting boundaries of connected components for the plurality of connection types that are each encoded in the reverse order as the connection type, and the connected components correspond to at least a portion of the three-dimensional mesh.

[0015] This may enable proper detection of boundaries between connected components in multiple connection types coded in reverse order. Therefore, it may be possible to efficiently identify connected components and efficiently identify the connectivity between multiple faces. Therefore, it may be possible to reduce processing delays in the reconstruction of 3D meshes.

[0016] In addition, the multiple examples of connection types regarding the connection relationship between the surface to be processed and the unprocessed surface may include a type in which the surface to be processed is not connected to the unprocessed surface (e.g., type E described below).

[0017] Furthermore, the encoding device of Example 2 may be the encoding device of Example 1, in which the related information indicates whether the current connection type, which is the connection type to be encoded, corresponds to the end of the connected component in the processing order.

[0018] This may enable the connection type corresponding to the end of a connected component to be properly detected among multiple connection types coded in reverse order, thereby enabling efficient identification of connected components and efficient identification of the connectivity relationships between multiple faces.

[0019] The encoding device of Example 3 may be the encoding device of Example 2, wherein the circuitry determines whether a condition is satisfied that the current connection type has a possibility of corresponding to the end, and encodes the associated information when it is determined that the condition is satisfied.

[0020] This may make it possible to omit encoding of related information when the current connection type is unlikely to correspond to the end of a connected component, thereby suppressing an increase in the amount of code.

[0021] Furthermore, the encoding device of Example 4 may be the encoding device of Example 3, in which the condition includes that the current connection type is a termination type, and the termination type is a type in which no unprocessed faces are connected to the left or right of the face to be processed along the path leading to the face to be processed in the processing order.

[0022] This may make it possible to omit encoding of related information when the current connection type is not the termination type, thereby making it possible to suppress an increase in the amount of code.

[0023] The encoding device of Example 5 may be the encoding device of Example 3 or 4, in which the condition includes that a stack to be pushed or popped according to the connection type is empty.

[0024] This may make it possible to omit encoding of related information when the stack is not empty, thereby making it possible to suppress an increase in the amount of code.

[0025] Furthermore, the encoding device of Example 6 may be any of the encoding devices of Examples 3 to 5, in which the condition includes that the connection type of the last face of at least one connected component among one or more connected components each defined as the connected component is not encoded.

[0026] This may allow for omitting the coding of related information if the connection type of the last face of each connected component has already been coded, thereby reducing the amount of coding.

[0027] Also, the encoding device of Example 7 may be the encoding device of Example 1, in which the related information indicates a connection type number, which is the number of one or more connection types to be encoded for each of one or more connected components each defined as the connected component.

[0028] This may enable efficient identification of each connected component according to the number of connection types, and efficient identification of the connectivity between multiple faces, thereby reducing processing delays in the reconstruction of a 3D mesh.

[0029] Also, the encoding device of Example 8 may be the encoding device of Example 7, in which the division is indicated according to the number of connection types indicated by the related information and the number of one or more connection types that have been encoded.

[0030] This may allow appropriate indication of the division of connected components in the multiple connection types coded in reverse order according to the relationship between the number of connection types indicated by the related information and the number of one or more encoded connection types, thereby allowing efficient identification of each connected component and efficient identification of the connection relationships of multiple faces.

[0031] In addition, the decoding device of Example 9 includes a memory and a circuit that can access the memory, and in operation, the circuit decodes, for each of a plurality of faces that constitute a three-dimensional mesh, a connection type relating to the connection relationship between the face to be processed and unprocessed faces in a reverse order to the processing order in which the connection type is defined, and decodes related information for detecting boundaries of connected components for the plurality of connection types that are each decoded in the reverse order as the connection type, and the connected components correspond to at least a portion of the three-dimensional mesh.

[0032] This may enable proper detection of boundaries between connected components in multiple connection types decoded in reverse order. Therefore, it may be possible to efficiently identify connected components and efficiently identify the connectivity between multiple faces. Therefore, it may be possible to reduce processing delays in reconstructing a 3D mesh.

[0033] In addition, the multiple examples of connection types regarding the connection relationship between the surface to be processed and the unprocessed surface may include a type in which the surface to be processed is not connected to the unprocessed surface (e.g., type E described below).

[0034] Furthermore, the decoding device of Example 10 may be the decoding device of Example 9, in which the related information indicates whether the current connection type, which is the connection type to be decoded, corresponds to the end of the connected component in the processing order.

[0035] This may enable the connection type corresponding to the end of a connected component to be properly detected among multiple connection types decoded in reverse order, thereby enabling efficient identification of connected components and efficient identification of the connectivity relationships between multiple faces.

[0036] Also, the decoding device of Example 11 may be the decoding device of Example 10, in which the circuit determines whether a condition that the current connection type has a possibility of corresponding to the end is satisfied, and if it is determined that the condition is satisfied, decodes the related information.

[0037] This may make it possible to omit decoding of related information when the current connection type is unlikely to correspond to the end of a connected component, thereby suppressing an increase in the amount of code.

[0038] Furthermore, the decoding device of Example 12 may be the decoding device of Example 11, wherein the condition includes that the current connection type is a terminal type, and the terminal type is a type in which no unprocessed faces are connected to the left or right of the face to be processed along the path leading to the face to be processed in the processing order.

[0039] This may make it possible to omit decoding of related information when the current connection type is not the termination type, thereby making it possible to suppress an increase in the amount of code.

[0040] Furthermore, the decoding device of Example 13 may be the decoding device of Example 11 or 12, in which the condition includes that a stack to be pushed or popped according to the connection type is empty.

[0041] This may make it possible to omit decoding of the related information when the stack is not empty, thereby making it possible to suppress an increase in the amount of code.

[0042] Furthermore, the decoding device of Example 14 may be any of the decoding devices of Examples 11 to 13, in which the condition includes that the connection type of the last face of at least one connected component among one or more connected components each defined as the connected component is not decoded.

[0043] This may allow for omitting the decoding of related information if the connection type of the last face of each connected component has already been decoded, thereby reducing the amount of coding.

[0044] Furthermore, the decoding device of Example 15 may be the decoding device of Example 9, wherein the related information indicates a connection type number, which is the number of one or more connection types to be decoded for each of one or more connected components each defined as the connected component.

[0045] This may enable efficient identification of each connected component according to the number of connection types, and efficient identification of the connectivity between multiple faces, thereby reducing processing delays in the reconstruction of a 3D mesh.

[0046] Also, the decoding device of Example 16 may be the decoding device of Example 15, in which the division is indicated according to the number of connection types indicated by the related information and the number of one or more connection types that have been decoded.

[0047] This may allow appropriate indication of the division of connected components in the multiple connection types decoded in reverse order according to the relationship between the number of connection types indicated by the related information and the number of one or more decoded connection types, thereby allowing efficient identification of each connected component and efficient identification of the connection relationships of multiple faces.

[0048] In addition, the encoding method of Example 17 encodes, for each of a plurality of faces constituting a three-dimensional mesh, a connection type relating to the connection relationship between the face being processed and unprocessed faces in a reverse order to the processing order in which the connection type is defined, and encodes related information for detecting boundaries of connected components for each of the plurality of connection types encoded in the reverse order as the connection type, and the connected components correspond to at least a portion of the three-dimensional mesh.

[0049] This may enable proper detection of boundaries between connected components in multiple connection types coded in reverse order. Therefore, it may be possible to efficiently identify each connected component and efficiently identify the connectivity between multiple faces. Therefore, it may be possible to reduce processing delays in the reconstruction of a 3D mesh.

[0050] In addition, the decoding method of Example 18 decodes, for each of the multiple faces that make up a three-dimensional mesh, a connection type relating to the connection relationship between the face being processed and unprocessed faces in the reverse order of the processing order in which the connection type is defined, and decodes related information for detecting boundaries of connected components for the multiple connection types that are each decoded in the reverse order as the connection type, and the connected components correspond to at least a portion of the three-dimensional mesh.

[0051] This may enable proper detection of boundaries between connected components in multiple connection types decoded in reverse order. Therefore, it may be possible to efficiently identify each connected component and efficiently identify the connectivity between multiple faces. Therefore, it may be possible to reduce processing delays in the reconstruction of a 3D mesh.

[0052] Furthermore, these comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0053] <Expressions and Terms> The following expressions and terms are used herein.

[0054] (1) Three-dimensional Mesh A three-dimensional mesh is a collection of multiple faces, and represents, for example, a three-dimensional object. A three-dimensional mesh is mainly composed of vertex information, connectivity information, and attribute information. A three-dimensional mesh may be expressed as a polygon mesh or a mesh. A three-dimensional mesh may also vary over time. A three-dimensional mesh may include metadata related to the vertex information, connectivity information, and attribute information, and may also include other additional information.

[0055] (2) Vertex Information Vertex information is information indicating a vertex. For example, the vertex information indicates the position of a vertex in a three-dimensional space. Furthermore, a vertex corresponds to a vertex of a face that constitutes a three-dimensional mesh. Vertex information may be expressed as "geometry." Furthermore, vertex information may be expressed as position information.

[0056] (3) Connection Information Connection information is information that indicates connections between vertices. For example, connection information indicates connections for forming faces or edges of a three-dimensional mesh. Connection information may be expressed as "Connectivity." Connection information may also be expressed as face information.

[0057] (4) Attribute Information Attribute information is information that indicates attributes of a vertex or a face. For example, attribute information indicates attributes such as a color, an image, and a normal vector associated with a vertex or a face. Attribute information may be expressed as "texture."

[0058] (5) Faces A face is an element that makes up a three-dimensional mesh. Specifically, a face is a polygon on a plane in three-dimensional space. For example, a face can be defined as a triangle in three-dimensional space.

[0059] (6) Plane A plane is a two-dimensional plane in a three-dimensional space. For example, a polygon is formed on a plane, and multiple polygons are formed on multiple planes.

[0060] (7) Bitstream: A bitstream corresponds to coded information. A bitstream may also be referred to as a stream, a coded bitstream, a compressed bitstream, or a coded signal.

[0061] (8) Encoding and Decoding The term encoding may be substituted with terms such as storing, including, writing, describing, signaling, sending, notifying, saving, or compressing, and these terms may be interchangeable. For example, encoding information may mean including the information in a bitstream. Also, encoding information into a bitstream may mean encoding the information to generate a bitstream that includes the encoded information.

[0062] Additionally, the term "decode" may be replaced with terms such as "read," "decode," "read," "load," "derive," "obtain," "receive," "extract," "reconstruct," "reconstruct," "decompress," or "decompress," and these terms may be interchangeable. For example, decoding information may mean obtaining information from a bitstream. Decoding information from a bitstream may mean decoding the bitstream to obtain information contained in the bitstream.

[0063] (9) Ordinal Numbers In the description, ordinal numbers such as first and second may be assigned to components, etc. These ordinal numbers may be changed as appropriate. Furthermore, new ordinal numbers may be assigned to components, etc., or removed. Furthermore, these ordinal numbers may be assigned to elements in order to identify them, and may not correspond to a meaningful order.

[0064] <Three-dimensional mesh> Fig. 1 is a conceptual diagram showing a three-dimensional mesh according to this embodiment. A three-dimensional mesh is composed of multiple faces. For example, each face is a triangle. The vertices of these triangles are defined in three-dimensional space. The three-dimensional mesh then represents a three-dimensional object. Each face may have a color or an image.

[0065] FIG. 2 is a conceptual diagram showing the basic elements of a three-dimensional mesh according to this embodiment. A three-dimensional mesh is composed of vertex information, connection information, and attribute information. The vertex information indicates the positions of the vertices of a face in three-dimensional space. The connection information indicates the connections between the vertices. A face can be identified by the vertex information and connection information. In other words, a colorless three-dimensional object is formed in three-dimensional space by the vertex information and connection information.

[0066] The attribute information may be associated with a vertex or a face. The attribute information associated with a vertex may be expressed as "Attribute Per Point." The attribute information associated with a vertex may indicate an attribute of the vertex itself, or may indicate an attribute of a face connected to the vertex.

[0067] For example, a color may be associated with a vertex as attribute information. The color associated with a vertex may be the color of the vertex itself, or the color of a face connected to the vertex. The color of a face may be the average of multiple colors associated with multiple vertices of the face. Furthermore, a normal vector may be associated with a vertex or a face as attribute information. Such a normal vector can represent the front and back of a face.

[0068] A two-dimensional image may be associated with a surface as attribute information. The two-dimensional image associated with a surface is also expressed as a texture image or an "Attribute Map." Information indicating mapping between the surface and the two-dimensional image may be associated with the surface as attribute information. Such information indicating mapping may be expressed as mapping information, vertex information of a texture image, or "Attribute UV Coordinate."

[0069] Furthermore, information such as color, image, and moving image used as attribute information may be expressed as "parametric space."

[0070] The attribute information allows texture to be reflected on the three-dimensional object. That is, a three-dimensional object having color is formed in three-dimensional space based on the vertex information, connection information, and attribute information.

[0071] In the above, the attribute information is associated with the vertices or faces, but it may also be associated with the edges.

[0072] 3 is a conceptual diagram illustrating mapping according to this embodiment. For example, a region of a two-dimensional image on a two-dimensional plane can be mapped onto a surface of a three-dimensional mesh in three-dimensional space. Specifically, coordinate information of the region in the two-dimensional image is associated with the surface of the three-dimensional mesh. As a result, an image of the mapped region in the two-dimensional image is reflected on the surface of the three-dimensional mesh.

[0073] By using the mapping, the 2D image used as attribute information can be separated from the 3D mesh. For example, in encoding the 3D mesh, the 2D image may be encoded by an image encoding method or a video encoding method.

[0074] <System Configuration> Fig. 4 is a block diagram showing an example of the configuration of a coding / decoding system according to this embodiment. In Fig. 4, the coding / decoding system includes a coding device 100 and a decoding device 200.

[0075] For example, the encoding device 100 obtains a three-dimensional mesh and encodes the three-dimensional mesh into a bitstream. Then, the encoding device 100 outputs the bitstream to the network 300. For example, the bitstream includes the encoded three-dimensional mesh and control information for decoding the encoded three-dimensional mesh. By encoding the three-dimensional mesh, information about the three-dimensional mesh is compressed.

[0076] The network 300 transmits a bitstream from the encoding device 100 to the decoding device 200. The network 300 may be the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 300 is not necessarily limited to bidirectional communication, and may be a unidirectional communication network for terrestrial digital broadcasting, satellite broadcasting, or the like.

[0077] Furthermore, the network 300 can be replaced by a recording medium such as a DVD (Digital Versatile Disc) or a BD (Blu-Ray Disc (registered trademark)).

[0078] The decoding device 200 obtains a bitstream and decodes a three-dimensional mesh from the bitstream. By decoding the three-dimensional mesh, information about the three-dimensional mesh is expanded. For example, the decoding device 200 decodes the three-dimensional mesh according to a decoding method corresponding to the encoding method used by the encoding device 100 to encode the three-dimensional mesh. That is, the encoding device 100 and the decoding device 200 perform encoding and decoding according to encoding methods and decoding methods that correspond to each other.

[0079] The 3D mesh before encoding may also be referred to as an original 3D mesh, and the 3D mesh after decoding may also be referred to as a reconstructed 3D mesh.

[0080] 5 is a block diagram showing an example of the configuration of a coding device 100 according to this embodiment. For example, the coding device 100 includes a vertex information encoder 101, a connection information encoder 102, and an attribute information encoder 103.

[0081] The vertex information encoder 101 is an electrical circuit that encodes vertex information. For example, the vertex information encoder 101 encodes the vertex information into a bitstream according to a format defined for the vertex information.

[0082] The connection information encoder 102 is an electrical circuit that encodes the connection information, for example, the connection information encoder 102 encodes the connection information into a bitstream according to a format defined for the connection information.

[0083] The attribute information encoder 103 is an electric circuit that encodes the attribute information. For example, the attribute information encoder 103 encodes the attribute information into a bit stream in accordance with a format defined for the attribute information.

[0084] The vertex information, connectivity information, and attribute information may be coded using variable-length coding or fixed-length coding, such as Huffman coding or context-adaptive binary arithmetic coding (CABAC).

[0085] The vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be integrated together, or each of the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be further subdivided into multiple components.

[0086] 6 is a block diagram showing another example of the configuration of the encoding device 100 according to this embodiment. For example, the encoding device 100 includes a pre-processor 104 and a post-processor 105 in addition to the configuration shown in FIG.

[0087] The preprocessor 104 is an electrical circuit that performs processing before encoding the vertex information, connectivity information, and attribute information. For example, the preprocessor 104 may perform a conversion process, a separation process, a multiplexing process, or the like on the 3D mesh before encoding. More specifically, for example, the preprocessor 104 may separate the vertex information, connectivity information, and attribute information from the 3D mesh before encoding.

[0088] The post-processor 105 is an electrical circuit that performs processing after the vertex information, connection information, and attribute information are encoded. For example, the post-processor 105 may perform conversion processing, separation processing, multiplexing processing, or the like on the encoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 105 may multiplex the encoded vertex information, connection information, and attribute information into a bitstream. Furthermore, for example, the post-processor 105 may further perform variable-length coding on the encoded vertex information, connection information, and attribute information.

[0089] 7 is a block diagram showing an example of the configuration of a decoding device 200 according to this embodiment. For example, the decoding device 200 includes a vertex information decoder 201, a connection information decoder 202, and an attribute information decoder 203.

[0090] The vertex information decoder 201 is an electrical circuit that decodes vertex information. For example, the vertex information decoder 201 decodes vertex information from a bitstream according to a format defined for the vertex information.

[0091] The connection information decoder 202 is an electrical circuit that decodes the connection information, for example, the connection information decoder 202 decodes the connection information from the bitstream according to a format defined for the connection information.

[0092] The attribute information decoder 203 is an electric circuit that decodes the attribute information. For example, the attribute information decoder 203 decodes the attribute information from the bitstream in accordance with a format defined for the attribute information.

[0093] The vertex information, connection information, and attribute information may be decoded using variable length decoding or fixed length decoding, which may correspond to Huffman coding, context-adaptive binary arithmetic coding (CABAC), or the like.

[0094] The vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be integrated together, or each of the vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be further subdivided into multiple components.

[0095] 8 is a block diagram showing another example of the configuration of the decoding device 200 according to this embodiment. For example, the decoding device 200 includes a pre-processor 204 and a post-processor 205 in addition to the configuration shown in FIG.

[0096] The preprocessor 204 is an electrical circuit that performs processing before decoding the vertex information, connection information, and attribute information. For example, the preprocessor 204 may perform conversion processing, separation processing, multiplexing processing, or the like on the bitstream before decoding the vertex information, connection information, and attribute information.

[0097] More specifically, for example, the preprocessor 204 may separate a sub-bitstream corresponding to vertex information, a sub-bitstream corresponding to connectivity information, and a sub-bitstream corresponding to attribute information from the bitstream. Also, for example, the preprocessor 204 may perform variable-length decoding on the bitstream in advance before decoding the vertex information, connectivity information, and attribute information.

[0098] The post-processor 205 is an electrical circuit that performs processing after the vertex information, connection information, and attribute information are decoded. For example, the post-processor 205 may perform conversion processing, separation processing, multiplexing processing, or the like on the decoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 205 may multiplex the decoded vertex information, connection information, and attribute information onto a three-dimensional mesh.

[0099] <Bitstream> Vertex information, connection information, and attribute information are coded and stored in a bitstream. The relationship between this information and the bitstream is shown below.

[0100] 9 is a conceptual diagram showing an example of the configuration of a bitstream according to this embodiment. In this example, connection information, vertex information, and attribute information are integrated in the bitstream. For example, the connection information, vertex information, and attribute information may be included in a single file.

[0101] Furthermore, multiple portions of this information may be stored sequentially, such as a first portion of connection information, a first portion of vertex information, a first portion of attribute information, a second portion of connection information, a second portion of vertex information, a second portion of attribute information, etc. These multiple portions may correspond to multiple portions that are different in time, multiple portions that are different in space, or multiple different faces.

[0102] Furthermore, the storage order of the connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0103] 10 is a conceptual diagram showing another example of the configuration of a bitstream according to this embodiment. In this example, a plurality of files are included in the bitstream, and connection information, vertex information, and attribute information are stored in different files. Here, a file containing connection information, a file containing vertex information, and a file containing attribute information are shown, but the storage format is not limited to this example. For example, two types of information among the connection information, vertex information, and attribute information may be included in one file, and the remaining type of information may be included in another file.

[0104] Alternatively, the information may be split and stored in more files. For example, multiple pieces of connectivity information may be stored in multiple files, multiple pieces of vertex information may be stored in multiple files, or multiple pieces of attribute information may be stored in multiple files. These multiple pieces may correspond to multiple temporally different pieces, multiple spatially different pieces, or multiple different faces.

[0105] Furthermore, the storage order of the connection information, vertex information, and attribute information is not limited to the above example, and a storage order different from the above example may be used.

[0106] 11 is a conceptual diagram showing another example of the configuration of a bitstream according to this embodiment. In this example, the bitstream is composed of multiple separable sub-bitstreams, and connection information, vertex information, and attribute information are stored in different sub-bitstreams.

[0107] Here, a sub-bitstream containing connection information, a sub-bitstream containing vertex information, and a sub-bitstream containing attribute information are shown, but the storage format is not limited to this example.

[0108] For example, two types of information among the connection information, vertex information, and attribute information may be included in one sub-bitstream, and the remaining type of information may be included in another sub-bitstream. Specifically, attribute information of a two-dimensional image or the like may be stored in a sub-bitstream that complies with an image coding method, separate from the sub-bitstreams of the connection information and vertex information.

[0109] Also, each sub-bitstream may include multiple files, and multiple pieces of connectivity information may be stored in multiple files, multiple pieces of vertex information may be stored in multiple files, or multiple pieces of attribute information may be stored in multiple files.

[0110] 9, 10, and 11, and a storage order different from the above examples may be used. For example, the vertex information, connection information, and attribute information may be stored in the bitstream in this order. Alternatively, the connection information, connection information, and attribute information may be stored in the bitstream in any of the following orders: connection information, attribute information, and vertex information; vertex information, attribute information, and connection information; attribute information, connection information, and vertex information; or attribute information, vertex information, and connection information.

[0111] Furthermore, each of the connection information, vertex information, and attribute information may be divided into a plurality of data, and the plurality of data may be stored in a cyclical or random order within the bitstream.

[0112] 12 is a block diagram showing a specific example of an encoding / decoding system according to this embodiment. In FIG. 12, the encoding / decoding system includes a three-dimensional data encoding system 110, a three-dimensional data decoding system 210, and an external connector 310.

[0113] The three-dimensional data encoding system 110 includes a controller 111, an input / output processor 112, a three-dimensional data encoder 113, a three-dimensional data generator 115, and a system multiplexer 114. The three-dimensional data decoding system 210 includes a controller 211, an input / output processor 212, a three-dimensional data decoder 213, a system demultiplexer 214, a presenter 215, and a user interface 216.

[0114] In the three-dimensional data encoding system 110, sensor data is input from a sensor terminal to a three-dimensional data generator 115. The three-dimensional data generator 115 generates three-dimensional data, such as point cloud data or mesh data, from the sensor data and inputs it to a three-dimensional data encoder 113.

[0115] For example, the three-dimensional data generator 115 generates vertex information, and generates connection information and attribute information corresponding to the vertex information. The three-dimensional data generator 115 may process the vertex information when generating the connection information and attribute information. For example, the three-dimensional data generator 115 may reduce the amount of data by deleting duplicate vertices, or may transform the vertex information (such as by shifting its position, rotating it, or normalizing it). The three-dimensional data generator 115 may also render the attribute information.

[0116] Furthermore, although the three-dimensional data generator 115 is a component of the three-dimensional data encoding system 110 in FIG. 12, it may be arranged externally and independently of the three-dimensional data encoding system 110.

[0117] The sensor terminal that provides the sensor data for generating the three-dimensional data may be, for example, a moving body such as an automobile, a flying object such as an airplane, a mobile terminal, a camera, etc. Furthermore, a distance sensor such as a LIDAR, a millimeter wave radar, an infrared sensor, or a range finder, a stereo camera, or a combination of multiple monocular cameras may also be used as the sensor terminal.

[0118] The sensor data may be the distance (position) of the object, monocular camera images, stereo camera images, color, reflectance, sensor attitude, orientation, gyro, sensing position (GPS information or altitude), speed, acceleration, sensing time, temperature, air pressure, humidity, or magnetism.

[0119] The three-dimensional data encoder 113 corresponds to the encoding device 100 shown in FIG. 5 and other figures. For example, the three-dimensional data encoder 113 encodes three-dimensional data to generate encoded data. The three-dimensional data encoder 113 also generates control information when encoding the three-dimensional data. The three-dimensional data encoder 113 then inputs the encoded data together with the control information to the system multiplexer 114.

[0120] The encoding method for the three-dimensional data may be an encoding method using geometry or an encoding method using a video codec. Here, the encoding method using geometry may also be referred to as a geometry-based encoding method. The encoding method using a video codec may also be referred to as a video-based encoding method.

[0121] The system multiplexer 114 multiplexes the encoded data and control information input from the 3D data encoder 113 to generate multiplexed data using a specified multiplexing method. The system multiplexer 114 may multiplex other media such as video, audio, subtitles, application data, or document files, or reference time information, along with the encoded data and control information of the 3D data. Furthermore, the system multiplexer 114 may multiplex attribute information related to the sensor data or the 3D data.

[0122] For example, the multiplexed data may have a file format for storage or a packet format for transmission. As these formats, ISOBMFF or a format based on ISOBMFF may be used. Also, MPEG-DASH, MMT, MPEG-2 TS Systems, RTP, or the like may be used.

[0123] The multiplexed data is then output as a transmission signal to the external connector 310 by the input / output processor 112. The multiplexed data may be transmitted as a transmission signal by wire or wirelessly. Alternatively, the multiplexed data is stored in an internal memory or a storage device. The multiplexed data may be transmitted to a cloud server via the Internet or may be stored in an external storage device.

[0124] For example, the transmission or storage of the multiplexed data is performed by a method according to the medium for transmission or storage, such as broadcasting or communication. The communication protocol may be http, ftp, TCP, UDP, IP, or a combination thereof. Furthermore, a pull-type communication method or a push-type communication method may be used.

[0125] For wired transmission, Ethernet (registered trademark), USB, RS-232C, HDMI (registered trademark), coaxial cable, etc. may be used. For wireless transmission, 3GPP (registered trademark), 3G / 4G / 5G defined by IEEE, wireless LAN, Wi-Fi, Bluetooth, or millimeter wave may be used. For broadcasting, for example, DVB-T2, DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3 may be used.

[0126] The sensor data may be input to the three-dimensional data generator 115 or the system multiplexer 114. The three-dimensional data or encoded data may be output as a transmission signal directly to the external connector 310 via the input / output processor 112. The transmission signal output from the three-dimensional data encoding system 110 is input to the three-dimensional data decoding system 210 via the external connector 310.

[0127] Furthermore, each operation of the three-dimensional data encoding system 110 may be controlled by a controller 111 that executes an application program.

[0128] In the three-dimensional data decoding system 210, a transmission signal is input to an input / output processor 212. The input / output processor 212 decodes multiplexed data having a file format or a packet format from the transmission signal and inputs the multiplexed data to a system demultiplexer 214. The system demultiplexer 214 obtains coded data and control information from the multiplexed data and inputs them to a three-dimensional data decoder 213. The system demultiplexer 214 may extract other media or reference time information from the multiplexed data.

[0129] The three-dimensional data decoder 213 corresponds to the decoding device 200 shown in Fig. 7 etc. For example, the three-dimensional data decoder 213 decodes three-dimensional data from the encoded data based on a predefined encoding method. The three-dimensional data is then presented to the user by the presenter 215.

[0130] Additionally, additional information such as sensor data may be input to the presenter 215. The presenter 215 may present three-dimensional data based on the additional information. Additionally, a user instruction may be input from a user terminal to the user interface 216. Then, the presenter 215 may present three-dimensional data based on the input instruction.

[0131] The input / output processor 212 may acquire the three-dimensional data and the encoded data from the external connector 310 .

[0132] Furthermore, each operation of the three-dimensional data decoding system 210 may be controlled by a controller 211 that executes an application program.

[0133] 13 is a conceptual diagram showing an example of the configuration of point cloud data according to this embodiment. The point cloud data is data of a group of points representing a three-dimensional object.

[0134] Specifically, a point cloud is made up of a plurality of points, and has position information indicating the three-dimensional coordinate position of each point and attribute information indicating the attribute of each point. The position information is also expressed as geometry.

[0135] The type of attribute information may be, for example, color, reflectance, etc. One point may be associated with attribute information of one type, one point may be associated with attribute information of multiple different types, or one point may be associated with attribute information having multiple values ​​for the same type.

[0136] 14 is a conceptual diagram showing an example of a data file of point cloud data according to this embodiment. This example shows a case where there is a one-to-one correspondence between position information items and attribute information items, and shows position information and attribute information for N points that make up the point cloud data. In this example, the position information is information indicating a three-dimensional coordinate position using three axes, x, y, and z, and the attribute information is information indicating a color using RGB. A PLY file or the like can be used as a representative data file for point cloud data.

[0137] 15 is a conceptual diagram showing an example of the configuration of mesh data according to this embodiment. Mesh data is data used in CG (Computer Graphics) and the like, and is three-dimensional mesh data that shows the three-dimensional shape of an object using multiple surfaces. Each surface is also expressed as a polygon, and has a polygonal shape such as a triangle or a rectangle.

[0138] Specifically, a 3D mesh is composed of a plurality of points constituting a point cloud, as well as a plurality of edges and a plurality of faces. Each point is also expressed as a vertex or a position. Each edge corresponds to a line segment connected by two vertices. Each face corresponds to an area surrounded by three or more edges.

[0139] Furthermore, a three-dimensional mesh has position information indicating the three-dimensional coordinate positions of vertices. The position information is also expressed as vertex information or geometry. A three-dimensional mesh also has connection information indicating the relationship between multiple vertices that make up an edge or a face. The connection information is also expressed as connectivity. A three-dimensional mesh also has attribute information indicating the attributes of the vertices, edges, or faces. The attribute information in a three-dimensional mesh is also expressed as texture.

[0140] For example, the attribute information may indicate the color, reflectance, or normal vector for a vertex, edge, or face. The direction of the normal vector may represent the front and back of the face.

[0141] The mesh data may be stored in a data file format such as an object file.

[0142] 16 is a conceptual diagram showing an example of a data file of mesh data according to this embodiment. In this example, the data file includes position information G(1) to G(N) of N vertices that make up the three-dimensional mesh, and attribute information A1(1) to A1(N) of the N vertices. Also, in this example, M pieces of attribute information A2(1) to A2(M) are included. The attribute information items do not need to correspond one-to-one to vertices or faces. Furthermore, attribute information need not exist.

[0143] The connection information is represented by a combination of vertex indices. n[1, 3, 4] indicates a triangular face formed by three vertices, n=1, n=3, and n=4. Also, m[2, 4, 6] indicates that the attribute information of m=2, m=4, and m=6 corresponds to the three vertices, respectively.

[0144] Furthermore, the actual contents of the attribute information may be written in a separate file. A pointer to that content may be associated with a vertex, a face, or the like. For example, attribute information indicating an image for a face may be stored in a two-dimensional attribute map file. The file name of the attribute map and two-dimensional coordinate values ​​in the attribute map may be written in attribute information A2(1) to A2(M). The method of specifying attribute information for a face is not limited to these methods, and any method may be used.

[0145] 17 is a conceptual diagram showing types of three-dimensional data according to this embodiment. Point cloud data and mesh data may represent static objects or dynamic objects. A static object is an object that does not change over time, and a dynamic object is an object that changes over time. A static object may correspond to three-dimensional data for any point in time.

[0146] For example, point cloud data for a given point in time may be referred to as a PCC frame, mesh data for a given point in time may be referred to as a mesh frame, and PCC frames and mesh frames may be simply referred to as frames.

[0147] The area of ​​the object may be limited to a certain range, as in normal video data, or may not be limited, as in map data. The density of points or surfaces may be determined in various ways. Sparse point cloud data or sparse mesh data may be used, or dense point cloud data or dense mesh data may be used.

[0148] Next, encoding and decoding of a point cloud or a three-dimensional mesh will be described. The device, process, or syntax for encoding and decoding vertex information of a three-dimensional mesh in the present disclosure may be applied to encoding and decoding of a point cloud. The device, process, or syntax for encoding and decoding of a point cloud in the present disclosure may be applied to encoding and decoding vertex information of a three-dimensional mesh.

[0149] Furthermore, a device, process, or syntax for encoding and decoding attribute information of a point cloud in the present disclosure may be applied to encoding and decoding connectivity information or attribute information of a three-dimensional mesh.Furthermore, a device, process, or syntax for encoding and decoding connectivity information or attribute information of a three-dimensional mesh in the present disclosure may be applied to encoding and decoding attribute information of a point cloud.

[0150] Furthermore, at least some of the processing may be shared between the encoding and decoding of point cloud data and the encoding and decoding of mesh data, thereby reducing the scale of the circuit and software program.

[0151] 18 is a block diagram showing an example configuration of a three-dimensional data encoder 113 according to this embodiment. In this example, the three-dimensional data encoder 113 includes a vertex information encoder 121, an attribute information encoder 122, a metadata encoder 123, and a multiplexer 124. The vertex information encoder 121, the attribute information encoder 122, and the multiplexer 124 may correspond to the vertex information encoder 101, the attribute information encoder 103, the post-processor 105, etc. in FIG.

[0152] In this example, the three-dimensional data encoder 113 encodes the three-dimensional data according to a geometry-based encoding method, which takes into account the three-dimensional structure. In addition, in the geometry-based encoding method, attribute information is encoded using configuration information obtained in encoding the vertex information.

[0153] Specifically, first, vertex information, attribute information, and metadata included in three-dimensional data generated from sensor data are input to a vertex information encoder 121, an attribute information encoder 122, and a metadata encoder 123, respectively. Here, connectivity information included in the three-dimensional data may be treated in the same way as attribute information. In addition, in the case of point cloud data, position information may be treated as vertex information.

[0154] The vertex information encoder 121 encodes the vertex information into compressed vertex information and outputs the compressed vertex information as encoded data to the multiplexer 124. The vertex information encoder 121 also generates metadata for the compressed vertex information and outputs it to the multiplexer 124. The vertex information encoder 121 also generates configuration information and outputs it to the attribute information encoder 122.

[0155] The attribute information encoder 122 uses the configuration information generated by the vertex information encoder 121 to encode the attribute information into compressed attribute information and outputs the compressed attribute information as encoded data to the multiplexer 124. The attribute information encoder 122 also generates metadata of the compressed attribute information and outputs it to the multiplexer 124.

[0156] The metadata encoder 123 encodes compressible metadata into compressed metadata and outputs the compressed metadata as encoded data to the multiplexer 124. The metadata encoded by the metadata encoder 123 may be used to encode vertex information and attribute information.

[0157] The multiplexer 124 multiplexes the compressed vertex information, the compressed vertex information metadata, the compressed attribute information, the compressed attribute information metadata, and the compressed metadata into a bitstream, and then inputs the bitstream to the system layer.

[0158] 19 is a block diagram showing an example configuration of a three-dimensional data decoder 213 according to this embodiment. In this example, the three-dimensional data decoder 213 includes a vertex information decoder 221, an attribute information decoder 222, a metadata decoder 223, and a demultiplexer 224. The vertex information decoder 221, the attribute information decoder 222, and the demultiplexer 224 may correspond to the vertex information decoder 201, the attribute information decoder 203, the preprocessor 204, and the like in FIG.

[0159] In this example, the three-dimensional data decoder 213 decodes three-dimensional data according to a geometry-based encoding method. The three-dimensional structure is taken into consideration in the decoding according to the geometry-based encoding method. Furthermore, in the decoding according to the geometry-based encoding method, attribute information is decoded using configuration information obtained in decoding vertex information.

[0160] Specifically, first, a bitstream is input from the system layer to a demultiplexer 224. The demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream. The compressed vertex information and compressed vertex information metadata are input to a vertex information decoder 221. The compressed attribute information and compressed attribute information metadata are input to an attribute information decoder 222. The metadata is input to a metadata decoder 223.

[0161] The vertex information decoder 221 decodes vertex information from the compressed vertex information using metadata of the compressed vertex information. The vertex information decoder 221 also generates configuration information and outputs it to the attribute information decoder 222. The attribute information decoder 222 decodes attribute information from the compressed attribute information using the configuration information generated by the vertex information decoder 221 and the metadata of the compressed attribute information. The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 may be used to decode the vertex information and the attribute information.

[0162] Thereafter, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. Note that, for example, this metadata is metadata of the vertex information and attribute information, and can be used in an application program.

[0163] 20 is a block diagram showing another example configuration of the three-dimensional data encoder 113 according to the present embodiment. In this example, the three-dimensional data encoder 113 includes a vertex image generator 131, an attribute image generator 132, a metadata generator 133, a video encoder 134, a metadata encoder 123, and a multiplexer 124. The vertex image generator 131, the attribute image generator 132, and the video encoder 134 may correspond to the vertex information encoder 101 and the attribute information encoder 103 in FIG. 6 , etc.

[0164] In this example, the 3D data encoder 113 encodes the 3D data according to a video-based encoding method. In encoding according to the video-based encoding method, multiple 2D images are generated from the 3D data, and the multiple 2D images are encoded according to a video encoding method. Here, the video encoding method may be High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), or the like.

[0165] Specifically, first, vertex information and attribute information included in three-dimensional data generated from sensor data are input to a metadata generator 133. The vertex information and attribute information are then input to a vertex image generator 131 and an attribute image generator 132, respectively. The metadata included in the three-dimensional data is then input to a metadata encoder 123. Here, connectivity information included in the three-dimensional data may be treated in the same way as attribute information. In the case of point cloud data, position information may be treated as vertex information.

[0166] The metadata generator 133 generates map information of a plurality of two-dimensional images from the vertex information and attribute information, and inputs the map information to the vertex image generator 131, the attribute image generator 132, and the metadata encoder 123.

[0167] The vertex image generator 131 generates a vertex image based on the vertex information and map information, and inputs the generated image to the video encoder 134. The attribute image generator 132 generates an attribute image based on the attribute information and map information, and inputs the generated image to the video encoder 134.

[0168] The video encoder 134 encodes the vertex images and attribute images into compressed vertex information and compressed attribute information, respectively, in accordance with a video encoding method, and outputs the compressed vertex information and compressed attribute information as encoded data to the multiplexer 124. The video encoder 134 also generates metadata for the compressed vertex information and metadata for the compressed attribute information, and outputs them to the multiplexer 124.

[0169] The metadata encoder 123 encodes the compressible metadata into compressed metadata and outputs the compressed metadata as encoded data to the multiplexer 124. The compressible metadata includes map information. The metadata encoded by the metadata encoder 123 may also be used to encode vertex information and attribute information.

[0170] The multiplexer 124 multiplexes the compressed vertex information, the compressed vertex information metadata, the compressed attribute information, the compressed attribute information metadata, and the compressed metadata into a bitstream, and then inputs the bitstream to the system layer.

[0171] 21 is a block diagram showing another example configuration of the 3D data decoder 213 according to this embodiment. In this example, the 3D data decoder 213 includes a vertex information generator 231, an attribute information generator 232, a video decoder 234, a metadata decoder 223, and a demultiplexer 224. The vertex information generator 231, the attribute information generator 232, and the video decoder 234 may correspond to the vertex information decoder 201 and the attribute information decoder 203 in FIG. 8, etc.

[0172] In this example, the 3D data decoder 213 decodes the 3D data according to a video-based coding method. In the decoding according to the video-based coding method, a plurality of 2D images are decoded according to a video coding method, and 3D data is generated from the plurality of 2D images. Here, the video coding method may be High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), or the like.

[0173] Specifically, first, a bitstream is input from the system layer to the demultiplexer 224. The demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream. The compressed vertex information, compressed vertex information metadata, compressed attribute information, and compressed attribute information metadata are input to the video decoder 234. The compressed metadata is input to the metadata decoder 223.

[0174] The video decoder 234 decodes the vertex images in accordance with the video encoding method. At this time, the video decoder 234 decodes the vertex images from the compressed vertex information using the metadata of the compressed vertex information. Then, the video decoder 234 inputs the vertex images to the vertex information generator 231. The video decoder 234 also decodes the attribute images in accordance with the video encoding method. At this time, the video decoder 234 decodes the attribute images from the compressed attribute information using the metadata of the compressed attribute information. Then, the video decoder 234 inputs the attribute images to the attribute information generator 232.

[0175] The metadata decoder 223 decodes metadata from the compressed metadata. The metadata decoded by the metadata decoder 223 includes map information used to generate vertex information and attribute information. The metadata decoded by the metadata decoder 223 may also be used to decode vertex images and attribute images.

[0176] The vertex information generator 231 reproduces vertex information from the vertex image in accordance with the map information included in the metadata decoded by the metadata decoder 223. The attribute information generator 232 reproduces attribute information from the attribute image in accordance with the map information included in the metadata decoded by the metadata decoder 223.

[0177] Thereafter, the vertex information, attribute information, and metadata are output as three-dimensional data from the three-dimensional data decoder 213. Note that, for example, this metadata is metadata of the vertex information and attribute information, and can be used in an application program.

[0178] Fig. 22 is a conceptual diagram showing a specific example of encoding processing according to this embodiment. Fig. 22 shows a three-dimensional data encoder 113 and a description encoder 148. In this example, the three-dimensional data encoder 113 includes a two-dimensional data encoder 141 and a mesh data encoder 142. The two-dimensional data encoder 141 includes a texture encoder 143. The mesh data encoder 142 includes a vertex information encoder 144 and a connection information encoder 145.

[0179] The vertex information encoder 144, the connection information encoder 145, and the texture encoder 143 may correspond to the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 in FIG.

[0180] For example, the two-dimensional data encoder 141 operates as a texture encoder 143 and generates a texture file by encoding the texture corresponding to the attribute information as two-dimensional data according to an image encoding method or a video encoding method.

[0181] The mesh data encoder 142 also operates as a vertex information encoder 144 and a connectivity information encoder 145, and generates a mesh file by encoding the vertex information and connectivity information. The mesh data encoder 142 may further encode mapping information for textures. The encoded mapping information may then be included in the mesh file.

[0182] The description encoder 148 also generates a description file by encoding a description corresponding to metadata such as text data. The description encoder 148 may encode the description at the system layer. For example, the description encoder 148 may be included in the system multiplexer 114 of FIG. 12 .

[0183] The above operations generate a bitstream containing texture files, mesh files, and description files, which may be multiplexed into the bitstream in file formats such as glTF (Graphics Language Transmission Format) or USD (Universal Scene Description).

[0184] The three-dimensional data encoder 113 may include two mesh data encoders as the mesh data encoder 142. For example, one mesh data encoder encodes vertex information and connectivity information of a static three-dimensional mesh, and the other mesh data encoder encodes vertex information and connectivity information of a dynamic three-dimensional mesh.

[0185] Correspondingly, two mesh files may then be included in the bitstream, for example one mesh file corresponding to a static 3D mesh and another mesh file corresponding to a dynamic 3D mesh.

[0186] Furthermore, the static three-dimensional mesh may be a three-dimensional mesh of an intraframe coded using intraprediction, and the dynamic three-dimensional mesh may be a three-dimensional mesh of an interframe coded using interprediction. Furthermore, information on the dynamic three-dimensional mesh may be differential information between vertex information or connectivity information of the three-dimensional mesh of an intraframe and vertex information or connectivity information of the three-dimensional mesh of an interframe.

[0187] Fig. 23 is a conceptual diagram showing a specific example of the decoding process according to this embodiment. Fig. 23 shows a three-dimensional data decoder 213, a description decoder 248, and a renderer 247. In this example, the three-dimensional data decoder 213 includes a two-dimensional data decoder 241, a mesh data decoder 242, and a mesh reconstructor 246. The two-dimensional data decoder 241 includes a texture decoder 243. The mesh data decoder 242 includes a vertex information decoder 244 and a connectivity information decoder 245.

[0188] The vertex information decoder 244, the connection information decoder 245, the texture decoder 243, and the mesh reconstructor 246 may correspond to the vertex information decoder 201, the connection information decoder 202, the attribute information decoder 203, and the post-processor 205 in Fig. 8. The presenter 247 may correspond to the presenter 215 in Fig. 12.

[0189] For example, the two-dimensional data decoder 241 operates as a texture decoder 243, and decodes the texture corresponding to the attribute information from the texture file as two-dimensional data in accordance with an image coding method or a video coding method.

[0190] The mesh data decoder 242 also operates as a vertex information decoder 244 and a connectivity information decoder 245 to decode vertex information and connectivity information from the mesh file. The mesh data decoder 242 may further decode mapping information for textures from the mesh file.

[0191] The description decoder 248 also decodes descriptions corresponding to metadata such as text data from the description file. The description decoder 248 may decode the descriptions at the system layer. For example, the description decoder 248 may be included in the system demultiplexer 214 of FIG. 12 .

[0192] The mesh reconstructor 246 reconstructs a 3D mesh from the vertex information, connectivity information, and textures according to the description. The renderer 247 renders and outputs the 3D mesh according to the description.

[0193] Through the above operations, a 3D mesh is reconstructed and output from a bitstream containing a texture file, a mesh file, and a description file.

[0194] The three-dimensional data decoder 213 may include two mesh data decoders as the mesh data decoder 242. For example, one mesh data decoder decodes vertex information and connectivity information of a static three-dimensional mesh, and the other mesh data decoder decodes vertex information and connectivity information of a dynamic three-dimensional mesh.

[0195] Correspondingly, two mesh files may then be included in the bitstream: for example, one mesh file corresponding to a static 3D mesh and another mesh file corresponding to a dynamic 3D mesh.

[0196] Furthermore, the static three-dimensional mesh may be a three-dimensional mesh of an intraframe coded using intraprediction, and the dynamic three-dimensional mesh may be a three-dimensional mesh of an interframe coded using interprediction. Furthermore, information on the dynamic three-dimensional mesh may be differential information between vertex information or connectivity information of the three-dimensional mesh of an intraframe and vertex information or connectivity information of the three-dimensional mesh of an interframe.

[0197] A dynamic 3D mesh coding method is sometimes called DMC (Dynamic Mesh Coding), and a video-based dynamic 3D mesh coding method is sometimes called V-DMC (Video-based Dynamic Mesh Coding).

[0198] The point cloud encoding method is sometimes called PCC (Point Cloud Compression). The point cloud video-based encoding method is sometimes called V-PCC (Video-based Point Cloud Compression). The point cloud geometry-based encoding method is sometimes called G-PCC (Geometry-based Point Cloud Compression).

[0199] <Implementation Example> Fig. 24 is a block diagram showing an implementation example of the encoding device 100 according to this embodiment. The encoding device 100 includes a circuit 151 and a memory 152. For example, multiple components of the encoding device 100 shown in Fig. 5 etc. are implemented by the circuit 151 and memory 152 shown in Fig. 24.

[0200] The circuit 151 is a circuit that performs information processing and is a circuit that can access the memory 152. For example, the circuit 151 is a dedicated or general-purpose electric circuit that encodes a three-dimensional mesh. The circuit 151 may be a processor such as a CPU. Alternatively, the circuit 151 may be a collection of multiple electric circuits.

[0201] The memory 152 is a dedicated or general-purpose memory that stores information used by the circuit 151 to encode the three-dimensional mesh. The memory 152 may be an electric circuit and may be connected to the circuit 151. The memory 152 may also be included in the circuit 151. The memory 152 may also be a collection of multiple electric circuits. The memory 152 may also be a magnetic disk, an optical disk, or the like, and may also be expressed as a storage, a recording medium, or the like. The memory 152 may also be a non-volatile memory or a volatile memory.

[0202] For example, the memory 152 may store a three-dimensional mesh or a bitstream, or may store a program for the circuit 151 to encode the three-dimensional mesh.

[0203] Note that the encoding device 100 does not necessarily have to implement all of the components shown in Figure 5 and the like, and does not necessarily have to perform all of the processes shown here. Some of the components shown in Figure 5 and the like may be included in another device, and some of the processes shown here may be executed by another device. Furthermore, the encoding device 100 may implement any combination of the components of the present disclosure, and may perform any combination of the processes of the present disclosure.

[0204] Fig. 25 is a block diagram showing an example implementation of a decoding device 200 according to this embodiment. The decoding device 200 includes a circuit 251 and a memory 252. For example, multiple components of the decoding device 200 shown in Fig. 7 and other figures are implemented by the circuit 251 and memory 252 shown in Fig. 25.

[0205] The circuit 251 is a circuit that performs information processing and is a circuit that can access the memory 252. For example, the circuit 251 is a dedicated or general-purpose electric circuit that decodes a three-dimensional mesh. The circuit 251 may be a processor such as a CPU. Alternatively, the circuit 251 may be a collection of multiple electric circuits.

[0206] The memory 252 is a dedicated or general-purpose memory that stores information for the circuit 251 to decode the 3D mesh. The memory 252 may be an electric circuit and may be connected to the circuit 251. The memory 252 may also be included in the circuit 251. The memory 252 may also be a collection of multiple electric circuits. The memory 252 may also be a magnetic disk, an optical disk, or the like, and may also be expressed as a storage, a recording medium, or the like. The memory 252 may also be a non-volatile memory or a volatile memory.

[0207] For example, the memory 252 may store a three-dimensional mesh or a bitstream, or may store a program for the circuit 251 to decode the three-dimensional mesh.

[0208] Note that the decoding device 200 does not necessarily have to implement all of the components shown in Figure 7 and the like, and does not necessarily have to perform all of the processes shown here. Some of the components shown in Figure 7 and the like may be included in another device, and some of the processes shown here may be executed by another device. Furthermore, the decoding device 200 may implement any combination of the components of the present disclosure, and may perform any combination of the processes of the present disclosure.

[0209] The encoding method and the decoding method including the steps performed by each component of the encoding device 100 and the decoding device 200 of the present disclosure may be executed by any device or system. For example, part or all of the encoding method and the decoding method may be executed by a computer including a processor, a memory, an input / output circuit, etc. In this case, the encoding method and the decoding method may be executed by the computer executing a program for causing the computer to execute the encoding method and the decoding method.

[0210] Furthermore, the program or the bitstream may be recorded on a non-transitory computer-readable recording medium such as a CD-ROM.

[0211] An example of a program may be a bitstream. For example, a bitstream including an encoded three-dimensional mesh includes syntax elements for causing the decoding device 200 to decode the three-dimensional mesh. The bitstream then causes the decoding device 200 to decode the three-dimensional mesh according to the syntax elements included in the bitstream. Thus, the bitstream may play a role similar to that of a program.

[0212] The bitstream may be an encoded bitstream containing the encoded 3D mesh, or may be a multiplexed bitstream containing the encoded 3D mesh and other information.

[0213] Furthermore, each component of the encoding device 100 and the decoding device 200 may be configured with dedicated hardware, general-purpose hardware that executes the above-mentioned programs, or a combination of these. The general-purpose hardware may be configured with a memory in which the programs are recorded and a general-purpose processor that reads and executes the programs from the memory. Here, the memory may be a semiconductor memory or a hard disk, and the general-purpose processor may be a CPU.

[0214] Furthermore, the dedicated hardware may be configured with a memory, a dedicated processor, etc. For example, the dedicated processor may execute the encoding method and the decoding method by referring to a memory for recording data.

[0215] Furthermore, as described above, each component of the encoding device 100 and the decoding device 200 may be an electric circuit. These electric circuits may form a single electric circuit as a whole, or each may be a separate electric circuit. Furthermore, these electric circuits may correspond to dedicated hardware, or may correspond to general-purpose hardware that executes the above-mentioned programs, etc. Furthermore, the encoding device 100 and the decoding device 200 may be implemented as an integrated circuit.

[0216] Furthermore, the encoding device 100 may be a transmitting device that transmits the three-dimensional mesh, and the decoding device 200 may be a receiving device that receives the three-dimensional mesh.

[0217] One or More Main Aspects: A 3D model digitally represents an object so that a user can explore it by zooming, panning, and rotating it in three dimensions while temporarily rendering the model. One way to construct such a representation is to build a 3D mesh using polygons, where the model stores the positions of the polygon vertices, their connectivity to each other, and their associated attributes (normals, UV patches, etc.). Examples of polygons are triangles and quadrilaterals.

[0218] Storing all this information in uncompressed form would require very large storage capacity and therefore very large bandwidth for transmission. The polygons that form the mesh often have repeating patterns and similar attributes, especially in temporal and spatial neighborhoods. These repetitions can be used to develop efficient encoding and decoding methods for storage and transmission.

[0219] Fig. 26 is a block diagram showing another example of the configuration of a coding / decoding system according to this embodiment. As shown in Fig. 26, the coding / decoding system includes a pair of a coding device 100 and a decoding device 200. The coding / decoding system accepts a three-dimensional mesh input in the form of three-dimensional coordinates of vertices (vertex information), connectivity (connection information), and associated attributes (attribute information).

[0220] The encoding device 100 encodes all relevant information into a bitstream (compressed bitstream), which may consist of multiple bitstreams. The bitstream is transmitted to the decoding device 200 via a transmission channel. The decoding device 200 decodes the bitstream and generates a 3D model (3D mesh) using the 3D coordinates, connectivity, and associated attributes of the decoded vertices.

[0221] 27 is a block diagram showing yet another example of the configuration of the encoding device 100 according to this embodiment. In this example, the encoding device 100 includes a preprocessor 521 and an encoding processor 522.

[0222] The preprocessor 521 reads the 3D mesh and processes it to extract a base mesh, displacement vectors, texture data, and attribute maps, which are then passed to the encoding processor 522. The encoding processor 522 compresses the base mesh, displacement vectors, texture data, and attribute maps separately, and combines them to generate a bitstream.

[0223] 28 is a block diagram showing yet another example of the configuration of the decoding device 200 according to this embodiment. In this example, the decoding device 200 includes a decoding processor 622 and a post-processor 623.

[0224] The decoder 622 reads the bitstream, separates the base mesh, displacement vectors, texture data and attribute maps from the bitstream, decodes them separately and passes them to the post-processor 623. The post-processor 623 processes the base mesh according to the displacement vectors and attribute maps to generate a 3D mesh.

[0225] For example, in an encoding method, the original 3D mesh is first decimated to obtain a base mesh containing fewer vertices, in which the vertices may not be in their original positions and the decimation may change the connectivity of the vertices.

[0226] The base mesh is then refined over multiple iterations, adding new vertices between the existing connected vertices of the base mesh. Displacement vectors between the refined 3D mesh and the input 3D mesh are then calculated. The displacement vectors are used by the decoding device 200 to place the refinement vertices at their predicted positions.

[0227] This information is converted into wavelet coefficients using a wavelet transform and coded using a video codec by mapping the coefficients onto the planes of the video frame. The textures in the 3D mesh and the maps used to combine all the information for reconstruction are coded separately and then combined into one bitstream.

[0228] The decoding device 200 first decodes a base mesh. Over multiple iterations, the base mesh is refined by adding new vertices between the existing connected vertices of the base mesh. After this, all vertices and connectivity are obtained. Again, vertices may be at different locations than their corresponding vertices in the input 3D mesh.

[0229] A video decoder is then used to decode the wavelet coefficients and apply an inverse wavelet transform to reconstruct the displacement vectors. Using this information, vertices are placed at their predicted positions, and textures are mapped onto the surfaces created by the vertices and their connectivity, completely decoding the 3D mesh.

[0230] Recent developments in 3D data acquisition, modeling, and rendering have fueled the application of 3D content on various platforms and devices. 3D media is a very convenient way for users to enjoy highly immersive experiences in fields such as digital entertainment, healthcare, and robotics.

[0231] A 3D mesh, used for example in rendering immersive media, is composed of a number of polygons that represent the boundary surfaces of a volumetric object. Each polygon contains vertices in 3D space and connectivity information that defines how the vertices are connected. Optionally, a 3D mesh may also include attributes such as color, normals, and material.

[0232] A dynamic mesh is a type of mesh in which at least one of the following changes over time: connectivity information, geometry information, mapping information, vertex attributes, and attribute maps. Dynamic meshes consist of a large amount of time-varying data and may require large amounts of storage. Therefore, efficient compression solutions are essential for the proper storage and transmission of such data.

[0233] A method called an edgebreaker is used to encode a 3D mesh, each of which is a triangle. The edgebreaker may correspond to a method for classifying triangles that make up a 3D mesh. In this method, the triangles that make up the 3D mesh are classified into five types according to the patterns of adjacent triangles, and the 3D mesh is represented by a character string consisting of C, L, E, R, and S.

[0234] Here, S indicates a branch, E indicates an end, C indicates a corner, and L and R indicate the adjacent positions of the triangle to be coded next. These types are also called type S, type E, type C, type L, and type R. For example, a type C triangle is not adjacent to a triangle that has already been coded. For a type L triangle, the next adjacent triangle to the right is coded. For a type R triangle, the next adjacent triangle to the left is coded.

[0235] Note that encoding (or decoding) a triangle may include encoding (or decoding) the connection type of the triangle, and the connection type of the encoded (or decoded) triangle may refer to the encoded (or decoded) connection type.

[0236] FIG. 29 is a conceptual diagram showing five types of connection types according to this embodiment. The connection types indicate the connection relationship between a triangle to be processed and an unprocessed triangle. Here, "to be visited" corresponds to "to be visited," "processed" corresponds to "visited," and "unprocessed" corresponds to "not visited." Note that vertices of a visited triangle may also be considered visited. Furthermore, a portion where no triangle exists and vertices adjacent to that portion may also be considered visited.

[0237] In Type S, multiple unprocessed triangles are separated and connected to both sides of the target triangle along the path leading to the target triangle in the processing order. In other words, multiple unprocessed triangles are branched and connected to both sides of the target triangle. For example, in Type S, vertex v has been visited. After processing a Type S triangle, the unprocessed triangle connected to the right side of the Type S triangle is processed first. Then, specifically after processing a Type E triangle, the unprocessed triangle connected to the left side of the Type S triangle is processed.

[0238] In Type C, multiple unprocessed triangles are connected to both sides of the target triangle without being separated along the path leading to the target triangle in the processing order. In other words, multiple unprocessed triangles are connected to both sides of the target triangle without being branched. For example, in Type C, vertex v is unvisited. After processing a Type C triangle, the unprocessed triangles connected to the right side of the Type C triangle are processed.

[0239] In type L, along the path leading to the target triangle in the processing order, no unprocessed triangles are connected to the left side of the target triangle, and an unprocessed triangle is connected to the right side of the target triangle. For example, in type L, vertex v has been visited. After processing a type L triangle, the unprocessed triangles connected to the right side of the type L triangle are processed.

[0240] In type R, along the path leading to the target triangle in the processing order, no unprocessed triangles are connected to the right of the target triangle, and an unprocessed triangle is connected to the left of the target triangle. For example, in type R, vertex v has been visited. After processing a type R triangle, the unprocessed triangles connected to the left of the type R triangle are processed.

[0241] In type E, no unprocessed triangles are connected to the right or left of the triangle being processed along the path leading to the triangle in the processing order. For example, in type E, vertex v has already been visited. After processing a triangle of type E, for example, an unprocessed triangle connected to the left of a processed triangle of type S is processed. Alternatively, after processing a triangle of type E, the series of processes ends.

[0242] For example, Type C is a type in which two unprocessed triangles are connected to the right and left of the triangle to be processed along the path leading to the triangle to be processed in the processing order, and all triangles connected to the vertices of the triangle to be processed, including these two triangles, are unprocessed.Furthermore, Type S is a type in which two unprocessed triangles are connected to the right and left of the triangle to be processed along the path leading to the triangle to be processed in the processing order, and is a type different from the corner type.

[0243] In particular, in the case of type C, the triangle to be processed and multiple unprocessed triangles may form a corner on the 3D mesh at vertex v. Therefore, type C indicates a corner and may also be expressed as a corner type. The other types S, L, R, and E may also be expressed as a branch type, a left type, a right type, and an end type, respectively. Furthermore, types equivalent to these types may also be defined by methods other than edge breakers.

[0244] Fig. 30 is a conceptual diagram showing an example of sequentially determined connection types according to this embodiment. As shown in Fig. 30, the connection types of the multiple triangles constituting a 3D mesh are sequentially determined to be one of the multiple types shown in Fig. 29, and then encoded and decoded. The connection types indicate the connection relationships between the multiple triangles and may indicate the processing order of the multiple triangles. The connection types may then be used to reconstruct a 3D model.

[0245] Alternatively, bit patterns 0, 110, 111, 101, and 100 may be assigned to types C, L, E, R, and S, respectively. These bit patterns may then be encoded. For the leading triangle, in addition to type C, L, E, R, or S, information about each vertex of the leading triangle (such as the three-dimensional coordinate position, texture map coordinate position, and normal vector) may also be encoded. For type C triangles, information about newly added vertices may also be encoded.

[0246] For example, the edge breaker in the present disclosure may be used in the encoding and decoding processes of a three-dimensional mesh, or in the encoding and decoding processes of a base mesh in Fig. 27 or 28. Specifically, the process of the present disclosure may be applied to the encoding and decoding processes of information indicating the connection relationship between multiple faces, which are multiple polygons that make up a three-dimensional mesh.

[0247] When decoding the character strings consisting of C, L, E, R, and S to restore the structure of the 3D mesh, a process called "zippering" is performed, in which adjacent triangles are sequentially connected. This process is performed when an L-type or E-type triangle appears, and is repeated for a series of connectable triangles. This restores the structure of the 3D mesh.

[0248] Also, a method called Spiral Reversi is known, in which the structure of a three-dimensional mesh is restored by tracing the character string composed of the above-mentioned C, L, E, R, and S in reverse order from the end. This method may make it possible to efficiently restore the structure of a three-dimensional mesh by suppressing the repetition of reading the character string. Furthermore, by encoding and decoding the character string composed of C, L, E, R, and S in reverse order from the end, it may be possible to use information about the restored three-dimensional mesh structure when encoding and decoding the character string.

[0249] Fig. 31 is a flowchart showing an example of a connection type encoding process and a decoding process according to this embodiment. Specifically, Fig. 31 shows an example of a method for encoding, for each triangle, the type (C, L, E, R, or S) associated with each triangle constituting a 3D mesh. Here, the encoding is entropy encoding, such as arithmetic encoding or Huffman encoding.

[0250] In this method, the value of a variable prev is set according to the previously coded connection type, and parameters for controlling entropy coding of the triangle connection type are determined according to prev, and entropy coding of the connection type to be coded is performed according to the determined parameters.

[0251] Specifically, first, prev is set to an initial value (e.g., 0) (S101). Then, entropy coding of each triangle connection type is repeated (S102 to S109). Here, parameters for controlling entropy coding of the triangle connection type are determined according to prev, and entropy coding of the connection type to be coded is performed (S103).

[0252] For example, when arithmetic coding is used for entropy coding, a context used to determine the predicted occurrence probability of information to be coded is determined from among a plurality of contexts according to prev. Also, when Huffman coding is used for entropy coding, a code word table used for coding is determined from among a plurality of code word tables according to prev.

[0253] Next, the value of prev, which is used to control entropy coding of the connection type to be coded next, is set according to the coded connection type (S104 to S107). Specifically, first, it is determined whether the coded connection type is type S. If the coded connection type is type S (Yes in S104), prev is set to 1 (S106).

[0254] If the encoded connection type is not type S (No in S104: any of C, R, L, and E), it is determined whether the encoded connection type is type E (S105). If the encoded connection type is type E (Yes in S105), prev is set to 2 (S107). If the encoded connection type is not type E (No in S105: any of C, R, and L), prev is set to 0 (S108).

[0255] As a result of trials, the inventors have found a new finding that the tendency of the occurrence probability of each type coded immediately after coding of type S or E is different from the tendency of the occurrence probability of each type coded immediately after coding of type C, R, or L. Therefore, as described above, by switching the control of entropy coding depending on the connection type coded immediately before, it may be possible to improve the coding efficiency of entropy coding.

[0256] 31 shows an example in which the control of entropy coding is switched by classifying the connection types into three groups: a type S group, a type E group, and a type C, R, and L group. However, a context may be determined according to each of the five types.

[0257] 31, the determination and setting of type E (S105 and S107) may be omitted. Then, the connection types may be classified into two groups, a type S group and a type C, R, L, and E group, and the control of entropy coding may be switched.

[0258] This may make it possible to improve coding efficiency compared to the case of only one group, while reducing memory usage compared to the case of three groups.

[0259] 31, the value of prev is set to 0, 1, or 2 depending on the encoded connection type. However, another method may be used to distinguish between three groups, such as a type S group, a type E group, and a type C, R, and L group, and switch the control of entropy encoding.

[0260] Furthermore, in the above description, the control of entropy encoding is switched, but encoding may be interpreted as decoding, and the control of entropy decoding may be switched.

[0261] In the above description, the context for arithmetic coding is selected according to the type associated with the triangle processed immediately before the triangle to be processed. However, the selection of the context is not limited to this example. In other words, the context is not limited to being selected according to the immediately preceding type, and the parameters for entropy coding or entropy decoding of the connectivity information to be processed may be determined according to the connectivity information (linkage information) that has already been coded or decoded.

[0262] Here, the coded or decoded link information may be link information of a polygon that was coded or decoded immediately before. Also, the entropy coding or entropy decoding may be arithmetic coding or arithmetic decoding, and in this case, the parameter may be a context or a parameter for deriving the context.

[0263] As an example of context selection, a first context may be used when the connectivity information of the polygon most recently coded or decoded is not of a type that indicates branching, and a second context may be used when the connectivity information of the polygon most recently coded or decoded is of a type that indicates branching.

[0264] In other words, the context may be determined based on whether the type of connectivity information of a processed polygon at a predetermined position relative to the polygon being processed indicates a branch or whether the type of connectivity information of a processed polygon at a predetermined position relative to the polygon being processed indicates a termination or not.

[0265] In addition, whether or not a context is determined based on the type of polygon that has already been processed may be switched. Also, the connection type used as the criterion may be switched. In this case, a parameter indicating whether or not switching is appropriate, the context to be selected, or the connection type of the polygon corresponding to the context may be coded.

[0266] Furthermore, the order of the above-described determination and setting processes may be changed or some of them may be omitted. For example, the determination of type S (S104) and the determination of type E (S105) may be interchanged, or one of them may be omitted.

[0267] Fig. 32 is a flowchart showing a first specific example of connection type encoding according to this embodiment. That is, Fig. 32 shows an example of the encoding process (S103) in Fig. 31. In this example, bit patterns 0, 110, 111, 101, and 100 are assigned to types C, L, E, R, and S, respectively, and entropy encoding is performed starting from the most significant bit.

[0268] First, it is determined whether the connection type of the triangle to be coded is Type C (S201). If the connection type is Type C (Yes in S201), the value of the first bit b0 is coded as 0 by entropy coding according to prev (S202). If the connection type is not Type C (No in S201: any of L, E, R, and S), the value of the first bit b0 is coded as 1 by entropy coding according to prev (S203).

[0269] Next, it is determined whether the connection type of the triangle to be coded is either type R or type S (S204). If the connection type is either type R or type S (Yes in S204), the value of the second bit b1 is set to 0 by entropy coding according to prev (S205).

[0270] Then, it is determined whether the connection type of the triangle to be coded is Type S (S207). If the connection type is Type S (Yes in S207), the value of the third bit b2 is coded as 0 by entropy coding according to prev (S211). If the connection type is not Type S (No: R in S207), the value of the third bit b2 is coded as 1 by entropy coding according to prev (S209).

[0271] If the connection type is not one of types R and S (No: L or E in S204), the value of the second bit b1 is coded as 1 by entropy coding according to prev (S206).

[0272] Then, it is determined whether the connection type of the triangle to be coded is Type L (S208). If the connection type is Type L (Yes in S208), the value of the third bit b2 is coded as 0 by entropy coding according to prev (S212). If the connection type is not Type L (No: E in S208), the value of the third bit b2 is coded as 1 by entropy coding according to prev (S210).

[0273] When arithmetic coding is used for entropy coding, a context used to determine the predicted occurrence probability of information to be coded is determined from among a plurality of contexts according to prev. When Huffman coding is used for entropy coding, a code word table used for coding is determined from among a plurality of code word tables according to prev.

[0274] Furthermore, in the entropy coding of the value of the third bit b2 (S207 to S212), the context or code word table may be determined according to the value of the second bit b1 in addition to prev.

[0275] The tendency of the occurrence probability of each type coded immediately after type S or E is different from the tendency of the occurrence probability of each type coded immediately after type C, R, or L. Therefore, as described above, the control of entropy coding is switched according to prev determined by the procedure of Fig. 31. This may make it possible to improve the coding efficiency of entropy coding.

[0276] Furthermore, the content of the event notified by the value of the third bit b2 differs depending on the value of the second bit b1 (if b1 = 0, type S or R, and if b1 = 1, type L or E). Therefore, in entropy coding of the value of the third bit b2, by determining the context or codeword table depending on the value of the second bit b1 in addition to prev, it may be possible to further improve coding efficiency.

[0277] In the above, encoding bits may include inputting the bits into a coding engine, where the bits may be compressed into fewer bits.

[0278] Fig. 33 is a flowchart showing a first specific example of connection type decoding according to this embodiment. That is, Fig. 33 shows an example of the decoding process (S103) in Fig. 31. In this example, entropy decoding is performed on a bit string encoded by the encoding method described with reference to Fig. 32, and one of types C, L, E, R, and S is set as the connection type of the triangle to be decoded.

[0279] First, the value of the first bit b0 is decoded by entropy decoding according to prev (S301). Then, it is determined whether the value of the first bit b0 is 0 (S302). If the value of the first bit b0 is 0 (Yes in S302), the connection type of the triangle to be decoded is set to Type C (S303).

[0280] If the value of the first bit b0 is not 0 (No in S302), the value of the second bit b1 is decoded by entropy decoding according to prev (S304). The value of the third bit b2 is also decoded by entropy decoding according to prev (S305). The bit patterns of the second bit b1 and the third bit b2 are then determined (S306-S308). Then, according to the determination result, the connection type of the triangle to be decoded is set to one of types L, E, R, or S (S309-S312).

[0281] Specifically, if the value of the second bit b1 and the value of the third bit b2 are 0 and 0, respectively (Yes in S306 and Yes in S307), the connection type is set to type S (S311). If the value of the second bit b1 and the value of the third bit b2 are 0 and 1, respectively (Yes in S306 and No in S307), the connection type is set to type R (S309).

[0282] If the value of the second bit b1 and the value of the third bit b2 are 1 and 0, respectively (No in S306 and Yes in S308), the connection type is set to Type L (S312). If the value of the second bit b1 and the value of the third bit b2 are 1 and 1, respectively (No in S306 and No in S308), the connection type is set to Type E (S310).

[0283] When arithmetic decoding is used for entropy decoding, a context used to determine the expected occurrence probability of information to be decoded is determined from a plurality of contexts according to "prev". When Huffman decoding is used for entropy decoding, a code word table used for decoding is determined from a plurality of code word tables according to "prev". Furthermore, in entropy decoding of the value of the third bit b2, a context or a code word table may be determined according to the value of the second bit b1 in addition to "prev".

[0284] Since the tendency of the occurrence probability of each type decoded immediately after type S or E is different from the tendency of the occurrence probability of each type decoded immediately after type C, R, or L, as described above, the control of entropy decoding is switched according to prev determined by the procedure of Fig. 31. This may enable efficient decoding and reduction of the amount of code.

[0285] Furthermore, the content of the event notified by the value of the third bit b2 differs depending on the value of the second bit b1 (if b1 = 0, type S or R, and if b1 = 1, type L or E). Therefore, in entropy decoding of the value of the third bit b2, it may be possible to further reduce the amount of code by determining the context or codeword table according to the value of the second bit b1 in addition to prev.

[0286] Fig. 34 is a flowchart showing a second specific example of connection type encoding according to this embodiment. That is, Fig. 34 shows an example of the encoding process (S103) in Fig. 31. In this example, bit patterns 0, 10, 110, 1110, and 1111 are assigned to types C, R, E, S, and L, respectively, and entropy encoding is performed starting from the most significant bit.

[0287] First, it is determined whether the connection type of the triangle to be coded is Type C (S401). If the connection type is Type C (Yes in S401), the value of the first bit b0 is coded as 0 by entropy coding according to prev (S405). If the connection type is not Type C (No in S401: any of L, E, R, and S), the value of the first bit b0 is coded as 1 by entropy coding according to prev (S402).

[0288] Next, it is determined whether the connection type of the triangle to be coded is Type R (S403). If the connection type is Type R (Yes in S403), the value of the second bit b1 is coded as 0 by entropy coding according to prev (S408). If the connection type is not Type R (No in S403: any of L, E, and S), the value of the second bit b1 is coded as 1 by entropy coding according to prev (S404).

[0289] Next, it is determined whether the connection type of the triangle to be coded is Type E (S406). If the connection type is Type E (Yes in S406), the value of the third bit b2 is coded as 0 by entropy coding according to prev (S410). If the connection type is not Type E (No: either L or S in S406), the value of the third bit b2 is coded as 1 by entropy coding according to prev (S407).

[0290] Next, it is determined whether the connection type of the triangle to be coded is Type S (S409). If the connection type is Type S (Yes in S409), the value of the fourth bit b3 is set to 0 by entropy coding according to prev (S412). If the connection type is not Type S (No: L in S409), the value of the fourth bit b3 is set to 1 by entropy coding according to prev (S411).

[0291] When arithmetic coding is used for entropy coding, a context used to determine the predicted occurrence probability of information to be coded is determined from among a plurality of contexts according to prev. When Huffman coding is used for entropy coding, a code word table used for coding is determined from among a plurality of code word tables according to prev.

[0292] The tendency of the occurrence probability of each type coded immediately after type S or E is different from the tendency of the occurrence probability of each type coded immediately after type C, R, or L. Therefore, as described above, the control of entropy coding is switched according to prev determined by the procedure of Fig. 31. This may make it possible to improve the coding efficiency of entropy coding.

[0293] In the above description, bit patterns 0, 10, 110, 1110, and 1111 are assigned to types C, R, E, S, and L, respectively. However, the types to be determined in the determination processes (S401, S403, S406, and S409) may be set in accordance with the bit pattern allocation method. This allows for arbitrary allocation.

[0294] Also, a shorter bit pattern may be assigned to a type that occurs more frequently, which may improve coding efficiency. Therefore, bit patterns 0 and 10 may be assigned to types C and R that occur more frequently, respectively.

[0295] Fig. 35 is a flowchart showing a second specific example of connection type decoding according to this embodiment. That is, Fig. 35 shows an example of the decoding process (S103) in Fig. 31. In this example, entropy decoding is performed on a bit string encoded by the encoding method described with reference to Fig. 34, and one of types C, L, E, R, and S is set as the connection type of the triangle to be decoded.

[0296] First, the value of the first bit b0 is decoded by entropy decoding according to prev (S501). Then, it is determined whether the value of the first bit b0 is 0 (S502). If the value of the first bit b0 is 0 (Yes in S502), the connection type of the triangle to be decoded is set to Type C (S505). If the value of the first bit b0 is not 0 (No in S502), the value of the second bit b1 is decoded by entropy decoding according to prev (S503).

[0297] Next, it is determined whether the value of the second bit b1 is 0 (S504). If the value of the second bit b1 is 0 (Yes in S504), the connection type of the triangle to be decoded is set to type R (S508). If the value of the second bit b1 is not 0 (No in S504), the value of the third bit b2 is decoded by entropy decoding according to prev (S506).

[0298] Next, it is determined whether the value of the third bit b2 is 0 (S507). If the value of the third bit b2 is 0 (Yes in S507), the connection type of the triangle to be decoded is set to Type E (S511). If the value of the third bit b2 is not 0 (No in S507), the value of the fourth bit b3 is decoded by entropy decoding according to prev (S509).

[0299] Next, it is determined whether the value of the fourth bit b3 is 0 (S510). If the value of the fourth bit b3 is 0 (Yes in S510), the connection type of the triangle to be decoded is set to type S (S513). If the value of the fourth bit b3 is not 0 (No in S510), the connection type of the triangle to be decoded is set to type L (S512).

[0300] When arithmetic decoding is used for entropy decoding, a context used to determine the predicted occurrence probability of information to be decoded is determined from among a plurality of contexts according to prev. When Huffman decoding is used for entropy decoding, a code word table used for decoding is determined from among a plurality of code word tables according to prev.

[0301] Since the tendency of the occurrence probability of each type decoded immediately after type S or E is different from the tendency of the occurrence probability of each type decoded immediately after type C, R, or L, as described above, the control of entropy decoding is switched according to prev determined by the procedure of Fig. 31. This may enable efficient decoding and reduction of the amount of code.

[0302] In the above description, bit patterns 0, 10, 110, 1110, and 1111 are assigned to C, R, E, S, and L, respectively. However, the types set in the setting processes (S505, S508, S511, S513, and S512) may be set in accordance with the bit pattern assignment method. This allows for arbitrary assignment.

[0303] Furthermore, a short bit pattern may be assigned to a type that occurs frequently, which may reduce the amount of code. Therefore, bit patterns 0 and 10 may be assigned to types C and R that occur frequently, respectively.

[0304] In the examples described with reference to Figures 31 to 35, the value of the variable prev is set according to the type of triangle that was previously coded. Then, parameters for controlling entropy coding of the triangle's connectivity type are determined according to prev. Then, entropy coding of the connectivity type of the triangle to be coded is performed according to the determined parameters.

[0305] 36 is a flowchart showing another example of the connection type encoding process and decoding process according to this embodiment. Specifically, in this example, the variable c_idx can be used instead of prev. For example, the value of c_idx is set according to the connection type of the triangle encoded immediately before, as well as the connection types of n triangles (n is an integer equal to or greater than 1) encoded in the past. In other words, the value of c_idx is set according to the connection types of the n+1 triangles that have already been encoded.

[0306] 36 will be described step by step. First, the variables c_idx and type[0] to type[n] (n is an integer equal to or greater than 1) are each set to an initial value (for example, 0) (S601).

[0307] The variable c_idx is used to determine a parameter for controlling the entropy coding of triangle connectivity types. The variables type[0] through type[n] hold integers corresponding to the connectivity types of the coded triangles. Specifically, type[x] holds an integer corresponding to the x+1 previous connectivity type, e.g., 0 for type C, 1 for type L, 2 for type E, 3 for type R, and 4 for type S.

[0308] Then, the entropy coding of the triangle connection type is repeated (S602 to S609). Specifically, first, parameters for controlling the entropy coding of the triangle connection type are determined according to c_idx, and the determined parameters are used to perform entropy coding of the triangle connection type to be coded (S603).

[0309] For example, when arithmetic coding is used for entropy coding, a context used to determine the expected occurrence probability of information to be coded may be determined from multiple contexts according to c_idx. Specifically, for example, multiple contexts may be held as a one-dimensional array, and an element from the one-dimensional array specified by c_idx may be used as the context.

[0310] Alternatively, when Huffman coding is used for entropy coding, the code word table to be used for coding may be determined from among a plurality of code word tables according to c_idx.

[0311] Next, in a loop process (S604 to S606), each of type[1] to type[n] is updated to an integer corresponding to the connection type of the triangle coded immediately before. That is, the value of type[k-1] is repeatedly copied to type[k] as k increases from n to 1 (S605).

[0312] Next, type[0] is set to an integer corresponding to the connection type of the triangle coded in the current coding process (S603) (S607). Then, c_idx is set according to type[0] to type[n] (S608). This c_idx is used when coding the connection type of the next triangle.

[0313] A specific example of setting c_idx (S608) is as follows. For example, idx_x (0≦x≦n) is determined according to type[x]. Here, type[x] can take five different values. On the other hand, idx_x can take five or fewer different values. Then, using an (n+1)-dimensional array, c_idx may be set by c_idx=array[idx_0][idx_1]...[idx_n]. According to this configuration, a value can be set to c_idx according to a combination of idx_0, idx_1,..., idx_n.

[0314] 37 is a diagram showing an example of the correspondence between the previous connection type and the values ​​used to determine the context according to this embodiment. For example, as shown in Table T106-01, five types of values ​​for type[0] correspond to five types of values ​​for idx_0. In this case, five types of context are used depending on the previous connection type.

[0315] For example, as in Tables T106-02, T106-03, and T106-04, five types of type[0] values ​​are associated with four types of idx_0 values. In this case, four types of context are used depending on the previous connection type. For example, as in Tables T106-05 and T106-06, five types of type[0] values ​​are associated with three types of idx_0 values. In this case, three types of context are used depending on the previous connection type.

[0316] 38 is a relationship diagram showing an example of the correspondence between the connection type two connections before and the values ​​used to determine the context according to this embodiment. For example, as shown in Table T106-11 and Table T106-12, five types of type[1] values ​​are associated with three types of idx_1 values. In this case, three types of context are used depending on the connection type two connections before.

[0317] Also, for example, as shown in Table T106-13, two types of idx_1 values ​​are associated with five types of type[1] values. In this case, two types of context are used depending on the connection type two connections ago.

[0318] For example, when n = 1, c_idx is determined according to the connection type of the immediately preceding triangle and the connection type of the triangle two triangles before that, and the context is determined according to c_idx. In this case, a combination of one table from among tables T106-01 to T106-06 and one table from among tables T106-11 to T106-13 can be used. Then, idx_0 and idx_1 are defined by the two tables, c_idx is determined by c_idx = array[idx_0][idx_1], and the context is determined by c_idx.

[0319] Also, for example, if four types of contexts are used depending on the connection type of the immediately preceding triangle, and three types of contexts are used depending on the connection type of the triangle two triangles before that, 12 types of contexts can be used by combining these.

[0320] In the above example, c_idx is set using an array. However, the method for setting the context used to encode the connection type of the triangle to be encoded according to the connection types of the n+1 triangles that have already been encoded is not limited to the array-based method. For example, c_idx may be determined using the following formula:

[0321]

[0322] Here, N(k) indicates the number of elements of idx_k (k=0, ..., n-1), and is expressed as N(0) = 5 when table T106-01 is used, and as N(1) = 3 when table T106-11 is used. Alternatively, the context may be held in an (n+1)-dimensional array, and the context may be determined directly according to the combination of idx_0, idx_1, ..., idx_n.

[0323] As a result of trials, the inventors have found the following: Type C occurs frequently, and Type C also occurs frequently consecutively. Therefore, by separating the contexts in which Type C is coded immediately before and two codes before it from the contexts in other cases, efficient coding and a reduction in the amount of code may be possible.

[0324] Furthermore, the tendency of the occurrence probability of each type coded immediately after type S differs from the tendency of the occurrence probability of each type coded immediately after other types. Therefore, by separating the context when type S is coded immediately before and two before it from the context when other types are coded, efficient coding and a reduction in the amount of code may be possible.

[0325] Specifically, by using Tables T106-01 to T106-05 and T106-11, it may be possible to perform efficient coding and reduce the amount of code.

[0326] Furthermore, to reduce memory usage, one of the contexts of types C and S may be integrated with a context of the other type. The inventors have found that in this case, by maintaining the independence of the context of type C and integrating the context of type S with a context of the other type, it may be possible to reduce memory usage while suppressing an increase in the amount of coding. Specifically, by using tables T106-06, 12, and 13, it may be possible to reduce memory usage while suppressing an increase in the amount of coding.

[0327] In the example of FIG. 36, type[n] is set to a value between 0 and 4 depending on the triangle connection type. However, another method for identifying the triangle connection type may be used. In addition, in tables T106-01 to T106-06 and T106-11 to T106-13, idx_n is set to a value between 0 and 4 depending on the triangle connection type. However, another method for uniquely determining an array element depending on the triangle connection type may be used.

[0328] Although an example of encoding is shown above, the same example can be applied to decoding by replacing encoding with decoding.

[0329] The shape of the face is not limited to a triangle, and other shapes may be used. For example, polygons other than triangles may be used. Furthermore, whether or not a context is determined based on the connection type of processed polygons may be switched, and the polygons, connection type groups, and context correspondences referenced in determining the context may be switched. In this case, parameters indicating the switching, the correspondences, and the like may be coded.

[0330] Furthermore, in any aspect of the present disclosure, instead of setting a context for arithmetic coding of the connection type of a polygon to be processed according to the connection type of a polygon that has already been processed, a parameter indicating the context may be derived according to the connection type of the polygon that has already been processed. For example, the parameter indicating the context may be derived by calculation using the value of prev or c_idx.

[0331] Fig. 39 is a flowchart showing a third specific example of connection type encoding according to this embodiment. That is, Fig. 39 shows an example of the encoding process (S603) in Fig. 36. In this example, bit patterns 0, 110, 111, 101, and 100 are assigned to types C, L, E, R, and S, respectively, and entropy encoding is performed starting from the most significant bit.

[0332] First, it is determined whether the connection type of the triangle to be coded is Type C (S701). If the connection type is Type C (Yes in S701), the value of the first bit b0 is set to 0 and coded by entropy coding according to c_idx (S702). If the connection type is not Type C (No in S701: any of L, E, R, and S), the value of the first bit b0 is set to 1 and coded by entropy coding according to c_idx (S703).

[0333] Next, it is determined whether the connection type of the previously coded triangle is Type C (S704). If the connection type of the previously coded triangle is Type C (Yes in S704), it is determined that the connection type of the triangle to be coded is neither Type E nor Type L. Then, without coding the value of the second bit b1, it is determined whether the connection type is Type S (S708).

[0334] If the connection type of the triangle previously coded is not type C (No in S704), it is determined whether the connection type of the triangle to be coded is either type R or type S (S705). If the connection type is either type R or type S (Yes in S705), 0 is coded by entropy coding as the value of the second bit b1 according to c_idx (S706).

[0335] Then, it is determined whether the connection type of the triangle to be coded is Type S (S708). If the connection type is Type S (Yes in S708), the value of the third bit b2 is set to 0 and coded by entropy coding according to c_idx (S710). If the connection type is not Type S (No: R in S708), the value of the third bit b2 is set to 1 and coded by entropy coding according to c_idx (S709).

[0336] If the connection type is not one of types R and S (No: L or E in S705), the value of the second bit b1 is coded as 1 by entropy coding according to c_idx (S707).

[0337] Then, it is determined whether the connection type of the triangle to be coded is Type L (S711). If the connection type is Type L (Yes in S711), the value of the third bit b2 is set to 0 and coded by entropy coding according to c_idx (S713). If the connection type is not Type L (No: E in S711), the value of the third bit b2 is set to 1 and coded by entropy coding according to c_idx (S712).

[0338] When arithmetic coding is used for entropy coding, a context used to determine the expected occurrence probability of information to be coded is determined from among a plurality of contexts according to c_idx.When Huffman coding is used for entropy coding, a code word table used for coding is determined from among a plurality of code word tables according to c_idx.

[0339] Furthermore, in the entropy coding of the value of the third bit b2 (S709, S710, S712, and S713), the context or code word table may be determined according to the value of the second bit b1 in addition to c_idx. Here, if the previous connection type is Type C, the connection type of the triangle to be coded is either Type R or Type S, and therefore 0 is used as the value of the second bit b1.

[0340] Neither Type L nor Type E logically occurs immediately after Type C. Therefore, immediately after Type C, it may be possible to encode the triangle connection type with a smaller amount of code by omitting encoding the second bit b1 that identifies Type L and Type E and encoding only the first bit b0 and the third bit b2.

[0341] In addition, instead of omitting the encoding of the second bit b1, it may be possible to reduce the amount of code for triangle connection types by always using the same context to encode the second bit b1 immediately after type C.

[0342] Furthermore, the content of the event notified by the value of the third bit b2 differs depending on the value of the second bit b1 (if b1 = 0, type S or R, and if b1 = 1, type L or E). Therefore, in entropy coding of the value of the third bit b2, by determining the context or code word table according to the value of the second bit b1 in addition to c_idx, it may be possible to further reduce the amount of code.

[0343] Alternatively, bit patterns 0, 011, 111, 101, and 001 may be assigned to types C, L, E, R, and S, respectively. Alternatively, unnecessary leading zeros may be omitted, and bit patterns 0, 11, 111, 101, and 1 may be assigned to types C, L, E, R, and S, respectively. Then, entropy coding may be performed starting from the least significant bit.

[0344] This allows types C, L, E, R and S to be assigned bit patterns 0, 110, 111, 101 and 100, respectively, and an operation corresponding to the above-described operation in which entropy coding is performed starting from the most significant bit can be performed.

[0345] In the above, encoding bits may include inputting the bits into a coding engine, where the bits may be compressed into fewer bits.

[0346] Fig. 40 is a flowchart showing a third specific example of connection type decoding according to this embodiment. That is, Fig. 40 shows an example of the decoding process (S603) in Fig. 36. In this example, entropy decoding is performed on a bit string encoded by the encoding method described with reference to Fig. 39, and one of types C, L, E, R, and S is set as the connection type of the triangle to be decoded.

[0347] First, the value of the first bit b0 is decoded by entropy decoding according to c_idx (S801). Then, it is determined whether the value of the first bit b0 is 0 (S802). If the value of the first bit b0 is 0 (Yes in S802), the connection type of the triangle to be decoded is set to Type C (S803). If the value of the first bit b0 is not 0 (No in S802), it is determined whether the connection type of the triangle decoded immediately before is Type C (S804).

[0348] If the connection type of the previously decoded triangle is Type C (Yes in S804), it is determined that the connection type of the triangle to be decoded is neither Type E nor Type L. Then, the value of the second bit b1 is set to 0 without being decoded (S805). If the connection type of the previously decoded triangle is not Type C (No in S804), the value of the second bit b1 is decoded by entropy decoding according to c_idx (S806).

[0349] Next, the value of the third bit b2 is decoded by entropy decoding according to c_idx (S807). Then, the bit patterns of the second bit b1 and the third bit b2 are determined (S808-S810). Then, according to the determination result, the connection type of the triangle to be decoded is set to one of types L, E, R, or S (S811-S814).

[0350] Specifically, if the value of the second bit b1 and the value of the third bit b2 are 0 and 0, respectively (Yes in S808 and Yes in S809), the connection type is set to type S (S812). If the value of the second bit b1 and the value of the third bit b2 are 0 and 1, respectively (Yes in S808 and No in S809), the connection type is set to type R (S811).

[0351] If the value of the second bit b1 and the value of the third bit b2 are 1 and 0, respectively (No in S808 and Yes in S810), the connection type is set to Type L (S814). If the value of the second bit b1 and the value of the third bit b2 are 1 and 1, respectively (No in S808 and No in S810), the connection type is set to Type E (S813).

[0352] When arithmetic decoding is used for entropy decoding, a context used to determine the expected occurrence probability of information to be decoded is determined from a plurality of contexts according to c_idx. When Huffman decoding is used for entropy decoding, a code word table used for decoding is determined from a plurality of code word tables according to c_idx. Furthermore, in the entropy decoding of the value of the third bit b2 (S807), the context or code word table may be determined according to the value of the second bit b1 in addition to c_idx.

[0353] Neither Type L nor Type E logically occurs immediately after Type C. Therefore, immediately after Type C, it may be possible to decode the triangle connection type with a smaller amount of code by omitting the decoding of the second bit b1 that identifies Type L and Type E and decoding only the first bit b0 and the third bit b2.

[0354] In addition, instead of omitting the decoding of the second bit b1, it may be possible to reduce the amount of code for the triangular connection type by always using the same context to decode the second bit b1 immediately after type C.

[0355] Furthermore, the content of the event notified by the value of the third bit b2 differs depending on the value of the second bit b1 (if b1 = 0, type S or R, and if b1 = 1, type L or E). Therefore, in entropy decoding of the value of the third bit b2, by determining the context or code word table according to the value of the second bit b1 in addition to c_idx, it may be possible to further reduce the amount of code.

[0356] Alternatively, bit patterns 0, 011, 111, 101, and 001 may be assigned to types C, L, E, R, and S, respectively. Alternatively, unnecessary leading zeros may be omitted, and bit patterns 0, 11, 111, 101, and 1 may be assigned to types C, L, E, R, and S, respectively. Then, entropy decoding may be performed in order from the least significant bit.

[0357] As a result, bit patterns 0, 110, 111, 101 and 100 are assigned to types C, L, E, R and S, respectively, and an operation corresponding to the above-described operation in which entropy decoding is performed in order from the most significant bit can be performed.

[0358] Fig. 41 is a flowchart showing a fourth specific example of connection type encoding according to this embodiment. That is, Fig. 41 shows an example of the encoding process (S603) in Fig. 36. In this example, bit patterns 0, 10, 110, 1110, and 1111 are assigned to types C, R, S, E, and L, respectively, and entropy encoding is performed starting from the most significant bit.

[0359] First, it is determined whether the connection type of the triangle to be encoded is type C (S901). If the connection type is type C (Yes in S901), 0 is encoded as the value of the first bit b0 by entropy encoding according to c_idx (S904). If the connection type is not type C (No in S901: any of L, E, R, and S), 1 is encoded as the value of the first bit b0 by entropy encoding according to c_idx (S902).

[0360] Next, it is determined whether the connection type of the triangle to be encoded is type R (S903). If the connection type is type R (Yes in S903), 0 is encoded as the value of the second bit b1 by entropy encoding according to c_idx (S906). If the connection type is not type R (No: L, E, or S in S903), 1 is encoded as the value of the second bit b1 by entropy encoding according to c_idx (S905).

[0361] Next, it is determined whether the connection type of the triangle previously coded (immediately before) is Type C (S907). If the connection type of the triangle previously coded is Type C (Yes in S907), it is determined that the connection type of the triangle to be coded is neither Type E nor Type L. Then, the value of the third bit b2 is not coded, and the process ends.

[0362] If the connection type of the previously coded triangle is not Type C (No in S907), it is determined whether the connection type of the triangle to be coded is Type S (S908). If the connection type is Type S (Yes in S908), a value of 0 is coded as the third bit b2 by entropy coding according to c_idx (S913). If the connection type is not Type S (No in S908: either L or E), a value of 1 is coded as the third bit b2 by entropy coding according to c_idx (S909).

[0363] Next, it is determined whether the connection type of the triangle to be coded is Type E (S910). If the connection type is Type E (Yes in S910), the value of the fourth bit b3 is set to 0 and coded by entropy coding according to c_idx (S912). If the connection type is not Type E (No: L in S910), the value of the fourth bit b3 is set to 1 and coded by entropy coding according to c_idx (S911).

[0364] When arithmetic coding is used for entropy coding, a context used to determine the expected occurrence probability of information to be coded is determined from among a plurality of contexts according to c_idx.When Huffman coding is used for entropy coding, a code word table used for coding is determined from among a plurality of code word tables according to c_idx.

[0365] Neither Type L nor Type E logically occurs immediately after Type C. Therefore, immediately after Type C, it may be possible to encode the triangle connection type with a smaller amount of code by omitting encoding the third bit b2 that identifies Type L and Type E and encoding only the first bit b0 and the second bit b1.

[0366] In addition, instead of omitting the encoding of the third bit b2, it may be possible to reduce the amount of code for triangle connection types by always using the same context to encode the third bit b2 immediately after type C.

[0367] In the above description, bit patterns 0, 10, 110, 1110, and 1111 are assigned to types C, R, S, E, and L, respectively. However, the types to be determined in the determination processes (S901, S903, S908, and S910) may be set in accordance with the bit pattern allocation method. This allows for arbitrary allocation.

[0368] For example, as explained above, bit patterns 1110 and 1111 may be assigned to types E and L, respectively. Alternatively, bit patterns 1111 and 1110 may be assigned to types E and L, respectively. This may make it possible to omit encoding the third bit b2 immediately after type C.

[0369] Alternatively, a shorter bit pattern may be assigned to a type that occurs more frequently. This may enable more efficient encoding and a reduction in the amount of code. Therefore, as explained above, bit patterns 0 and 10 may be assigned to types C and R that occur more frequently, respectively. Alternatively, bit patterns 10 and 0 may be assigned to types C and R, respectively.

[0370] Fig. 42 is a flowchart showing a fourth specific example of connection type decoding according to this embodiment. That is, Fig. 42 shows an example of the decoding process (S603) in Fig. 36. In this example, entropy decoding is performed on a bit string encoded by the encoding method described with reference to Fig. 41, and one of types C, L, E, R, and S is set as the connection type of the triangle to be decoded.

[0371] First, the value of the first bit b0 is decoded by entropy decoding according to c_idx (S1001). Then, it is determined whether the value of the first bit b0 is 0 (S1002). If the value of the first bit b0 is 0 (Yes in S1002), the connection type of the triangle to be decoded is set to Type C (S1005). If the value of the first bit b0 is not 0 (No in S1002), the value of the second bit b1 is decoded by entropy decoding according to c_idx (S1003).

[0372] Next, it is determined whether the value of the second bit b1 is 0 (S1004). If the value of the second bit b1 is 0 (Yes in S1004), the connection type of the triangle to be decoded is set to Type R (S1010). If the value of the second bit b1 is not 0 (No in S1004), it is determined whether the connection type of the triangle decoded immediately before is Type C (S1006).

[0373] If the connection type of the previously decoded triangle is Type C (Yes in S1006), the connection type of the triangle to be decoded is determined to be Type S. Then, the value of the third bit b2 is not decoded, and the connection type of the triangle to be decoded is set to Type S (S1011). If the connection type of the previously decoded triangle is not Type C (No in S1006), the value of the third bit b2 is decoded by entropy decoding according to c_idx (S1007).

[0374] Next, it is determined whether the value of the third bit b2 is 0 (S1008). If the value of the third bit b2 is 0 (Yes in S1008), the connection type of the triangle to be decoded is set to type S (S1011). If the value of the third bit b2 is not 0 (No in S1008), the value of the fourth bit b3 is decoded by entropy decoding according to c_idx (S1009).

[0375] Next, it is determined whether the value of the fourth bit b3 is 0 (S1012). If the value of the fourth bit b3 is 0 (Yes in S1012), the connection type of the triangle to be decoded is set to Type E (S1014). If the value of the fourth bit b3 is not 0 (No in S1012), the connection type of the triangle to be decoded is set to Type L (S1013).

[0376] When arithmetic decoding is used for entropy decoding, a context used to determine the expected occurrence probability of information to be decoded is determined from among multiple contexts according to c_idx.When Huffman decoding is used for entropy decoding, a code word table used for decoding is determined from among multiple code word tables according to c_idx.

[0377] Neither Type L nor Type E logically occurs immediately after Type C. Therefore, immediately after Type C, it may be possible to decode the triangle connection type with a smaller amount of code by omitting the decoding of the third bit b2 that identifies Type L and Type E and decoding only the first bit b0 and the second bit b1.

[0378] In addition, instead of omitting the decoding of the third bit b2, it may be possible to reduce the amount of code for the triangular connection type by always using the same context to decode the third bit b2 immediately after type C.

[0379] In the above description, bit patterns 0, 10, 110, 1110, and 1111 are assigned to types C, R, S, E, and L, respectively. However, the types to be set in the setting processes (S1005, S1010, S1011, S1014, and S1013) may be set in accordance with the bit pattern assignment method. This allows for arbitrary assignment.

[0380] For example, as explained above, bit patterns 1110 and 1111 may be assigned to types E and L, respectively. Alternatively, bit patterns 1111 and 1110 may be assigned to types E and L, respectively. This may allow omitting the decoding of the third bit b2 immediately after type C.

[0381] Alternatively, a shorter bit pattern may be assigned to a type that occurs more frequently. This may reduce the amount of code. Therefore, as explained above, bit patterns 0 and 10 may be assigned to types C and R that occur more frequently, respectively. Alternatively, bit patterns 10 and 0 may be assigned to types C and R, respectively.

[0382] 43 is a relationship diagram showing another example of the correspondence between the connection type two steps before and the value used to determine the context according to this embodiment. For example, in the setting process (S608) of FIG. 36, when n=1, c_idx is set using the value shown in FIG. 38 according to each type of triangle. The value shown in FIG. 43 may be used instead of the value shown in FIG. 38.

[0383] For example, as in tables T107-01 to T107-03 and T107-14, five types of type[1] values ​​are associated with three types of idx_1 values. In this case, three types of context are used depending on the connection type two types prior. Also, as in tables T107-04 and T107-05, five types of type[1] values ​​are associated with two types of idx_1 values. In this case, two types of context are used depending on the connection type two types prior.

[0384] Also, for example, as shown in Table T107-06, one type of idx_1 value is associated with five types of type[1] values. In this case, one type of context is used depending on the connection type two connections before.

[0385] Furthermore, the table used to determine idx_1 may be determined according to the connection type (type[0]) of the immediately preceding (previously) encoded / decoded triangle. Specifically, when idx_0 is determined using table T106-01, the table for determining idx_1 may be determined as follows.

[0386] (1) If the connectivity type of the previously encoded or decoded triangle is Type C, Table T107-05 or Table T107-06 may be used.

[0387] (2) If the connectivity type of the previously coded or decoded triangle is type S, then Table T107-06 may be used.

[0388] (3) If the connectivity type of the previously encoded or decoded triangle is Type L, Table T107-01 or Table T107-06 may be used.

[0389] (4) If the connectivity type of the previously coded or decoded triangle is Type R, Table T107-02, Table T107-14, or Table T106-13 may be used.

[0390] (5) If the connectivity type of the previously encoded or decoded triangle is Type E, Table T107-03 or Table T107-04 may be used.

[0391] As a result of trials, the inventors have discovered the following: If the previous (immediately preceding) connection type is different, the correlation between the connection type of the triangle to be coded and the connection type two blocks prior to it will differ. Therefore, by switching the type two blocks prior to it that is made independent in determining the context depending on the type of the previous (immediately preceding) triangle, it may be possible to improve the coding efficiency of entropy coding. Therefore, by using the above combination, it may be possible to reduce memory usage while suppressing a decrease in the coding efficiency of entropy coding.

[0392] Furthermore, in any aspect of the present disclosure, instead of setting a context for arithmetic coding of the connection type of a polygon to be processed according to the connection type of a polygon that has already been processed, a parameter indicating the context may be derived according to the connection type of the polygon that has already been processed. For example, the parameter indicating the context may be derived by calculation using the value of prev or c_idx.

[0393] As described above, if the connection type of the immediately preceding triangle is Type C, the connection type of the triangle being processed is neither Type L nor Type E. Specifically, for example, as shown in Figure 29, if the connection type of the triangle being processed is Type C, an unprocessed triangle is connected to the left of the next triangle. On the other hand, if the connection type of the triangle is Type L or E, no unprocessed triangle is connected to the left of the triangle in question. Therefore, if the connection type of the triangle being processed is Type C, the connection type of the next triangle is neither Type L nor Type E.

[0394] Therefore, if the connection type of the immediately preceding triangle is Type C, information is encoded to identify the connection type of the triangle to be processed from the candidate set of Types C, R, and S. This may make it possible to reduce the amount of code compared to encoding information to identify the connection type from the candidate set of all types.

[0395] Here, when the connection type of the immediately preceding triangle is type C, the candidate set of types C, R, and S is used as the candidate set of connection types for the triangle to be processed. However, the present invention is not limited to this example, and the candidate set of connection types to be processed may be determined by the previously processed connection type.

[0396] In the above example, values ​​such as those shown in Tables T106-01 to T106-06 in Fig. 37 and Tables T107-01 to T107-06 and T107-14 in Fig. 43 are set to idx_0 and idx_1 according to the connection types of the n+1 (here, n=1) triangles that have already been coded. Then, c_idx is set according to idx_0 and idx_1 (S608 in Fig. 36). Furthermore, c_idx may be set according to Crun, which indicates the number of consecutively coded type C triangles, in addition to the connection types of the n+1 triangles that have already been coded.

[0397] Fig. 44 is a flowchart showing yet another example of the connection-type encoding process and decoding process according to this embodiment. In this procedure, processes related to the initialization, update, and use of Crun are added. The other parts are the same as the procedure in Fig. 36. Here, the added parts will be described.

[0398] Specifically, in the initialization process (S1101), Crun is set to 0. Also, in the update process (S1108 to S1110), 1 is added to Crun (S1109) or Crun is set to 0 (S1110) depending on the triangle connection type. Also, in the c_idx setting process (S1111), Crun is used.

[0399] As a result, the number of consecutively coded type Cs is set in Crun. Then, in the c_idx setting process (S1111), c_idx is set using the connection types of the coded n+1 triangles as well as the number of consecutively coded type Cs.

[0400] Here, the number of consecutively encoded Type Cs specifically corresponds to the number of one or more Type Cs consecutively encoded up to the immediately preceding (previous) one, and corresponds to the number of one or more connection types consecutively encoded as Type C up to the immediately preceding one. The number of consecutively encoded Type Cs can also be expressed as the number of consecutive Type Cs, the consecutive length of Type Cs, the number of consecutive values ​​equal to Type C, the consecutive length of values ​​equal to Type C, or the number of consecutively encoded Type C triangles, etc. Type C corresponds to the above-mentioned corner type.

[0401] 45 is a relationship diagram showing an example of the correspondence relationship between at least one of the connection type two before and the number of consecutive Type Cs and the values ​​used to determine a context according to the present embodiment. In this example, three different values ​​are used for idx_1 and three different contexts are used depending on at least one of the connection type two before and the number of consecutive Type Cs.

[0402] Specifically, if the connection type (type[0]) of the immediately preceding (previously) coded triangle is Type C, in the c_idx setting process (S1111), idx_1 for setting c_idx may be determined using the example of Table T108-01 in Fig. 45. In the example of Table T108-01, if Crun is less than the threshold TH(0), idx_1 is set to 0, if Crun is equal to or greater than the threshold TH(0) and less than the threshold TH(1), idx_1 is set to 1, and if Crun is equal to or greater than the threshold TH(1), idx_1 is set to 2.

[0403] Here, threshold value TH(0) is smaller than threshold value TH(1). As a specific set value, threshold value TH(0) may be 2 or a value greater than or equal to 2 and less than or equal to 4. Threshold value TH(1) may be 7 or a value greater than or equal to 5.

[0404] When a value of 2 or close to 2 is used as the threshold value TH(0), different contexts are used for cases where there are no consecutive or few consecutive type Cs and cases where there are consecutive type Cs. Therefore, it may be possible to encode the connection type of the next triangle according to the occurrence probability of the connection type of the next triangle in each case. This may improve coding efficiency.

[0405] Note that when the threshold TH(0) is 2, the condition Crun<TH(0) in the example of Table T108-01 may be determined by another method that achieves the same result. For example, the condition may be determined such that type[0] is a value representing type C and type[1] is a value representing a type other than C.

[0406] Furthermore, in cases where type C is not consecutive, type[1] is frequently type R. Therefore, even if idx_1 is set as in the example of table T108-02 in FIG. 45, it is possible to obtain the same effect as when threshold TH(0) is 2 in table T108-1.

[0407] In general meshes, the number of consecutive Type Cs is often no more than 6 to 8. On the other hand, in large meshes with a large number of triangles constituting the mesh, the number of consecutive Type Cs can exceed 100.

[0408] Therefore, when a value of 7 or close to 7 is used as the threshold TH(1), different contexts are used for the case of a general mesh where the number of consecutive Type Cs is about 6 to 8 or less, and the case of a large mesh where an extremely large number of consecutive Type Cs occur. Therefore, it may be possible to encode the connection type of the next triangle according to the occurrence probability of the connection type of the next triangle in each case. This may improve coding efficiency.

[0409] Although the above example shows encoding, the same example can be applied to decoding by replacing encoding with decoding.

[0410] 44, the table used to determine idx_1 may be determined according to the connection type (type[0]) of the immediately preceding triangle that was coded / decoded. Specifically, when idx_0 is determined using table T106-01, the table for determining idx_1 may be determined as follows.

[0411] (1) If the connection type of the triangle previously coded or decoded is Type C, Table T108-01 may be used. In this case, c_idx may be determined to be 0, 1, or 2 depending on the number of consecutive Type C triangles.

[0412] (2) If the connection type of the previously coded or decoded triangle is type S, Table T107-06 may be used. In this case, c_idx may be determined to be 3.

[0413] (3) If the connection type of the previously coded or decoded triangle is Type L, Table T107-06 may be used. In this case, c_idx may be determined to be 4.

[0414] (4) If the connection type of the triangle previously coded or decoded is Type R, Table T107-14 may be used. In this case, c_idx may be determined to be 5, 6, or 7 depending on the connection type of the triangle coded or decoded two blocks before.

[0415] (5) If the connectivity type of the previously coded or decoded triangle is Type E, Table T107-06 may be used. In this case, c_idx may be determined to be 8.

[0416] As a result of trials, the inventors have discovered the following: If the immediately previous connection type (type[0]) is different, the correlation between the connection type of the triangle to be coded and the connection type two prior to that (type[1]) or the connection type three or more prior to that (type[n], n≧2) will differ. Therefore, by switching the method for determining idx_1 used to determine the context depending on the immediately previous connection type, it may be possible to improve the coding efficiency of entropy coding.

[0417] Therefore, by using the above combination, it may be possible to reduce the memory usage while suppressing a decrease in the coding efficiency of entropy coding.

[0418] In the above combination, if the connection type of the triangle that was previously coded / decoded is Type C, the same effect may be obtained even if idx_1 is determined using Table T108-02.

[0419] Fig. 46 is a flowchart showing a fifth specific example of connection type encoding according to this embodiment. That is, Fig. 46 shows an example of the encoding process (S603) in Fig. 36 and the encoding process (S1103) in Fig. 44.

[0420] In this example, the character strings of types C, L, E, R, and S are coded from the end to the beginning, i.e., in reverse order. That is, for the multiple triangles that make up the mesh, the connection type of each triangle is determined according to the processing order, and the connection type is coded in the reverse order of the processing order used to determine the connection type. Also, in this example, bit patterns 0, 110, 111, 101, and 100 are assigned to types C, L, E, R, and S, respectively, and entropy coding is performed on each bit pattern starting from the most significant bit.

[0421] Specifically, first, it is determined whether the connection type of the immediately preceding (previously) coded triangle is one of types L and E (S1201).

[0422] If the connection type of the previously coded triangle is Type L or Type E (Yes in S1201), it is determined that the connection type of the triangle to be coded is not Type C. Then, without coding the value of the first bit b0, it is determined whether the connection type of the triangle to be coded is either Type R or Type S (S1205).

[0423] If the connection type of the previously coded triangle is neither Type L nor Type E (No in S1201), it is determined whether the connection type of the triangle to be coded is Type C (S1202). If the connection type of the triangle to be coded is Type C (Yes in S1202), 0 is coded by entropy coding as the value of the first bit b0 according to c_idx (S1203).

[0424] If the connection type of the triangle to be coded is not type C (No: L, E, R, or S in S1202), the value of the first bit b0 is set to 1 by entropy coding according to c_idx (S1204). Then, it is determined whether the connection type of the triangle to be coded is either type R or type S (S1205).

[0425] If the connection type of the triangle to be coded is Type R or Type S (Yes in S1205), the value of the second bit b1 is set to 0 by entropy coding according to c_idx (S1206). Then, it is determined whether the connection type of the triangle to be coded is Type S (S1208).

[0426] If the connection type of the triangle to be coded is Type S (Yes in S1208), the value of the third bit b2 is set to 0 and coded by entropy coding according to c_idx (S1210). If the connection type of the triangle to be coded is not Type S (No: R in S1208), the value of the third bit b2 is set to 1 and coded by entropy coding according to c_idx (S1209).

[0427] If the connection type of the triangle to be coded is neither Type R nor Type S (No: L or E in S1205), the value of the second bit b1 is set to 1 by entropy coding according to c_idx (S1207). Then, it is determined whether the connection type of the triangle to be coded is Type L (S1211).

[0428] If the connection type of the triangle to be coded is Type L (Yes in S1211), the value of the third bit b2 is set to 0 and coded by entropy coding according to c_idx (S1213). If the connection type of the triangle to be coded is not Type L (No: E in S1211), the value of the third bit b2 is set to 1 and coded by entropy coding according to c_idx (S1212).

[0429] When arithmetic coding is used for entropy coding, a context used to determine the expected occurrence probability of information to be coded is determined from a plurality of contexts according to c_idx. When Huffman coding is used for entropy coding, a code word table used for coding is determined from a plurality of code word tables according to c_idx.

[0430] Furthermore, in entropy coding of the value of the third bit b2 (S1209, S1210, S1212, and S1213), the context or code word table may be determined according to the value of the second bit b1 in addition to c_idx.

[0431] Type C does not logically occur immediately after Type L or Type E. Therefore, immediately after Type L or Type E, it may be possible to encode the triangle connection type with a smaller amount of code by omitting encoding the first bit b0 that identifies Type C and encoding only the second bit b1 and the third bit b2.

[0432] In addition, instead of omitting the encoding of the first bit b0, it may be possible to reduce the amount of code for triangle connection types by always using the same context to encode the first bit b0 immediately after type L or type E.

[0433] Furthermore, the content of the event notified by the value of the third bit b2 differs depending on the value of the second bit b1 (if b1 = 0, type S or R, and if b1 = 1, type L or E). Therefore, in entropy coding of the value of the third bit b2, by determining the context or code word table according to the value of the second bit b1 in addition to c_idx, it may be possible to further reduce the amount of code.

[0434] Alternatively, bit patterns 0, 011, 111, 101, and 001 may be assigned to types C, L, E, R, and S, respectively. Alternatively, unnecessary leading zeros may be omitted, and bit patterns 0, 11, 111, 101, and 1 may be assigned to types C, L, E, R, and S, respectively. Entropy coding may then be performed starting from the least significant bit of each bit pattern.

[0435] This allows types C, L, E, R and S to be assigned bit patterns 0, 110, 111, 101 and 100, respectively, and an operation corresponding to the above-described operation in which entropy coding is performed starting from the most significant bit can be performed.

[0436] In the above, encoding bits may include inputting the bits into a coding engine, where the bits may be compressed into fewer bits.

[0437] Fig. 47 is a flowchart showing a fifth specific example of connection-type decoding according to this embodiment. That is, Fig. 47 shows an example of the decoding process (S603) in Fig. 36 and the decoding process (S1103) in Fig. 44.

[0438] In this example, entropy decoding is performed on a bit string that has been coded using the coding method described with reference to Figure 46. That is, entropy decoding of a bit string in which character strings of types C, L, E, R, and S have been coded from the end to the beginning (i.e., in reverse order) is performed in the same decoding order as the coding order. Then, one of types C, L, E, R, and S is set as the connection type of the triangle to be decoded.

[0439] Specifically, first, it is determined whether the connection type of the triangle that has been decoded immediately before is one of types L and E (S1301).

[0440] If the connection type of the previously decoded triangle is Type L or Type E (Yes in S1301), it is determined that the connection type of the triangle to be decoded is not Type C. Then, the value of the first bit b0 is not decoded, and the value of the second bit b1 is decoded by entropy decoding according to c_idx (S1305). In this case, the value of the first bit b0 may be regarded as 1.

[0441] If the connection type of the previously decoded triangle is neither Type L nor Type E (No in S1301), the value of the first bit b0 is decoded by entropy decoding according to c_idx (S1302), and it is then determined whether the value of the first bit b0 is 0 (S1303).

[0442] If the value of the first bit b0 is 0 (Yes in S1303), the connection type of the triangle to be decoded is set to Type C (S1304). If the value of the first bit b0 is not 0 (No in S1303), the value of the second bit b1 is decoded by entropy decoding according to c_idx (S1305).

[0443] Next, the value of the third bit b2 is decoded by entropy decoding according to c_idx (S1306). Then, the bit patterns of the second bit b1 and the third bit b2 are determined (S1307 to S1309). Then, according to the determination result, the connection type of the triangle to be decoded is set to one of types L, E, R, or S (S1310 to S1313).

[0444] Specifically, if the value of the second bit b1 and the value of the third bit b2 are 0 and 0, respectively (Yes in S1307 and Yes in S1308), the connection type is set to type S (S1311). If the value of the second bit b1 and the value of the third bit b2 are 0 and 1, respectively (Yes in S1307 and No in S1308), the connection type is set to type R (S1310).

[0445] If the value of the second bit b1 and the value of the third bit b2 are 1 and 0, respectively (No in S1307 and Yes in S1309), the connection type is set to Type L (S1313). If the value of the second bit b1 and the value of the third bit b2 are 1 and 1, respectively (No in S1307 and No in S1309), the connection type is set to Type E (S1312).

[0446] When arithmetic decoding is used for entropy decoding, a context used to determine the expected occurrence probability of information to be decoded is determined from a plurality of contexts according to c_idx. When Huffman decoding is used for entropy decoding, a code word table used for decoding is determined from a plurality of code word tables according to c_idx. Furthermore, in the entropy decoding of the value of the third bit b2 (S1306), the context or code word table may be determined according to the value of the second bit b1 in addition to c_idx.

[0447] Type C does not logically occur immediately after Type L or Type E. Therefore, immediately after Type L or Type E, it may be possible to decode the triangle connection type with a smaller amount of code by omitting decoding the first bit b0 that identifies Type C and decoding only the second bit b1 and the third bit b2.

[0448] In addition, instead of omitting the decoding of the first bit b0, it may be possible to reduce the amount of code for triangular connection types by always using the same context to decode the first bit b0 immediately after type L or type E.

[0449] Furthermore, the content of the event notified by the value of the third bit b2 differs depending on the value of the second bit b1 (if b1 = 0, type S or R, and if b1 = 1, type L or E). Therefore, in entropy decoding of the value of the third bit b2, by determining the context or code word table according to the value of the second bit b1 in addition to c_idx, it may be possible to further reduce the amount of code.

[0450] Alternatively, bit patterns 0, 011, 111, 101, and 001 may be assigned to types C, L, E, R, and S, respectively. Alternatively, unnecessary leading zeros may be omitted, and bit patterns 0, 11, 111, 101, and 1 may be assigned to types C, L, E, R, and S, respectively. Then, entropy decoding may be performed in order from the least significant bit of each bit pattern.

[0451] As a result, bit patterns 0, 110, 111, 101 and 100 are assigned to types C, L, E, R and S, respectively, and an operation corresponding to the above-described operation in which entropy decoding is performed in order from the most significant bit can be performed.

[0452] As described above, in the processing order used to determine the connection type, if the previous connection type is Type C, the connection type to be processed is neither Type L nor Type E. Therefore, in the reverse processing order used to determine the connection type, if the previous connection type is Type L or E, the connection type to be processed is not Type C.

[0453] Therefore, when encoding and decoding are performed in the reverse order of the processing order for determining the connection type, if the immediately preceding connection type is type L or E, encoding and decoding are performed on information for identifying the connection type to be processed from the candidate set of types L, E, R, and S. This may make it possible to reduce the amount of code compared to encoding and decoding information for identifying the connection type from a candidate set of all types.

[0454] Here, when the immediately preceding connection type is type L or E, a candidate set of types L, E, R, and S is used as the candidate set of connection types for the triangle to be processed. However, the present invention is not limited to this example, and the candidate set of connection types to be processed may be determined by the previously processed connection type.

[0455] For example, five connection types are defined here. However, the connection types may be defined in other ways. For example, four or fewer connection types may be defined, or six or more connection types may be defined. Furthermore, the shape of the faces is not limited to triangles, and may be other polygonal shapes. Then, the connection types may be defined based on the shapes.

[0456] Then, a candidate set of connection types for the face to be processed may be defined from the connection types of previously processed faces according to the shape of each face, the definition of each type, and geometric characteristics. This may enable efficient encoding of the connection types and may reduce the amount of coding. Note that in the above, encoding may be interpreted as decoding.

[0457] 48 is a conceptual diagram showing an example of the transition of stack occupancy according to this embodiment. Specifically, the diagram shows an example of the transition of stack occupancy for holding branch positions of connected strings in entropy encoding and entropy decoding of connection types (types C, L, E, R, or S) associated with each triangle constituting a 3D mesh by an edge breaker.

[0458] The edge breaker follows a plurality of triangles that make up the three-dimensional mesh based on a predetermined rule, and assigns each triangle a connection type of C, L, E, R, or S. Then, a character string consisting of the symbols C, L, E, R, or S indicating the connection type is generated in the order of the right-pointing arrows in the middle row of Fig. 48.

[0459] Here, the symbol C, L, E, R, or S indicating the connection type may also be expressed as a CLERS symbol. The symbol to be processed may also be expressed as a current symbol. A character string consisting of the symbols C, L, E, R, or S may also be expressed as a CLERS character string, a symbol string, or a CLERS symbol string.

[0460] When the CLERS string generated by the edge breaker is processed in the forward direction (the order of the right-pointing arrow in the middle of FIG. 48), the branch position is pushed onto the stack in type S, and the stack occupation amount increases by 1. Then, in type E, to return the processing to the branch position, the branch position held in the stack is popped, and the stack occupation amount decreases by 1.

[0461] However, if type E appears when the stack is empty (occupancy is 0) (for example, at the end of the first connected component or the end of the second connected component in Figure 48), the current symbol is determined to be the end of the connected component.

[0462] After that, if there is subsequent data, processing of a new connected component is started. For example, after the end of the first connected component in Figure 48, it is determined that processing of the next second connected component is to begin. On the other hand, if there is no subsequent data, processing of that mesh data ends. For example, at the end of the second connected component in Figure 48, it is determined that the current symbol is the end of the mesh data, and after processing the current symbol, processing of the mesh data ends.

[0463] A connected component corresponds to a subset of a 3D mesh. That is, a connected component corresponds to at least a portion of a 3D mesh. A 3D mesh is often composed of multiple connected components. A connected component is sometimes abbreviated as CC.

[0464] For example, multiple portions of a three-dimensional mesh that are not contiguous with one another may be defined as multiple connected components. Furthermore, multiple portions of a three-dimensional mesh that are contiguous with one another may be defined as multiple connected components.

[0465] Specifically, it may be efficient to divide multiple contiguous parts of a three-dimensional mesh into multiple connected components and perform encoding and decoding processes for each connected component. In such cases, multiple contiguous parts of a three-dimensional mesh may be divided into multiple connected components, and encoding and decoding processes may be performed for each connected component.

[0466] More specifically, texture information of a 3D mesh is mapped to a 2D image and coded. However, it may be difficult to map the texture information of a 3D mesh to a single 2D image. In such cases, the 3D mesh may be divided into multiple submeshes and coded. Multiple connected components may correspond to such multiple submeshes. Furthermore, a connected component may be an arbitrary unit for defining a series of connection types.

[0467] Here, multiple portions of a CLERS string that correspond to multiple connected components in a 3D mesh may be referred to as multiple connected components. That is, the connected components in the CLERS string correspond to the connected components in the 3D mesh. Each connected component may include multiple connection types of multiple triangles.

[0468] Furthermore, the CLERS character string generated by the edge breaker may be processed in the reverse direction from the end (that is, in the order of the left-pointing arrows in the lower part of FIG. 48) using a so-called spiral reversi or the like.

[0469] In this case, the concatenated string is first cut in type E, and then connected to the cut position in type S. Therefore, the cut position is pushed onto the stack in type E, and the stack occupation amount increases by 1. Then, in type S, the cut position held on the stack to connect the concatenated string to the cut position is popped, and the stack occupation amount decreases by 1.

[0470] Although type E appears at the end of the connected component, the connected string of the connected component to be processed is not connected to any connected component that has already been processed, so the cut position is not pushed onto the stack.

[0471] Here, when the CLERS string is processed in the reverse direction, it is difficult to determine the end or end of a connected component based on the stack occupancy and the connection type of the triangles that appear. Therefore, in this embodiment, a configuration and process for determining the end or end of a connected component when the CLERS string of a mesh composed of multiple connected components is encoded and decoded in reverse order from the end is shown.

[0472] The end of a connected component basically means the end of the connected component in the forward direction. Furthermore, a segment of a connected component means the boundary of the connected component. The end of a connected component corresponds to a segment of the connected component. The segment of the connected component may be determined by determining the end of the connected component. The determination of the end or segment of the connected component may be the determination of the end of the connected component, the determination of the segment of the connected component, or both of these. Here, the expression "determine" may be replaced with expressions such as "determine," "identify," or "detect."

[0473] Furthermore, the CLERS string described in the present disclosure may be generated in advance by a device different from the encoding device 100. In this case, the pre-generated CLERS string or mesh data including the CLERS string may be read from a storage medium and input to the encoding device 100. Alternatively, the pre-generated CLERS string or mesh data including the CLERS string may be received via the network 300 or the like and input to the encoding device 100.

[0474] Furthermore, the CLERS character string input to the encoding device 100 may be arranged in a forward direction or a reverse direction. When the CLERS character string input to the encoding device 100 is arranged in a forward direction, the encoding device 100 may rearrange the CLERS character string arranged in a forward direction to a CLERS character string arranged in a reverse direction. Alternatively, the encoding device 100 may perform a conversion process to convert the CLERS character string arranged in a forward direction into a CLERS character string arranged in a reverse direction.

[0475] Similarly, the CLERS character string output by the decoding device 200 may be arranged in either a forward or reverse direction. When the CLERS character string output by the decoding device 200 is arranged in a forward direction, the decoding device 200 may rearrange the CLERS character string arranged in a backward direction to a CLERS character string arranged in a forward direction. Alternatively, the decoding device 200 may perform a conversion process to convert the CLERS character string arranged in a backward direction into a CLERS character string arranged in a forward direction.

[0476] 49 is a syntax diagram showing a first syntax example of a symbol string including a connected component according to the present embodiment. In this example, when a CLERS string made up of multiple connected components generated by an edge breaker is encoded and decoded in reverse order from the end, whether a symbol is at the end of a connected component or not is notified so that the end or delimiter of the connected component can be determined.

[0477] Here, num_clers_symbol indicates the total number of symbols contained in the CLERS string of the mesh data, num_connected_component indicates the total number of connected components contained in the mesh data, and clers_symbol[i] indicates the i-th symbol when the first symbol of the CLERS string in the forward direction is defined as the 0th symbol.

[0478] Also, end_of_connected_component[i] indicates whether the i-th symbol is the end of the connected component. In this example, if end_of_connected_component[i] is 1, it indicates that the i-th symbol is the end of the connected component, and if it is 0, it indicates that the i-th symbol is not the end of the connected component.

[0479] As shown in this example, the notification of end_of_connected_component[i] may be omitted if it can be determined that the i-th symbol is not the end of the connected component without notification. If the notification is omitted, end_of_connected_component[i] may be set to 0 (not the end of the connected component), considered to be 0, or estimated to be 0.

[0480] For example, the last symbol of a connected component is "E". Therefore, if the current symbol is not "E", the notification of end_of_connected_component[i] may be omitted. Note that clers_symbol[i] may be expressed by a bit pattern assigned to the i-th symbol (connection type) from among bit patterns 0, 110, 111, 101, 100, etc. In the example of Figure 49, 'E' means the bit pattern assigned to type E.

[0481] Furthermore, when processing the last symbol of a connected component, the stack is assumed to be empty (occupancy is 0). Therefore, if the stack is not empty, the notification of end_of_connected_component[i] may be omitted. Note that in the example of Figure 49, stackEmpty indicates that the stack is empty when it is 1, and indicates that the stack is not empty when it is 0.

[0482] Furthermore, if the number of unprocessed (incomplete) connected components among the multiple connected components included in the mesh data is 1, that is, if there are no unprocessed connected components other than the connected component being processed, it is assumed that the last symbol of each connected component has been processed for all connected components. Therefore, in this case, the notification of end_of_connected_component[i] may be omitted.

[0483] In the example of Figure 49, the number of end_of_connected_component[i], which indicates that the i-th symbol is the end of the connected component, is counted as connected_component_count. Then, connected_component_count is compared with num_connected_component. If connected_component_count is equal to num_connected_component, notification is omitted.

[0484] 49, if the number of symbols processed as the end of a connected component is equal to the total number of connected components, notification is omitted. The fact that the number of symbols processed as the end of a connected component is equal to the total number of connected components corresponds to the fact that the symbol corresponding to the end of each connected component has already been encoded or decoded.

[0485] All or some of the above conditions may be applied in combination, or only one of the conditions may be applied.

[0486] As described above, when a symbol is notified, whether the symbol is the end of a connected component or not is notified. This makes it possible to correctly detect the end or boundary of a connected component even when a CLERS string of a mesh made up of multiple connected components is coded and decoded in reverse order from the end.

[0487] Furthermore, regarding notification of whether a symbol is the end of a connected component, if it can be determined that the symbol is not the end of a connected component without notification, the notification is omitted. This may make it possible to encode and decode a CLERS string of a mesh composed of multiple connected components in reverse order from the end with a smaller amount of code.

[0488] 50 is a flowchart showing a first decoding example of a symbol string including a connected component according to this embodiment. In this example, a CLERS character string including multiple connected components is entropy-decoded in reverse order from the end, and a three-dimensional mesh structure is restored. Also, in this example, in loop processing (S1601 to S1608), the symbols of the CLERS character string are decoded one by one, and the three-dimensional mesh structure is restored according to the decoded symbols. Note that the three-dimensional mesh structure here may refer to the structure of a portion of the three-dimensional mesh.

[0489] First, a symbol is decoded by entropy decoding (S1602). The symbol corresponds to clers_symbol[i] in Fig. 49. In this case, the entropy decoding parameters may be determined by referring to the structure of the restored 3D mesh, or may be determined in combination with other examples such as Fig. 47.

[0490] Next, it is determined whether the decoded symbol is likely to be the end of a connected component (S1603). This process corresponds to determining the condition for notifying end_of_connected_component[i] in Fig. 49. Note that the determination may be made based on a combination of some of the conditions shown in Fig. 49, such as when the decoded symbol is "E" and the stack is empty.

[0491] If the decoded symbol is likely to be the end of a connected component (Yes in S1603), entropy decoding is performed on information indicating whether the decoded symbol is the end of a connected component (S1604). The information indicating whether the decoded symbol is the end of a connected component corresponds to end_of_connected_component[i] in Fig. 49. In this case, parameters for entropy decoding may be determined by referring to the structure of the restored 3D mesh.

[0492] Next, based on the information decoded in the decoding process (S1604), it is determined whether the decoded symbol is the end of a connected component (S1605). If the decoded symbol is the end of a connected component (Yes in S1605), a process is performed to complete the 3D mesh structure of the connected component whose decoding has been completed (the connected component including the previously decoded symbol) (S1606).

[0493] Specifically, at the beginning of a connected component, there is a triangle to which no symbol is assigned, i.e., a triangle to which no symbol is encoded. This leading triangle is connected to the 3D mesh structure. That is, the vertex indices of this leading triangle are associated with the vertices of triangles whose symbols are decoded. This association of vertex indices completes the 3D mesh structure of the connected component.

[0494] Next, the structure of the three-dimensional mesh is restored according to the decoded symbols (S1607), as will be described in detail later with reference to FIG.

[0495] Then, all symbols are entropy-decoded, and after the process of restoring the 3D mesh structure according to the decoded symbols is completed, the process of completing the 3D mesh structure of the last connected component that has been decoded is performed (S1609). That is, the above-mentioned process (S1606) is performed on the last connected component that has been decoded.

[0496] As described above, whether a symbol is the end of a connected component or not is notified. This makes it possible to correctly detect the end or boundary of a connected component even when the CLERS string of a mesh made up of multiple connected components is coded and decoded in reverse order from the end. This makes it possible to appropriately select triangles to be linked at the boundary of the connected component, and correctly restore the structure of the 3D mesh.

[0497] Then, when entropy decoding a symbol, it becomes possible to determine the entropy decoding parameters by referring to the structure of the restored three-dimensional mesh, which may make it possible to reduce the amount of coding.

[0498] Here, the structure of the restored three-dimensional mesh may be, for example, the number of restored triangles located around the vertices of the triangle associated with the symbol to be decoded, etc. Furthermore, the entropy decoding parameters may be, for example, parameters such as context information used to determine the occurrence probability in arithmetic decoding.

[0499] 50, an example of decoding processing has been described, but similar operations can be applied to encoding processing by replacing decoding with encoding. When similar operations are applied to encoding processing, a CLERS character string is generated by an edge breaker before executing the loop processing (S1601 to S1608).

[0500] For example, since the encoding device 100 can identify the end (delimiter) of a connected component, it encodes information for notifying the decoding device 200 of the end (delimiter) of the connected component. Specifically, the encoding device 100 sets end_of_connected_component[i] in FIG. 49 to 1 at the end (delimiter) of the connected component and performs encoding. Then, end_of_connected_component[i] can be used by the decoding device 200 to detect the end (delimiter) of the connected component.

[0501] In addition, in the encoding process, the three-dimensional mesh structure restoration process (S1607), the three-dimensional mesh structure completion process (S1606 and S1609), and the end determination process (S1605) may be omitted. Alternatively, to enable the same reference as in the decoding process, these processes may be performed in the encoding process, and the three-dimensional mesh structure may be reconstructed.

[0502] Furthermore, the encoding device 100 and the decoding device 200 according to the present disclosure may switch between a mode in which a CLERS character string is encoded and decoded in reverse order from the end and a mode in which a CLERS character string is encoded and decoded in order from the beginning, in units of a sequence, a frame, a submesh, etc. This mode may also be expressed as an adaptive mode.

[0503] Regarding mode switching, mode information indicating the mode may be notified in a header or the like corresponding to a switching unit such as a sequence, frame, or submesh. Alternatively, one mode may be applied according to information for specifying an application tool, such as profile information.

[0504] Furthermore, the encoding device 100 and the decoding device 200 according to the present disclosure may fixedly use a mode in which a CLERS character string is encoded and decoded in reverse order from the end. When the mode in which a CLERS character string is encoded and decoded in reverse order from the end is fixedly used, the CLERS character string is encoded and decoded using a syntax structure including information indicating the end of a connected component without notifying mode information in a header or the like.

[0505] In the above example, information indicating the end of a connected component is used between multiple connected components in the CLERS character string. This information may be expressed as information indicating the boundary (or delimiter) of the connected component. Furthermore, other notification information unrelated to the boundary of the connected component may be written at the boundary of the connected component in the coded data of the CLERS character string.

[0506] Fig. 51 is a flowchart showing a process of restoring a mesh structure according to symbols according to this embodiment. Here, an example of the process (S1607) of restoring a three-dimensional mesh structure according to the symbols decoded in Fig. 50 is shown.

[0507] Specifically, the process to be performed is selected according to the decoded symbol (S1701, S1705, S1707, and S1712). The process to be performed according to the decoded symbol is shown in Figures 52 to 56. Figures 52, 53, 54, 55, and 56 correspond to the processes to be performed according to types C, L, S, R, and E, respectively. The symbols (steps) in Figures 52 to 56 correspond to the symbols (steps) in Figure 51.

[0508] For example, if the decoded symbol is "C" (Yes in S1701), the right side of the current triangle is connected to the previous triangle (S1702). Here, the current triangle is the triangle corresponding to the decoded symbol, and the previous triangle is the triangle corresponding to the symbol decoded immediately before.

[0509] Next, a new index is assigned to the vertex above the current triangle. Then, the vertex indexes are copied from the multiple triangles that share the vertex in counterclockwise order, starting from the previous triangle (S1703). Finally, the triangle whose vertex index was copied is connected to the left side of the current triangle (S1704).

[0510] If the decoded symbol is "L" (Yes in S1705), the right side of the current triangle is connected to the previous triangle (S1706).

[0511] If the decoded symbol is "S" (Yes in S1707), the right side of the current triangle is connected to the previous triangle (S1708). Next, it is determined whether there is a triangle to be notified as a handle on the left side of the current triangle (S1709). The triangle to be notified as a handle is a triangle that forms a cyclic structure called a handle. For example, a handle corresponds to multiple triangles that form a cyclic structure that traces from the right side of a type S triangle to the left side of the triangle in the forward processing order.

[0512] If a triangle notified as a handle exists (Yes in S1709), the triangle notified as a handle is connected to the left side of the current triangle (S1710). In this case, the triangle (cutting position) is not popped from the stack. On the other hand, if a triangle notified as a handle does not exist (No in S1709), the triangle popped from the stack is connected to the left side of the current triangle (S1711).

[0513] If the decoded symbol is "R" (Yes in S1712), the left side of the current triangle is connected to the previous triangle (S1713).

[0514] If the decoded symbol is "E" (No in S1712), it is determined whether the current triangle is the end of the connected component (S1714). If the current triangle is not the end of the connected component (No in S1714), the previous triangle is pushed onto the stack (S1715). Note that the determination of the end (S1714) may be performed in the same manner as the determination of the end (S1605) in Fig. 50. For example, the end of the connected component may be determined according to end_of_connected_component[i] in Fig. 49.

[0515] In the above description, the right, left, and top sides of the triangle may be determined based on a predetermined rule, for example, the right, left, and top sides of the triangle may be determined along a forward path corresponding to the processing order for determining the CLERS string.

[0516] Although an example of a decoding process has been described in Fig. 51, similar operations can be applied to an encoding process by replacing "decoding" with "encoding." In addition, a three-dimensional mesh structure may be reconstructed in the encoding process to enable the same reference as in the decoding process.

[0517] 57 is a syntax diagram showing a second syntax example of a symbol string including connected components according to this embodiment. In this example, when a CLERS string consisting of multiple connected components generated by an edge breaker is encoded and decoded in reverse order from the end, the number of symbols included in each connected component is notified so that the end or boundary of the connected component can be determined.

[0518] Here, num_connected_component indicates the total number of connected components contained in the mesh data.

[0519] Furthermore, num_clers_symbol[i] indicates the total number of symbols contained in the i-th connected component when the first connected component of the three-dimensional mesh is defined as the 0th connected component. In other words, num_clers_symbol[i] indicates the total number of symbols contained in the i-th connected component when the first connected component in the forward direction is defined as the 0th connected component among one or more connected components included in the CLERS string of the entire three-dimensional mesh.

[0520] Furthermore, clers_symbol[i] indicates the i-th symbol when the first symbol of the CLERS string in the forward direction is defined as the 0th symbol. Furthermore, total_num_clers_symbol indicates the total number of symbols (count number) included in the CLERS string of the entire 3D mesh, and is derived by adding num_clers_symbol[i] for each i. Furthermore, clers_count indicates the number of decoded symbols for the current connected component, which is the connected component to be decoded.

[0521] For example, the same number of clers_symbol[i] as the number derived as total_num_clers_symbol is notified sequentially. Then, in each connected component, the number of clers_symbol[i] notified sequentially is counted sequentially as clers_count.

[0522] Then, it is determined whether clers_count is equal to num_clers_symbol[connected_component_index]. If clers_count is equal to num_clers_symbol[connected_component_index], it is determined that decoding of all symbols included in the connected component has been completed. In other words, it is determined that the decoding process has reached a boundary of the connected component, and the next symbol to be decoded is the end of the connected component to be decoded.

[0523] Note that connected_component_index is the index of the current connected component. Also, in the above description, decoding can be read as encoding.

[0524] As described above, by notifying the number of symbols contained in each connected component, it becomes possible to correctly detect the end or boundary of a connected component even when the CLERS string of a mesh composed of multiple connected components is encoded and decoded in reverse order from the end.

[0525] 58 is a flowchart showing a second decoding example of a symbol string including a connected component according to this embodiment. In this example, a CLERS character string including multiple connected components is entropy-decoded in reverse order from the end, and a three-dimensional mesh structure is restored. Also, in this example, in loop processing (S1902 to S1907), the symbols of the CLERS character string are decoded one by one, and the three-dimensional mesh structure is restored according to the decoded symbols. Note that the three-dimensional mesh structure here may refer to the structure of a portion of the three-dimensional mesh.

[0526] First, before the loop processing, the number of connected components and the number of symbols included in each connected component are obtained (S1901). Here, the number of connected components corresponds to num_connected_component in Fig. 57. Also, the number of symbols included in each connected component corresponds to num_classes_symbol[i] in Fig. 57.

[0527] Next, loop processing is started. In the loop processing, first, a symbol is decoded by entropy decoding (S1903). Here, the symbol corresponds to clers_symbol[i] in FIG. 57. At this time, the entropy decoding parameters may be determined by referring to the structure of the restored three-dimensional mesh, or may be determined by combining with other examples such as FIG. 47. Then, the three-dimensional mesh structure is restored according to the decoded symbol (S1904). Details are as described above with reference to FIG. 51.

[0528] Next, it is determined whether the decoding process has reached a boundary of a connected component (S1905). This determination process corresponds to determining whether the decoding of all symbols included in the connected component has been completed. This determination process also corresponds to determining whether clers_count is equal to num_clers_symbol[connected_component_index] in FIG. 57. In other words, it is determined whether the decoding process has reached a boundary of a connected component and the next symbol to be decoded is the end of the next connected component to be decoded.

[0529] If it is determined that the decoding process has reached the boundary of the connected component (Yes in S1905), processing is performed to complete the three-dimensional mesh structure of the connected component whose decoding has been completed (the connected component that includes the previously decoded symbol) (S1906).

[0530] Specifically, at the beginning of a connected component, there is a triangle to which no symbol is assigned, i.e., a triangle to which no symbol is encoded. This leading triangle is connected to the 3D mesh structure. That is, the vertex indices of this leading triangle are associated with the vertices of triangles whose symbols are decoded. This association of vertex indices completes the 3D mesh structure of the connected component.

[0531] As described above, the number of symbols included in each connected component is notified. This makes it possible to correctly detect the end or boundary of a connected component even when the CLERS string of a mesh made up of multiple connected components is encoded and decoded in reverse order from the end. This makes it possible to appropriately select triangles to be connected at the boundary of the connected component, and correctly restore the structure of the 3D mesh.

[0532] Then, when entropy decoding a symbol, it becomes possible to determine the entropy decoding parameters by referring to the structure of the restored three-dimensional mesh, which may make it possible to reduce the amount of coding.

[0533] Here, the structure of the restored three-dimensional mesh may be, for example, the number of restored triangles located around the vertices of the triangle associated with the symbol to be decoded, etc. Furthermore, the entropy decoding parameters may be, for example, parameters such as context information used to determine the occurrence probability in arithmetic decoding.

[0534] 58, an example of decoding processing has been described, but similar operations can be applied to encoding processing by replacing decoding with encoding. When similar operations are applied to encoding processing, a CLERS character string is generated by an edge breaker before executing the loop processing (S1902 to S1907).

[0535] For example, since the encoding device 100 can identify the boundaries of connected components, it encodes information for notifying the decoding device 200 of the boundaries of connected components. Specifically, the encoding device 100 identifies and encodes num_connected_component and num_clers_symbol[i] in Fig. 57 based on the boundaries of connected components. Then, num_connected_component and num_clers_symbol[i] can be used by the decoding device 200 to detect boundaries of connected components.

[0536] In addition, in the encoding process, the three-dimensional mesh structure restoration process (S1904), the three-dimensional mesh structure completion process (S1906), and the delimiter determination process (S1905) may be omitted. Alternatively, to enable the same reference as in the decoding process, these processes may be performed in the encoding process, and the three-dimensional mesh structure may be reconstructed.

[0537] Furthermore, the encoding device 100 and the decoding device 200 according to the present disclosure may switch between a mode in which a CLERS character string is encoded and decoded in reverse order from the end and a mode in which a CLERS character string is encoded and decoded in order from the beginning, in units of a sequence, a frame, a submesh, etc. This mode may also be expressed as an adaptive mode.

[0538] Regarding mode switching, mode information indicating the mode may be notified in a header or the like corresponding to a switching unit such as a sequence, frame, or submesh. Alternatively, one mode may be applied according to information for specifying an application tool, such as profile information.

[0539] Furthermore, the encoding device 100 and the decoding device 200 according to the present disclosure may use a fixed mode for encoding and decoding a CLERS character string in reverse order from the end. When the fixed mode for encoding and decoding a CLERS character string in reverse order from the end is used, the CLERS character string is encoded and decoded using a syntax structure including information indicating the number of symbols included in a connected component, without notifying mode information in a header or the like.

[0540] In the above example, information indicating the number of symbols included in the connected component is used. This information may be expressed as information indicating the length of the connected component. This information may also be information that can be combined with other information to derive the number of symbols included in the connected component.

[0541] Fig. 59 is a flowchart showing a sixth specific example of connection-type encoding according to this embodiment. That is, Fig. 59 shows an example of the encoding process (S603) in Fig. 36 and the encoding process (S1103) in Fig. 44. Furthermore, the encoding process in Fig. 59 may be applied to encoding process corresponding to the decoding process (S1602) in Fig. 50, or may be applied to encoding process corresponding to the decoding process (S1903) in Fig. 58.

[0542] In this example, the character strings of types C, L, E, R, and S are coded from the end to the beginning, i.e., in reverse order. That is, for the multiple triangles that make up the mesh, the connection type of each triangle is determined according to the processing order, and the connection type is coded in the reverse order of the processing order used to determine the connection type. Also, in this example, bit patterns 0, 10, 110, 1110, and 1111 are assigned to types C, R, S, E, and L, respectively, and entropy coding is performed starting from the most significant bit of each bit pattern.

[0543] In other words, this example corresponds to an example in which the character string is encoded in reverse order and a truncated unary bit pattern is used. Also, the fact that Type C does not occur immediately after Type L or Type E in the reverse order is reflected in the encoding process.

[0544] Specifically, first, it is determined whether the connection type of the immediately preceding (previously) coded triangle is one of types L and E (S2001).

[0545] If the connection type of the previously coded triangle is L or E (Yes in S2001), it is determined that the connection type of the triangle to be coded is not Type C. Then, without coding the value of the first bit b0, it is determined whether the connection type of the triangle to be coded is Type R (S2004).

[0546] If the connection type of the previously coded triangle is neither Type L nor Type E (No in S2001), it is determined whether the connection type of the triangle to be coded is Type C (S2002). If the connection type of the triangle to be coded is Type C (Yes in S2002), the value of the first bit b0 is set to 0 by entropy coding according to c_idx (S2005).

[0547] If the connection type of the triangle to be coded is not Type C (No: L, E, R, or S in S2002), the value of the first bit b0 is set to 1 by entropy coding according to c_idx (S2003). Then, it is determined whether the connection type of the triangle to be coded is Type R (S2004).

[0548] If the connection type of the triangle to be coded is Type R (Yes in S2004), the value of the second bit b1 is coded as 0 by entropy coding according to c_idx (S2007).

[0549] If the connection type of the triangle to be coded is not Type R (No: L, E, or S in S2004), the value of the second bit b1 is set to 1 by entropy coding according to c_idx (S2006). Then, it is determined whether the connection type of the triangle to be coded is Type S (S2008).

[0550] If the connection type of the triangle to be coded is Type S (Yes in S2008), the value of the third bit b2 is coded as 0 by entropy coding according to c_idx (S2013).

[0551] If the connection type of the triangle to be coded is not Type S (No: L or E in S2008), the value of the third bit b2 is set to 1 by entropy coding according to c_idx (S2009). Then, it is determined whether the connection type of the triangle to be coded is Type E (S2010).

[0552] If the connection type of the triangle to be coded is Type E (Yes in S2010), the value of the fourth bit b3 is coded as 0 by entropy coding according to c_idx (S2012).

[0553] If the connection type of the triangle to be coded is not Type E (No: L in S2010), the value of the fourth bit b3 is set to 1 by entropy coding according to c_idx (S2011).

[0554] When arithmetic coding is used for entropy coding, a context used to determine the expected occurrence probability of information to be coded is determined from a plurality of contexts according to c_idx. When Huffman coding is used for entropy coding, a code word table used for coding is determined from a plurality of code word tables according to c_idx.

[0555] Type C does not logically occur immediately after Type L or Type E. Therefore, immediately after Type L or Type E, it may be possible to encode the triangle connection type with a smaller amount of code by omitting encoding the first bit b0 that identifies Type C and encoding only the three bits from the second bit b1 to the fourth bit b3.

[0556] In addition, instead of omitting the encoding of the first bit b0, it may be possible to reduce the amount of code for triangle connection types by always using the same context to encode the first bit b0 immediately after type L or type E.

[0557] In the above example, bit patterns 0, 10, 110, 1110, and 1111 are assigned to types C, R, S, E, and L, respectively. However, the type to be determined in the determination processes (S2002, S2004, S2008, and S2010) may be determined in accordance with the method of assigning bit patterns. By setting the bits omitted in the immediately preceding connection type determination process (S2001) as bits related to type C determination, any bit pattern can be assigned.

[0558] Alternatively, short bit patterns may be assigned to types that occur frequently. This may enable efficient encoding and a reduction in the amount of code. Therefore, bit patterns 0 and 10, or bit patterns 10 and 0, may be assigned to types C and R, which occur frequently. Bit patterns 1110 and 1111, or bit patterns 1111 and 1110, may be assigned to types E and L, which occur less frequently.

[0559] Fig. 60 is a flowchart showing a sixth specific example of connection-type decoding according to this embodiment. That is, Fig. 60 shows an example of the decoding process (S603) in Fig. 36 and the decoding process (S1103) in Fig. 44. Furthermore, the decoding process in Fig. 60 may be applied to the decoding process (S1602) in Fig. 50 or the decoding process (S1903) in Fig. 58.

[0560] In this example, entropy decoding is performed on a bit string that has been coded using the coding method described with reference to Figure 59. That is, entropy decoding of a bit string in which character strings of types C, L, E, R, and S have been coded from the end to the beginning (i.e., in reverse order) is performed in the same decoding order as the coding order. Then, one of types C, L, E, R, and S is set as the connection type of the triangle to be decoded.

[0561] In other words, this example corresponds to an example in which the decoding process of a character string is performed in reverse order and a truncated unary is used for the bit pattern. Also, the fact that Type C does not occur immediately after Type L or Type E in the reverse order is reflected in the decoding process.

[0562] Specifically, first, it is determined whether the connection type of the triangle that has been decoded immediately before is one of types L and E (S2101).

[0563] If the connection type of the previously decoded triangle is Type L or Type E (Yes in S2101), it is determined that the connection type of the triangle to be decoded is not Type C. Then, the value of the first bit b0 is not decoded, and the value of the second bit b1 is decoded by entropy decoding according to c_idx (S2104). In this case, the value of the first bit b0 may be regarded as 1.

[0564] If the connection type of the previously decoded triangle is neither Type L nor Type E (No in S2101), the value of the first bit b0 is decoded by entropy decoding according to c_idx (S2102). Then, it is determined whether the value of the first bit b0 is 0 (S2103).

[0565] If the value of the first bit b0 is 0 (Yes in S2103), the connection type of the triangle to be decoded is set to Type C (S2108). If the value of the first bit b0 is not 0 (No in S2103), the value of the second bit b1 is decoded by entropy decoding according to c_idx (S2104). Then, it is determined whether the value of the second bit b1 is 0 (S2105).

[0566] If the value of the second bit b1 is 0 (Yes in S2105), the connection type of the triangle to be decoded is set to type R (S2110). If the value of the second bit b1 is not 0 (No in S2105), the value of the third bit b2 is decoded by entropy decoding according to c_idx (S2106). Then, it is determined whether the value of the third bit b2 is 0 (S2107).

[0567] If the value of the third bit b2 is 0 (Yes in S2107), the connection type of the triangle to be decoded is set to type S (S2111). If the value of the third bit b2 is not 0 (No in S2107), the value of the fourth bit b3 is decoded by entropy decoding according to c_idx (S2109). Then, it is determined whether the value of the fourth bit b3 is 0 (S2112).

[0568] If the value of the fourth bit b3 is 0 (Yes in S2112), the connection type of the triangle to be decoded is set to Type E (S2114). If the value of the fourth bit b3 is not 0 (No in S2112), the connection type of the triangle to be decoded is set to Type L (S2113).

[0569] When arithmetic decoding is used for entropy decoding, a context used to determine the expected occurrence probability of information to be decoded is determined from among multiple contexts according to c_idx.When Huffman decoding is used for entropy decoding, a code word table used for decoding is determined from among multiple code word tables according to c_idx.

[0570] Type C does not logically occur immediately after Type L or Type E. Therefore, immediately after Type L or Type E, it may be possible to decode the triangle connection type with a smaller amount of code by omitting decoding the first bit b0 that identifies Type C and decoding only the three bits from the second bit b1 to the fourth bit b3.

[0571] In addition, instead of omitting the decoding of the first bit b0, it may be possible to reduce the amount of code for triangular connection types by always using the same context to decode the first bit b0 immediately after type L or type E.

[0572] In the above example, bit patterns 0, 10, 110, 1110, and 1111 are assigned to types C, R, S, E, and L, respectively. However, the type to be set in the setting processes (S2108, S2110, S2111, S2114, and S2113) may be determined in accordance with the bit pattern assignment method. By setting the bits omitted in the immediately preceding connection type determination process (S2101) as bits related to type C determination, any bit pattern can be assigned.

[0573] Alternatively, shorter bit patterns may be assigned to types that occur more frequently, which may allow for a reduction in the amount of code. Therefore, bit patterns 0 and 10, or bit patterns 10 and 0, may be assigned to types C and R, which occur more frequently. Bit patterns 1110 and 1111, or bit patterns 1111 and 1110, may be assigned to types E and L, which occur less frequently.

[0574] <Representative Example> Fig. 61 is a flowchart showing an example of basic encoding processing according to this embodiment. For example, the circuit 151 of the encoding device 100 shown in Fig. 24 performs the encoding processing shown in Fig. 61 using the memory 152 in operation.

[0575] Specifically, for each face constituting the three-dimensional mesh, the circuit 151 encodes the connection type relating to the connection relationship between the face to be processed and unprocessed faces in the reverse order of the processing order in which the connection type is defined (S2201). Here, the multiple examples of the connection type relating to the connection relationship between the face to be processed and unprocessed faces may include a type in which the face to be processed is not connected to any unprocessed faces (e.g., the above-mentioned type E).

[0576] The circuit 151 also encodes association information for detecting boundaries of connected components for a plurality of connection types, each of which is encoded in reverse order as a connection type (S2202). Here, a connected component corresponds to at least a portion of a 3D mesh. The association information may be encoded before or during the encoding of the plurality of connection types.

[0577] This may enable proper detection of boundaries between connected components in multiple connection types coded in reverse order. Therefore, it may be possible to efficiently identify connected components and efficiently identify the connectivity between multiple faces. Therefore, it may be possible to reduce processing delays in the reconstruction of 3D meshes.

[0578] For example, the related information may indicate whether the current connection type, which is the connection type to be coded, corresponds to the end of a connected component in processing order. This may make it possible to appropriately detect the connection type corresponding to the end of a connected component among multiple connection types coded in reverse order. This may make it possible to efficiently identify connected components and efficiently identify the connectivity relationships between multiple faces.

[0579] Furthermore, for example, the circuit 151 may determine whether a condition is satisfied that indicates that the current connection type may correspond to the end. If it is determined that the condition is satisfied, the circuit 151 may encode the related information. This may make it possible to omit encoding the related information when the current connection type is unlikely to correspond to the end of the connected component. Therefore, it may be possible to suppress an increase in the amount of code.

[0580] Furthermore, for example, the condition may include that the current connection type is a termination type. Here, the termination type is a type in which no unprocessed faces are connected to the left or right of the face to be processed along the path leading to the face to be processed in the processing order. This may make it possible to omit encoding of related information when the current connection type is not a termination type. Therefore, it may be possible to suppress an increase in the amount of code.

[0581] Furthermore, for example, the condition may include that the stack to be pushed or popped according to the connection type is empty. This may allow for omitting encoding of related information if the stack is not empty. Therefore, it may be possible to suppress an increase in the amount of encoding.

[0582] Furthermore, for example, the condition may include that the connection type of the last face of at least one of one or more connected components each defined as a connected component is not coded. This may make it possible to omit coding of the related information if the connection type of the last face of each connected component has already been coded. Therefore, it may be possible to suppress an increase in the amount of coding.

[0583] Furthermore, for example, the related information may indicate a connection type number, which is the number of one or more connection types encoded for each of one or more connected components defined as a connected component. This may enable efficient identification of each connected component according to the connection type number and efficient identification of the connection relationships between multiple faces. This may therefore enable reduction in processing delays in the reconstruction of a 3D mesh.

[0584] Furthermore, for example, the division may be indicated according to the number of connection types indicated by the related information and the number of one or more connection types that have been encoded. More specifically, for example, if the number of connection types indicated by the related information matches the number of one or more connection types that have been encoded, it is assumed that the encoding process has reached a division of the connected component. In other words, the division of the connected component may be indicated by a match between the number of connection types indicated by the related information and the number of one or more connection types that have been encoded.

[0585] This may allow appropriate indication of the division of connected components in the multiple connection types coded in reverse order according to the relationship between the number of connection types indicated by the related information and the number of one or more encoded connection types, thereby allowing efficient identification of each connected component and efficient identification of the connection relationships of multiple faces.

[0586] The related information may be defined as information related to identifying the boundary of a connected component. The related information may correspond to end_of_connected_component[i] or num_classes_symbol[i].

[0587] In the above, circuit 151 of encoding device 100 performs each process, but connection information encoder 102 or 145 of encoding device 100 may perform each of the above processes, or other components may perform each of the above processes.

[0588] Fig. 62 is a flowchart showing an example of basic decoding processing according to this embodiment. For example, the circuit 251 of the decoding device 200 shown in Fig. 25 performs the decoding processing shown in Fig. 62 using the memory 252 in operation.

[0589] Specifically, for each face constituting the three-dimensional mesh, the circuit 251 decodes the connection type relating to the connection relationship between the face to be processed and unprocessed faces in the reverse order of the processing order in which the connection types are defined (S2301). Here, the multiple examples of the connection type relating to the connection relationship between the face to be processed and unprocessed faces may include a type in which the face to be processed is not connected to any unprocessed faces (e.g., the above-mentioned type E).

[0590] The circuit 251 also decodes associated information for detecting boundaries of connected components for multiple connection types, each of which is decoded in reverse order as a connection type (S2302). Here, a connected component corresponds to at least a portion of a 3D mesh. The associated information may be decoded before or during the decoding of the multiple connection types.

[0591] This may enable proper detection of boundaries between connected components in multiple connection types decoded in reverse order. Therefore, it may be possible to efficiently identify connected components and efficiently identify the connectivity between multiple faces. Therefore, it may be possible to reduce processing delays in reconstructing a 3D mesh.

[0592] For example, the related information may indicate whether the current connection type, which is the connection type to be decoded, corresponds to the end of a connected component in the processing order. This may make it possible to appropriately detect the connection type corresponding to the end of a connected component among multiple connection types decoded in reverse order. Therefore, it may be possible to efficiently identify connected components and efficiently identify the connection relationships between multiple faces.

[0593] Furthermore, for example, the circuit 251 may determine whether a condition is satisfied that indicates that the current connection type may correspond to the end. If the circuit 251 determines that the condition is satisfied, the circuit 251 may decode the related information. This may make it possible to omit decoding of the related information when the current connection type is unlikely to correspond to the end of a connected component. Therefore, it may be possible to suppress an increase in the amount of code.

[0594] Furthermore, for example, the condition may include that the current connection type is a termination type. Here, the termination type is a type in which no unprocessed faces are connected to the left or right of the face to be processed along the path leading to the face to be processed in the processing order. This may make it possible to omit decoding of related information when the current connection type is not a termination type. Therefore, it may be possible to suppress an increase in the amount of code.

[0595] Furthermore, for example, the condition may include that the stack to be pushed or popped according to the connection type is empty. This may allow for skipping the decoding of related information if the stack is not empty. Therefore, it may be possible to suppress an increase in the amount of code.

[0596] Furthermore, for example, the condition may include that the connection type of the last face of at least one of one or more connected components each defined as a connected component has not been decoded. This may make it possible to omit decoding of the related information if the connection type of the last face of each connected component has already been decoded. Therefore, it may be possible to suppress an increase in the amount of code.

[0597] Furthermore, for example, the related information may indicate a connection type number, which is the number of one or more connection types decoded for each of one or more connected components defined as a connected component. This may enable efficient identification of each connected component according to the connection type number and efficient identification of the connection relationships between multiple faces. Therefore, it may be possible to reduce processing delays in reconstructing a 3D mesh.

[0598] Furthermore, for example, the division may be indicated according to the number of connection types indicated by the related information and the number of one or more decoded connection types. More specifically, for example, if the number of connection types indicated by the related information matches the number of one or more decoded connection types, it is assumed that the decoding process has reached a division of the connected component. In other words, the division of the connected component may be indicated by a match between the number of connection types indicated by the related information and the number of one or more decoded connection types.

[0599] This may allow appropriate indication of the division of connected components in the multiple connection types decoded in reverse order according to the relationship between the number of connection types indicated by the related information and the number of one or more decoded connection types, thereby allowing efficient identification of each connected component and efficient identification of the connection relationships of multiple faces.

[0600] The related information may be defined as information related to identifying the boundary of a connected component. The related information may correspond to end_of_connected_component[i] or num_classes_symbol[i].

[0601] In the above, circuit 251 of decoding device 200 performs each process, but connection information decoder 202 or 245 of decoding device 200 may perform each of the above processes, or other components may perform each of the above processes.

[0602] In the above description, a connection may be expressed as a linkage, and a connection type may be expressed as connection information or linkage information.

[0603] <Other Examples> Although aspects of the encoding device 100 and the decoding device 200 have been described above according to the embodiments, the aspects of the encoding device 100 and the decoding device 200 are not limited to the embodiments. Modifications conceivable by those skilled in the art may be applied to the embodiments, and multiple components in the embodiments may be combined in any manner.

[0604] For example, a process performed by a specific component in the embodiment may be performed by another component instead of the specific component. Also, the order of multiple processes may be changed, or multiple processes may be performed in parallel.

[0605] Furthermore, as described above, at least some of the configurations of the present disclosure may be implemented as an integrated circuit. At least some of the processes of the present disclosure may be used as an encoding method or a decoding method. A program for causing a computer to execute the encoding method or the decoding method may be used. A non-transitory computer-readable recording medium on which the program is recorded may be used. A bitstream for causing the decoding device 200 to perform a decoding process may be used.

[0606] Furthermore, at least some of the configurations and processes of the present disclosure may be used as a transmitting device, a receiving device, a transmitting method, or a receiving method. A program for causing a computer to execute the transmitting method or the receiving method may be used. Furthermore, a non-transitory computer-readable recording medium on which the program is recorded may be used.

[0607] Also, for example, a phrase "at least one of" a first element, a second element, and a third element corresponds to the first element, the second element, the third element, or any combination thereof.

[0608] In the present disclosure, setting a first element to a first value may mean determining the value of the element to the first value. Furthermore, the expression "setting a first element to a first value" may be replaced with the expression "setting the first element to the first value."

[0609] The present disclosure is useful, for example, in encoding devices, decoding devices, transmitting devices, receiving devices, etc. related to three-dimensional meshes, and is applicable to computer graphics systems, three-dimensional data display systems, etc.

[0610] 100 Encoding device 101, 121, 144 Vertex information encoder 102, 145 Connection information encoder 103, 122 Attribute information encoder 104, 204, 521 Preprocessor 105, 205, 623 Postprocessor 110 Three-dimensional data encoding system 111, 211 Controller 112, 212 Input / output processor 113 Three-dimensional data encoder 114 System multiplexer 115 Three-dimensional data generator 123 Metadata encoder 124 Multiplexer 131 Vertex image generator 132 Attribute image generator 133 Metadata generator 134 Video encoder 141 Two-dimensional data encoder 142 Mesh data encoder 143 Texture encoder 148 Description encoder 151, 251 Circuit 152, 252 Memory 200 Decoding device 201, 221, 244 Vertex information decoder 202, 245 Connection information decoder 203, 222 Attribute information decoder 210 Three-dimensional data decoding system 213 Three-dimensional data decoder 214 System demultiplexer 215, 247 Presentation device 216 User interface 223 Metadata decoder 224 Demultiplexer 231 Vertex information generator 232 Attribute information generator 234 Video decoder 241 Two-dimensional data decoder 242 Mesh data decoder 243 Texture decoder 246 Mesh reconstructor 248 Description decoder 300 Network 310 External connection device 522 Encoding processor 622 Decoding processor

Claims

1. An encoding device comprising a memory and a circuit accessible to the memory, wherein in operation, for each of a plurality of faces constituting a three-dimensional mesh, a connection type regarding a connection relationship between the face to be processed and an unprocessed face is encoded in a reverse order of a processing order in which the connection type is defined, and in a plurality of connection types each encoded in the reverse order as the connection type, related information for detecting a delimiter of a connected component is encoded, the connected component corresponding to at least a part of the three-dimensional mesh.

2. The encoding device according to claim 1, wherein the related information indicates whether a current connection type, which is the connection type to be encoded, corresponds to the end of the connected component in the processing order.

3. The encoding device according to claim 2, wherein the circuit determines whether a condition under which the current connection type has a possibility of corresponding to the end is satisfied, and when it is determined that the condition is satisfied, encodes the related information.

4. The encoding device according to claim 3, wherein the condition includes that the current connection type is a terminal type, and the terminal type is a type in which no unprocessed face is connected to the left or right of the face to be processed along a path leading to the face to be processed in the processing order.

5. The encoding device according to claim 3 or 4, wherein the condition includes that a stack in which push or pop is performed according to the connection type is empty.

6. The encoding device according to claim 3 or 4, wherein the condition includes that the connection type of a face at the end of at least one of one or more connected components each defined as the connected component is not encoded.

7. The encoding device according to claim 1, wherein the related information indicates a number of connection types, which is a number of one or more connection types encoded for each of one or more connected components each defined as the connected component.

8. The encoding device according to claim 7, wherein the delimiter is indicated according to the number of connection types indicated by the related information and the number of one or more encoded connection types.

9. A decoding device comprising a memory and a circuit accessible to the memory, wherein in operation, for each of a plurality of planes constituting a three-dimensional mesh, the circuit decodes a connection type regarding the connection relationship between the plane to be processed and the unprocessed planes in a reverse order to the processing order in which the connection type is defined, and in a plurality of connection types each decoded in the reverse order as the connection type, decodes related information for detecting a delimiter of a connected component, the connected component corresponding to at least a part of the three-dimensional mesh.

10. The decoding device according to claim 9, wherein the related information indicates whether a current connection type, which is the connection type to be decoded, corresponds to the end of the connected component in the processing order.

11. The decoding device according to claim 10, wherein the circuit determines whether a condition under which the current connection type has a possibility of corresponding to the end is satisfied, and when it is determined that the condition is satisfied, decodes the related information.

12. The decoding device according to claim 11, wherein the condition includes that the current connection type is a terminal type, and the terminal type is a type in which no unprocessed plane is connected to the left or right of the plane to be processed along a path leading to the plane to be processed in the processing order.

13. The decoding device according to claim 11 or 12, wherein the condition includes that a stack in which push or pop is performed according to the connection type is empty.

14. The decoding device according to claim 11 or 12, wherein the condition includes that the connection type of the end plane of at least one of one or more connected components each defined as the connected component has not been decoded.

15. The decoding device according to claim 9, wherein the related information indicates the number of connection types, which is the number of one or more connection types decoded for each of one or more connected components each defined as the connected component.

16. The decoding device according to claim 15, wherein the delimiter is indicated according to the number of connection types indicated by the related information and the number of one or more decoded connection types.

17. For each of a plurality of faces constituting a three-dimensional mesh, an attachment type regarding the attachment relationship between the face to be processed and the unprocessed faces is encoded in a reverse order of the processing order in which the attachment type is defined. In a plurality of attachment types each encoded in the reverse order as the attachment type, related information for detecting a delimiter of a connected component is encoded. The connected component corresponds to at least a part of the three-dimensional mesh. Encoding method.

18. For each of a plurality of faces constituting a three-dimensional mesh, an attachment type regarding the attachment relationship between the face to be processed and the unprocessed faces is decoded in a reverse order of the processing order in which the attachment type is defined. In a plurality of attachment types each decoded in the reverse order as the attachment type, related information for detecting a delimiter of a connected component is decoded. The connected component corresponds to at least a part of the three-dimensional mesh. Decoding method.

Images