Methods and apparatus for encoding and decoding three-dimensional grid

Abstract

Disclosed in the present invention are methods and apparatus for encoding and decoding a three-dimensional grid. The encoding method comprises performing connection relationship encoding and geometric information encoding on an inputted three-dimensional grid, and obtaining an output bit stream. The connection relationship encoding comprises encoding a triangular pattern in the three-dimensional grid and additional information used for indicating a triangle position relationship, and the connection relationship encoding is implemented on the basis of entropy encoding of context information. The decoding method comprises performing connection relationship decoding and geometric information decoding on an input bit stream, and obtaining a connection relationship and geometric information; and reconstructing a three-dimensional grid on the basis of the obtained connection relationship and geometric information. The connection relationship decoding comprises decoding a triangular pattern in a three-dimensional grid, while also decoding additional information used for indicating a triangle position relationship, so as to implement instant stitching of a current triangle, and the connection relationship decoding is implemented on the basis of entropy decoding of context information. According to the present invention, the efficiency of entropy encoding can be improved, and the instant stitching of a current triangle can be implemented in a decoding process.

Description

A three-dimensional grid encoding and decoding method and device Technical Field

[0001] The present invention belongs to the field of coding and decoding, and in particular relates to a three-dimensional grid coding and decoding method and device. Background Art

[0002] In recent years, 3D models have become a new generation of digital media, following audio, images, and video. 3D meshes are a commonly used representation of 3D models. Compared to traditional multimedia like images and videos, 3D meshes offer greater interactivity and realism, leading to their increasing application in a variety of fields, including commerce, manufacturing, construction, education, medicine, entertainment, art, and the military.

[0003] With the advancement of 3D mesh modeling and scanning technology, people have higher requirements for the accuracy and details of 3D graphics, which has also led to an exponential increase in the amount of data of the original 3D mesh. This has posed a major challenge to the transmission, storage and computer processing of the 3D mesh. Therefore, it is particularly important to achieve efficient compression of the 3D mesh while ensuring quality.

[0004] A three-dimensional mesh is composed of elements at multiple levels, including vertices, edges, and faces. Vertices are the basic elements that make up a three-dimensional mesh and are described by coordinates in three-dimensional space. Edges are the parts that connect two vertices in a three-dimensional mesh. Faces are polygons connected by closed edges. Nowadays, most faces in three-dimensional meshes are triangles. A three-dimensional mesh mainly contains the following three types of information: 1. Geometric information. Geometric information mainly includes the position coordinates of all vertices in the three-dimensional mesh in three-dimensional space; 2. Connectivity relationship, also known as topological information. Connectivity relationship is used to describe the connection information between vertices and facets in a three-dimensional mesh; 3. Other optional attribute information. Attribute information includes other information attached to the three-dimensional mesh, including color information, normal vectors, texture coordinates, etc.

[0005] Connectivity is a crucial component of 3D meshes, describing the connections between vertices. It influences both the overall shape and local details of the model. Therefore, proposing a new compression algorithm to efficiently compress the connectivity of 3D meshes and, based on this, accurately and efficiently encode the entire 3D mesh is undoubtedly of great significance.

[0006] Summary of the Invention

[0007] In order to solve the above problems existing in the prior art, the present invention provides a three-dimensional grid encoding and decoding method and device. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0008] In a first aspect, an embodiment of the present invention provides a three-dimensional grid encoding method, the method comprising:

[0009] Encode the connection relationship and geometric information of the input 3D mesh to obtain the output code stream;

[0010] The connection relationship encoding includes encoding the pattern of triangles in the three-dimensional grid and encoding additional information for indicating the positional relationship of the triangles; the connection relationship encoding is implemented based on entropy encoding of context information.

[0011] In a second aspect, an embodiment of the present invention provides a three-dimensional grid encoding device, the device comprising:

[0012] The encoding module is used to encode the connection relationship and geometric information of the input 3D mesh to obtain an output code stream;

[0013] The connection relationship encoding includes encoding the pattern of triangles in the three-dimensional grid and encoding additional information for indicating the positional relationship of the triangles; the connection relationship encoding is implemented based on entropy encoding of context information.

[0014] In a third aspect, an embodiment of the present invention provides a three-dimensional grid decoding method, the method comprising:

[0015] Decode the connection relationship and geometric information of the input code stream to obtain the connection relationship and geometric information respectively;

[0016] Reconstructing a three-dimensional mesh based on the obtained connection relationship and geometric information;

[0017] The connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding additional information indicating the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

[0018] In a fourth aspect, an embodiment of the present invention provides a three-dimensional grid decoding device, the device comprising:

[0019] The decoding module is used to decode the connection relationship and geometric information of the input code stream to obtain the connection relationship and geometric information respectively;

[0020] A reconstruction module, used to reconstruct a three-dimensional mesh based on the obtained connection relationship and geometric information;

[0021] The connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding additional information indicating the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

[0022] Beneficial effects of the present invention:

[0023] In a three-dimensional mesh encoding and decoding scheme proposed in an embodiment of the present invention, at the encoding end, connection relationship encoding includes encoding the pattern of triangles in the three-dimensional mesh and encoding additional information used to indicate the positional relationship of the triangles. The connection relationship encoding is implemented based on entropy encoding of context information. Accordingly, at the decoding end, the input code stream is subjected to connection relationship decoding and geometric information decoding to obtain the connection relationship and geometric information respectively; the three-dimensional mesh is reconstructed based on the obtained connection relationship and geometric information; wherein, the connection relationship decoding includes decoding the pattern of triangles in the three-dimensional mesh and decoding the additional information used to indicate the positional relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is implemented based on entropy decoding of context information. In this embodiment of the present invention, the encoding and transmission of additional information used to indicate the positional relationship of the triangles by the encoding end enables the decoding end to use the triangle positional relationship to ensure the immediacy of stitching during stitching.

[0024] Furthermore, in an embodiment of the present invention, relevant information of a triangle is used as context information, which includes at least one or more of vertex geometry information, encoded triangle patterns, and the number of adjacent encoded triangles at triangle vertices; since the number of adjacent encoded triangles at triangle vertices and the pattern are strongly correlated, entropy encoding of the pattern is performed using relevant information such as the number of adjacent encoded triangles at triangle vertices, which can more accurately estimate the probability of pattern occurrence, thereby improving the accuracy of probability estimation and improving coding efficiency, and the decoding end can obtain context information consistent with the encoding end to ensure accurate decoding and instant mesh reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic diagram of a triangular fan structure in an existing TFAN algorithm;

[0026] FIG2 is a schematic diagram of triangle fan classification in the existing TFAN algorithm;

[0027] FIG3 is a schematic flow chart of a three-dimensional grid coding method provided by an embodiment of the present invention;

[0028] FIG4 is a schematic diagram showing the principle of applying a three-dimensional grid coding method according to an embodiment of the present invention to a separate coding frame;

[0029] FIG5 is a schematic diagram showing the principle of applying a three-dimensional grid coding method according to an embodiment of the present invention to a hybrid coding framework;

[0030] FIG6 is a schematic diagram of a framework of a connection relationship encoding method proposed in an embodiment of the present invention;

[0031] Figure 7 shows the correspondence between angles, vertices, and triangles in the Corner-Table data structure;

[0032] Figure 8 is a schematic diagram of the CLERS model;

[0033] FIG9 is a schematic diagram of an entropy coding framework according to an embodiment of the present invention;

[0034] FIG10 is a schematic diagram of context modeling according to an embodiment of the present invention;

[0035] FIG11 is a schematic diagram of a process for recording the number of vertex rotations according to an embodiment of the present invention;

[0036] FIG12 is a schematic diagram showing an S-mode triangle target vertex connection that cannot be rotated according to an embodiment of the present invention;

[0037] FIG13 is a schematic diagram illustrating how to determine whether a traversed triangle pattern has a ring property when adjacent triangles have been traversed on both sides of the traversed triangle according to an embodiment of the present invention;

[0038] FIG14 is a schematic diagram showing the zip rotation vertex and two adjacent S-shaped triangles according to an embodiment of the present invention;

[0039] FIG15 is a schematic diagram of various parallelogram prediction methods according to an embodiment of the present invention;

[0040] FIG16 is a schematic flow chart of a three-dimensional grid decoding method provided by an embodiment of the present invention;

[0041] FIG17 is a schematic diagram showing the principle of applying a three-dimensional grid decoding method according to an embodiment of the present invention to a separate decoding framework;

[0042] FIG18 is a schematic diagram showing the principle of applying a three-dimensional grid decoding method according to an embodiment of the present invention to a hybrid decoding framework;

[0043] FIG19 is a schematic diagram of a framework of a connection relationship decoding method according to an embodiment of the present invention;

[0044] FIG20 is a schematic diagram of decoding and reconstructing a connection relationship according to an embodiment of the present invention;

[0045] FIG21 is a schematic diagram of the suturing process of the L-mode according to an embodiment of the present invention;

[0046] FIG22 is a schematic diagram showing an L-mode rotation vertex adjacent to two S-mode triangles according to an embodiment of the present invention;

[0047] FIG23 is a schematic diagram of the suturing process of the R mode according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Currently, TFAN is one of the best performing connectivity encoding algorithms. In the TFAN connectivity encoding algorithm, a triangular fan structure is first defined. A triangular fan is composed of a series of triangles with the same vertex and the same direction, as shown in Figure 1. The triangular fan is defined as follows:

[0050] 1) Any two consecutive triangles in a triangle fan must be adjacent and share a common side.

[0051] 2) All triangles in a triangle fan have the same direction.

[0052] 3) All triangles in a triangle fan share a vertex, which is called the center point of the triangle fan. The shared vertex is shown as v0.

[0053] The TFAN connectivity encoding algorithm encodes vertices in a 3D mesh by traversing them. First, a first-in, first-out queue is defined, and all vertices are marked as untraversed. Then, starting with a random initial vertex, a triangular fan structure centered at the initial vertex is determined. The remaining vertices in this triangular fan are placed in the queue, marked as traversed, and encoded. Vertices are then removed from the queue and the above process is repeated until the queue is empty and all vertices in the 3D mesh are marked, at which point encoding ends.

[0054] [Corrected 01.08.2024 according to Rule 91] The coding of triangle sectors is mainly based on the degree of the center point of the triangle sector (that is, the number of triangles in the triangle sector adjacent to the center point), the direction of the triangle, and the situation of vertex traversal. As shown in Figure 2, different values ​​of C represent different categories of triangle sectors. In the figure, gray triangles represent traversed triangles, white triangles represent untraversed triangles, and red and black vertices represent traversed and untraversed vertices respectively (the colors are displayed in grayscale in the figure). When encoding, the category, vertex degree, and vertex traversal information of the current triangle sector are encoded, among which the vertex traversal information is an optional encoding option.

[0055] Since TFAN has many types of triangular fans and the encoding of connection relationships is relatively inefficient, in order to achieve efficient encoding of connection relationships and real-time reconstruction at the decoding end, an embodiment of the present invention proposes a new three-dimensional mesh connection relationship encoding and decoding scheme. On this basis, combined with the encoding and decoding scheme of geometric information, a complete encoding and decoding scheme for three-dimensional meshes is proposed. Specifically, an embodiment of the present invention proposes a three-dimensional mesh encoding method and device, as well as a corresponding three-dimensional mesh decoding method and device.

[0056] It should be noted that the execution entities of the three-dimensional mesh encoding method / device or three-dimensional mesh decoding method / device provided in the embodiments of the present invention can be executed in corresponding electronic devices. The electronic device can be a server or a terminal device, but is not limited to these.

[0057] In a first aspect, an embodiment of the present invention provides a three-dimensional grid encoding method, which can be applied to an encoding end. As shown in FIG3 , the encoding method may include:

[0058] Scode1 encodes the connection relationship and geometric information of the input 3D mesh to obtain the output code stream;

[0059] The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of the triangles; the connection relationship coding is implemented based on entropy coding of context information.

[0060] In an embodiment of the present invention, connection relationship encoding and geometric information encoding are performed in units of triangles, that is, they are implemented by traversing triangles. In an embodiment of the present invention, connection relationship encoding and geometric information encoding can have the same triangle traversal order, but can also have different triangle traversal orders. It can be understood that the latter method requires additional transmission of some relevant information compared to the former method, which will not be described in detail here.

[0061] The method for encoding the connectivity of a 3D mesh, proposed in an embodiment of the present invention, primarily involves selecting a triangle as the initial triangle on the 3D mesh to be encoded. Next, starting from this initial triangle, the connectivity of the triangles is traversed by determining the pattern of each triangle according to a specific traversal rule. Finally, the determined triangle pattern is entropy-encoded, and additional information indicating the positional relationships of the triangles is simultaneously determined and encoded. Furthermore, this embodiment of the present invention implements the pattern using context-based adaptive arithmetic coding, which incorporates improved context information to enhance entropy coding efficiency.

[0062] It should be noted that the three-dimensional grid coding method proposed in the embodiment of the present invention can be applied to a three-dimensional grid coding framework in which connection relationships and geometric information are separately encoded, that is, it is applicable to a separate coding framework; it can also be applied to a three-dimensional grid coding framework in which connection relationships and geometric information are mixedly encoded, that is, it is also applicable to a mixed coding framework.

[0063] Specifically, for a separate coding framework, the input 3D mesh is coded for connectivity and geometry information to obtain an output code stream, including:

[0064] (1) Traversing the triangles in the three-dimensional grid to perform connection relationship coding and obtain the connection relationship coding result;

[0065] Specifically, the connection relationship encoding can be performed by traversing the triangles in the three-dimensional grid, and the connection relationship code stream can be obtained from the connection relationship encoding results of all triangles;

[0066] (2) traversing the triangles in the three-dimensional grid to perform geometric information encoding and obtain geometric information encoding results; wherein, the connection relationship encoding and geometric information encoding are performed in units of triangles;

[0067] This step can be performed by traversing the triangles in the three-dimensional mesh according to the connection relationship coding order to perform geometric information coding, and obtaining a geometric information code stream from the geometric information coding results of all triangles;

[0068] (3) Obtaining an output code stream based on the connection relationship encoding result and the geometric information encoding result;

[0069] Specifically, the connection relationship code stream and the geometric information code stream may be mixed to obtain the output code stream.

[0070] Please refer to Figure 4 for the main processing flow of a 3D mesh encoding framework for separate encoding of connectivity and geometric information according to an embodiment of the present invention. In this separate encoding framework, the input 3D mesh undergoes connectivity encoding and geometric information encoding, both of which are achieved by traversing triangles. Taking the example of a case where connectivity encoding and geometric information encoding have the same triangle traversal order, this approach demonstrates that geometric information encoding requires the use of the connectivity encoding order, which reflects the triangle traversal order during the connectivity encoding process. In other words, the connectivity encoding stream is obtained by traversing all triangles to complete connectivity encoding, and the geometric information stream is also obtained by traversing all triangles to complete geometric information encoding in the same order. The output stream is a mixture of these two streams. Of course, the above diagrams illustrate only the case where connectivity encoding and geometric information encoding have the same triangle traversal order. For cases where connectivity encoding and geometric information encoding have different triangle traversal orders, please refer to the above text for understanding and will not be further illustrated here. The connectivity encoding in the above steps is implemented using the concepts described above and will be described in detail later for clarity.

[0071] Specifically, for the hybrid coding framework, the input 3D mesh is coded for connectivity and geometry information to obtain an output code stream, including:

[0072] (1) Traversing each triangle in the three-dimensional mesh, performing connection relationship encoding on the traversed triangle to obtain a connection relationship encoding result of the triangle, and performing geometric information encoding on the triangle to obtain a geometric information encoding result of the triangle. The connection relationship encoding result and the geometric information encoding result of the triangle constitute the encoding result of the triangle;

[0073] (2) The output code stream is obtained from the encoding results of all triangles.

[0074] Please refer to Figure 5 for the main processing flow of the three-dimensional mesh coding framework for hybrid coding of connection relationship and geometric information applied in the embodiment of the present invention. In this hybrid coding framework, the input three-dimensional mesh is traversed, and each traversed triangle will be coded for connection relationship and geometric information, that is, when a triangle is traversed, the pattern of the triangle, any additional information that may exist, and the geometric coordinates of the vertices of the triangle that have not yet been encoded are encoded. Therefore, for the hybrid coding framework, the triangle traversal order for geometric information encoding and connection relationship encoding is consistent, and ultimately there will be only one code stream, which is a mixture of the connection relationship coding results and geometric information coding results of the triangles traversed in sequence, as the output code stream. Similarly, the connection relationship coding in the above steps is also implemented using the ideas in the previous text.

[0075] The following describes the main process of the connection relationship encoding method proposed in an embodiment of the present invention. Please refer to Figure 6 for understanding. It can be understood that for a separate coding frame, the output of Figure 6 is a connection relationship code stream, and for a mixed coding frame, the output of Figure 6 is the connection relationship encoding result of the current triangle.

[0076] Specifically, in Scode1, the process of encoding the connection relationship of the input 3D mesh mainly includes A1 to A4:

[0077] A1, initialize the three-dimensional grid and establish related data structures;

[0078] This step corresponds to the "Initializing the 3D Mesh Data Structure" section in Figure 6. The purpose of initializing the 3D mesh and establishing related data structures is to extract relevant information about the triangles in the 3D mesh, such as the relationships between vertices, corners, or edges, to facilitate subsequent traversal. These data structures may include, but are not limited to, Corner-Table data structures or Half-Edge data structures. For ease of understanding and description, this embodiment of the present invention uses the Corner-Table data structure as an example.

[0079] The Corner-Table data structure contains four arrays: V, O, M, and U. The lengths of arrays V, O, and U are three times the number of mesh patches, and the length of array M is equal to the number of patches. Mesh patches are triangles. Array V stores the vertex corresponding to each corner in the 3D mesh, and array O stores the opposite angle of each corner. Arrays M and U both store binary symbols, each indicating whether the vertex and triangle in the 3D mesh have been traversed. During initialization, each corner is assigned a sequence number based on the counterclockwise order of the triangle's interior angles. This allows the sequence number of a given corner to quickly identify the other two corners of the same triangle. Let the current corner in the triangle be c, the first corner encountered by c in clockwise order be cp, the first corner encountered by c in counterclockwise order be cn, the vertex corresponding to c be cv, the opposite angle of c be co, and the triangle to which c belongs be ct, as shown in Figure 7. For the definitions and configuration rules of angles and vertices, please refer to Understanding the Corner-Table Data Structure.

[0080] A2, for each traversed triangle, determine the traversal result of the triangle based on the data structure;

[0081] This step corresponds to the "Selecting an initial triangle and traversing the connection relationship" and "Determining the triangle mode and next traversal direction based on the triangle traversal results" sections in Figure 6. The traversal results for the triangle include the triangle mode. In an optional embodiment, the triangle mode can be a CLERS mode, but this is not a limitation.

[0082] In an optional embodiment, step A2 includes:

[0083] For each traversed triangle, determine the corresponding vertex of the triangle based on the data structure, and determine the traversal result of the triangle, including the pattern of the triangle, based on the traversal status of the corresponding vertices of the triangle and the traversal status of the triangles on both sides. Mark the triangle as a traversed triangle, and mark the three vertices of the triangle as traversed vertices.

[0084] Taking the Corner-Table data structure and the CLERS mode as an example, the above step A2 may include:

[0085] For each traversed triangle, the vertex corresponding to angle c of the triangle is determined based on the relationship between the angle and the opposite angle, as well as the angle and the vertex in the Corner-Table data structure. Based on the traversal of the vertex corresponding to angle c of the triangle obtained and the traversal of the triangles on the left and right sides, the traversal result of the triangle is determined, including the CLERS mode of the triangle, and the triangle is marked as a traversed triangle, and the three vertices of the triangle are marked as traversed vertices. It can be understood that in this specific example, the current angle is angle c.

[0086] Specifically, when traversing triangles on a 3D mesh, embodiments of the present invention divide the 3D mesh into traversed and untraversed portions. An initial triangle may be selected, for example through random selection, and traversal may begin from this initial triangle. Traversed triangles are continuously inserted into the traversed portion, and additional triangles are traversed based on the relative orientation of the newly added triangles. Furthermore, a different mode is output for each triangle based on the position of the vertex corresponding to the current corner of the triangle and the traversal of the triangles to its left and right. This also determines the traversal direction for the next triangle.

[0087] Assume that the relevant data structure is the Corner-Table data structure and the mode is the CLERS mode. Please refer to Figure 8 for understanding. Here, X is the current triangle, and v is the vertex corresponding to the current angle of the triangle, i.e., angle c. When traversing, different modes are assigned to the triangle according to the five different situations of the CLERS mode, and jump to the next triangle in a different direction.

[0088] Specifically, the traversal result of the triangle is determined based on the traversal of the vertex corresponding to the obtained angle c of the triangle and the traversal of the triangles on the left and right sides, including:

[0089] 1) For the current triangle traversed, if the vertex corresponding to the c corner of the triangle has not been traversed, determine that the CLERS mode to which the triangle belongs is C mode, and the traversal direction of the next triangle is to the right of the triangle;

[0090] Please refer to the pattern diagram marked with C in Figure 8. Pattern C indicates that v has not been traversed and should be traversed to the right of the current triangle.

[0091] 2) For the current triangle traversed, if the vertex corresponding to the c corner of the triangle has been traversed, and the triangle to the left of the triangle has been traversed, but the triangle to the right has not been traversed, the CLERS mode to which the triangle belongs is determined to be L mode, and the traversal direction of the next triangle is to the right of the triangle;

[0092] Please refer to the pattern diagram marked with L in Figure 8. The L pattern indicates that v has been traversed and should be traversed to the right of the current triangle.

[0093] 3) For the current triangle traversed, if the vertex corresponding to the c corner of the triangle has been traversed, and the triangle to the right of the triangle has been traversed, but the triangle to the left has not been traversed, the CLERS mode to which the triangle belongs is determined to be R mode, and the traversal direction of the next triangle is the left side of the triangle;

[0094] Please refer to the pattern diagram marked with R in Figure 8. The R pattern indicates that v has been traversed and should be traversed to the left of the current triangle.

[0095] 4) For the current triangle traversed, if the vertex corresponding to the c corner of the triangle has been traversed, and the left and right triangles of the triangle have not been traversed, the CLERS mode to which the triangle belongs is determined to be S mode. The traversal direction of the next triangle is to traverse the right branch of the triangle first, and then traverse the left branch of the triangle after traversing to the bottom;

[0096] Please refer to the pattern diagram marked with S in Figure 8. The S pattern indicates that v has been traversed and should first traverse the right branch, and then traverse the left branch after traversing to the end.

[0097] 5) For the current triangle traversed, if the vertex corresponding to the c corner of the triangle has been traversed, and the left and right triangles of the triangle have been traversed, it is determined that the CLERS mode to which the triangle belongs is E mode, there is no traversal direction for the next triangle, and the traversal ends.

[0098] Please refer to the pattern diagram marked with E in Figure 8. Pattern E indicates that v has been traversed and the traversal is completed.

[0099] Of course, for other modes other than the CLERS mode, the method of determining the traversal result of the triangle can be implemented in a similar way to the above process based on the traversal of the corresponding vertices of the triangle and the traversal of the triangles on both sides, and no examples are given here.

[0100] A3, using the relevant information of the traversed triangle as context information, entropy encoding the pattern of the triangle;

[0101] The relevant information includes at least one or more of vertex geometry information, encoded triangle mode, and the number of adjacent encoded triangles of a triangle vertex;

[0102] Among them, A3 corresponds to the part of "entropy encoding the triangle pattern according to relevant context information" in Figure 6.

[0103] In an optional implementation manner, A3 may include steps A31 and A32:

[0104] A31, using a context-based adaptive binary arithmetic coding method and a preset binarization method to binarize the pattern of each triangle to be encoded to obtain a corresponding binary character sequence;

[0105] The pattern of a triangle is strongly correlated with the number of traversed adjacent triangles. This pattern can be entropy-encoded using Context-Adaptive Binary Arithmetic Coding (CABAC). CABAC consists of three stages: binarization, context modeling, and binary arithmetic coding. The present invention primarily improves these two stages.

[0106] Among them, for binarization processing, an embodiment of the present invention binarizes the triangular pattern to obtain a corresponding binary character sequence. The preset binarization methods used include but are not limited to truncated unary code, Golomb code and fixed-length code.

[0107] During the binarization process, the symbol with a greater probability of occurrence (i.e., the symbol corresponding to the pattern) has a shorter codeword after binarization. Table 1 shows a possible binarization code table using a truncated unary code as an example. The codewords in Table 1 represent the corresponding binary character sequences.

[0108] Table 1 CLERS pattern and binary code table of triangle

[0109] A32, obtains relevant information of the triangle to be encoded, uses the obtained relevant information to calculate the context index corresponding to each binary character of the triangle to be encoded, and determines the corresponding context model based on the obtained context index, and uses the obtained context model to perform binary arithmetic encoding on the binary character sequence of the triangle to be encoded, and updates the context model.

[0110] Among them, for context index calculation, an embodiment of the present invention uses relevant information, such as vertex geometry information, encoded triangle pattern, and the number of adjacent encoded triangles of triangle vertices, as context information to calculate the context index.

[0111] The entropy coding framework of embodiments A31-A32 of the present invention can be seen in Figure 9. The entropy coding framework first binarizes the triangle pattern, then uses the relevant information to calculate the context index, thereby determining the context model and completing the context modeling. Finally, the context model is used to implement entropy coding of the triangle pattern, and the context model also needs to be updated.

[0112] Specifically, two different kinds of relevant information are given below to illustrate the calculation process of the context index respectively.

[0113] The first method is to obtain relevant information of the triangle to be encoded when the relevant information is the number of adjacent encoded triangles of the triangle vertex, and use the obtained relevant information to calculate the context index corresponding to each binary character of the triangle to be encoded, including:

[0114] i) for the triangle to be encoded, obtaining the numbers P1 and P2 of adjacent traversed triangles corresponding to the vertex corresponding to the predetermined angle and the vertex corresponding to the preset angle respectively as relevant information of the triangle to be encoded;

[0115] ii) Substituting P1 and P2 into a preset formula for calculating the index of the adjacent coded situation of two triangle vertices to obtain the index of the adjacent coded situation of two triangle vertices of the triangle to be coded;

[0116] iii) For each binary character in the binary character sequence corresponding to the to-be-encoded triangle, the context index of the binary character is obtained using a preset context index calculation formula according to the position of the binary character and the adjacent encoded index of the two vertices of the triangle.

[0117] The following uses the Corner-Table data structure and the CLERS model as an example. The three steps above correspond to ①②③ below in the first method:

[0118] ① For the triangle to be coded, obtain the number of adjacent traversed triangles P1 and P2 corresponding to the vertex corresponding to the angle cn and the vertex corresponding to the angle cp respectively as the relevant information of the triangle to be coded;

[0119] In this case, the current angle is the c angle, the preset angle is the cp angle, and the predetermined angle is the cn angle; specifically:

[0120] After completing the binarization of the triangle pattern, the context index can be calculated based on the number of previously coded triangles adjacent to the current triangle vertex. As shown in Figure 10, the yellow triangle is the current triangle to be coded. v1 and v2 are the vertices corresponding to the triangle's angles cn and cp. Let P1 and P2 be the number of traversed triangles adjacent to v1 and v2 when encoding the current triangle, respectively. These P1 and P2 are used as the relevant information for the current triangle.

[0121] ② Substitute P1 and P2 into the preset triangle two vertex adjacent encoded index calculation formula to obtain the triangle two vertex adjacent encoded index of the triangle to be encoded;

[0122] The embodiment of the present invention uses P1 and P2 to calculate the context index and the number of bits of the binary characters, i.e., binary symbols, in the binary character sequence, and performs binary arithmetic coding on the binary symbol sequence.

[0123] The following uses truncated unary codes as an example of a binarization method to describe a context index calculation process for a triangle pattern. Of course, other binarization methods are similar and will not be described in detail here.

[0124] Let traversedValence represent the index of the adjacent coded situation of two vertices of a triangle. The preset calculation formula of the adjacent coded situation index of two vertices of a triangle includes: traversedValence=P1+10×P2 (1)

[0125] Among them, traversedValence represents the index of the adjacent encoded situation of two vertices of a triangle.

[0126] Of course, the preset index calculation formula for the adjacent encoded situation of two vertices of a triangle in the embodiment of the present invention is not limited to the above form. For example, 10 can also be set to other values ​​according to needs, or other weights can be used to perform weighted calculation on P1 and P2, and so on.

[0127] ③ For each binary character in the binary character sequence corresponding to the triangle to be encoded, the context index of the binary character is obtained by using the preset context index calculation formula according to the position of the binary character and the adjacent encoded index of the two vertices of the triangle.

[0128] When the binarization method used is truncated unary code, the length of the longest codeword corresponding to the binarized pattern is 4. It is necessary to establish a suitable context model for each binary character at different positions. The context index can be calculated based on the current adjacent traversed triangle situation and the binary character position. Let the binary symbol position be bitIndex, and the value range is bitIndex∈{0,1,2,3}. The preset context index calculation formula includes: ctxIdx=bitIndex+4×traversedValence (2)

[0129] Among them, traversedValence represents the index of the adjacent encoded situation of two vertices of the triangle; bitIndex represents the position of the binary character; ctxIdx represents the context index of the binary character.

[0130] It is understandable that for a binary character sequence corresponding to a triangular pattern, a corresponding ctxIdx is calculated for each binary character. It is understandable that after the calculation of the context index is completed, the context model can be obtained accordingly, the binary character can be binary arithmetic coded using the context model, and the context model can be updated. Regarding the process of obtaining the context model for encoding after obtaining the context index, please refer to the relevant content of CABAC, which will not be explained in detail here.

[0131] It should be noted that the above formula (2) is only an example of a preset context index calculation formula and does not constitute a limitation thereto.

[0132] The embodiment of the present invention calculates the context index using the number of encoded triangles adjacent to the current triangle vertex. Compared with the prior art, the context of the embodiment of the present invention uses more information related to the triangle pattern and can encode the connection relationship more efficiently.

[0133] The second method is to obtain relevant information of the to-be-encoded triangle when the relevant information is in an encoded triangle mode, and calculate the context index corresponding to each binary character of the to-be-encoded triangle using the obtained relevant information, including:

[0134] ① For each binary character in the binary character sequence corresponding to the triangle to be encoded, determine the context index of the binary character using only the pattern of the previous triangle of the triangle to be encoded;

[0135] Still using the previous example, let's assume that the codeword of the triangle pattern after the current input is binarized is value, that is, the binary character sequence represented by 0 / 1. The context index ctxIdx is initialized to 0. When the binarization method used is truncated unary code, the length of the longest codeword corresponding to the binarized pattern is 4. Different context models can be established for binary characters at different positions. The context index ctxIdx of the current binary character can only consider the pattern of the previous triangle to calculate the context index of the current triangle to be encoded. See the following formula: ctxIdx = value (3)

[0136] or,

[0137] ② For each binary character in the binary character sequence corresponding to the triangle to be encoded, the context index of the binary character is determined by using the pattern of the previous triangle of the triangle to be encoded and the context index obtained by the previous triangle.

[0138] In this case, the context index of the current triangle is calculated by considering the mode of the previous triangle and the context index obtained by the previous triangle. The calculation formula can be: ctxIdx=value+((ctxIdx & 1) <<3)+((ctxIdx & 4) <<2) (4)

[0139] Among them, & represents logical AND; (ctxIdx & 1)<<3) represents logical AND of ctxIdx with 1, followed by a left shift of three bits; (ctxIdx & 4)<<2 represents logical AND of ctxIdx with 4, followed by a left shift of two bits.

[0140] It is also understood that after calculating the context index, a corresponding context model can be obtained, the context model can be used to perform binary arithmetic coding on the binary character, and the context model can be updated. For the process of obtaining the context model after obtaining the context index and performing encoding, please refer to the relevant content of CABAC, which will not be explained in detail here.

[0141] A4: If the traversed triangle meets the preset conditions, the corresponding additional information is determined, incorporated into the traversal result of the triangle, and the additional information is entropy-encoded;

[0142] After determining the pattern of the traversed triangles, if the traversed triangles meet preset conditions, embodiments of the present invention further perform some special processing to obtain additional information indicating the positional relationship of the triangles and perform entropy encoding for transmission in the output bitstream, allowing the decoder to use this additional information for appropriate stitching and reconstruction. The entropy encoding of the additional information can be implemented using any entropy encoding method, without limitation herein.

[0143] The following is a detailed description of the different additional information:

[0144] 1) In an optional implementation, if the traversed triangle meets a preset condition, corresponding additional information is determined, including:

[0145] If the traversed triangle pattern matches the predetermined pattern, information representing the position of the triangle in the 3D mesh is determined as additional information. To ensure that the codec can utilize the same context information, embodiments of the present invention require that information representing the position of the triangle in the 3D mesh be transmitted as additional information for triangles in the predetermined pattern.

[0146] In an optional implementation, if the pattern of the traversed triangle is a predetermined pattern, determining the representation information of the position of the triangle in the three-dimensional grid as additional information includes:

[0147] B1, determining the representation information of the position of the current vertex of the triangle in the three-dimensional grid;

[0148] For a triangle to be encoded in a predetermined pattern, the encoder will encode and transmit the position of the current corner vertex in the three-dimensional grid, so that the decoding end can use the context information to decode the triangle pattern as the predetermined pattern, and immediately reconstruct the vertices of the triangle to their actual positions in the grid, so that subsequent decoding can use the same context information as the encoder.

[0149] Taking the CLERS mode as an example, the predetermined mode may be the S mode. In this case, B1 is the characterization information for determining the position of the c-angle vertex of the triangle in the three-dimensional grid. Other cases are not described in detail here.

[0150] In an optional implementation manner, the characterization information for determining the position of the current vertex of the triangle in the three-dimensional mesh in B1 may include:

[0151] B11, using the vertex rotation method, starts from the preset corner vertex of the triangle, continuously changing the rotation vertex until the vertex position corresponding to the current corner is found, and records the number of vertex rotations experienced during the period as a simplified representation of the position of the current corner vertex in the three-dimensional grid.

[0152] For triangles with a predetermined pattern, the position of the current corner vertex in the three-dimensional grid will be encoded and transmitted, also known as the position of the entry edge to the vertex. However, the actual information often has a large amount of data, and direct encoding and transmission will increase the transmission pressure. Therefore, the embodiment of the present invention uses the number of vertex rotations to indirectly represent the position of the current corner vertex of the triangle in the three-dimensional grid. As a simplified representation of information, it is only necessary to encode and transmit a numerical value with a smaller amount of data. Therefore, it can reduce the transmission pressure and improve the transmission efficiency. However, the decoding end can still use the rotation information to effectively locate the position of the current corner vertex of the triangle in the three-dimensional grid, thereby ensuring the accuracy of decoding and reconstruction.

[0153] Taking the Corner-Table data structure and CLERS mode as an example, the current angle in B11 is the c angle, and the preset angle is the cp angle.

[0154] Specifically, referring to the schematic diagram of the process of recording the number of vertex rotations shown in FIG11 , the rotation is started from the vertex of angle cp, and the rotation vertex is continuously changed until the vertex position corresponding to angle c is found. During this process, the number of times the rotation vertex is changed is recorded. The specific steps of B11 may include:

[0155] Step a1, record the vertex corresponding to the triangle angle c as the target vertex v1, v1 = cv; let v t = cpv, b = c, number of rotations l = 0;

[0156] Among them, v1, vt , b and l are parameters set in the embodiment of the present invention; cv and c can be understood by referring to the Corner-Table data structure, and those skilled in the art can understand from the data structure that cpv represents the vertex corresponding to the cp angle.

[0157] Step a2, let b = bop, if the vertical angle of b has been traversed, continue around v t Rotate and repeat step a2; if the vertical angle of b is not traversed, let v t =bnv, b=bp, l=l+1, go to step a3;

[0158] Similarly, according to the Corner-Table data structure, bop represents the previous corner of b (the previous corner here refers to the first corner of bo in the clockwise direction), bnv represents the vertex corresponding to the next corner of b (the next corner here refers to the first corner of b in the counterclockwise direction), and bp represents the previous corner of b (the previous corner here refers to the first corner of b in the clockwise direction). l = l + 1 means that the current l is incremented by 1 to obtain the updated l.

[0159] Step a3, determine v t Is it equal to v1? If so, determine that the current l is the number of vertex rotations and end the rotation search process; if not, return to step a2 and continue around vertex v t Perform a rotation search.

[0160] In another optional implementation, the characterization information for determining the position of the current vertex of the triangle in the three-dimensional grid in B1 may include:

[0161] Complete representation information is obtained based on the position of the current corner vertex of the triangle in the three-dimensional mesh. The complete representation information includes a triangle traversal index when the current corner vertex is traversed for the first time.

[0162] This method can be performed directly or when simplified representation information cannot be obtained through vertex rotation, which is reasonable.

[0163] Taking the Corner-Table data structure and the CLERS mode as an example, when the encoding end encounters the S mode, it may encounter a special situation as shown in Figure 12. Figure 12 is a schematic diagram of an embodiment of the present invention in which the target vertex of the S-mode triangle cannot be rotated and connected. The mode of the yellow triangle in the figure is S. The vertex corresponding to the c-angle of the triangle in the S mode, that is, the target vertex, cannot be found by the above rotation method. Therefore, the relevant information of the position of the vertex in the three-dimensional grid can be transmitted as a complete representation information, so that the decoding end can use the complete representation information to complete the stitching of the corresponding vertices. For example, the triangle traversal index when traversing to the vertex corresponding to the c-angle for the first time can be transmitted. The triangle traversal index refers to the number of the triangle when traversing to the vertex corresponding to the c-angle for the first time according to the triangle traversal order of the encoding connection relationship.

[0164] 2) In an optional implementation, if the traversed triangle meets a preset condition, corresponding additional information is determined, including:

[0165] When there are encoded adjacent triangles on both sides of the traversed triangle, it is determined whether the pattern of the traversed triangle has a ring property. If so, the diagonal information of the triangle is recorded as additional information.

[0166] Specifically, when there are encoded adjacent triangles on both sides of the traversed triangle, if it is determined that the pattern of the traversed triangle has a ring property, it means that the triangle on one side of the predetermined pattern can be traversed to the triangle on the other side at one time. Please refer to Figure 13 for a schematic diagram.

[0167] Taking the Corner-Table data structure and the CLERS pattern as an example, when there are encoded adjacent triangles on both sides of the traversed triangle, this means that the CLERS pattern is one of E, R, or L. In this case, the pattern of the traversed triangle has a ring property and can be called a handle. Of course, the above situation is only an example and does not limit the embodiments of the present invention.

[0168] Taking the above example as an example, the situation of traversing from one branch of the S-mode triangle to another branch is called a handle. For E, R, and L modes, it is necessary to detect whether the mode is a handle at the encoding end. If it is a handle, the diagonal information of the triangle is recorded, that is, the handle information, which represents the diagonal information of the handle, that is, recording c and co. If it is not a handle, it is necessary to determine whether the rotation vertex is connected to two or more S-mode triangles. If so, it is necessary to record the sector number; if not, skip the recording step and proceed to subsequent operations. See below for the recording of sector numbers.

[0169] For the handle, since the decoder cannot find the diagonal corner and the vertex of angle c through reconstruction operations, the encoder needs to directly transmit the diagonal information to the decoder, and then the decoder updates the vertex of angle c based on this diagonal information.

[0170] Of course, when there are encoded adjacent triangles on both sides of the traversed triangle, and it is determined that the other possible situations of the pattern of the traversed triangle has the property of a ring are not limited to those shown above, and accordingly, the recorded diagonal information of the triangle is also not subject to the above restrictions.

[0171] 3) In an optional implementation, if the traversed triangle meets a preset condition, corresponding additional information is determined, including:

[0172] If there are encoded adjacent triangles on both sides of a traversed triangle, and the triangle pattern does not have a ring property, the system determines whether the rotation vertex used in the current pattern is connected to two or more triangles in the predetermined pattern. If so, the system determines, based on the triangle traversal order, which sector the corner to be stitched lies on, and obtains the sector number as additional information. Specifically, the reconstruction operation may be, but is not limited to, a zip operation.

[0173] Taking the Corner-Table data structure, CLERS mode, and the reconstruction operation as zip operation as an example, when there are encoded adjacent triangles on both sides of the traversed triangle, it means that the CLERS mode is one of E, R, and L, and the predetermined mode is S mode. Specifically:

[0174] When performing a zip operation (or stitching operation) on the decoder, if the surrounding vertex is adjacent to two or more S-pattern triangles, diagonal connection errors may occur, as shown in Figure 14. Therefore, when the encoder encounters a triangle in an E, R, or L pattern that requires a zip operation, and if the triangle pattern does not have a ring property, the encoder determines whether the zip rotation vertex of the current pattern is connected to two or more S-pattern triangles. If so, the decoder transmits the sector number of the angle to be zipped based on the triangle traversal order. In other words, the decoder assigns a sector number for encoding and transmission. This allows the decoder to determine the sector number of the angle to be stitched based on the sector number and perform the zip operation, thereby improving the accuracy of 3D mesh reconstruction on the decoder.

[0175] Specifically, for the recorded sector number, the encoder determines whether it is necessary to determine whether the rotation vertex is connected to two or more pattern S triangles. The specific determination method includes:

[0176] ① If the current triangle mode is L, check whether the left branch is a handle. If so, record the handle information; otherwise, determine whether the rotation vertex is connected to two or more triangles of mode S.

[0177] ① If the current triangle mode is R, check whether the right branch is a handle. If so, record the handle information; otherwise, determine whether the rotation vertex is connected to two or more triangles of mode S.

[0178] ③ If the current triangle pattern is E, check the branches on both sides to ensure the order is consistent between the codec and encoder. The following demonstrates a feasible detection method: First, check whether the right branch is a handle. If so, record the handle information; otherwise, determine whether the rotation vertex is connected to two or more triangles in the pattern S. Then, check whether the left branch is a handle. If so, record the handle information and determine whether the rotation vertex is connected to two or more triangles in the pattern S. Otherwise, determine whether the rotation vertex is connected to two or more triangles in the pattern S.

[0179] The above is the main content of connection relationship encoding. The following is a brief description of the process of geometric information encoding using a separate encoding framework as an example.

[0180] In an optional implementation, geometric information encoding is performed on triangles in a three-dimensional mesh according to a connection relationship encoding order, and a geometric information code stream is obtained from the geometric information encoding results of all triangles, including:

[0181] (1) Based on the connection relationship coding order, the geometric information of each triangle in the three-dimensional grid is predicted using the parallelogram prediction method to obtain the geometric coordinate residual;

[0182] Among them, the prediction methods include single parallelogram prediction, multi-parallelogram prediction and weighted parallelogram prediction.

[0183] Please refer to Figure 15 to understand various parallelogram prediction methods. In Figure 15, (a), (b), and (c) represent single parallelogram prediction, multi-parallelogram prediction, and weighted parallelogram prediction, respectively. For the specific prediction process, please refer to the relevant technical understanding and will not be elaborated here.

[0184] (2) Perform entropy coding on the predicted geometric coordinate residuals to obtain a geometric information code stream.

[0185] Among them, the entropy coding method can be selected as needed. For the specific process, please refer to the relevant technical understanding.

[0186] Those skilled in the art will appreciate that the hybrid coding framework is similar to the geometric coordinate prediction process and entropy coding described above, and will not be described in detail here. Furthermore, the geometric information coding process of the embodiment of the present invention is not limited to the above example.

[0187] A 3D mesh encoding method proposed in an embodiment of the present invention specifically encodes connectivity relationships, including encoding the pattern of triangles within the 3D mesh and encoding additional information indicating the positional relationships of the triangles. Connectivity encoding is achieved through entropy coding of contextual information. Encoding and transmitting the additional information indicating the triangle positions on the encoder side enables the decoder to utilize these relationships during stitching, ensuring accurate and immediate stitching.

[0188] Furthermore, in an embodiment of the present invention, relevant information of a triangle is used as context information, which includes at least one or more of vertex geometry information, encoded triangle patterns, and the number of adjacent encoded triangles at triangle vertices; since the number of adjacent encoded triangles at triangle vertices and the pattern are strongly correlated, entropy encoding of the triangle pattern is performed using triangle-related information such as the number of adjacent encoded triangles at triangle vertices. This can more accurately estimate the probability of pattern occurrence, thereby improving the accuracy of probability estimation and improving coding efficiency, and can obtain context information consistent with that at the encoding end at the decoding end to ensure accurate decoding and mesh reconstruction.

[0189] Furthermore, for triangles in a predetermined pattern, the additional information encoded in embodiments of the present invention includes information representing the triangle's position within the 3D mesh. This allows the decoder to use contextual information consistent with that of the encoder to decode the triangle pattern as the predetermined pattern and immediately reconstruct the triangle's vertices to their actual positions within the mesh, allowing subsequent decoding to use the same contextual information as that of the encoder. Recording the number of vertex rotations as the current vertex's position within the 3D mesh reduces transmission pressure and improves transmission efficiency while ensuring the accuracy of 3D reconstruction at the decoder.

[0190] Furthermore, when there are encoded adjacent triangles on both sides of the traversed triangle, the additional information encoded and transmitted by the embodiment of the present invention includes the diagonal information of the triangle recorded when the detection pattern has a ring property, thereby facilitating the accuracy of the stitching process on the decoding end. Also, when there are encoded adjacent triangles on both sides of the traversed triangle, the embodiment of the present invention determines that the pattern of the triangle does not have a ring property, and when the rotation vertex used in the current pattern is connected to two or more triangles of a predetermined pattern during the reconstruction operation, the sector number corresponding to the sector connected to the rotation vertex whose angle needs to be reconstructed is determined based on the traversal order of the triangles, and encoded and transmitted, which enables the decoding end to ensure accuracy during stitching.

[0191] In a second aspect, corresponding to the three-dimensional grid encoding method provided in the first aspect, an embodiment of the present invention further provides a three-dimensional grid encoding device, applied to an encoding end, the device comprising:

[0192] The encoding module is used to encode the connection relationship and geometric information of the input 3D mesh to obtain an output code stream;

[0193] The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of the triangles; the connection relationship coding is implemented based on entropy coding of context information.

[0194] Among them, the specific processing process of the above modules can be found in the three-dimensional grid coding method described in the first aspect, which will not be repeated here.

[0195] In a third aspect, corresponding to the three-dimensional grid encoding method provided in the first aspect, an embodiment of the present invention further provides a three-dimensional grid decoding method, which is applied to a decoding end, as shown in FIG16 , and includes:

[0196] Sdecode1 decodes the connection information and geometric information of the input code stream to obtain the connection relationship and geometric information respectively;

[0197] Sdecode2, reconstructs the 3D mesh based on the obtained connectivity and geometric information;

[0198] Among them, the connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and at the same time decoding the additional information used to indicate the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

[0199] Compared with the three-dimensional grid encoding method provided in the first aspect of the embodiment of the present invention, which can be applied to a separate encoding framework and a hybrid encoding framework, the three-dimensional grid decoding method provided in the third aspect of the embodiment of the present invention can also be applied to a separate decoding framework and a hybrid decoding framework.

[0200] Specifically, for a separate decoding framework, the input code stream is decoded for connection information and geometric information to obtain connection relationships and geometric information respectively; and a three-dimensional mesh is reconstructed based on the obtained connection relationships and geometric information, including:

[0201] (1) De-streaming the input code stream to obtain a connection information code stream and a geometric information code stream; wherein the connection relationship code stream and the geometric information code stream are based on triangles;

[0202] (2) decoding the connection relationship of the connection information code stream to obtain the connection relationship;

[0203] (3) Decoding the geometric information code stream to obtain geometric information;

[0204] The geometric information decoding may be performed according to the decoding order of the connection relationship.

[0205] (4) Reconstruct the 3D mesh using connection relationships and geometric information.

[0206] Please refer to Figure 17 for the main processing flow of the three-dimensional mesh decoding framework for separate decoding of connection relationships and geometric information in the embodiment of the present invention. In this separate decoding framework, both connection relationship decoding and geometric information decoding are achieved by traversing triangles, but there is no restriction on whether the triangle traversal order is the same. Taking the same triangle traversal order as an example, this method is reflected in the need to utilize the connection relationship decoding order for geometric information decoding, and the connection relationship decoding order reflects the triangle traversal order in the connection relationship decoding process. In other words, the connection relationship output after connection relationship decoding is obtained after traversing all triangles to complete connection relationship decoding, and the geometric information is also obtained after traversing all triangles in the same order to complete geometric information decoding. Of course, the same triangle traversal order is only used as a possible example. For the case where the triangle traversal order is different, the processing process of the embodiment of the present invention will not be described in detail here. The connection relationship decoding method in the above steps corresponds to the encoding idea in the previous text. For the sake of clarity, it is described in detail later.

[0207] Specifically, for the hybrid decoding framework, the input code stream is decoded for connection information and geometric information to obtain connection relationship and geometric information respectively; and a 3D mesh is reconstructed based on the obtained connection relationship and geometric information, including:

[0208] (1) The input code stream is traversed in triangle order, and the connection relationship and geometric information of each traversed triangle are decoded to obtain the connection relationship and geometric information of the triangle;

[0209] (2) Reconstruct the three-dimensional mesh of the triangle using the connection relationship and geometric information of the triangle.

[0210] See Figure 18 for the main processing flow of a 3D mesh decoding framework for separate decoding of connectivity and geometry information, as applied in an embodiment of the present invention. In this hybrid decoding framework, the input bitstream undergoes triangle traversal, and each traversed triangle undergoes both connectivity decoding and geometry decoding before 3D mesh reconstruction. This means that each triangle is traversed, decoding its pattern and the geometric coordinates of its undecoded vertices. Therefore, the triangle traversal order for both geometry decoding and connectivity decoding is consistent, until all triangles are traversed to obtain the final reconstructed 3D mesh.

[0211] Similarly, the connection relationship decoding method in the above steps also corresponds to the encoding idea in the previous article.

[0212] The following describes the main process of the connection relationship decoding method proposed in an embodiment of the present invention. Please refer to Figure 19 for understanding. It can be understood that for a separate coding frame, the corresponding code stream input in Figure 19 is a connection relationship code stream, and for a mixed coding frame, the corresponding code stream input in Figure 19 can be the overall input code stream.

[0213] The decoding end first performs entropy decoding on the pattern triangle based on certain context information, that is, relevant information about the triangle, such as one or more of the vertex geometry information, the encoded triangle pattern, and the number of encoded triangles adjacent to the triangle vertex, but must be consistent with the encoding end. Then, the current triangle is connected to the reconstructed 3D mesh using different stitching methods based on its pattern, and then the position of the next triangle is determined, and the pattern of the next triangle is decoded using the context information entropy. This is repeated until all triangles are decoded and reconstructed to obtain a reconstructed 3D mesh.

[0214] Specifically, in Sdecode1, the process of decoding the connection relationship of the input code stream mainly includes D1 to D3:

[0215] D1, entropy decoding the input bitstream to obtain the current triangle mode and additional information indicating the triangle position relationship;

[0216] The entropy decoding process at the decoding end corresponds to the entropy encoding process at the encoding end, which can be understood in conjunction with the previous description. The mode can be the CLERS mode, which will not be explained in detail here. It can be understood that in addition to decoding the mode to which the current triangle belongs, additional information indicating the positional relationship of the triangles is also decoded.

[0217] The following is a detailed description of the decoding of the additional information.

[0218] 1) In an optional implementation, the process of performing entropy decoding on the input bitstream to obtain additional information indicating the positional relationship of the triangles includes:

[0219] The input code stream is entropy decoded, and when the mode of the current triangle is the predetermined mode, the additional information obtained is the representation information of the position of the triangle in the three-dimensional grid.

[0220] Specifically, entropy decoding may be performed on the input code stream to obtain representation information of the position of the current corner vertex of the triangle in the three-dimensional grid.

[0221] Taking the Corner-Table data structure and CLERS mode as an example, the predetermined mode is the S mode and the current angle is the c angle.

[0222] Optionally, the representation information of the position of the current corner vertex of the triangle in the three-dimensional mesh may include simplified representation information or complete representation information.

[0223] If the current triangle is in a predetermined pattern, the decoder needs to decode its vertex position information to use this representation information to complete the stitching of the corresponding vertices. For the simplified and complete representation information, please refer to the description of the encoder above and will not be repeated here.

[0224] 2) In an optional implementation, the process of performing entropy decoding on the input bitstream to obtain additional information indicating the positional relationship of the triangles includes:

[0225] The input code stream is entropy decoded. When there are traversed adjacent triangles on both sides of the traversed triangle and the pattern has a ring property, the additional information obtained is the diagonal information of the triangle, which is used for reconstruction operation.

[0226] Among them, taking the CLERS mode as an example, when there are traversed adjacent triangles on both sides of the traversed triangle, it can be one of the E, R, and L modes. The ring property and the diagonal information of the triangle can be described in the previous encoding end, which will not be repeated here; the reconstruction operation can be a zip operation.

[0227] 3) In an optional implementation, the process of performing entropy decoding on the input bitstream to obtain additional information indicating the positional relationship of the triangles includes:

[0228] The input code stream is entropy decoded. If there are traversed adjacent triangles on both sides of the traversed triangle, if the pattern does not have a ring property and the corresponding rotation vertex is connected to two or more triangles of a predetermined pattern, the additional information obtained is the fan number of the triangle.

[0229] Among them, taking the CLERS mode as an example, when there are traversed adjacent triangles on both sides of the traversed triangle, it can be one of the E, R, and L modes, and the predetermined mode can be the S mode. Please refer to the description of the encoding end above for the meaning of the sector number, which will not be repeated here. In the above-mentioned several implementation methods, entropy decoding can be selected as needed and is not limited here.

[0230] D2: Connecting the current triangle to the reconstructed 3D mesh based on the pattern and its corresponding stitching method. This step may include: determining initial diagonal information of the current triangle based on the pattern, and updating the diagonal information based on the stitching method corresponding to the pattern to connect the current triangle to the reconstructed 3D mesh. If the traversed triangle meets a preset condition, the stitching method for the triangle is implemented based on the corresponding additional information.

[0231] For a clear layout, the relevant content of D2 is explained after D3.

[0232] D3: Determine the position of the next triangle according to the pattern, obtain relevant information required to decode the triangle, and use the obtained relevant information to decode the pattern of the triangle.

[0233] It can be understood that after obtaining the relevant information required for decoding the triangle, the process returns to the pattern entropy decoding process.

[0234] The relevant information includes at least one or more of: vertex geometry information, encoded triangle mode, and the number of adjacent encoded triangles of a triangle vertex.

[0235] Specifically, for D2, if the context information uses the number of adjacent encoded triangles at a triangle vertex, then after completing entropy decoding of the triangle pattern, it is necessary to immediately connect the current triangle to the reconstructed mesh. This ensures that the context information used for entropy decoding of the next triangle remains consistent with that of the encoder. The process of reconstructing the mesh on the decoder is the process of reconstructing the encoder's network structure (e.g., Corner Table, Half-Edge, etc.). This section of the present invention uses the reconstruction of the Corner Table as an example to explain the connection relationship reconstruction process.

[0236] Since the connection relationship of the triangle can be reconstructed by determining the diagonal relationship of each angle and the relationship between the vertices corresponding to the angle, reconstructing the connection relationship is essentially to reconstruct the O table and V table in the Corner-Table.

[0237] During triangle reconstruction, the initial diagonal relationship is first set based on the current triangle mode. Setting it to -1 or -2 indicates that the diagonal relationship of the current angle is undetermined. Other symbols can also be used instead of -1 or -2, as shown in Figure 20.

[0238] Taking the Corner-Table data structure and CLERS mode as an example, the initial diagonal information of the current triangle is determined according to the mode, including:

[0239] ① If the CLERS mode of the current triangle is C mode, the initial diagonal information is determined to be co=2, cpo=1, cno=-1;

[0240] ② If the CLERS mode of the current triangle is L mode, the initial diagonal information is determined to be co = 2, cpo = 1, cno = -2;

[0241] ③ If the CLERS mode of the current triangle is R mode, the initial diagonal information is determined to be co = -2, cpo = 1, cno = 2;

[0242] ④ If the CLERS mode of the current triangle is S mode, the initial diagonal information is determined to be co=2, cpo=1, cno=3;

[0243] ⑤ If the CLERS mode of the current triangle is E mode, the initial diagonal information is determined to be co=-2, cpo=1, cno=-2.

[0244] In the above process, some corners cannot establish the correct diagonal relationship, that is, the edges corresponding to some corners are not stitched. Therefore, the embodiment of the present invention needs to stitch the corresponding edges of the triangles to the reconstructed mesh immediately after decoding the triangle pattern.

[0245] In the embodiments of the present invention, the diagonal information is updated differently for different modes. Taking the above example, the three modes (E, R, and L) are divided into handle and non-handle cases. In the handle case, the diagonal information is updated using the decoded diagonal information. In the non-handle case, the diagonal information is updated using the corresponding method in the mode stitching method. For the S and C modes, the diagonal information is updated using the corresponding method in the mode stitching method.

[0246] The following are stitching methods for different modes on the decoder side, using the Corner-Table data structure and the CLERS mode as an example.

[0247] (1) Suturing methods corresponding to L-mode include:

[0248] Step b1: Determine whether the left branch of the current triangle angle c is a handle. If so, update the diagonal information of angle c to the decoded diagonal information, update cnv, and end stitching; otherwise, jump to step b2;

[0249] The method for determining whether the left branch of the current triangle's corner c is a handle is discussed in the related technical literature and will not be explained here. If it is a handle, since the handle cannot be stitched to find the opposite corner, the encoder must transmit the opposite corner information. Updating cnv means cnv = copv. At this point, ending stitching means ending the stitching of this triangle and then proceeding to the next triangle.

[0250] Step b2, determine whether the cpv is connected to multiple S-mode triangles, if yes, jump to step b6, otherwise jump to step b3;

[0251] Step b3, initialize angle b, set b = cn, as shown in the first figure in Figure 21;

[0252] Step b4: Determine whether bo is greater than 0. If so, set b = bon, as shown in the second figure in Figure 21, and repeat step b4. If bo is less than or equal to 0, as shown in the third figure in Figure 21, jump to step b5.

[0253] Step b5, set bo = c, co = b, bpv = cnv, as shown in the fourth figure in Figure 21, and end the suture;

[0254] Step b6, read from the corresponding sector number the number of sectors on which the angle that needs to be zipped according to the traversal order of the triangle is located, and obtain the t angle of the sector, let b = tp, as shown in Figure 22, and jump to step b5; wherein, the t angle of the sector is the angle connecting the sector and its rotation vertex.

[0255] To understand the L-pattern stitching process, refer to Figure 21 for a schematic diagram of the stitching process for pattern L and Figure 22 for a schematic diagram of a rotated vertex of the L-pattern adjacent to two S-pattern triangles. For the symbols used for the corners and vertices in each stitching process, refer to the Corner-Table and the encoding method described above.

[0256] (2) Suturing methods corresponding to E-mode include:

[0257] Step c1: determine whether the right branch of the current triangle angle c is a handle. If so, update the diagonal information of angle c to the decoded diagonal information and update cvp; where updating cvp means cpv = conv;

[0258] Step c2: determine whether the left branch of angle c is a handle. If so, update cnv and end stitching; otherwise, jump to step c3;

[0259] Step c3, determine whether the cpv is connected to multiple S-mode triangles, if yes, jump to step c8, otherwise jump to step c4;

[0260] Step c4, initialize angle b, set b = cn;

[0261] Step c5, determine whether bo is greater than 0. If so, set b = bon and repeat step c5; if bo is less than or equal to 0, jump to step c6;

[0262] Step c6, let bo = c, co = b, bpv = cnv;

[0263] Step c7, let c = cp, if co is not equal to -2, jump to step c4, otherwise end suturing;

[0264] Step c8, read from the corresponding sector number the number of sectors on which the angle that needs to be zipped according to the traversal order of the triangle is located, obtain the t angle of the sector, set b = tp, and jump to step c6; wherein, the t angle of the sector is the angle connecting the sector and its rotation vertex.

[0265] Except for handle detection, the stitching methods of L-mode and E-mode are basically the same, which can be used as a reference for understanding.

[0266] (3) Suturing methods corresponding to R mode include:

[0267] Step d1: determine whether the right branch of the current triangle c corner is a handle. If so, update the diagonal information of the c corner to the decoded diagonal information, update the CPV, and end the stitching; otherwise, jump to step d2;

[0268] Step d2, determine whether cnv is connected to multiple S-pattern triangles, if yes, jump to step d5, otherwise jump to step d3;

[0269] Step d3, initialize angle b, set b = cp, as shown in the first figure of Figure 23;

[0270] Step d4, determine whether bo is greater than 0. If so, set b = bop, as shown in the second figure of Figure 23, and repeat step d4; if bo is less than or equal to 0, as shown in the third figure of Figure 23, jump to step d5;

[0271] Step d5, set bo = c, co = b, bnv = cpv, as shown in the fourth figure of Figure 23, and end the suturing;

[0272] Step d6, read from the corresponding sector number the number of sectors on which the angle that needs to be zipped according to the traversal order of the triangle is located, obtain the t angle of the sector, set b = tn, and jump to step d5; wherein the t angle of the sector is the angle connecting the sector and its rotation vertex.

[0273] Please refer to the schematic diagram of the suturing process of mode R shown in Figure 23 to understand the suturing process of mode R.

[0274] (IV) Suturing methods corresponding to the S pattern include:

[0275] Based on the decoded representation of the position of the triangle's c-angle vertex in the 3D mesh, the corresponding vertex is found and connected according to the same rules as the encoding end;

[0276] When performing the stitching operation, if the representation information is complete, the vertex-related information is read for stitching; if the representation information is simplified, the number of vertex rotations is read, and the rotation is performed using the same method as the encoding end, and finally the vertices of cp are updated.

[0277] Specifically, for the S-mode, the decoder simply connects the cp corner vertex to the mesh. The decoder first decodes the position of the c corner vertex in the mesh, then finds the corresponding vertex and connects it using the same rules as the encoder, as shown in Figure 11. On the decoder, when performing the stitching operation, if the situation is as shown in Figure 12, the vertex-related information is read and stitched. Otherwise, the rotation count is read and the rotation is performed using the same method as the encoder, ultimately updating the cp vertex.

[0278] The above is the process of decoding the connection relationship at the decoding end.

[0279] For geometric information decoding, taking a separate decoding framework as an example, the geometric information code stream can be decoded according to the connection relationship decoding order to obtain geometric information, including:

[0280] 1) Perform entropy decoding on the geometric information code stream to obtain the prediction residual;

[0281] 2) Using the geometric coordinate prediction method corresponding to the encoding end to obtain the predicted geometric coordinates;

[0282] 3) The reconstructed geometric coordinates are obtained based on the sum of the predicted geometric coordinates and the prediction residuals.

[0283] Those skilled in the art will appreciate that the above decoding process corresponds to the processing process on the encoding side, and the prediction method of the geometric coordinates is consistent with that on the encoding side, which will not be described in detail here. The geometric information decoding process in the hybrid decoding framework can be similar to the above process and will not be described in detail here.

[0284] A three-dimensional mesh decoding method proposed in an embodiment of the present invention performs connectivity decoding and geometric information decoding on an input bitstream to obtain connectivity and geometric information, respectively; then, reconstructs a three-dimensional mesh based on the obtained connectivity and geometric information. Connectivity decoding includes decoding the pattern of triangles in the three-dimensional mesh and simultaneously decoding additional information indicating the positional relationships of the triangles to enable instant stitching of the current triangles. Connectivity decoding is implemented based on entropy decoding of contextual information. Connectivity decoding utilizes triangle-related information and a decoding method corresponding to the encoder to decode the corresponding portion of the input bitstream to obtain the triangle pattern. The decoded triangle pattern is then used to construct a mesh data structure to achieve three-dimensional mesh reconstruction. This allows accurate decoding and mesh reconstruction to be performed using contextual information consistent with that obtained on the encoder.

[0285] Furthermore, for triangles of a predetermined pattern, an embodiment of the present invention also decodes additional information, including the position of the current corner vertex in the three-dimensional grid, so that when reconstructing the triangle in three dimensions, the same context information as the encoding end can be used to effectively locate the vertex, thereby ensuring the accuracy of reconstruction.

[0286] Furthermore, when adjacent triangles have already been traversed on either side of a traversed triangle, embodiments of the present invention decode additional information, including the diagonal information of the triangle transmitted by the encoder. This improves the accuracy of the stitching process on the decoder side when the triangle pattern exhibits a ring property. Furthermore, additional information decoded includes the sector number transmitted by the encoder to ensure stitching accuracy.

[0287] In a fourth aspect, corresponding to the three-dimensional grid decoding method provided in the third aspect, an embodiment of the present invention further provides a three-dimensional grid decoding device, applied to a decoding end, the device comprising:

[0288] The decoding module is used to decode the connection relationship and geometric information of the input code stream to obtain the connection relationship and geometric information respectively;

[0289] A reconstruction module, used to reconstruct a three-dimensional mesh based on the obtained connection relationship and geometric information;

[0290] Among them, connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and at the same time decoding additional information used to indicate the position relationship of the triangles to achieve instant stitching of the current triangle; connection relationship decoding is achieved based on entropy decoding of context information.

[0291] For the specific processing procedures of the above modules, please refer to the three-dimensional grid decoding method described in the third aspect, which will not be described in detail here.

[0292] The above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A three-dimensional grid coding method, characterized in that: include: Encode the connection relationship and geometric information of the input three-dimensional grid to obtain an output code stream; The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of triangles; the connection relationship coding is implemented based on entropy coding of context information.

2. The three-dimensional grid coding method according to claim 1, characterized in that: The step of encoding the connection relationship and the geometric information of the input three-dimensional mesh to obtain an output code stream includes: Traversing the triangles in the three-dimensional grid to perform connection relationship coding to obtain a connection relationship coding result; Traversing the triangles in the three-dimensional grid to perform geometric information encoding to obtain a geometric information encoding result; wherein the connection relationship encoding and the geometric information encoding are performed in units of triangles; An output code stream is obtained according to the connection relationship encoding result and the geometric information encoding result.

3. The three-dimensional grid coding method according to claim 1, characterized in that: The process of encoding the connection relationship of the input 3D mesh includes: Initializing the three-dimensional grid and establishing a related data structure; For each traversed triangle, determine the traversal result of the triangle based on the data structure; wherein the traversal result of the triangle includes the mode of the triangle; Using relevant information of the traversed triangle as context information, entropy encoding the pattern of the triangle; wherein the relevant information includes at least one or more of vertex geometry information, the encoded triangle pattern, and the number of adjacent encoded triangles of the triangle vertex; If the traversed triangle meets the preset conditions, the corresponding additional information is determined, incorporated into the traversal result of the triangle, and the additional information is entropy encoded.

4. The three-dimensional grid coding method according to claim 3, characterized in that: If the traversed triangle meets the preset conditions, the corresponding additional information is determined, including: If the pattern of the traversed triangle is a predetermined pattern, characterization information of the position of the triangle in the three-dimensional grid is determined as additional information.

5. The three-dimensional grid coding method according to claim 3, characterized in that: If the traversed triangle meets the preset conditions, the corresponding additional information is determined, including: When there are encoded adjacent triangles on both sides of the traversed triangle, it is determined whether the pattern of the traversed triangle has a ring property. If so, the diagonal information of the triangle is recorded as a kind of additional information.

6. The three-dimensional grid coding method according to claim 3 or 4, characterized in that: If the traversed triangle meets the preset conditions, the corresponding additional information is determined, including: When there are encoded adjacent triangles on both sides of the traversed triangle, if the pattern of the triangle does not have a ring property, it is determined whether the rotation vertex used in the current pattern during the reconstruction operation is connected to two or more triangles of the predetermined pattern. If so, the angle to be stitched is determined based on the traversal order of the triangles. The sector number is obtained as additional information.

7. A three-dimensional grid encoding device, characterized in that: include: The encoding module is used to encode the connection relationship and geometric information of the input three-dimensional grid to obtain an output code stream; The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of triangles; the connection relationship coding is implemented based on entropy coding of context information.

8. A three-dimensional grid decoding method, characterized in that: include: Decode the connection relationship and the geometric information of the input code stream to obtain the connection relationship and the geometric information respectively; Reconstructing a three-dimensional mesh based on the obtained connection relationship and geometric information; The connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding the additional information indicating the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

9. The three-dimensional grid decoding method according to claim 8, characterized in that: The input code stream is decoded for connection relationship and geometric information to obtain connection relationship and geometric information respectively; Reconstruct the 3D mesh based on the obtained connection relationship and geometric information, including: De-streaming the input code stream to obtain a connection relationship code stream and a geometric information code stream; wherein the connection relationship code stream and the geometric information code stream are in units of triangles; Performing connection relationship decoding on the connection relationship code stream to obtain a connection relationship; Decoding the geometric information code stream to obtain geometric information; The three-dimensional mesh is reconstructed using the connection relationship and the geometric information.

10. The three-dimensional grid decoding method according to claim 8, characterized in that: The process of decoding the connection relationship of the input code stream includes: Performing entropy decoding on the input code stream to obtain a mode of a current triangle and additional information indicating a positional relationship of the triangle; Based on the mode and its corresponding stitching method, the current triangle is connected to the reconstructed three-dimensional mesh; wherein, if the traversed triangle meets the preset condition, the stitching method of the triangle is implemented based on the corresponding additional information; The position of the next triangle is determined according to the pattern, and relevant information required for decoding the triangle is obtained, and the pattern of the triangle is entropy decoded using the obtained relevant information.

11. The three-dimensional grid decoding method according to claim 10, characterized in that: The relevant information at least includes: One or more of vertex geometry information, encoded triangle mode, and the number of adjacent encoded triangles of a triangle vertex.

12. The three-dimensional grid decoding method according to claim 10, characterized in that: The process of entropy decoding the input bitstream to obtain additional information indicating the position relationship of the triangles includes: The input code stream is entropy decoded, and when the current mode of the triangle is a predetermined mode, the additional information obtained is the representation information of the position of the triangle in the three-dimensional grid.

13. The three-dimensional grid decoding method according to claim 10, characterized in that: The process of entropy decoding the input bitstream to obtain additional information indicating the position relationship of the triangles includes: The input code stream is entropy decoded, and when there are traversed adjacent triangles on both sides of the traversed triangle and the pattern has a ring property, the additional information obtained is the diagonal information of the triangle.

14. The three-dimensional grid decoding method according to claim 10 or 12, characterized in that: The process of entropy decoding the input bitstream to obtain additional information indicating the position relationship of the triangles includes: The input code stream is entropy decoded. If there are traversed adjacent triangles on both sides of the traversed triangle, if the pattern does not have a ring property and the corresponding rotation vertices are connected to two or more triangles of a predetermined pattern, the additional information obtained is the fan number of the triangle.

15. A three-dimensional grid decoding device, characterized in that: include: A decoding module, used for decoding the connection relationship and the geometric information of the input code stream to obtain the connection relationship and the geometric information respectively; A reconstruction module, used for reconstructing a three-dimensional mesh based on the obtained connection relationship and geometric information; The connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding the additional information indicating the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

Images