Three-dimensional grid coding and decoding method and device

Abstract

The invention discloses a three-dimensional grid coding and decoding method and device. The coding method comprises the following steps: carrying out connection relation coding and geometric information coding on an input three-dimensional grid to obtain an output code stream; the connection relation encoding comprises encoding modes of triangles in the three-dimensional grid and additional information used for indicating the position relation of the triangles; connection relation coding is realized based on entropy coding of context information. The decoding method comprises the following steps: carrying out connection relation decoding and geometric information decoding on an input code stream to obtain a connection relation and geometric information; reconstructing a three-dimensional grid based on the obtained connection relation and geometric information; the connection relation decoding comprises the steps of decoding modes of triangles in the three-dimensional grid, and decoding additional information used for indicating the position relation of the triangles at the same time so as to realize instant stitching of the current triangles; connection relationship decoding is implemented based on entropy decoding of context information. According to the invention, the entropy coding efficiency can be improved, and the instant stitching of the current triangle is realized in the decoding process.

Description

Technical Field

[0001] The invention belongs to the field of coding and decoding, and in particular relates to a three-dimensional grid coding and decoding method and device. Background Art

[0002] In recent years, 3D models have become a new generation of digital media after audio, images, and videos. 3D mesh is a commonly used 3D model representation method. Compared with traditional multimedia such as images and videos, it has stronger interactivity and realism, making it more and more widely used in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and military.

[0003] With the advancement of 3D mesh modeling and scanning technology, people have higher requirements for the accuracy and details of 3D graphics, which has also led to an exponential increase in the amount of data of the original 3D mesh. This has posed a major challenge to the transmission, storage and computer processing of the 3D mesh. Therefore, it is particularly important to achieve efficient compression of the 3D mesh while ensuring quality.

[0004] A 3D mesh is composed of multiple elements at different levels, including vertices, edges, and faces. Vertices are the basic elements that make up a 3D mesh and are described by coordinates in 3D space. Edges are the parts that connect two vertices in a 3D mesh. Faces are polygons connected by closed edges. Nowadays, most faces in 3D meshes are triangles. A 3D mesh mainly contains the following three types of information: 1. Geometric information. Geometric information mainly includes the position coordinates of all vertices in the 3D mesh in 3D space; 2. Connectivity relationship, also known as topological information. Connectivity relationship is used to describe the connection information between vertices and facets in a 3D mesh; 3. Other optional attribute information. Attribute information includes other information attached to the 3D mesh, including color information, normal vectors, texture coordinates, etc.

[0005] Connectivity is used to describe the connection of vertices in a 3D mesh, which affects the overall shape and local details of the model and is an important part of the 3D mesh. Therefore, it is undoubtedly of great significance to propose a new compression algorithm to achieve efficient compression of the connectivity relationship of the 3D mesh, and then accurately and efficiently achieve the overall encoding of the 3D mesh based on this. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a three-dimensional grid encoding and decoding method and device. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a three-dimensional grid coding method, the method comprising:

[0008] Encode the connection relationship and geometric information of the input three-dimensional grid to obtain an output code stream;

[0009] The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of triangles; the connection relationship coding is implemented based on entropy coding of context information.

[0010] In a second aspect, an embodiment of the present invention provides a three-dimensional grid encoding device, the device comprising:

[0011] The encoding module is used to encode the connection relationship and geometric information of the input three-dimensional grid to obtain an output code stream;

[0012] The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of triangles; the connection relationship coding is implemented based on entropy coding of context information.

[0013] In a third aspect, an embodiment of the present invention provides a three-dimensional grid decoding method, the method comprising:

[0014] Decode the connection relationship and the geometric information of the input code stream to obtain the connection relationship and the geometric information respectively;

[0015] Reconstructing a three-dimensional mesh based on the obtained connection relationship and geometric information;

[0016] The connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding the additional information indicating the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

[0017] In a fourth aspect, an embodiment of the present invention provides a three-dimensional grid decoding device, the device comprising:

[0018] A decoding module, used for decoding the connection relationship and the geometric information of the input code stream to obtain the connection relationship and the geometric information respectively;

[0019] A reconstruction module, used for reconstructing a three-dimensional mesh based on the obtained connection relationship and geometric information;

[0020] The connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding the additional information indicating the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

[0021] Beneficial effects of the present invention:

[0022] In a three-dimensional mesh encoding and decoding scheme proposed in an embodiment of the present invention, at the encoding end, the connection relationship encoding includes encoding the pattern of triangles in the three-dimensional mesh and encoding additional information for indicating the positional relationship of the triangles, and the connection relationship encoding is implemented based on the entropy encoding of the context information. Correspondingly, at the decoding end, the input code stream is decoded for connection relationship and geometric information to obtain connection relationship and geometric information respectively; the three-dimensional mesh is reconstructed based on the obtained connection relationship and geometric information; wherein, the connection relationship decoding includes decoding the pattern of triangles in the three-dimensional mesh, and at the same time decoding the additional information for indicating the positional relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is implemented based on the entropy decoding of the context information. In the embodiment of the present invention, the encoding and transmission of the additional information for indicating the positional relationship of the triangles by the encoding end enables the decoding end to use the triangle positional relationship to ensure the immediacy of stitching when stitching.

[0023] Furthermore, in an embodiment of the present invention, relevant information of a triangle is used as context information, which includes at least one or more of vertex geometry information, encoded triangle patterns, and the number of adjacent encoded triangles at triangle vertices. Since the number of adjacent encoded triangles at triangle vertices and the pattern are strongly correlated, entropy encoding of the pattern is performed using relevant information such as the number of adjacent encoded triangles at triangle vertices. This can more accurately estimate the probability of occurrence of the pattern, thereby improving the accuracy of probability estimation and improving encoding efficiency. In addition, context information consistent with that at the encoding end can be obtained at the decoding end to ensure accurate decoding and instant mesh reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the triangular fan structure in the existing TFAN algorithm;

[0025] Figure 2 It is a schematic diagram of triangle fan classification in the existing TFAN algorithm;

[0026] Figure 3 A schematic diagram of a flow chart of a three-dimensional grid encoding method provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of the principle of applying the three-dimensional grid coding method of an embodiment of the present invention to a separate coding frame;

[0028] Figure 5 A schematic diagram of the principle of applying the three-dimensional grid coding method of an embodiment of the present invention to a hybrid coding framework;

[0029] Figure 6 A schematic diagram of a framework of a connection relationship encoding method proposed in an embodiment of the present invention;

[0030] Figure 7The corresponding relationship between angles, vertices and triangles in the Corner-Table data structure;

[0031] Figure 8 It is a schematic diagram of the CLERS model;

[0032] Fig. 9 Schematic diagram of an entropy coding framework according to an embodiment of the present invention;

[0033] Fig.10 A schematic diagram of context modeling according to an embodiment of the present invention;

[0034] Fig.11 A schematic diagram of a process for recording the number of vertex rotations according to an embodiment of the present invention;

[0035] Fig.12 This is a schematic diagram of the connection in which the target vertex of the S-mode triangle cannot be rotated according to an embodiment of the present invention;

[0036] Fig.13 A schematic diagram for understanding the determination that the pattern of the traversed triangle has a ring property when there are traversed adjacent triangles on both sides of the traversed triangle in an embodiment of the present invention;

[0037] Fig.14 This is a schematic diagram of the zip rotation vertex and two S-shaped triangles adjacent to each other in an embodiment of the present invention;

[0038] Fig.15 Schematic diagram of various parallelogram prediction methods in an embodiment of the present invention;

[0039] Fig.16 A schematic diagram of a flow chart of a three-dimensional grid decoding method provided by an embodiment of the present invention;

[0040] Fig.17 It is a schematic diagram of the principle of applying the three-dimensional grid decoding method of an embodiment of the present invention to a separate decoding framework;

[0041] Fig.18 It is a schematic diagram of the principle of applying the three-dimensional grid decoding method of an embodiment of the present invention to a hybrid decoding framework;

[0042] Fig.19 A schematic diagram of a framework of a connection relationship decoding method proposed in an embodiment of the present invention;

[0043] Fig. 20 A schematic diagram of decoding and reconstructing a connection relationship according to an embodiment of the present invention;

[0044] Fig.21 Schematic diagram of the suturing process of the L mode in an embodiment of the present invention;

[0045] Fig. 22This is a schematic diagram of an L-mode rotation vertex adjacent to two S-mode triangles according to an embodiment of the present invention;

[0046] Fig.23 Schematic diagram of the suturing process of the R mode in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] At present, TFAN is an algorithm with better performance in connectivity relation coding algorithms. In the TFAN connectivity relation coding algorithm, a triangular fan structure is first defined. The triangular fan is composed of a series of triangles with the same vertex and the same direction, such as Figure 1 As shown, the triangular fan is defined as follows:

[0049] 1) Any two consecutive triangles in a triangle fan must be adjacent and share a common side.

[0050] 2) All triangles in the triangle fan have the same direction.

[0051] 3) All triangles in the triangle fan share a vertex, which is called the center point of the triangle fan. The shared vertex is shown as v0.

[0052] The TFAN connection relationship encoding algorithm is based on traversing the vertices in the three-dimensional mesh. First, a first-in-first-out queue is defined, and all vertices are marked as untraversed. Then, starting from a random initial vertex, a triangular fan structure centered on the initial point is determined, and the other vertices in the triangular fan are placed in the queue, marked as traversed, and the triangular fan is encoded. After that, vertices are continuously taken out of the queue, and the above process is repeated until the queue is empty and all vertices in the three-dimensional mesh are marked, and the encoding ends.

[0053] The coding of triangle fans is mainly based on the degree of the center point of the triangle fan (that is, the number of triangles in the triangle fan adjacent to the center point), the direction of the triangle, and the vertex traversal to classify the triangle fan. Figure 2 As shown in the figure, different values ​​of C represent different categories of triangle fans. In the figure, gray triangles represent traversed triangles, white triangles represent untraversed triangles, and red and black vertices represent traversed and untraversed vertices, respectively. When encoding, the category, vertex degree, and vertex traversal information of the current triangle fan are encoded, among which vertex traversal information is an optional encoding option.

[0054] Since TFAN has many types of triangular fans and the encoding of connection relationships is inefficient, in order to achieve efficient encoding of connection relationships and real-time reconstruction at the decoding end, an embodiment of the present invention proposes a new three-dimensional mesh connection relationship encoding and decoding scheme, and on this basis, combined with the encoding and decoding scheme of geometric information, proposes a complete encoding and decoding scheme for three-dimensional meshes. Specifically, an embodiment of the present invention proposes a three-dimensional mesh encoding method and device, and a corresponding three-dimensional mesh decoding method and device.

[0055] It should be noted that the corresponding execution subjects of the three-dimensional mesh encoding method / device or three-dimensional mesh decoding method / device provided in the embodiments of the present invention can be run in corresponding electronic devices, wherein the electronic device can be a server or a terminal device, but is not limited thereto.

[0056] In the first aspect, the embodiment of the present invention proposes a three-dimensional grid coding method, which can be applied to the coding end, see Figure 3 As shown, the encoding method may include:

[0057] Scode1, encodes the connection relationship and geometric information of the input 3D mesh to obtain the output code stream;

[0058] The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of the triangles; the connection relationship coding is implemented based on entropy coding of context information.

[0059] In an embodiment of the present invention, connection relationship encoding and geometric information encoding are performed in units of triangles, that is, they are implemented by traversing triangles. In an embodiment of the present invention, connection relationship encoding and geometric information encoding can have the same triangle traversal order, but can also have different triangle traversal orders. It can be understood that the latter method requires additional transmission of some relevant information compared to the former method, which is not described in detail here.

[0060] The main idea of ​​the three-dimensional mesh connection relationship encoding method proposed in the embodiment of the present invention is to first select a triangle as an initial triangle on the three-dimensional mesh to be encoded, then traverse the connection relationship of the triangles by determining the pattern of the triangles one by one according to a certain traversal rule starting from the initial triangle, and finally entropy encoding the determined triangle pattern, and at the same time determine and encode additional information for indicating the positional relationship of the triangles. Moreover, the embodiment of the present invention uses context-based adaptive arithmetic coding to implement the pattern, wherein the context information is improved, in order to improve the entropy coding efficiency.

[0061] It should be noted that the three-dimensional grid coding method proposed in the embodiment of the present invention can be applied to a three-dimensional grid coding framework in which connection relations and geometric information are separately encoded, that is, it is applicable to a separate coding framework; it can also be applied to a three-dimensional grid coding framework in which connection relations and geometric information are mixedly encoded, that is, it is also applicable to a mixed coding framework.

[0062] Specifically, for a separate coding frame, the connection relationship coding and geometric information coding of the input three-dimensional grid are performed to obtain an output code stream, including:

[0063] (1) traversing the triangles in the three-dimensional grid to perform connection relationship encoding and obtain the connection relationship encoding result;

[0064] Specifically, the connection relationship encoding may be performed by traversing the triangles in the three-dimensional grid, and a connection relationship code stream is obtained from the connection relationship encoding results of all triangles;

[0065] (2) traversing the triangles in the three-dimensional mesh to perform geometric information encoding and obtain a geometric information encoding result; wherein the connection relationship encoding and the geometric information encoding are performed in units of triangles;

[0066] This step can be performed by traversing the triangles in the three-dimensional mesh to encode the geometric information according to the connection relationship encoding order, and obtaining the geometric information code stream from the geometric information encoding results of all triangles;

[0067] (3) Obtaining an output code stream based on the connection relationship encoding result and the geometric information encoding result;

[0068] Specifically, the connection relationship code stream and the geometric information code stream may be mixed to obtain the output code stream.

[0069] The main processing flow of the three-dimensional grid coding framework for separate coding of connection relationship and geometric information in the embodiment of the present invention can be found in Figure 4Understand. Under this separate coding framework, the input three-dimensional mesh will be separately connected and coded for geometric information. Both codes are implemented by traversing triangles. For example, the connection relationship coding and geometric information coding have the same triangle traversal order. This method is reflected in the need to use the connection relationship coding order for geometric information coding. The connection relationship coding order reflects the triangle traversal order in the connection relationship coding process. In other words, the connection relationship code stream is obtained after traversing all triangles to complete the connection relationship coding, and the geometric information code stream is also obtained after traversing all triangles in the same order to complete the geometric information coding. The output code stream is obtained by mixing these two code streams. Of course, the above only illustrates the case where the connection relationship coding and the geometric information coding have the same triangle traversal order. For the case where the connection relationship coding and the geometric information coding have different triangle traversal orders, please refer to the above text for understanding. It will not be explained in detail in the form of diagrams. The connection relationship coding in the above steps is implemented using the ideas in the previous text. For a clear layout, it will be explained in detail in the following text.

[0070] Specifically, for the hybrid coding framework, the connection relationship coding and geometric information coding of the input three-dimensional mesh are performed to obtain an output code stream, including:

[0071] (1) traversing each triangle in the three-dimensional mesh, performing connection relationship encoding on the traversed triangle to obtain a connection relationship encoding result of the triangle, and performing geometric information encoding on the triangle to obtain a geometric information encoding result of the triangle, wherein the connection relationship encoding result and the geometric information encoding result of the triangle constitute the encoding result of the triangle;

[0072] (2) The output code stream is obtained from the encoding results of all triangles.

[0073] The main processing flow of the three-dimensional grid coding framework applied to the hybrid coding of connection relationship and geometric information in the embodiment of the present invention can be found in Figure 5 Understand. In this hybrid coding framework, the input three-dimensional mesh is traversed, and each traversed triangle will be coded for connection relationship and geometric information, that is, traversing a triangle will encode the triangle's pattern, possible additional information, and the geometric coordinates of the vertices of the triangle that have not yet been encoded. Therefore, for the hybrid coding framework, the triangle traversal order for geometric information encoding and connection relationship encoding is consistent, and ultimately there will be only one code stream, which is a mixture of the connection relationship encoding results and geometric information encoding results of the triangles traversed in sequence, as the output code stream. Similarly, the connection relationship encoding in the above steps is also implemented using the ideas in the previous text.

[0074] The following is a description of the main process of the connection relationship encoding method proposed in the embodiment of the present invention. Figure 6Understand, it is understandable that for a single coding framework, Figure 6 The output is the connection relationship code stream. For the hybrid coding framework, Figure 6 The output is the encoding result of the connection relationship of the current triangle.

[0075] Specifically, in Scode1, the process of encoding the connection relationship of the input three-dimensional grid mainly includes A1 to A4:

[0076] A1, initialize the three-dimensional grid and establish related data structures;

[0077] This step corresponds to Figure 6 The purpose of initializing the three-dimensional mesh and establishing the relevant data structure is to extract the relevant information of the triangles in the three-dimensional mesh, such as the relationship between vertices, angles or edges, for subsequent traversal; the relevant data structure may include: Corner-Table data structure or half-edge data structure, etc., which are not limited here. For ease of understanding and description, the embodiment of the present invention is explained by taking the Corner-Table data structure as an example.

[0078] The Corner-Table data structure contains four arrays: V, O, M, and U. The lengths of arrays V, O, and U are three times the number of mesh patches, and the length of array M is equal to the number of patches. Mesh patches refer to triangles. Array V stores the vertices corresponding to each corner in the three-dimensional mesh, and array O stores the opposite angle of each corner. Arrays M and U both store binary symbols, and the stored elements indicate whether the vertices and triangles in the three-dimensional mesh have been traversed. During the initialization process, each corner will be assigned a serial number in the counterclockwise order of the interior angles of the triangle, so that the other two corners of the same triangle can be quickly found based on the serial number of a certain angle. Let the current angle in the triangle be c, the first angle encountered by angle c in clockwise order be cp, the first angle encountered by angle c in counterclockwise order be cn, the vertex corresponding to c is cv, the opposite angle of c is co, and the triangle to which c belongs is ct. Figure 7 As shown, for the definition and setting rules of corners and vertices, please refer to Corner-Table data structure understanding.

[0079] A2, for each traversed triangle, determine the traversal result of the triangle based on the data structure;

[0080] This step corresponds to Figure 6In the "Select an initial triangle, traverse the connection relationship" and "Determine the triangle mode and next traversal direction based on the triangle traversal situation" part. Among them, the traversal result of the triangle includes the triangle mode. In an optional implementation, the triangle mode can be a CLERS mode, but this is not limited here.

[0081] In an optional embodiment, step A2 includes:

[0082] For each traversed triangle, determine the corresponding vertices of the triangle based on the data structure, and determine the traversal result of the triangle, including the mode of the triangle, based on the traversal status of the corresponding vertices of the triangle and the traversal status of the triangles on both sides. Mark the triangle as a traversed triangle, and mark the three vertices of the triangle as traversed vertices.

[0083] Taking the case where the relevant data structure is a Corner-Table data structure and the mode is a CLERS mode as an example, the above step A2 may include:

[0084] For each traversed triangle, the vertex corresponding to the c angle of the triangle is determined according to the relationship between the angle and the opposite angle in the Corner-Table data structure, and the angle and the vertex. According to the traversal of the vertex corresponding to the c angle of the triangle obtained and the traversal of the triangles on the left and right sides, the traversal result of the triangle is determined, including the CLERS mode of the triangle, and the triangle is marked as a traversed triangle, and the three vertices of the triangle are marked as traversed vertices. It can be understood that in this specific example, the current angle is the c angle.

[0085] Specifically, when traversing triangles on a three-dimensional mesh, the embodiment of the present invention divides the three-dimensional mesh into two parts: traversed and untraversed. An initial triangle may be selected by random selection or the like, and traversal may be started from the initial triangle, the traversed triangles are continuously inserted into the traversed part, and other triangles are traversed according to the relative position of the newly added triangle. At the same time, according to the position of the vertex corresponding to the current angle of the triangle and the traversal of the triangles on the left and right sides, different modes are output for each triangle, and the traversal direction of the next triangle is determined.

[0086] We still take the Corner-Table data structure and the CLERS mode as an example. Figure 8 Understand, where X is the current triangle, and v is the vertex corresponding to the current angle of the triangle, i.e. angle c. When traversing, different modes are assigned to the triangle according to the five different situations of the CLERS mode, and jump to the next triangle in different directions.

[0087] Specifically, according to the traversal of the vertex corresponding to the obtained angle c of the triangle and the traversal of the triangles on the left and right sides, the traversal result of the triangle is determined, including:

[0088] 1) For the current triangle traversed, if the vertex corresponding to the c angle of the triangle has not been traversed, determine that the CLERS mode to which the triangle belongs is C mode, and the traversal direction of the next triangle is the right side of the triangle;

[0089] See also Figure 8 The pattern diagram marked with C, C pattern means that v has not been traversed and should be traversed to the right of the current triangle.

[0090] 2) For the current triangle traversed, if the vertex corresponding to the c angle of the triangle has been traversed, and the triangle on the left side of the triangle has been traversed, but the triangle on the right side has not been traversed, it is determined that the CLERS mode to which the triangle belongs is the L mode, and the traversal direction of the next triangle is the right side of the triangle;

[0091] See also Figure 8 The pattern diagram marked with L indicates that v has been traversed and should be traversed to the right of the current triangle.

[0092] 3) For the current triangle traversed, if the vertex corresponding to the c angle of the triangle has been traversed, and the triangle on the right side of the triangle has been traversed, but the triangle on the left side has not been traversed, it is determined that the CLERS mode to which the triangle belongs is the R mode, and the traversal direction of the next triangle is the left side of the triangle;

[0093] See also Figure 8 The pattern diagram marked with R indicates that v has been traversed and should be traversed to the left of the current triangle.

[0094] 4) For the current triangle traversed, if the vertex corresponding to the c angle of the triangle has been traversed, and the left and right triangles of the triangle have not been traversed, it is determined that the CLERS mode to which the triangle belongs is the S mode, and the traversal direction of the next triangle is to first traverse the right branch of the triangle, and then traverse to the left branch of the triangle after traversing to the end;

[0095] See also Figure 8 The pattern diagram marked with S indicates that v has been traversed and should first traverse the right branch, then traverse the left branch after traversing to the end.

[0096] 5) For the current triangle traversed, if the vertex corresponding to the c angle of the triangle has been traversed, and the left and right triangles of the triangle have been traversed, it is determined that the CLERS mode to which the triangle belongs is E mode, there is no traversal direction for the next triangle, and the traversal ends.

[0097] See also Figure 8In the pattern diagram marked with E, the E pattern means that v has been traversed and the traversal is finished.

[0098] Of course, for other modes besides the CLERS mode, the method of determining the traversal result of the triangle can be implemented in a similar way to the above process based on the traversal conditions of the corresponding vertices of the triangle and the traversal conditions of the triangles on both sides, and no examples are given here.

[0099] A3, using the relevant information of the traversed triangle as context information, entropy encoding the pattern of the triangle;

[0100] The relevant information includes at least one or more of vertex geometry information, encoded triangle mode, and the number of adjacent encoded triangles of a triangle vertex;

[0101] Among them, A3 corresponds to Figure 6 The section "Entropy encoding of triangle patterns based on relevant context information" in .

[0102] In an optional implementation manner, A3 may include steps A31 and A32:

[0103] A31, for each pattern of the to-be-coded triangle, using a context-based adaptive binary arithmetic coding method, using a preset binarization method to binarize the pattern of the to-be-coded triangle, to obtain a corresponding binary character sequence;

[0104] There is a strong correlation between the pattern of a triangle and the number of adjacent traversed triangles. The pattern can be entropy encoded using context-based adaptive binary arithmetic coding (CABAC for short). CABAC is mainly divided into three stages: binarization, context modeling, and binary arithmetic coding. The embodiments of the present invention mainly improve the two stages of binarization and context modeling.

[0105] Among them, for binarization processing, an embodiment of the present invention binarizes the triangular pattern to obtain a corresponding binary character sequence, and the preset binarization method used includes but is not limited to truncated unary code, Golomb code and fixed-length code.

[0106] In the binarization process, the symbol with a greater probability of occurrence (i.e., the symbol corresponding to the pattern) has a shorter codeword after binarization. Table 1 shows a possible binarization code table using truncated unary code as an example. The codewords in Table 1 represent the corresponding binary character sequences.

[0107] Table 1 CLERS pattern and binary code table of triangle

[0108] model Codeword C 0 R 10 S 110 L 1110 E 1111

[0109] A32, obtains relevant information of the triangle to be encoded, uses the obtained relevant information to calculate the context index corresponding to each binary character of the triangle to be encoded, and determines the corresponding context model according to the obtained context indexes, and uses the obtained context model to perform binary arithmetic encoding on the binary character sequence of the triangle to be encoded, and updates the context model.

[0110] Among them, for context index calculation, an embodiment of the present invention uses relevant information, such as vertex geometry information, encoded triangle pattern, and one or more of the numbers of adjacent encoded triangles of triangle vertices, as context information to calculate the context index.

[0111] The entropy coding framework of embodiments A31 to A32 of the present invention can be found in Fig. 9 As shown in the figure, the entropy coding framework first binarizes the triangle pattern, then uses the relevant information to calculate the context index, thereby determining the context model, completing the context modeling, and finally using the context model to implement entropy coding of the triangle pattern, and the context model also needs to be updated.

[0112] Specifically, two different related information are given below to respectively illustrate the calculation process of the context index.

[0113] The first method is to obtain relevant information of the triangle to be encoded when the relevant information is the number of adjacent encoded triangles of the triangle vertex, and calculate the context index corresponding to each binary character of the triangle to be encoded using the obtained relevant information, including:

[0114] i) for the triangle to be encoded, obtaining the numbers P1 and P2 of adjacent traversed triangles corresponding to the vertices corresponding to the predetermined angle and the vertices corresponding to the preset angle respectively as relevant information of the triangle to be encoded;

[0115] ii) Substituting P1 and P2 into a preset formula for calculating the index of the adjacent encoded situation of two vertices of a triangle, to obtain the index of the adjacent encoded situation of two vertices of the triangle to be encoded;

[0116] iii) For each binary character in the binary character sequence corresponding to the to-be-encoded triangle, the context index of the binary character is obtained according to the position of the binary character and the adjacent encoded index of two vertices of the triangle using a preset context index calculation formula.

[0117] The following uses the Corner-Table data structure and the CLERS model as an example to illustrate that the above three steps correspond to ①②③ in the following text of the first method:

[0118] ① For the triangle to be encoded, obtain the numbers P1 and P2 of adjacent traversed triangles corresponding to the vertices corresponding to the angle cn and the vertices corresponding to the angle cp respectively as the relevant information of the triangle to be encoded;

[0119] In this case, the current angle is the c angle, the preset angle is the cp angle, and the predetermined angle is the cn angle; specifically:

[0120] After completing the binarization of the triangle pattern, the context index can be calculated based on the number of adjacent coded triangles of the current triangle vertex. Fig.10 As shown in the figure, the yellow triangle is the current triangle to be encoded. v1 and v2 are the vertices corresponding to the cn and cp angles of the triangle. Let P1 and P2 be the number of adjacent traversed triangles of v1 and v2 when encoding the current triangle, and take P1 and P2 as the relevant information of the current triangle.

[0121] ② Substitute P1 and P2 into the preset triangle two vertices adjacent encoded index calculation formula to obtain the triangle two vertices adjacent encoded index of the triangle to be encoded;

[0122] The embodiment of the present invention uses P1 and P2 to calculate the context index and the number of bits of binary characters, namely binary symbols, in a binary character sequence, and performs binary arithmetic coding on the binary symbol sequence.

[0123] The following takes truncated unary code as a binarization processing method as an example to give a context index calculation process of a triangle pattern. Of course, the other binarization processing methods are similar and will not be described in detail here.

[0124] Let traversedValence represent the index of the adjacent encoded situation of two vertices of a triangle. The preset calculation formula of the adjacent encoded situation index of two vertices of a triangle includes:

[0125] traversedValence=P1+10×P2 (1)

[0126] Among them, traversedValence represents the index of the adjacent encoded situation of two vertices of a triangle.

[0127] Of course, the preset index calculation formula for the adjacent encoded situation of two vertices of a triangle in the embodiment of the present invention is not limited to the above form. For example, 10 can also be set to other values ​​according to needs, or other weights can be used to perform weighted calculation on P1 and P2, and so on.

[0128] ③ For each binary character in the binary character sequence corresponding to the triangle to be encoded, the context index of the binary character is obtained according to the position of the binary character and the adjacent encoded index of the two vertices of the triangle, using the preset context index calculation formula.

[0129] When the binarization method used is truncated unary code, the length of the longest codeword corresponding to the binarized mode is 4. It is necessary to establish a suitable context model for each binary character at different positions, and the context index can be calculated based on the current adjacent traversed triangle situation and the binary character position. Let the binary symbol position be bitIndex, and the value range is bitIndex∈{0,1,2,3}, then the preset context index calculation formula includes:

[0130] ctxIdx=bitIndex+4×traversedValence (2)

[0131] Among them, traversedValence represents the index of the adjacent encoded situation of two vertices of the triangle; bitIndex represents the position of the binary character; ctxIdx represents the context index of the binary character.

[0132] It is understandable that for a binary character sequence corresponding to a triangular pattern, a corresponding ctxIdx will be calculated for each binary character. It is understandable that after the calculation of the context index is completed, the context model can be obtained accordingly, the binary character can be binary arithmetic encoded using the context model, and the context model can be updated. For the process of obtaining the context model for encoding after obtaining the context index, please refer to the relevant content of CABAC, which will not be explained in detail here.

[0133] It should be noted that the above formula (2) is only an example of a preset context index calculation formula and does not constitute a limitation thereto.

[0134] The embodiment of the present invention calculates the context index using the number of encoded triangles adjacent to the current triangle vertex. Compared with the prior art, the context of the embodiment of the present invention uses more information related to the triangle pattern and can encode the connection relationship more efficiently.

[0135] The second method is to obtain relevant information of the to-be-encoded triangle when the relevant information is in the encoded triangle mode, and calculate the context index corresponding to each binary character of the to-be-encoded triangle using the obtained relevant information, including:

[0136] ① For each binary character in the binary character sequence corresponding to the to-be-encoded triangle, determine the context index of the binary character using only the pattern of the previous triangle of the to-be-encoded triangle;

[0137] Still using the previous example, let the codeword of the triangle pattern after the current input binarization be value, that is, the binary character sequence represented by 0 / 1, and the context index ctxIdx be initialized to 0. When the binarization method used is truncated unary code, the length of the longest codeword corresponding to the binarization pattern is 4. Different context models can be established for binary characters at different positions. The context index ctxIdx of the current binary character can only consider the use of the previous triangle pattern to calculate the context index of the current triangle to be encoded. See the following formula:

[0138] ctxIdx=value (3)

[0139] or,

[0140] ② For each binary character in the binary character sequence corresponding to the to-be-encoded triangle, the context index of the binary character is determined by using the pattern of the previous triangle of the to-be-encoded triangle and the context index obtained by the previous triangle.

[0141] In this case, the context index of the current triangle is calculated by using the mode of the previous triangle and the context index obtained by the previous triangle. The calculation formula can be:

[0142] ctxIdx=value+((ctxIdx & 1) << 3) + ((ctxIdx & 4) << 2) (4)

[0143] Among them, & represents logical AND; (ctxIdx&1)<<3) represents the logical AND of ctxIdx to 1, and then shifted left by three bits; (ctxIdx&4)<<2 represents the logical AND of ctxIdx to 4, and then shifted left by two bits.

[0144] It is also understandable that after the calculation of the context index is completed, the context model can be obtained accordingly, the binary character can be binary arithmetic coded using the context model, and the context model can be updated. For the process of obtaining the context model for encoding after obtaining the context index, please refer to the relevant content of CABAC, which will not be described in detail here.

[0145] A4, if the traversed triangle meets the preset conditions, determine the corresponding additional information, include it in the traversal result of the triangle, and perform entropy coding on the additional information;

[0146] After determining the pattern of the traversed triangles, if the traversed triangles meet the preset conditions, the embodiment of the present invention needs to perform some special processing to obtain additional information indicating the positional relationship of the triangles and perform entropy coding for transmission in the output bit stream, so that the decoding end can use the additional information to perform corresponding stitching and reconstruction processing. The entropy coding of the additional information can be implemented by any entropy coding method, which is not limited here.

[0147] The following is a detailed description of the different additional information:

[0148] 1) In an optional implementation manner, if the traversed triangle meets a preset condition, the corresponding additional information is determined, including:

[0149] If the traversed triangle pattern is a predetermined pattern, the representation information of the position of the triangle in the three-dimensional mesh is determined as additional information. To ensure that the codec can use the same context information, the embodiment of the present invention needs to transmit the representation information of the position of the triangle in the three-dimensional mesh as additional information for the triangle of the predetermined pattern.

[0150] In an optional implementation manner, if the pattern of the traversed triangle is a predetermined pattern, determining the representation information of the position of the triangle in the three-dimensional grid as additional information includes:

[0151] B1, determining the representation information of the position of the current corner vertex of the triangle in the three-dimensional grid;

[0152] For a triangle to be encoded in a predetermined mode, the encoder will encode and transmit the position of the current corner vertex in the three-dimensional grid, so that the decoding end can use the context information to decode the triangle pattern as the predetermined mode, and immediately reconstruct the vertices of the triangle to their actual positions in the grid, so that subsequent decoding can use the same context information as the encoder.

[0153] Taking the CLERS mode as an example, the predetermined mode may be the S mode. In this case, B1 is the characterization information for determining the position of the c-angle vertex of the triangle in the three-dimensional grid. Other cases are not described in detail here.

[0154] In an optional implementation manner, the characterization information for determining the position of the current corner vertex of the triangle in the three-dimensional grid in B1 may include:

[0155] B11, using the vertex rotation method, starts to rotate from the preset corner vertex of the triangle, and continuously changes the rotation vertex until the vertex position corresponding to the current corner is found. The number of vertex rotations experienced during the period is recorded as the simplified representation information of the position of the current corner vertex in the three-dimensional grid.

[0156] For triangles with a predetermined pattern, the position of the current corner vertex in the three-dimensional grid will be encoded and transmitted, also called the position of the entry edge to the vertex, but the actual information often has a large amount of data, and direct encoding and transmission will increase the transmission pressure. Therefore, the embodiment of the present invention uses the number of vertex rotations to indirectly characterize the position of the current corner vertex of the triangle in the three-dimensional grid as a simplified representation of the information. In this way, only a numerical value with a small amount of data needs to be encoded and transmitted, thereby reducing the transmission pressure and improving the transmission efficiency. However, the decoding end can still use the rotation information to effectively locate the position of the current corner vertex of the triangle in the three-dimensional grid, thereby ensuring the accuracy of decoding and reconstruction.

[0157] Taking the Corner-Table data structure and CLERS mode as an example, the current angle in B11 is the c angle, and the preset angle is the cp angle.

[0158] For details, see Fig.11 As shown in the schematic diagram of the process of recording the number of vertex rotations, the rotation starts from the cp angle vertex, and the rotation vertex is continuously changed until the vertex position corresponding to the c angle is found. In this process, the number of times the rotation vertex is changed is recorded. The specific steps of B11 may include:

[0159] Step a1, record the vertex corresponding to the c angle of the triangle as the target vertex v1, v1 = cv; let v t = cpv, b = c, number of rotations l = 0;

[0160] Among them, v1, v t , b and l are parameters set in the embodiment of the present invention; cv and c can be understood by referring to the Corner-Table data structure, and those skilled in the art can understand from the data structure that cpv represents the vertex corresponding to the cp angle.

[0161] Step a2, let b = bop. If the vertical angle of b has been traversed, continue around v t Rotate and repeat step a2; if the vertical angle of b is not traversed, let v t =bnv, b=bp, l=l+1, execute step a3;

[0162] Similarly, according to the Corner-Table data structure, bop represents the previous angle of the diagonal of b (the previous angle here refers to the first angle of bo in the clockwise direction), bnv represents the vertex corresponding to the next angle of b (the next angle here refers to the first angle of b in the counterclockwise direction), and bp represents the previous angle of b (the previous angle here refers to the first angle of b in the clockwise direction). l=l+1 means adding one to the current l as the updated l.

[0163] Step a3, determine v t Is it equal to v1? If so, determine that the current l is the number of vertex rotations and end the rotation search process; if not, return to step a2 and continue to rotate around vertex v t Perform a rotation search.

[0164] In another optional implementation manner, the characterization information for determining the position of the current corner vertex of the triangle in the three-dimensional grid in B1 may include:

[0165] Complete representation information is obtained based on the position of the current corner vertex of the triangle in the three-dimensional mesh. The complete representation information includes a triangle traversal index when the current corner vertex is traversed for the first time.

[0166] This method can be performed directly or when it is impossible to obtain simplified representation information through vertex rotation, which is reasonable.

[0167] Taking the Corner-Table data structure and CLERS mode as an example, when the encoding end encounters the S mode, you may encounter the following Fig.12 The special case shown, Fig.12 This is a schematic diagram of the S-mode triangle target vertex that cannot be rotated and connected in an embodiment of the present invention, in which the mode of the yellow triangle is S. The vertex corresponding to the c angle of the triangle in the S mode, that is, the target vertex, cannot be found by the above rotation method, so the relevant information of the position of the vertex in the three-dimensional grid can be transmitted as a complete representation information, so that the decoding end can use the complete representation information to complete the stitching of the corresponding vertices. For example, the triangle traversal index when traversing to the vertex corresponding to the c angle for the first time can be transmitted. The triangle traversal index refers to the number of the triangle when traversing to the vertex corresponding to the c angle for the first time according to the triangle traversal order of the encoded connection relationship.

[0168] 2) In an optional implementation, if the traversed triangle meets the preset conditions, the corresponding additional information is determined, including:

[0169] When there are encoded adjacent triangles on both sides of the traversed triangle, it is determined whether the pattern of the traversed triangle has a ring property. If so, the diagonal information of the triangle is recorded as a kind of additional information.

[0170] Specifically, when there are encoded adjacent triangles on both sides of the traversed triangle, if it is determined that the traversed triangle pattern has a ring property, it means that the triangle on one side of the predetermined pattern can be traversed to the triangle on the other side at one time. For a schematic diagram, please refer to Fig.13 understand.

[0171] Taking the Corner-Table data structure and the CLERS pattern as an example, when there are encoded adjacent triangles on both sides of the traversed triangle, it means that the CLERS pattern is one of E, R, and L. In this case, the pattern of the traversed triangle has a ring property and the pattern can be called a handle. Of course, the above situation is only an example and is not intended to limit the embodiments of the present invention.

[0172] Taking the above example as an example, the situation of traversing from one branch of the S-mode triangle to another branch is called a handle. For the E, R, and L modes, it is necessary to detect whether the mode is a handle at the encoding end. If it is a handle, the diagonal information of the triangle is recorded, that is, the handle information, which represents the diagonal information of the handle, which refers to recording c and co. If it is not a handle, it is necessary to determine whether the rotation vertex is connected to two or more S-mode triangles. If so, it is necessary to record the sector number; if not, skip the recording step and perform subsequent operations. See below for the recording of sector numbers.

[0173] For the handle, since the decoder has no way to find the diagonal angle and the vertex of angle c through reconstruction operations, the encoder needs to directly transmit the diagonal information to the decoder, and then the decoder updates the vertex of angle c based on this diagonal information.

[0174] Of course, when there are encoded adjacent triangles on both sides of the traversed triangle and it is determined that the pattern of the traversed triangle has the property of a ring, the other possible situations are not limited to those shown above, and accordingly, the recorded diagonal information of the triangle is also not subject to the above limitations.

[0175] 3) In an optional implementation manner, if the traversed triangle meets the preset conditions, the corresponding additional information is determined, including:

[0176] When there are encoded adjacent triangles on both sides of the traversed triangle, if the pattern of the triangle does not have a ring property, it is determined whether the rotation vertex used in the current pattern is connected to two or more triangles of the predetermined pattern during the reconstruction operation. If so, the angle to be stitched is determined according to the traversal order of the triangles on which the rotation vertex is connected, and the sector number is obtained as additional information. Specifically, the reconstruction operation can be a zip operation, but is not limited thereto.

[0177] Taking the Corner-Table data structure, CLERS mode, and the reconstruction operation as a zip operation as an example, when there are encoded adjacent triangles on both sides of the traversed triangle, it means that the CLERS mode is one of E, R, and L, and the predetermined mode is S mode. Specifically:

[0178] When performing the zip operation or stitching operation on the decoding end, if the surrounding vertex is adjacent to two or more S-pattern triangles, it may cause diagonal connection errors, such as Fig.14 As shown. Therefore, when the encoding end encounters a triangle of E, R, or L mode that needs to be zipped, when the mode of the triangle does not have a ring property, the embodiment of the present invention determines whether the zip rotation vertex of the current mode is connected to two or more triangles of S mode. If connected, the angle that needs to be zipped is located on the sector connected to the vertex according to the traversal order of the triangles, that is, a sector number is given for encoding and transmission, so that the decoding end can determine the sector on which the angle that needs to be stitched is located according to the sector number, and perform the zip operation, so as to improve the accuracy of three-dimensional mesh reconstruction at the decoding end.

[0179] Specifically, for recording the sector number, the encoding end determines whether it is necessary to determine whether the rotation vertex is connected to two or more pattern S triangles. The specific determination method includes:

[0180] ① If the current triangle mode is L, check whether the left branch is a handle. If so, record the handle information; otherwise, determine whether the rotation vertex is connected to two or more triangles of mode S.

[0181] ① If the current triangle mode is R, check whether the right branch is a handle. If so, record the handle information; otherwise, determine whether the rotation vertex is connected to two or more triangles of mode S.

[0182] ③ The current triangle mode is E. You need to check the branches on both sides to ensure that the order of the codec is consistent. The following is a feasible detection method: first check whether the right branch is a handle. If so, record the handle information; otherwise, determine whether the rotation vertex is connected to two or more triangles in mode S. Then check whether the left branch is a handle. If so, record the handle information and determine whether the rotation vertex is connected to two or more triangles in mode S; otherwise, determine whether the rotation vertex is connected to two or more triangles in mode S.

[0183] The above is the main content of connection relationship encoding. The following is a brief description of the process of geometric information encoding, taking a separate encoding framework as an example.

[0184] In an optional implementation, according to the connection relationship coding order, the triangles in the three-dimensional mesh are traversed to perform geometric information coding, and the geometric information coding results of all triangles are used to obtain a geometric information code stream, including:

[0185] (1) According to the connection relationship coding order, the geometric information of each triangle in the three-dimensional grid is predicted using the parallelogram prediction method to obtain the geometric coordinate residual;

[0186] Among them, the prediction methods include single parallelogram prediction, multi-parallelogram prediction and weighted parallelogram prediction.

[0187] See also Fig.15 Understand various parallelogram prediction methods, Fig.15 In FIG. 1 , (a), (b), and (c) represent single parallelogram prediction, multi-parallelogram prediction, and weighted parallelogram prediction, respectively. For the specific prediction process, please refer to the relevant technical understanding, which will not be described here.

[0188] (2) Perform entropy coding on the predicted geometric coordinate residual to obtain a geometric information code stream.

[0189] Among them, the entropy coding method can be selected as needed, and the specific process can be understood by referring to the relevant technology.

[0190] Those skilled in the art can understand that the hybrid coding framework is similar to the above-mentioned geometric coordinate prediction process and entropy coding, and will not be described in detail here. In addition, the geometric information coding process of the embodiment of the present invention is not limited to the above-mentioned example.

[0191] A three-dimensional mesh coding method proposed in an embodiment of the present invention is for connection relationship coding, including coding a pattern of triangles in a three-dimensional mesh and coding additional information for indicating the positional relationship of the triangles, wherein the connection relationship coding is implemented based on entropy coding of context information. The encoding end codes and transmits the additional information for indicating the positional relationship of the triangles, so that the decoding end can use the triangle positional relationship to ensure the accuracy and immediacy of stitching when stitching.

[0192] Furthermore, in an embodiment of the present invention, relevant information of a triangle is used as context information, which includes at least one or more of vertex geometry information, encoded triangle patterns, and the number of adjacent encoded triangles at triangle vertices. Since the number of adjacent encoded triangles at triangle vertices and the pattern are strongly correlated, entropy encoding of the triangle pattern is performed using triangle-related information such as the number of adjacent encoded triangles at triangle vertices. This can more accurately estimate the probability of occurrence of the pattern, thereby improving the accuracy of probability estimation and improving encoding efficiency. In addition, context information consistent with that at the encoding end can be obtained at the decoding end to ensure accurate decoding and mesh reconstruction.

[0193] Furthermore, for triangles of a predetermined pattern, the additional information encoded by the embodiment of the present invention includes information representing the position of the triangle in a three-dimensional grid, so that the decoding end can use the context information consistent with the encoding end to decode the triangle pattern as a predetermined pattern, and immediately reconstruct the vertices of the triangle to their actual positions in the grid, so that subsequent decoding can use the same context information as the encoding end. Among them, the method of recording the number of vertex rotations as the position of the current vertex in the three-dimensional grid can reduce the transmission pressure and improve the transmission efficiency while ensuring the accuracy of the three-dimensional reconstruction at the decoding end.

[0194] Furthermore, when there are encoded adjacent triangles on both sides of the traversed triangle, the additional information encoded and transmitted by the embodiment of the present invention includes the diagonal information of the triangle recorded when the detection mode has a ring property, which is beneficial to the accuracy of the stitching process at the decoding end. And, when there are encoded adjacent triangles on both sides of the traversed triangle, the embodiment of the present invention determines that the pattern of the triangle does not have a ring property, and when the rotation vertex used in the current mode is connected to two or more triangles of the predetermined mode during the reconstruction operation, the sector number corresponding to the sector connected to the rotation vertex, which is located at the angle that needs to be reconstructed, is determined according to the traversal order of the triangle, and encoded and transmitted, so that the decoding end can ensure accuracy when stitching.

[0195] In a second aspect, corresponding to the three-dimensional grid encoding method provided in the first aspect, an embodiment of the present invention further provides a three-dimensional grid encoding device, which is applied to an encoding end, and the device includes:

[0196] The encoding module is used to encode the connection relationship and geometric information of the input three-dimensional grid to obtain an output code stream;

[0197] The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of the triangles; the connection relationship coding is implemented based on entropy coding of context information.

[0198] Among them, for the specific processing process of the above modules, please refer to the three-dimensional grid coding method described in the first aspect, which will not be repeated here.

[0199] In a third aspect, corresponding to the three-dimensional grid encoding method provided in the first aspect, an embodiment of the present invention further provides a three-dimensional grid decoding method, which is applied to a decoding end, such as Fig.16 As shown, the method includes:

[0200] Sdecode1 decodes the connection information and geometric information of the input code stream to obtain the connection relationship and geometric information respectively;

[0201] Sdecode2, reconstructs the 3D mesh based on the obtained connection relationship and geometric information;

[0202] Among them, the connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and at the same time decoding the additional information used to indicate the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

[0203] Compared with the three-dimensional grid encoding method provided in the first aspect of the embodiment of the present invention, which can be applied to a single encoding framework and a hybrid encoding framework, the three-dimensional grid decoding method provided in the third aspect of the embodiment of the present invention can also be applied to a single decoding framework and a hybrid decoding framework.

[0204] Specifically, for a separate decoding framework, the input code stream is decoded for connection information and geometric information to obtain connection relationship and geometric information respectively; and a three-dimensional mesh is reconstructed based on the obtained connection relationship and geometric information, including:

[0205] (1) De-streaming the input code stream to obtain a connection information code stream and a geometric information code stream; wherein the connection relationship code stream and the geometric information code stream are based on triangles;

[0206] (2) decoding the connection relationship of the connection information code stream to obtain the connection relationship;

[0207] (3) Decoding the geometric information code stream to obtain geometric information;

[0208] Among them, the geometric information decoding can be performed according to the connection relationship decoding order.

[0209] (4) Reconstruct the 3D mesh using connection relationships and geometric information.

[0210] The main processing flow of the 3D mesh decoding framework for decoding connection relationship and geometric information separately according to the embodiment of the present invention can be found in Fig.17Understand. In this separate decoding framework, both connection relationship decoding and geometric information decoding are implemented by traversing triangles, but there is no restriction on whether the triangle traversal order is the same. Taking the same triangle traversal order as an example, this method is reflected in the geometric information decoding. The connection relationship decoding order needs to be utilized, and the connection relationship decoding order reflects the triangle traversal order in the connection relationship decoding process. In other words, the connection relationship output after the connection relationship decoding is obtained after traversing all triangles to complete the connection relationship decoding, and the geometric information is also obtained after traversing all triangles in the same order to complete the geometric information decoding. Of course, the same triangle traversal order is only used as a possible example. For the case where the triangle traversal order is different, the processing process of the embodiment of the present invention will not be described in detail here. The connection relationship decoding method in the above steps corresponds to the encoding idea in the previous text. For the sake of clear layout, it is specifically described in the following text.

[0211] Specifically, for the hybrid decoding framework, the input code stream is decoded for connection information and geometric information to obtain connection relationship and geometric information respectively; and a three-dimensional mesh is reconstructed based on the obtained connection relationship and geometric information, including:

[0212] (1) Traverse the triangles of the input bitstream in sequence according to the triangle traversal order, and decode the connection relationship and geometric information of each traversed triangle to obtain the connection relationship and geometric information of the triangle;

[0213] (2) Reconstruct the three-dimensional mesh of the triangle using the connection relationship and geometric information of the triangle.

[0214] The main processing flow of the 3D mesh decoding framework for decoding connection relationship and geometric information separately according to the embodiment of the present invention can be found in Fig.18 Understand. In this hybrid decoding framework, the input bitstream will perform triangle traversal, and each traversed triangle will be decoded for connection relationship and geometric information, and then the 3D mesh will be reconstructed. That is, when a triangle is traversed, the pattern of the triangle and the geometric coordinates of the vertices of the triangle that have not yet been decoded are decoded. Therefore, the triangle traversal order of geometric information decoding and connection relationship decoding is consistent, until all triangles are traversed and the final reconstructed 3D mesh is obtained.

[0215] Similarly, the connection relationship decoding method in the above steps also corresponds to the encoding idea in the previous text.

[0216] The following is a description of the main process of the connection relationship decoding method proposed in the embodiment of the present invention. Fig.19 Understand, it is understandable that for a single coding framework, Fig.19 The corresponding code stream of the input is the connection relationship code stream. For the hybrid coding framework, Fig.19The corresponding code stream of the input may be the entire input code stream.

[0217] The decoding end first performs entropy decoding on the pattern triangle based on certain context information, that is, relevant information about the triangle, such as one or more of vertex geometry information, encoded triangle patterns, and the number of encoded triangles adjacent to triangle vertices, but must be consistent with the encoding end. Then, the current triangle is connected to the reconstructed 3D mesh using different stitching methods based on its pattern, and then the position of the next triangle is determined, and the pattern of the next triangle is entropy decoded through the context information. This process is repeated until all triangles are decoded and reconstructed to obtain a reconstructed 3D mesh.

[0218] Specifically, in Sdecode1, the process of decoding the connection relationship of the input code stream mainly includes D1 to D3:

[0219] D1, entropy decoding the input bit stream to obtain the mode of the current triangle and additional information indicating the position relationship of the triangle;

[0220] The entropy decoding process at the decoding end corresponds to the entropy encoding process at the encoding end, which can be understood in conjunction with the foregoing. The mode can be the CLERS mode, which is not described in detail here. It can be understood that in addition to decoding the mode to which the current triangle belongs, additional information indicating the positional relationship of the triangle is also decoded.

[0221] The decoding of the additional information is described in detail below.

[0222] 1) In an optional implementation manner, the process of entropy decoding the input bit stream to obtain additional information indicating the position relationship of the triangles includes:

[0223] The input code stream is entropy decoded, and when the mode of the current triangle is the predetermined mode, the additional information obtained is the representation information of the position of the triangle in the three-dimensional grid.

[0224] Specifically, entropy decoding is performed on the input code stream to obtain the representation information of the position of the current corner vertex of the triangle in the three-dimensional grid.

[0225] Taking the Corner-Table data structure and CLERS mode as an example, the predetermined mode is the S mode, and the current angle is the c angle.

[0226] Optionally, the representation information of the position of the current corner vertex of the triangle in the three-dimensional mesh may include simplified representation information or complete representation information.

[0227] If the current triangle is in a predetermined mode, the decoder needs to decode its vertex position information to complete the stitching of the corresponding vertices using the representation information. For simplified representation information and complete representation information, please refer to the description of the encoder above, which will not be repeated here.

[0228] 2) In an optional implementation manner, the process of entropy decoding the input bit stream to obtain additional information indicating the position relationship of the triangles includes:

[0229] The input bitstream is entropy decoded. When there are traversed adjacent triangles on both sides of the traversed triangle and the pattern has a ring property, the additional information obtained is the diagonal information of the triangle for reconstruction operation.

[0230] Among them, taking the CLERS mode as an example, when there are traversed adjacent triangles on both sides of the traversed triangle, it can be one of the E, R, and L modes. The information of the diagonal of the triangle with ring properties can be found in the description of the encoding end above, which will not be repeated here; the reconstruction operation can be a zip operation.

[0231] 3) In an optional implementation manner, the process of entropy decoding the input bit stream to obtain additional information indicating the position relationship of the triangles includes:

[0232] The input code stream is entropy decoded. If there are traversed adjacent triangles on both sides of the traversed triangle, if the pattern does not have a ring property and the corresponding rotation vertices are connected to two or more triangles of a predetermined pattern, the additional information obtained is the fan number of the triangle.

[0233] Among them, taking the CLERS mode as an example, when there are traversed adjacent triangles on both sides of the traversed triangle, it can be one of the E, R, and L modes, and the predetermined mode can be the S mode. Please refer to the description of the encoding end above for the meaning of the sector number, which will not be repeated here. In the above-mentioned several implementation methods, entropy decoding can be selected as needed and is not limited here.

[0234] D2, based on the mode and its corresponding stitching method, connect the current triangle to the reconstructed three-dimensional mesh; this step may include: determining the initial diagonal information of the current triangle according to the mode, and updating the diagonal information according to the stitching method corresponding to the mode, so as to connect the current triangle to the reconstructed three-dimensional mesh; wherein, if the traversed triangle meets the preset condition, the stitching method of the triangle is implemented based on the corresponding additional information;

[0235] For the sake of clarity, the relevant content of D2 is explained after D3.

[0236] D3, determine the position of the next triangle according to the pattern, obtain relevant information required to decode the triangle, and use the obtained relevant information to decode the pattern of the triangle.

[0237] It can be understood that after obtaining the relevant information required for decoding the triangle, the process returns to the pattern entropy decoding process.

[0238] The relevant information includes at least one or more of: vertex geometry information, encoded triangle mode, and the number of adjacent encoded triangles of the triangle vertex.

[0239] Specifically, for D2, if the context information uses the number of adjacent encoded triangles of the triangle vertex, then after the entropy decoding of the triangle pattern is completed, the current triangle needs to be immediately connected to the reconstructed mesh, so that the context information used for the entropy decoding of the next triangle is consistent with the encoder. The process of reconstructing the mesh at the decoding end is the process of reconstructing the network structure of the encoder (such as Corner-Table, half-edge, etc.). This part of the embodiment of the present invention uses the reconstruction of Corner-Table as an example to introduce the connection relationship reconstruction process.

[0240] Since the connection relationship of the triangle can be reconstructed by determining the diagonal relationship of each angle and the relationship between the corresponding vertices of the angle, reconstructing the connection relationship is essentially to reconstruct the O table and V table in the Corner-Table.

[0241] In the process of rebuilding a triangle, the initial diagonal relationship is first set according to the current triangle mode. Setting it to -1 or -2 means that the diagonal of the current angle is not determined. Other symbols can also be used instead of -1 or -2. Fig. 20 shown.

[0242] Taking the Corner-Table data structure and CLERS mode as an example, the initial diagonal information of the current triangle is determined according to the mode, including:

[0243] ① If the CLERS mode of the current triangle is C mode, the initial diagonal information is determined to be co=2, cpo=1, cno=-1;

[0244] ② If the CLERS mode of the current triangle is L mode, the initial diagonal information is determined to be co=2, cpo=1, cno=-2;

[0245] ③ If the CLERS mode of the current triangle is R mode, the initial diagonal information is determined to be co=-2, cpo=1, cno=2;

[0246] ④ If the CLERS mode of the current triangle is S mode, the initial diagonal information is determined to be co=2, cpo=1, cno=3;

[0247] ⑤ If the CLERS mode of the current triangle is E mode, the initial diagonal information is determined to be co=-2, cpo=1, cno=-2.

[0248] In the above process, some corners cannot establish a correct diagonal relationship, that is, the edges corresponding to some corners are not stitched. Therefore, the embodiment of the present invention needs to stitch the corresponding edges of the triangles to the reconstructed grid immediately after decoding the triangle pattern.

[0249] In the embodiment of the present invention, the diagonal information is updated in different ways for different modes. Taking the above example, it is explained here that for the three modes of E, R, and L, there are handle cases and non-handle cases. For the handle case, the diagonal information is updated using the decoded diagonal information, and for the non-handle case, the diagonal information is updated using the corresponding method in the mode stitching method. For the S and C modes, the diagonal information is updated using the corresponding method in the mode stitching method.

[0250] The following are stitching methods for different modes at the decoder end, taking the Corner-Table data structure and the CLERS mode as an example.

[0251] (I) The suturing method corresponding to the L mode includes:

[0252] Step b1, determine whether the left branch of the current triangle c angle is a handle, if so, update the diagonal information of the c angle to the decoded diagonal information, update cnv, and end stitching; otherwise jump to step b2;

[0253] The method of judging whether the left branch of the current triangle c angle is a handle can be found in the related technical understanding, which will not be explained here. If it is a handle, since the handle cannot find the diagonal through the stitching operation, the encoder needs to transmit the diagonal information. Updating cnv means cnv = copv, and ending stitching means ending the stitching of the triangle, and then operating on the next triangle.

[0254] Step b2, determine whether cpv is connected to multiple S-mode triangles, if yes, jump to step b6, otherwise jump to step b3;

[0255] Step b3, initialize angle b, set b = cn, such as Fig.21 The first picture in ;

[0256] Step b4, determine whether bo is greater than 0. If it is greater than 0, set b = bon. Fig.21 Repeat step b4 for the second graph in the figure; if bo is less than or equal to 0, such as Fig.21If the third figure is in , jump to step b5;

[0257] Step b5, let bo = c, co = b, bpv = cnv, such as Fig.21 The fourth figure in the figure shows the end of the suture;

[0258] Step b6, read from the corresponding sector number the number of sectors on which the angle to be zipped is located according to the traversal order of the triangle, and obtain the t angle of the sector, and let b = tp, such as Fig. 22 , jump to step b5; wherein the t angle of the sector is the angle connecting the sector and its rotation vertex.

[0259] See also Fig.21 The schematic diagram of the stitching process of mode L shown in FIG. Fig. 22 The L-mode rotation vertex and two S-mode triangles are shown in the figure to understand the stitching process of the L-mode. For the symbols of the corners and vertices involved in the stitching process of each mode, please refer to the Corner-Table and the encoding method in the previous article.

[0260] (II) Suturing methods corresponding to the E-mode include:

[0261] Step c1, determine whether the right branch of the current triangle c angle is a handle, if so, update the diagonal information of the c angle to the decoded diagonal information, and update the cvv; wherein updating the cvv means that cpv=conv;

[0262] Step c2, determine whether the left branch of angle c is a handle, if so, update cnv and end stitching; otherwise jump to step c3;

[0263] Step c3, determine whether cpv is connected to multiple S-mode triangles, if yes, jump to step c8, otherwise jump to step c4;

[0264] Step c4, initialize angle b, set b = cn;

[0265] Step c5, determine whether bo is greater than 0. If so, set b=bon and repeat step c5; if bo is less than or equal to 0, jump to step c6;

[0266] Step c6, let bo = c, co = b, bpv = cnv;

[0267] Step c7, let c = cp, if co is not equal to -2, jump to step c4, otherwise end the suturing;

[0268] Step c8, read from the corresponding sector number the number of sectors on which the angle that needs to be zipped according to the traversal order of the triangle is located, obtain the t angle of the sector, set b=tp, and jump to step c6; wherein the t angle of the sector is the angle connecting the sector and its rotation vertex.

[0269] Except for handle detection, the stitching methods of L mode and E mode are basically the same, which can be used as a reference for understanding.

[0270] (III) Suturing methods corresponding to R mode include:

[0271] Step d1, determine whether the right branch of the current triangle c angle is a handle, if so, update the diagonal information of the c angle to the decoded diagonal information, update the cpv, and end the stitching; otherwise jump to step d2;

[0272] Step d2, determine whether cnv is connected to multiple S-mode triangles, if yes, jump to step d5, otherwise jump to step d3;

[0273] Step d3, initialize angle b, set b = cp, such as Fig.23 The first picture of

[0274] Step d4, determine whether bo is greater than 0, if it is greater than 0, then set b = bop, such as Fig.23 Repeat step d4 for the second graph; if bo is less than or equal to 0, such as Fig.23 If the third figure is , jump to step d5;

[0275] Step d5, let bo = c, co = b, bnv = cpv, such as Fig.23 The fourth picture shows the end of the suture;

[0276] Step d6, read from the corresponding sector number the number of sectors on which the angle that needs to be zipped according to the traversal order of the triangle is located, obtain the t angle of the sector, set b=tn, and jump to step d5; wherein the t angle of the sector is the angle connecting the sector and its rotation vertex.

[0277] See also Fig.23 The schematic diagram of the suturing process of mode R is shown to help you understand the suturing process of mode R.

[0278] (IV) Suturing methods corresponding to the S pattern include:

[0279] According to the decoded representation information of the position of the c-angle vertex of the triangle in the three-dimensional grid, the corresponding vertex is found and connected according to the rules consistent with the encoding end;

[0280] When performing the stitching operation, if the representation information is complete, the vertex related information is read for stitching; if the representation information is simplified, the number of vertex rotations is read, and the rotation is performed using the same method as the encoding end, and finally the vertices of cp are updated.

[0281] Specifically, for the S mode, it is only necessary to connect the cp corner vertex to the grid at the decoding end. At this time, the decoding end first decodes the position of the c corner vertex in the grid, and then finds the corresponding vertex and connects it according to the same rules as the encoding end, such as Fig.11 As shown. On the decoding end, when performing stitching, if Fig.12 In the case shown, read the vertex related information for stitching, otherwise read the number of rotations, use the same method as the encoding end to rotate, and finally update the vertices of cp.

[0282] The above is the process of decoding the connection relationship at the decoding end.

[0283] For geometric information decoding, taking a separate decoding framework as an example, the geometric information code stream can be decoded according to the connection relationship decoding order to obtain geometric information, including:

[0284] 1) Perform entropy decoding on the geometric information bitstream to obtain the prediction residual;

[0285] 2) using a geometric coordinate prediction method corresponding to the encoding end to obtain predicted geometric coordinates;

[0286] 3) The reconstructed geometric coordinates are obtained based on the sum of the predicted geometric coordinates and the prediction residuals.

[0287] Those skilled in the art can understand that the above decoding process corresponds to the processing process of the encoding end, and the prediction method of the geometric coordinates is consistent with the encoding end, which is not described in detail here. The geometric information decoding process in the hybrid decoding framework can be similar to the above process and is not described in detail here.

[0288] A three-dimensional mesh decoding method proposed in an embodiment of the present invention is to decode the connection relationship and the geometric information of the input code stream to obtain the connection relationship and the geometric information respectively; reconstruct the three-dimensional mesh based on the obtained connection relationship and geometric information; wherein the connection relationship decoding includes decoding the pattern of triangles in the three-dimensional mesh, and at the same time decoding the additional information used to indicate the position relationship of the triangles to achieve the instant stitching of the current triangle; the connection relationship decoding is implemented based on the entropy decoding of the context information. The connection relationship decoding uses the relevant information of the triangles and the decoding method corresponding to the encoding end to decode the corresponding part of the input code stream to obtain the pattern of the triangles, and uses the decoded triangle pattern to construct the data structure of the mesh to achieve three-dimensional mesh reconstruction, so that the context information consistent with the encoding end can be obtained for accurate decoding and mesh reconstruction.

[0289] Furthermore, for triangles in a predetermined pattern, an embodiment of the present invention also decodes additional information, including the position of the current corner vertex in the three-dimensional mesh, so that when reconstructing the triangle in three dimensions, the same context information as the encoding end can be used to effectively locate the vertex, thereby ensuring the accuracy of reconstruction.

[0290] Furthermore, when there are traversed adjacent triangles on both sides of the traversed triangle, the embodiment of the present invention also decodes additional information, including diagonal information of the triangle transmitted by the encoding end, so as to facilitate the accuracy of the stitching process of the decoding end when the triangle pattern has a ring property in this case. In addition, the additional information decoded includes the sector number transmitted by the encoding end to ensure the accuracy of stitching.

[0291] In a fourth aspect, corresponding to the three-dimensional grid decoding method provided in the third aspect, an embodiment of the present invention further provides a three-dimensional grid decoding device, which is applied to a decoding end, and the device includes:

[0292] A decoding module, used for decoding the connection relationship and the geometric information of the input code stream to obtain the connection relationship and the geometric information respectively;

[0293] A reconstruction module, used for reconstructing a three-dimensional mesh based on the obtained connection relationship and geometric information;

[0294] Among them, connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding the additional information used to indicate the position relationship of the triangles to achieve instant stitching of the current triangle; connection relationship decoding is achieved based on entropy decoding of context information.

[0295] For the specific processing process of each of the above modules, please refer to the three-dimensional grid decoding method described in the third aspect, which will not be described here.

[0296] The above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A three-dimensional grid coding method, characterized in that: include: Encode the connection relationship and geometric information of the input three-dimensional grid to obtain an output code stream; The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of triangles; the connection relationship coding is implemented based on entropy coding of context information.

2. The three-dimensional grid coding method according to claim 1, characterized in that: The step of encoding the connection relationship and the geometric information of the input three-dimensional mesh to obtain an output code stream includes: Traversing the triangles in the three-dimensional grid to perform connection relationship coding to obtain a connection relationship coding result; Traversing the triangles in the three-dimensional grid to perform geometric information encoding to obtain a geometric information encoding result; wherein the connection relationship encoding and the geometric information encoding are performed in units of triangles; An output code stream is obtained according to the connection relationship encoding result and the geometric information encoding result.

3. The three-dimensional grid coding method according to claim 1, characterized in that: The process of encoding the connection relationship of the input 3D mesh includes: Initializing the three-dimensional grid and establishing a related data structure; For each traversed triangle, determine the traversal result of the triangle based on the data structure; wherein the traversal result of the triangle includes the mode of the triangle; Using relevant information of the traversed triangle as context information, entropy encoding the pattern of the triangle; wherein the relevant information includes at least one or more of vertex geometry information, the encoded triangle pattern, and the number of adjacent encoded triangles of the triangle vertex; If the traversed triangle meets the preset conditions, the corresponding additional information is determined, incorporated into the traversal result of the triangle, and the additional information is entropy encoded.

4. The three-dimensional grid coding method according to claim 3, characterized in that: If the traversed triangle meets the preset conditions, the corresponding additional information is determined, including: If the pattern of the traversed triangle is a predetermined pattern, characterization information of the position of the triangle in the three-dimensional grid is determined as additional information.

5. The three-dimensional grid coding method according to claim 3, characterized in that: If the traversed triangle meets the preset conditions, the corresponding additional information is determined, including: When there are encoded adjacent triangles on both sides of the traversed triangle, it is determined whether the pattern of the traversed triangle has a ring property. If so, the diagonal information of the triangle is recorded as a kind of additional information.

6. The three-dimensional grid coding method according to claim 3 or 4, characterized in that: If the traversed triangle meets the preset conditions, the corresponding additional information is determined, including: When there are encoded adjacent triangles on both sides of the traversed triangle, if the pattern of the triangle does not have a ring property, it is determined whether the rotation vertex used in the current pattern during the reconstruction operation is connected to two or more triangles of the predetermined pattern. If so, the angle to be stitched is determined based on the traversal order of the triangles. The sector number is obtained as additional information.

7. A three-dimensional grid encoding device, characterized in that: include: The encoding module is used to encode the connection relationship and geometric information of the input three-dimensional grid to obtain an output code stream; The connection relationship coding includes coding the pattern of triangles in the three-dimensional grid and coding additional information for indicating the positional relationship of triangles; the connection relationship coding is implemented based on entropy coding of context information.

8. A three-dimensional grid decoding method, characterized in that: include: Decode the connection relationship and the geometric information of the input code stream to obtain the connection relationship and the geometric information respectively; Reconstructing a three-dimensional mesh based on the obtained connection relationship and geometric information; The connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding the additional information indicating the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

9. The three-dimensional grid decoding method according to claim 8, characterized in that: The input code stream is decoded for connection relationship and geometric information to obtain connection relationship and geometric information respectively; Reconstruct the 3D mesh based on the obtained connection relationship and geometric information, including: De-streaming the input code stream to obtain a connection relationship code stream and a geometric information code stream; wherein the connection relationship code stream and the geometric information code stream are in units of triangles; Performing connection relationship decoding on the connection relationship code stream to obtain a connection relationship; Decoding the geometric information code stream to obtain geometric information; The three-dimensional mesh is reconstructed using the connection relationship and the geometric information.

10. The three-dimensional grid decoding method according to claim 8, characterized in that: The process of decoding the connection relationship of the input code stream includes: Performing entropy decoding on the input code stream to obtain a mode of a current triangle and additional information indicating a positional relationship of the triangle; Based on the mode and its corresponding stitching method, the current triangle is connected to the reconstructed three-dimensional mesh; wherein, if the traversed triangle meets the preset condition, the stitching method of the triangle is implemented based on the corresponding additional information; The position of the next triangle is determined according to the pattern, and relevant information required for decoding the triangle is obtained, and the pattern of the triangle is entropy decoded using the obtained relevant information.

11. The three-dimensional grid decoding method according to claim 10, characterized in that: The relevant information at least includes: One or more of vertex geometry information, encoded triangle mode, and the number of adjacent encoded triangles of a triangle vertex.

12. The three-dimensional grid decoding method according to claim 10, characterized in that: The process of entropy decoding the input bitstream to obtain additional information indicating the position relationship of the triangles includes: The input code stream is entropy decoded, and when the current mode of the triangle is a predetermined mode, the additional information obtained is the representation information of the position of the triangle in the three-dimensional grid.

13. The three-dimensional grid decoding method according to claim 10, characterized in that: The process of entropy decoding the input bitstream to obtain additional information indicating the position relationship of the triangles includes: The input code stream is entropy decoded, and when there are traversed adjacent triangles on both sides of the traversed triangle and the pattern has a ring property, the additional information obtained is the diagonal information of the triangle.

14. The three-dimensional grid decoding method according to claim 10 or 12, characterized in that: The process of entropy decoding the input bitstream to obtain additional information indicating the position relationship of the triangles includes: The input code stream is entropy decoded. If there are traversed adjacent triangles on both sides of the traversed triangle, if the pattern does not have a ring property and the corresponding rotation vertices are connected to two or more triangles of a predetermined pattern, the additional information obtained is the fan number of the triangle.

15. A three-dimensional grid decoding device, characterized in that: include: A decoding module, used for decoding the connection relationship and the geometric information of the input code stream to obtain the connection relationship and the geometric information respectively; A reconstruction module, used for reconstructing a three-dimensional mesh based on the obtained connection relationship and geometric information; The connection relationship decoding includes decoding the pattern of triangles in the three-dimensional grid, and decoding the additional information indicating the position relationship of the triangles to achieve instant stitching of the current triangle; the connection relationship decoding is achieved based on entropy decoding of context information.

Images