Three-dimensional grid coding and decoding method and device

Abstract

The invention discloses a three-dimensional grid coding and decoding method and device, and relates to the technical field of coding and decoding, and the method comprises the steps: obtaining sub-grid merging information according to a plurality of sub-grids of an original grid; and coding the sub-grids and the sub-grid merging information to obtain a target code stream. According to the invention, the original grid can be obtained losslessly.

Description

Technical Field

[0001] The present invention belongs to the technical field of coding and decoding, and in particular relates to a three-dimensional grid coding and decoding method and a device thereof. Background Art

[0002] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have become a new generation of digital media after audio, images, and videos. Three-dimensional grids and point clouds are two commonly used three-dimensional model representation methods. Among them, three-dimensional grid models have stronger interactivity and realism compared to traditional multimedia such as images and videos, making them more and more widely used in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and military.

[0003] With the increasing demand for visual effects of 3D mesh models and the emergence of many more mature 3D scanning technologies and 3D modeling software, the data size and complexity of 3D mesh models obtained by 3D scanning equipment or 3D modeling software are also growing rapidly. Therefore, how to efficiently compress 3D mesh data is the key to conveniently transmit, store and process 3D mesh data. Among them, 3D meshes include three main types of information: topological information, geometric information and attribute information. Topological information, also known as connectivity relationship information, is used to describe the connection relationship between elements such as vertices and patches in the mesh; geometric information is the 3D coordinates of all vertices in the mesh; attribute information records other information attached to the mesh, such as normal vectors, texture coordinates and colors. The compression of 3D mesh data is often performed on the above three types of information according to their data characteristics. In addition, for 3D meshes with texture maps, the texture maps also need to be compressed.

[0004] In the related technology, MPEG, the international standards organization for video image neighborhood, is developing a new dynamic 3D mesh compression standard V-DMC, and has adopted a 3D mesh encoding and decoding solution based on Edgebreaker to achieve compression of 3D mesh geometric information, connection information, and attribute information. At the same time, the standard also adopts the concept of submesh, allowing the input 3D mesh to be divided into multiple submeshes and each submesh can be encoded and decoded independently. However, for the current V-DMC implementation, in lossless mode, for the encoding and decoding of a 3D mesh containing multiple submeshes, when merging submeshes at the decoding end, the final decoded mesh obtained by directly adding each reconstructed submesh to a new mesh does not delete the boundary points of adjacent submeshes that are repeatedly divided, thereby failing to restore the decoded mesh losslessly.

[0005] Therefore, there is an urgent need to propose a three-dimensional mesh encoding and decoding method to ensure lossless implementation of three-dimensional mesh encoding and decoding including multiple sub-meshes. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a three-dimensional grid encoding and decoding method and a device thereof. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] In a first aspect, the present invention provides a three-dimensional grid coding method, comprising:

[0008] According to multiple sub-grids of the original grid, obtain sub-grid merging information;

[0009] The sub-grid and sub-grid merging information are encoded to obtain the target code stream.

[0010] In a second aspect, the present invention further provides a three-dimensional grid decoding method, comprising:

[0011] Decode the target code stream to obtain multiple sub-grids of the original grid and sub-grid merging information;

[0012] According to the sub-grid merging information, the sub-grids that need to be merged are merged to obtain a decoded grid.

[0013] In a third aspect, the present invention further provides a three-dimensional grid encoding device, comprising:

[0014] A first acquisition unit, configured to acquire sub-grid merging information according to multiple sub-grids of the original grid;

[0015] The second acquisition unit is used to encode the sub-grid and the sub-grid merging information to obtain a target code stream.

[0016] In a fourth aspect, the present invention further provides a three-dimensional grid decoding device, comprising:

[0017] A first acquisition unit, used for decoding a target code stream, and acquiring multiple sub-grids of an original grid and sub-grid merging information;

[0018] The second acquisition unit is used to merge the sub-grids that need to be merged according to the sub-grid merging information to obtain a decoding grid.

[0019] Beneficial effects of the present invention:

[0020] The present invention provides a three-dimensional mesh coding and decoding method and a device thereof, wherein the coding and decoding methods include two methods, both of which can realize lossless merging of sub-grids. Different strategies can be selected for merging sub-grids at a decoding end according to sub-grid merging information obtained at an encoding end. The first method is implemented based on mesh adjustment information. First, the encoded sub-grids are decoded and reconstructed at the encoding end to obtain reconstructed sub-grids. Then, a merging method is selected according to the sub-grid merging information to merge the reconstructed sub-grids. The merging method can be to merge the reconstructed sub-grids by merging repeated information between sub-grids, or to directly add each information of the reconstructed sub-grids to a blank grid to obtain a reconstructed grid. Thirdly, the reconstructed grid is compared with the original grid. If there is a situation that the reconstructed grid is different from the original grid due to merging, the grid adjustment information can be obtained and encoded during the comparison process. At the decoding end, first, a merging method is selected according to the sub-grid merging information to merge the reconstructed sub-grids to obtain a reconstructed grid. Then, the merged grid is adjusted according to the grid adjustment information to obtain a final decoded grid. The second method is implemented based on the indication information of merging repeated information. The corresponding vertex mapping relationship information can be obtained before and after the sub-mesh division at the encoding end. Based on this information and the indication information of merging repeated information between sub-meshes, the repeated division identification information of the vertices on the sub-mesh that needs to merge repeated information and the corresponding relationship information of the repeatedly divided vertices that need to be merged are obtained at the encoding end, and the information is encoded; at the same time, the indication information of merging repeated information between sub-meshes is encoded, which can include two types of information: one is the identification information indicating whether the sub-mesh needs to perform the merge repeated information operation, and the other is the indication information indicating which sub-mesh to merge with for the sub-mesh that needs to perform the merge repeated information operation. At the decoding end, the sub-meshes are merged according to the indication information of merging repeated information, the repeatedly divided points that need to be merged are merged, the repeated points in the original mesh on the boundary that is not repeatedly divided are retained, and the sub-meshes that do not need to merge repeated information are directly added to the reconstructed mesh to obtain the final decoded mesh, so as to achieve lossless recovery of the original mesh.

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a three-dimensional grid encoding method provided by an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of sub-grid division provided by an embodiment of the present invention;

[0024] Figure 3 is another schematic diagram of sub-grid division provided by an embodiment of the present invention;

[0025] Figure 4 is another schematic diagram of sub-grid division provided by an embodiment of the present invention;

[0026] Figure 5 is another flow chart of the three-dimensional grid encoding method provided by an embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of a three-dimensional mesh encoding method based on Edgebreaker provided in an embodiment of the present invention;

[0028] Figure 7 is a schematic diagram of sub-grid merging provided by an embodiment of the present invention;

[0029] Figure 8 is another flow chart of the three-dimensional grid encoding method provided by an embodiment of the present invention;

[0030] Fig. 9 is another flow chart of the three-dimensional grid encoding method provided by an embodiment of the present invention;

[0031] Fig.10 is a flow chart of a three-dimensional grid decoding method provided by an embodiment of the present invention;

[0032] Fig.11 is another flow chart of a three-dimensional grid decoding method provided by an embodiment of the present invention;

[0033] Fig.12 is a flow chart of a three-dimensional mesh decoding method based on Edgebreaker provided in an embodiment of the present invention;

[0034] Fig.13 is another flow chart of a three-dimensional grid decoding method provided by an embodiment of the present invention;

[0035] Fig.14 is a schematic diagram of a three-dimensional grid encoding device provided by an embodiment of the present invention;

[0036] Fig.15 It is a schematic diagram of a three-dimensional grid decoding device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0038] In the prior art, the three-dimensional mesh compression tool currently provided by MPEG contains multiple sub-meshes. Before encoding, the input mesh is divided into sub-meshes using a bounding box-based partitioning scheme. Then, the connection information, geometric information and UV coordinates of the three-dimensional mesh are independently encoded and stored for each sub-mesh using an Edgebreaker-based method. The core module, i.e., the module for encoding connection information, uses the Edgebreaker algorithm. Conventional compression methods are used for encoding geometric information and attribute information such as texture coordinates, i.e., data quantization, prediction compression (parallelogram prediction) and entropy coding. Since the tool uses a connection relationship-driven encoding method, the encoding of geometric information and attribute information will follow the encoding order of the connection information. At the decoding end, after decoding each sub-mesh, the information of each sub-mesh is directly added to a new mesh to obtain the final decoded mesh, without deleting the boundary points of adjacent sub-meshes that are repeatedly divided, so that the decoded mesh cannot be restored losslessly.

[0039] In view of this, the present invention provides a three-dimensional mesh encoding and decoding method, which achieves lossless restoration of the original mesh during the encoding and decoding process by acquiring sub-mesh merging information.

[0040] See also Figure 1 , Figure 1 is a flow chart of a three-dimensional mesh coding method provided by an embodiment of the present invention. The three-dimensional mesh coding method provided by the present invention includes:

[0041] According to multiple sub-grids of the original grid, obtain sub-grid merging information;

[0042] The sub-grid and sub-grid merging information are encoded to obtain the target code stream.

[0043] Specifically, see Figure 1 The three-dimensional mesh encoding method provided in this embodiment is for an original mesh including multiple sub-meshes, where the original mesh is a three-dimensional mesh. Sub-mesh merging information is determined at the encoding end, and the sub-mesh merging information is encoded to generate a code stream; wherein the sub-mesh merging information includes information such as which sub-meshes are merged, a sub-mesh merging strategy, and a sub-mesh merging method; after the sub-meshes are independently encoded, the target code stream is decoded at the decoding end, and the decoded and reconstructed sub-meshes are merged according to the sub-mesh merging information obtained by decoding, so as to realize lossless decoding of the three-dimensional mesh.

[0044] Specifically, see Figure 1First, the original grid is divided into subgrids, such as a subgrid division method based on boundingbox, which is not limited here, to obtain multiple subgrids, and the subgrids are compressed to obtain subgrid code streams. For example, each subgrid can be independently compressed by an Edgebreaker method, which is not limited here; secondly, in the process of subgrid division, subgrid merging information is obtained. For the strategy of obtaining subgrid merging information, the encoding end reconstructs the subgrid according to the merging information to obtain the merged reconstructed grid, and then compares the reconstructed grid with the original grid. If there is a reconstructed grid caused by the merging, If the mesh is different from the original mesh, the sub-mesh merging information that needs to be adjusted for the reconstructed mesh is obtained. The mapping relationship between vertices that can be obtained in the sub-mesh division process can also be used to obtain the corresponding relationship between the vertices of the sub-meshes that need to merge the repeated information, thereby obtaining the sub-mesh merging information. The strategy for obtaining the sub-mesh merging information is not limited here. The sub-mesh merging information is encoded, such as entropy coding and other methods can be used. The encoding method of the sub-mesh merging information is not limited here, and the sub-mesh merging information code stream is obtained; in addition, the texture map is encoded, such as a video codec can be used to compress the texture map to obtain a texture map code stream; the various code streams obtained above are mixed to obtain a mixed stream, that is, the target code stream.

[0045] In this embodiment, considering that after the original grid is divided into subgrids at the encoding end, there is information that is repeatedly divided between subgrids that were originally connected in the original grid, therefore, the reconstructed subgrids are merged and the duplicate information is deleted to achieve a lossless decoding process. It can be understood that after the subgrids are independently encoded and decoded, the reconstructed subgrids need to be merged at the decoding end to merge the information that is repeatedly divided between the subgrids and restore the original grid losslessly.

[0046] In this embodiment, the information that is repeatedly divided between sub-grids needs to be merged, according to the sub-grid merging information. Figure 2 , Figure 2 is a schematic diagram of sub-grid division provided by an embodiment of the present invention. In the original grid, the same color represents the same independent part. In the sub-grid after division, black, blue and green respectively represent the independent parts that are not connected to each other in the original grid. Red represents the information that is repeatedly divided between sub-grids. For the sub-grids that need to merge the repeated information, such as Figure 2 Subgrid 1, subgrid 2 and subgrid 3, as well as subgrid 5 and subgrid 6, need to encode information about which subgrids need to be merged with the repeated information when the subgrids are merged; in addition, if the subgrid with which the repeated information needs to be merged is a special subgrid, such as the subgrid with the smallest number, the above information does not need to be encoded.

[0047] For sub-grids that need to merge duplicate information, you also need to consider special cases such as the duplicate points of the original grid existing on the grid boundary. See Figure 3 and Figure 4 , Figure 3 is another schematic diagram of sub-grid division provided by an embodiment of the present invention, Figure 4 is another schematic diagram of sub-grid division provided by an embodiment of the present invention, the 0 point and the 0' point in the original grid, Figure 3 0 and 0' points in Neutron Grid 1, and Figure 3 and Figure 4 Points 2 and 2' in sub-grid 2 are duplicate point pairs with the same geometric coordinates but different vertex indices. The red mark in the sub-grid is the information that is repeatedly divided in the original grid, and the green mark is the duplicate information in the original grid that needs to be retained when the sub-grids are merged. After the sub-grid 1 and sub-grid 2 are independently losslessly encoded, it is necessary to merge the reconstructed sub-grid 1 and the reconstructed sub-grid 2 at the decoding end to merge the duplicate information in sub-grid 1 or sub-grid 2. At this time, it is also necessary to consider that the boundary points are points that are repeatedly divided and need to be merged, such as Figure 3 The 0' point of subgrid 1 or the 2' point of subgrid 2 is still a point that has not been repeatedly divided and needs to be retained, such as Figure 4 2' point in subgrid 2 in .

[0048] In view of the general idea of ​​the above sub-grid merging, the three-dimensional grid encoding method provided by the present invention includes two methods. The first method is implemented based on grid adjustment information. First, the encoded sub-grids are decoded and reconstructed at the encoding end to obtain a reconstructed sub-grid; then, according to the sub-grid merging information, the reconstructed sub-grids are merged. The merging method can be to merge the reconstructed sub-grids by merging the repeated information between the sub-grids, or to directly add the information of the reconstructed sub-grids to a blank grid to obtain a reconstructed grid. The specific merging method is not limited here; then, the reconstructed grid is compared with the original grid. If there is a situation where the reconstructed grid is different from the original grid due to the merging, the grid adjustment information can be obtained and encoded in the comparison process; at the decoding end, first, according to the sub-grid merging information, the reconstructed sub-grids are merged to obtain a reconstructed grid; then, the merged grid is adjusted according to the grid adjustment information to obtain the final decoded grid. The second method is implemented based on the indication information of merging repeated information. The corresponding vertex mapping relationship information can be obtained before and after the sub-mesh division at the encoding end. Based on this information and the indication information of merging repeated information between sub-meshes, the repeated division identification information of the vertices on the sub-mesh that needs to merge repeated information and the corresponding relationship information of the repeatedly divided vertices that need to be merged are obtained at the encoding end, and the information is encoded; at the same time, the indication information of merging repeated information between sub-meshes is encoded, and the information can include two types of information: one is the identification information indicating whether the sub-mesh needs to perform the merge repeated information operation, and the other is the indication information indicating which sub-mesh to merge with for the sub-mesh that needs to perform the merge repeated information operation. At the decoding end, according to the indication information of merging repeated information (including the repeated division identification information of the repeatedly divided vertices between the sub-meshes that need to merge repeated information, the corresponding relationship information of the repeatedly divided vertices that need to be merged, and the indication information of merging repeated information between sub-meshes), the sub-meshes are merged, the repeatedly divided points that need to be merged are merged, the repeated points in the original mesh existing on the boundary that is not repeatedly divided are retained, and the sub-meshes that do not need to merge repeated information are directly added to the reconstructed mesh to obtain the final decoded mesh.

[0049] For the first encoding method, which is based on grid adjustment information, please see the following details.

[0050] In an optional embodiment of the present invention, see Figure 5 , Figure 5 It is another flow chart of the three-dimensional mesh encoding method provided by an embodiment of the present invention, wherein the sub-mesh merging information includes mesh adjustment information, and the mesh adjustment information includes at least one of adding vertex index information, deleting vertex index information and connection relationship adjustment information.

[0051] For more details, please see Figure 5In this embodiment, the mesh adjustment information is used to adjust the reconstructed mesh to further obtain the lossless original mesh; when the reconstructed mesh vertices are missing, the vertex index information is added; when the reconstructed mesh vertices are repeated, the vertex index information is deleted; when the vertex connection relationship in the reconstructed mesh is wrong, the connection relationship between the vertices is adjusted; in this way, the original mesh is restored losslessly.

[0052] In an optional embodiment of the present invention, please continue to refer to Figure 5 , the grid adjustment information is obtained by merging and reconstructing subgrids, wherein the process of merging and reconstructing subgrids includes:

[0053] According to the sub-grid merging information, the reconstructed sub-grids are merged, and the duplicate information between the sub-grids is removed to obtain the reconstructed grid.

[0054] For more details, please see Figure 5 In this embodiment, at the encoding end, the compressed sub-grids are decoded to obtain the reconstructed sub-grids, and the reconstructed sub-grids are merged to remove the duplicate information between the sub-grids to obtain the reconstructed grid for the next step of operation.

[0055] It should be noted that, in this embodiment, the method of obtaining the reconstructed sub-grid also includes directly adding the information included in each sub-grid into a blank grid to obtain the reconstructed grid.

[0056] In an optional embodiment of the present invention, please continue to refer to Figure 5 ,The process of obtaining grid adjustment information includes:

[0057] The reconstructed mesh is compared with the original mesh, and mesh adjustment information is obtained according to the difference between the reconstructed mesh and the original mesh.

[0058] For more details, please see Figure 5 The three-dimensional mesh encoding method provided in this embodiment is implemented based on the strategy of mesh adjustment information. Sub-grids are obtained and encoded at the encoding end, the encoded sub-grids are reconstructed, the reconstructed sub-grids are merged according to the sub-grid merging information, duplicate information is removed, and then the reconstructed mesh is compared with the original mesh. If the reconstructed mesh is different from the original mesh due to the merging, the mesh adjustment information can be obtained and encoded. At the decoding end, the reconstructed sub-grids are merged according to the sub-grid merging information to obtain a reconstructed mesh. After the merging is completed, the reconstructed mesh is adjusted according to the mesh adjustment information to obtain a decoded mesh, so as to realize lossless encoding and decoding of a three-dimensional mesh including multiple sub-grids.

[0059] In this embodiment, see Figure 5At the encoding end, first, the original grid is divided into subgrids, for example, a subgrid division method based on a bounding box can be used, and the subgrid division method is not limited here; in the subgrid division process, subgrid merging information is obtained, such as which subgrids the subgrid is merged with, and the subgrid merging method, and the information is encoded; secondly, each subgrid is independently compressed to obtain a subgrid bitstream, for example, each subgrid is independently compressed using a method such as an Edgebreaker-based method, and the subgrid compression method is not limited here; according to the subgrid merging information, the subgrid merging method to be used is determined, and the reconstructed subgrids are merged. Optionally, the merging method can be to merge and reconstruct the subgrids by merging repeated information between subgrids, or to directly add each information of the reconstructed subgrid to a blank grid to obtain a reconstructed grid, and the subgrid merging method is not limited here; then, the reconstructed grid is compared with the original grid to obtain grid adjustment information, and the information is encoded to obtain a grid adjustment information bitstream; in addition, the texture map is compressed using a video codec to obtain a texture map bitstream; the subgrid bitstream, the grid adjustment information bitstream and the texture map bitstream are mixed.

[0060] It should also be noted that the grid adjustment information can also be obtained in the sub-grid division module, and the sub-grid division module clearly knows which sub-grid information is repeated information and which information is not repeated information.

[0061] Please refer to the specific embodiments below for a detailed description of the encoding end.

[0062] In this embodiment, the implementation strategy based on the grid adjustment information includes a three-dimensional grid lossless coding framework of multiple sub-grids, which mainly includes: sub-grid division, sub-grid compression, sub-grid merging, grid comparison, and grid adjustment information coding. Specifically:

[0063] (1) Subgrid division;

[0064] In this embodiment, the sub-grid division method can use various methods such as division based on bounding box, and the specific division method is not limited here. The following description is made by taking the division method based on bounding box as an example.

[0065] First, according to the geometric coordinates of the input original mesh, the boundingbox range in the three-dimensional space is obtained; then, it is divided equally based on the coordinate range of the x-axis, y-axis or z-axis. The number of equal divisions can be set in advance or set according to the number of sub-meshes required by the input. After the equal division is completed, the bounding box range of each sub-mesh can be obtained; thirdly, for each bounding box range, all faces in the original mesh are traversed, and the vertices in the input mesh that meet the bounding box range are divided into the current sub-mesh in units of faces. After traversing each divided boundingbox, the final divided sub-mesh is obtained.

[0066] In the process of dividing the subgrids, it is possible to obtain which subgrids each subgrid needs to be merged with, and encode the subgrid merging information. For example, each subgrid is numbered according to a certain rule, 0-i respectively, and the minimum number here is 0 (the minimum number can also use 1, in this case 1-i). For the i-th subgrid, encode whether it needs to merge with other subgrids. The identification information ismerge may also not be encoded for each subgrid, such as encoding isallmerge information at the grid layer to indicate that all subgrids use this method, or it may not be encoded and the default method may be used. For the first subgrid, such as the 0th subgrid, ismerge may not be encoded; the ismerge information of each subsequent subgrid in this embodiment indicates that these encoding schemes may be used.

[0067] If the ismerge of the i-th sub-grid is 1, it is necessary to further encode which sub-grids it should be merged with, and encode the sub-grid indexes to be merged. When merging, they can be merged in sequence by serial number. For example, the i-1th sub-grid has been merged with the i-2th sub-grid, and it is marked as merged with the i-1th and sub-grids. It can also express merging with the i-2 sub-grids at the same time.

[0068] You can also encode ismerge as 1 for the sub-meshes that need to be merged, and encode ismerge as 0 for the sub-meshes that do not need to be merged. Without further encoding which sub-mesh to merge with, all sub-meshes with ismerge as 1 are merged.

[0069] In the process of sub-grid division, the grid adjustment information after the sub-grids are merged can also be directly given, that is, the repeated information in the original grid can be deleted during the merging process. At this time, the grid adjustment information can be encoded, that is, the encoding points, connection relationships and other information can be added to the merged grid.

[0070] (2) Subgrid coding;

[0071] In the sub-grid encoding process, each sub-grid is encoded independently. The encoding of each sub-grid is mainly divided into connection relationship encoding, geometric information encoding and attribute information encoding. The Edgebreaker-based method can be used to encode each sub-grid, and the specific encoding method is not emphasized here. The following is a specific description of the Edgebreaker-based encoding method.

[0072] See also Figure 6 , Figure 6 : is a schematic diagram of a three-dimensional mesh encoding method based on Edgebreaker provided in an embodiment of the present invention. First, the non-manifold structure in the sub-mesh is split to obtain a manifold mesh. In this process, the repeated point information generated by the splitting of the non-manifold is recorded, and the non-manifold structure information is encoded in a geometric coding order and / or an attribute coding order, such as an entropy coding method. The encoding method of the non-manifold structure information is not limited here; then, the connection relationship, geometric information and attribute information of the manifold mesh are encoded. For the connection relationship encoding, the Edgebreaker method can be used to encode the connection relationship of the three-dimensional mesh, by establishing a CornerTable to represent the connection relationship of the mesh, and using the CornerTable to traverse all triangles in the mesh to generate an Edgebreaker CLERS mode string, and the CLERS mode string is encoded using an entropy coding method to obtain a connection information code stream; the geometric information can be encoded according to the connection relationship encoding order using a variety of methods such as a difference prediction coding algorithm, a parallelogram prediction coding algorithm, etc. to obtain a geometric information code stream, and the specific encoding method is not emphasized here. The three-dimensional mesh attribute information generally includes texture coordinates, normal vectors, etc. Taking texture coordinates as an example, the texture coordinates can be encoded in the order of connection relationship using methods such as difference prediction coding, parallelogram prediction coding, and similar triangle prediction coding to obtain attribute information code streams. The specific encoding method is not emphasized here. The connection information code stream, geometric information code stream, attribute information code stream, and non-popular structure information code stream are mixed to obtain a sub-mesh code stream.

[0073] (3) Subgrid merging;

[0074] There are many ways to implement the sub-grid merging process. The sub-grids can be directly merged by merging the repeated information between the sub-grids, or the sub-grids can be merged according to the optional indication information of merging the repeated information between the sub-grids, or the information of each sub-grid can be directly added to a blank grid to obtain a reconstructed grid. In the implementation process, the indication information indicating which merging method is used can be added to the sub-grid merging information and encoded; or the corresponding merging method can be used by default at the encoding end to merge the sub-grids without transmitting the indication information of the merging method. The specific merging method is not emphasized here. The following introduces the implementation method of directly merging the repeated information between sub-grids.

[0075] Traverse the input reconstructed sub-grids and perform pairwise merging operations, see Figure 7 , Figure 7 This is a schematic diagram of sub-grid merging provided by an embodiment of the present invention. First, the reconstructed grid is initialized to any reconstructed sub-grid, and a mapping relationship list between its geometric coordinates and vertex indices is constructed. For the sub-grid selected for initialization, the information of the initialization grid can be notified to the decoding end by transmitting indication information, or the codec end can be directly defaulted to adopt the same initialization selection method, for example, the first reconstructed sub-grid that is traversed is selected as the initialized reconstructed grid, and then the next sub-grid is selected to merge with the current reconstructed grid to obtain a new reconstructed grid. When merging, the points in the current sub-grid are traversed, and the mapping relationship list between the geometric coordinates and the vertex index of the current reconstructed grid can be used to determine whether the geometric coordinates of the current point have been traversed; if the geometric coordinates of the current point have not been traversed, it is directly added to the reconstructed grid, and its index in the current sub-grid triangle is updated to its index in the reconstructed grid; if the geometric coordinates of the current point have been traversed, the point in the current sub-grid is deleted, and its index in the triangle is updated to the index of the point with the same geometric coordinates in the reconstructed grid. Finally, the mapping relationship list between the geometric coordinates and the vertex index in the reconstructed grid is updated, and after traversing all the points in the current sub-grid, the vertex merging operation is completed. Then add the triangles whose vertex indices are updated during the traversal process to the reconstructed mesh to obtain the updated reconstructed mesh. For the above deletion operation, you can delete the vertices immediately, or you can first record the information of the vertices to be deleted, and after all the vertices of the current sub-mesh are traversed and the connection relationship is updated, delete all the vertices to be deleted. The specific implementation method is not emphasized here, and this process is iterated until all sub-meshes are traversed to obtain the reconstructed mesh.

[0076] In addition, for attribute vertices and attribute triangles, such as texture coordinates and normal vectors, the above process can be executed separately once, or during the execution of the above process, attribute points corresponding to geometric coordinate points can be added or deleted accordingly, and the connection relationship of the attribute triangles can be updated.

[0077] The sub-grid merging process can merge the sub-grids in sequence according to their serial numbers. For the i-th sub-grid, its ismerge information is used to determine whether to merge with other sub-grids. When merging is required, the corresponding sub-grids are merged based on the merging information of the sub-grids.

[0078] (4) Comparison grid;

[0079] In the process of comparing grids, grid adjustment information is obtained by comparing the difference between the original grid and the reconstructed grid, and the reconstructed grid is adjusted according to the information at the decoding end, so as to obtain a decoded grid that is exactly the same as the original grid, thereby realizing lossless compression of a three-dimensional grid containing multiple sub-grids.

[0080] There are two main types of differences between the reconstructed mesh and the original mesh obtained at the encoding end: one is the difference in the number of vertices, and the other is the difference in connection relationships.

[0081] First, compare the difference in the number of vertices. Taking geometric coordinates as an example, traverse the geometric coordinates of each vertex in the mesh, and compare the number of repetitions of the geometric coordinates in the reconstructed mesh with the number of repetitions in the original mesh. If the number of repetitions of a certain geometric coordinate in the original mesh is greater than the number of repetitions of the coordinate in the reconstructed mesh, add new duplicate points in the reconstructed mesh, and write the vertex index to be repeated into the added vertex index list. If the number of repetitions of a certain geometric coordinate in the original mesh is less than the number of repetitions of the coordinate in the reconstructed mesh, delete the redundant duplicate points in the reconstructed mesh, and write the vertex index to be deleted into the deleted vertex index list. Here, there are many ways to implement the operation of deleting vertices. You can delete the vertices immediately, or you can first record the vertices to be deleted, and then after comparing the differences in the connection relationships, uniformly delete all the vertices to be deleted. The specific implementation method is not emphasized. Therefore, by comparing the difference in the number of vertices, the added vertex index information and the deleted vertex index information are obtained.

[0082] Secondly, compare the difference in connection relationship. After the processing of the above-mentioned encoding and decoding module, the number of facets in the reconstructed mesh and the original mesh is the same, but the vertex index in each facet may be different. First, according to the traversal order of the facets when encoding the connection relationship, the corresponding relationship between the facets of the reconstructed mesh and the facets of the original mesh is established. Then, the matching relationship of the vertices between the reconstructed mesh and the original mesh is established: for vertices without geometric duplicate points, the matching relationship between the vertex between the reconstructed mesh and the original mesh can be established according to the three-dimensional coordinates; for vertices with geometric duplicate points, the facets connected to each duplicate point in the original mesh and the reconstructed mesh are compared to determine whether they correspond. If the facets connected to a duplicate point A in the original mesh and the facets connected to a duplicate point B in the reconstructed mesh are both corresponding, then point A and point B are matched; if a duplicate point A in the original mesh does not match all the duplicate points in the reconstructed mesh, then a certain rule (the rule for selecting vertices is not emphasized here) specifies that an unmatched duplicate point in the reconstructed mesh is matched with point A. After the step of comparing the difference in the number of geometric vertices, the number of duplicate points corresponding to each geometric coordinate in the reconstructed mesh has been made equal to the number of duplicate points corresponding to the original mesh, so according to the above method, each vertex in the reconstructed mesh can be matched to a vertex in the original mesh. After establishing the corresponding relationship between the patches and the matching relationship between the original mesh and the reconstructed mesh, compare whether there is a difference between the index of the vertex in each patch in the original mesh and the index of the matching vertex in the corresponding patch in the reconstructed mesh. If there is a difference, the connection relationship modification information is obtained, and the connection relationship modification information includes the patch index to be modified, the position of the vertex index to be modified in the patch, and the modified vertex index value.

[0083] In addition, for attribute information such as texture coordinates and normal vectors, the above process can be performed once respectively to obtain attribute vertices and connection relationship adjustment information. It is also possible to judge and process the attribute vertices corresponding to the geometric vertices in the above process of executing geometric coordinates to obtain attribute vertices and connection relationship adjustment information.

[0084] After comparing the meshes, the added vertex index information, deleted vertex index information and connection relationship adjustment information are obtained, and each list is entropy encoded to obtain the mesh adjustment information sub-stream. Here, an identifier can be encoded to indicate whether the mesh needs to be adjusted, and its specific representation method is not limited. For example, the encoding flag can be used to explicitly indicate or the mesh adjustment information size can be used to implicitly indicate whether the mesh needs to be adjusted.

[0085] It should be noted that the grid adjustment information can also be completed in the sub-grid division module. In this case, there may be no comparison grid module.

[0086] For the second encoding method, which is based on the implementation of the indication information of merging repeated information, please see the following details.

[0087] In an optional embodiment of the present invention, see Figure 8 , Figure 8 It is another flow chart of the three-dimensional mesh encoding method provided by an embodiment of the present invention, wherein the sub-mesh merging information includes indication information for merging repeated information, and the indication information for merging repeated information includes repeated division identification information of vertices on the sub-mesh whose repeated information needs to be merged and corresponding relationship information of the repeatedly divided vertices that need to be merged.

[0088] For more details, please see Figure 8 The three-dimensional mesh encoding method provided in this embodiment is implemented based on the strategy of indicating information for merging repeated information. At the encoding end, based on the mapping relationship between vertices obtained before and after the sub-mesh division and the indicating information for merging repeated information between sub-meshes, the corresponding relationship of vertices on the boundary between sub-meshes that were originally connected after the sub-mesh division can be obtained, thereby identifying the repeatedly divided points that need to be merged between sub-meshes and their corresponding relationship between sub-meshes, and encoding the information according to the coding order of the vertices. At the same time, the indicating information for merging repeated information between sub-meshes is encoded. At the decoding end, based on the indicating information for merging repeated information decoded (including the representation information of the repeatedly divided points that need to be merged between sub-meshes, the corresponding relationship information between the repeatedly divided points that need to be merged between sub-meshes, and the indicating information for merging repeated information between sub-meshes), the repeatedly divided points between sub-meshes that need to be merged are merged, and at the same time, the repeated points in the original mesh existing on the boundary that is not repeatedly divided are retained, and the sub-meshes that do not need to merge the boundary are directly added to the reconstructed mesh. Through the above operations, the final decoded mesh can be obtained, ensuring lossless encoding and decoding of a three-dimensional mesh containing multiple sub-meshes.

[0089] The coding framework of this strategy is as follows Figure 8As shown, at the encoding end, first, the duplicate points existing in the original mesh are judged, and the duplicate point information in the original mesh is recorded. The duplicate point information can be a mapping relationship between the geometric coordinates of the duplicate points and the vertex index in the original mesh, and its specific representation method is not limited here. Then, sub-mesh division is performed, for example, a division method based on a bounding box can be used, and the sub-mesh division method is not limited here. Then, the mapping relationship between the vertex indexes between the sub-mesh and the original mesh that can be obtained during the sub-mesh division process and the indication information of merging duplicate information between sub-meshes are used to obtain the corresponding relationship between the vertex indexes that are repeatedly divided between the originally connected sub-meshes. Among them, the indication information of merging duplicate information between sub-meshes can include two types of information: one is identification information indicating whether the sub-mesh needs to perform a merge duplicate information operation, and the other is indication information indicating which sub-mesh it needs to merge with for the sub-mesh that needs to perform a merge duplicate information operation. Then, based on the correspondence between the repeatedly divided vertex indices and the repeated point information in the original mesh, the points in the sub-mesh that need to merge the repeated information can be determined. For the sub-mesh that needs to perform the operation of merging repeated information, the repeatedly divided points that need to be merged and the correspondence between their vertex indices between sub-meshes are recorded, and the indication information for merging the repeated information (including the repeated division identifier of the vertex, the correspondence information between the repeatedly divided vertices that need to be merged, and the indication information for merging the repeated information between sub-meshes) is encoded. The sub-mesh is compressed using a three-dimensional mesh encoding and decoding method such as that based on Edgebreaker. The compression method of the sub-mesh is not limited here. The texture map is compressed using a video codec to obtain a texture map bitstream.

[0090] Please refer to the specific embodiments below for a detailed description of the encoding end.

[0091] In this embodiment, based on the strategy of merging the indication information of repeated information, the three-dimensional grid lossless coding framework including multiple sub-grids mainly includes: sub-grid division, input grid repeated point judgment, repeated division information judgment, sub-grid compression and encoding of indication information for merging repeated information, etc. Specifically:

[0092] (1) Subgrid division;

[0093] In this embodiment, the sub-grid division method can use various methods such as division based on bounding box, and the specific division method is not limited here. The following description is made by taking the division method based on bounding box as an example.

[0094] First, based on the geometric coordinates of the input original mesh, the bounding box range in the three-dimensional space where it is located can be obtained; then, it is divided equally based on the coordinate range of the x-axis, y-axis or z-axis. The number of equal divisions can be set in advance or set according to the number of sub-meshes required by the input. After the equal division is completed, the bounding box range of each sub-mesh can be obtained; again, for each bounding box range, all faces in the original mesh are traversed, and the vertices in the input mesh that meet the bounding box range are divided into the current sub-mesh in units of faces. After traversing each divided bounding box, the final divided sub-mesh is obtained, and in this process, the mapping relationship between the sub-mesh and the original mesh vertex index is retained, and the indication information of merging duplicate information between sub-meshes is obtained.

[0095] (2) Input grid duplicate point judgment;

[0096] Taking the mapping relationship between geometric coordinates and vertex indices as an example, all points in the original mesh are traversed, and the mapping relationship between the geometric coordinates and vertex indices of the original mesh is constructed in the process.

[0097] (3) Repeated division of information judgment;

[0098] The specific representation method of the repeated point information in the original mesh is not limited here. The following is an example of the mapping relationship between the geometric coordinates of the repeated points in the original mesh and the vertex index. For the corresponding relationship information between the current sub-mesh and the merged mesh of the vertices to be merged, it can be the repeated point group index information, that is, the points with the same repeated point group index are the repeatedly divided points that need to be merged between the current sub-mesh and the merged mesh whose repeated information needs to be merged. The specific representation method is not limited here. The following is an example of the repeated point group index.

[0099] In this process, you can obtain all the information of the repeatedly divided vertices that need to be merged on the sub-mesh, or you can only obtain the information of the repeatedly divided vertices that need to be merged in special cases; the special case includes the case where the repeated points in the original mesh exist on the sub-mesh boundary. The following is a specific description of obtaining the information of all the repeatedly divided vertices that need to be merged on the sub-mesh as an example. This process can use the merging process of the sub-mesh to obtain the indication information of merging the repeated information. For the merging process, please refer to Figure 7 .

[0100] First, based on the first type of information in the indication information for merging duplicate information between sub-meshes, the merged mesh is initialized to any sub-mesh for which the first type of information indicates that the operation of merging duplicate information does not need to be performed, the index information of the merged sub-meshes in the current merged mesh is recorded, and a mapping relationship list between the geometric coordinates and the vertex index in the current merged mesh is constructed, and a mapping relationship list between the vertex index of the current merged mesh and the original mesh is constructed based on the mapping relationship list between the vertex index of the sub-mesh and the original mesh. As for the method of initializing the selection of sub-meshes, the information of initializing the sub-meshes when merging sub-meshes can be notified to the decoding end by transmitting indication information, or the corresponding initial sub-mesh selection method can be directly used by default at the encoding and decoding end, for example, the first traversed sub-mesh for which the first type of information indicates that the operation of merging duplicate information does not need to be performed is selected as the initial sub-mesh. Then, the sub-grids indicated by the first category of information as needing to perform a merge of duplicate information are traversed, and based on the second category of information in the indication information of merging duplicate information between sub-grids carried by the sub-grids and the index information of the sub-grids that have been merged in the current merged grid, it is determined whether there is a sub-grid that needs to be merged with the current merged grid; if not, the next sub-grid that is traversed and indicated by the first category of information as not needing to perform an operation of merging duplicate information is directly added to the updated reconstructed grid of the current merged grid, and the index information of the sub-grids that have been merged in the current merged grid is updated; if it exists, the operation of merging duplicate information is performed, and the index information of the sub-grids that have been merged in the current merged grid is updated, and then the sub-grids that are still present and indicated by the first category of information as needing to perform an operation of merging duplicate information are continued to be traversed, and the above-mentioned judgment process is repeated.

[0101] When you need to merge duplicate information on a sub-grid or merge duplicate information between a sub-grid and the current merged grid, refer to Fig. 9 , Fig. 9It is another flow chart of the three-dimensional mesh coding method provided by an embodiment of the present invention, traversing all points in the current sub-mesh, and judging whether the geometric coordinates of the current point have been traversed through the mapping relationship list between the geometric coordinates and the vertex index in the current merged mesh. If the geometric coordinates of the current point have not been traversed, it is not a repeatedly divided point that needs to be merged when merging the mesh, the repeated division mark of the point is set to 0, it is added to the merged mesh, and its index in the current sub-mesh triangle is updated to its index in the merged mesh to update the connection relationship; if the geometric coordinates of the current point have been traversed, first judge whether it is a repeated point in the original mesh according to the repeated point information of the original mesh. If it is not a repeated point in the original mesh, the point is a repeatedly divided point that needs to be merged, then the repeated division mark of the point in the current sub-mesh is set to 1, and according to the mapping relationship list between the geometric coordinates and the vertex index in the current merged mesh, the corresponding points with the current geometric coordinates in the current sub-mesh and the current merged mesh are added to the same repeated point group, that is, the same repeated point group index is set for them.

[0102] At the same time, the geometric coordinates of the point in the current sub-mesh are deleted, and its index in the triangle is updated to the vertex index in the corresponding merged mesh to update the connection relationship; if it is a repeated point in the original mesh, then according to the mapping relationship list between the vertex index of the current sub-mesh and the original mesh, the mapping relationship list between the vertex index of the current merged mesh and the original mesh, and the mapping relationship list between the geometric coordinates and the vertex index in the current merged mesh, it can be determined whether the corresponding point with the current geometric coordinates in the current merged mesh and the current sub-mesh is a repeatedly divided point. If it is not a repeatedly divided point, it is not a point that needs to merge repeated information when merging meshes, and the repeated division flag of the current point is set to 0, and it is added to the reconstructed mesh, and its index in the current sub-mesh triangle is updated to its index in the current merged mesh to update the connection relationship; if it is a repeatedly divided point, the repeated division flag of the point in the current sub-mesh is set to 1, and according to the mapping relationship list between the geometric coordinates and the vertex index in the current merged mesh, the corresponding points with the current geometric coordinates in the current sub-mesh and the current merged mesh are added to the same repeated point group, that is, the same repeated point group index is set for them. At the same time, delete the geometric coordinates of the point in the current sub-mesh, and update its index in the triangle to the corresponding vertex index in the current merged mesh to update the connection relationship. Finally, update the mapping relationship list between the geometric coordinates and the vertex index in the merged mesh and the mapping relationship list between the merged mesh and the vertex index of the original mesh. After traversing all the vertices of the current sub-mesh, add the updated triangles in the current sub-mesh to the merged mesh to obtain the updated merged mesh. For the above deletion operation, you can delete the vertices immediately, or you can first record the information of the vertices to be deleted. After all the vertices of the current sub-mesh have been traversed and the connection relationship has been updated, delete all the vertices that need to be deleted. The specific implementation method is not emphasized here.

[0103] The above process is iterated until all sub-meshes are traversed, and finally the merged information of the vertices on the sub-mesh on which the operation of merging duplicate information needs to be performed can be obtained, that is, the first type of information in the indication information for merging duplicate information between sub-meshes indicates the duplicate division identifier of the vertices in the sub-mesh on which the operation of merging duplicate information needs to be performed and the duplicate point group index information of the duplicated divided vertices that need to be merged.

[0104] For attribute information such as texture coordinates and normal vectors, the above process can be performed once respectively, or in the process of executing the geometric coordinates, the attribute vertices corresponding to the geometric vertices can be judged and processed accordingly to obtain indication information of attribute merging duplicate information.

[0105] (4) Subgrid coding;

[0106] In the sub-grid encoding process, each sub-grid is encoded independently. The encoding of each sub-grid is mainly divided into connection relationship encoding, geometric information encoding and attribute information encoding. The Edgebreaker-based method can be used to encode each sub-grid, and the specific encoding method is not emphasized here. The following is a specific description of the Edgebreaker-based encoding method.

[0107] Please continue to see Figure 6 First, the non-manifold structure in the sub-grid is split to obtain a manifold mesh. In this process, the duplicate point information generated by the splitting of the non-manifold is recorded, and the non-manifold structure information is encoded in a geometric coding order and / or an attribute coding order, such as an entropy coding method. The encoding method of the non-manifold structure information is not limited here; then, the connection relationship, geometric information and attribute information of the manifold mesh are encoded. For the connection relationship encoding, the Edgebreaker method can be used to encode the connection relationship of the three-dimensional mesh. The connection relationship of the mesh is represented by establishing a CornerTable, and the CornerTable is used to traverse all triangles in the mesh to generate an Edgebreaker CLERS mode string. The CLERS mode string is encoded using entropy coding to obtain a connection information code stream; the geometric information can be encoded according to the connection relationship encoding order using a variety of methods such as a difference prediction coding algorithm, a parallelogram prediction coding algorithm, etc. to obtain a geometric information code stream. The specific encoding method is not emphasized here. Three-dimensional mesh attribute information generally includes texture coordinates, normal vectors, etc. Taking texture coordinates as an example, the encoded texture coordinates can be encoded in the order of connection relationship using methods such as difference prediction coding, parallelogram prediction coding, and similar triangle prediction coding to obtain an attribute information code stream. The specific encoding method is not emphasized here.

[0108] (5) Indicative information encoding for merging repeated information;

[0109] The encoding method of the indication information of merging repeated information may be entropy encoding or the like. The specific encoding method is not emphasized here. The indication information of merging repeated information is encoded according to the encoding order of the vertices to obtain the indication information code stream of merging repeated information.

[0110] Based on the same inventive concept, see Fig.10 , Fig.10 is a flow chart of a three-dimensional mesh decoding method provided by an embodiment of the present invention. The present invention also provides a three-dimensional mesh decoding method, including:

[0111] Decode the target code stream to obtain multiple sub-grids of the original grid and sub-grid merging information;

[0112] According to the sub-grid merging information, the sub-grids that need to be merged are merged to obtain a decoded grid.

[0113] For more details, please see Fig.10 The present invention provides a 3D mesh decoding method. At the decoding end, first, the target bitstream is decoded, and the sub-grids are decoded using a method such as Edgebreaker to obtain a reconstructed sub-grid; then, according to the sub-grid merging information, the sub-grids are merged to obtain a decoded mesh; in addition, the texture map bitstream is decoded using a video decoder to obtain a decoded texture map. Through the above-mentioned framework, lossless 3D mesh encoding and decoding containing multiple sub-grids can be achieved.

[0114] Please continue to see Fig.10 , for the target code stream obtained at the encoding end, it is decoded to obtain the reconstructed sub-grid and sub-grid merging information respectively, and according to the sub-grid merging information, the reconstructed sub-grids are merged to obtain the reconstructed grid, so as to achieve lossless restoration of the original grid.

[0115] In an optional embodiment of the present invention, the sub-grid merging information includes sub-grids indicating that duplicate information needs to be merged with each other.

[0116] Specifically, in this embodiment, the decoded sub-grid merging information is obtained at the decoding end, and the sub-grid merging information includes sub-grids indicating the need to merge repeated information with each other. Based on the sub-grid merging information, it is determined which sub-grids need to be merged with each other, providing instructions for the subsequent merging of sub-grids.

[0117] In an optional embodiment of the present invention, the sub-grid merging information includes indicating a merging method of the sub-grids.

[0118] Specifically, in this embodiment, decoded sub-grid merging information is obtained at the decoding end, and the sub-grid merging information includes an indication of a merging method of sub-grids. According to the sub-grid merging information, it is determined which method is used for merging sub-grids, providing a method for subsequent merging of sub-grids.

[0119] In an optional embodiment of the present invention, see Fig.11 , Fig.11 It is another flow chart of the three-dimensional mesh decoding method provided by an embodiment of the present invention, which determines whether the sub-mesh merging information includes mesh adjustment information; wherein the mesh adjustment information includes at least one of adding vertex index information, deleting vertex index information and connection relationship adjustment information.

[0120] Specifically, in this embodiment, please continue to refer to Fig.11At the decoding end, based on the sub-grid merging information, it is determined whether the sub-grid merging information includes grid adjustment information. If the grid adjustment information is included, the grid adjustment information is used to process the reconstructed sub-grid, and the reconstructed grid is obtained losslessly according to this method.

[0121] In an optional embodiment of the present invention, the method for merging sub-grids includes:

[0122] According to the sub-grid merging information, the reconstructed sub-grids are merged, and the duplicate information between the sub-grids is removed to obtain the reconstructed grid.

[0123] In an optional embodiment of the present invention, the sub-grid merging information includes grid adjustment information, and the reconstructed grid is adjusted according to the grid adjustment information to obtain a decoded grid.

[0124] For more details, please see Fig.11 In this embodiment, according to the encoding method of the encoding end, the decoding end is determined to use a corresponding method to reconstruct the grid. First, each code stream is decoded, and the sub-grid is decoded using a method such as Edgebreaker to obtain a reconstructed sub-grid; then, according to the sub-grid merging information obtained by decoding, the reconstructed sub-grid is merged to obtain a reconstructed grid; finally, the reconstructed grid is adjusted according to the mesh adjustment information obtained by decoding to obtain the final decoded grid. In addition, a video decoder is used to decode the texture map code stream to obtain a decoded texture map. Through the above process, lossless three-dimensional mesh encoding and decoding containing multiple sub-grids can be achieved.

[0125] Please refer to the specific embodiments below for a detailed description of the decoding end.

[0126] In this embodiment, based on the implementation strategy of the grid adjustment information, the three-dimensional grid lossless decoding framework including multiple sub-grids mainly includes: sub-grid decoding, grid adjustment information decoding, sub-grid merging, and adjusting and reconstructing the grid, etc. Specifically:

[0127] (1) Subgrid decoding;

[0128] In the process of sub-grid decoding, each sub-grid is decoded independently. The decoding of each sub-grid is mainly divided into connection relationship decoding, geometric information decoding and attribute information decoding. The decoding method corresponding to the encoding method used by the encoding end is used to decode each information. The following is a specific description of the decoding method based on Edgebreaker as an example.

[0129] See also Fig.12 , Fig.12It is a flow chart of the Edgebreaker three-dimensional mesh decoding method provided by an embodiment of the present invention. First, the connection relationship, geometric information, attribute information and non-manifold structure information of the manifold mesh are decoded. For connection relationship decoding, the pattern string is first decoded to obtain the pattern string, and the pattern string is traversed in a certain order (forward or reverse order), and the connection relationship is reconstructed according to the corresponding pattern in the string. In addition, the traversal order of the vertices is output to the geometric information and attribute information decoding module. The geometric information is decoded according to the decoding order of the connection relationship using a decoding method corresponding to the encoding end to obtain the geometric information. The three-dimensional mesh attributes are decoded according to the decoding order of the connection relationship using a decoding method corresponding to the encoding end to decode the three-dimensional mesh attribute information. The non-manifold structure information decoding uses a decoding method corresponding to the encoding end to decode the non-manifold structure information. According to the decoded connection relationship, geometric information and attribute information, the manifold mesh can be directly reconstructed. Then, according to the decoded non-manifold structure information, the reconstructed sub-mesh can be obtained.

[0130] (2) Subgrid merging;

[0131] Subgrids are merged according to the decoded subgrid merging information. According to the method of merging subgrids at the decoding end, for example, the subgrids can be directly merged by merging repeated information between subgrids, or the subgrids can be merged according to the optional merged repeated information between subgrids, or the information of each subgrid can be directly added to a blank grid to obtain a reconstructed grid. In the implementation process, according to the subgrid merging information, the corresponding merging method is selected; the specific merging method can refer to the encoding terminal grid merging method.

[0132] (3) Adjust the reconstruction grid;

[0133] Decode the mesh adjustment flag or decode the mesh adjustment information to determine whether the decoded mesh needs to be adjusted. If adjustment is required, adjust the reconstructed mesh according to at least one of the added vertex index information, deleted vertex index information, and connection relationship modification information obtained by decoding in the mesh adjustment information sub-code stream. Specifically, first, create a duplicate point for each vertex in the added vertex index list; second, use the vertex deletion method corresponding to the encoding end to delete the vertices in the deleted vertex index list. The vertices can be deleted immediately, or the vertices to be deleted can be recorded first; then, after adjusting the connection relationship, all vertices to be deleted are deleted uniformly. Here, the vertex deletion method corresponding to the encoding end can be used, and the specific implementation method is not emphasized; then, adjust the connection relationship of the mesh according to the connection relationship modification information list. Read the face index to be modified and the position of the vertex index to be modified in the face in turn, and modify the vertex index at the position. Finally, the decoded mesh is obtained to achieve lossless encoding and decoding of a three-dimensional mesh containing multiple sub-meshes.

[0134] In an optional embodiment of the present invention, see Fig.13 , Fig.13 It is another flow chart of the three-dimensional mesh decoding method provided by an embodiment of the present invention, which determines whether the merged information in the sub-mesh includes indication information for merging repeated information; wherein the indication information for merging repeated information includes repeated division identification information of vertices on the sub-mesh whose repeated information needs to be merged and corresponding relationship information of the repeatedly divided vertices that need to be merged.

[0135] In an optional embodiment of the present invention, the sub-grid merging information includes indication information for merging duplicate information. According to the merged duplicate information, the repeatedly divided points that need to be merged are merged, and the duplicate points in the original grid that exist on the boundaries that are not repeatedly divided are retained. For the sub-grids that do not need to merge duplicate information, they are directly added to the reconstructed grid to obtain a decoded grid.

[0136] For more details, please see Fig.13 The three-dimensional mesh decoding method provided in this embodiment is implemented based on the strategy of merging the indication information of repeated information. At the decoding end, first, each code stream is decoded, and the sub-mesh code stream is decoded using a three-dimensional mesh decoding method based on Edgebreaker to obtain a reconstructed sub-mesh; then, the sub-mesh is merged according to the indication information of merging between the decoded sub-meshes and the indication information of merging repeated information to obtain the final decoded mesh. In addition, a texture map is decoded using a video decoder to obtain a texture map. Through the above-mentioned framework, lossless encoding and decoding of a three-dimensional mesh containing multiple sub-meshes can be achieved.

[0137] Please refer to the specific embodiments below for a detailed description of the decoding end.

[0138] In this embodiment, based on the implementation strategy of the indication information for merging repeated information, the three-dimensional grid lossless decoding framework including multiple subgrids mainly includes: subgrid decoding, decoding of the indication information for merging repeated information, and subgrid merging, etc. Specifically:

[0139] (1) Subgrid decoding;

[0140] In the process of sub-grid decoding, each sub-grid is decoded independently. The decoding of each sub-grid is mainly divided into connection relationship decoding, geometric information decoding and attribute information decoding. The decoding method corresponding to the encoding method used by the encoding end is used to decode each information. The following is a specific description of the decoding method based on Edgebreaker as an example.

[0141] Please continue to see Fig.12First, the connection relationship, geometric information, attribute information and non-manifold structure information of the manifold mesh are decoded. For the connection relationship decoding, the pattern string is first decoded, and the pattern string is traversed in a certain order (forward or reverse order), and the connection relationship is reconstructed according to the corresponding pattern in the string. In addition, the traversal order of the vertices is output to the geometric information and attribute information decoding module. The geometric information is decoded according to the decoding order of the connection relationship using a decoding method corresponding to the encoding end to obtain the geometric information. The three-dimensional mesh attributes are decoded according to the decoding order of the connection relationship using a decoding method corresponding to the encoding end to decode the three-dimensional mesh attribute information. The non-manifold structure information decoding uses a decoding method corresponding to the encoding end to decode the non-manifold structure information. According to the decoded connection relationship, geometric information and attribute information, the manifold mesh can be directly reconstructed. Then, according to the decoded non-manifold structure information, the reconstructed sub-mesh can be obtained.

[0142] (2) decoding of indication information combining repeated information;

[0143] The repeated division identifiers of the vertices and the corresponding relationship information of the repeatedly divided points to be merged are decoded in the vertex decoding order, and the indication information of merging the repeated information between the sub-grids is obtained by decoding at the same time.

[0144] (3) Subgrid merging;

[0145] Traverse and reconstruct the subgrid according to the traversal method corresponding to the judgment of the repeated division information by the encoding end. First, initialize the output decoding grid according to the indication information of the initial subgrid, or use the default initial subgrid selection method to initialize the decoding grid, and record the subgrid index information that has been merged in the current decoding grid. Then, traverse the subgrids indicated by the first type of information in the indication information of merging repeated information between subgrids, and judge whether there is a subgrid that needs to be merged with the current decoding grid according to the second type of information in the indication information of merging repeated information between subgrids and the subgrid index information that has been merged in the current decoding grid. If it does not exist, the next subgrid that is traversed and indicates that the operation of merging repeated information does not need to be performed is directly added to the updated decoding grid of the current decoding grid, and the subgrid index information that has been merged in the current decoding grid is updated; if it exists, the operation of merging repeated information is performed, and the subgrid index information that has been merged in the current decoding grid is updated, and then the subgrids that still exist and indicate that the operation of merging repeated information needs to be performed continue to traverse, and repeat the above judgment process.

[0146] When it is necessary to merge duplicate information on the sub-mesh and merge duplicate information between the sub-mesh and the current merged mesh, traverse all vertices in the current sub-mesh and determine whether it is a vertex to be deleted based on the vertex duplicate division identifier obtained by decoding. If it is not a vertex to be deleted, directly add the current point to the decoded mesh, and update its index in the triangle in the current sub-mesh to the index after it is added to the decoded mesh; if it is a vertex to be deleted, the corresponding relationship between the current point and the vertex in the current decoded mesh can be obtained based on the corresponding relationship information of the repeatedly divided points to be merged obtained by decoding. Delete the current vertex and update its index in the current sub-mesh triangle to the corresponding vertex index in the current decoded mesh. Until all points in the current sub-mesh are traversed, add the updated triangle of the current sub-mesh to the decoded mesh to obtain the updated decoded mesh. For the above deletion operation, the vertex can be deleted immediately, or the information of the vertex to be deleted can be recorded first, and all vertices to be deleted are deleted uniformly after all vertices of the current sub-mesh are traversed and the connection relationship is updated. The specific implementation method is not emphasized here.

[0147] The above process is iterated until all reconstructed sub-grids are traversed to obtain the final decoded grid.

[0148] For attribute information such as texture coordinates and normal vectors, the above process can be performed once respectively, or in the process of executing the geometric coordinates, the attribute vertices corresponding to the geometric vertices can be judged and processed accordingly to perform sub-mesh merging operations.

[0149] In summary, the present invention provides a three-dimensional mesh encoding and decoding method, including two methods, both of which can achieve lossless merging of sub-grids. Different strategies can be selected for obtaining sub-grid merging information at the encoding end and merging sub-grids at the decoding end; the first method is implemented based on mesh adjustment information. First, the encoded sub-grids are decoded and reconstructed at the encoding end to obtain reconstructed sub-grids; then, a merging method is selected according to the sub-grid merging information to merge the reconstructed sub-grids. The merging method can be to merge the reconstructed sub-grids by merging the repeated information between the sub-grids, or to directly add the information of the reconstructed sub-grids to a blank grid to obtain the reconstructed grid. The specific merging method is not limited here; again, the reconstructed grid is compared with the original grid. If there is a situation where the reconstructed grid is different from the original grid due to the merging, the grid adjustment information can be obtained and encoded during the comparison process; at the decoding end, first, a merging method is selected according to the sub-grid merging information, and the reconstructed sub-grids are merged to obtain a reconstructed grid; then, the merged grid is adjusted according to the grid adjustment information to obtain the final decoded grid. The second method is implemented based on the indication information of merging repeated information. The corresponding vertex mapping relationship information can be obtained before and after the sub-mesh division at the encoding end. Based on this information and the indication information of merging repeated information between sub-meshes, the repeated division identification information of the vertices on the sub-mesh that needs to merge repeated information and the corresponding relationship information of the repeatedly divided vertices that need to be merged are obtained at the encoding end, and the information is encoded; at the same time, the indication information of merging repeated information between sub-meshes is encoded, and the information can include two types of information: one is the identification information indicating whether the sub-mesh needs to perform the merge repeated information operation, and the other is the indication information indicating which sub-mesh to merge with for the sub-mesh that needs to perform the merge repeated information operation. At the decoding end, according to the indication information of merging repeated information (including the repeated division identification information of the repeatedly divided vertices between the sub-meshes that need to merge repeated information, the corresponding relationship information of the repeatedly divided vertices that need to be merged, and the indication information of merging repeated information between sub-meshes), the sub-meshes are merged, the repeatedly divided points that need to be merged are merged, the repeated points in the original mesh existing on the boundary that is not repeatedly divided are retained, and the sub-meshes that do not need to merge repeated information are directly added to the reconstructed mesh to obtain the final decoded mesh.

[0150] The above two methods can be used separately. For example, when the bitstream is not marked, the decoder adopts one of the schemes by default. The codec can also support both schemes at the same time, dynamically indicating which scheme to use through the coding merge scheme identifier, or it can be used as one or two of the coding and decoding merge schemes at the same time as other schemes.

[0151] Based on the same inventive concept, see Fig.14 , Fig.14is a schematic diagram of a three-dimensional grid coding device provided by an embodiment of the present invention. The present invention further provides a three-dimensional grid coding device, which is applied to a three-dimensional grid coding method provided by the above embodiment. Please refer to the above three-dimensional grid coding method, which will not be described in detail in this embodiment. The three-dimensional grid coding device includes:

[0152] A first acquisition unit 201 is used to acquire sub-grid merging information according to multiple sub-grids of the original grid;

[0153] The second acquisition unit 202 is used to encode the sub-grid and the sub-grid merging information to obtain a target code stream.

[0154] Based on the same inventive concept, see Fig.15 , Fig.15 is a schematic diagram of a three-dimensional grid decoding device provided by an embodiment of the present invention. The present invention further provides a three-dimensional grid decoding device, which is applied to a three-dimensional grid decoding method provided by the above embodiment. Please refer to the above three-dimensional grid decoding method, which will not be described in detail in this embodiment. The three-dimensional grid decoding device includes:

[0155] A first acquisition unit 301 is used to decode a target code stream and acquire multiple sub-grids and sub-grid merging information of an original grid;

[0156] The second acquisition unit 302 is used to merge the sub-grids that need to be merged according to the sub-grid merging information to obtain a decoding grid.

[0157] The encoding device and decoding device in the embodiment of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or can be other devices other than a terminal. Other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in this embodiment.

[0158] The encoding device provided in this embodiment can achieve Figure 1 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0159] The decoding device provided in this embodiment can achieve Fig.10 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0160] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant is intended to cover non-exclusive inclusion, so that the article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the article or device including the elements. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The orientation or position relationship indicated by "up", "down", "left", "right", etc. is based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

[0161] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification.

[0162] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A three-dimensional grid coding method, characterized in that: include: According to multiple sub-grids of the original grid, obtain sub-grid merging information; The sub-grid and the sub-grid merging information are encoded to obtain a target code stream.

2. The three-dimensional grid coding method according to claim 1, characterized in that: The sub-grid merging information includes grid adjustment information, and the grid adjustment information includes at least one of adding vertex index information, deleting vertex index information, and connection relationship adjustment information.

3. The three-dimensional grid coding method according to claim 2, characterized in that: The grid adjustment information is obtained by merging and reconstructing sub-grids, wherein the process of merging and reconstructing sub-grids includes: According to the sub-grid merging information, the reconstructed sub-grids are merged, and the duplicate information between the sub-grids is removed to obtain the reconstructed grid.

4. The three-dimensional grid coding method according to claim 3, characterized in that: The process of obtaining the grid adjustment information includes: The reconstructed grid is compared with the original grid, and grid adjustment information is obtained according to the difference between the reconstructed grid and the original grid.

5. The three-dimensional grid coding method according to claim 1, characterized in that: The sub-grid merging information includes indication information for merging repeated information, and the indication information for merging repeated information includes repeated division identification information of vertices on the sub-grids where repeated information needs to be merged and corresponding relationship information of repeatedly divided vertices that need to be merged.

6. A three-dimensional grid decoding method, characterized in that: include: Decode the target code stream to obtain multiple sub-grids of the original grid and sub-grid merging information; According to the sub-grid merging information, the sub-grids that need to be merged are merged to obtain a decoded grid.

7. The three-dimensional grid decoding method according to claim 6, characterized in that: The sub-grid merging information includes sub-grids indicating that duplicate information needs to be merged with each other.

8. The three-dimensional grid decoding method according to claim 7, characterized in that: The sub-grid merging information includes an indication of a merging method of the sub-grids.

9. The three-dimensional grid decoding method according to claim 8, characterized in that: Determine whether the sub-grid merging information includes grid adjustment information; wherein the grid adjustment information includes at least one of adding vertex index information, deleting vertex index information, and connection relationship adjustment information.

10. The three-dimensional grid decoding method according to claim 9, characterized in that: The sub-grid merging method includes: The reconstructed subgrids are merged according to the subgrid merging information, and the duplicate information between the subgrids is removed to obtain a reconstructed grid.

11. The three-dimensional grid decoding method according to claim 10, characterized in that: The sub-grid merging information includes grid adjustment information, and the reconstructed grid is adjusted according to the grid adjustment information to obtain a decoded grid.

12. The three-dimensional grid decoding method according to claim 6, characterized in that: Determine whether the merge information in the sub-grid includes indication information for merging repeated information; wherein the indication information for merging repeated information includes repeated division identification information of vertices on the sub-grid whose repeated information needs to be merged and corresponding relationship information of repeatedly divided vertices that need to be merged.

13. The three-dimensional grid decoding method according to claim 12, characterized in that: The sub-grid merging information includes indication information for merging repeated information. According to the indication information for merging repeated information, the repeatedly divided points that need to be merged are merged, and the repeated points in the original grid that exist on the boundaries that are not repeatedly divided are retained. For the sub-grids that do not need to merge repeated information, they are directly added to the reconstructed grid to obtain a decoded grid.

14. A three-dimensional grid encoding device, characterized in that: include: A first acquisition unit, configured to acquire sub-grid merging information according to multiple sub-grids of the original grid; The second acquisition unit is used to encode the sub-grid and the sub-grid merging information to obtain a target code stream.

15. A three-dimensional grid decoding device, characterized in that: include: A first acquisition unit, used for decoding a target code stream, and acquiring multiple sub-grids of an original grid and sub-grid merging information; The second acquisition unit is used to merge the sub-grids that need to be merged according to the sub-grid merging information to obtain a decoding grid.

Images