Three-dimensional grid coding method, three-dimensional grid decoding method, device and equipment

Abstract

The invention discloses a three-dimensional grid coding method, device and equipment and a three-dimensional grid decoding method, device and equipment, and belongs to the technical field of coding and decoding, the three-dimensional grid coding method in the embodiment of the invention comprises the following steps: performing color space format conversion on a texture map of a first color space format of a three-dimensional grid to obtain a texture map of a second color space format; and encoding the texture image in the second color space format to obtain a first texture image code stream.

Description

Technical Field

[0001] The present application belongs to the field of coding and decoding technology, and specifically relates to a three-dimensional grid coding method, a three-dimensional grid decoding method, a device and equipment. Background Art

[0002] With the rapid development of multimedia technology, 3D models have become a new generation of digital media after audio, images, and videos. 3D mesh and point cloud are two commonly used ways to represent 3D models. Compared with traditional multimedia such as images and videos, 3D mesh models have stronger interactivity and realism, and are widely used.

[0003] In the related art, when encoding a texture map of a three-dimensional grid, the texture map in the BGR444 format is directly encoded, and the flexibility of encoding the texture map of the three-dimensional grid is relatively poor. Summary of the invention

[0004] The embodiments of the present application provide a three-dimensional mesh encoding method, a three-dimensional mesh decoding method, an apparatus and a device, which can solve the problem of poor flexibility in encoding the texture map of the three-dimensional mesh.

[0005] In a first aspect, a three-dimensional grid encoding method is provided, which is performed by an encoding end and includes:

[0006] Performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format;

[0007] The texture image in the second color space format is encoded to obtain a first texture image code stream.

[0008] In a second aspect, a three-dimensional mesh decoding method is provided, which is executed by a decoding end and includes:

[0009] Decoding a first texture image bitstream in a bitstream corresponding to the three-dimensional grid to obtain a texture image in a second color space format;

[0010] Performing color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0011] A mesh reconstruction process is performed based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0012] In a third aspect, a three-dimensional grid encoding device is provided, the device comprising:

[0013] A conversion module, used for performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format;

[0014] The first encoding module is used to encode the texture image in the second color space format to obtain a first texture image code stream.

[0015] In a fourth aspect, a three-dimensional grid decoding device is provided, the device comprising:

[0016] A first decoding module is used to decode a first texture image bitstream in a bitstream corresponding to the three-dimensional grid to obtain a texture image in a second color space format;

[0017] A conversion module, used for performing color space format conversion on the texture image in the second color space format to obtain the texture image in the first color space format;

[0018] A reconstruction module is used to perform mesh reconstruction processing based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0019] In a fifth aspect, an electronic device is provided, which terminal includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect; or, the program or instruction, when executed by the processor, implements the steps of the method described in the second aspect.

[0020] In a sixth aspect, an electronic device is provided, comprising a processor and a communication interface, wherein the processor is used to: convert a texture image in a first color space format of a three-dimensional grid to obtain a texture image in a second color space format; and encode the texture image in the second color space format to obtain a first texture image code stream.

[0021] In the seventh aspect, an electronic device is provided, comprising a processor and a communication interface, wherein the processor is used to: decode a first texture map code stream in a code stream corresponding to a three-dimensional mesh to obtain a texture map in a second color space format; perform color space format conversion on the texture map in the second color space format to obtain a texture map in a first color space format; perform mesh reconstruction based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0022] In an eighth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the three-dimensional mesh encoding method as described in the first aspect are implemented; or, when the program or instruction is executed by a processor, the steps of the three-dimensional mesh decoding method as described in the second aspect are implemented.

[0023] In the ninth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0024] In the tenth aspect, a computer program / program product is provided, wherein the computer program / program product is stored in a non-volatile storage medium, and the program / program product is executed by at least one processor to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0025] In an embodiment of the present application, a texture map in a first color space format of a three-dimensional mesh is converted into a color space format to obtain a texture map in a second color space format; and the texture map in the second color space format is encoded to obtain a first texture map code stream. In this way, by converting the color space format of the texture map of the three-dimensional mesh during the three-dimensional mesh encoding process, it is supported to convert the texture map of the three-dimensional mesh into a texture map in another color space format for encoding, which can improve the flexibility of the texture map encoding of the three-dimensional mesh. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the five modes of Edgebreaker in the related art;

[0027] Figure 2 It is a schematic diagram of the vertex traversal process and operation mode string in Edgebreaker in the related art;

[0028] Figure 3 is a flow chart of a three-dimensional grid encoding method provided in an embodiment of the present application;

[0029] Figure 4 is a flow chart of a three-dimensional grid decoding method provided in an embodiment of the present application;

[0030] Figure 5 It is a schematic diagram of a three-dimensional grid coding framework provided in an embodiment of the present application;

[0031] Figure 6 It is a schematic diagram of a three-dimensional grid decoding framework provided in an embodiment of the present application;

[0032] Figure 7 is a schematic diagram of a manifold grid provided in an embodiment of the present application;

[0033] Figure 8 This is a Corner relationship diagram provided in an embodiment of the present application;

[0034] Fig. 9It is a grid traversal schematic diagram provided in an embodiment of the present application;

[0035] Fig.10 is a three-dimensional grid schematic diagram provided in an embodiment of the present application;

[0036] Fig.11 It is a schematic diagram of UV coordinate prediction based on three-dimensional to two-dimensional projection provided in an embodiment of the present application;

[0037] Fig.12 is a structural schematic diagram of a three-dimensional grid encoding device provided in an embodiment of the present application;

[0038] Fig.13 is a structural schematic diagram of a three-dimensional grid decoding device provided in an embodiment of the present application;

[0039] Fig.14 It is a structural diagram of a communication device provided in an embodiment of the present application;

[0040] Fig.15 It is a structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0042] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0043] The term "indication" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication; an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.

[0044] The encoding and decoding end corresponding to the encoding and decoding method in the embodiment of the present application may be a terminal, which may also be referred to as a terminal device or a user terminal (User Equipment, UE). The terminal may be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer) or a notebook computer, a personal digital assistant (Personal Digital Assistant, PDA), a handheld computer, a netbook, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device (Wearable Device) or a vehicle-mounted device (Vehicle User Equipment, VUE), a pedestrian terminal (Pedestrian User Equipment, PUE) and other terminal side devices, and the wearable device includes: a smart watch, a bracelet, a headset, glasses, etc. It should be noted that the specific type of the terminal is not limited in the embodiment of the present application.

[0045] For ease of understanding, some contents involved in the embodiments of the present application are described below:

[0046] 1. 3D Grid

[0047] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have become a new generation of digital media after audio, images, and videos. Three-dimensional grids and point clouds are two commonly used ways to represent three-dimensional models. Compared with traditional multimedia such as images and videos, three-dimensional grid models have stronger interactivity and realism, making them more and more widely used in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and military.

[0048] With the increasing demand for visual effects of 3D mesh models and the emergence of more mature 3D scanning technologies and 3D modeling software, the data size and complexity of 3D mesh models obtained by 3D scanning equipment or 3D modeling software are also growing rapidly. Therefore, how to efficiently compress 3D mesh data is the key to conveniently transmit, store and process 3D mesh data.

[0049] A 3D mesh often contains three main types of information: topological information, geometric information, and attribute information. Topological information, also known as connectivity information, is used to describe the connection between elements such as vertices and facets in the mesh; geometric information is the 3D coordinates of all vertices in the mesh; attribute information records other information attached to the mesh, such as normal vectors, texture coordinates, and colors. The compression of 3D mesh data is often based on the compression of these three types of information according to their data characteristics. In addition, for 3D meshes with texture maps, the texture maps need to be compressed.

[0050] Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It aims to improve the storage and transmission of 3D graphics and greatly accelerate the encoding, transmission and decoding of 3D data. Draco supports the compression of 3D mesh geometry information, connection information and attribute information. Draco supports lossy mode and near-lossless mode. In addition, the Edgebreaker method used by Draco to encode connection relationships is currently one of the most efficient methods for encoding 3D mesh connection information.

[0051] However, Edgebreaker requires the mesh to be encoded to be a manifold structure. For meshes with non-manifold structures, Draco must split them into manifold structures to encode them correctly. However, Draco does not merge the split structures at the decoding end, which makes the mesh output by the decoding end have more split points than the original mesh input by the encoding end. This will cause Draco to be unable to losslessly encode such meshes with non-manifold structures.

[0052] The Moving Picture Expert Group (MPEG), an international standards organization in the video image field, is developing a new dynamic 3D mesh compression standard, Video-based dynamic mesh coding (V-DMC or VDMC). Currently, when encoding static 3D meshes, it also chooses to adopt an Edgebreaker-based solution and temporarily reuses the Draco codec. At the same time, MPEG is also trying to implement the Edgebreaker-based 3D mesh codec provided by MPEG to achieve compression of 3D mesh geometry information, connection information, and attribute information. Since Edgebreaker requires the mesh to be encoded to be a manifold structure, the currently proposed solution will involve splitting the non-manifold structure 3D mesh into a manifold structure for encoding. Therefore, in lossless mode, the Edgebreaker-based 3D mesh codec implementation provided by MPEG will record and encode the duplicate point information generated by non-manifold splitting, and add an identifier to each vertex of the split manifold mesh to determine whether it is a duplicate point generated by non-manifold splitting. Based on the identifier and the duplicate point information generated by non-manifold splitting, the non-manifold structure of the original 3D mesh is restored at the decoding end to achieve lossless encoding of the 3D mesh.

[0053] The Edgebreaker-based 3D mesh compression tool currently provided by MPEG encodes and stores the connection information, geometric information and attribute information (e.g., UV coordinates) of the 3D mesh separately. UV coordinates are two-dimensional coordinates, with U in the horizontal direction and V in the vertical direction. The core module, i.e., the module for encoding the connection information, uses the Edgebreaker algorithm. Taking the attribute information as UV coordinates as an example, the encoding of the geometric information and UV coordinates adopts conventional compression methods, i.e., quantization, predictive compression (e.g., parallelogram prediction) and entropy coding of the data. Since the tool adopts a connection relationship-driven encoding method, the encoding of the geometric information and UV coordinates will follow the encoding order of the connection information. In this way, the vertex order of the connection relationship encoding is implicitly included in the vertex order of the geometric information to avoid the separate transmission of the vertex order of the connection relationship encoding, thereby saving this part of the bit overhead.

[0054] 2. Edgebreaker

[0055] The Edgebreaker (EB) method is a 3D mesh connection relationship encoding method with good compression performance, easy implementation, and the ability to provide an upper limit for compression ratio. The Edgebreaker method itself only describes the compression method for 3D mesh connection information, and the compression of 3D meshes can only be achieved through geometric information compression and entropy coding.

[0056] Edgebreaker encoding technology can compress triangular meshes that are homeomorphic to the sphere to 2 bits or less per triangle. The encoding algorithm uses five different modes (called C, L, E, R, and S) to access each triangle of the mesh in depth-first order. Each triangle is marked according to the mode it is in, and a CLERS string is generated to obtain a compact representation of the mesh connectivity.

[0057] The five modes of the Edgebreaker method are: Figure 1 As shown. The Edgebreaker method divides the mesh into a traversed part and an untraversed part, and the boundary between the two parts is called the active boundary. In the Edgebreaker encoding process, the triangle to be traversed is accessed through the active edge on the active boundary, and the mode to be used is selected according to the relationship between the active edge and the triangle in which it is located. The other vertex in the triangle where the active edge is located is called the third vertex. If the third vertex is not on the active boundary, the current triangle is marked as C mode. If the third vertex is on the active boundary and is the next vertex of the current active edge vertex in counterclockwise order, the current triangle is marked as R mode. If the third vertex is on the active boundary and is the previous vertex of the current active edge vertex in counterclockwise order, the current triangle is marked as L mode. If the third vertex is on the active boundary and is both the previous vertex of the current active edge vertex and the next vertex of the current active edge in counterclockwise order, the current triangle is marked as E mode. If the third vertex is on the active boundary, but is neither the previous vertex of the current active edge vertex nor the next vertex of the current active edge in counterclockwise order, the current triangle is marked as S mode.

[0058] After marking each triangle, the active boundary is updated and the next active edge is selected according to certain rules. After traversing all triangles, the obtained CLERS string is entropy encoded to obtain higher compression efficiency.

[0059] Figure 2 A schematic diagram of a two-dimensional grid using EB coding is given. According to the coding rule of EB, the final entropy coded mode codeword is CCRRSLCRSERRELCRRRCRRRE.

[0060] Since the five modes of the Edgebreaker method cannot process non-manifold meshes, the mesh must be converted into a manifold mesh before using the Edgebreaker method.

[0061] In the related art, when encoding a texture map of a three-dimensional grid, the texture map in the BGR444 format is directly encoded, resulting in low encoding and decoding efficiency of the three-dimensional grid.

[0062] In the following, in combination with the accompanying drawings, the three-dimensional grid encoding method, three-dimensional grid decoding method, device and equipment provided by the embodiments of the present application are described in detail through some embodiments and their application scenarios.

[0063] See also Figure 3 , Figure 3 is a flowchart of a three-dimensional grid encoding method provided in an embodiment of the present application, which can be applied to an encoding end, such as Figure 3 As shown, the three-dimensional grid encoding method includes the following steps:

[0064] Step 101: Perform color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format.

[0065] Among them, the first color space format can be the original color space format of the texture map of the three-dimensional grid. The first color space format can be an RGB color space format. For example, the first color space format can be BGR444, BGR555, RGB555, or RGB 565, etc.; or, the first color space format can be a YUV color space format; or, the first color space format can be a CMY color space format; or, the first color space format can be a CMYK color space format; or, the first color space format can be an HSV color space format; etc., this embodiment does not limit the first color space format. RGB refers to red, green, and blue. CMY refers to cyan, magenta, and yellow. In YUV, Y represents luminance, and UV represents chrominance and concentration, respectively. CMYK refers to cyan, magenta, yellow, and black. HSV refers to hue-saturation-value.

[0066] In addition, the second color space format may be a color space format after color space format conversion. The second color space format and the first color space format are different color space formats. The second color space format may be a YUV color space format. For example, the second color space format may be YUV444, YUV420, or YUV422, etc.; or, the second color space format may be an RGB color space format; the second color space format may also be an HSV color space format; or the second color space format may also be a CMY color space format; or the second color space format may also be a Lab color space format; etc. This embodiment does not limit the second color space format. In Lab, L is brightness, and a and b are two color channels.

[0067] In one implementation, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different formats of the same color space.

[0068] It should be noted that different color space format conversions can be performed for different application scenarios. To improve encoding and decoding efficiency, the amount of relevant information of the color space of the texture image in the second color space format can be less than the amount of relevant information of the texture image in the first color space format. For example, the texture image in the second color space format is a texture image obtained by removing certain redundant information from the texture image in the first color space format; or, the texture image in the second color space format is a texture image obtained by removing the correlation between color components from the texture image in the first color space format.

[0069] It should be noted that the color space format conversion of the texture image in the first color space format of the three-dimensional grid to obtain the texture image in the second color space format may be a lossless color space format conversion of the texture image in the first color space format of the three-dimensional grid to obtain the texture image in the second color space format.

[0070] Step 102: Encode the texture image in the second color space format to obtain a first texture image code stream.

[0071] In an embodiment of the present application, a texture map in a first color space format of a three-dimensional mesh is converted into a color space format to obtain a texture map in a second color space format; and the texture map in the second color space format is encoded to obtain a first texture map code stream. In this way, by converting the color space format of the texture map of the three-dimensional mesh during the three-dimensional mesh encoding process, it is supported to convert the texture map of the three-dimensional mesh into a texture map in another color space format for encoding, which can improve the flexibility of the texture map encoding of the three-dimensional mesh.

[0072] Optionally, performing color space format conversion on the texture image of the three-dimensional grid in the first color space format to obtain the texture image in the second color space format includes:

[0073] When it is determined to perform color space format conversion on the texture map of the three-dimensional grid, the texture map of the first color space format of the three-dimensional grid is converted into a color space format to obtain a texture map of the second color space format.

[0074] The encoding end may determine whether to perform color space format conversion on the texture map of the three-dimensional mesh. The encoding end may determine whether to perform color space format conversion on the texture map of the three-dimensional mesh based on different encoding requirements. For example, when higher encoding and decoding efficiency is required, it may be determined to perform color space format conversion on the texture map of the three-dimensional mesh.

[0075] In this implementation, when it is determined that the color space format conversion is performed on the texture map of the three-dimensional mesh, the color space format conversion is performed on the texture map of the first color space format of the three-dimensional mesh to obtain the texture map of the second color space format. In this way, by judging whether to perform the color space format conversion on the texture map of the three-dimensional mesh, the encoding end can be supported to selectively convert the color space format of the texture map, thereby improving the flexibility of encoding the texture map during the three-dimensional mesh encoding process.

[0076] Optionally, the method further comprises:

[0077] In the case where it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream.

[0078] The encoding end may determine whether to perform color space format conversion on the texture map of the three-dimensional mesh. The encoding end may determine whether to perform color space format conversion on the texture map of the three-dimensional mesh based on different encoding requirements. For example, in a scenario where the encoding and decoding efficiency is not required, it may be determined not to perform color space format conversion on the texture map of the three-dimensional mesh.

[0079] In this implementation, when it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream. In this way, by judging whether to perform color space format conversion on the texture map of the three-dimensional grid, the encoding end can be supported to selectively not perform color space format conversion on the texture map, which can further improve the flexibility of encoding the texture map during the three-dimensional grid encoding process.

[0080] Optionally, the code stream corresponding to the three-dimensional mesh includes an encoding result of first indication information, where the first indication information is used to indicate whether the encoding end performs color space format conversion on the texture map of the three-dimensional mesh;

[0081] Wherein, when the first indication information instructs the encoder to perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes the first texture map bitstream; or

[0082] When the first indication information indicates that the encoding end does not perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes a second texture map bitstream.

[0083] For example, if the first indication information is "0", it instructs the encoder not to perform color space format conversion on the texture image of the three-dimensional mesh; otherwise, the first indication information instructs the encoder to perform color space format conversion on the texture image of the three-dimensional mesh.

[0084] It should be noted that, when the first indication information indicates that the color space format conversion of the texture map of the three-dimensional mesh is to be performed, the first indication information can also be used to indicate the conversion type of the texture map of the three-dimensional mesh from the first color space format to the second color space format; thereby, the decoding end can determine what type of color space format conversion the encoding end has performed based on the first indication information, so that the decoding end can perform the inverse conversion of the corresponding color space format to restore the original texture map.

[0085] In this embodiment, the code stream corresponding to the three-dimensional mesh includes the first texture map code stream or the second texture map code stream, and the code stream corresponding to the three-dimensional mesh also includes the encoding result of the first indication information, and the first indication information is used to indicate whether the encoding end performs color space format conversion on the texture map of the three-dimensional mesh. In this way, the encoding end is indicated by the first indication information whether to perform color space format conversion on the texture map of the three-dimensional mesh. The decoding end can determine whether to perform color space format conversion on the decoded texture map through the first indication information, which facilitates the decoding end to reconstruct the three-dimensional mesh.

[0086] Optionally, when the first indication information indicates to perform color space format conversion on the texture image of the three-dimensional mesh, the first indication information is also used to indicate the conversion type of the texture image of the three-dimensional mesh from the first color space format to the second color space format.

[0087] The first indication information may be used to indicate that the texture map of the three-dimensional grid is converted from the first color space format to the second color space format at the encoding end. In one embodiment, the first indication information may be a value in a preset list, and different values ​​in the preset list represent different types of conversion of color space formats. For example, if the first indication information is "0", it indicates that the encoding end does not perform color space format conversion on the texture map of the three-dimensional grid; if the first indication information is "1", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR444 format of the RGB color space to the YUV444 format of the YUV color space; if the first indication information is "2", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR444 format of the RGB color space to the RGB444 format of the RGB color space; if the first indication information is "3", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR444 format of the RGB color space to the YUV420 format of the YUV color space; if the first indication information is "4", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR444 format of the RGB color space to the YUV422 format of the YUV color space; if the first indication information is "5", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR555 format of the RGB color space to the YUV444 format of the YUV color space; and so on.

[0088] In this embodiment, when the first indication information indicates that the color space format conversion of the texture map of the three-dimensional mesh is to be performed, the first indication information is also used to indicate the conversion type of the texture map of the three-dimensional mesh from the first color space format to the second color space format, so that the decoding end can determine what type of color space format conversion the encoding end has performed through the first indication information, so that the decoding end can perform an inverse conversion of the corresponding color space format to restore the original texture map.

[0089] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0090] The texture image in the second color space format may be a texture image obtained by removing certain redundant information from the texture image in the first color space format; or, the texture image in the second color space format may be a texture image obtained by removing the correlation between color components from the texture image in the first color space format.

[0091] In the related art, when encoding the texture map of the three-dimensional grid, the texture map in the BGR444 format is directly encoded. Since the decoding efficiency of the decoding end for decoding videos in different color space formats is different, and the decoding end is less efficient in decoding videos in the BGR444 format, the encoding and decoding efficiency of the three-dimensional grid is low. In this embodiment, during the three-dimensional grid encoding process, the texture map of the three-dimensional grid is converted into a texture map in a color space format with higher encoding and decoding efficiency by performing a color space format conversion on the texture map of the three-dimensional grid, thereby improving the encoding and decoding efficiency of the three-dimensional grid.

[0092] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0093] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0094] In this implementation, the first color space format is an RGB color space format, and the second color space format is a YUV color space format, so that when encoding the texture map of the three-dimensional mesh, the texture map of the RGB color space is converted into a texture map of the YUV color space for encoding. Since the efficiency of the decoding end in decoding the video of the YUV color space is higher than the efficiency of the decoding end in decoding the video of the RGB color space, the encoding and decoding efficiency of the three-dimensional mesh can be improved through color space format conversion.

[0095] The present application also provides a three-dimensional grid coding method, the method comprising:

[0096] In the case where it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream.

[0097] Optionally, the code stream corresponding to the three-dimensional mesh includes the encoding result of the second texture map code stream and first indication information, and the first indication information instructs the encoding end not to perform color space format conversion on the texture map of the three-dimensional mesh.

[0098] See also Figure 4 , Figure 4 is a flowchart of a three-dimensional grid decoding method provided in an embodiment of the present application, which can be applied to a decoding end device, such as Figure 4 As shown, the three-dimensional grid decoding method includes the following steps:

[0099] Step 201: Decode a first texture image bitstream in a bitstream corresponding to a three-dimensional grid to obtain a texture image in a second color space format;

[0100] Step 202: performing color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0101] Step 203: Perform mesh reconstruction processing based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0102] Optionally, the method further comprises:

[0103] Decoding the encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information;

[0104] The decoding process of the first texture image code stream in the code stream corresponding to the three-dimensional grid to obtain the texture image in the second color space format includes:

[0105] In the case where it is determined based on the first indication information to convert the color space format of the texture map, the first texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain the texture map in the second color space format.

[0106] Optionally, performing color space format conversion on the texture image in the second color space format to obtain the texture image in the first color space format includes:

[0107] When it is determined based on the first indication information that the texture image is converted from the second color space format to the first color space format, the texture image in the second color space format is converted from the second color space format to the first color space format to obtain the texture image in the first color space format.

[0108] Optionally, the method further comprises:

[0109] When it is determined based on the first indication information that the texture map is not to be converted into a color space format, a second texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain a texture map in the first color space format.

[0110] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0111] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0112] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0113] It should be noted that this embodiment is Figure 3 The corresponding decoding side implementation in the embodiment shown in the figure can be found in the specific implementation. Figure 3 In order to avoid repeated description, the relevant description of the embodiment shown will not be repeated in this embodiment, and the same beneficial effects can still be achieved.

[0114] The present application also provides a three-dimensional grid decoding method, the method comprising:

[0115] Decoding the encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information;

[0116] When the first indication information indicates that the color space format conversion is not performed on the texture map, decoding the second texture map bitstream in the bitstream corresponding to the three-dimensional grid to obtain the texture map in the first color space format;

[0117] A mesh reconstruction process is performed based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0118] The embodiment of the present application also proposes a three-dimensional mesh texture image lossless encoding and decoding framework and method based on Edgebreaker. Figure 5 As shown in FIG. 1 , the coding framework mainly includes a preprocessing module (including a non-manifold structure splitting module), a connection relationship coding module, a geometric information coding module, an attribute information coding module, a texture image lossless color space format conversion, and a texture image coding module. When encoding a 3D mesh, Edgebreaker can be used to perform lossless coding on the 3D mesh.

[0119] Taking the first color space format as RGB color space format, the second color space format as YUV color space format, and the attribute information as UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture) as an example, the process of the encoder performing three-dimensional mesh encoding is as follows:

[0120] (1) At the encoding end, it is determined whether the geometric coordinates and the UV coordinates have the same connection relationship and the information is recorded. The information indicating whether the geometric coordinates and the UV coordinates in the grid have the same connection relationship can be represented by, for example, setting an identifier indicating whether the geometric coordinates and the UV coordinates in the grid have the same connection relationship. The representation method is not limited here.

[0121] (2) Split the non-manifold points and non-manifold edges that may exist in the mesh and convert the mesh into a manifold mesh. Splitting the non-manifold structure is achieved by adding duplicate points with the same geometric information and attribute information. When splitting the non-manifold, if the geometric coordinates and UV coordinates have the same connection relationship, that is, the geometric coordinates and UV coordinates are in a one-to-one correspondence, then only one set of duplicate point index information generated by splitting the non-manifold structure needs to be recorded; if the geometric coordinates and UV coordinates have different connection relationships, such as one geometric coordinate vertex corresponds to multiple texture coordinate vertices, then the geometric duplicate point index information and UV duplicate point index information generated by splitting the non-manifold structure need to be recorded separately. The representation method of the duplicate point index information generated by splitting the non-manifold structure can be the target vertex index that needs to be merged when restoring the non-manifold, or it can be the duplicate point group index: points with the same index belong to the same duplicate point group, that is, they have the same vertex information, and its representation method is not restricted here.

[0122] (3) Encode information indicating whether the geometric coordinates and UV coordinates have the same connection relationship. Use the Edgebreaker method to encode the connection relationship to obtain a CLERS pattern string that can concisely represent the connection relationship. Use entropy coding to compress the pattern string to obtain a sub-code stream of the connection relationship. Use methods such as parallelogram prediction to encode the geometric information of the mesh to obtain a geometric information sub-code stream. If the mesh has attribute information such as UV coordinates, use methods such as similar triangle prediction to encode the UV coordinates of the mesh to obtain an attribute information sub-code stream.

[0123] (4) Encode an indication information (i.e., the first indication information) to indicate whether to perform lossless color space format conversion on the texture image. If the texture image is to be converted to a lossless color space format, the texture image is converted from the RGB color space format (i.e., the first color space format) to the YUV color space format (i.e., the second color space format), and the texture image in the YUV format is encoded using a video encoder to obtain a texture image sub-stream. Here, only the conversion from the RGB color space format to the YUV color space is emphasized, and the specific format of the texture image is not limited. For example, the texture image in the RGB color space can be in the BGR444 format or the RGB444 format. If the texture image is not to be converted to a color space format, the video encoder is directly used to losslessly encode the video in the RGB color space.

[0124] (5) The encoding of non-manifold structure information includes three parts of information to be encoded: information indicating whether there is a non-manifold structure in the mesh, identification information of whether the vertex is a duplicate point generated by removing the non-manifold, and index information of duplicate points generated by removing the non-manifold structure. Among them, the information indicating whether there is a non-manifold structure in the mesh can be represented by setting an identification of whether there is a non-manifold structure in the mesh, or by the number of duplicate points generated by removing the non-manifold structure in the mesh. The representation method is not limited here. When encoding the non-manifold structure information, the information indicating whether there is a non-manifold structure in the mesh is first encoded. If there is a non-manifold structure in the mesh, it is divided into two cases: if the geometric coordinates and the UV coordinates have the same connection relationship, an identifier is set for each vertex to indicate whether it is a duplicate point generated by splitting the non-manifold, and this information and the index information of the duplicate points generated by splitting the non-manifold are encoded; if the geometric coordinates and the UV coordinates have different connection relationships, an identifier is set for each geometric vertex and each UV vertex to indicate whether it is a geometric duplicate point or a UV duplicate point generated by splitting the non-manifold, and the non-manifold identification information of the geometric vertex and the index information of the geometric duplicate points generated by splitting the non-manifold, the non-manifold identification information of the UV vertex and the index information of the UV duplicate points generated by splitting the non-manifold are encoded respectively.

[0125] For example, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture). Figure 6 As shown, the process of three-dimensional grid decoding at the decoding end is as follows:

[0126] (1) At the decoding end, information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship is obtained by decoding. The CLERS pattern string is obtained by entropy decoding, and the connection relationship between the geometric coordinates and the UV coordinates is reconstructed using the pattern string. The geometric information of the mesh is decoded using methods such as parallelogram inverse prediction, and the UV coordinates of the mesh are decoded using methods such as similar triangle inverse prediction. When decoding non-manifold structure information, the indication information of whether there is a non-manifold structure in the mesh is first decoded. If there is a non-manifold structure in the mesh, the non-manifold identification information of the vertex and the index information of the duplicate points generated by the non-manifold removal are further decoded to obtain the duplicate points generated by the non-manifold removal, and the duplicate points generated by the non-manifold removal are merged, and the connection relationship is adjusted to restore the non-manifold structure in the mesh.

[0127] (2) Decode the indication information (such as the first indication information) of whether to perform lossless color space format conversion, and use the video decoder to decode the texture map. If the color space format of the texture map is converted to a lossless color space format at the encoding end, the decoded texture map is in the YUV format. According to the indication information, the texture map is converted to a lossless color space format at the decoding end, and the texture map is converted from the YUV color space format (i.e., the second color space format) to the RGB color space format (i.e., the first color space format), thereby realizing lossless encoding and decoding of the texture map. Here, only the conversion from the YUV color space format to the RGB color space format is emphasized, and the specific format of the texture map is not limited. For example, the texture map can be in the BGR444 format or the RGB444 format in the RGB color space. If the color space format of the texture map is not converted to a lossless color space format at the encoding end, the decoded texture map is in the RGB format. In this case, there is no need to perform color space format conversion at the decoding end.

[0128] Through the coding and decoding framework of the embodiment of the present application, lossless coding and decoding of three-dimensional meshes can be completed.

[0129] The three-dimensional grid coding framework of the embodiment of the present application is as follows Figure 5 As shown, in this embodiment, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture) as an example for explanation.

[0130] (1) In the preprocessing step, first determine whether the geometric coordinates and UV coordinates have the same connection relationship, then split the input grid with non-manifold structure to obtain a manifold grid, and record the duplicate point information generated by splitting the non-manifold. It should be noted that this step can also include other preprocessing modules required for encoding, such as filtering out duplicate points and adding virtual points. For the convenience of explanation, Figure 5 Only the modules for splitting non-manifold structures targeted by the embodiments of the present application are listed.

[0131] (2) Encode information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship; for manifold meshes, use the Edgebreaker method to encode the connection information, obtain a pattern string, and perform entropy encoding on it; encode the geometric information of the manifold mesh, for example, the parallelogram prediction coding method can be used, and the encoding method of the geometric information is not limited here; if the mesh has attribute information such as UV coordinates, it can be encoded using methods such as similar triangle prediction coding, and the encoding method of the attribute information is not limited here; when encoding non-manifold structure information, first encode whether there is non-manifold representation information in the mesh. If there is a non-manifold structure in the mesh, it is divided into two cases according to whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship: if the geometric coordinates and UV coordinates in the mesh have the same connection relationship, that is, the geometric coordinates and UV coordinates are in a one-to-one correspondence, a group of non-manifold identification information of vertices and the index information of duplicate points generated by removing the non-manifold are encoded in the same encoding order; if the geometric coordinates and UV coordinates in the mesh have different connection relationships, that is, the geometric coordinates and UV coordinates are not in a one-to-one correspondence, then the non-manifold identification information of the geometric vertices and the index information of the geometric duplicate points generated by removing the non-manifold, the non-manifold identification information of the UV vertices and the index information of the UV duplicate points generated by removing the non-manifold are encoded in the encoding order of the geometric coordinates and UV coordinates. The non-manifold identification information of the vertex here is to identify whether the vertex is a duplicate point generated by removing the non-manifold by encoding a flag bit for each vertex in the split manifold mesh.

[0132] (3) An indication information can be encoded to indicate whether to perform lossless color space format conversion on the texture image. If the texture image is to be converted to a lossless color space format, the texture image is converted from the RGB color space format (i.e., the first color space format) to the YUV color space format (i.e., the second color space format) in the lossless color space format conversion module, and the texture image in the YUV format is encoded using a video encoder to obtain a texture image sub-stream. If the texture image is not to be converted to a color space format, the video encoder is directly used to losslessly encode the video in the RGB color space format. Finally, the multiple bitstreams are mixed to obtain the final output bitstream.

[0133] It should be noted that the order of encoding the connection relationship of the three-dimensional mesh and the vertex information is not emphasized. The geometric information, attribute information and non-manifold structure information can be encoded at the same time as the connection relationship is encoded; or the geometric information, attribute information and non-manifold structure information can be encoded in the order of encoding the connection relationship after the connection relationship is encoded. When encoding the geometric information and attribute information of the vertices, the geometric information and attribute information of the repeated points generated by the non-manifold splitting can be skipped, that is, only encoded once, or can not be skipped. It is not emphasized here whether to skip the encoding of the geometric information and attribute information of the repeated points generated by the non-manifold splitting.

[0134] The three-dimensional grid decoding framework of the embodiment of the present application is as follows Figure 6 As shown, in this embodiment, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture) as an example for explanation.

[0135] (1) First, decompose the code stream to obtain sub-code streams. Then decode the information indicating whether the geometric coordinates and UV coordinates in the grid have the same connection relationship; entropy decode the connection relationship sub-code stream to obtain the pattern string. If the geometric coordinates and UV coordinates have the same connection relationship, only one set of connection relationships needs to be reconstructed; if the geometric coordinates and UV coordinates have different connection relationships, the connection relationship of the geometric coordinates and the connection relationship of the UV coordinates need to be reconstructed respectively; use the decoding method corresponding to the encoding end to decode the geometric information of the grid; use the decoding method corresponding to the encoding end to decode the attribute information of the grid; when decoding non-manifold structure information, first decode whether there is non-manifold information in the grid. If there is non-manifold structure in the grid, then There are two cases depending on whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship: if the geometric coordinates and UV coordinates in the mesh have the same connection relationship, that is, the geometric coordinates and UV coordinates are in a one-to-one correspondence, further decoding is performed to obtain a set of non-manifold identification information of vertices and duplicate point index information generated by removing the non-manifold; if the geometric coordinates and UV coordinates in the mesh have different connection relationships, that is, the geometric coordinates and UV coordinates are not in a one-to-one correspondence, then the non-manifold identification information of the geometric vertices and the geometric duplicate point index information generated by removing the non-manifold, the non-manifold identification information of the UV vertices and the UV duplicate point index information generated by removing the non-manifold are decoded respectively. In the post-processing step, the duplicate points generated by removing the non-manifold are merged, and the connection relationship is adjusted to restore the non-manifold structure in the mesh. The lossless encoding and decoding of the mesh containing the non-manifold structure is completed. It should be noted that the post-processing step may also include post-processing modules required for correct decoding, such as restoring filtered duplicate points and deleting added virtual vertices. For the convenience of explanation, Figure 6 Only the modules for restoring non-manifold structures targeted by the embodiments of the present application are listed.

[0136] (2) Decode the indication information (such as the first indication information) indicating whether to perform lossless color space format conversion, and use the video decoder to decode the texture map. If the color space format of the texture map is converted to a lossless color space format at the encoding end, the decoded texture map is a YUV format texture map. According to the indication information, the texture map is converted to a lossless color space format at the decoding end, and the texture map is converted from the YUV color space format (i.e., the second color space format) to the RGB color space format (i.e., the first color space format), thereby realizing lossless encoding and decoding of the texture map. If the color space format of the texture map is not converted to a lossless color space format at the encoding end, the decoded texture map is a RGB format texture map, and in this case, there is no need to perform color space format conversion at the decoding end.

[0137] The following is a description of the three-dimensional grid encoding and decoding method through a specific embodiment:

[0138] Encoding side:

[0139] like Figure 5 As shown, the three-dimensional mesh lossless coding framework is mainly divided into five parts: splitting non-manifold structure in preprocessing, connection relationship coding, geometric information coding, attribute information coding and non-manifold structure information coding. In this embodiment, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture) as an example, and each part is introduced separately:

[0140] (1) Pretreatment

[0141] Input: original mesh;

[0142] Output: information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship, manifold mesh, and duplicate point information generated by decomposing non-manifolds.

[0143] Before dismantling the non-manifold structure, first determine whether the geometric coordinates and UV coordinates have the same connection relationship. You can first determine whether the number of geometric vertices is consistent with the number of UV vertices. If they are consistent, then determine whether the geometric triangles are consistent with the UV triangles, that is, whether the geometric connection relationship is consistent with the UV connection relationship. If the geometric triangles are inconsistent with the UV triangles, the geometric coordinates and UV coordinates have different connection relationships; if the number of geometric vertices is inconsistent with the number of UV vertices, but the number of geometric triangles is consistent with the number of UV triangles, then the geometric coordinates and UV coordinates also have different connection relationships. It can also be determined in other ways, and the determination method is not limited here.

[0144] Splitting non-manifold structures is mainly divided into two parts: splitting non-manifold edges and splitting non-manifold points.

[0145] The first step in splitting non-manifold edges is to find them. The criterion for determining a non-manifold edge is that an edge exists in three or more triangles at the same time. The specific implementation method is: either establish a data structure to store the triangles where each edge is located, and find the non-manifold edge by querying the number of triangles corresponding to the edge; or construct a corner table (CornerTable) to establish the correspondence between the angles and edges in the grid, and then find the non-manifold edge. Specifically, for a manifold grid, each edge is opposite to at most two corners, and the two opposite corners are called diagonals. For example Figure 7 As shown in the figure, angle a and angle d are opposite to side bc, and angle a and angle d are opposite angles; for non-manifold edges, there will be three or more opposite angles. Therefore, non-manifold edges can also be found through the correspondence between angles and edges.

[0146] The second step of splitting non-manifold edges is to add vertices and modify the connection relationship. After finding the non-manifold edge, create duplicate vertices for the two vertices of the non-manifold edge, select a triangle t where the non-manifold edge is located, make the third vertex in the triangle and the two newly added vertices form a new triangle t', replace the original triangle t with t', and iterate this process until the non-manifold edge is converted into a manifold edge.

[0147] To split non-manifold points, you first need to build a CornerTable and establish a corresponding relationship between each vertex and the corner of the vertex. Perform two steps for each vertex. In the first step, starting from a certain corner of the vertex, traverse all the corners adjacent to the corner and forming a fan in turn, and mark the vertex and the traversed corners as traversed. If after executing the above process, there are still vertices that have not been traversed, it means that the vertex is a non-manifold point. In the second step, for each non-manifold point, create a duplicate point and modify the connection relationship, connect the corners that have not been traversed in the first step to the newly added duplicate point, and split the non-manifold point into two manifold vertices. Repeat this process until all vertices are converted into manifold points.

[0148] If the geometric coordinates and UV coordinates have the same connection relationship, the above process is executed once, and a set of duplicate point information generated by the process is recorded; if the geometric coordinates and UV coordinates have different connection relationships, the above process is executed once for the geometry and UV respectively: the CornerTable of the geometry and UV is constructed respectively, and the non-manifold edges of the geometry and the non-manifold points of the geometry, the non-manifold edges of the UV and the non-manifold points of the UV are split respectively, and the geometric duplicate point information and UV duplicate point information generated by the process are recorded respectively.

[0149] (2) Connection relationship coding

[0150] Input: The connectivity of the manifold mesh, which indicates whether the geometric coordinates and UV coordinates in the mesh have the same connectivity.

[0151] Output: Encoded connection relationship sub-code stream and vertex encoding order.

[0152] This embodiment uses the Edgebreaker method to encode the connection relationship of the three-dimensional mesh, establishes a CornerTable to represent the connection relationship of the mesh, and uses the CornerTable to traverse all triangles in the mesh to generate the CLERS pattern string of Edgebreaker.

[0153] CornerTable is used to represent the relationship between corners, vertices and triangles in a mesh. Before building CornerTable, you first need to number the corners. Traverse the triangles in the order of the triangle patches in the mesh, and number the corners of each triangle in counterclockwise order. If the mesh has f triangle patches, then the mesh will have 3f corners. The advantage of this numbering is that the triangle number of the current corner can be calculated through the corner number, as shown in formula (1); you can also calculate the previous corner c of the current corner c in the counterclockwise direction. p and the next angle c n The serial number is shown in formula (2) and formula (3).

[0154] f i =c / 3 (1)

[0155] Among them, c is the index of the current angle, f i is the serial number of the triangle where the current angle c is located, " / " is integer division, and the result is rounded down.

[0156] c p =(f i *3)+(c-1)%3 (2)

[0157] Among them, c is the index of the current angle, f i is the serial number of the triangle where the current angle c is located, "*" is multiplication, and "%" is modulo operation.

[0158] c p =(f i *3)+(c+1)%3 (3)

[0159] Among them, c is the index of the current angle, f i is the serial number of the triangle where the current angle c is located, "*" is multiplication, and "%" is modulo operation.

[0160] CornerTable consists of four parts: V, O, U, and M. The V table stores the vertex index corresponding to each corner, the O table stores the diagonal index of each corner, the U table stores the flag of whether each triangle has been traversed during the traversal process, and the M table stores the flag of whether each vertex has been traversed during the traversal process.

[0161] CornerTable can be used to construct Figure 8 The relationship shown in the figure, where c represents the current angle, cp represents the previous angle of the current angle c (counterclockwise), and cn represents the next angle of the current angle c. co is the opposite angle of the current angle c, which can be obtained by querying the O table. ct is the serial number of the triangle where c is located, which can be calculated by formula 1. cv represents the vertex of the current angle, which can be obtained by querying the V table. cl represents the angle to the left of the current angle c, which is obtained by querying the opposite angle of cp in the O table; cr represents the angle to the right of the current angle c, which is obtained by querying the opposite angle of cn in the O table.

[0162] After using CornerTable to build the relationship between corners, vertices and triangles, the mesh can be traversed in a spiral order to obtain the CLERS pattern string of Edgebreaker that represents the mesh connection relationship. At this time, the judgment conditions and traversal rules of the five modes are as follows: Fig. 9 As shown. The current traversed angle is x. If the vertex xv corresponding to x has not been visited, the current triangle is in C mode, and the next triangle to be traversed is the triangle where xr is located; otherwise, if the triangle where xl is located has been visited, the current triangle is in L mode, and the next triangle to be traversed is the triangle where xr is located; if the triangle where xr is located has been visited, the current triangle is in R mode, and the next triangle to be traversed is the triangle where xl is located; if the vertex xv has been visited, but the triangles where xl and xr are located have not been visited, the current triangle is in S mode, and the traversal path will generate two branches. Using the principle of depth-first traversal, the first triangle traversed is the triangle where xr is located, and the triangle where xl is located is stored in the stack. After the traversal of the branch where xr is located is completed, the triangle where xl is located is traversed; if the triangles where xl and xr are located have been visited, the mode of the current triangle is E, and the traversal has reached the end of the current traversal path branch.

[0163] Randomly select an initial triangle in the mesh, traverse the triangles in the mesh according to the above rules, and generate a CLERS pattern string. When the traversal path ends, but there are still untraversed triangles in the mesh, randomly select an untraversed triangle and start the next traversal until all triangles in the mesh have been traversed.

[0164] Entropy coding is used to compress the CLERS pattern string to obtain the final connection information code stream.

[0165] If the geometric coordinates and UV coordinates have the same connection relationship, the above process only needs to be performed once; if the geometric coordinates and UV coordinates have different connection relationships, since the number of geometric triangles and texture triangles is consistent and corresponding, but the corresponding geometric coordinate index and UV coordinate index are not one-to-one corresponding, it is only necessary to build and encode the CLERS pattern string once according to the CornerTable of the geometry. In this case, for UV, after the pattern string encoding of each connected area of ​​the geometry is completed, the traversed corners are traversed, and the difference information between the UV connection relationship and the geometric connection relationship is recorded and encoded according to the TC table and OTC table of UV. Among them, the TC table stores the UV vertex index corresponding to each corner in the texture triangle, and the OTC table stores each diagonal index in the texture triangle.

[0166] (3) Geometric Information Coding

[0167] Input: Geometric information and connection relationship encoding order of manifold mesh;

[0168] Output: geometry information sub-code stream and geometry information encoding order.

[0169] There are many ways to encode geometric information, such as difference prediction coding algorithm, parallelogram prediction coding algorithm, multi-parallelogram prediction coding algorithm, etc. The specific coding method is not emphasized here. Take the parallelogram prediction coding algorithm as an example: there are four vertices a, b, c, and d, forming the following Fig.10 Two adjacent triangles in the mesh shown.

[0170] Among them, the geometric information of points a, b, and c has been encoded, and the geometric information of point d is to be encoded. The predicted value d' of the geometric coordinates of point d can be calculated using formula (4).

[0171] d′(x,y,z)=b(x,y,z)+c(x,y,z)-a(x,y,z)(4)

[0172] After obtaining d', calculate the difference between d' and the three-dimensional coordinates of point d, as shown in formula (5):

[0173] Δd(x,y,z)=d(x,y,z)-d′(x,y,z)(5)

[0174] Perform entropy coding on Δd to obtain a code stream of geometric information.

[0175] For triangles that cannot be predicted using parallelograms, such as triangles at the boundary of a mesh, the difference coding method is used to encode geometric information, that is, the coordinate values ​​of adjacent encoded vertices are used as the predicted values ​​of the current vertex coordinates to calculate and predict the residual.

[0176] (4) Attribute Information Coding

[0177] Input: attribute information and connection relationship encoding order of manifold mesh;

[0178] Output: attribute information sub-code stream and the encoding order of attribute information.

[0179] The three-dimensional mesh attribute information generally includes UV coordinates, normal vectors, etc. Taking UV coordinates as an example, there are many encoding methods that can be used for UV coordinates, including difference prediction encoding, parallelogram prediction encoding, and similar triangle prediction encoding. The specific encoding method is not emphasized here. The similar triangle prediction algorithm is described below.

[0180] First, select a triangle in a three-dimensional mesh as the initial triangle, directly encode the UV coordinates of the three vertices of the initial triangle without predicting them, and store the edges of the initial triangle in an edge set. This set can be a data structure that meets certain access criteria. Then take out an edge τ from the set and predict the UV coordinates of the opposite vertex of τ in the next new triangle. And put the two sides of the new triangle except τ into the set. Let the point to be predicted be point C, the two end points of edge τ be N and P respectively, the opposite vertex of the triangle adjacent to the new triangle through τ is O, and the projection point of C on τ is X. For example Fig.11 As shown, since the UV coordinates of points N, P, and O are encoded before point C, these three points can be used to predict the UV coordinates of point C. The specific calculation process is as follows:

[0181] Let C uv ,X uv ,N uv ,P uv ,O uv are the UV coordinates of each point, C G ,X G ,N G ,P G ,O G are the geometric coordinates of each point.

[0182] First, use equations (6) and (7) to calculate the UV coordinates of point X:

[0183]

[0184]

[0185] in, The vectors with G as the subscript are all vector representations of the geometric coordinates between the corresponding points; similarly, The vector representations with UV as the subscript are all vector representations of the UV coordinates between the corresponding points.

[0186] Use formula (8) to calculate the vector Rotated() means flipping the vector 90 degrees:

[0187]

[0188] Finally, use equations (9), (10) and (11) to calculate the predicted UV coordinates Pred of point C C :

[0189]

[0190]

[0191]

[0192] in, and Both represent the vector of the UV coordinates between point C and point O, but they are expressed in different calculation methods / value forms.

[0193] After obtaining the predicted UV coordinate value, subtract it from the original UV coordinate to get the residual value.

[0194] The steps to encode UV coordinates are as follows:

[0195] (a) Select an initial triangle from the connectivity relationship, and directly encode the UV coordinates of the three vertices of the initial triangle without prediction. Store the edges of the initial triangle in the edge set.

[0196] (b) Select edge τ from the set according to the access criteria, and encode the UV coordinates of the vertices of the new triangle formed by τ. Using the projection relationship of the triangle from three dimensions to two dimensions, the predicted value of the point to be encoded is calculated according to the UV coordinate prediction value calculation process described above. The residual is obtained by subtracting the predicted value from the original UV coordinate value.

[0197] (c) Add the two edges of the new triangle to the edge set and remove the edge τ at the top of the set. Take the next edge from the set and continue encoding the UV coordinates of the vertices of the adjacent triangles of the edge, and return to step 3 until the UV coordinates of all vertices are encoded.

[0198] (d) Entropy encode the UV coordinate residual and output the UV coordinate code stream.

[0199] (5) Non-manifold structural information encoding

[0200] Input: information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship, information indicating whether there is a non-manifold structure in the mesh, non-manifold identifiers of vertices, index information of duplicate points generated by removing non-manifolds, encoding order of geometric information, and encoding order of attribute information;

[0201] Output: non-manifold structure information sub-code stream.

[0202] First, encode information indicating whether there is a non-manifold structure in the grid. Take setting the flag of whether there is a non-manifold structure as an example: if there is no non-manifold structure in the grid, that is, the number of duplicate points generated by removing the non-manifold is 0, set the flag to 0, and there is no need to encode the non-manifold flag of the vertex and the index information of the duplicate points generated by removing the non-manifold; if there is a non-manifold structure in the grid, that is, the number of duplicate points generated by removing the non-manifold is greater than 0, set the flag to 1, and then encode the non-manifold flag of the vertex and the index information of the duplicate points generated by removing the non-manifold.

[0203] The duplicate points generated by removing non-manifolds include geometric duplicate points and UV duplicate points. According to whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship, the non-manifold identifiers of the encoded vertices and the index information of the duplicate points generated by removing the non-manifolds are divided into two cases: the first case is that the geometric coordinates and UV coordinates in the mesh have the same connection relationship, that is, the geometric coordinates and UV coordinates are in a one-to-one correspondence, then the non-manifold identifiers of a group of vertices and the index information of the duplicate points generated by removing the non-manifolds are directly encoded, and the index information can be obtained according to the encoding order of the geometric coordinates and UV coordinates at the encoding end; the second case is that the geometric coordinates and UV coordinates in the mesh have different connection relationships, that is, the geometric coordinates and UV coordinates are not in a one-to-one correspondence, then the non-manifold identifiers of the geometric vertices and the index information of the geometric duplicate points generated by removing the non-manifolds, the non-manifold identifiers of the UV vertices and the index information of the UV duplicate points generated by removing the non-manifolds are encoded respectively, and the index information can be obtained according to the encoding order of the geometric coordinates and the encoding order of the UV coordinates at the encoding end.

[0204] The specific implementation is: if the geometric coordinates and UV coordinates in the mesh have the same connection relationship, then a flag is set for each vertex in the manifold mesh to indicate whether the point at the current position is a duplicate point generated by the removal of the non-manifold, and the duplicate point index information generated by the removal of the non-manifold is encoded; if the geometric coordinates and UV coordinates in the mesh have different connection relationships, then a flag is set for each geometric vertex and each UV vertex in the manifold mesh to indicate whether the geometric vertex and UV vertex at the current position are duplicate points generated by the removal of the non-manifold, and the geometric duplicate point index information and UV duplicate point index information generated by the removal of the non-manifold are encoded respectively. Then, the flag bit binary string sequence arranged in the corresponding coding order and the duplicate point index information generated by the removal of the non-manifold are entropy encoded to obtain the non-manifold structure information code stream.

[0205] There are multiple ways to store the non-manifold structure information bitstream in the total bitstream: one is to store the non-manifold structure information bitstream as a separate sub-bitstream; another is to store the geometric non-manifold structure information bitstream in the geometric information sub-bitstream, and store the UV non-manifold structure information bitstream in the attribute information sub-bitstream; the geometric non-manifold structure information bitstream and the UV non-manifold structure information bitstream can also be stored as two sub-bitstreams in the total bitstream. The storage method of the non-manifold structure information bitstream in the total bitstream is not emphasized here.

[0206] (6) Lossless color space format conversion module

[0207] Input: Texture image in BGR444 format (BGR444 format is used as an example here);

[0208] Output: Texture map in YUV444 format.

[0209] An indication information can be encoded to indicate whether to perform lossless color space format conversion on the texture map. If the texture map is subjected to lossless color space format conversion, the texture map in the input RGB color space format (i.e., the first color space format) is subjected to lossless color space format conversion to obtain a texture map in the YUV color space format (i.e., the second color space format). The lossless color space format conversion can be implemented in a variety of ways, and the embodiments of the present application do not limit the specific implementation methods of the lossless color space format conversion. For example, the color space format conversion method shown in Formula (12)-Formula (14) can be used to convert the texture map from the RGB color space format to the YUV color space format.

[0210]

[0211] V=RG (13)

[0212]

[0213] Alternatively, the color space format conversion method shown in equations (15)-(17) may be used to convert the texture image from the RGB color space format to the YUV color space format.

[0214]

[0215] U=RG (16)

[0216] V=BG (17)

[0217] Among them, round() represents the round-down function.

[0218] (7) Texture image encoding module

[0219] If the texture image is converted to a lossless color space format, a video encoder is used to losslessly encode the video in the YUV color space format. If the texture image is not converted to a color space format, a video encoder is used to losslessly encode the video in the RGB color space format. The video encoder used is not limited here.

[0220] Decoding side:

[0221] like Figure 6 As shown, the three-dimensional mesh lossless decoding framework of the embodiment of the present application is mainly divided into six parts: connection relationship decoding, geometric information decoding, attribute information decoding, non-manifold structure information decoding, reconstruction of manifold mesh, and recovery of non-manifold structure in post-processing. In this embodiment, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture) as an example, and each part is introduced separately:

[0222] (1) Connection relationship decoding

[0223] Input: the connection relationship sub-code stream to be decoded, indicating whether the geometric coordinates and UV coordinates have the same connection relationship;

[0224] Output: connectivity of the manifold mesh and the decoded vertex order.

[0225] First, decode the connection relationship sub-code stream to obtain the pattern string. Traverse the pattern string in a certain order (forward or reverse order), and reconstruct the connection relationship according to the corresponding pattern in the string. Consistent with the encoding end, if the geometric coordinates and UV coordinates have the same connection relationship, only one set of connection relationships needs to be reconstructed; if the geometric coordinates and UV coordinates have different connection relationships, first reconstruct the geometric connection relationship based on the decoded string, and then traverse the corners in the currently decoded connected area and the difference information between the UV connection relationship and the geometric connection relationship in the currently decoded connected area to reconstruct the connection relationship of the UV coordinates. In addition, the traversal order of the vertices is output to the geometric information and attribute information decoding module.

[0226] (2) Geometric Information Decoding

[0227] Input: geometric information sub-code stream, decoding order of connection relationship;

[0228] Output: Geometric information of the manifold mesh.

[0229] The decoding process of the mesh geometric coordinates is the inverse process of the encoding process: first, entropy decode the coordinate prediction residual. Then, based on the decoded triangle, predict the predicted coordinates of the point to be decoded according to the parallelogram rule. The predicted coordinates plus the residual value obtained by entropy decoding can get the geometric coordinate position to be decoded. The vertex traversal order here is the same as the vertex order of the encoded geometric information. It should be noted that the geometric coordinates of the initial triangle do not use predictive coding, but directly encode their geometric coordinate values. After the geometric coordinates of the triangle are decoded at the decoding end, the geometric coordinates of the vertices of other triangles are traversed and decoded as the initial triangle. In addition, other decoding methods may also be used here. The specific decoding method is not emphasized, as long as it corresponds to the encoding end.

[0230] (3) Attribute Information Decoding

[0231] Input: attribute information code stream to be decoded, decoding order of connection relationship;

[0232] Output: attribute information of manifold mesh reconstruction.

[0233] Taking UV coordinates as an example, the decoding of UV coordinates adopts the decoding method corresponding to the encoding end, and the specific decoding method is not emphasized here. The decoding process using the similar triangle prediction algorithm is described below.

[0234] The steps to decode UV coordinates are as follows:

[0235] a. Entropy decode the UV coordinate code stream.

[0236] b. Decode the UV coordinates of the three vertices of the initial triangle. The prediction value is not calculated here. The initial triangle directly encodes its UV coordinates instead of encoding the residual. Store the edges of the initial triangle in the edge set.

[0237] c. Select edge τ from the set according to the access criteria, and decode the UV coordinates of the vertices of the new triangle formed with τ. First, use the mapping relationship from 3D to 2D of the triangle to calculate the predicted value of the UV coordinate of the point to be decoded in the same way as the encoding end. Then add the predicted value to the residual of entropy decoding to obtain the reconstructed UV coordinate.

[0238] d. Add the two edges of the new triangle to the edge set, remove the edge τ at the top of the set, take the next edge from the set, continue decoding the UV coordinates of the opposite vertices of the triangles adjacent to the edge, and return to step c until the UV coordinates of all vertices are decoded.

[0239] (4) Decoding of non-manifold structural information

[0240] Input: non-manifold structure information sub-code stream, information indicating whether the geometric coordinates and UV coordinates in the grid have the same connection relationship, geometric information decoding order, and attribute information decoding order;

[0241] Output: Information indicating whether there is a non-manifold structure in the mesh, the non-manifold identification of the vertex, and the index information of duplicate points generated by removing the non-manifold.

[0242] First, decode the information indicating whether there is a non-manifold structure in the grid. Take the information as an example of whether there is a non-manifold flag: if the flag is 0, there is no need to decode the non-manifold flag of the vertex and the duplicate point index information generated by removing the non-manifold, and skip the subsequent module for recovering the non-manifold structure; if the flag is 1, decode the non-manifold flag of the vertex and the duplicate point index information generated by removing the non-manifold.

[0243] The decoding of the non-manifold identification of vertices and the repeated point index information generated by removing the non-manifold adopts the method corresponding to the encoding end, and is divided into two cases according to whether the geometric coordinates and UV coordinates have the same connection relationship: if the geometric coordinates and UV coordinates have the same connection relationship, entropy decoding is used to obtain a set of non-manifold identifications of vertices and repeated point index information generated by removing the non-manifold; if the geometric coordinates and UV coordinates have different connection relationships, the non-manifold identification of geometric vertices and geometric repeated point index information generated by removing the non-manifold, and the non-manifold identification of UV vertices and UV repeated point index information generated by removing the non-manifold are respectively decoded. This information is recorded and output to the non-manifold structure recovery module.

[0244] (5) Reconstructing the manifold mesh

[0245] Input: connection relationship of manifold mesh, geometric information of manifold mesh, attribute information of manifold mesh;

[0246] Output: Manifold mesh.

[0247] The manifold mesh can be directly reconstructed by using the connection relationship, geometric information, and attribute information of the manifold mesh.

[0248] (6) Restoring non-manifold structures in post-processing

[0249] Input: manifold mesh, non-manifold identifiers of vertices, duplicate point index information generated by removing non-manifolds, and information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship;

[0250] Output: Reconstructed non-manifold mesh.

[0251] The recovery process of non-manifold edges and non-manifold points is the same. Taking the duplicate point index information generated by splitting the non-manifold as the duplicate point group index as an example, we can first traverse all vertices in the decoding order of the vertices, and establish a hash table based on the non-manifold identifier of the vertex. The key of the hash table is the duplicate point group index generated by splitting the non-manifold, and the value is the index of the target vertex to be merged. If the current vertex is not a duplicate point generated by splitting the non-manifold, the current vertex corresponds to the index of the current vertex, that is, its index is not updated; if the current vertex is a duplicate point generated by splitting the non-manifold and is the target vertex to be merged into the duplicate point group, the current vertex corresponds to the index of the current vertex, that is, its index is not updated, and the duplicate point group index of the current vertex and the index of the current vertex in the reconstructed manifold mesh are added to the hash table; if the current vertex is a duplicate point generated by splitting the non-manifold, but is not the target vertex to be merged into the duplicate point group, the index of the target vertex to be merged into the reconstructed manifold mesh is searched in the hash table according to the duplicate point group index of the current point, so as to merge the duplicate points generated by splitting the non-manifold, that is, update the index of the current vertex to the index of the target vertex to be merged into. Finally, update the geometric coordinate list and UV coordinate list, and update the indexes of the geometric vertices and UV vertices in the connection relationship to obtain the reconstructed non-manifold mesh.

[0252] If the geometric coordinates and UV coordinates have the same connection relationship, the above process is executed once to obtain a non-manifold mesh with the same connection relationship between the reconstructed geometric coordinates and UV coordinates; if the geometric coordinates and UV coordinates have different connection relationships, the geometric vertices and UV vertices are traversed in the decoding order of the geometric vertices and UV vertices respectively, and a geometric hash table and a UV hash table are established according to the non-manifold identifiers of the geometric vertices and UV vertices respectively, and the index information of the geometric duplicate points generated by splitting the non-manifold and the index of the geometric target vertex to be merged in the reconstructed manifold mesh, the index information of the UV duplicate points generated by splitting the non-manifold and the index of the UV target vertex to be merged in the reconstructed manifold mesh are stored respectively, and the above judgment and merging operations of the duplicate points generated by splitting the non-manifold are performed respectively, and the geometric coordinate list and the UV coordinate list are updated, and the geometric vertex index and the UV vertex index in the connection relationship are updated to obtain a non-manifold mesh with different connection relationships between the reconstructed geometric coordinates and UV coordinates.

[0253] (8) Texture map decoding module

[0254] Decode the indication information (such as the first indication information) of whether to perform lossless color space format conversion, and use the video decoder to decode the texture map. If the color space format is converted to a lossless color space format at the encoding end, the decoded texture map is in the YUV format; if the color space format is not converted to a lossless color space format at the encoding end, the decoded texture map is in the RGB format.

[0255] (9) Lossless color space format conversion module

[0256] Input: Texture map in YUV 444 format;

[0257] Output: Texture map in BGR444 format (BGR444 format is used as an example here).

[0258] If the color space format is losslessly transformed, the texture image in the input YUV color space format (i.e., the second color space format) is losslessly converted to obtain a texture image in the RGB color space format (i.e., the first color space format). There are many ways to implement the lossless color space format conversion, and this embodiment does not limit the specific implementation method of the lossless color space format conversion. For example, the inverse conversion method corresponding to the color space format conversion method shown in equations (12)-(14) is shown in (18)-(20).

[0259]

[0260] R=V+G (19)

[0261]

[0262] The inverse conversion method corresponding to the color space format conversion method shown in equations (15) to (17) is shown in (21) to (23).

[0263]

[0264] R=U+G (22)

[0265] B=V+G (23)

[0266] Among them, round() represents the round-down function.

[0267] In the related art, for the lossless coding of the texture map of the three-dimensional mesh, the texture map in the BGR444 format is directly encoded using a video encoder, resulting in low efficiency in lossless coding of the texture map. The embodiment of the present application proposes a lossless coding and decoding method for a three-dimensional mesh texture map based on Edgebreaker. Taking the first color space format as the RGB color space format and the second color space format as the YUV color space format as an example, the decoding end decodes to obtain indication information indicating whether to perform lossless color space format conversion. If the color space format of the texture map is converted to a lossless color space format at the encoding end, the texture map in the YUV format is obtained by decoding using a video decoder. The texture map is converted to a lossless color space format, and the texture map is converted from the YUV color space format (i.e., the second color space format) to the RGB color space format (i.e., the first color space format), thereby realizing lossless coding and decoding of the texture map. It should be noted that this embodiment converts the texture map from the YUV color space format to the RGB color space format without limiting the specific format of the texture map. For example, the texture map in the RGB color space format can be in the BGR444 format or the RGB444 format. If the color space format of the texture image is not converted into a lossless color space format at the encoding end, a video in RGB color space format is obtained by decoding with a video decoder. The embodiment of the present application can improve the encoding and decoding efficiency of a three-dimensional grid.

[0268] It should be noted that the 3D mesh coding method provided in the embodiment of the present application can be executed by a 3D mesh coding device, or a control module in the 3D mesh coding device for executing the 3D mesh coding method. In the embodiment of the present application, the 3D mesh coding device executing the mesh coding method is taken as an example to illustrate the 3D mesh coding device provided in the embodiment of the present application.

[0269] See also Fig.12 , Fig.12 is a structural diagram of a three-dimensional grid encoding device provided in an embodiment of the present application, such as Fig.12 As shown, the three-dimensional grid encoding device 300 includes:

[0270] The conversion module 301 is used to convert the texture image of the three-dimensional grid in the first color space format into a color space format to obtain the texture image in the second color space format;

[0271] The first encoding module 302 is used to encode the texture image in the second color space format to obtain a first texture image code stream.

[0272] Optionally, the conversion module is specifically used for:

[0273] When it is determined to perform color space format conversion on the texture map of the three-dimensional grid, the texture map of the first color space format of the three-dimensional grid is converted into a color space format to obtain a texture map of the second color space format.

[0274] Optionally, the device further comprises:

[0275] The second encoding module is used to encode the texture image in the first color space format to obtain a second texture image code stream when it is determined that the texture image of the three-dimensional grid does not need to be converted into a color space format.

[0276] Optionally, the code stream corresponding to the three-dimensional mesh includes an encoding result of first indication information, where the first indication information is used to indicate whether the encoding end performs color space format conversion on the texture map of the three-dimensional mesh;

[0277] Wherein, when the first indication information instructs the encoder to perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes the first texture map bitstream; or

[0278] When the first indication information indicates that the encoding end does not perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes a second texture map bitstream.

[0279] Optionally, when the first indication information indicates to perform color space format conversion on the texture image of the three-dimensional mesh, the first indication information is also used to indicate the conversion type of the texture image of the three-dimensional mesh from the first color space format to the second color space format.

[0280] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0281] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0282] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0283] The three-dimensional grid coding device in the embodiment of the present application can be a device, a device or electronic device with an operating system, or a component, an integrated circuit, or a chip in a terminal. The device or electronic device can be a mobile terminal or a non-mobile terminal. Exemplarily, the mobile terminal can include but is not limited to the types of terminals listed above, and the non-mobile terminal can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0284] The three-dimensional grid encoding device provided in the embodiment of the present application can achieve Figure 3 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0285] It should be noted that the 3D mesh decoding method provided in the embodiment of the present application can be executed by a 3D mesh decoding device, or a control module in the 3D mesh decoding device for executing the 3D mesh decoding method. In the embodiment of the present application, the 3D mesh decoding device executing the mesh decoding method is taken as an example to illustrate the 3D mesh decoding device provided in the embodiment of the present application.

[0286] See also Fig.13 , Fig.13 is a structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application, such as Fig.13 As shown, the three-dimensional grid decoding device 400 includes:

[0287] A first decoding module 401 is used to decode a first texture image bitstream in a bitstream corresponding to a three-dimensional grid to obtain a texture image in a second color space format;

[0288] A conversion module 402, configured to perform color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0289] The reconstruction module 403 is used to perform mesh reconstruction processing based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0290] Optionally, the device further comprises:

[0291] A second decoding module, configured to decode the encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information;

[0292] The first decoding module is specifically used for:

[0293] In the case where it is determined based on the first indication information to convert the color space format of the texture map, the first texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain the texture map in the second color space format.

[0294] Optionally, the first decoding module is specifically configured to:

[0295] When it is determined based on the first indication information that the texture image is converted from the second color space format to the first color space format, the texture image in the second color space format is converted from the second color space format to the first color space format to obtain the texture image in the first color space format.

[0296] Optionally, the device further comprises:

[0297] The third decoding module is used to decode the second texture map bitstream in the bitstream corresponding to the three-dimensional grid to obtain the texture map in the first color space format when it is determined based on the first indication information that the texture map is not to be converted into a color space format.

[0298] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0299] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0300] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0301] The three-dimensional grid decoding device in the embodiment of the present application can be a device, a device or electronic device with an operating system, or a component, an integrated circuit, or a chip in a terminal. The device or electronic device can be a mobile terminal or a non-mobile terminal. Exemplarily, the mobile terminal can include but is not limited to the types of terminals listed above, and the non-mobile terminal can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0302] The three-dimensional grid decoding device provided in the embodiment of the present application can achieve Figure 4 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0303] Alternatively, if Fig.14 As shown, the embodiment of the present application further provides a communication device 500, including a processor 501 and a memory 502, and the memory 502 stores a program or instruction that can be run on the processor 501. For example, when the communication device 500 is an encoding end device, the program or instruction is executed by the processor 501 to implement the various steps of the above-mentioned three-dimensional grid encoding method embodiment, and can achieve the same technical effect. When the communication device 500 is a decoding end device, the program or instruction is executed by the processor 501 to implement the various steps of the above-mentioned three-dimensional grid decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0304] The embodiment of the present application also provides an electronic device, including a processor and a communication interface, wherein the processor is used to: perform color space format conversion on a texture image of a first color space format of a three-dimensional grid to obtain a texture image of a second color space format; and perform encoding processing on the texture image of the second color space format to obtain a first texture image code stream. The electronic device embodiment corresponds to the above-mentioned three-dimensional grid encoding method embodiment, and each implementation process and implementation method of the above-mentioned three-dimensional grid encoding method embodiment can be applied to the electronic device embodiment and can achieve the same technical effect.

[0305] The embodiment of the present application also provides an electronic device, including a processor and a communication interface, wherein the processor is used to: decode a first texture image code stream in a code stream corresponding to a three-dimensional mesh to obtain a texture image in a second color space format; perform color space format conversion on the texture image in the second color space format to obtain a texture image in a first color space format; perform mesh reconstruction based on the texture image in the first color space format to obtain a reconstructed three-dimensional mesh. This electronic device embodiment corresponds to the above-mentioned three-dimensional mesh decoding method embodiment, and each implementation process and implementation method of the above-mentioned three-dimensional mesh decoding method embodiment can be applied to this electronic device embodiment and can achieve the same technical effect.

[0306] Specifically, the electronic device may be a terminal. Fig.15 A schematic diagram of the hardware structure of a terminal for implementing an embodiment of the present application.

[0307] The terminal 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609 and at least some of the components of a processor 610.

[0308] Those skilled in the art will appreciate that the terminal 600 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 610 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.15 The terminal structure shown in the figure does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0309] It should be understood that in the embodiment of the present application, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042, and the GPU 6041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 6071 and at least one of other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0310] In the embodiment of the present application, after receiving downlink data from the network side device, the RF unit 601 can transmit the data to the processor 610 for processing; in addition, the RF unit 601 can send uplink data to the network side device. Generally, the RF unit 601 includes but is not limited to an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0311] The memory 609 can be used to store software programs or instructions and various data. The memory 609 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 609 may include a volatile memory or a non-volatile memory, or the memory 609 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 609 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0312] The processor 610 may include one or more processing units; optionally, the processor 610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 610.

[0313] Wherein, when the terminal is a coding end device:

[0314] The processor 610 is configured to:

[0315] Performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format;

[0316] The texture image in the second color space format is encoded to obtain a first texture image code stream.

[0317] Optionally, the processor 610 is specifically configured to:

[0318] When it is determined to perform color space format conversion on the texture map of the three-dimensional grid, the texture map of the first color space format of the three-dimensional grid is converted into a color space format to obtain a texture map of the second color space format.

[0319] Optionally, the processor 610 is further configured to:

[0320] In the case where it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream.

[0321] Optionally, the code stream corresponding to the three-dimensional mesh includes an encoding result of first indication information, where the first indication information is used to indicate whether the encoding end performs color space format conversion on the texture map of the three-dimensional mesh;

[0322] Wherein, when the first indication information instructs the encoder to perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes the first texture map bitstream; or

[0323] When the first indication information indicates that the encoding end does not perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes a second texture map bitstream.

[0324] Optionally, when the first indication information indicates to perform color space format conversion on the texture image of the three-dimensional mesh, the first indication information is also used to indicate the conversion type of the texture image of the three-dimensional mesh from the first color space format to the second color space format.

[0325] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0326] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0327] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0328] Wherein, when the terminal is a decoding end device:

[0329] The processor 610 is configured to:

[0330] Decoding a first texture image bitstream in a bitstream corresponding to the three-dimensional grid to obtain a texture image in a second color space format;

[0331] Performing color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0332] A mesh reconstruction process is performed based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0333] Optionally, the processor 610 is further configured to:

[0334] Decoding the encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information;

[0335] The processor 610 is specifically configured to:

[0336] In the case where it is determined based on the first indication information to convert the color space format of the texture map, the first texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain the texture map in the second color space format.

[0337] Optionally, the processor 610 is further configured to:

[0338] When it is determined based on the first indication information that the texture image is converted from the second color space format to the first color space format, the texture image in the second color space format is converted from the second color space format to the first color space format to obtain the texture image in the first color space format.

[0339] Optionally, the processor 610 is further configured to:

[0340] When it is determined based on the first indication information that the texture map is not to be converted into a color space format, a second texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain a texture map in the first color space format.

[0341] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0342] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0343] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0344] Specifically, the terminal of the embodiment of the present application also includes: instructions or programs stored in the memory 609 and executable on the processor 610, and the processor 610 calls the instructions or programs in the memory 609 to execute Fig.12 or Fig.13 The methods executed by the modules shown achieve the same technical effects, and therefore will not be described here in detail to avoid repetition.

[0345] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned three-dimensional mesh encoding method embodiment are implemented, or when the program or instruction is executed by a processor, the various processes of the above-mentioned three-dimensional mesh decoding method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0346] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0347] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned three-dimensional mesh encoding method embodiment, or to implement the various processes of the above-mentioned three-dimensional mesh decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0348] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0349] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0350] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0351] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A three-dimensional grid encoding method, performed by an encoding end, characterized in that: include: Performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format; The texture image in the second color space format is encoded to obtain a first texture image code stream.

2. The method according to claim 1, characterized in that The step of converting the texture image in the first color space format of the three-dimensional grid to obtain the texture image in the second color space format includes: When it is determined to perform color space format conversion on the texture map of the three-dimensional grid, the texture map of the first color space format of the three-dimensional grid is converted into a color space format to obtain a texture map of the second color space format.

3. The method according to claim 2, characterized in that The method further comprises: In the case where it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream.

4. The method according to any one of claims 1 to 3, characterized in that The bitstream corresponding to the three-dimensional grid includes an encoding result of first indication information, where the first indication information is used to indicate whether the encoding end performs color space format conversion on the texture image of the three-dimensional grid; Wherein, when the first indication information instructs the encoder to perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes the first texture map bitstream; or When the first indication information indicates that the encoding end does not perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes a second texture map bitstream.

5. The method according to claim 4, characterized in that When the first indication information indicates to perform color space format conversion on the texture image of the three-dimensional mesh, the first indication information is also used to indicate the conversion type of the texture image of the three-dimensional mesh from the first color space format to the second color space format.

6. The method according to any one of claims 1 to 5, characterized in that The amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

7. The method according to any one of claims 1 to 6, characterized in that The second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

8. The method according to any one of claims 1 to 7, characterized in that The first color space format is an RGB color space format, and the second color space format is a YUV color space format.

9. A three-dimensional grid decoding method, executed by a decoding end, characterized in that: include: Decoding a first texture image bitstream in a bitstream corresponding to the three-dimensional grid to obtain a texture image in a second color space format; Performing color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format; A mesh reconstruction process is performed based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

10. The method according to claim 9, characterized in that The method further comprises: Decoding the encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information; The decoding process of the first texture image bitstream in the bitstream corresponding to the three-dimensional grid to obtain the texture image in the second color space format includes: In the case where it is determined based on the first indication information to convert the color space format of the texture map, the first texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain the texture map in the second color space format.

11. The method according to claim 10, characterized in that The step of converting the texture image in the second color space format into a color space format to obtain the texture image in the first color space format includes: When it is determined based on the first indication information that the texture image is converted from the second color space format to the first color space format, the texture image in the second color space format is converted from the second color space format to the first color space format to obtain the texture image in the first color space format.

12. The method according to claim 10 or 11, characterized in that: The method further comprises: When it is determined based on the first indication information that the texture map is not to be converted into a color space format, a second texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain a texture map in the first color space format.

13. The method according to any one of claims 9 to 12, characterized in that: The amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

14. The method according to any one of claims 9 to 13, characterized in that: The second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

15. The method according to any one of claims 9 to 14, characterized in that: The first color space format is an RGB color space format, and the second color space format is a YUV color space format.

16. A three-dimensional grid encoding device, characterized in that: The device comprises: A conversion module, used for performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format; The first encoding module is used to encode the texture image in the second color space format to obtain a first texture image code stream.

17. A three-dimensional grid decoding device, characterized in that: The device comprises: A first decoding module is used to decode a first texture image bitstream in a bitstream corresponding to the three-dimensional grid to obtain a texture image in a second color space format; A conversion module, used for performing color space format conversion on the texture image in the second color space format to obtain the texture image in the first color space format; A reconstruction module is used to perform mesh reconstruction processing based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

18. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 8; or, the program or instruction, when executed by the processor, implements the steps of the three-dimensional mesh decoding method as described in any one of claims 9 to 15.

19. A chip, characterized in that: The chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 8, or to implement the steps of the three-dimensional mesh decoding method as described in any one of claims 9 to 15.

20. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 8 are implemented, or when the program or instruction is executed by the processor, the steps of the three-dimensional mesh decoding method as described in any one of claims 9 to 15 are implemented.

Images