Three-dimensional mesh encoding method and apparatus, three-dimensional mesh decoding method and apparatus, and device

Abstract

A three-dimensional mesh encoding method and apparatus, a three-dimensional mesh decoding method and apparatus, and a device. The three-dimensional mesh encoding method comprises: performing color space format conversion on a texture map in a first color space format of a three-dimensional mesh to obtain a texture map in a second color space format (101); and encoding the texture map in the second color space format to obtain a first texture map code stream (102). The present application supports the conversion of a texture map of a three-dimensional mesh into a texture map in another color space format for encoding, and can improve the flexibility of encoding of texture maps of three-dimensional meshes.

Description

Three-dimensional grid encoding method, three-dimensional grid decoding method, device and equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese Patent Application No. 202310795676.3 filed in China on June 30, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application belongs to the field of coding and decoding technology, and specifically relates to a three-dimensional grid coding method, a three-dimensional grid decoding method, a device and equipment. Background Art

[0004] With the rapid development of multimedia technology, 3D models have become a new generation of digital media, following audio, images, and video. Three-dimensional meshes and point clouds are two commonly used representations of 3D models. Compared to traditional multimedia such as images and videos, 3D mesh models offer greater interactivity and realism, and thus have a wide range of applications.

[0005] In the related art, when encoding a texture map of a three-dimensional mesh, the texture map in the Blue-Green-Red (BGR) 444 format is directly encoded, which has poor flexibility in encoding the texture map of the three-dimensional mesh.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a three-dimensional mesh encoding method, a three-dimensional mesh decoding method, an apparatus and a device, which can solve the problem of poor flexibility in encoding the texture map of the three-dimensional mesh.

[0008] In a first aspect, a three-dimensional mesh encoding method is provided, which is performed by an encoding end and includes:

[0009] Performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format;

[0010] The texture image in the second color space format is encoded to obtain a first texture image code stream.

[0011] In a second aspect, a three-dimensional mesh decoding method is provided, which is executed by a decoding end and includes:

[0012] Decoding a first texture image code stream in a code stream corresponding to the three-dimensional grid to obtain a texture image in a second color space format;

[0013] Performing color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0014] A mesh reconstruction process is performed based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0015] In a third aspect, a three-dimensional grid encoding device is provided, the device comprising:

[0016] a conversion module, configured to convert a texture image of a three-dimensional grid in a first color space format into a color space format to obtain a texture image in a second color space format;

[0017] The first encoding module is used to encode the texture image in the second color space format to obtain a first texture image code stream.

[0018] In a fourth aspect, a three-dimensional grid decoding device is provided, the device comprising:

[0019] A first decoding module is used to decode a first texture map code stream in a code stream corresponding to the three-dimensional grid to obtain a texture map in a second color space format;

[0020] a conversion module, configured to perform color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0021] The reconstruction module is used to perform mesh reconstruction processing based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0022] In a fifth aspect, an electronic device is provided, which terminal includes a processor, a memory, and a program or instruction stored in the memory and runnable on the processor, wherein when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented; or, when the program or instruction is executed by the processor, the steps of the method described in the second aspect are implemented.

[0023] In a sixth aspect, an electronic device is provided, comprising a processor and a communication interface, wherein the processor is used to: convert the texture image of a three-dimensional grid in a first color space format into a color space format to obtain a texture image in a second color space format; and encode the texture image in the second color space format to obtain a first texture image code stream.

[0024] In the seventh aspect, an electronic device is provided, including a processor and a communication interface, wherein the processor is used to: decode the first texture map code stream in the code stream corresponding to the three-dimensional grid to obtain a texture map in a second color space format; perform color space format conversion on the texture map in the second color space format to obtain a texture map in a first color space format; perform grid reconstruction processing based on the texture map in the first color space format to obtain a reconstructed three-dimensional grid.

[0025] In an eighth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the three-dimensional mesh encoding method as described in the first aspect are implemented, or when the program or instruction is executed by a processor, the steps of the three-dimensional mesh decoding method as described in the second aspect are implemented.

[0026] In the ninth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0027] In the tenth aspect, a computer program / program product is provided, which is stored in a non-volatile storage medium, and the program / program product is executed by at least one processor to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0028] In an embodiment of the present application, a texture image in a first color space format of a three-dimensional mesh is converted to a second color space format. The texture image in the second color space format is then encoded to obtain a first texture image code stream. In this way, by converting the texture image in a color space format during the three-dimensional mesh encoding process, the texture image can be converted to a texture image in another color space format for encoding, thereby increasing the flexibility of the three-dimensional mesh texture image encoding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] FIG1 is a schematic diagram of five modes of Edgebreaker in the related art;

[0030] FIG2 is a schematic diagram of a vertex traversal process and an operation mode string in Edgebreaker in the related art;

[0031] FIG3 is a flow chart of a three-dimensional grid coding method provided in an embodiment of the present application;

[0032] FIG4 is a flow chart of a three-dimensional grid decoding method provided in an embodiment of the present application;

[0033] FIG5 is a schematic diagram of a three-dimensional grid coding framework provided in an embodiment of the present application;

[0034] FIG6 is a schematic diagram of a three-dimensional grid decoding framework provided in an embodiment of the present application;

[0035] FIG7 is a schematic diagram of a manifold grid provided in an embodiment of the present application;

[0036] FIG8 is a schematic diagram of a Corner relationship provided in an embodiment of the present application;

[0037] FIG9 is a schematic diagram of a grid traversal provided in an embodiment of the present application;

[0038] FIG10 is a schematic diagram of a three-dimensional grid provided in an embodiment of the present application;

[0039] FIG11 is a schematic diagram of UV coordinate prediction based on three-dimensional to two-dimensional projection provided by an embodiment of the present application;

[0040] FIG12 is a schematic structural diagram of a three-dimensional grid encoding device provided in an embodiment of the present application;

[0041] FIG13 is a schematic structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application;

[0042] FIG14 is a schematic structural diagram of a communication device provided in an embodiment of the present application;

[0043] FIG15 is a schematic structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0045] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in this application represents at least one of the connected objects. For example, "A or B" covers three options, namely, Option 1: including A but not including B; Option 2: including B but not including A; Option 3: including both A and B. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0046] The term "indication" in this application can be either a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, the operation to be performed, or the requested result, etc. in the instruction sent; an indirect indication can be understood as the receiver determining the corresponding information based on the instruction sent by the sender, or making a judgment and determining the operation to be performed or the requested result, etc. based on the judgment result.

[0047] The encoding and decoding end corresponding to the encoding and decoding method in the embodiment of the present application can be a terminal, which can also be called a terminal device or user equipment (UE). The terminal can be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer) or a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile Internet device (Mobile Internet Device, MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device (Wearable Device) or a vehicle-mounted device (VUE), a pedestrian terminal (Pedestrian User Equipment, PUE) and other terminal-side devices. Wearable devices include: smart watches, bracelets, headphones, glasses, etc. It should be noted that the specific type of the terminal is not limited in the embodiment of the present application.

[0048] For ease of understanding, some of the contents involved in the embodiments of this application are described below:

[0049] 1. 3D Grid

[0050] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have become the next generation of digital media, following audio, images, and video. Three-dimensional meshes and point clouds are two commonly used representations of 3D models. Compared to traditional multimedia such as images and videos, 3D mesh models offer greater interactivity and realism, leading to their increasing application in various fields, including commerce, manufacturing, construction, education, medicine, entertainment, art, and the military.

[0051] With the increasing demand for visually appealing 3D mesh models and the emergence of more sophisticated 3D scanning technologies and 3D modeling software, the size and complexity of 3D mesh data captured by 3D scanning equipment or 3D modeling software are rapidly increasing. Therefore, efficiently compressing 3D mesh data is crucial for its convenient transmission, storage, and processing.

[0052] A 3D mesh often contains three main types of information: topology, geometry, and attributes. Topology, also known as connectivity information, describes the connections between elements such as vertices and patches in the mesh; geometry is the 3D coordinates of all vertices in the mesh; and attribute information records other information attached to the mesh, such as normals, texture coordinates, and color. 3D mesh data compression often targets these three types of information, individually based on their data characteristics. Furthermore, for 3D meshes with texture maps, the texture maps themselves must be compressed.

[0053] Draco is a library for compressing and decompressing three-dimensional (3D) geometric meshes and point clouds. It aims to improve the storage and transmission of 3D graphics and significantly accelerate the encoding, transmission, and decoding of 3D data. Draco supports compression of 3D mesh geometry, connectivity, and attribute information. Draco supports both lossy and near-lossless modes. Furthermore, Draco's Edgebreaker method for encoding connectivity relationships is currently one of the most efficient methods for encoding 3D mesh connectivity information.

[0054] However, Edgebreaker requires the mesh to be encoded to have a manifold structure. For meshes with non-manifold structures, Draco must split them into manifold structures to correctly encode them. However, Draco does not merge these split structures at the decoder end. This results in the decoder outputting meshes with more split points than the original mesh input to the encoder. This prevents Draco from losslessly encoding these meshes with non-manifold structures.

[0055] The Moving Picture Experts Group (MPEG), an international standards organization specializing in video and image processing, is developing a new standard for dynamic 3D mesh compression, called Video-based dynamic mesh coding (V-DMC or VDMC). Currently, they are also adopting an Edgebreaker-based solution for encoding static 3D meshes, temporarily reusing the Draco codec. MPEG is also exploring the implementation of an Edgebreaker-based 3D mesh codec provided by MPEG to compress 3D mesh geometry, connectivity, and attribute information. Because Edgebreaker requires the mesh to be encoded to be a manifold structure, currently proposed solutions involve splitting non-manifold 3D meshes into manifold structures for encoding. Therefore, in lossless mode, the Edgebreaker-based 3D mesh codec implementation provided by MPEG records and encodes the duplicate point information generated by non-manifold splitting, and adds an identifier to each vertex of the split manifold mesh to determine whether it is a duplicate point generated by non-manifold splitting. Based on this identifier and the duplicate point information generated by non-manifold splitting, the non-manifold structure of the original 3D mesh is restored at the decoding end to achieve lossless encoding of the 3D mesh.

[0056] The Edgebreaker-based 3D mesh compression tool currently provided by MPEG encodes and stores the connectivity information, geometric information, and attribute information (e.g., UV coordinates) of a 3D mesh separately. UV coordinates are two-dimensional coordinates, with U being the horizontal direction and V being the vertical direction. The core module, the one that encodes the connectivity information, uses the Edgebreaker algorithm. Taking UV coordinates as an example, conventional compression methods are used to encode the geometric information and UV coordinates, namely, data quantization, predictive compression (e.g., parallelogram prediction), and entropy coding. Because the tool uses a connectivity-driven encoding method, the encoding of the geometric information and UV coordinates follows the encoding order of the connectivity information. This method implicitly embeds the vertex order of the connectivity encoding within the vertex order of the geometric information, avoiding the need to transmit the vertex order of the connectivity encoding separately, thereby saving this bit overhead.

[0057] 2. Edgebreaker

[0058] The Edgebreaker method (EB) is a 3D mesh connectivity encoding method with advantages such as good compression performance, ease of implementation, and an upper limit on the compression ratio. The Edgebreaker method itself only describes how to compress the 3D mesh's connectivity information; further compression is achieved through geometric information compression and entropy coding.

[0059] Edgebreaker encoding can compress triangular meshes that are homeomorphic to a sphere to a compression efficiency of 2 bits per triangle or less. The encoding algorithm uses five different modes (called C, L, E, R, and S) to access each triangle of the mesh in depth-first order. Each triangle is labeled according to the mode it is in, generating a CLERS string that provides a compact representation of the mesh's connectivity.

[0060] Figure 1 shows the five modes of the Edgebreaker method. The Edgebreaker method divides the mesh into a traversed portion and an untraversed portion, with the boundary between the two portions being called the active boundary. During the Edgebreaker encoding process, the triangle to be traversed is accessed through the active edges on the active boundary, and the mode to be used is determined based on the relationship between the active edge and the triangle in which it resides. The other vertex in the triangle containing the active edge is called the third vertex. If the third vertex is not on the active boundary, the current triangle is marked in C mode. If the third vertex is on the active boundary and is the next vertex of the current active edge in counterclockwise order, the current triangle is marked in R mode. If the third vertex is on the active boundary and is the previous vertex of the current active edge in counterclockwise order, the current triangle is marked in L mode. If the third vertex is on the active boundary and is both the previous vertex and the next vertex of the current active edge in counterclockwise order, the current triangle is marked in E mode. If the third vertex is on the active edge, but is neither the previous vertex nor the next vertex of the current active edge in counterclockwise order, then the current triangle is marked as S mode.

[0061] After marking each triangle, the active boundary is updated and the next active edge is selected according to certain rules. After traversing all triangles, the obtained CLERS string is entropy encoded to achieve higher compression efficiency.

[0062] Figure 2 shows a schematic diagram of a two-dimensional grid using EB coding. According to the EB coding rules, the final entropy coding pattern codeword is CCRRSLCRSERRELCRRRCRRRE.

[0063] Since the five modes of the Edgebreaker method cannot process non-manifold meshes, the mesh must be converted into a manifold mesh before using the Edgebreaker method.

[0064] In the related art, when encoding a texture map of a three-dimensional mesh, the texture map in the BGR444 format is directly encoded, resulting in low encoding and decoding efficiency of the three-dimensional mesh.

[0065] The following, in conjunction with the accompanying drawings, describes in detail the three-dimensional grid encoding method, three-dimensional grid decoding method, device and equipment provided by the embodiments of the present application through some embodiments and their application scenarios.

[0066] 3 , which is a flow chart of a three-dimensional grid encoding method provided in an embodiment of the present application, which can be applied to an encoding end. As shown in FIG3 , the three-dimensional grid encoding method includes the following steps:

[0067] Step 101: Perform color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format.

[0068] The first color space format may be the original color space format of the texture map of the three-dimensional grid. The first color space format may be an RGB color space format. For example, the first color space format may be BGR444, BGR555, RGB555, or RGB 565, etc. Alternatively, the first color space format may be a YUV color space format. Alternatively, the first color space format may be a CMY color space format. Alternatively, the first color space format may be a CMYK color space format. Alternatively, the first color space format may be an HSV color space format. This embodiment does not limit the first color space format. RGB refers to red, green, and blue. CMY refers to cyan, magenta, and yellow. In YUV, Y represents luminance, and UV represents chrominance and chroma, respectively. CMYK refers to cyan, magenta, yellow, and black. HSV refers to hue-saturation-value.

[0069] In addition, the second color space format can be a color space format after color space format conversion. The second color space format and the first color space format are different color space formats. The second color space format can be a YUV color space format. For example, the second color space format can be YUV444, YUV420, or YUV422, etc.; or, the second color space format can be an RGB color space format; the second color space format can also be an HSV color space format; or the second color space format can also be a CMY color space format; or the second color space format can also be a Lab color space format; etc. This embodiment does not limit the second color space format. In Lab, L is brightness, and a and b are two color channels.

[0070] In one embodiment, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different formats of the same color space.

[0071] It should be noted that different color space format conversions can be performed for different application scenarios. To improve encoding and decoding efficiency, the amount of relevant information of the color space of the texture image in the second color space format can be less than the amount of relevant information of the texture image in the first color space format. For example, the texture image in the second color space format is a texture image obtained by removing certain redundant information from the texture image in the first color space format; or, the texture image in the second color space format is a texture image obtained by removing the correlation between color components from the texture image in the first color space format.

[0072] It should be noted that the color space format conversion of the texture image of the three-dimensional grid in the first color space format to obtain the texture image of the second color space format may be a lossless color space format conversion of the texture image of the three-dimensional grid in the first color space format to obtain the texture image of the second color space format.

[0073] Step 102: Encode the texture image in the second color space format to obtain a first texture image code stream.

[0074] In an embodiment of the present application, a texture image in a first color space format of a three-dimensional mesh is converted to a second color space format. The texture image in the second color space format is then encoded to obtain a first texture image code stream. In this way, by converting the texture image in a color space format during the three-dimensional mesh encoding process, the texture image can be converted to a texture image in another color space format for encoding, thereby increasing the flexibility of the three-dimensional mesh texture image encoding process.

[0075] Optionally, performing color space format conversion on the texture image of the three-dimensional grid in the first color space format to obtain the texture image in the second color space format includes:

[0076] When it is determined to perform color space format conversion on the texture map of the three-dimensional grid, the texture map of the three-dimensional grid in the first color space format is converted into a color space format to obtain a texture map in the second color space format.

[0077] The encoder can determine whether to perform color space format conversion on the texture image of the three-dimensional mesh. The encoder can determine whether to perform color space format conversion on the texture image of the three-dimensional mesh based on different encoding requirements. For example, when higher encoding and decoding efficiency is required, it can be determined to perform color space format conversion on the texture image of the three-dimensional mesh.

[0078] In this embodiment, when it is determined that the color space format conversion is performed on the texture map of the three-dimensional grid, the color space format conversion is performed on the texture map of the first color space format of the three-dimensional grid to obtain the texture map of the second color space format. In this way, by judging whether the color space format conversion is performed on the texture map of the three-dimensional grid, the encoding end can be supported to selectively convert the color space format of the texture map, which can improve the flexibility of encoding the texture map during the three-dimensional grid encoding process.

[0079] Optionally, the method further includes:

[0080] When it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream.

[0081] The encoder can determine whether to perform color space format conversion on the texture image of the three-dimensional mesh. The encoder can determine whether to perform color space format conversion on the texture image of the three-dimensional mesh based on different encoding requirements. For example, in scenarios where encoding and decoding efficiency is not required, it can be determined not to perform color space format conversion on the texture image of the three-dimensional mesh.

[0082] In this embodiment, when it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream. In this way, by judging whether the color space format conversion is performed on the texture map of the three-dimensional grid, the encoding end can be supported to selectively not perform the color space format conversion on the texture map, which can further improve the flexibility of encoding the texture map during the three-dimensional grid encoding process.

[0083] Optionally, the code stream corresponding to the three-dimensional mesh includes an encoding result of first indication information, where the first indication information is used to instruct the encoder whether to perform color space format conversion on the texture map of the three-dimensional mesh;

[0084] Wherein, when the first indication information instructs the encoder to perform color space format conversion on the texture map of the three-dimensional mesh, the code stream corresponding to the three-dimensional mesh also includes the first texture map code stream; or

[0085] When the first indication information instructs the encoder not to perform color space format conversion on the texture map of the three-dimensional mesh, the code stream corresponding to the three-dimensional mesh further includes a second texture map code stream.

[0086] For example, if the first indication information is "0", it instructs the encoder to not perform color space format conversion on the texture image of the three-dimensional mesh; otherwise, the first indication information instructs the encoder to perform color space format conversion on the texture image of the three-dimensional mesh.

[0087] It should be noted that, when the first indication information indicates that the color space format conversion of the texture map of the three-dimensional mesh is performed, the first indication information can also be used to indicate the conversion type of the texture map of the three-dimensional mesh from the first color space format to the second color space format; thereby, the decoding end can determine what type of color space format conversion the encoding end has performed based on the first indication information, so that the decoding end can perform the inverse conversion of the corresponding color space format to restore the original texture map.

[0088] In this embodiment, the code stream corresponding to the three-dimensional mesh includes the first texture map code stream or the second texture map code stream, and the code stream corresponding to the three-dimensional mesh also includes the encoding result of the first indication information, and the first indication information is used to indicate to the encoding end whether to perform color space format conversion on the texture map of the three-dimensional mesh. In this way, the encoding end is indicated by the first indication information whether to perform color space format conversion on the texture map of the three-dimensional mesh. The decoding end can determine whether to perform color space format conversion on the decoded texture map through the first indication information, thereby facilitating the decoding end to reconstruct the three-dimensional mesh.

[0089] Optionally, when the first indication information indicates that the color space format conversion of the texture image of the three-dimensional mesh is performed, the first indication information is also used to indicate the conversion type of the texture image of the three-dimensional mesh from the first color space format to the second color space format.

[0090] The first indication information may be used to instruct the encoding end to convert the texture image of the three-dimensional mesh from the first color space format to the second color space format. In one embodiment, the first indication information may be a value in a preset list, where different values ​​in the preset list represent different color space format conversion types. For example, if the first indication information is "0", it indicates that the encoding end does not perform color space format conversion on the texture map of the three-dimensional grid; if the first indication information is "1", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR444 format of the RGB color space to the YUV444 format of the YUV color space; if the first indication information is "2", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR444 format of the RGB color space to the RGB444 format of the RGB color space; if the first indication information is "3", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR444 format of the RGB color space to the YUV420 format of the YUV color space; if the first indication information is "4", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR444 format of the RGB color space to the YUV422 format of the YUV color space; if the first indication information is "5", it indicates that the texture map of the three-dimensional grid at the encoding end is converted from the BGR555 format of the RGB color space to the YUV444 format of the YUV color space; and so on.

[0091] In this embodiment, when the first indication information indicates that the color space format conversion of the texture map of the three-dimensional mesh is to be performed, the first indication information is also used to indicate the conversion type of the texture map of the three-dimensional mesh from the first color space format to the second color space format, so that the decoding end can determine what type of color space format conversion the encoding end has performed through the first indication information, so that the decoding end can perform the inverse conversion of the corresponding color space format to restore the original texture map.

[0092] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0093] The texture image in the second color space format may be a texture image obtained by removing certain redundant information from the texture image in the first color space format; or, the texture image in the second color space format may be a texture image obtained by removing the correlation between color components from the texture image in the first color space format.

[0094] In related art, when encoding a 3D mesh's texture map, the texture map is directly encoded in the BGR444 format. Because the decoding end has different decoding efficiencies for videos in different color space formats, and the decoding end has lower decoding efficiency for BGR444 format videos, the encoding and decoding efficiency of the 3D mesh is low. In this embodiment, during the 3D mesh encoding process, the texture map of the 3D mesh is converted to a texture map in a color space format with higher encoding and decoding efficiency by performing a color space format conversion on the texture map of the 3D mesh. This improves the encoding and decoding efficiency of the 3D mesh.

[0095] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0096] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0097] In this embodiment, the first color space format is an RGB color space format, and the second color space format is a YUV color space format, so that when encoding the texture map of the three-dimensional mesh, the texture map of the RGB color space is converted into a texture map of the YUV color space for encoding. Since the efficiency of decoding the video in the YUV color space at the decoding end is higher than the efficiency of decoding the video in the RGB color space at the decoding end, the encoding and decoding efficiency of the three-dimensional mesh can be improved through color space format conversion.

[0098] The present application also provides a three-dimensional grid coding method, the method comprising:

[0099] When it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream.

[0100] Optionally, the code stream corresponding to the three-dimensional mesh includes the encoding result of the second texture map code stream and first indication information, and the first indication information instructs the encoding end not to perform color space format conversion on the texture map of the three-dimensional mesh.

[0101] 4 is a flowchart of a three-dimensional grid decoding method provided in an embodiment of the present application, which can be applied to a decoding end device. As shown in FIG4 , the three-dimensional grid decoding method includes the following steps:

[0102] Step 201: Decode a first texture image code stream in a code stream corresponding to a three-dimensional grid to obtain a texture image in a second color space format;

[0103] Step 202: performing color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0104] Step 203: Perform mesh reconstruction based on the texture image in the first color space format to obtain a reconstructed three-dimensional mesh.

[0105] Optionally, the method further includes:

[0106] decoding an encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information;

[0107] The decoding process of the first texture map code stream in the code stream corresponding to the three-dimensional grid to obtain the texture map in the second color space format includes:

[0108] In the case where it is determined based on the first indication information that the color space format conversion is to be performed on the texture map, the first texture map code stream in the code stream corresponding to the three-dimensional grid is decoded to obtain the texture map in the second color space format.

[0109] Optionally, performing color space format conversion on the texture image in the second color space format to obtain the texture image in the first color space format includes:

[0110] When it is determined based on the first indication information that the texture image is converted from the second color space format to the first color space format, the texture image in the second color space format is converted from the second color space format to the first color space format to obtain the texture image in the first color space format.

[0111] Optionally, the method further includes:

[0112] When it is determined based on the first indication information that the texture map is not to be converted into a color space format, a second texture map code stream in the code stream corresponding to the three-dimensional grid is decoded to obtain a texture map in the first color space format.

[0113] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0114] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0115] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0116] It should be noted that this embodiment is an implementation of the decoding side corresponding to the embodiment shown in Figure 3. Its specific implementation can refer to the relevant description of the embodiment shown in Figure 3. In order to avoid repetition, this embodiment will not be repeated, and the same beneficial effects can still be achieved.

[0117] The present invention also provides a three-dimensional grid decoding method, which includes:

[0118] Decoding the encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information;

[0119] When the first indication information indicates that the color space format conversion is not to be performed on the texture map, decoding the second texture map bitstream in the bitstream corresponding to the three-dimensional grid to obtain the texture map in the first color space format;

[0120] A mesh reconstruction process is performed based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0121] This application also proposes a framework and method for lossless encoding and decoding of 3D mesh texture images based on Edgebreaker. As shown in Figure 5, this encoding framework primarily includes a preprocessing module (including a module for splitting non-manifold structures), a connection relationship encoding module, a geometry information encoding module, an attribute information encoding module, a texture image lossless color space format conversion module, and a texture image encoding module. Edgebreaker can be used to losslessly encode 3D meshes.

[0122] Taking the case where the first color space format is RGB, the second color space format is YUV, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture) as an example, the process of 3D mesh encoding at the encoder is as follows:

[0123] (1) At the encoding end, it is determined whether the geometric coordinates and UV coordinates have the same connection relationship and this information is recorded. The information indicating whether the geometric coordinates and UV coordinates in a grid have the same connection relationship can be represented, for example, by setting an identifier indicating whether the geometric coordinates and UV coordinates in a grid have the same connection relationship. This representation method is not limited herein.

[0124] (2) Split the non-manifold points and non-manifold edges that may exist in the mesh and convert the mesh into a manifold mesh. Splitting the non-manifold structure is achieved by adding duplicate points with the same geometric information and attribute information. When splitting the non-manifold, if the geometric coordinates and UV coordinates have the same connection relationship, that is, the geometric coordinates and UV coordinates are in a one-to-one correspondence, then only one set of duplicate point index information generated by splitting the non-manifold structure needs to be recorded; if the geometric coordinates and UV coordinates have different connection relationships, such as one geometric coordinate vertex corresponds to multiple texture coordinate vertices, then the geometric duplicate point index information and UV duplicate point index information generated by splitting the non-manifold structure need to be recorded separately. The representation of the duplicate point index information generated by splitting the non-manifold structure can be the target vertex index that needs to be merged when restoring the non-manifold, or it can be the duplicate point group index: points with the same index belong to the same duplicate point group, that is, they have the same vertex information. There is no restriction on its representation here.

[0125] (3) Encode information indicating whether the geometric coordinates and UV coordinates have the same connection relationship. Use the Edgebreaker method to encode the connection relationship to obtain a CLERS pattern string that can concisely represent the connection relationship. Use entropy coding to compress the pattern string to obtain a sub-code stream of the connection relationship. Use methods such as parallelogram prediction to encode the geometric information of the mesh to obtain a geometric information sub-code stream. If the mesh has attribute information such as UV coordinates, use methods such as similar triangle prediction to encode the UV coordinates of the mesh to obtain an attribute information sub-code stream.

[0126] (4) Encode an indication information (i.e., the first indication information) to indicate whether to perform lossless color space format conversion on the texture image. If the texture image is to be converted to a lossless color space format, the texture image is converted from the RGB color space format (i.e., the first color space format) to the YUV color space format (i.e., the second color space format), and the texture image in the YUV format is encoded using a video encoder to obtain a texture image sub-code stream. Here, only the conversion from the RGB color space format to the YUV color space is emphasized, and the specific format of the texture image is not limited. For example, the texture image in the RGB color space can be in the BGR444 format or the RGB444 format. If the texture image is not to be converted to a color space format, the video in the RGB color space is directly encoded using a video encoder for lossless encoding.

[0127] (5) The encoding of non-manifold structure information includes three parts of information to be encoded: information indicating whether a non-manifold structure exists in the grid, identification information of whether a vertex is a duplicate point generated by removing the non-manifold, and index information of duplicate points generated by removing the non-manifold structure. Among them, the information indicating whether a non-manifold structure exists in the grid can be represented by setting an identification of whether a non-manifold structure exists in the grid, or by the number of duplicate points generated by removing the non-manifold structure in the grid. The representation method is not limited here. When encoding non-manifold structure information, the information indicating whether a non-manifold structure exists in the grid is first encoded. If there is a non-manifold structure in the mesh, it is divided into two cases: if the geometric coordinates and UV coordinates have the same connection relationship, an identifier is set for each vertex to indicate whether it is a duplicate point generated by splitting the non-manifold, and this information and the index information of the duplicate points generated by splitting the non-manifold are encoded; if the geometric coordinates and UV coordinates have different connection relationships, an identifier is set for each geometric vertex and each UV vertex to indicate whether it is a geometric duplicate point or UV duplicate point generated by splitting the non-manifold, and the non-manifold identification information of the geometric vertex and the index information of the geometric duplicate point generated by splitting the non-manifold, the non-manifold identification information of the UV vertex and the index information of the UV duplicate point generated by splitting the non-manifold are encoded respectively.

[0128] Taking the case where the first color space format is RGB, the second color space format is YUV, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture), as an example, as shown in FIG6 , the process of three-dimensional mesh decoding at the decoding end is as follows:

[0129] (1) At the decoding end, decoding obtains information indicating whether the geometric coordinates and UV coordinates in the grid have the same connection relationship. Entropy decoding obtains the CLERS pattern string, and the pattern string is used to reconstruct the connection relationship between the geometric coordinates and UV coordinates. The geometric information of the grid is decoded using methods such as parallelogram inverse prediction, and the UV coordinates of the grid are decoded using methods such as similar triangle inverse prediction. When decoding non-manifold structure information, the indication information of whether there is a non-manifold structure in the grid is first decoded. If there is a non-manifold structure in the grid, further decoding is performed to obtain the non-manifold identification information of the vertex and the index information of the duplicate points generated by the non-manifold removal. The duplicate points generated by the non-manifold removal are merged, and the connection relationship is adjusted to restore the non-manifold structure in the grid.

[0130] (2) Decode the indication information (such as the first indication information) of whether to perform lossless color space format conversion, and use the video decoder to decode the texture map. If the color space format of the texture map is converted to a lossless color space format at the encoding end, the decoded texture map is a YUV format texture map. According to the indication information, the texture map is converted to a lossless color space format at the decoding end, and the texture map is converted from the YUV color space format (i.e., the second color space format) to the RGB color space format (i.e., the first color space format), thereby realizing lossless encoding and decoding of the texture map. Here, only the conversion from the YUV color space format to the RGB color space format is emphasized, and the specific format of the texture map is not limited. For example, the texture map can be in the BGR444 format or the RGB444 format in the RGB color space. If the color space format of the texture map is not converted to a lossless color space format at the encoding end, the decoded texture map is an RGB format texture map. In this case, there is no need to perform color space format conversion at the decoding end.

[0131] Through the encoding and decoding framework of the embodiment of the present application, lossless encoding and decoding of three-dimensional meshes can be completed.

[0132] The three-dimensional grid coding framework of an embodiment of the present application is shown in Figure 5. In this embodiment, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture) as an example for explanation.

[0133] (1) In the preprocessing step, first determine whether the geometric coordinates and UV coordinates have the same connection relationship, then split the input grid with non-manifold structure to obtain a manifold grid, and record the duplicate point information generated by splitting the non-manifold. It should be noted that this step can also include other preprocessing modules required for encoding, such as filtering out duplicate points and adding virtual points. For ease of explanation, Figure 5 only lists the modules for splitting non-manifold structures targeted by the embodiment of the present application.

[0134] (2) Encode information indicating whether the geometric coordinates and UV coordinates in the grid have the same connection relationship; for manifold grids, use the Edgebreaker method to encode the connection information, obtain a pattern string, and perform entropy encoding on it; encode the geometric information of the manifold grid, for example, the parallelogram prediction coding method can be used, and the encoding method of the geometric information is not limited here; if the grid has attribute information such as UV coordinates, it can be encoded using methods such as similar triangle prediction coding, and the encoding method of the attribute information is not limited here; when encoding non-manifold structure information, first encode whether there is non-manifold representation information in the grid. If a non-manifold structure exists in the mesh, two cases are classified based on whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship: if the geometric coordinates and UV coordinates in the mesh have the same connection relationship, that is, the geometric coordinates and UV coordinates have a one-to-one correspondence, the non-manifold identification information of a group of vertices and the index information of duplicate points generated by removing the non-manifold are encoded in the same coding order; if the geometric coordinates and UV coordinates in the mesh have different connection relationships, that is, the geometric coordinates and UV coordinates do not have a one-to-one correspondence, the non-manifold identification information of the geometric vertices and the index information of the geometric duplicate points generated by removing the non-manifold, and the non-manifold identification information of the UV vertices and the index information of the UV duplicate points generated by removing the non-manifold are encoded in the coding order of the geometric coordinates and UV coordinates respectively. The non-manifold identification information of the vertex here is obtained by encoding a flag bit for each vertex in the split manifold mesh to identify whether the vertex is a duplicate point generated by removing the non-manifold.

[0135] (3) An indication information can be encoded to indicate whether to perform lossless color space format conversion on the texture image. If the texture image is to be converted to a lossless color space format, the texture image is converted from the RGB color space format (i.e., the first color space format) to the YUV color space format (i.e., the second color space format) in the lossless color space format conversion module, and the texture image in the YUV format is encoded using a video encoder to obtain a texture image sub-stream. If the texture image is not to be converted to a color space format, the video in the RGB color space format is directly losslessly encoded using a video encoder. Finally, the multiple streams are mixed to obtain the final output stream.

[0136] It should be noted that the order of encoding the 3D mesh's connectivity relationships and vertex information is not emphasized. Geometric information, attribute information, and non-manifold structural information can be encoded simultaneously with the connectivity relationships. Alternatively, the geometric information, attribute information, and non-manifold structural information can be encoded sequentially according to the order in which the connectivity relationships are encoded. When encoding the geometry and attribute information of vertices, the geometry and attribute information of duplicate points generated by non-manifold removal can be skipped and encoded only once, or not skipped at all. Whether or not to skip encoding the geometry and attribute information of duplicate points generated by non-manifold removal is not emphasized here.

[0137] The three-dimensional grid decoding framework of an embodiment of the present application is shown in Figure 6. In this embodiment, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply referred to as UV or texture) as an example for explanation.

[0138] (1) First, decompose the code stream to obtain each sub-code stream. Then decode the information indicating whether the geometric coordinates and UV coordinates in the grid have the same connection relationship; entropy decode the connection relationship sub-code stream to obtain the pattern string. If the geometric coordinates and UV coordinates have the same connection relationship, only one set of connection relationships needs to be reconstructed; if the geometric coordinates and UV coordinates have different connection relationships, the connection relationship of the geometric coordinates and the connection relationship of the UV coordinates need to be reconstructed respectively; use the decoding method corresponding to the encoding end to decode the geometric information of the grid; use the decoding method corresponding to the encoding end to decode the attribute information of the grid; when decoding non-manifold structure information, first decode whether there is non-manifold information in the grid. If there is non-manifold structure in the grid, then Depending on whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship, there are two cases: if the geometric coordinates and UV coordinates in the mesh have the same connection relationship, that is, the geometric coordinates and UV coordinates are in a one-to-one correspondence, further decoding is performed to obtain non-manifold identification information of a set of vertices and duplicate point index information generated by removing the non-manifold; if the geometric coordinates and UV coordinates in the mesh have different connection relationships, that is, the geometric coordinates and UV coordinates are not in a one-to-one correspondence, then the non-manifold identification information of the geometric vertices and the geometric duplicate point index information generated by removing the non-manifold, and the non-manifold identification information of the UV vertices and the UV duplicate point index information generated by removing the non-manifold are decoded respectively. In the post-processing step, the duplicate points generated by removing the non-manifold are merged and the connection relationship is adjusted to restore the non-manifold structure in the mesh. Lossless encoding and decoding of the mesh containing the non-manifold structure is completed. It should be noted that the post-processing step may also include post-processing modules required for correct decoding, such as restoring filtered duplicate points and deleting added virtual vertices. For ease of explanation, Figure 6 only lists the modules for restoring the non-manifold structure targeted by the embodiment of the present application.

[0139] (2) Decode the indication information (such as the first indication information) indicating whether to perform lossless color space format conversion, and use the video decoder to decode the texture map. If the color space format of the texture map is converted to a lossless color space format at the encoding end, the decoded texture map is a YUV format texture map. According to the indication information, the texture map is converted to a lossless color space format at the decoding end, and the texture map is converted from the YUV color space format (i.e., the second color space format) to the RGB color space format (i.e., the first color space format), thereby realizing lossless encoding and decoding of the texture map. If the color space format of the texture map is not converted to a lossless color space format at the encoding end, the decoded texture map is a RGB format texture map. In this case, no color space format conversion is required at the decoding end.

[0140] The following describes a three-dimensional grid encoding and decoding method through a specific embodiment:

[0141] Encoding side:

[0142] As shown in Figure 5, the 3D mesh lossless coding framework is mainly divided into five parts: preprocessing to split non-manifold structures, connection relationship encoding, geometric information encoding, attribute information encoding, and non-manifold structure information encoding. In this embodiment, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (also described as texture coordinates, or simply UV or texture) as an example to introduce each part separately:

[0143] (1) Pretreatment

[0144] Input: original mesh;

[0145] Output: information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship, manifold mesh, and duplicate point information generated by decomposing non-manifolds.

[0146] Before dismantling the non-manifold structure, first determine whether the geometric coordinates and UV coordinates have the same connection relationship. You can first determine whether the number of geometric vertices is consistent with the number of UV vertices. If they are consistent, then determine whether the geometric triangles and UV triangles, that is, the geometric connection relationship and UV connection relationship, are consistent. If the geometric triangles and UV triangles are inconsistent, then the geometric coordinates and UV coordinates have different connection relationships; if the number of geometric vertices and the number of UV vertices are inconsistent, but the number of geometric triangles and the number of UV triangles are consistent, then the geometric coordinates and UV coordinates also have different connection relationships. Other methods can also be used to determine this, and the determination method is not limited here.

[0147] Splitting non-manifold structures is mainly divided into two parts: splitting non-manifold edges and splitting non-manifold points.

[0148] The first step in splitting non-manifold edges is to find them. The criterion for determining a non-manifold edge is that an edge exists in three or more triangles at the same time. The specific implementation method is: either establish a data structure to store the triangles where each edge is located, and find the non-manifold edge by querying the number of triangles corresponding to the edge; or construct a corner table (CornerTable) to establish a correspondence between angles and edges in the grid, and then find the non-manifold edge. Specifically, for a manifold grid, each edge is opposite to at most two angles, and the two opposite angles are called opposite angles. As shown in Figure 7, angle a and angle d are opposite to edge bc, and angle a and angle d are opposite angles; for non-manifold edges, there will be three or more opposite angles. Therefore, non-manifold edges can also be found through the correspondence between angles and edges.

[0149] The second step in splitting a non-manifold edge is to add vertices and modify the connectivity. After finding a non-manifold edge, duplicate vertices are created for each of the two vertices of the non-manifold edge. A triangle t containing the non-manifold edge is selected, and the third vertex of that triangle forms a new triangle t' with the two newly added vertices. This process is then repeated until the non-manifold edge is converted to a manifold edge.

[0150] To split non-manifold points, you first need to build a CornerTable and establish a correspondence between each vertex and the corner of the vertex. Perform two steps for each vertex. In the first step, starting from a corner of the vertex, traverse all the corners adjacent to the corner and forming a fan in sequence, and mark the vertex and the traversed corners as traversed. If after executing the above process, there are still vertices that have not been traversed, then it means that the vertex is a non-manifold point. In the second step, for each non-manifold point, create a duplicate point and modify the connection relationship, connecting the corners that were not traversed in the first step to the newly added duplicate point, and splitting the non-manifold point into two manifold vertices. Repeat this process until all vertices are converted into manifold points.

[0151] If the geometric coordinates and UV coordinates have the same connection relationship, the above process is executed once, and a set of duplicate point information generated by the process is recorded; if the geometric coordinates and UV coordinates have different connection relationships, the above process is executed once for the geometry and UV respectively: the CornerTable of the geometry and UV are constructed respectively, and the non-manifold edges and non-manifold points of the geometry and the non-manifold edges and non-manifold points of the UV are split respectively, and the geometric duplicate point information and UV duplicate point information generated by the process are recorded respectively.

[0152] (2) Connection relationship encoding

[0153] Input: The connectivity of the manifold mesh, which indicates whether the geometric coordinates and UV coordinates in the mesh have the same connectivity.

[0154] Output: Encoded connection relationship sub-code stream and vertex encoding order.

[0155] This embodiment uses the Edgebreaker method to encode the connection relationship of a three-dimensional mesh. It establishes a CornerTable to represent the connection relationship of the mesh, and uses the CornerTable to traverse all triangles in the mesh to generate the Edgebreaker CLERS pattern string.

[0156] CornerTable is used to represent the relationship between corners, vertices, and triangles in a mesh. Before building CornerTable, you first need to number the corners. Traverse the triangles in the order of the triangles in the mesh, and number the corners of each triangle in counterclockwise order. If the mesh has f triangles, then the mesh will have 3f corners. The advantage of this numbering is that the number of the triangle where the current corner is located can be calculated through the number of the corner, as shown in formula (1); you can also calculate the previous corner c of the current corner c in the counterclockwise direction. p and the next angle c n The serial number is shown in formula (2) and formula (3). i =c / 3 (1)

[0157] Among them, c is the index of the current angle, f i is the number of the triangle where the current angle c is located. " / " is integer division, and the result is rounded down. p =(f i *3)+(c-1)%3 (2)

[0158] Among them, c is the index of the current angle, f i is the number of the triangle where the current angle c is located, "*" is multiplication, and "%" is modulo operation. p =(f i *3)+(c+1)%3 (3)

[0159] Among them, c is the index of the current angle, f i is the serial number of the triangle where the current angle c is located, "*" is multiplication, and "%" is modulo operation.

[0160] CornerTable consists of four parts: V-table, O-table, U-table, and M-table. The V-table stores the vertex index corresponding to each corner, the O-table stores the diagonal index of each corner, the U-table stores the flags of whether each triangle has been traversed during the traversal process, and the M-table stores the flags of whether each vertex has been traversed during the traversal process.

[0161] CornerTable can be used to construct the relationship shown in Figure 8, where c represents the current corner, cp represents the previous corner of the current corner c (counterclockwise), and cn represents the next corner of the current corner c. co is the opposite corner of the current corner c, which can be obtained by querying the O table. ct is the serial number of the triangle in which c is located, which can be calculated using Equation 1. cv represents the vertex of the current corner, which can be obtained by querying the V table. cl represents the corner to the left of the current corner c, which is obtained by querying the opposite corner of cp in the O table; cr represents the corner to the right of the current corner c, which is obtained by querying the opposite corner of cn in the O table.

[0162] After using CornerTable to establish the relationships between corners, vertices, and triangles, we can traverse the mesh in a spiral order to obtain the Edgebreaker CLERS pattern string representing the mesh connectivity. The judgment conditions and traversal rules for the five patterns are shown in Figure 9. The currently traversed angle is x. If the vertex xv corresponding to x has not been visited, the current triangle is in C mode, and the next triangle to be traversed is the triangle where xr is located; otherwise, if the triangle where xl is located has been visited, the current triangle is in L mode, and the next triangle to be traversed is the triangle where xr is located; if the triangle where xr is located has been visited, the current triangle is in R mode, and the next triangle to be traversed is the triangle where xl is located; if the vertex xv has been visited, and the triangles where xl and xr are located have not been visited, then the current triangle is in S mode. At this time, the traversal path will produce two branches. The depth-first traversal principle is adopted. The first triangle traversed is the triangle where xr is located, and the triangle where xl is located is stored in the stack. After waiting for the branch where xr is located to be traversed, the triangle where xl is located is traversed; if the triangles where xl and xr are located have been visited, the mode of the current triangle is E, and the end point of the current traversal path branch has been traversed.

[0163] Randomly select an initial triangle in the mesh, traverse the triangles in the mesh according to the above rules, and generate a CLERS pattern string. When the traversal path ends but there are still untraversed triangles in the mesh, randomly select an untraversed triangle and start the next traversal until all triangles in the mesh have been traversed.

[0164] Entropy coding is used to compress the CLERS pattern string to obtain the final connection information code stream.

[0165] If the geometric coordinates and UV coordinates have the same connection relationship, the above process only needs to be executed once; if the geometric coordinates and UV coordinates have different connection relationships, since the number of geometric triangles and texture triangles is the same and corresponding, but the corresponding geometric coordinate index and UV coordinate index are not one-to-one corresponding, it is only necessary to build a CLERS pattern string and encode it according to the CornerTable of the geometry. In this case, for UV, after the pattern string encoding of each connected area of ​​the geometry is completed, the traversed corners are traversed, and the difference information between the UV connection relationship and the geometric connection relationship is recorded and encoded according to the TC table and OTC table of UV. Among them, the TC table stores the UV vertex index corresponding to each corner in the texture triangle, and the OTC table stores each diagonal index in the texture triangle.

[0166] (3) Geometric Information Coding

[0167] Input: Geometric information and connection relationship encoding order of manifold mesh;

[0168] Output: geometry information sub-code stream and geometry information coding order.

[0169] There are many methods for encoding geometric information, such as differential prediction coding, parallelogram prediction coding, and multi-parallelogram prediction coding. The specific encoding method is not emphasized here. For example, consider the parallelogram prediction coding algorithm: Assume that four vertices, a, b, c, and d, form two adjacent triangles in the mesh shown in Figure 10.

[0170] The geometric information of points a, b, and c has been encoded, and the geometric information of point d is yet to be encoded. The predicted value d' of the geometric coordinates of point d can be calculated using formula (4). d'(x,y,z)=b(x,y,z)+c(x,y,z)-a(x,y,z) (4)

[0171] After obtaining d', calculate the difference between d' and the three-dimensional coordinates of point d, as shown in formula (5): Δd(x,y,z)=d(x,y,z)-d′(x,y,z) (5)

[0172] Perform entropy coding on Δd to obtain a code stream of geometric information.

[0173] For triangles that cannot be predicted using parallelograms, such as those at the mesh boundary, a difference encoding method is used to encode the geometric information. This method uses the coordinates of adjacent encoded vertices as the predicted values ​​of the current vertex coordinates, and calculates and predicts the residual.

[0174] (4) Attribute Information Coding

[0175] Input: attribute information and connection relationship encoding order of manifold mesh;

[0176] Output: attribute information sub-code stream and the encoding order of attribute information.

[0177] 3D mesh attribute information generally includes UV coordinates and normal vectors. Taking UV coordinates as an example, there are many ways to encode UV coordinates, including differential prediction coding, parallelogram prediction coding, and similar triangle prediction coding. The specific encoding methods are not discussed here. The following describes the similar triangle prediction algorithm.

[0178] First, select a triangle in a three-dimensional mesh as the initial triangle, directly encode the UV coordinates of the three vertices of the initial triangle without predicting them, and store the edges of the initial triangle in an edge set. This set can be a data structure that meets certain access criteria. Then take out an edge τ from a set and predict the UV coordinates of the vertex opposite to τ in the next new triangle. And put the two sides of the new triangle except τ into the set. Let the point to be predicted be point C, the two end points of the edge τ be N and P respectively, the vertex opposite to the triangle adjacent to the new triangle through τ be O, and the projection point of C on τ be X. As shown in Figure 11, since the UV coordinates of points N, P, and O are encoded before point C, these three points can be used to predict the UV coordinates of point C. The specific calculation process is as follows:

[0179] Let C uv ,X uv ,N uv ,P uv ,O uv are the UV coordinates of each point, C G ,X G ,N G ,P G ,O G are the geometric coordinates of each point. First, use equations (6) and (7) to calculate the UV coordinates of point X:

[0180] in, The vectors with G as the subscript are all vector representations of the geometric coordinates between the corresponding points; similarly, The vector representations with UV as the subscript are all vector representations of the UV coordinates between the corresponding points.

[0181] Use formula (8) to calculate the vector Rotated() means flipping the vector 90 degrees:

[0182] Finally, use equations (9), (10) and (11) to calculate the predicted UV coordinates Pred of point C C :

[0183] in, and Both represent the vector of the UV coordinates between point C and point O, but are calculated and expressed in different ways.

[0184] After obtaining the UV coordinate prediction value, subtract it from the original UV coordinate to obtain the residual value.

[0185] The steps to encode UV coordinates are as follows:

[0186] (a) Select an initial triangle from the connectivity relationship and directly encode the UV coordinates of the three vertices of the initial triangle without prediction. Store the edges of the initial triangle in the edge set.

[0187] (b) Select edge τ from the set according to the access criteria and encode the UV coordinates of the vertices of the new triangle formed by τ. Using the triangle's 3D-to-2D projection relationship, calculate the predicted value of the point to be encoded according to the UV coordinate prediction value calculation process described above. Subtract the predicted value from the original UV coordinate value to obtain the residual.

[0188] (c) Add the two edges of the new triangle to the edge set and remove the top edge τ. Take the next edge from the set and continue encoding the UV coordinates of the vertices of the triangles adjacent to that edge. Return to step 3 until the UV coordinates of all vertices are encoded.

[0189] (d) Entropy encode the UV coordinate residual and output the UV coordinate code stream.

[0190] (5) Non-manifold structural information encoding

[0191] Input: information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship, information indicating whether there is a non-manifold structure in the mesh, non-manifold identifiers of vertices, index information of duplicate points generated by removing non-manifolds, encoding order of geometric information, and encoding order of attribute information;

[0192] Output: non-manifold structure information sub-code stream.

[0193] First, encode information indicating whether a non-manifold structure exists in the mesh. For example, if a non-manifold structure does not exist, meaning the number of duplicate points generated by removing non-manifolds is zero, set the flag to 0 and do not encode the vertex non-manifold identifiers or the index of duplicate points generated by removing non-manifolds. If a non-manifold structure exists, meaning the number of duplicate points generated by removing non-manifolds is greater than zero, set the flag to 1 and then encode the vertex non-manifold identifiers and the index of duplicate points generated by removing non-manifolds.

[0194] The duplicate points generated by non-manifold removal include geometric duplicate points and UV duplicate points. Depending on whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship, the non-manifold identifiers of the vertices and the index information of the duplicate points generated by non-manifold removal are encoded into two cases: the first case is when the geometric coordinates and UV coordinates in the mesh have the same connection relationship, that is, the geometric coordinates and UV coordinates are in a one-to-one correspondence, then the non-manifold identifiers of a group of vertices and the index information of the duplicate points generated by non-manifold removal are directly encoded, and the index information can be obtained based on the encoding order of the geometric coordinates and UV coordinates on the encoding end; the second case is when the geometric coordinates and UV coordinates in the mesh have different connection relationships, that is, the geometric coordinates and UV coordinates are not in a one-to-one correspondence, then the non-manifold identifiers of the geometric vertices and the index information of the geometric duplicate points generated by non-manifold removal, and the non-manifold identifiers of the UV vertices and the index information of the UV duplicate points generated by non-manifold removal are encoded separately, and the index information can be obtained based on the encoding order of the geometric coordinates and UV coordinates on the encoding end.

[0195] The specific implementation is as follows: if the geometric coordinates and UV coordinates in the mesh have the same connection relationship, a flag is set for each vertex in the manifold mesh to indicate whether the point at the current position is a duplicate point generated by non-manifold removal, and the index information of the duplicate points generated by non-manifold removal is encoded. If the geometric coordinates and UV coordinates in the mesh have different connection relationships, a flag is set for each geometric vertex and each UV vertex in the manifold mesh to indicate whether the geometric vertex and UV vertex at the current position are duplicate points generated by non-manifold removal, and the index information of the geometric duplicate points and UV duplicate points generated by non-manifold removal are encoded respectively. Then, the binary string sequence of the flag bits and the index information of the duplicate points generated by non-manifold removal, arranged in the corresponding coding order, are entropy coded to obtain the non-manifold structure information code stream.

[0196] The non-manifold structure information bitstream can be stored in multiple ways within the total bitstream: one is to store the non-manifold structure information bitstream as a separate sub-bitstream; another is to store the geometric non-manifold structure information bitstream within the geometric information sub-bitstream and the UV non-manifold structure information bitstream within the attribute information sub-bitstream; or, alternatively, to store the geometric non-manifold structure information bitstream and the UV non-manifold structure information bitstream as two sub-bitstreams within the total bitstream. The storage method of the non-manifold structure information bitstream within the total bitstream is not emphasized here.

[0197] (6) Lossless color space format conversion module

[0198] Input: Texture image in BGR444 format (BGR444 format is used as an example here);

[0199] Output: Texture map in YUV444 format.

[0200] An indication information can be encoded to indicate whether the texture map is subjected to lossless color space format conversion. If the texture map is subjected to lossless color space format conversion, the texture map in the input RGB color space format (i.e., the first color space format) is subjected to lossless color space format conversion to obtain a texture map in the YUV color space format (i.e., the second color space format). The lossless color space format conversion here can be implemented in a variety of ways, and the embodiments of the present application do not limit the specific implementation of the lossless color space format conversion. For example, the color space format conversion method shown in Formula (12)-Formula (14) can be used to convert the texture map from the RGB color space format to the YUV color space format. V=RG (13)

[0201] Alternatively, the color space format conversion method shown in equations (15)-(17) can be used to convert the texture image from the RGB color space format to the YUV color space format. U=RG (16) V=BG (17)

[0202] Among them, round() represents the rounding down function.

[0203] (7) Texture image encoding module

[0204] If the texture image is losslessly converted to a color space format, a video encoder is used to losslessly encode the video in the YUV color space format. If the texture image is not losslessly converted to a color space format, a video encoder is used to losslessly encode the video in the RGB color space format. The video encoder used is not limited here.

[0205] Decoding end:

[0206] As shown in Figure 6, the 3D mesh lossless decoding framework of the embodiment of the present application is mainly divided into six parts: connection relationship decoding, geometric information decoding, attribute information decoding, non-manifold structure information decoding, reconstruction of the manifold mesh, and recovery of the non-manifold structure in post-processing. In this embodiment, the first color space format is RGB color space format, the second color space format is YUV color space format, and the attribute information is UV coordinates (which can also be described as texture coordinates, or simply UV or texture) as an example, and each part is introduced separately:

[0207] (1) Connection relationship decoding

[0208] Input: The connection relationship sub-code stream to be decoded, indicating whether the geometric coordinates and UV coordinates have the same connection relationship;

[0209] Output: connectivity of the manifold mesh and decoded vertex order.

[0210] First, decode the connection relationship sub-code stream to obtain a pattern string. Traverse the pattern string in a certain order (forward or reverse order), and reconstruct the connection relationship according to the corresponding pattern in the string. Consistent with the encoding end, if the geometric coordinates and UV coordinates have the same connection relationship, only one set of connection relationships needs to be reconstructed; if the geometric coordinates and UV coordinates have different connection relationships, first reconstruct the geometric connection relationship based on the decoded string, and then traverse the corners in the currently decoded connected area and the difference information between the UV connection relationship and the geometric connection relationship in the currently decoded connected area to reconstruct the connection relationship of the UV coordinates. In addition, the traversal order of the vertices is output to the geometric information and attribute information decoding module.

[0211] (2) Geometric information decoding

[0212] Input: Geometric information sub-code stream, decoding order of connection relationship;

[0213] Output: Geometric information of the manifold mesh.

[0214] The decoding process of the mesh geometric coordinates is the inverse process of the encoding process: first, entropy decoding is performed to obtain the coordinate prediction residual. Then, based on the decoded triangle, the predicted coordinates of the point to be decoded are predicted according to the parallelogram rule. The predicted coordinates are added to the residual value obtained by entropy decoding to obtain the geometric coordinate position to be decoded. The vertex traversal order here is the same as the vertex order of the encoded geometric information. It should be noted that the geometric coordinates of the initial triangle do not use predictive coding, but directly encode their geometric coordinate values. After the geometric coordinates of the triangle are decoded at the decoding end, it is used as the initial triangle to start traversing and decoding the geometric coordinates of the vertices of other triangles. In addition, other decoding methods may also be used here. The specific decoding method is not emphasized, as long as it corresponds to the encoding end.

[0215] (3) Attribute information decoding

[0216] Input: attribute information code stream to be decoded, decoding order of connection relationship;

[0217] Output: attribute information of manifold mesh reconstruction.

[0218] Taking UV coordinates as an example, the decoding method corresponding to the encoding end is used for UV coordinate decoding. The specific decoding method is not emphasized here. The following describes the decoding process using the similar triangle prediction algorithm.

[0219] The steps to decode UV coordinates are as follows:

[0220] a. Entropy decoding of UV coordinate code stream.

[0221] b. Decode the UV coordinates of the three vertices of the initial triangle. Predictions are not calculated here; the initial triangle directly encodes its UV coordinates rather than encoding the residual. Store the edges of the initial triangle in the edge set.

[0222] c. Select edge τ from the set according to the access criteria and decode the UV coordinates of the vertices of the new triangle formed with τ. First, use the 3D-to-2D triangle mapping relationship to calculate the UV coordinate prediction value of the point to be decoded using the same calculation method as the encoder. Then, add the predicted value to the residual error from entropy decoding to obtain the reconstructed UV coordinate.

[0223] d. Add the two edges of the new triangle to the edge set and remove the top edge τ. Take the next edge from the set and continue decoding the UV coordinates of the opposite vertex of the triangle adjacent to that edge. Return to step c until the UV coordinates of all vertices are decoded.

[0224] (4) Decoding of non-manifold structural information

[0225] Input: non-manifold structure information sub-code stream, information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship, the decoding order of geometric information and the decoding order of attribute information;

[0226] Output: Information indicating whether there is a non-manifold structure in the mesh, the non-manifold identification of the vertex, and the index information of the duplicate points generated by removing the non-manifold.

[0227] First, decode the information indicating whether there is a non-manifold structure in the mesh. For example, if this information is a flag indicating whether there is a non-manifold, if the flag is 0, then the non-manifold flags of the vertices and the index of duplicate points generated by removing the non-manifold do not need to be decoded, and the subsequent module for recovering the non-manifold structure is skipped. If the flag is 1, then the non-manifold flags of the vertices and the index of duplicate points generated by removing the non-manifold are decoded.

[0228] The decoding of vertex non-manifold identifiers and duplicate point index information generated by non-manifold removal uses a method similar to the encoding end. This is divided into two cases based on whether the geometric coordinates and UV coordinates have the same connectivity. If the geometric coordinates and UV coordinates have the same connectivity, entropy decoding is used to obtain a set of vertex non-manifold identifiers and duplicate point index information generated by non-manifold removal. If the geometric coordinates and UV coordinates have different connectivity, the non-manifold identifiers of the geometric vertices and the geometric duplicate point index information generated by non-manifold removal, and the non-manifold identifiers of the UV vertices and the UV duplicate point index information generated by non-manifold removal, respectively, are decoded. This information is recorded and output to the non-manifold structure recovery module.

[0229] (5) Reconstructing the manifold mesh

[0230] Input: connection relationship of manifold mesh, geometric information of manifold mesh, attribute information of manifold mesh;

[0231] Output: Manifold mesh.

[0232] The manifold mesh can be directly reconstructed by using the connection relationship, geometric information, and attribute information of the manifold mesh.

[0233] (6) Restoring non-manifold structures in post-processing

[0234] Input: manifold mesh, non-manifold identifiers of vertices, index information of duplicate points generated by removing non-manifolds, and information indicating whether the geometric coordinates and UV coordinates in the mesh have the same connection relationship;

[0235] Output: Reconstructed non-manifold mesh.

[0236] The recovery process for non-manifold edges and non-manifold points is the same. For example, using the duplicate point index information generated by splitting a non-manifold as the duplicate point group index, we can first traverse all vertices in the decoding order of the vertices and create a hash table based on the non-manifold identifiers of the vertices. The key of the hash table is the duplicate point group index generated by splitting the non-manifold, and the value is the index of the target vertex to be merged. If the current vertex is not a duplicate point generated by non-manifold splitting, the current vertex corresponds to the current vertex's index, meaning its index is not updated. If the current vertex is a duplicate point generated by non-manifold splitting and is the target vertex to be merged into its duplicate point group, the current vertex corresponds to the current vertex's index, meaning its index is not updated. The duplicate point group index of the current vertex and the index of the current vertex in the reconstructed manifold mesh are added to the hash table. If the current vertex is a duplicate point generated by non-manifold splitting but is not the target vertex to be merged into its duplicate point group, the hash table is searched for the index of the target vertex to be merged into in the reconstructed manifold mesh based on the duplicate point group index of the current vertex, so as to merge the duplicate points generated by non-manifold splitting. That is, the current vertex index is updated to the index of the target vertex to be merged into. Finally, the geometric coordinate list and UV coordinate list are updated, and the indices of the geometric vertices and UV vertices in the connection relationship are updated to obtain the reconstructed non-manifold mesh.

[0237] If the geometric coordinates and UV coordinates have the same connection relationship, the above process is executed once to obtain a non-manifold mesh with the same connection relationship between the reconstructed geometric coordinates and UV coordinates; if the geometric coordinates and UV coordinates have different connection relationships, the geometric vertices and UV vertices are traversed in the decoding order of the geometric vertices and UV vertices respectively, and a geometric hash table and a UV hash table are established according to the non-manifold identifiers of the geometric vertices and UV vertices respectively, and the index information of the geometric duplicate points generated by disassembling the non-manifold and the index of the geometric target vertex to be merged in the reconstructed manifold mesh, the index information of the UV duplicate points generated by disassembling the non-manifold and the index of the UV target vertex to be merged in the reconstructed manifold mesh are stored respectively, and the above judgment and merging operations of the duplicate points generated by disassembling the non-manifold are performed respectively, and the geometric coordinate list and the UV coordinate list are updated, and the geometric vertex index and the UV vertex index in the connection relationship are updated to obtain a non-manifold mesh with different connection relationships between the reconstructed geometric coordinates and UV coordinates.

[0238] (8) Texture map decoding module

[0239] Decode the indication information (such as the first indication information) of whether to perform lossless color space format conversion, and use the video decoder to decode the texture map. If the color space format is converted to a lossless color space format at the encoding end, the decoded texture map is in the YUV format; if the color space format is not converted to a lossless color space format at the encoding end, the decoded texture map is in the RGB format.

[0240] (9) Lossless color space format conversion module

[0241] Input: Texture map in YUV 444 format;

[0242] Output: Texture map in BGR444 format (BGR444 format is used as an example here).

[0243] If the color space format is losslessly transformed, the texture image in the input YUV color space format (i.e., the second color space format) is losslessly converted to obtain a texture image in the RGB color space format (i.e., the first color space format). The lossless color space format conversion can be implemented in a variety of ways, and this embodiment does not limit the specific implementation of the lossless color space format conversion. For example, the inverse conversion method corresponding to the color space format conversion method shown in equations (12) to (14) is shown in (18) to (20). R=V+G (19)

[0244] The inverse conversion method corresponding to the color space format conversion method shown in equations (15) to (17) is shown in (21) to (23). R=U+G (22) B=V+G (23)

[0245] Among them, round() represents the rounding down function.

[0246] In the related art, for the lossless encoding of the texture map of the three-dimensional mesh, the texture map in the BGR444 format is directly encoded using a video encoder, resulting in low efficiency in lossless encoding of the texture map. The embodiment of the present application proposes a lossless encoding and decoding method for a three-dimensional mesh texture map based on Edgebreaker. Taking the first color space format as the RGB color space format and the second color space format as the YUV color space format as an example, the decoding end obtains the indication information indicating whether to perform lossless color space format conversion. If the color space format of the texture map is converted to a lossless color space format at the encoding end, the texture map obtained by decoding using the video decoder is a YUV format texture map. The texture map is converted to a lossless color space format, and the texture map is converted from the YUV color space format (i.e., the second color space format) to the RGB color space format (i.e., the first color space format), thereby achieving lossless encoding and decoding of the texture map. It should be noted that this embodiment converts the texture map from the YUV color space format to the RGB color space format without limiting the specific format of the texture map. For example, the texture map in the RGB color space format can be in the BGR444 format or the RGB444 format. If the color space format of the texture image is not converted to a lossless color space format at the encoding end, a video in RGB color space format is obtained by decoding using a video decoder. The embodiment of the present application can improve the encoding and decoding efficiency of a three-dimensional mesh.

[0247] It should be noted that the 3D mesh encoding method provided in the embodiments of this application can be executed by a 3D mesh encoding device, or by a control module within the 3D mesh encoding device that is configured to execute the 3D mesh encoding method. The 3D mesh encoding device provided in the embodiments of this application is described using the 3D mesh encoding method executed by the 3D mesh encoding device as an example.

[0248] Please refer to FIG12 , which is a structural diagram of a three-dimensional grid encoding device provided in an embodiment of the present application. As shown in FIG12 , the three-dimensional grid encoding device 300 includes:

[0249] The conversion module 301 is configured to convert the texture image of the three-dimensional grid in the first color space format into a color space format to obtain the texture image in the second color space format.

[0250] The first encoding module 302 is configured to encode the texture image in the second color space format to obtain a first texture image code stream.

[0251] Optionally, the conversion module is specifically configured to:

[0252] When it is determined to perform color space format conversion on the texture map of the three-dimensional grid, the texture map of the three-dimensional grid in the first color space format is converted into a color space format to obtain a texture map in the second color space format.

[0253] Optionally, the device further comprises:

[0254] The second encoding module is configured to, when it is determined that the color space format conversion is not to be performed on the texture map of the three-dimensional grid, encode the texture map in the first color space format to obtain a second texture map code stream.

[0255] Optionally, the code stream corresponding to the three-dimensional mesh includes an encoding result of first indication information, where the first indication information is used to instruct the encoder whether to perform color space format conversion on the texture map of the three-dimensional mesh;

[0256] Wherein, when the first indication information instructs the encoder to perform color space format conversion on the texture map of the three-dimensional mesh, the code stream corresponding to the three-dimensional mesh also includes the first texture map code stream; or

[0257] When the first indication information instructs the encoder not to perform color space format conversion on the texture map of the three-dimensional mesh, the code stream corresponding to the three-dimensional mesh further includes a second texture map code stream.

[0258] Optionally, when the first indication information indicates that the color space format conversion of the texture image of the three-dimensional mesh is performed, the first indication information is also used to indicate the conversion type of the texture image of the three-dimensional mesh from the first color space format to the second color space format.

[0259] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0260] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0261] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0262] The three-dimensional grid coding device in the embodiments of the present application can be a device, a device or electronic device with an operating system, or a component, integrated circuit, or chip in a terminal. The device or electronic device can be a mobile terminal or a non-mobile terminal. For example, the mobile terminal can include but is not limited to the types of terminals listed above, and the non-mobile terminal can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), an ATM or a self-service machine, etc., which are not specifically limited in the embodiments of the present application.

[0263] The three-dimensional grid coding device provided in the embodiment of the present application can implement each process implemented in the method embodiment of Figure 3 and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0264] It should be noted that the 3D lattice decoding method provided in the embodiments of the present application can be executed by a 3D lattice decoding device, or by a control module within the 3D lattice decoding device that is configured to execute the 3D lattice decoding method. The 3D lattice decoding device provided in the embodiments of the present application is described using the 3D lattice decoding method executed by the 3D lattice decoding device as an example.

[0265] Please refer to FIG. 13 , which is a structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application. As shown in FIG. 13 , the three-dimensional grid decoding device 400 includes:

[0266] A first decoding module 401 is configured to decode a first texture image bitstream in a bitstream corresponding to a three-dimensional grid to obtain a texture image in a second color space format;

[0267] A conversion module 402 is configured to perform color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0268] The reconstruction module 403 is configured to perform mesh reconstruction based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0269] Optionally, the device further comprises:

[0270] a second decoding module, configured to decode an encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information;

[0271] The first decoding module is specifically configured to:

[0272] In the case where it is determined based on the first indication information that the color space format conversion is to be performed on the texture map, the first texture map code stream in the code stream corresponding to the three-dimensional grid is decoded to obtain the texture map in the second color space format.

[0273] Optionally, the first decoding module is specifically configured to:

[0274] When it is determined based on the first indication information that the texture image is converted from the second color space format to the first color space format, the texture image in the second color space format is converted from the second color space format to the first color space format to obtain the texture image in the first color space format.

[0275] Optionally, the device further comprises:

[0276] The third decoding module is used to decode the second texture map code stream in the code stream corresponding to the three-dimensional grid to obtain the texture map in the first color space format when it is determined based on the first indication information that the texture map is not to be converted into a color space format.

[0277] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0278] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0279] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0280] The three-dimensional grid decoding device in the embodiments of the present application can be a device, a device or electronic device with an operating system, or a component, integrated circuit, or chip in a terminal. The device or electronic device can be a mobile terminal or a non-mobile terminal. For example, the mobile terminal can include but is not limited to the types of terminals listed above, and the non-mobile terminal can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine (ATM), or a self-service machine, etc., which are not specifically limited in the embodiments of the present application.

[0281] The three-dimensional grid decoding device provided in the embodiment of the present application can implement each process implemented by the method embodiment of Figure 4 and achieve the same technical effect. To avoid repetition, it will not be described here.

[0282] Optionally, as shown in FIG14 , an embodiment of the present application further provides a communication device 500, comprising a processor 501 and a memory 502. The memory 502 stores a program or instruction that can be executed on the processor 501. For example, when the communication device 500 is an encoding end device, the program or instruction, when executed by the processor 501, implements the various steps of the above-described three-dimensional mesh encoding method embodiment and can achieve the same technical effect. When the communication device 500 is a decoding end device, the program or instruction, when executed by the processor 501, implements the various steps of the above-described three-dimensional mesh decoding method embodiment and can achieve the same technical effect. To avoid repetition, these steps are not further described here.

[0283] An embodiment of the present application also provides an electronic device, including a processor and a communication interface. The processor is configured to: convert a texture image in a first color space format of a three-dimensional mesh to obtain a texture image in a second color space format; and encode the texture image in the second color space format to obtain a first texture image code stream. This electronic device embodiment corresponds to the aforementioned three-dimensional mesh encoding method embodiment. Each implementation process and method of the aforementioned three-dimensional mesh encoding method embodiment is applicable to this electronic device embodiment and can achieve the same technical effects.

[0284] An embodiment of the present application also provides an electronic device, including a processor and a communication interface. The processor is configured to: decode a first texture image code stream in a code stream corresponding to a three-dimensional mesh to obtain a texture image in a second color space format; perform color space format conversion on the texture image in the second color space format to obtain a texture image in a first color space format; and perform mesh reconstruction based on the texture image in the first color space format to obtain a reconstructed three-dimensional mesh. This electronic device embodiment corresponds to the above-mentioned three-dimensional mesh decoding method embodiment. Each implementation process and implementation method of the above-mentioned three-dimensional mesh decoding method embodiment are applicable to this electronic device embodiment and can achieve the same technical effects.

[0285] Specifically, the electronic device may be a terminal. FIG15 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of the present application.

[0286] The terminal 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609 and at least some of the components of the processor 610.

[0287] Those skilled in the art will appreciate that the terminal 600 may also include a power supply (such as a battery) to power various components. The power supply may be logically connected to the processor 610 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The terminal structure shown in FIG15 does not limit the terminal. The terminal may include more or fewer components than shown, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0288] It should be understood that in an embodiment of the present application, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042, and the GPU 6041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 6071 and at least one of other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0289] In the embodiment of the present application, after receiving downlink data from a network-side device, the radio frequency unit 601 may transmit the data to the processor 610 for processing. Furthermore, the radio frequency unit 601 may send uplink data to the network-side device. Typically, the radio frequency unit 601 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like.

[0290] The memory 609 can be used to store software programs or instructions and various data. The memory 609 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 609 may include a volatile memory or a non-volatile memory, or the memory 609 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 609 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0291] Processor 610 may include one or more processing units. Optionally, processor 610 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 610.

[0292] Wherein, when the terminal is an encoding terminal device:

[0293] The processor 610 is configured to:

[0294] Performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format;

[0295] The texture image in the second color space format is encoded to obtain a first texture image code stream.

[0296] Optionally, the processor 610 is specifically configured to:

[0297] When it is determined to perform color space format conversion on the texture map of the three-dimensional grid, the texture map of the three-dimensional grid in the first color space format is converted into a color space format to obtain a texture map in the second color space format.

[0298] Optionally, the processor 610 is further configured to:

[0299] When it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream.

[0300] Optionally, the code stream corresponding to the three-dimensional mesh includes an encoding result of first indication information, where the first indication information is used to instruct the encoder whether to perform color space format conversion on the texture map of the three-dimensional mesh;

[0301] Wherein, when the first indication information instructs the encoder to perform color space format conversion on the texture map of the three-dimensional mesh, the code stream corresponding to the three-dimensional mesh also includes the first texture map code stream; or

[0302] When the first indication information instructs the encoder not to perform color space format conversion on the texture map of the three-dimensional mesh, the code stream corresponding to the three-dimensional mesh further includes a second texture map code stream.

[0303] Optionally, when the first indication information indicates that the color space format conversion of the texture image of the three-dimensional mesh is performed, the first indication information is also used to indicate the conversion type of the texture image of the three-dimensional mesh from the first color space format to the second color space format.

[0304] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0305] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0306] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0307] Wherein, when the terminal is a decoding end device:

[0308] The processor 610 is configured to:

[0309] Decoding a first texture image code stream in a code stream corresponding to the three-dimensional grid to obtain a texture image in a second color space format;

[0310] Performing color space format conversion on the texture image in the second color space format to obtain a texture image in the first color space format;

[0311] A mesh reconstruction process is performed based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

[0312] Optionally, the processor 610 is further configured to:

[0313] decoding an encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information;

[0314] The processor 610 is specifically configured to:

[0315] In the case where it is determined based on the first indication information that the color space format conversion is to be performed on the texture map, the first texture map code stream in the code stream corresponding to the three-dimensional grid is decoded to obtain the texture map in the second color space format.

[0316] Optionally, the processor 610 is further configured to:

[0317] When it is determined based on the first indication information that the texture image is converted from the second color space format to the first color space format, the texture image in the second color space format is converted from the second color space format to the first color space format to obtain the texture image in the first color space format.

[0318] Optionally, the processor 610 is further configured to:

[0319] When it is determined based on the first indication information that the texture map is not to be converted into a color space format, a second texture map code stream in the code stream corresponding to the three-dimensional grid is decoded to obtain a texture map in the first color space format.

[0320] Optionally, the amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

[0321] Optionally, the second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

[0322] Optionally, the first color space format is an RGB color space format, and the second color space format is a YUV color space format.

[0323] Specifically, the terminal of the embodiment of the present application also includes: instructions or programs stored in the memory 609 and executable on the processor 610. The processor 610 calls the instructions or programs in the memory 609 to execute the methods of the modules shown in FIG12 or FIG13 and achieve the same technical effect. To avoid repetition, it will not be described here.

[0324] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned three-dimensional grid encoding method embodiment are implemented, or when the program or instruction is executed by a processor, the various processes of the above-mentioned three-dimensional grid decoding method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0325] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0326] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned three-dimensional grid encoding method embodiment, or to implement the various processes of the above-mentioned three-dimensional grid decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0327] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0328] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0329] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0330] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A three-dimensional grid encoding method, performed by an encoding end, comprising: Performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format; The texture image in the second color space format is encoded to obtain a first texture image code stream.

2. The method according to claim 1, wherein: The step of converting the texture image in the first color space format of the three-dimensional grid to obtain the texture image in the second color space format includes: When it is determined to perform color space format conversion on the texture map of the three-dimensional grid, the texture map of the first color space format of the three-dimensional grid is converted into a color space format to obtain a texture map of the second color space format.

3. The method according to claim 2, wherein: The method further comprises: In the case where it is determined that the color space format conversion is not performed on the texture map of the three-dimensional grid, the texture map in the first color space format is encoded to obtain a second texture map code stream.

4. The method according to any one of claims 1 to 3, wherein: The bitstream corresponding to the three-dimensional mesh includes an encoding result of the first indication information, where the first indication information is used to indicate whether the encoding end performs color space format conversion on the texture image of the three-dimensional mesh; Wherein, when the first indication information instructs the encoder to perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes the first texture map bitstream; or When the first indication information indicates that the encoding end does not perform color space format conversion on the texture map of the three-dimensional mesh, the bitstream corresponding to the three-dimensional mesh also includes a second texture map bitstream.

5. The method according to claim 4, wherein: When the first indication information indicates to perform color space format conversion on the texture image of the three-dimensional mesh, the first indication information is also used to indicate the conversion type of the texture image of the three-dimensional mesh from the first color space format to the second color space format.

6. The method according to any one of claims 1 to 5, wherein: The amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

7. The method according to any one of claims 1 to 6, wherein: The second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

8. The method according to any one of claims 1 to 7, wherein: The first color space format is an RGB color space format, and the second color space format is a YUV color space format.

9. A three-dimensional grid decoding method, executed by a decoding end, comprising: Decoding a first texture image bitstream in a bitstream corresponding to the three-dimensional grid to obtain a texture image in a second color space format; The texture image in the second color space format is converted into a color space format to obtain a texture image in the first color space format. Texture map; A mesh reconstruction process is performed based on the texture map in the first color space format to obtain a reconstructed three-dimensional mesh.

10. The method according to claim 9, wherein: The method further comprises: Decoding the encoding result of the first indication information in the code stream corresponding to the three-dimensional grid to obtain the first indication information; The decoding process of the first texture image code stream in the code stream corresponding to the three-dimensional grid to obtain the texture image in the second color space format includes: In the case where it is determined based on the first indication information to convert the color space format of the texture map, the first texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain the texture map in the second color space format.

11. The method according to claim 10, wherein: The step of converting the texture image in the second color space format into a color space format to obtain the texture image in the first color space format includes: When it is determined based on the first indication information that the texture image is converted from the second color space format to the first color space format, the texture image in the second color space format is converted from the second color space format to the first color space format to obtain the texture image in the first color space format.

12. The method according to claim 10 or 11, wherein: The method further comprises: When it is determined based on the first indication information that the texture map is not to be converted into a color space format, a second texture map bitstream in the bitstream corresponding to the three-dimensional grid is decoded to obtain a texture map in the first color space format.

13. The method according to any one of claims 9 to 12, wherein: The amount of relevant information about the color space of the texture image in the second color space format is less than the amount of relevant information about the color space of the texture image in the first color space format.

14. The method according to any one of claims 9 to 13, wherein: The second color space format and the first color space format are color space formats of different color spaces; or, the second color space format and the first color space format are different color space formats of the same color space.

15. The method according to any one of claims 9 to 14, wherein: The first color space format is an RGB color space format, and the second color space format is a YUV color space format.

16. A three-dimensional grid encoding device, the device comprising: A conversion module, used for performing color space format conversion on a texture image of a three-dimensional grid in a first color space format to obtain a texture image in a second color space format; The first encoding module is used to encode the texture image in the second color space format to obtain a first texture image code stream.

17. A three-dimensional grid decoding device, the device comprising: A first decoding module is used to decode a first texture image bitstream in a bitstream corresponding to the three-dimensional grid to obtain a texture image in a second color space format; A conversion module, used for performing color space format conversion on the texture image in the second color space format to obtain the texture image in the first color space format; A reconstruction module is used to perform mesh reconstruction processing based on the texture map in the first color space format to obtain a reconstructed Three-dimensional grid.

18. An electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 8; or, the program or instruction, when executed by the processor, implements the steps of the three-dimensional mesh decoding method as described in any one of claims 9 to 15.

19. A chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 8, or to implement the steps of the three-dimensional mesh decoding method as described in any one of claims 9 to 15.

20. A readable storage medium storing a program or instruction, wherein the program or instruction, when executed by a processor, implements the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 8, or, when executed by a processor, implements the steps of the three-dimensional mesh decoding method as described in any one of claims 9 to 15.

Images