Three-dimensional grid encoding method, decoding method, device, encoding end and decoding end

Abstract

The present application discloses a three-dimensional mesh code stream, a decoding method, an apparatus, and an electronic device, belonging to the field of communication technology. The three-dimensional mesh encoding method of an embodiment of the present application includes: an encoding end compresses and encodes a first mesh to obtain a mesh code stream, wherein the first mesh is a mesh determined according to a three-dimensional mesh to be encoded; the encoding end reconstructs the three-dimensional mesh based on the mesh code stream to obtain a first reconstructed mesh; the encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is a first reconstructed mesh including texture coordinates; the encoding end converts a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; the encoding end encodes the second texture map to obtain a texture map code stream.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and specifically relates to a three-dimensional grid encoding method, decoding method, device, encoding end, and decoding end. Background Art

[0002] Three-dimensional models have become a new generation of digital media, following audio, images, and video. Compared to traditional multimedia such as images and videos, they offer greater interactivity and realism. A three-dimensional mesh is a commonly used representation of three-dimensional models. A three-dimensional mesh is composed of three basic elements: vertices, edges, and faces. Vertices are the most fundamental elements in a 3D mesh, defining a position in three-dimensional space; edges are line segments connecting two vertices in a 3D mesh; and faces are polygons formed by closed paths along the edges of a 3D mesh. Furthermore, mesh parameterization maps a 3D mesh from three-dimensional space to a two-dimensional plane. This mapping is typically described by a set of parametric coordinates, known as UV coordinates or texture coordinates, associated with the vertices of the 3D mesh. In the prior art, when a 3D mesh is encoded and transmitted or stored, its texture coordinates must also be encoded and stored or transmitted to enable reconstruction of the texture coordinates at the decoder. Since the texture coordinate bitstream often occupies a large portion of the 3D mesh bitstream, this not only results in low encoding efficiency but also consumes storage and transmission resources. Summary of the Invention

[0003] The embodiments of the present application provide a three-dimensional mesh encoding method, decoding method, device, encoding end, and decoding end, which can improve the encoding efficiency of the three-dimensional mesh and reduce the storage resources or transmission resources occupied by the code stream of the three-dimensional mesh.

[0004] In a first aspect, a three-dimensional grid encoding method is provided, the method comprising:

[0005] The encoding end compresses and encodes the first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded;

[0006] The encoding end performs three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid;

[0007] The encoder performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;

[0008] The encoding end converts the first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is the texture map corresponding to the three-dimensional mesh to be encoded;

[0009] The encoding end encodes the second texture map to obtain a texture map code stream.

[0010] In a second aspect, a three-dimensional grid encoding device is provided, the device comprising:

[0011] A first encoding module, configured to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded;

[0012] A first reconstruction module, configured to perform three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid;

[0013] A first processing module is configured to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;

[0014] a conversion module, configured to convert a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded;

[0015] The second encoding module is used to encode the second texture map to obtain a texture map code stream.

[0016] In a third aspect, a three-dimensional grid decoding method is provided, the method comprising:

[0017] The decoding end obtains the grid code stream and texture map code stream;

[0018] The decoding end decodes the grid code stream to obtain a third grid, and decodes the texture map code stream to obtain a third texture map;

[0019] The decoding end performs grid parameterization processing on the third grid to obtain a fourth grid, wherein the fourth grid is a grid including texture coordinates.

[0020] In a fourth aspect, a three-dimensional grid decoding device is provided, the device comprising:

[0021] A first acquisition module is used to acquire a grid code stream and a texture map code stream;

[0022] a decoding module, configured to decode the mesh code stream to obtain a third mesh, and decode the texture map code stream to obtain a third texture map;

[0023] The second processing module is configured to perform mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

[0024] In a fifth aspect, an encoding end is provided, which includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0025] In a sixth aspect, an encoding end is provided, comprising a processor and a communication interface, wherein the processor is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded; perform three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid; perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; convert a first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional grid to be encoded; encode the second texture map to obtain a texture map code stream.

[0026] In a seventh aspect, a decoding end is provided, which includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the third aspect are implemented.

[0027] In an eighth aspect, a decoding end is provided, comprising a processor and a communication interface, wherein the processor is used to obtain a grid code stream and a texture map code stream; decode the grid code stream to obtain a third grid, and decode the texture map code stream to obtain a third texture map; perform grid parameterization processing on the third grid to obtain a fourth grid, wherein the fourth grid is a grid including texture coordinates.

[0028] In the ninth aspect, a three-dimensional mesh encoding and decoding system is provided, comprising: an encoding end and a decoding end, wherein the encoding end can be used to execute the steps of the three-dimensional mesh encoding method as described in the first aspect, and the decoding end can be used to execute the steps of the three-dimensional mesh decoding method as described in the third aspect.

[0029] In the tenth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the third aspect are implemented.

[0030] In the eleventh aspect, a chip is provided, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the third aspect.

[0031] In the twelfth aspect, a computer program / program product is provided, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the third aspect.

[0032] In an embodiment of the present application, the encoding end compresses and encodes a first mesh to obtain a mesh code stream, wherein the first mesh is a mesh determined according to a three-dimensional mesh to be encoded; the encoding end reconstructs a three-dimensional mesh based on the mesh code stream to obtain a first reconstructed mesh; the encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is a first reconstructed mesh including texture coordinates; the encoding end converts a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; the encoding end encodes the second texture map to obtain a texture map code stream, that is, in the embodiment of the present application, the encoding end only generates a mesh code stream and a texture map code stream of the three-dimensional mesh, but does not generate a code stream corresponding to the texture coordinates of the three-dimensional mesh. This not only improves the encoding efficiency of the three-dimensional mesh, but also reduces the storage resources or transmission resources occupied by the code stream of the three-dimensional mesh. In addition, in the embodiment of the present application, the mesh parameterization processing of the first reconstructed mesh is performed, which not only reduces the data processing amount but also ensures the accuracy of the texture coordinates of the first reconstructed mesh compared to the mesh parameterization processing of the first mesh in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a block diagram of a wireless communication system to which embodiments of the present application may be applied;

[0034] Figure 2 This is one of the schematic diagrams of a grid parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application;

[0035] Figure 3 This is a flow chart of a three-dimensional grid coding method provided in an embodiment of the present application;

[0036] Figure 4 is a schematic diagram of basic grid compression provided by an embodiment of the present application;

[0037] Figure 5 This is a second schematic diagram of a grid parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application;

[0038] Figure 6a This is one of the schematic diagrams of the connected regions with overlapping regions before splitting provided in the embodiment of the present application;

[0039] Figure 6b This is one of the schematic diagrams of the connected areas with overlapping areas after splitting provided in the embodiment of the present application;

[0040] Figure 6c This is the second schematic diagram of the splitting of the connected areas with overlapping areas provided in the embodiment of the present application;

[0041] Figure 7a This is a third schematic diagram of a grid parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application;

[0042] Figure 7b This is a schematic diagram of extracting connected regions provided by an embodiment of the present application;

[0043] Figure 8 is a schematic diagram of a simplified grid provided in an embodiment of the present application;

[0044] Figure 9 is a schematic diagram of mesh subdivision deformation provided by an embodiment of the present application;

[0045] Figure 10 is a schematic diagram of grid subdivision provided in an embodiment of the present application;

[0046] Figure 11 is a schematic diagram of a three-dimensional grid coding framework provided in an embodiment of the present application;

[0047] Figure 12 This is a flowchart of a three-dimensional grid decoding method provided by an embodiment of the present application;

[0048] Figure 13 is a schematic diagram of basic grid decoding provided by an embodiment of the present application;

[0049] Figure 14 is a schematic diagram of a three-dimensional grid decoding framework provided in an embodiment of the present application;

[0050] Figure 15 This is a structural diagram of a three-dimensional grid encoding device provided in an embodiment of the present application;

[0051] Figure 16 This is a structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application;

[0052] Figure 17 is a structural diagram of a communication device provided in an embodiment of the present application;

[0053] Figure 18 is a structural diagram of the encoding end provided in an embodiment of the present application;

[0054] Figure 19 This is a structural diagram of the decoding end provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0056] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in this application represents at least one of the connected objects. For example, "A or B" covers three options, namely, Option 1: including A but not including B; Option 2: including B but not including A; Option 3: including both A and B. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0057] The term "indication" in this application can be either a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, the operation to be performed, or the requested result, etc. in the instruction sent; an indirect indication can be understood as the receiver determining the corresponding information based on the instruction sent by the sender, or making a judgment and determining the operation to be performed or the requested result, etc. based on the judgment result.

[0058] It is worth noting that the technology described in the embodiments of the present application is not limited to the Long Term Evolution (LTE) / LTE-Advanced (LTE-A) system, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the technology described can be used for the systems and radio technologies mentioned above, as well as for other systems and radio technologies. The following description describes a New Radio (NR) system for illustrative purposes, and NR terminology is used in most of the following description, but these technologies can also be applied to systems other than NR systems, such as 6th generation (6G) systems. th Generation, 6G) communication system.

[0059] Figure 1The block diagram of a wireless communication system applicable to the embodiments of the present application is shown. The wireless communication system includes a terminal 11 and a network-side device 12. The terminal 11 can be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer), a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), an augmented reality (AR), a virtual reality (VR) device, a robot, a wearable device (Wearable Device), an aircraft (flight vehicle), a vehicle user equipment (VUE), a ship-borne device, a pedestrian user equipment (PUE), a smart home (home appliances with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture, etc.), a game console, a personal computer (PC), a teller machine, or a self-service machine, etc., and other terminal-side devices. Wearable devices include: smart watches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle-mounted device can also be called a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiment of the present application. The network side device 12 may include an access network device or a core network device, wherein the access network device may also be called a radio access network (Radio Access Network, RAN) device, a radio access network function or a radio access network unit. The access network device may include a base station, a wireless local area network (WLAN) access point (AP) or a wireless fidelity (WiFi) node, etc.Among them, the base station can be referred to as Node B (NB), Evolved Node B (eNB), the next generation Node B (gNB), New Radio Node B (NR Node B), access point, Relay Base Station (RBS), Serving Base Station (SBS), Base Transceiver Station (BTS), radio base station, radio transceiver, Basic Service Set (BSS), Extended Service Set (ESS), Home Node B (HNB), Home Evolved Node B (home evolved Node B), Transmission Reception Point (TRP) or other appropriate terms in the field. As long as the same technical effect is achieved, the base station is not limited to specific technical vocabulary. It should be noted that in the embodiment of the present application, only the base station in the NR system is used as an example for introduction, and the specific type of the base station is not limited.

[0060] The core network equipment may include but is not limited to at least one of the following: core network node, core network function, mobility management entity (MME), access mobility management function (AMF), session management function (SMF), user plane function (UPF), policy control function (PCF), policy and charging rules function unit (PCRF), edge application service discovery function (EASDF), unified data management (UDM), unified data repository (UDR), home user server (HSS), centralized network configuration (CNC), network storage function (NRF), network exposure function (NEF), local NEF (L-NEF), binding support function (BSF), application function ( Function, AF), etc. It should be noted that in the embodiments of the present application, only the core network device in the NR system is introduced as an example, and the specific type of the core network device is not limited.

[0061] It should be noted that the encoding end of the embodiment of the present application can be an electronic device or component of an electronic device, such as a terminal, a network-side device, or a server. The decoding end of the embodiment of the present application can also be an electronic device or component of an electronic device, such as a terminal, a network-side device, or a server. In addition, the encoding end and decoding end of the embodiment of the present application can be located in the same electronic device or in different electronic devices.

[0062] For ease of understanding, some of the contents involved in the embodiments of this application are described below:

[0063] 1. 3D Grid

[0064] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have become the next generation of digital media, following audio, images, and video. Three-dimensional meshes are a commonly used representation method for 3D models. Compared with traditional multimedia such as images and videos, 3D meshes offer greater interactivity and realism, leading to their increasing application in various fields, including commerce, manufacturing, construction, education, medicine, entertainment, art, and the military.

[0065] While there are many ways to represent 3D meshes, triangular meshes remain the most common. A 3D mesh can be considered to be composed of three basic elements: vertices, edges, and faces. Vertices are the most basic elements in a mesh, defining a position in 3D space. Edges are line segments connecting two vertices in the mesh. Faces can be considered polygons formed by closed paths of edges. For a triangular mesh, each face is a triangle.

[0066] The information contained in a three-dimensional mesh is usually divided into three categories: geometric information, connection information, and attribute information. Among them, geometric information refers to the position of each vertex of the three-dimensional mesh in three-dimensional space. Connection information describes the association relationship between the elements in the three-dimensional mesh, that is, the connection relationship between vertices. Attribute information is an optional information that can associate attributes with corresponding mesh elements (such as vertex color, normal vector, etc. can be associated with mesh vertices). In addition, mesh parameterization can be used to map the three-dimensional mesh from three-dimensional space to a two-dimensional plane area. This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with mesh vertices. This two-dimensional mapping can be used to represent high-resolution attribute information, such as textures, normal vectors, etc.

[0067] In nearly all application fields using 3D meshes (such as computational simulation, entertainment, medical imaging, digitized artifacts, computer design, and e-commerce), the increasing demand for visually appealing 3D models is leading to increasing complexity and precision. Consequently, the amount of data required to represent the 3D mesh is also increasing. These issues have led to increasing complexity in the processing, visualization, transmission, and storage of 3D meshes. 3D mesh compression can be considered a solution to these problems. It can reduce the size of model data, facilitating the processing, storage, and transmission of 3D meshes.

[0068] Recently, the Moving Pictures Experts Group (MPEG), an international standardization organization for audio and video coding and compression, has begun developing a compression standard for 3D meshes. This standard, Video-based Dynamic Mesh Coding (VDMC), is based on the existing Visual Volumetric Video-based Coding (V3C) standard. The V3C standard provides a general method for compressing 3D models, which can be represented by point clouds, meshes, or panoramic videos. Compatibility of 3D mesh compression methods with this standard will facilitate the method's widespread adoption and applicability. Therefore, optimizing the 3D mesh encoding and decoding methods in VDMC and integrating these optimizations with the V3C standard is of great significance. One alternative optimization method is to optimize displacement coding. In existing frameworks, displacements are calculated by calculating the distances between the vertices of the reconstructed mesh and the original mesh, aiming to improve mesh quality. Furthermore, in existing frameworks, displacements are encoded using a video encoder. Providing multiple options for displacement encoding can help improve coding performance.

[0069] 2. orthoAtlas

[0070] OrthoAtlas is a projection-based mesh parameterization method that generates texture coordinates for 3D meshes through orthogonal projection. The main process is as follows: Figure 2 As shown, the following steps are included:

[0071] a) Calculate mesh properties, including the neighboring faces of each face and the area and normal vector of each face;

[0072] b) Determine the projection plane of each face based on the normal vector;

[0073] c) Start clustering all faces according to the projection plane to form a connected region, first selecting the starting face of the cluster;

[0074] d) Iterating from the starting face, determining whether adjacent faces of the face added to the connected region can be added to the connected region;

[0075] e) After each connected region is iterated, multiple connected regions are obtained;

[0076] f) determining whether to merge adjacent connected regions based on the error metric;

[0077] g) Check whether there are overlapping areas during projection, remove the overlapping areas and regenerate connected areas;

[0078] h) Arrange all the projected regions into a two-dimensional image.

[0079] The following describes the above main processes:

[0080] (1) Calculating grid properties

[0081] Calculate some properties of the mesh, including the adjacency list of the triangles and the normal vector and area of ​​the triangles; triangle adjacency can be defined as sharing a single vertex or sharing an edge; the normal vector and area of ​​the triangle can be derived from the triangles after its projection (later used to calculate the distortion after projection).

[0082] (2) Clustering

[0083] The triangles are clustered, and the area formed by the same type of triangles is called a connected area. There are two methods to select the starting point of the clustering algorithm:

[0084] The first method is to obtain the projection direction based on the statistical results of the normal vectors of all faces. The triangle face whose normal vector is closest to the projection direction can be selected as the starting point of the clustering algorithm and it has not yet been added to the connected area.

[0085] The second method selects the first available face as the starting point of the clustering algorithm (that is, the face has not yet been added to the connected region) and determines whether to add adjacent faces to the connected region according to the criteria; the criteria include:

[0086] Has the same projection direction as the connected region;

[0087] The angle between the normal vector of the last face added to the connected region and the normal vector of the last face added to the connected region is less than the threshold;

[0088] The area is less than a threshold of the total area;

[0089] The number of triangles is less than the threshold.

[0090] (3) Merger

[0091] After clustering is completed, there may be many small partitions that need to be merged. When merging, a cost function is used to determine whether two partitions can be merged. The function is defined by the perimeter and stretch degree of the connected area. The detailed steps are as follows:

[0092] First, the connected regions are sorted in a specific order. The sorting criteria can be based on the number of triangles contained and the proximity between the average normal vector of the connected region and the corresponding projection direction. The cost function of each connected region is COST(M i )=L 2 (M i )+λPERIMETER(M i );

[0093] Then, take the first connected region after sorting each time, obtain the connected regions adjacent to the connected region to form a candidate list, and then for each candidate region, first calculate the cost function after the two regions are merged: COST(M i +M j )=L 2 (M i +M j )+λPERIMETER(M i +M j ), for all (COST(M i +M j ) <COST(M i )+COST(M j )) of the region, select the region that minimizes the cost function, merge the extracted regions into this region and update the ranking of the connected regions; where L 2 () represents the stretching degree of the connected region after projection, PERIMETER() represents the perimeter of the above connected region, and λ represents the weight coefficient.

[0094] The stretching function in the cost function measures the degree of distortion after projection, which is obtained by calculating the area of ​​the triangle after projection in each direction.

[0095] (4) Detect overlapping areas

[0096] Finally, we need to check whether there are overlapping vertices or faces in each connected region, splitting the overlapping regions from the connected regions to form new connected regions. Detecting overlapping regions means determining whether the currently projected face overlaps with the projected region. If so, the current face is removed from the connected region, and this face and the other removed faces form a new connected region. Finally, within the removed connected regions, we check whether there are disconnected regions and split these disconnected regions into multiple connected regions.

[0097] The following describes in detail the three-dimensional grid coding method provided by the embodiments of the present application through some embodiments and their application scenarios in combination with the accompanying drawings.

[0098] See Figure 3 , Figure 3 This is a flow chart of a three-dimensional grid encoding method provided by an embodiment of the present application. The method can be executed by the encoding end, such as Figure 3 As shown, the following steps are included:

[0099] Step 301: The encoding end compresses and encodes a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded.

[0100] In this embodiment, the first mesh can be a 3D mesh to be encoded, or it can be a mesh obtained by processing the 3D mesh to be encoded, for example, a mesh obtained by performing mesh simplification on the 3D mesh to be encoded. The mesh code stream is the code stream obtained by compressing and encoding the first mesh. It should be noted that the 3D mesh to be encoded can also be referred to as the input mesh of the encoder, the first mesh can also be referred to as the base mesh, and the mesh code stream can also be referred to as the base mesh code stream.

[0101] The compression encoding of the first grid can be performed, for example, using compression modes such as intra-frame mode, inter-frame mode or skip mode. Figure 4 The above three compression modes are described with examples.

[0102] like Figure 4 As shown, in intra-frame mode, the first mesh is encoded by a static mesh encoder. In this compression mode, the first mesh must contain geometric coordinates, connection relationships, and attribute information associated with the vertices. In inter-frame mode, the first mesh first uses rate-distortion optimization (RDO) to decide whether to use skip mode. If skip mode is used, the reference mesh in the mesh buffer is directly selected as the first mesh of the current frame. If skip mode is not used, the motion vector of the vertex is calculated based on the reference frame in the mesh buffer and the current frame, and the corresponding motion vector is encoded by the motion vector encoder. It should be noted that after encoding the first mesh, the encoded mesh needs to be reconstructed, that is, the first mesh needs to be reconstructed so that it can be provided to subsequent modules for processing.

[0103] Step 302: The encoding end reconstructs a three-dimensional grid based on the grid code stream to obtain a first reconstructed grid.

[0104] In this embodiment, after the first mesh is compressed and encoded to obtain a mesh code stream, a three-dimensional mesh can be reconstructed based on the mesh code stream. Exemplarily, the three-dimensional mesh can be reconstructed by decoding the mesh code stream.

[0105] Specifically, corresponding decoding can be performed based on the compression mode of the above-mentioned grid code stream. For example, when the compression mode of the above-mentioned grid code stream is intra-frame mode, the above-mentioned grid code stream can be decoded based on the static grid decoder to obtain a reconstructed basic grid, that is, the first reconstructed grid; when the compression mode of the above-mentioned basic grid is inter-frame mode, the motion vector of the above-mentioned grid code stream can be decoded, and the corresponding basic grid can be reconstructed according to the reference frame in the grid buffer; when the compression mode of the above-mentioned basic grid is skip mode, the corresponding reference frame in the grid buffer can be used as the basic grid of the current frame.

[0106] Step 303: The encoder performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates.

[0107] In this embodiment, the mesh parameterization process is used to generate corresponding texture coordinates for the first reconstructed mesh. Exemplarily, the mesh parameterization process can be performed on the first reconstructed mesh based on an Isochart algorithm, an orthogonal projection algorithm, or the like.

[0108] Step 304: The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map.

[0109] In this embodiment, the first texture map may also be referred to as an input texture map, which may be a texture map corresponding to the three-dimensional mesh to be encoded. For example, an optional texture map conversion method may include the following steps:

[0110] Step a1: Calculate the texture coordinates of each pixel on the texture map to be generated (ie, the second texture map).

[0111] Step a2: For each texture coordinate, determine whether the texture coordinate is within a certain face of the second reconstructed mesh.

[0112] Step a3: If the texture coordinate does not belong to any surface of the second reconstructed mesh, the pixel is marked as an empty pixel, which can be filled with a filling algorithm.

[0113] Step a4: If the texture coordinate belongs to a face of the second reconstructed mesh, perform the following steps:

[0114] Step a41: Mark the pixel as filled;

[0115] Step a42: Calculate the center of gravity coordinates of the texture on the current surface based on the texture coordinates;

[0116] Step a43: Map the two-dimensional texture coordinates to three-dimensional geometric coordinates according to the barycentric coordinates and the corresponding surface, that is, map them to points on the second reconstructed grid corresponding to the texture coordinates;

[0117] Step a44: Find the point closest to the three-dimensional coordinate on the input original grid (i.e., the three-dimensional grid to be encoded);

[0118] Step a45, calculating the center of gravity coordinates of the three-dimensional coordinates according to the surface on which they are located and mapping them to two dimensions to calculate their texture coordinates;

[0119] Step a46: sampling the input original texture map (i.e., the first texture map) using the texture coordinates to obtain the value of the corresponding pixel position;

[0120] Step a47: assign the value to the corresponding pixel on the texture map to be generated (ie, the second texture map mentioned above).

[0121] It should be noted that, for empty pixels, existing filling algorithms (such as the Push-Pull algorithm) can be used to fill these empty pixels.

[0122] Step 305: The encoding end encodes the second texture image to obtain a texture image code stream.

[0123] Exemplarily, a video encoder may be used to encode the second texture map to obtain a texture map code stream.

[0124] It is understood that, upon obtaining a mesh code stream and a texture map code stream, the encoder can store or transmit the mesh code stream and the texture map code stream. It should also be noted that, if the encoder does not generate a code stream for the texture coordinates of the mesh, the decoder can perform mesh parameterization on the mesh obtained by decoding the mesh code stream to generate the texture coordinates of the mesh, thereby ensuring that the decoder can properly apply the decoded mesh.

[0125] In the three-dimensional mesh encoding method provided in an embodiment of the present application, an encoding end compresses and encodes a first mesh to obtain a mesh code stream, wherein the first mesh is a mesh determined according to a three-dimensional mesh to be encoded; the encoding end reconstructs the three-dimensional mesh based on the mesh code stream to obtain a first reconstructed mesh; the encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is a first reconstructed mesh including texture coordinates; the encoding end converts a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; the encoding end encodes the second texture map to obtain a texture map code stream, that is, in the embodiment of the present application, the encoding end only generates a mesh code stream and a texture map code stream of the three-dimensional mesh, but does not generate a code stream corresponding to the texture coordinates of the three-dimensional mesh. This not only improves the encoding efficiency of the three-dimensional mesh, but also reduces the storage resources or transmission resources occupied by the code stream of the three-dimensional mesh. In addition, in the embodiment of the present application, the mesh parameterization processing of the first reconstructed mesh is performed, which not only reduces the data processing amount but also ensures the accuracy of the texture coordinates of the first reconstructed mesh compared to the mesh parameterization processing of the first mesh in the prior art.

[0126] Optionally, the encoding end performs grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, including:

[0127] The encoding end performs grid parameterization processing on the first reconstructed grid using an orthogonal projection algorithm to obtain a second reconstructed grid.

[0128] It is understood that the mesh parameterization methods performed on the encoder and decoder in the embodiments of the present application should be consistent, which facilitates more accurate reconstruction of the desired mesh. Therefore, this embodiment uses an orthogonal projection algorithm to perform mesh parameterization on the first reconstructed mesh, which has low implementation complexity and is convenient for application on the decoder.

[0129] Optionally, the encoding end performs mesh parameterization processing on the first reconstructed mesh using an orthogonal projection algorithm to obtain a second reconstructed mesh, including:

[0130] The encoder divides the first reconstructed grid into at least two connected regions, wherein each of the connected regions includes at least one face of the first reconstructed grid, and faces having the same projection surface and being connected in three-dimensional space belong to the same connected region;

[0131] The encoding end merges the at least two connected regions to obtain at least one first connected region;

[0132] The encoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each first connected region to detect overlapping regions, and splitting the first connected regions having overlapping regions into at least two non-overlapping connected regions;

[0133] The encoding end merges the connected regions obtained after the first processing to obtain at least one second connected region;

[0134] The encoding end arranges the projection area of ​​the at least one second connected area in two dimensions to obtain a second reconstructed grid.

[0135] The following combination Figure 5 This embodiment is described with an example.

[0136] like Figure 5 As shown, the encoding end uses the orthogonal projection algorithm to perform mesh parameterization processing on the first reconstructed mesh, which includes the following processes in sequence: initial partitioning, merging partitions, detecting overlapping areas, simple area merging, and two-dimensional arrangement. Among them, the above-mentioned initial partitioning, that is, the encoding end divides the first reconstructed mesh into at least two connected areas. The above-mentioned merging partitions, that is, the encoding end merges the at least two connected areas to obtain at least one first connected area. The above-mentioned detecting overlapping areas, that is, the encoding end performs the first processing on the at least one first connected area respectively. The above-mentioned simple area merging, that is, the encoding end merges the connected areas obtained after the first processing to obtain at least one second connected area. The above-mentioned two-dimensional arrangement, that is, the encoding end performs two-dimensional arrangement on the projection areas of the at least one second connected area to obtain the second reconstructed mesh. The following takes the face of the first reconstructed mesh as an example to illustrate the above-mentioned processes.

[0137] Exemplarily, the above initial partitioning may include the following process:

[0138] Define any triangular face T of the first reconstructed mesh, whose three vertices are A, B, C, (x i ,y i ,z i ) is the geometric coordinate of the i-th vertex, then the normal vector of the current face T AB×AC=((y B -y A )*(z C -z A )-(y C -y A )*(z B -z A ),(z B -z A )*(x C-x A )-(z C -z A )*(x B -x A ),(x B -x A )*(y C -y A )-(x C -x A )*(y B -y A )), where AB and AC represent the vectors corresponding to the two sides of the triangle T, and × represents the vector cross product. After the normal vector is calculated by the above formula, the normal vector is normalized to obtain the unit normal vector of the current triangle. At the same time, the area S of the current triangle can be calculated by the normal vector The modulus length is obtained, that is,

[0139] After obtaining the normal vector and area of ​​each triangular face of the first reconstructed mesh, the projection surface of each triangular face is determined according to the pre-set projection surface. This process is to calculate the dot product of the normal vector of the current triangular face and the normal vector of each pre-set projection surface, and take the projection surface corresponding to the largest product point as the projection surface of the current triangular face. Then, the triangular faces in the first reconstructed mesh that belong to the same projection surface and are connected in three-dimensional space are divided into the same connected area. Through this process, the first reconstructed mesh can be split into multiple connected areas. Among them, the above-mentioned pre-set projection surface can be reasonably set according to actual needs. For example, the above-mentioned pre-set projection surface can be the 6 faces of a cube.

[0140] After the initial partitioning, you can try to merge adjacent connected areas. When merging, you need to use a cost function to determine whether two connected areas can be merged. The cost function can be defined by the perimeter and stretching degree of the connected areas. For example, the above merging partitions may include the following process:

[0141] First, the connected regions obtained by the first reconstructed mesh are sorted. Specifically, the connected regions can be sorted according to at least one of the number of triangles contained in each connected region and the degree of proximity between the average normal vector of each connected region and the corresponding projection direction (i.e., the size of the dot product of the two vectors). For example, the connected regions are first sorted in ascending order according to the number of triangles contained in each connected region; for multiple connected regions with the same number of triangles contained, they can be sorted according to the degree of proximity between the average normal vector of each connected region and the corresponding projection direction, and the connected regions with higher proximity are ranked higher. The average normal vector of the connected region can refer to the average value of the normal vectors of all the triangles contained in the connected region.

[0142] Then, each time the first connected region in the connected region sorting is taken, all connected regions adjacent to the connected region are obtained to form a candidate list. For each candidate region in the candidate list, the cost function of the connected region and the candidate region is calculated: COST(M i +M j )=L 2 (M i +M j )+λ*PERIMETER(M i +M j ), the cost function after merging is less than the sum of the cost functions of the two regions before merging (i.e. COST(M i +M j ) <COST(M i )+COST(M j )) of all candidate regions, select the candidate region that minimizes the cost function, merge the current connected region with the candidate region, update the projection surface of the merged connected region, and update the sorting of the connected regions, and iterate until all connected regions are traversed.

[0143] Among them, the adjacency of two connected regions can be understood as the two connected regions sharing at least one edge. i and M j One represents the first connected region in the connected region sorting, and the other represents the candidate region of the first connected region. i +M j ) represents the above M i and M j The cost function of the connected regions after merging. L 2 (M i +M j ) represents the above M i and M j The degree of stretching after the projection of the merged connected region. i +M j ) represents the above M i and M j The perimeter of the connected region after merging. λ represents the weight coefficient. COST(M i ) represents the above M i Cost function. COST(M j ) The above M j The cost function of .

[0144] For example, the stretching degree of the connected region after projection can be determined as follows:

[0145] Assume a triangular surface T with two-dimensional texture coordinates p1, p2, p3, where pi = (si, ti), i takes the values ​​1, 2, and 3 respectively, and the corresponding three-dimensional coordinates are expressed as q1, q2, q3. The calculation process of the affine mapping S(p) = S(s, t) = q is shown in formula (1):

[0146]

[0147] in,<p,p2,p3> represents the area of ​​the triangle pp2p3,<p,p3,p1> represents the area of ​​the triangle pp3p1,<p,p1,p2> Represents the area of ​​the triangle pp1p2, and p represents the coordinates of any point on the triangle p1p2p3. Since the mapping is affine, its partial derivative is a constant on (s, t), and its calculation process is shown in formulas (2) and (3):

[0148]

[0149]

[0150] Where A=<p1,p2,p3> =((s2-s1)(t3-t1)-(s3-s1)(t2-t1)) / 2.

[0151] Then the Jacobian matrix [S S ,S t The calculation process of the larger and smaller singular values ​​of ] is shown in formula (4) and formula (5):

[0152]

[0153]

[0154] Where a = S s ·S s ,b=S s ·S t ,c=S t ·S t . Singular value γ max , γ min They represent the maximum and minimum lengths obtained when a unit length vector is mapped from a two-dimensional texture domain to a three-dimensional surface, i.e., the maximum and minimum local "stretching". The stretching measure on the triangle T is defined as shown in formula (6):

[0155]

[0156] In the entire three-dimensional grid M={T i The definition of the upper stretching measure is shown in formula (7):

[0157]

[0158] Among them, A′(T i ) is the triangle T i The surface area in three-dimensional space.

[0159] From the above, we can see that the cost function of this example is calculated by L 2 (M i ) and PERIMETER(M i ) definition, the two variables are weighed by the λ parameter. After the merging partition step, the number of partitions in the connected region will be reduced compared to the initial partition.

[0160] After the merging partition process, each connected area can be projected. During the projection process, it is necessary to prevent the projected area of ​​the current triangle from occluding the projected area. Exemplarily, the above detection of overlapping areas may include the following process:

[0161] First, traverse all the triangles in the current connected area and project each triangle according to the projection surface corresponding to the current connected area. That is, calculate the two-dimensional coordinates of the three vertices on the current triangle after projection. Then determine whether the projection area of ​​the current triangle overlaps with the projection area of ​​the projected triangle in the current connected area. For example, the overlap of the projection area can be divided into three cases, namely: one or more vertices on the projection area of ​​the current triangle are within the projection area of ​​the projected triangle in the current connected area; the vertices of the projection area of ​​the projected triangle in the current connected area are within the projection area of ​​the current triangle; one or more edges of the projection area of ​​the current triangle intersect with the edges of the projection area of ​​the projected triangle in the current connected area.

[0162] By judging the above three situations, if the projection area of ​​the current triangle overlaps with the projection area of ​​the projected triangle in the current connected area, the projection of the current triangle is skipped first, and the traversal continues to the next triangle in the current connected area. Then, after a round of traversal, it is determined whether there are unprojected triangles in the current connected area, and these unprojected triangles are projected as new connected areas and it is determined whether there is overlap in the projection area. This iteration is repeated until the final projection area does not overlap. Through this process, the connected area with overlapping areas can be split into multiple non-overlapping connected areas. For example, Figures 6a to 6c As shown, a connected region with overlapping areas (i.e. Figure 6a The connected area shown in the figure) is split into two non-overlapping connected areas (i.e. Figure 6b and Figure 6c The two connected regions are shown).

[0163] It can be understood that the process of projecting the triangular surface of the connected area is also the process of obtaining the initial texture coordinates of the vertices of the triangular surface.

[0164] After the above-mentioned overlapping area detection process, for some connected areas with a small number of faces, it is possible to try to merge these connected areas with their adjacent connected areas to reduce the number of connected areas. Exemplarily, the above-mentioned simple area merging can be to merge some connected areas with a small number of faces with the connected areas connected to them, for example, to merge connected areas with a number of faces less than a preset value with the connected areas connected to them, wherein the preset value can be reasonably set according to actual needs, for example, 4, 3, or 2. It is understandable that in the process of simple area merging, it is necessary to ensure that there is no overlap between the projection areas of the various faces of the merged connected areas.

[0165] After the simple partitions are merged, the parameterized final partitioning result of the first reconstructed mesh is obtained. These partitions are arranged into a two-dimensional image according to certain rules to obtain the final texture coordinates of all vertices, which is also the final parameterized result of the first reconstructed mesh. For example, the above two-dimensional arrangement may include the following process:

[0166] First, the scaling ratio of the entire first reconstructed mesh vertices is calculated so that they can be arranged in a two-dimensional image of the specified size. Then, the width and height occupied by each connected area after projection are calculated respectively. Then, the image occupied by the connected area is rasterized, that is, the pixel value of the area occupied by the triangle face is set to 1, and the pixel value of the rest of the area is set to 0.

[0167] Next, for each connected region, try to arrange it on the final image. That is, for the connected region to be arranged, traverse each pixel block on the two-dimensional image as the upper left corner of the connected region, and then try different placement directions. If it is found that the connected region can be placed in the area with the current pixel as the upper left corner (that is, the area occupied by the current connected region on the two-dimensional image does not occupy other placed areas), then all pixels in the area are set to occupied and the texture coordinates of all points in the current connected region are adjusted (including rotation, scaling, and translation). Then, the position of the upper left corner of the current connected region on the two-dimensional image and the width and height of the current connected region are recorded. If it is still not successfully placed after traversing all pixels, the current connected region is scaled and then re-attempted to arrange the scaled connected region on the two-dimensional image. If the current connected region still cannot be successfully placed after scaling, the entire grid is scaled and then rearranged.

[0168] After the above steps, the final parameterization result of the first reconstructed mesh, that is, the texture coordinates of the first reconstructed mesh, can be obtained, and accordingly the first reconstructed mesh including the texture coordinates is obtained.

[0169] In some optional embodiments, between the step of detecting overlapping regions and the step of merging simple regions, a step of extracting connected regions may be further included, such as Figure 7a Specifically, after detecting and splitting overlapping areas, a connected area may be split into multiple connected areas. Among these split connected areas, there may be some disconnected faces or areas, such as Figure 6c As shown in the figure, the above extraction of connected regions is to extract the regions that are not connected to each other in a connected region, and project each of them as an independent connected region, as shown in the figure. Figure 7b Correspondingly, the above-mentioned simple region merging can be a simple region merging of the connected regions obtained after the connected regions are extracted.

[0170] In this embodiment, after performing the first processing on the at least one first connected region, the encoding end again merges the connected regions obtained after the first processing, which can further reduce the number of connected regions and further improve the encoding efficiency.

[0171] Optionally, the encoding end merges the connected regions obtained after the first processing to obtain at least one second connected region, including:

[0172] The encoder end obtains a candidate region set for a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region is any connected region among the connected regions obtained after the first processing;

[0173] The encoder side obtains at least one target candidate region from the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection area of ​​the third connected region does not overlap with a projection area of ​​the target candidate region;

[0174] The encoder merges the third connected region into one of the at least one target candidate region.

[0175] In this embodiment, the target candidate region is a candidate region whose projection area does not overlap with the projection area of ​​the third connected region after being merged with the third connected region. When the target candidate region is merged with the third connected region, the projection surface of the target candidate region and the third connected region can be the projection surface corresponding to the target candidate region, or can be the projection surface corresponding to the merged connected region, etc.

[0176] The above-mentioned merging of the third connected region into one of the at least one target candidate region may, for example, be performed by merging the third connected region into any target candidate region, or by merging the third connected region into the target candidate region with the largest number of faces.

[0177] In some optional embodiments, the third connected region may be a connected region obtained after the first processing, in which the number of faces contained therein is less than a preset value, or a connected region obtained by splitting connected regions with overlapping regions.

[0178] For example, taking the above-mentioned merged candidate area set as a merged candidate area list, after determining the merged candidate area list, the encoder traverses each candidate area in the merged candidate area list, attempts to merge the current connected area into the candidate area, and then detects whether there will be overlap after projection. The projection surface can be the projection surface corresponding to the currently merged candidate area. The overlap detection process here is the same as the above-mentioned overlapping area detection process. If the projection area after projection overlaps, the current candidate area is deleted from the merged candidate area list. In this way, the remaining candidate areas in the merged candidate area list are areas that can be merged with the current area and will not overlap after projection, that is, the above-mentioned target candidate areas.

[0179] This embodiment merges candidate regions whose projection areas after merging do not overlap with the projection area of ​​the third connected region from the connected regions adjacent to the third connected region. This ensures that the projection areas of the connected regions after merging do not overlap, thereby ensuring the accuracy of the texture coordinates of the resulting mesh.

[0180] Optionally, the encoder merging the third connected region into one of the at least one target candidate region includes:

[0181] The encoder merges the third connected region into the first target candidate region;

[0182] The first target candidate area is the target candidate area having the smallest angle between the projection surface and the normal vector of the third connected area among the at least one target candidate area, or the first target candidate area is the target candidate area having the largest number of surfaces among the at least one target candidate area.

[0183] In this embodiment, the target candidate area with the smallest angle between the projection plane and the normal vector of the third connected area, that is, the target candidate area with the closest normal vector to the projection plane and the third connected area, can ensure the minimum distortion after projection.

[0184] Exemplarily, each target candidate area can be traversed, and the target candidate area whose normal vector between the projection surface and the third connected area is closest can be selected as the first target candidate area to ensure that the distortion after projection is minimized; when there are multiple target candidate areas whose normal vector between the projection surface and the third connected area is closest, the target candidate area with the largest number of faces can be selected from the multiple target candidate areas as the first target candidate area, which is conducive to reducing the number of connected areas.

[0185] It can be understood that merging the third connected region into the first target candidate region means adding all faces in the third connected region to the first target candidate region, deleting the original third connected region, and further obtaining the texture coordinates of the newly added vertices in the merged connected region.

[0186] Optionally, there are at least two overlapping vertices between the candidate region and the third connected region.

[0187] Optionally, the first grid is determined according to a simplified grid, and the simplified grid is a grid obtained by performing grid simplification processing on the three-dimensional grid to be encoded;

[0188] The number of vertices of the simplified mesh is smaller than the number of vertices of the three-dimensional mesh to be encoded, or the number of faces of the simplified mesh is smaller than the number of faces of the three-dimensional mesh to be encoded.

[0189] In this embodiment, the first mesh is determined according to the simplified mesh. For example, the first mesh may be the simplified mesh, or the first mesh may be a mesh obtained by displacing each vertex of the simplified mesh.

[0190] The above-mentioned mesh simplification process can be understood as simplifying the three-dimensional mesh to be encoded into a mesh with relatively fewer points and faces, while maintaining the shape of the original mesh as much as possible.

[0191] For example, Figure 8As shown, the vertices at both ends of an edge of a 3D mesh to be encoded are merged into a single vertex, and the connection between the two vertices is deleted. This process is repeated throughout the mesh according to a certain rule to reduce the number of faces and vertices in the mesh to a target value, where the target value can be determined based on the number required for lossy encoding. Specifically, during the mesh simplification process, a certain error metric can be selected to optimize the simplification result. For example, the sum of the coefficients of the equations of all adjacent faces of a vertex can be selected as the error metric for that vertex. In other words, the error of a vertex is measured by the sum of the coefficients of the equations of all adjacent faces of that vertex. The error of the corresponding edge is the sum of the errors of the two vertices on that edge. In other words, the error resulting from merging an edge is the sum of the distances from the merged vertex to all adjacent faces of the original two vertices of the edge. After determining the error metric, iterative mesh simplification is performed. For example, the errors of the vertices of the 3D mesh to be encoded are calculated to obtain the error of each edge of the 3D mesh to be encoded. Each edge is then sorted from smallest to largest error, and the edge with the smallest error is selected for merging each time. Simultaneously, the position of the merged vertex is calculated, and the errors of all edges associated with the merged vertex are updated. That is, the order of edge arrangement is updated to ensure that each iteration is based on the global error metric. Through iteration, the faces of the 3D mesh to be encoded are simplified to the number required to meet the lossy encoding requirements.

[0192] This embodiment performs mesh simplification processing on the three-dimensional mesh to be encoded to obtain a simplified mesh, and compresses and encodes the first mesh determined by the simplified mesh. Since the number of faces or vertices of the mesh is reduced after the mesh simplification processing, this not only improves the efficiency of mesh encoding, but also further reduces the transmission resources or storage resources required for the mesh code stream.

[0193] Optionally, the method further includes:

[0194] The encoding end subdivides and deforms the simplified grid to obtain a second grid;

[0195] The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, including:

[0196] The encoder calculates the displacement of each vertex of the second mesh according to the second reconstructed mesh and the second mesh;

[0197] The encoding end encodes the displacement of each vertex of the second mesh to obtain a displacement code stream;

[0198] The encoding end performs three-dimensional grid reconstruction according to the first reconstructed grid and the displacement code stream to obtain a third reconstructed grid;

[0199] The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the third reconstructed grid to obtain a second texture map.

[0200] For example, the basic idea of ​​subdivision deformation can be as follows Figure 9 As shown in Figure 1, the input 2D curve (represented by a 2D polyline), called the original curve, is first downsampled to generate a basic curve / polyline, called the simplified curve. Then, a subdivision curve can be generated based on the polyline of the simplified curve. The polyline of the subdivision curve is then deformed to obtain a deformed curve that is closer to the original curve, that is, a displacement vector is calculated for each vertex of the subdivision curve, as shown in Figure 1. Figure 9 As shown by the arrows in , the shape of the subdivision deformation curve is made as close as possible to the shape of the original curve; then, each vertex can be moved accordingly according to the displacement vector of each vertex of the subdivision curve to obtain a subdivision deformation curve with a shape close to the original curve.

[0201] Specifically, the simplified mesh can be subdivided to obtain a subdivided mesh. The specific implementation method of the subdivision can be selected arbitrarily. One optional implementation method is the midpoint subdivision method. For example, if the faces included in the simplified mesh are triangular faces, each triangular face can be subdivided into four sub-triangular faces in each subdivision iteration, such as Figure 10 As shown in Figure 2, a new vertex is introduced in the middle of each edge of the simplified mesh, and the subdivision of geometric information and attribute information is performed independently because the connection relationship between geometric information and attribute information is usually different.

[0202] For the above subdivision method, the midpoint v of the newly introduced edge (v1, v2) can be calculated using formula (8): 12 Position Pos(v 12 ):

[0203]

[0204] Among them, Pos(v1) and Pos(v2) are the geometric coordinates of vertices v1 and v2 respectively. 12 The texture coordinates can be calculated in the same way as the geometric coordinates, so I will not go into details here.

[0205] For the subdivided mesh, the nearest neighbor of each point on the simplified mesh (including points on the simplified mesh surface) is found. Optionally, a data structure such as a k-dimensional tree (kdTree) can be used to accelerate the search. The geometric coordinate displacement of each vertex in the subdivided mesh is obtained by calculating the distance between the geometric coordinates of each vertex on the subdivided mesh and its nearest neighbor on the simplified mesh. The corresponding vertex can then be moved based on the displacement of the geometric coordinates of each vertex in the subdivided mesh to obtain the subdivided deformed mesh, i.e., the second mesh mentioned above.

[0206] After obtaining the subdivided deformed mesh, the encoder can calculate the displacement of each vertex of the subdivided deformed mesh based on the subdivided deformed mesh and the second reconstructed mesh. For example, the encoder can subdivide the second reconstructed mesh, determine the correspondence between each vertex of the subdivided second reconstructed mesh and each vertex of the subdivided deformed mesh, and calculate the displacement of each vertex based on each vertex of the subdivided second reconstructed mesh and each vertex of the corresponding subdivided deformed mesh. The encoder can then encode the calculated displacements. This embodiment does not limit the method for encoding the calculated displacements. For example, the calculated displacements can be encoded using a video encoding method. Specifically, the steps may include: converting the coordinate system of the displacement, i.e., converting the coordinate system of the displacement of each vertex to a coordinate system constructed by the normal vector of the vertex corresponding to the displacement and two components tangent to the normal vector; then, performing a transformation on the displacement, such as a wavelet transform, quantizing the transformed coefficients, arranging them in the image in scanning order, and applying video encoding to the image; or, directly encoding the calculated displacements using entropy coding.

[0207] The video encoding method may include the following steps:

[0208] (1) Coordinate system conversion:

[0209] The generated displacements are in the same global coordinate system as the input mesh. An optional optimization method is to transform them into a local coordinate system, where each vertex's local coordinate system is defined by the vertex's normal vector on the subdivided deformed mesh. The advantage of this approach is that the normal component of the displacement has a more significant impact on the quality of the reconstructed mesh than the two tangential components, so a larger quantization parameter can be set for the tangential component.

[0210] (2) Wavelet transform:

[0211] For the quantized displacement, wavelet transform can be applied thereto. An optional wavelet transform such as linear wavelet transform is shown below:

[0212] The prediction process is defined as shown in formula (12):

[0213]

[0214] Where v is the newly inserted midpoint on the edge (v1, v2), Signal(v), Signal(v1) and Signal(v2) are the displacements corresponding to vertices v, v1 and v2 respectively, and "←" indicates that the content on the left of the arrow is updated by the content on the right of the arrow.

[0215] The update process is defined as shown in formula (13):

[0216]

[0217] where v * is the set of all vertices adjacent to vertex v.

[0218] (3) Wavelet coefficient quantization:

[0219] There are many ways to quantize wavelet coefficients. One method is shown in equations (14) and (15):

[0220] disp[v].d[k]=floor(disp[v].d[k]*scale[k]) -(14)

[0221]

[0222] Where disp[v] represents the wavelet coefficient of the displacement at the vth vertex, d[k] represents the kth coefficient of the displacement, and floor indicates rounding down. bitDepthPosition represents the bit depth of the current grid position, and qp[k] represents the quantization parameter of the kth coefficient. As mentioned earlier, after the displacement is converted to a coordinate system, its normal component has a more significant impact on quality than the tangential component, so a larger quantization parameter can be used for the tangential component.

[0223] At the same time, according to the characteristics of wavelet transform, different quantization parameters can be used for the newly generated vertices and the original vertices. That is, for the subdivided vertices, the quantization parameter update is shown in formula (16):

[0224] scale[k]=scale[k]*lodScale[k] -(16)

[0225] Among them, lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.

[0226] (4) Wavelet coefficient arrangement:

[0227] For the wavelet coefficients generated after wavelet transform, arrange them on the two-dimensional image as follows:

[0228] Traverse the wavelet coefficients in order from low frequency to high frequency;

[0229] For each coefficient, determine the index of the NxM pixel block (e.g., N=M=16) where it should be stored in raster scan order of the block;

[0230] Calculate the position of the corresponding NxM pixel block on the image according to the Morton order.

[0231] The arrangement method is not limited here, that is, other arrangement schemes can be used, such as zigzag order, raster order, etc. The encoder can explicitly specify the corresponding arrangement scheme in the bitstream.

[0232] (5) Video encoding:

[0233] After arranging the wavelet coefficients onto a two-dimensional image, they can be directly encoded using a video encoder. The solution of this application is independent of the video encoding technology used; the type of video encoder can be explicitly specified in the bitstream. Furthermore, if quantization is applied before video encoding, lossless methods can be used for video encoding, eliminating the need to reconstruct displacements at the encoder end.

[0234] The calculated displacement is directly compressed using existing entropy coding. For example, context adaptive binary arithmetic coding (CABAC) can be used to compress the calculated displacement. It should be noted that this embodiment can also directly use entropy coding on the coefficients after wavelet transformation. Corresponding quantization can be performed before entropy coding.

[0235] After obtaining the displacement code stream, the encoding end can reconstruct the displacement based on the displacement code stream and perform a three-dimensional mesh reconstruction based on the reconstructed displacement and the first reconstructed mesh to obtain a third reconstructed mesh. For example, the first reconstructed mesh can be subdivided to obtain a subdivided first reconstructed mesh. Then, each vertex of the subdivided first reconstructed mesh corresponding to the displacement of each reconstructed vertex can be moved to obtain a third reconstructed mesh. The third reconstructed mesh is also a reconstructed subdivided deformed mesh. The first texture map can then be converted based on the three-dimensional mesh to be encoded and the third reconstructed mesh to obtain a second texture map. The process of converting the first texture map based on the three-dimensional mesh to be encoded and the third reconstructed mesh can be referred to the relevant description of the texture map conversion method described above and will not be repeated here.

[0236] In this embodiment, when a mesh simplification process is performed on a three-dimensional mesh to be encoded, the simplified mesh is subdivided and deformed to obtain a second mesh, the displacement of each vertex of the second mesh is calculated based on the second reconstructed mesh and the second mesh, the displacement of each vertex of the second mesh is encoded to obtain a displacement code stream, and a three-dimensional mesh is reconstructed based on the first reconstructed mesh and the displacement code stream to obtain a third reconstructed mesh, thereby ensuring the quality of the reconstructed mesh.

[0237] In some optional embodiments, when the simplified mesh is subjected to subdivision deformation, the method further includes:

[0238] The encoder determines the displacement of the geometric coordinates of each vertex of the simplified mesh based on the displacement of the geometric coordinates of each vertex of the subdivided mesh; and the encoder moves each vertex of the simplified mesh based on the displacement of the geometric coordinates of each vertex of the simplified mesh to obtain a first mesh. The displacement of the geometric coordinates of each vertex of the subdivided mesh can be found in the relevant description of the previous embodiment and is not further described here.

[0239] Optionally, the method further includes:

[0240] The encoding end performs a first operation on first encoding information, wherein the first encoding information includes the grid code stream and the texture map code stream, and the first operation includes at least one of the following: storage and transmission.

[0241] Optionally, the first coding information further includes at least one of the following:

[0242] Displacement code stream;

[0243] The first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

[0244] It is understandable that, when the encoding end obtains the above-mentioned displacement code stream through encoding, the above-mentioned first encoding information also includes the displacement code stream, and when the above-mentioned displacement code stream is not obtained through encoding, the above-mentioned first encoding information does not include the displacement code stream.

[0245] The above-mentioned information related to grid coding, for example, the intra-frame coding flag, is used to indicate whether the current frame needs to be constructed according to the reference frame grid; the reference frame list, is used to indicate the index of the reference frame required for the current frame, and the reference frame list can be applied to the reconstruction of the first grid; the static grid encoder type, etc. The above-mentioned information related to texture map coding, for example, the video encoder type. The above-mentioned information related to grid subdivision and deformation, for example, the subdivision deformation flag, is used to indicate whether the reconstructed grid needs to be subdivided and deformed; the subdivision method, that is, the method of grid subdivision when reconstructing the subdivided deformed grid. The above-mentioned information related to grid displacement, for example, the displacement encoding method; the displacement transformation method; the wavelet coefficient arrangement method, etc. The above-mentioned information related to grid parameterization processing, for example, the algorithm used for grid parameterization processing (for example, Isochart algorithm, orthogonal projection algorithm, etc.).

[0246] It can be understood that the above-mentioned first auxiliary information can be used to assist the decoding end in decoding. For example, the decoding end decodes the grid code stream according to the information related to the grid coding, decodes the texture map code stream according to the information related to the texture map coding, subdivides and deforms the decoded grid according to the information related to the grid subdivision and deformation, performs grid parameterization on the decoded grid according to the information related to the grid parameterization processing, and decodes the displacement code stream according to the information related to the grid displacement. This is conducive to the decoding end to decode the encoded grid more accurately.

[0247] The following combination Figure 11 The three-dimensional grid coding method provided in the embodiments of the present application is illustrated by way of example.

[0248] See also Figure 11 , the encoding end performs mesh simplification processing on the input mesh to obtain a simplified mesh, and subdivides and deforms the simplified mesh to obtain a base mesh (i.e., the first mesh) and a subdivided deformed mesh (i.e., the second mesh). The base mesh is compressed and encoded to obtain a mesh code stream. The base mesh is reconstructed based on the mesh code stream to obtain a reconstructed base mesh (i.e., the first reconstructed mesh). The first reconstructed mesh is mesh parameterized to obtain a first reconstructed mesh (i.e., the second reconstructed mesh) containing texture coordinates. The displacement of each vertex of the subdivided deformed mesh is calculated based on the subdivided deformed mesh and the second reconstructed mesh. The displacement may include the displacement of the geometric coordinates and the displacement of the texture coordinates. The calculated displacement of each vertex is transformed, quantized, and the processed displacement is encoded to obtain a displacement code stream. Furthermore, the encoding end performs displacement reconstruction based on the displacement code stream to obtain a reconstructed position, and reconstructs a subdivided deformed grid (i.e., a third reconstructed grid) based on the reconstructed displacement and the first reconstructed grid, and converts the input texture map (i.e., the first texture map) based on the reconstructed subdivided deformed grid and the input grid to obtain a converted texture map (i.e., the second texture map), and uses a video encoder to encode the converted texture map to obtain a texture map code stream, and then the encoding end can store or transmit the obtained grid code stream, texture map code stream, and displacement code stream.

[0249] See Figure 12 , Figure 12 This is a flow chart of a three-dimensional grid decoding method provided by an embodiment of the present application. The method can be executed by a decoding end, such as Figure 12 As shown, the following steps are included:

[0250] Step 1201: The decoding end obtains the grid code stream and the texture map code stream.

[0251] In this embodiment, the mesh code stream and texture map code stream are generated by the encoder according to the three-dimensional mesh coding method. Exemplarily, the decoder can receive the mesh code stream and texture map code stream from the encoder.

[0252] Step 1202: The decoding end decodes the grid code stream to obtain a third grid, and decodes the texture map code stream to obtain a third texture map.

[0253] In this embodiment, the third grid is a grid obtained after decoding the grid code stream. The third grid can also be called a reconstructed base grid. The third texture image is a texture image obtained after decoding the texture image code stream.

[0254] Specifically, the decoding end should use the decoding method corresponding to the trellis coding method of the encoding end to decode the trellis code stream. For example, if the encoding end uses intra-frame mode encoding, the decoding end should also use intra-frame mode decoding; if the encoding end uses inter-frame mode encoding, the decoding end should also use inter-frame mode decoding; if the encoding end uses skip mode encoding, the decoding end should also use skip mode decoding. Figure 13 The following three decoding methods are given as examples:

[0255] like Figure 13 As shown, if it is intra mode, a static grid decoder is used for decoding; if it is inter mode, the corresponding motion vector is decoded, and the corresponding basic grid is reconstructed according to the reference frame in the grid buffer; if it is skip mode, the corresponding reference frame in the grid buffer is directly used as the basic grid of the current frame.

[0256] Similarly, the decoding end should use a decoding method corresponding to the texture image encoding method of the encoding end to decode the texture image bit stream. For example, if the encoding end uses a video encoder, the decoding end uses the corresponding video encoder to decode the texture image bit stream. In some optional embodiments, the decoding end can also perform color space conversion on the texture image obtained by decoding to obtain a color space that is consistent with the texture image input by the encoding end. Figure 1 consistent image format.

[0257] Step 1203: The decoding end performs grid parameterization processing on the third grid to obtain a fourth grid, wherein the fourth grid is a grid including texture coordinates.

[0258] In this embodiment, the decoder can use the same mesh parameterization processing method as the encoder to perform mesh parameterization processing on the third mesh to obtain a third mesh containing texture coordinates, namely the fourth mesh. The decoder can then apply the third texture map and the fourth mesh.

[0259] In an embodiment of the present application, a grid parameterization process is performed on the grid obtained by decoding the grid code stream at the decoding end to obtain the texture coordinates of the grid. This can reduce the storage resources or transmission resources occupied by the three-dimensional grid code stream while ensuring the texture coordinates of the grid that can be obtained by decoding.

[0260] Optionally, the decoding end performs grid parameterization processing on the third grid to obtain a fourth grid, including:

[0261] The decoding end performs grid parameterization processing on the third grid using an orthogonal projection algorithm to obtain a fourth grid.

[0262] Optionally, the decoding end performs grid parameterization processing on the third grid using an orthogonal projection algorithm to obtain a fourth grid, including:

[0263] The decoding end divides the third grid into at least two connected regions, wherein each of the connected regions includes at least one face of the third grid, and faces having the same projection surface and being connected in three-dimensional space belong to the same connected region;

[0264] The decoding end merges the at least two connected regions to obtain at least one first connected region;

[0265] The decoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each first connected region to detect overlapping regions, and splitting the first connected regions having overlapping regions into at least two non-overlapping connected regions;

[0266] The decoding end merges the connected regions obtained after the first processing to obtain at least one second connected region;

[0267] The decoding end arranges the projection area of ​​the at least one second connected area in two dimensions to obtain a fourth grid.

[0268] Optionally, the decoding end merges the connected regions obtained after the first processing to obtain at least one second connected region, including:

[0269] The decoding end obtains a candidate region set for a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region is any connected region among the connected regions obtained after the first processing;

[0270] The decoding end obtains at least one target candidate region from the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection area of ​​the third connected region does not overlap with a projection area of ​​the target candidate region;

[0271] The decoding end merges the third connected region into one of the at least one target candidate region.

[0272] Optionally, the decoding end merges the third connected region into one of the at least one target candidate region, including:

[0273] The decoding end merges the third connected region into the first target candidate region;

[0274] The first target candidate area is the target candidate area having the smallest angle between the projection surface and the normal vector of the third connected area among the at least one target candidate area, or the first target candidate area is the target candidate area having the largest number of surfaces among the at least one target candidate area.

[0275] Optionally, there are at least two overlapping vertices between the candidate region and the third connected region.

[0276] Optionally, the method further includes:

[0277] The decoding end obtains the displacement code stream;

[0278] The decoding end subdivides and deforms the fourth grid to obtain a fifth grid;

[0279] The decoding end performs three-dimensional grid reconstruction according to the fifth grid and the displacement obtained by decoding the displacement code stream to obtain a fourth reconstructed grid.

[0280] In this embodiment, upon receiving a displacement bitstream, the decoder decodes the bitstream. For example, if the encoder compressed the displacement using video encoding, the decoder decodes the bitstream using a corresponding video decoder and restores it from the two-dimensional image in the corresponding order according to the permutation scheme. It then performs inverse transformation, inverse quantization, and other operations to restore the displacement consistent with that obtained by the encoder. If entropy encoding is used, direct entropy decoding can be performed.

[0281] After the base mesh and displacement decoding and reconstruction are completed, the subdivided deformed mesh is reconstructed based on the two parts. For example, if the displacement encoding process uses a reference frame as a reference, that is, the displacement of the vertices in the area matching the reference frame in the decoded displacement is the residual of the displacement of the current frame vertex relative to the reference frame vertex, then the displacement of the vertices in the matching area of ​​the current frame is first restored by the vertices and residuals of the subdivided deformed mesh reconstructed by the reference frame, and then the subsequent reconstruction steps are performed. That is, the reconstructed base mesh is subdivided, which is consistent with the subdivision method of the encoding end. The corresponding displacement is added to each vertex of the subdivided mesh, as shown in formula (17):

[0282] deformedmesh[i].v[k]=subdivmesh[i].v[k]+displacement[k] -(17)

[0283] Among them, subdivmesh[i].v[k] is the geometric coordinates of the k-th vertex after the base mesh is subdivided in the current frame (index is i), displacement[k] is the displacement corresponding to the k-th vertex, and deformedmesh[i].v[k] is the geometric coordinates of the k-th vertex after subdivision and deformation in the current frame.

[0284] Optionally, the method further includes:

[0285] The decoding end obtains first auxiliary information, wherein the first auxiliary information includes at least one of the following: information related to grid coding, information related to texture map coding, information related to grid subdivision and deformation, information related to grid displacement, and information related to grid parameterization processing.

[0286] In this embodiment, when the decoding end obtains the first auxiliary information, it can perform decoding based on the first auxiliary information. For example, the decoding end decodes the grid code stream according to information related to grid encoding, decodes the texture map code stream according to information related to texture map encoding, subdivides and deforms the decoded grid according to information related to grid subdivision and deformation, performs grid parameterization on the decoded grid according to information related to grid parameterization processing, and decodes the displacement code stream according to information related to grid displacement. This is conducive to the decoding end decoding the encoded grid more accurately.

[0287] It should be noted that the above-mentioned first auxiliary information can be encoded into an independent code stream, or it can be carried in the above-mentioned grid code stream, texture map code stream and displacement code stream. For example, information related to grid coding is carried in the above-mentioned grid code stream, information related to texture map coding is carried in the above-mentioned texture map code stream, and information related to grid displacement is carried in the displacement code stream, etc.

[0288] It should be noted that the implementation of the above embodiments can be found in Figure 3 The relevant description of the illustrated embodiment is omitted here.

[0289] The following combination Figure 14 The three-dimensional grid decoding method provided in the embodiment of the present application is illustrated by way of example.

[0290] See also Figure 14 The decoding end obtains the decoded code streams of each part and reconstructs it. After the basic mesh is reconstructed, the mesh parameterization is performed to obtain the corresponding basic mesh containing texture coordinates. The basic mesh is subdivided and the displacement obtained by decoding is applied to obtain the subdivided deformed mesh reconstructed by the decoding end, that is, the reconstructed mesh finally output by the decoding end.

[0291] It should be noted that the 3D mesh encoding method provided in the embodiments of this application can be executed by a 3D mesh encoding device, or by a control module within the 3D mesh encoding device that is configured to execute the 3D mesh encoding method. In the embodiments of this application, the 3D mesh encoding device executing the 3D mesh encoding method is used as an example to illustrate the 3D mesh encoding device provided in the embodiments of this application.

[0292] See Figure 15 , Figure 15 This is a structural diagram of a three-dimensional grid coding device provided in an embodiment of the present application. Figure 15 As shown, the three-dimensional grid encoding device 1500 includes:

[0293] A first encoding module 1501 is configured to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded;

[0294] A first reconstruction module 1502 is configured to perform three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid;

[0295] A first processing module 1503 is configured to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;

[0296] a conversion module 1504, configured to convert a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded;

[0297] The second encoding module 1505 is configured to encode the second texture map to obtain a texture map code stream.

[0298] Optionally, the first processing module is specifically configured to:

[0299] An orthogonal projection algorithm is used to perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid.

[0300] Optionally, the first processing module is specifically configured to:

[0301] Dividing the first reconstructed mesh into at least two connected regions, wherein each of the connected regions includes at least one face of the first reconstructed mesh, and faces having the same projection surface and being connected in three-dimensional space belong to the same connected region;

[0302] Merging the at least two connected regions to obtain at least one first connected region;

[0303] performing a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each first connected region to detect overlapping regions, and splitting the first connected regions having overlapping regions into at least two non-overlapping connected regions;

[0304] Merging the connected regions obtained after the first processing to obtain at least one second connected region;

[0305] The projection area of ​​the at least one second connected area is two-dimensionally arranged to obtain a second reconstructed grid.

[0306] Optionally, the first processing module is specifically configured to:

[0307] Obtaining a set of candidate regions for a third connected region, wherein the set of candidate regions includes at least one candidate region, the candidate region being a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region being any connected region among the connected regions obtained after the first processing;

[0308] Acquire at least one target candidate region of the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection area of ​​the third connected region does not overlap with a projection area of ​​the target candidate region;

[0309] The third connected region is merged into one of the at least one target candidate region.

[0310] Optionally, the first processing module is specifically configured to:

[0311] Merging the third connected region into the first target candidate region;

[0312] The first target candidate area is the target candidate area having the smallest angle between the projection surface and the normal vector of the third connected area among the at least one target candidate area, or the first target candidate area is the target candidate area having the largest number of surfaces among the at least one target candidate area.

[0313] Optionally, there are at least two overlapping vertices between the candidate region and the third connected region.

[0314] Optionally, the first grid is determined according to a simplified grid, and the simplified grid is a grid obtained by performing grid simplification processing on the three-dimensional grid to be encoded;

[0315] The number of vertices of the simplified mesh is smaller than the number of vertices of the three-dimensional mesh to be encoded, or the number of faces of the simplified mesh is smaller than the number of faces of the three-dimensional mesh to be encoded.

[0316] Optionally, the device further comprises:

[0317] A first subdivision deformation module, configured to perform subdivision deformation on the simplified mesh to obtain a second mesh;

[0318] The conversion module is specifically used for:

[0319] Calculating a displacement of each vertex of the second mesh according to the second reconstructed mesh and the second mesh;

[0320] Encoding the displacement of each vertex of the second mesh to obtain a displacement code stream;

[0321] Performing three-dimensional grid reconstruction based on the first reconstructed grid and the displacement code stream to obtain a third reconstructed grid;

[0322] The first texture map is converted according to the three-dimensional grid to be encoded and the third reconstructed grid to obtain a second texture map.

[0323] Optionally, the device further comprises:

[0324] The first execution module is configured to perform a first operation on first coded information, wherein the first coded information includes the grid code stream and the texture map code stream, and the first operation includes at least one of the following: storage and transmission.

[0325] Optionally, the first coding information further includes at least one of the following:

[0326] Displacement code stream;

[0327] The first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

[0328] The three-dimensional grid coding device in the embodiment of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or a network-side device, or it can be a device other than a terminal or a network-side device. For example, the terminal can include but is not limited to the types of terminals 11 listed above, and the network-side device can include but is not limited to the types of network-side devices 12 listed above. Other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiment of the present application.

[0329] The three-dimensional grid coding device provided in the embodiment of the present application can achieve Figure 3 The various processes implemented by the method embodiment achieve the same technical effect and are not described here again to avoid repetition.

[0330] It should be noted that the 3D mesh decoding method provided in the embodiments of the present application can be executed by a 3D mesh decoding device, or by a control module within the 3D mesh decoding device that is configured to execute the 3D mesh decoding method. The 3D mesh decoding device provided in the embodiments of the present application is described using the 3D mesh decoding device executing the 3D mesh decoding method as an example.

[0331] See Figure 16 , Figure 16 This is a structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application. Figure 16 As shown, the three-dimensional grid decoding device 1600 includes:

[0332] The first acquisition module 1601 is used to acquire a grid code stream and a texture map code stream;

[0333] A decoding module 1602 is configured to decode the mesh code stream to obtain a third mesh, and decode the texture map code stream to obtain a third texture map;

[0334] The second processing module 1603 is configured to perform mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

[0335] Optionally, the second processing module is specifically configured to:

[0336] An orthogonal projection algorithm is used to perform grid parameterization processing on the third grid to obtain a fourth grid.

[0337] Optionally, the second processing module is specifically configured to:

[0338] Dividing the third grid into at least two connected regions, wherein each of the connected regions includes at least one face of the third grid, and faces having the same projection surface and being connected in three-dimensional space belong to the same connected region;

[0339] Merging the at least two connected regions to obtain at least one first connected region;

[0340] performing a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each first connected region to detect overlapping regions, and splitting the first connected regions having overlapping regions into at least two non-overlapping connected regions;

[0341] Merging the connected regions obtained after the first processing to obtain at least one second connected region;

[0342] The projection area of ​​the at least one second connected area is two-dimensionally arranged to obtain a fourth grid.

[0343] Optionally, the second processing module is specifically configured to:

[0344] Obtaining a set of candidate regions for a third connected region, wherein the set of candidate regions includes at least one candidate region, the candidate region being a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region being any connected region among the connected regions obtained after the first processing;

[0345] Acquire at least one target candidate region of the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection area of ​​the third connected region does not overlap with a projection area of ​​the target candidate region;

[0346] The third connected region is merged into one of the at least one target candidate region.

[0347] Optionally, the second processing module is specifically configured to:

[0348] Merging the third connected region into the first target candidate region;

[0349] The first target candidate area is the target candidate area having the smallest angle between the projection surface and the normal vector of the third connected area among the at least one target candidate area, or the first target candidate area is the target candidate area having the largest number of surfaces among the at least one target candidate area.

[0350] Optionally, there are at least two overlapping vertices between the candidate region and the third connected region.

[0351] Optionally, the device further comprises:

[0352] The second acquisition module is used to obtain the displacement code stream;

[0353] A second subdivision deformation module, configured to perform subdivision deformation on the fourth mesh to obtain a fifth mesh;

[0354] The second reconstruction module is configured to perform three-dimensional grid reconstruction according to the fifth grid and the displacement obtained by decoding the displacement code stream to obtain a fourth reconstructed grid.

[0355] Optionally, the device further comprises:

[0356] The third acquisition module is used to obtain first auxiliary information, wherein the first auxiliary information includes at least one of the following: information related to grid coding, information related to texture map coding, information related to grid subdivision and deformation, information related to grid displacement, and information related to grid parameterization processing.

[0357] The three-dimensional grid decoding device in the embodiment of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a network-side device or a terminal, or it can be other devices other than network-side devices or terminals. For example, the network-side device can include but is not limited to the types of network-side devices 12 listed above, the terminal can include but is not limited to the types of terminals 11 listed above, and other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiment of the present application.

[0358] The three-dimensional grid decoding device provided in the embodiment of the present application can achieve Figure 12 The various processes implemented by the method embodiment achieve the same technical effect and are not described here again to avoid repetition.

[0359] Optional, such as Figure 17As shown, an embodiment of the present application further provides a communication device 1700, including a processor 1701 and a memory 1702. The memory 1702 stores a program or instruction that can be executed on the processor 1701. For example, when the communication device 1700 is an encoding end, the program or instruction, when executed by the processor 1701, implements the various steps of the above-mentioned three-dimensional mesh encoding method embodiment and can achieve the same technical effect. When the communication device 1700 is a decoding end, the program or instruction, when executed by the processor 1701, implements the various steps of the above-mentioned three-dimensional mesh decoding method embodiment and can achieve the same technical effect. To avoid repetition, it is not repeated here.

[0360] An embodiment of the present application also provides a decoding end, including a processor and a communication interface, the processor is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded; based on the grid code stream, a three-dimensional grid is reconstructed to obtain a first reconstructed grid; the first reconstructed grid is grid parameterized to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; a first texture map is converted according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional grid to be encoded; the second texture map is encoded to obtain a texture map code stream. This encoding end embodiment corresponds to the above-mentioned encoding end side method embodiment, and each implementation process and implementation method of the above-mentioned method embodiment can be applied to this encoding end embodiment and can achieve the same technical effect. Specifically, Figure 18 A schematic diagram of the hardware structure of an encoding end to implement an embodiment of the present application.

[0361] The encoding end 1800 includes but is not limited to: a radio frequency unit 1801, a network module 1802, an audio output unit 1803, an input unit 1804, a sensor 1805, a display unit 1806, a user input unit 1807, an interface unit 1808, a memory 1809 and at least some of the components of the processor 1810.

[0362] Those skilled in the art will understand that the encoding end 1800 may also include a power supply (such as a battery) to power each component, and the power supply may be logically connected to the processor 1810 through a power management system, thereby realizing functions such as charging, discharging, and power consumption management through the power management system. Figure 18 The encoding end structure shown in the figure does not constitute a limitation on the encoding end. The encoding end may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0363] It should be understood that in an embodiment of the present application, the input unit 1804 may include a graphics processing unit (GPU) 18041 and a microphone 18042, and the graphics processor 18041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1806 may include a display panel 18061, and the display panel 18061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1807 includes a touch panel 18071 and at least one of other input devices 18072. The touch panel 18071 is also called a touch screen. The touch panel 18071 may include two parts: a touch detection device and a touch controller. Other input devices 18072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0364] In the embodiment of the present application, after receiving downlink data from a network-side device, the RF unit 1801 may transmit the data to the processor 1810 for processing. Furthermore, the RF unit 1801 may send uplink data to the network-side device. Typically, the RF unit 1801 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like.

[0365] The memory 1809 can be used to store software programs or instructions and various data. The memory 1809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1809 may include a volatile memory or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 1809 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0366] Processor 1810 may include one or more processing units. Optionally, processor 1810 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 1810.

[0367] Among them, processor 1810 is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded; reconstruct the three-dimensional grid based on the grid code stream to obtain a first reconstructed grid; perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; convert a first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, wherein the first texture map is the texture map corresponding to the three-dimensional grid to be encoded; and encode the second texture map to obtain a texture map code stream.

[0368] It can be understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the aforementioned method embodiment and achieve the same or corresponding technical effects. To avoid repetition, it will not be repeated here.

[0369] This embodiment of the present application also provides a decoding end, comprising a processor and a communication interface. The processor is configured to obtain a mesh code stream and a texture map code stream; decode the mesh code stream to obtain a third mesh, and decode the texture map code stream to obtain a third texture map; and perform mesh parameterization on the third mesh to obtain a fourth mesh, wherein the fourth mesh includes texture coordinates. This decoding end embodiment corresponds to the above-mentioned decoding end method embodiment. The various implementation processes and methods of the above-mentioned method embodiment are applicable to this decoding end embodiment and can achieve the same technical effects.

[0370] Specifically, the embodiment of the present application also provides a decoding end. Figure 19 As shown, decoding end 1900 includes an antenna 1901, a radio frequency device 1902, a baseband device 1903, a processor 1904, and a memory 1905. Antenna 1901 is connected to radio frequency device 1902. In the uplink direction, radio frequency device 1902 receives information via antenna 1901 and sends the received information to baseband device 1903 for processing. In the downlink direction, baseband device 1903 processes the information to be transmitted and sends it to radio frequency device 1902. Radio frequency device 1902 processes the received information and then sends it through antenna 1901.

[0371] The method executed by the decoding end in the above embodiment may be implemented in the baseband device 1903 , which includes a baseband processor.

[0372] The baseband device 1903 may include, for example, at least one baseband board on which a plurality of chips are arranged, such as Figure 19As shown, one of the chips is, for example, a baseband processor, which is connected to the memory 1905 through a bus interface to call the program in the memory 1905 to execute the network device operations shown in the above method embodiment.

[0373] The decoding end may further include a network interface 1906 , which is, for example, a Common Public Radio Interface (CPRI).

[0374] Specifically, the decoding end 1900 of the embodiment of the present application further includes: instructions or programs stored in the memory 1905 and executable on the processor 1904, and the processor 1904 calls the instructions or programs in the memory 1905 to execute Figure 16 The methods executed by the modules shown achieve the same technical effects, so they will not be described here to avoid repetition.

[0375] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned three-dimensional grid coding method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0376] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. In some examples, the readable storage medium may be a non-transitory readable storage medium.

[0377] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned three-dimensional grid coding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0378] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0379] An embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the various processes of the above-mentioned three-dimensional grid coding method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0380] The embodiment of the present application also provides a three-dimensional grid encoding and decoding system, including: an encoding end and a decoding end, wherein the encoding end is used to perform the following Figure 3And each process of each method embodiment above, the decoding end is used to perform the following Figure 12 And each process of each method embodiment mentioned above, and can achieve the same technical effect, to avoid repetition, no further details are given here.

[0381] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0382] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general-purpose hardware platform, or of course, by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes a number of instructions for enabling a terminal or network-side device to execute the methods described in each embodiment of the present application.

[0383] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of this application and the scope of protection of the claims. These implementation methods are all within the protection of this application.

Claims

1. A three-dimensional grid coding method, characterized in that: include: The encoding end compresses and encodes the first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded; The encoding end performs three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid; The encoder performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates; The encoding end converts the first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is the texture map corresponding to the three-dimensional mesh to be encoded; The encoding end encodes the second texture map to obtain a texture map code stream.

2. The method according to claim 1, characterized in that The encoder performs grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, including: The encoding end performs grid parameterization processing on the first reconstructed grid using an orthogonal projection algorithm to obtain a second reconstructed grid.

3. The method according to claim 2, characterized in that The encoding end performs grid parameterization processing on the first reconstructed grid using an orthogonal projection algorithm to obtain a second reconstructed grid, including: The encoder divides the first reconstructed grid into at least two connected regions, wherein each of the connected regions includes at least one face of the first reconstructed grid, and faces having the same projection surface and being connected in three-dimensional space belong to the same connected region; The encoding end merges the at least two connected regions to obtain at least one first connected region; The encoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each first connected region to detect overlapping regions, and splitting the first connected regions having overlapping regions into at least two non-overlapping connected regions; The encoding end merges the connected regions obtained after the first processing to obtain at least one second connected region; The encoding end arranges the projection area of ​​the at least one second connected area in two dimensions to obtain a second reconstructed grid.

4. The method according to claim 3, characterized in that The encoding end merges the connected regions obtained after the first processing to obtain at least one second connected region, including: The encoder end obtains a candidate region set for a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region is any connected region among the connected regions obtained after the first processing; The encoder obtains at least one target candidate region from the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection area of ​​the third connected region does not overlap with a projection area of ​​the target candidate region; The encoder merges the third connected region into one of the at least one target candidate region.

5. The method according to claim 4, characterized in that The encoder merges the third connected region into one of the at least one target candidate region, including: The encoder merges the third connected region into the first target candidate region; The first target candidate area is the target candidate area having the smallest angle between the projection surface and the normal vector of the third connected area among the at least one target candidate area, or the first target candidate area is the target candidate area having the largest number of surfaces among the at least one target candidate area.

6. The method according to claim 4 or 5, characterized in that There are at least two overlapping vertices between the candidate region and the third connected region.

7. The method according to any one of claims 1 to 5, characterized in that The first grid is determined according to a simplified grid, where the simplified grid is a grid obtained by performing grid simplification processing on the three-dimensional grid to be encoded; The number of vertices of the simplified mesh is smaller than the number of vertices of the three-dimensional mesh to be encoded, or the number of faces of the simplified mesh is smaller than the number of faces of the three-dimensional mesh to be encoded.

8. The method according to claim 7, characterized in that The method further comprises: The encoding end subdivides and deforms the simplified grid to obtain a second grid; The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, including: The encoder calculates the displacement of each vertex of the second mesh according to the second reconstructed mesh and the second mesh; The encoding end encodes the displacement of each vertex of the second mesh to obtain a displacement code stream; The encoding end performs three-dimensional grid reconstruction according to the first reconstructed grid and the displacement code stream to obtain a third reconstructed grid; The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the third reconstructed grid to obtain a second texture map.

9. The method according to any one of claims 1 to 5, characterized in that The method further comprises: The encoding end performs a first operation on first encoding information, wherein the first encoding information includes the grid code stream and the texture map code stream, and the first operation includes at least one of the following: storage and transmission.

10. The method according to claim 9, characterized in that The first coding information further includes at least one of the following: Displacement code stream; The first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

11. A three-dimensional grid decoding method, characterized in that: include: The decoding end obtains the grid code stream and texture map code stream; The decoding end decodes the grid code stream to obtain a third grid, and decodes the texture map code stream to obtain a third texture map; The decoding end performs grid parameterization processing on the third grid to obtain a fourth grid, wherein the fourth grid is a grid including texture coordinates.

12. The method according to claim 11, characterized in that The decoding end performs grid parameterization processing on the third grid to obtain a fourth grid, including: The decoding end performs grid parameterization processing on the third grid using an orthogonal projection algorithm to obtain a fourth grid.

13. The method according to claim 12, characterized in that The decoding end performs grid parameterization processing on the third grid using an orthogonal projection algorithm to obtain a fourth grid, including: The decoding end divides the third grid into at least two connected regions, wherein each of the connected regions includes at least one face of the third grid, and faces having the same projection surface and being connected in three-dimensional space belong to the same connected region; The decoding end merges the at least two connected regions to obtain at least one first connected region; The decoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each first connected region to detect overlapping regions, and splitting the first connected regions having overlapping regions into at least two non-overlapping connected regions; The decoding end merges the connected regions obtained after the first processing to obtain at least one second connected region; The decoding end arranges the projection area of ​​the at least one second connected area in two dimensions to obtain a fourth grid.

14. The method according to claim 13, characterized in that The decoding end merges the connected regions obtained after the first processing to obtain at least one second connected region, including: The decoding end obtains a candidate region set for a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region is any connected region among the connected regions obtained after the first processing; The decoding end obtains at least one target candidate region of the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection area of ​​the third connected region does not overlap with a projection area of ​​the target candidate region; The decoding end merges the third connected region into one of the at least one target candidate region.

15. The method according to claim 14, characterized in that The decoding end merges the third connected region into one of the at least one target candidate region, including: The decoding end merges the third connected region into the first target candidate region; The first target candidate area is the target candidate area having the smallest angle between the projection surface and the normal vector of the third connected area among the at least one target candidate area, or the first target candidate area is the target candidate area having the largest number of surfaces among the at least one target candidate area.

16. The method according to claim 14 or 15, characterized in that There are at least two overlapping vertices between the candidate region and the third connected region.

17. The method according to any one of claims 11 to 15, characterized in that The method further comprises: The decoding end obtains the displacement code stream; The decoding end subdivides and deforms the fourth grid to obtain a fifth grid; The decoding end reconstructs a three-dimensional grid according to the fifth grid and the displacement obtained by decoding the displacement code stream to obtain a fourth reconstructed grid.

18. The method according to any one of claims 11 to 15, characterized in that The method further comprises: The decoding end obtains first auxiliary information, wherein the first auxiliary information includes at least one of the following: information related to grid coding, information related to texture map coding, information related to grid subdivision and deformation, information related to grid displacement, and information related to grid parameterization processing.

19. A three-dimensional grid encoding device, characterized in that: include: A first encoding module, configured to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded; A first reconstruction module, configured to perform three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid; A first processing module is configured to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates; a conversion module, configured to convert a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; The second encoding module is used to encode the second texture map to obtain a texture map code stream.

20. A three-dimensional grid decoding device, characterized in that: include: A first acquisition module is used to acquire a grid code stream and a texture map code stream; a decoding module, configured to decode the mesh code stream to obtain a third mesh, and decode the texture map code stream to obtain a third texture map; The second processing module is configured to perform mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

21. An electronic device, characterized in that: It includes a processor and a memory, the memory storing a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, it implements the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 10, or implements the steps of the three-dimensional mesh decoding method as described in any one of claims 11 to 18.

22. A chip, characterized in that: The chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 10, or to implement the steps of the three-dimensional mesh decoding method as described in any one of claims 11 to 18.

23. A readable storage medium, characterized in that The readable storage medium stores a program or instruction, which, when executed by a processor, implements the steps of the three-dimensional mesh encoding method as described in any one of claims 1 to 10, or implements the steps of the three-dimensional mesh decoding method as described in any one of claims 11 to 18.

Images