Three-dimensional grid coding method and device, three-dimensional grid decoding method and device, coding end and decoding end

Abstract

The invention discloses a three-dimensional grid code stream, a decoding method and device and electronic equipment, and belongs to the technical field of communication, and the three-dimensional grid coding method comprises the steps that a coding end carries out compression coding on a first grid to obtain a grid code stream, and the first grid is a grid determined according to a to-be-coded three-dimensional grid; the coding end performs three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid; the coding end performs grid parameterization processing on the first reconstruction grid to obtain a second reconstruction grid, and the second reconstruction grid is the first reconstruction grid comprising texture coordinates; the coding end converts a first texture map according to the to-be-coded three-dimensional grid and the second reconstruction grid to obtain a second texture map, and the first texture map is a texture map corresponding to the to-be-coded three-dimensional grid; and the encoding end encodes the second texture map to obtain a texture map code stream.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and specifically relates to a three-dimensional grid encoding method, decoding method, device, encoding end and decoding end. Background Art

[0002] Three-dimensional models have become a new generation of digital media after audio, images, and videos. Compared with traditional multimedia such as images and videos, they have stronger interactivity and realism. Three-dimensional mesh is a commonly used three-dimensional model representation method. The three-dimensional mesh is composed of three basic elements: vertices, edges, and faces. Among them, the vertex is the most basic element in the three-dimensional mesh, which defines a position in a three-dimensional space; the edge is a line segment connecting two vertices in the three-dimensional mesh; the face is a polygon formed by the closed path of the edge of the three-dimensional mesh. In addition, the three-dimensional mesh is mapped from the three-dimensional space to the two-dimensional plane area through mesh parameterization. This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with the vertices of the three-dimensional mesh. In the prior art, when the three-dimensional mesh is encoded and transmitted or stored, the texture coordinates of the three-dimensional mesh need to be encoded and stored or transmitted accordingly to achieve the reconstruction of the texture coordinates of the three-dimensional mesh at the decoding end. Since the code stream of the texture coordinates often occupies most of the code stream of the three-dimensional mesh, this will not only lead to low encoding efficiency, but also occupy storage resources or transmission resources. Summary of the invention

[0003] The embodiments of the present application provide a three-dimensional grid encoding method, decoding method, device, encoding end and decoding end, which can improve the encoding efficiency of the three-dimensional grid and reduce the occupation of storage resources or transmission resources by the code stream of the three-dimensional grid.

[0004] In a first aspect, a three-dimensional grid encoding method is provided, the method comprising:

[0005] The encoding end compresses and encodes the first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded;

[0006] The encoding end performs three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid;

[0007] The encoder performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;

[0008] The encoding end converts the first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is the texture map corresponding to the three-dimensional mesh to be encoded;

[0009] The encoding end encodes the second texture map to obtain a texture map bit stream.

[0010] In a second aspect, a three-dimensional grid encoding device is provided, the device comprising:

[0011] A first encoding module, configured to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded;

[0012] A first reconstruction module, used for performing three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid;

[0013] A first processing module, configured to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;

[0014] A conversion module, configured to convert a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded;

[0015] The second encoding module is used to encode the second texture map to obtain a texture map code stream.

[0016] In a third aspect, a three-dimensional grid decoding method is provided, the method comprising:

[0017] The decoding end obtains the grid code stream and the texture map code stream;

[0018] The decoding end decodes the grid code stream to obtain a third grid, and decodes the texture map code stream to obtain a third texture map;

[0019] The decoding end performs mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

[0020] In a fourth aspect, a three-dimensional grid decoding device is provided, the device comprising:

[0021] A first acquisition module is used to acquire a grid code stream and a texture map code stream;

[0022] A decoding module, used for decoding the grid code stream to obtain a third grid, and decoding the texture map code stream to obtain a third texture map;

[0023] The second processing module is used to perform mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

[0024] In a fifth aspect, an encoding end is provided, which includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0025] In a sixth aspect, a coding end is provided, comprising a processor and a communication interface, wherein the processor is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded; reconstruct a three-dimensional grid based on the grid code stream to obtain a first reconstructed grid; perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; convert a first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional grid to be encoded; encode the second texture map to obtain a texture map code stream.

[0026] In a seventh aspect, a decoding end is provided, which includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the third aspect are implemented.

[0027] In an eighth aspect, a decoding end is provided, comprising a processor and a communication interface, wherein the processor is used to obtain a grid code stream and a texture map code stream; decode the grid code stream to obtain a third grid, and decode the texture map code stream to obtain a third texture map; perform grid parameterization processing on the third grid to obtain a fourth grid, wherein the fourth grid is a grid including texture coordinates.

[0028] In a ninth aspect, a three-dimensional mesh encoding and decoding system is provided, comprising: an encoding end and a decoding end, wherein the encoding end can be used to execute the steps of the three-dimensional mesh encoding method as described in the first aspect, and the decoding end can be used to execute the steps of the three-dimensional mesh decoding method as described in the third aspect.

[0029] In the tenth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the third aspect are implemented.

[0030] In the eleventh aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the third aspect.

[0031] In the twelfth aspect, a computer program / program product is provided, wherein the computer program / program product is stored in a storage medium, and the computer program / program product is executed by at least one processor to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the third aspect.

[0032] In an embodiment of the present application, the encoding end compresses and encodes the first mesh to obtain a mesh code stream, wherein the first mesh is a mesh determined according to the three-dimensional mesh to be encoded; the encoding end reconstructs the three-dimensional mesh based on the mesh code stream to obtain a first reconstructed mesh; the encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is a first reconstructed mesh including texture coordinates; the encoding end converts a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; the encoding end encodes the second texture map to obtain a texture map code stream, that is, in the embodiment of the present application, the encoding end only generates a mesh code stream and a texture map code stream of the three-dimensional mesh, but does not generate a code stream of the texture coordinates of the three-dimensional mesh, which can not only improve the encoding efficiency of the three-dimensional mesh, but also reduce the occupation of the three-dimensional mesh code stream on storage resources or transmission resources. In addition, in the embodiment of the present application, the mesh parameterization processing of the first reconstructed mesh is performed, which can not only reduce the amount of data processing, but also ensure the accuracy of the texture coordinates of the first reconstructed mesh, compared with the mesh parameterization processing of the first mesh in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a block diagram of a wireless communication system to which the embodiments of the present application can be applied;

[0034] Figure 2 It is one of the schematic diagrams of a mesh parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application;

[0035] Figure 3 is a flow chart of a three-dimensional grid encoding method provided in an embodiment of the present application;

[0036] Figure 4 is a schematic diagram of basic grid compression provided by an embodiment of the present application;

[0037] Figure 5 This is a second schematic diagram of a mesh parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application;

[0038] Figure 6a This is one of the schematic diagrams of the connected regions with overlapping regions before splitting provided in the embodiment of the present application;

[0039] Figure 6b This is one of the schematic diagrams of the connected areas with overlapping areas after splitting provided in the embodiments of the present application;

[0040] Figure 6c This is the second schematic diagram of the connected areas with overlapping areas after splitting provided in the embodiment of the present application;

[0041] Figure 7a This is a third schematic diagram of a mesh parameterization process based on an orthogonal projection algorithm provided in an embodiment of the present application;

[0042] Figure 7b is a schematic diagram of extracting connected regions provided in an embodiment of the present application;

[0043] Figure 8 is a schematic diagram of a simplified grid provided in an embodiment of the present application;

[0044] Fig. 9 is a schematic diagram of a mesh subdivision deformation provided in an embodiment of the present application;

[0045] Fig.10 is a schematic diagram of grid subdivision provided in an embodiment of the present application;

[0046] Fig.11 is a schematic diagram of a three-dimensional grid coding framework provided in an embodiment of the present application;

[0047] Fig.12 is a flow chart of a three-dimensional grid decoding method provided in an embodiment of the present application;

[0048] Fig.13 is a schematic diagram of basic grid decoding provided by an embodiment of the present application;

[0049] Fig.14 is a schematic diagram of a three-dimensional grid decoding framework provided in an embodiment of the present application;

[0050] Fig.15 is a structural diagram of a three-dimensional grid encoding device provided in an embodiment of the present application;

[0051] Fig.16 is a structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application;

[0052] Fig.17 is a structural diagram of a communication device provided in an embodiment of the present application;

[0053] Fig.18 is a structural diagram of the encoding end provided in an embodiment of the present application;

[0054] Fig.19 It is a structural diagram of the decoding end provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0056] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0057] The term "indication" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication; an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.

[0058] It is worth noting that the technology described in the embodiments of the present application is not limited to the Long Term Evolution (LTE) / LTE-Advanced (LTE-A) system, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the described technology can be used for the systems and radio technologies mentioned above as well as for other systems and radio technologies. The following description describes a New Radio (NR) system for illustrative purposes, and NR terminology is used in most of the following descriptions, but these technologies can also be applied to systems other than NR systems, such as the 6th generation (6 th Generation, 6G) communication system.

[0059] Figure 1A block diagram of a wireless communication system applicable to an embodiment of the present application is shown. The wireless communication system includes a terminal 11 and a network side device 12. Among them, the terminal 11 can be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer), a notebook computer, a personal digital assistant (Personal Digital Assistant, PDA), a handheld computer, a netbook, an ultra-mobile personal computer (Ultra-mobile Personal Computer, UMPC), a mobile Internet device (Mobile Internet Device, MID), an augmented reality (Augmented Reality, AR), a virtual reality (Virtual Reality, VR) device, a robot, a wearable device (Wearable Device), an aircraft (flight vehicle), a vehicle-mounted device (Vehicle User Equipment, VUE), a ship-mounted device, a pedestrian terminal (Pedestrian User Equipment, PUE), a smart home (home appliances with wireless communication functions, such as refrigerators, televisions, washing machines or furniture, etc.), a game console, a personal computer (Personal Computer, PC), a teller machine or a self-service machine and other terminal side devices. Wearable devices include: smart watches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle-mounted device can also be called a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiment of the present application. The network side device 12 may include an access network device or a core network device, wherein the access network device may also be referred to as a radio access network (Radio Access Network, RAN) device, a radio access network function or a radio access network unit. The access network device may include a base station, a wireless local area network (Wireless Local Area Network, WLAN) access point (Access Point, AP) or a wireless fidelity (Wireless Fidelity, WiFi) node, etc.Among them, the base station may be referred to as a Node B (NB), an evolved Node B (eNB), a next generation Node B (gNB), a New Radio Node B (NR Node B), an access point, a Relay Base Station (RBS), a Serving Base Station (SBS), a Base Transceiver Station (BTS), a radio base station, a radio transceiver, a Basic Service Set (BSS), an Extended Service Set (ESS), a Home Node B (HNB), a Home Evolved Node B, a Transmission Reception Point (TRP) or other appropriate terms in the field. As long as the same technical effect is achieved, the base station is not limited to specific technical terms. It should be noted that in the embodiments of the present application, only the base station in the NR system is used as an example for introduction, and the specific type of the base station is not limited.

[0060] The core network equipment may include but is not limited to at least one of the following: core network node, core network function, mobility management entity (Mobility Management Entity, MME), access mobility management function (Access and Mobility Management Function, AMF), session management function (Session Management Function, SMF), user plane function (User Plane Function, UPF), policy control function (Policy Control Function, PCF), policy and charging rules function unit (Policy and Charging Rules Function, PCRF), edge application service discovery function (Edge Application Server Discovery Function, EASDF), unified data management (Unified Data Management, UDM), unified data storage (Unified Data Repository, UDR), home user server (Home Subscriber Server, HSS), centralized network configuration (CNC), network storage function (Network Repository Function, NRF), network exposure function (Network Exposure Function, NEF), local NEF (Local NEF, or L-NEF), binding support function (Binding Support Function, BSF), application function (Application Function, AF), etc. It should be noted that in the embodiments of the present application, only the core network device in the NR system is introduced as an example, and the specific type of the core network device is not limited.

[0061] It should be noted that the encoding end of the embodiment of the present application may be an electronic device or a component of an electronic device such as a terminal or a network side device or a server. The decoding end of the embodiment of the present application may also be an electronic device or a component of an electronic device such as a terminal or a network side device or a server. In addition, the encoding end and the decoding end of the embodiment of the present application may be located in the same electronic device or in different electronic devices.

[0062] For ease of understanding, some contents involved in the embodiments of the present application are described below:

[0063] 1. 3D Grid

[0064] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have become a new generation of digital media after audio, images, and videos. Three-dimensional grids are a commonly used way to represent three-dimensional models. Compared with traditional multimedia such as images and videos, three-dimensional grids have stronger interactivity and realism, making them more and more widely used in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and military.

[0065] Although there are many ways to represent 3D meshes, triangular meshes are still the most common representation method. A 3D mesh can be considered to be composed of three basic elements: vertices, edges, and faces. Vertices are the most basic elements in a mesh, and they define a position in a 3D space. Edges are line segments connecting two vertices in a mesh. Faces can be considered as polygons formed by closed paths of edges. For a triangular mesh, each face is a triangle.

[0066] The information contained in a three-dimensional mesh is usually divided into three categories: geometric information, connection information, and attribute information. Among them, geometric information refers to the position of each vertex of the three-dimensional mesh in three-dimensional space. Connection information describes the association between elements in the three-dimensional mesh, that is, the connection relationship between vertices. Attribute information is an optional information that can associate attributes to corresponding mesh elements (such as vertex color, normal vector, etc. can be associated with mesh vertices). In addition, mesh parameterization can be used to map the three-dimensional mesh from three-dimensional space to a two-dimensional plane area. This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with mesh vertices. This two-dimensional mapping can be used to represent high-resolution attribute information, such as texture, normal vector, etc.

[0067] In almost all application fields that use 3D meshes (such as computational simulation, entertainment, medical imaging, digital cultural relics, computer design, e-commerce, etc.), as people's demand for 3D models in visual effects is getting higher and higher, 3D models are becoming more and more complex, and the accuracy of 3D models is getting higher and higher. Therefore, the amount of data required to represent 3D meshes is also increasing accordingly. The above problems have led to the increasing complexity of 3D mesh processing, visualization, transmission and storage. 3D mesh compression can be regarded as a way to solve the above problems. It can reduce the size of model data and facilitate the processing, storage and transmission of 3D meshes.

[0068] Recently, the Moving Pictures Experts Group (MPEG), an international standardization organization in the field of audio and video coding compression, has begun to develop a compression standard for three-dimensional meshes, Video-based Dynamic Mesh Coding (VDMC), which is specified based on the existing Visual Volumetric Video-based Coding (V3C) standard. The V3C standard provides a general method for compressing three-dimensional models, which can be presented in the form of point clouds, meshes or panoramic videos. Making the compression method of three-dimensional meshes compatible with this standard is conducive to the promotion and applicability of this method. Therefore, it is of great significance to optimize the three-dimensional mesh encoding and decoding method in VDMC and combine the optimization method with the V3C standard. An optional optimization method is to optimize the displacement coding method. In the existing framework, the displacement is obtained by calculating the distance between the reconstructed mesh and the original mesh vertex, and its purpose is to improve the quality of the mesh. In addition, in the existing framework, the displacement is encoded using a video encoder, and providing multiple options for the encoding method of the displacement helps to improve the encoding performance.

[0069] 2. orthoAtlas

[0070] OrthoAtlas is a projection-based mesh parameterization method that generates texture coordinates for 3D meshes through orthogonal projection. The main process is as follows: Figure 2 As shown, the following steps are included:

[0071] a) Calculate mesh properties, including the neighboring faces of each face and the area and normal vector of each face;

[0072] b) Determine the projection plane of each face based on the normal vector;

[0073] c) Start clustering all faces according to the projection plane to form a connected area, and first select the starting face of the cluster;

[0074] d) Iterating from the starting face, determining whether the adjacent faces of the face added to the connected region can be added to the connected region;

[0075] e) After each connected region is iterated, multiple connected regions are obtained;

[0076] f) determining whether to merge adjacent connected regions based on the error metric;

[0077] g) Detect whether there are overlapping areas during projection, remove the overlapping surfaces and regenerate the connected areas;

[0078] h) Arrange all the projected regions into a two-dimensional image.

[0079] The following describes the above main processes:

[0080] (1) Calculate grid properties

[0081] Calculate some properties of the mesh, including the adjacency list of the triangles and the normal vector and area of ​​the triangles; triangle adjacency can be defined as sharing a single vertex or sharing an edge; the normal vector and area of ​​a triangle can be derived from the projected triangle (later used to calculate the distortion after projection).

[0082] (2) Clustering

[0083] The triangles are clustered, and the area formed by the same type of triangles is called a connected area. There are two methods to select the starting point of the clustering algorithm:

[0084] The first method is to obtain the projection direction based on the statistical results of all face normal vectors. Then the triangular face whose normal vector is closest to the projection direction can be selected as the starting point of the clustering algorithm and it has not yet been added to the connected area.

[0085] The second method is to select the first available face as the starting point of the clustering algorithm (that is, the face has not yet been added to the connected area), and determine whether to add the adjacent face to the connected area according to the criteria; the criteria include:

[0086] has the same projection direction as the connected region;

[0087] The angle between the normal vector of the last face added to the connected region and the normal vector of the last face added to the connected region is less than the threshold;

[0088] The area is less than a threshold of the total area;

[0089] The number of triangles is less than the threshold.

[0090] (3) Merger

[0091] After clustering is completed, there may be many small area partitions that need to be merged. When merging, a cost function is needed to determine whether two partitions can be merged. The function is defined by the perimeter and stretching degree of the connected area. The detailed steps are as follows:

[0092] First, the connected regions are sorted in a specific order. The sorting criteria can be based on the number of triangles contained and the proximity between the average normal vector of the connected region and the corresponding projection direction. The cost function of each connected region is COST (M i )=L 2 (M i )+λPERIMETER(M i );

[0093] Then, take the first connected region after sorting each time, obtain the connected regions adjacent to the connected region to form a candidate list, and then for each candidate region, first calculate the cost function after the two regions are merged: COST(M i +M j )=L 2 (M i +M j )+λPERIMETER(M i +M j ), for all (COST(M i +M j ) <COST(M i )+COST(M j )) of the region, select the region that minimizes the cost function, merge the extracted region into this region and update the ranking of the connected regions; where L 2 () represents the stretching degree of the connected area after projection, PERIMETER() represents the perimeter of the above connected area, and λ represents the weight coefficient.

[0094] The stretching function in the cost function measures the degree of distortion after projection, which is obtained by calculating the area of ​​the triangle after projection in each direction.

[0095] (4) Detect overlapping areas

[0096] Finally, it is necessary to check whether there are overlapping vertices or faces in each connected area, and split the overlapping area from the connected area to form a new connected area. Detecting overlapping areas means determining whether the currently projected surface overlaps with the projected area. If an overlap occurs, the current surface is removed from the connected area, and the surface and other removed surfaces form a new connected area. Finally, in the removed connected area, check whether there is a disconnected area, and split the disconnected area into multiple connected areas.

[0097] The following is a detailed description of the three-dimensional grid coding method provided in the embodiment of the present application through some embodiments and their application scenarios in combination with the accompanying drawings.

[0098] See also Figure 3 , Figure 3 is a flowchart of a three-dimensional grid encoding method provided in an embodiment of the present application. The method can be executed by an encoding end, such as Figure 3 As shown, the following steps are included:

[0099] Step 301: The encoding end compresses and encodes a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded.

[0100] In this embodiment, the first grid may be a three-dimensional grid to be encoded, or may be a grid obtained after processing the three-dimensional grid to be encoded, for example, a grid obtained after grid simplification processing of the three-dimensional grid to be encoded. The grid code stream is also a code stream obtained after compression encoding of the first grid. It should be noted that the three-dimensional grid to be encoded may also be referred to as an input grid of the encoding end, the first grid may also be referred to as a basic grid, and the grid code stream may also be referred to as a basic grid code stream.

[0101] The compression encoding of the first grid may be performed by, for example, adopting compression modes such as intra-frame mode, inter-frame mode or skip mode to compress the first grid. Figure 4 The above three compression modes are described with examples.

[0102] like Figure 4 As shown, in the intra-frame mode, the first mesh is encoded by a static mesh encoder. In this compression mode, the first mesh must contain geometric coordinates, connection relationships, and attribute information associated with vertices. In the inter-frame mode, the first mesh first determines whether to use the skip mode through rate-distortion optimization (RDO). If the skip mode is used, the reference mesh in the mesh buffer is directly selected as the first mesh of the current frame. If the skip mode is not used, the motion vector of the vertex is calculated based on the reference frame in the mesh buffer and the current frame, and the corresponding motion vector is encoded by the motion vector encoder. It should be noted that after encoding the first mesh, the encoded mesh needs to be reconstructed, that is, the first mesh needs to be reconstructed so as to be provided to the subsequent modules for processing.

[0103] Step 302: The encoding end reconstructs a three-dimensional grid based on the grid code stream to obtain a first reconstructed grid.

[0104] In this embodiment, after the first mesh is compressed and encoded to obtain a mesh code stream, a three-dimensional mesh reconstruction may be performed based on the mesh code stream. Exemplarily, the three-dimensional mesh reconstruction may be achieved by decoding the mesh code stream.

[0105] Specifically, corresponding decoding can be performed based on the compression mode of the above-mentioned grid code stream. For example, when the compression mode of the above-mentioned grid code stream is the intra-frame mode, the above-mentioned grid code stream can be decoded based on the static grid decoder to obtain a reconstructed basic grid, that is, a first reconstructed grid; when the compression mode of the above-mentioned basic grid is the inter-frame mode, the motion vector of the above-mentioned grid code stream can be decoded, and the corresponding basic grid can be reconstructed according to the reference frame in the grid buffer; when the compression mode of the above-mentioned basic grid is the skip mode, the corresponding reference frame in the grid buffer can be used as the basic grid of the current frame.

[0106] Step 303: The encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates.

[0107] In this embodiment, the mesh parameterization process is used to generate corresponding texture coordinates for the first reconstructed mesh. Exemplarily, the mesh parameterization process can be performed on the first reconstructed mesh based on an Isochart algorithm, an orthogonal projection algorithm, or the like.

[0108] Step 304: The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map.

[0109] In this embodiment, the first texture map may also be referred to as an input texture map, which may be a texture map corresponding to the three-dimensional mesh to be encoded. Exemplarily, an optional texture map conversion method may include the following steps:

[0110] Step a1: Calculate the texture coordinates of each pixel on the texture map to be generated (ie, the second texture map).

[0111] Step a2: for each texture coordinate, determine whether the texture coordinate is within a certain face of the second reconstructed mesh.

[0112] Step a3: If the texture coordinate does not belong to any surface of the second reconstructed mesh, the pixel is marked as an empty pixel, which can be filled with a filling algorithm.

[0113] Step a4: If the texture coordinate belongs to a face of the second reconstructed mesh, perform the following steps:

[0114] Step a41, marking the pixel as filled;

[0115] Step a42, calculating the center of gravity coordinates of the texture in the current surface according to the texture coordinates;

[0116] Step a43, mapping the two-dimensional texture coordinates to three-dimensional geometric coordinates according to the barycentric coordinates and the corresponding surface, that is, mapping to the points on the second reconstructed grid corresponding to the texture coordinates;

[0117] Step a44, searching for the point closest to the three-dimensional coordinate on the input original grid (i.e. the three-dimensional grid to be encoded);

[0118] Step a45, calculating the center of gravity coordinates of the three-dimensional coordinates according to the surface where the three-dimensional coordinates are located and mapping them to two dimensions to calculate the texture coordinates;

[0119] Step a46, sampling the input original texture map (i.e., the first texture map) by using the texture coordinates to obtain the value of the corresponding pixel position;

[0120] Step a47, assigning the value to the corresponding pixel on the texture map to be generated (ie, the second texture map mentioned above).

[0121] It should be noted that, for empty pixels, existing filling algorithms (such as the Push-Pull algorithm) may be used to fill these empty pixels.

[0122] Step 305: The encoding end encodes the second texture map to obtain a texture map bit stream.

[0123] Exemplarily, a video encoder may be used to encode the second texture map to obtain a texture map bit stream.

[0124] It is understandable that, when the encoder obtains the grid code stream and the texture map code stream, the encoder can store or transmit the grid code stream and the texture map code stream. It should also be noted that, when the encoder does not generate a code stream of the texture coordinates of the grid, the decoder can perform a grid parameterization process on the grid obtained by decoding the grid code stream to generate the texture coordinates of the grid, thereby ensuring that the decoder can normally apply the decoded grid.

[0125] In the three-dimensional mesh encoding method provided in the embodiment of the present application, the encoding end compresses and encodes the first mesh to obtain a mesh code stream, wherein the first mesh is a mesh determined according to the three-dimensional mesh to be encoded; the encoding end reconstructs the three-dimensional mesh based on the mesh code stream to obtain a first reconstructed mesh; the encoding end performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is a first reconstructed mesh including texture coordinates; the encoding end converts a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; the encoding end encodes the second texture map to obtain a texture map code stream, that is, in the embodiment of the present application, the encoding end only generates a mesh code stream and a texture map code stream of the three-dimensional mesh, but does not generate a code stream of the texture coordinates of the three-dimensional mesh, which not only improves the encoding efficiency of the three-dimensional mesh, but also reduces the occupation of the three-dimensional mesh code stream on storage resources or transmission resources. In addition, in the embodiment of the present application, the mesh parameterization processing of the first reconstructed mesh is performed, which can not only reduce the amount of data processing, but also ensure the accuracy of the texture coordinates of the first reconstructed mesh, compared with the mesh parameterization processing of the first mesh in the prior art.

[0126] Optionally, the encoding end performs grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, including:

[0127] The encoding end uses an orthogonal projection algorithm to perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid.

[0128] It is understandable that the way in which the encoder and the decoder perform grid parameterization processing in the embodiment of the present application should be consistent, which is conducive to more accurate reconstruction of the grid required for the application. Therefore, in this embodiment, the orthogonal projection algorithm is used to perform grid parameterization processing on the first reconstructed grid, which has low implementation complexity and is easy to apply to the decoding end.

[0129] Optionally, the encoding end uses an orthogonal projection algorithm to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, including:

[0130] The encoding end divides the first reconstructed grid into at least two connected regions, wherein each of the connected regions includes at least one face of the first reconstructed grid, and faces having the same projection face and being connected in three-dimensional space belong to the same connected region;

[0131] The encoding end merges the at least two connected regions to obtain at least one first connected region;

[0132] The encoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions;

[0133] The encoding end merges the connected regions obtained after the first processing to obtain at least one second connected region;

[0134] The encoding end arranges the projection area of ​​the at least one second connected area in two dimensions to obtain a second reconstructed grid.

[0135] The following combination Figure 5 This embodiment is described by way of example.

[0136] like Figure 5 As shown, the encoder uses an orthogonal projection algorithm to perform mesh parameterization processing on the first reconstructed mesh, which includes the following processes in sequence: initial partitioning, merging partitions, detecting overlapping areas, simple area merging, and two-dimensional arrangement. Among them, the above-mentioned initial partitioning, that is, the above-mentioned encoder divides the first reconstructed mesh into at least two connected areas. The above-mentioned merging partitions, that is, the encoder merges the at least two connected areas to obtain at least one first connected area. The above-mentioned detection of overlapping areas, that is, the encoder performs a first process on the at least one first connected area respectively. The above-mentioned simple area merging, that is, the encoder merges the connected areas obtained after the first process, to obtain at least one second connected area. The above-mentioned two-dimensional arrangement, that is, the encoder arranges the projection areas of the at least one second connected area in two dimensions to obtain a second reconstructed mesh. The following takes the face of the first reconstructed mesh as a triangular face as an example to illustrate the above-mentioned various processes.

[0137] Exemplarily, the above initial partitioning may include the following process:

[0138] Define any triangular face T of the first reconstructed mesh, whose three vertices are A, B, C, (x i ,y i ,z i ) is the geometric coordinate of the i-th vertex, then the normal vector of the current face T AB×AC=((y B -y A )*(z C -z A )-(y C -y A )*(z B -z A ),(z B -z A )*(x C-x A )-(z C -z A )*(x B -x A ),(x B -x A )*(y C -y A )-(x C -x A )*(y B -y A )), where AB and AC represent the vectors corresponding to the two sides of the triangle T, and × represents the vector cross product. After the normal vector is calculated by the above formula, the normal vector is normalized to obtain the unit normal vector of the current triangle. At the same time, the area S of the current triangle can be obtained by calculating the normal vector The modulus length is obtained, that is,

[0139] After obtaining the normal vector and area of ​​each triangular face of the first reconstructed mesh, the projection surface of each triangular face is determined according to the pre-set projection surface. This process is to calculate the dot product of the normal vector of the current triangular face and the normal vector of each pre-set projection surface, and take the projection surface corresponding to the largest product point as the projection surface of the current triangular face. Then, the triangular faces in the first reconstructed mesh that belong to the same projection surface and are connected in three-dimensional space are divided into the same connected area. Through this process, the first reconstructed mesh can be split into multiple connected areas. Among them, the above-mentioned pre-set projection surface can be reasonably set according to actual needs. For example, the above-mentioned pre-set projection surface can be the 6 faces of a cube.

[0140] After the initial partitioning, you can try to merge adjacent connected areas. When merging, you need to use a cost function to determine whether two connected areas can be merged. The cost function can be defined by the perimeter and stretching degree of the connected area. Exemplarily, the above merging partitions may include the following process:

[0141] First, the connected areas obtained by the first reconstructed mesh division are sorted. Specifically, the connected areas can be sorted according to at least one of the number of triangular faces contained in each connected area and the degree of proximity between the average normal vector of each connected area and the corresponding projection direction (i.e., the size of the dot product of the two vectors). For example, first sort the connected areas in ascending order according to the number of triangular faces contained in each connected area; for multiple connected areas with the same number of triangular faces, they can be sorted according to the degree of proximity between the average normal vector of each connected area and the corresponding projection direction, and the connected areas with higher proximity are ranked higher. The average normal vector of the above-mentioned connected area can refer to the average value of the normal vectors of all triangular faces contained in the above-mentioned connected area.

[0142] Then, each time the first connected region in the connected region sorting is taken, all connected regions adjacent to the connected region are obtained to form a candidate list. For each candidate region in the candidate list, the cost function after the connected region is merged with the candidate region is calculated: COST(M i +M j )=L 2 (M i +M j )+λ*PERIMETER(M i +M j ), the cost function after merging is less than the sum of the cost functions of the two regions before merging (i.e. COST(M i +M j ) <COST(M i )+COST(M j )), select the candidate region that minimizes the cost function, merge the current connected region with the candidate region, update the projection surface of the merged connected region, and update the sorting of the connected regions, and iterate until all connected regions are traversed.

[0143] Among them, the adjacency of two connected regions can be understood as the two connected regions sharing at least one edge. i and M j One represents the first connected region in the connected region sorting, and the other represents the candidate region of the first connected region. i +M j ) indicates that the above M i and M j The cost function of the merged connected regions. L 2 (M i +M j ) indicates that the above M i and M j The degree of stretching of the merged connected region after projection. i +M j ) indicates that the above M i and M j The perimeter of the connected region after merging. λ represents the weight coefficient. COST(M i ) indicates that the above M i Cost function of COST(M j ) The above M j The cost function of .

[0144] Exemplarily, the stretching degree of the above connected region after projection can be determined in the following manner:

[0145] Assume a triangular face T with two-dimensional texture coordinates p1, p2, p3, where pi = (si, ti), i takes the values ​​of 1, 2 and 3 respectively, and the corresponding three-dimensional coordinates are expressed as q1, q2, q3. The calculation process of the affine mapping S(p) = S(s, t) = q is shown in formula (1):

[0146]

[0147] in,<p,p2,p3> represents the area of ​​the triangle face pp2p3,<p,p3,p1> represents the area of ​​the triangle face pp3p1,<p,p1,p2> represents the area of ​​the triangle pp1p2, and p represents the coordinates of any point on the triangle p1p2p3. Since the mapping is affine, its partial derivative is a constant on (s, t), and its calculation process is shown in formula (2) and formula (3):

[0148]

[0149]

[0150] Where A=<p1,p2,p3> =((s2-s1)(t3-t1)-(s3-s1)(t2-t1)) / 2.

[0151] Then the Jacobian matrix [S S ,S t ], the calculation process is shown in formula (4) and formula (5):

[0152]

[0153]

[0154] Where a = S s ·S s ,b=S s ·S t ,c=S t ·S t . Singular values ​​γ max , γ min They represent the maximum and minimum lengths of a unit length vector when it is mapped from a two-dimensional texture domain to a three-dimensional surface, i.e., the maximum and minimum local "stretching". The stretching measure on the triangular surface T is defined as shown in formula (6):

[0155]

[0156] In the entire three-dimensional grid M={T i The definition of the upper stretching measure is shown in formula (7):

[0157]

[0158] Among them, A′(T i ) is the triangle face T i The surface area in three dimensions.

[0159] From the above, we can see that the cost function of this example is calculated by L 2 (M i ) and PERIMETER(M i ) definition, the two variables are weighed by the λ parameter. After the merging partition step, the number of partitions in the connected region will be reduced compared to the initial partition.

[0160] After the merging partition process, each connected area can be projected. During the projection process, it is necessary to prevent the projection area of ​​the current triangle from blocking the projected area. Exemplarily, the above detection of overlapping areas may include the following process:

[0161] First, traverse all the triangular faces of the current connected area, and project each triangular face according to the projection surface corresponding to the current connected area. That is, calculate the two-dimensional coordinates of the three vertices on the current triangular face after projection. Then determine whether the projection area of ​​the current triangular face overlaps with the projection area of ​​the projected triangular face of the current connected area. Exemplarily, the overlap of the projection area can be divided into three situations, namely: one or more vertices on the projection area of ​​the current triangular face are within the projection area of ​​the projected triangular face of the current connected area; the vertices of the projection area of ​​the projected triangular face of the current connected area are within the projection area of ​​the current triangular face; one or more edges of the projection area of ​​the current triangular face intersect with the edges of the projection area of ​​the projected triangular face of the current connected area.

[0162] By judging the above three situations, if the projection area of ​​the current triangle overlaps with the projection area of ​​the projected triangle in the current connected area, the projection of the current triangle is skipped first, and the next triangle in the current connected area is traversed. Then, after a round of traversal, it is determined whether there are unprojected triangles in the current connected area, and these unprojected triangles are projected as new connected areas and it is determined whether there is overlap in the projection area. This is repeated until the final projection area does not overlap. Through this process, the connected area with overlapping areas can be split into multiple non-overlapping connected areas. For example, Figures 6a to 6c As shown, a connected region with overlapping areas (i.e. Figure 6a The connected region shown in the figure) is split into two non-overlapping connected regions (i.e. Figure 6b and Figure 6c The two connected regions shown).

[0163] It can be understood that the process of projecting the triangular surface of the connected area is also the process of obtaining the initial texture coordinates of the vertices of the triangular surface.

[0164] After the above-mentioned overlapping area detection process, for some connected areas with fewer faces, it is possible to try to merge these connected areas with the connected areas adjacent to them to reduce the number of connected areas. Exemplarily, the above-mentioned simple area merging can be to merge some connected areas with fewer faces with the connected areas connected to them, for example, to merge the connected areas with faces less than a preset value with the connected areas connected to them, wherein the above-mentioned preset value can be reasonably set according to actual needs, for example, 4 or 3 or 2. It can be understood that in the process of simple area merging, it is necessary to ensure that there is no overlap between the projection areas of the faces of the merged connected areas.

[0165] After the simple partitions are merged, the result of the parameterized final partition of the first reconstructed mesh can be obtained, and these partitions are arranged in a two-dimensional image according to certain rules to obtain the final texture coordinates of all vertices, that is, the final parameterized result of the first reconstructed mesh. Exemplarily, the above two-dimensional arrangement may include the following process:

[0166] First, the scaling ratio of the entire first reconstructed mesh vertices is calculated so that they can be arranged in a two-dimensional image of a specified size. Then, the width and height occupied by each connected area after projection are calculated respectively. Next, the image occupied by the connected area is rasterized, that is, the pixel value of the area occupied by the triangle face is set to 1, and the pixel value of the rest of the area is set to 0.

[0167] Next, for each connected region, try to arrange it on the final image, that is, for the current connected region to be arranged, traverse each pixel block on the two-dimensional image as the upper left corner of the connected region, and then try different placement directions. If it is found that the connected region can be placed in the area with the current pixel as the upper left corner (that is, the occupied area of ​​the current connected region on the two-dimensional image does not occupy other placed areas), then all pixels in the area are set to occupied and the texture coordinates of all points in the current connected region are adjusted (including rotation, scaling and translation), and then the position of the upper left corner of the current connected region on the two-dimensional image and the width and height of the current connected region are recorded; if it is still not successfully placed after traversing all pixels, the current connected region is scaled, and then the scaled connected region is re-tried to be arranged on the two-dimensional image. If the current connected region still cannot be successfully placed after scaling, the entire grid is scaled and then rearranged.

[0168] After the above steps, the final parameterization result of the first reconstructed mesh, that is, the texture coordinates of the first reconstructed mesh, can be obtained, and accordingly the first reconstructed mesh including the texture coordinates is obtained.

[0169] In some optional embodiments, between the step of detecting overlapping regions and the step of merging simple regions, a step of extracting connected regions may be further included, such as Figure 7a Specifically, after detecting and splitting the overlapping areas, a connected area may be split into multiple connected areas. Among these split connected areas, there may be some surfaces or areas that are not connected to each other, such as Figure 6c As shown in , the above-mentioned connected region extraction is to extract the unconnected regions in a connected region, and project each of them as an independent connected region, such as Figure 7b Correspondingly, the above-mentioned simple region merging can be a simple region merging of the connected regions obtained after the connected regions are extracted.

[0170] In this embodiment, after the encoding end performs the first processing on the at least one first connected region respectively, it merges the connected regions obtained after the first processing again, which can further reduce the number of connected regions and further improve the encoding efficiency.

[0171] Optionally, the encoding end performs a merging process on the connected regions obtained after the first process to obtain at least one second connected region, including:

[0172] The encoder end obtains a candidate region set for a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region is any connected region among the connected regions obtained after the first processing;

[0173] The encoder obtains at least one target candidate region of the merged candidate region set, wherein when the third connected region is merged into the target candidate region, a projection region of the third connected region does not overlap with a projection region of the target candidate region;

[0174] The encoder merges the third connected region into one of the at least one target candidate region.

[0175] In this embodiment, the target candidate region is a candidate region whose projection area does not overlap with the projection area of ​​the third connected region after being merged with the third connected region. Wherein, when the target candidate region is merged with the third connected region, the projection surface of the target candidate region and the third connected region may be the projection surface corresponding to the target candidate region, or may be the projection surface corresponding to the merged connected region, etc.

[0176] The above merging of the third connected region into one of the at least one target candidate region may, for example, merge the third connected region into any target candidate region, or merge the third connected region into the target candidate region with the largest number of faces.

[0177] In some optional embodiments, the third connected region may be a connected region obtained after the first processing and having a number of faces less than a preset value, or a connected region obtained by splitting connected regions with overlapping regions.

[0178] Exemplarily, taking the above-mentioned merged candidate area set as a merged candidate area list as an example, after determining the merged candidate area list, the encoder traverses each candidate area in the merged candidate area list, tries to merge the current connected area into the candidate area, and then detects whether there will be overlap after projection. The projection surface can be the projection surface corresponding to the current merged candidate area. The overlap detection process here is the same as the above-mentioned overlapping area detection process. If the projection area after projection overlaps, the current candidate area is deleted from the merged candidate area list. In this way, the remaining candidate areas in the merged candidate area list are areas that can be merged with the current area and will not overlap after projection, that is, the above-mentioned target candidate areas.

[0179] This embodiment merges the candidate areas whose projection areas after merging do not overlap with the projection area of ​​the third connected area from the connected areas adjacent to the third connected area. This ensures that the projection areas of the connected areas after merging do not overlap, thereby ensuring the accuracy of the texture coordinates of the obtained mesh.

[0180] Optionally, the encoder merges the third connected region into one of the at least one target candidate region, including:

[0181] The encoder merges the third connected region into the first target candidate region;

[0182] Among them, the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.

[0183] In this embodiment, the target candidate area with the smallest angle between the normal vectors of the projection plane and the third connected area, that is, the target candidate area with the closest normal vectors to the projection plane and the third connected area, can ensure that the distortion after projection is minimized.

[0184] Exemplarily, each target candidate area can be traversed, and the target candidate area whose normal vector is closest to the projection surface and the third connected area can be selected as the first target candidate area to ensure that the distortion after projection is minimized; when there are multiple target candidate areas whose normal vector is closest to the projection surface and the third connected area, the target candidate area with the largest number of faces can be selected from the multiple target candidate areas as the first target candidate area, which is helpful in reducing the number of connected areas.

[0185] It can be understood that merging the third connected region into the first target candidate region means adding all faces in the third connected region to the first target candidate region, deleting the original third connected region, and further obtaining texture coordinates of newly added vertices in the merged connected region.

[0186] Optionally, there are at least two overlapping vertices between the candidate region and the third connected region.

[0187] Optionally, the first grid is determined according to a simplified grid, and the simplified grid is a grid obtained by performing grid simplification processing on the three-dimensional grid to be encoded;

[0188] The number of vertices of the simplified mesh is smaller than the number of vertices of the three-dimensional mesh to be encoded, or the number of faces of the simplified mesh is smaller than the number of faces of the three-dimensional mesh to be encoded.

[0189] In this embodiment, the first mesh is determined according to the simplified mesh. For example, the first mesh may be the simplified mesh, or the first mesh may be a mesh obtained by displacing each vertex of the simplified mesh.

[0190] The above-mentioned mesh simplification process can be understood as simplifying the three-dimensional mesh to be encoded into a mesh with relatively fewer points and faces, and maintaining the shape of the original mesh as much as possible.

[0191] For example, Figure 8As shown, the vertices at both ends of the edge of the three-dimensional mesh to be encoded are merged into one vertex and the connection between the two vertices is deleted. The entire mesh repeats this process according to a certain rule to reduce the number of faces and vertices of the mesh to the target value, wherein the above target value can be determined according to the number required for lossy encoding. Specifically, in the process of mesh simplification, a certain error metric can be selected to optimize the simplified result. For example, the sum of the equation coefficients of all adjacent faces of the vertex can be selected as the error metric of the vertex, that is, the error of the vertex is measured by the sum of the equation coefficients of all adjacent faces of the vertex. The error of the corresponding edge is the sum of the errors of the two vertices on the edge, that is, the error generated by the merging of an edge is the sum of the distances from the merged vertex to all adjacent faces of the original two vertices of the edge. After determining the error metric, iterative mesh simplification begins, for example, calculating the errors of the vertices of the three-dimensional mesh to be encoded, thereby obtaining the error of each edge of the three-dimensional mesh to be encoded. Then, each edge is arranged from small to large in error, and the edge with the smallest error is selected for merging each time. At the same time, the position of the merged vertex is calculated, and the errors of all edges related to the merged vertex are updated. That is, the order of edge arrangement is updated to ensure that each iteration is based on the global error metric. The faces of the 3D mesh to be encoded are simplified to the number required for lossy encoding through iteration.

[0192] This embodiment performs mesh simplification processing on the three-dimensional mesh to be encoded to obtain a simplified mesh, and compresses and encodes the first mesh determined by the simplified mesh. Since the number of faces or vertices of the mesh is reduced after the mesh simplification processing, this can not only improve the efficiency of mesh encoding, but also further reduce the transmission resources or storage resources required for the mesh code stream.

[0193] Optionally, the method further comprises:

[0194] The encoding end subdivides and deforms the simplified grid to obtain a second grid;

[0195] The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, including:

[0196] The encoding end calculates the displacement of each vertex of the second mesh according to the second reconstructed mesh and the second mesh;

[0197] The encoding end encodes the displacement of each vertex of the second mesh to obtain a displacement code stream;

[0198] The encoding end performs three-dimensional grid reconstruction according to the first reconstruction grid and the displacement code stream to obtain a third reconstruction grid;

[0199] The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the third reconstructed grid to obtain a second texture map.

[0200] For example, the basic idea of ​​subdivision deformation can be as follows Fig. 9 As shown in FIG. 1 , the input 2D curve (represented by a 2D polyline), called the original curve, is first downsampled to generate a basic curve / polyline, called a simplified curve. Then, a subdivision curve can be generated based on the polyline of the simplified curve. The polyline of the subdivision curve is then deformed to obtain a deformed curve that is closer to the original curve, that is, a displacement vector is calculated for each vertex of the subdivision curve, such as Fig. 9 As shown by the arrows in , the shape of the subdivision deformation curve is made as close as possible to the shape of the original curve; then, each vertex can be moved correspondingly according to the displacement vector of each vertex of the subdivision curve to obtain a subdivision deformation curve with a shape close to the original curve.

[0201] Specifically, the simplified mesh may be subdivided to obtain a subdivided mesh, wherein the specific implementation method of the subdivision may be selected arbitrarily. An optional implementation method is a midpoint subdivision method. For example, when the faces included in the simplified mesh are triangular faces, each triangular face may be subdivided into four sub-triangular faces in each subdivision iteration, such as Fig.10 As shown, a new vertex is introduced in the middle of each edge of the simplified mesh, and the subdivision of geometric information and attribute information is performed independently because the connection relationship between geometric information and attribute information is usually different.

[0202] For the above subdivision method, the midpoint v of the newly introduced edge (v1, v2) can be calculated using formula (8): 12 The position Pos(v 12 ):

[0203]

[0204] Among them, Pos(v1) and Pos(v2) are the geometric coordinates of vertices v1 and v2 respectively. 12 The texture coordinates can be calculated in the same way as the geometric coordinates, which will not be elaborated here.

[0205] For the subdivided mesh, find the nearest neighbor point of each point on the simplified mesh (including the points on the simplified mesh surface). Optionally, the search can be accelerated by using a data structure such as a k-dimensional tree (kdTree). The displacement of the geometric coordinates of each vertex of the subdivided mesh is obtained by calculating the distance between the geometric coordinates of each vertex on the subdivided mesh and its nearest neighbor point on the simplified mesh. Then, the corresponding vertex can be moved based on the displacement of the geometric coordinates of each vertex of the subdivided mesh to obtain a subdivided deformed mesh, that is, the second mesh mentioned above.

[0206] After obtaining the subdivided deformed mesh, the encoder can calculate the displacement of each vertex of the subdivided deformed mesh based on the subdivided deformed mesh and the second reconstructed mesh. For example, the second reconstructed mesh can be subdivided to determine the correspondence between each vertex of the subdivided second reconstructed mesh and each vertex of the subdivided deformed mesh, and the displacement of each vertex can be calculated according to each vertex of the subdivided second reconstructed mesh and each vertex of the corresponding subdivided deformed mesh. Then, the encoder can encode the calculated displacement. In this embodiment, the above encoding method of the calculated displacement is not limited. For example, the calculated displacement can be encoded by a video encoding method, which can specifically include the following steps: converting the coordinate system where the displacement is located, that is, converting the coordinate system of the displacement of each vertex to a coordinate system constructed by the normal vector of the vertex corresponding to the displacement and two components tangent to the normal vector, and then transforming the displacement, such as wavelet transform, quantizing the transformed coefficients, and arranging them in the image in the order of scanning, and applying video encoding to the image; or, the calculated displacement can be directly encoded by entropy coding.

[0207] The video encoding method may include the following steps:

[0208] (1) Coordinate system conversion:

[0209] The generated displacements are in the same global coordinate system as the input mesh. An optional optimization method is to transform them into a local coordinate system, where the local coordinate system of each vertex is defined by the vertex normal vector on the subdivided deformed mesh. The advantage of this approach is that the normal component of the displacement has a more significant impact on the quality of the reconstructed mesh than the two tangential components, so a larger quantization parameter can be set for the tangential component.

[0210] (2) Wavelet transform:

[0211] For the quantized displacement, wavelet transform can be applied thereto. An optional wavelet transform such as linear wavelet transform is shown below:

[0212] The prediction process is defined as shown in formula (12):

[0213]

[0214] Among them, v is the newly inserted midpoint on the edge (v1, v2), Signal(v), Signal(v1) and Signal(v2) are the displacements corresponding to vertices v, v1 and v2 respectively, and "←" means updating the content on the left side of the arrow by the content on the right side of the arrow.

[0215] The update process is defined as shown in formula (13):

[0216]

[0217] where v * is the set of all vertices adjacent to vertex v.

[0218] (3) Wavelet coefficient quantization:

[0219] There are many ways to quantize wavelet coefficients. One method is shown in equations (14) and (15):

[0220] disp[v].d[k]=floor(disp[v].d[k]*scale[k]) -(14)

[0221]

[0222] Where disp[v] represents the wavelet coefficient of the displacement of the vth vertex, d[k] represents the kth coefficient of its displacement, and floor represents rounding down. bitDepthPosition represents the bit depth of the current grid position, and qp[k] represents the quantization parameter of the kth coefficient. As mentioned above, after the displacement is converted to a coordinate system, its normal component has a more significant effect on the quality than the tangential component, so a larger quantization parameter can be used for the tangential component.

[0223] At the same time, according to the characteristics of wavelet transform, different quantization parameters can be used for the newly generated vertices and the original vertices. That is, for the subdivided vertices, the quantization parameter update is shown in formula (16):

[0224] scale[k]=scale[k]*lodScale[k] -(16)

[0225] Among them, lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.

[0226] (4) Wavelet coefficient arrangement:

[0227] For the wavelet coefficients generated after wavelet transform, arrange them on a two-dimensional image as follows:

[0228] Traverse the wavelet coefficients in order from low frequency to high frequency;

[0229] For each coefficient, determine the index of the NxM pixel block where it is located (e.g. N=M=16), where it should be stored in the raster scan order of the block;

[0230] The position of the corresponding NxM pixel block on the image is calculated according to the Morton order.

[0231] The arrangement method is not limited here, that is, other arrangement schemes may be used, such as zigzag order, raster order, etc. The encoder may explicitly specify the corresponding arrangement scheme in the bitstream.

[0232] (5) Video encoding:

[0233] After arranging the wavelet coefficients on the two-dimensional image, the video encoder can be used to encode it directly. The solution of the present application is independent of which video encoding technology is used, and the type of video encoder can be explicitly specified in the bit stream. At the same time, if quantization is applied before video encoding, the video encoding here can use a lossless method, and there is no need to reconstruct the displacement at the encoding end.

[0234] The above-mentioned displacement obtained by calculation is directly compressed and encoded using the existing entropy coding. For example, the displacement obtained by calculation can be compressed and encoded using Context Adaptive Binary Arithmetic Coding (CABAC). It should be noted that the present embodiment can also directly use entropy coding for the coefficients after wavelet transformation. Corresponding quantization can be performed before entropy coding.

[0235] After obtaining the displacement code stream, the encoding end can reconstruct the displacement based on the displacement code stream, and perform three-dimensional mesh reconstruction based on the reconstructed displacement and the first reconstructed mesh to obtain a third reconstructed mesh. For example, the first reconstructed mesh can be meshed subdivided to obtain a subdivided first reconstructed mesh, and then the vertices of the first reconstructed mesh corresponding to the displacement of each reconstructed vertex can be moved to obtain a third reconstructed mesh, which is also a reconstructed subdivided deformed mesh. Then, the first texture map can be converted according to the three-dimensional mesh to be encoded and the third reconstructed mesh to obtain a second texture map. The process of converting the first texture map according to the three-dimensional mesh to be encoded and the third reconstructed mesh can refer to the relevant description of the aforementioned texture map conversion method, which will not be repeated here.

[0236] In this embodiment, when the three-dimensional mesh to be encoded is subjected to mesh simplification processing, the simplified mesh is subdivided and deformed to obtain a second mesh, the displacement of each vertex of the second mesh is calculated based on the second reconstructed mesh and the second mesh, the displacement of each vertex of the second mesh is encoded to obtain a displacement code stream, and the three-dimensional mesh is reconstructed based on the first reconstructed mesh and the displacement code stream to obtain a third reconstructed mesh, thereby ensuring the quality of the reconstructed mesh.

[0237] In some optional embodiments, when the simplified mesh is subjected to subdivision deformation, the method further comprises:

[0238] The encoding end determines the displacement of the geometric coordinates of each vertex of the simplified mesh according to the displacement of the geometric coordinates of each vertex of the subdivided mesh; the encoding end moves each vertex of the simplified mesh according to the displacement of the geometric coordinates of each vertex of the simplified mesh to obtain a first mesh. The displacement of the geometric coordinates of each vertex of the subdivided mesh can be referred to the relevant description of the above embodiment, which will not be repeated here.

[0239] Optionally, the method further comprises:

[0240] The encoding end performs a first operation on first encoding information, wherein the first encoding information includes the grid code stream and the texture map code stream, and the first operation includes at least one of the following: storage, transmission.

[0241] Optionally, the first coding information further includes at least one of the following:

[0242] Displacement code stream;

[0243] The first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

[0244] It can be understood that, when the encoding end encodes to obtain the above-mentioned displacement code stream, the above-mentioned first encoding information also includes the displacement code stream, and when the above-mentioned displacement code stream is not encoded to obtain the above-mentioned displacement code stream, the above-mentioned first encoding information does not include the displacement code stream.

[0245] The above information related to grid coding, for example, the intra-frame coding flag, used to indicate whether the current frame needs to be constructed according to the reference frame grid; the reference frame list, used to indicate the index of the reference frame required for the current frame, and the reference frame list can be applied to the reconstruction of the first grid; the static grid encoder type, etc. The above information related to texture map coding, for example, the video encoder type. The above information related to grid subdivision and deformation, for example, the subdivision deformation flag, used to indicate whether the reconstructed grid needs to be subdivided and deformed; the subdivision method, that is, the method of grid subdivision when reconstructing the subdivided deformed grid. The above information related to grid displacement, for example, the encoding method of displacement; the displacement transformation method; the wavelet coefficient arrangement method, etc. The above information related to grid parameterization processing, for example, the algorithm used for grid parameterization processing (for example, Isochart algorithm, orthogonal projection algorithm, etc.).

[0246] It can be understood that the above-mentioned first auxiliary information can be used to assist the decoding end in decoding. For example, the decoding end decodes the grid code stream according to the information related to the grid coding, decodes the texture map code stream according to the information related to the texture map coding, subdivides and deforms the decoded grid according to the information related to the grid subdivision and deformation, performs grid parameterization on the decoded grid according to the information related to the grid parameterization processing, and decodes the displacement code stream according to the information related to the grid displacement. This is conducive to the decoding end to decode the encoded grid more accurately.

[0247] The following combination Fig.11 The three-dimensional grid coding method provided in the embodiment of the present application is illustrated by way of example.

[0248] See also Fig.11 , the encoding end performs mesh simplification processing on the input mesh to obtain a simplified mesh, and subdivides and deforms the simplified mesh to obtain a basic mesh (i.e., the first mesh) and a subdivided deformed mesh (i.e., the second mesh). The basic mesh is compressed and encoded to obtain a mesh code stream. The basic mesh is reconstructed based on the mesh code stream to obtain a reconstructed basic mesh (i.e., the first reconstructed mesh). The first reconstructed mesh is mesh parameterized to obtain a first reconstructed mesh (i.e., the second reconstructed mesh) containing texture coordinates. The displacement of each vertex of the subdivided deformed mesh is calculated based on the subdivided deformed mesh and the second reconstructed mesh. The displacement may include the displacement of the geometric coordinates and the displacement of the texture coordinates. The calculated displacement of each vertex is transformed, quantized, and the processed displacement is encoded to obtain a displacement code stream. Furthermore, the encoding end performs displacement reconstruction based on the displacement code stream to obtain a reconstructed location, and reconstructs a subdivision deformed grid (i.e., a third reconstructed grid) based on the reconstructed displacement and the first reconstructed grid, and converts the input texture map (i.e., the first texture map) based on the reconstructed subdivision deformed grid and the input grid to obtain a converted texture map (i.e., the second texture map), and uses a video encoder to encode the converted texture map to obtain a texture map code stream, and then the encoding end can store or transmit the obtained grid code stream, texture map code stream and displacement code stream.

[0249] See also Fig.12 , Fig.12 is a flowchart of a three-dimensional grid decoding method provided in an embodiment of the present application. The method can be executed by a decoding end, such as Fig.12 As shown, the following steps are included:

[0250] Step 1201: The decoding end obtains a grid code stream and a texture map code stream.

[0251] In this embodiment, the mesh code stream and texture map code stream are generated by the encoder according to the three-dimensional mesh coding method. Exemplarily, the decoder can receive the mesh code stream and texture map code stream from the encoder.

[0252] Step 1202: The decoding end decodes the grid code stream to obtain a third grid, and decodes the texture map code stream to obtain a third texture map.

[0253] In this embodiment, the third grid is a grid obtained after the grid code stream is decoded. The third grid can also be called a reconstructed basic grid. The third texture map is a texture map obtained after the texture code stream is decoded.

[0254] Specifically, the decoding end should use the decoding method corresponding to the grid coding method of the encoding end to decode the grid code stream. For example, if the encoding end uses intra-frame mode encoding, the decoding end also uses intra-frame mode decoding; if the encoding end uses inter-frame mode encoding, the decoding end also uses inter-frame mode decoding; if the encoding end uses skip mode encoding, the decoding end also uses skip mode decoding. Fig.13 The following are examples of the three decoding methods mentioned above:

[0255] like Fig.13 As shown, if it is intra-frame mode, a static grid decoder is used for decoding; if it is inter-frame mode, the corresponding motion vector is decoded, and the corresponding basic grid is reconstructed according to the reference frame in the grid buffer; if it is skip mode, the corresponding reference frame in the grid buffer is directly used as the basic grid of the current frame.

[0256] Similarly, the decoding end should use a decoding method corresponding to the texture image encoding method of the encoding end to decode the texture image bitstream. For example, if the encoding end uses a video encoder, the decoding end uses the corresponding video encoder to decode the texture image bitstream. In some optional embodiments, the decoding end can also perform color space conversion on the texture image obtained by decoding to obtain a color space that is consistent with the texture image input by the encoding end. Figure 1 consistent image format.

[0257] Step 1203: The decoding end performs mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

[0258] In this embodiment, the decoding end can use the same mesh parameterization processing method as the encoding end to perform mesh parameterization processing on the third mesh to obtain a third mesh containing texture coordinates, that is, the fourth mesh. Then, the decoding end can apply the third texture map and the fourth mesh.

[0259] In the embodiment of the present application, a mesh obtained by decoding a mesh code stream at a decoding end is subjected to mesh parameterization processing to obtain the texture coordinates of the mesh. This can reduce the storage resources or transmission resources occupied by the code stream of the three-dimensional mesh while ensuring the texture coordinates of the mesh that can be obtained by decoding.

[0260] Optionally, the decoding end performs grid parameterization processing on the third grid to obtain a fourth grid, including:

[0261] The decoding end performs grid parameterization processing on the third grid by using an orthogonal projection algorithm to obtain a fourth grid.

[0262] Optionally, the decoding end performs grid parameterization processing on the third grid using an orthogonal projection algorithm to obtain a fourth grid, including:

[0263] The decoding end divides the third grid into at least two connected regions, wherein each of the connected regions includes at least one face of the third grid, and faces having the same projection face and being connected in three-dimensional space belong to the same connected region;

[0264] The decoding end merges the at least two connected regions to obtain at least one first connected region;

[0265] The decoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions;

[0266] The decoding end merges the connected regions obtained after the first processing to obtain at least one second connected region;

[0267] The decoding end arranges the projection area of ​​the at least one second connected area in two dimensions to obtain a fourth grid.

[0268] Optionally, the decoding end performs a merging process on the connected regions obtained after the first process to obtain at least one second connected region, including:

[0269] The decoding end obtains a candidate region set of a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region in the connected regions obtained after the first processing, and the third connected region is any connected region in the connected regions obtained after the first processing;

[0270] The decoding end obtains at least one target candidate region of the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection region of the third connected region does not overlap with a projection region of the target candidate region;

[0271] The decoding end merges the third connected region into one of the at least one target candidate region.

[0272] Optionally, the decoding end merges the third connected region into one of the at least one target candidate region, comprising:

[0273] The decoding end merges the third connected region into the first target candidate region;

[0274] Among them, the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.

[0275] Optionally, there are at least two overlapping vertices between the candidate region and the third connected region.

[0276] Optionally, the method further comprises:

[0277] The decoding end obtains the displacement code stream;

[0278] The decoding end subdivides and deforms the fourth grid to obtain a fifth grid;

[0279] The decoding end reconstructs a three-dimensional grid according to the fifth grid and the displacement obtained by decoding the displacement code stream to obtain a fourth reconstructed grid.

[0280] In this embodiment, when the decoder obtains the displacement code stream, it decodes the displacement code stream. For example, if the encoder compresses the displacement by video encoding, the decoder decodes it by a corresponding video decoder and restores it from the two-dimensional image in the corresponding order according to the arrangement scheme. Then, it performs inverse transformation, inverse quantization and other operations to restore the displacement consistent with the encoder. If entropy encoding is used, it can be directly entropy decoded.

[0281] After the basic mesh and displacement decoding and reconstruction are completed, the subdivided deformed mesh is reconstructed based on the two parts. Exemplarily, if the displacement encoding process uses a reference frame as a reference, that is, the displacement of the vertices in the area matching the reference frame in the decoded displacement is the residual of the displacement of the current frame vertex relative to the reference frame vertex, then firstly, the displacement of the vertices in the current frame matching area is restored by the vertices and residuals of the subdivided deformed mesh reconstructed by the reference frame, and then the subsequent reconstruction steps are performed. That is, the reconstructed basic mesh is subdivided, which is consistent with the subdivision method of the encoding end. The corresponding displacement is added to each vertex of the subdivided mesh, as shown in formula (17):

[0282] deformedmesh[i].v[k]=subdivmesh[i].v[k]+displacement[k] -(17)

[0283] Among them, subdivmesh[i].v[k] is the geometric coordinates of the kth vertex after the base mesh is subdivided in the current frame (index is i), displacement[k] is the displacement corresponding to the kth vertex, and deformedmesh[i].v[k] is the geometric coordinates of the kth vertex after subdivision and deformation in the current frame.

[0284] Optionally, the method further comprises:

[0285] The decoding end obtains first auxiliary information, wherein the first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

[0286] In this embodiment, when the decoding end obtains the first auxiliary information, it can perform decoding based on the first auxiliary information. For example, the decoding end decodes the grid code stream according to information related to grid encoding, decodes the texture map code stream according to information related to texture map encoding, subdivides and deforms the decoded grid according to information related to grid subdivision and deformation, performs grid parameterization on the decoded grid according to information related to grid parameterization, and decodes the displacement code stream according to information related to grid displacement. This is conducive to the decoding end to decode the encoded grid more accurately.

[0287] It should be noted that the above-mentioned first auxiliary information can be encoded into an independent code stream, or it can be carried in the above-mentioned grid code stream, texture map code stream and displacement code stream. For example, information related to grid coding is carried in the above-mentioned grid code stream, information related to texture map coding is carried in the above-mentioned texture map code stream, information related to grid displacement is carried in the displacement code stream, etc.

[0288] It should be noted that the implementation methods of the above embodiments can be found in Figure 3 The relevant description of the illustrated embodiment is not repeated here.

[0289] The following combination Fig.14 The three-dimensional grid decoding method provided in the embodiment of the present application is illustrated by way of example.

[0290] See also Fig.14 The decoding end obtains the decoding of each part of the code stream and reconstructs it. After the basic grid is reconstructed, the grid parameterization is performed to obtain the corresponding basic grid containing texture coordinates. The basic grid is subdivided and the displacement obtained by decoding is applied to obtain the subdivided deformed grid reconstructed by the decoding end, that is, the reconstructed grid finally output by the decoding end.

[0291] It should be noted that the 3D mesh coding method provided in the embodiment of the present application can be executed by a 3D mesh coding device, or a control module in the 3D mesh coding device for executing the 3D mesh coding method. In the embodiment of the present application, the 3D mesh coding device executing the 3D mesh coding method is taken as an example to illustrate the 3D mesh coding device provided in the embodiment of the present application.

[0292] See also Fig.15 , Fig.15 is a structural diagram of a three-dimensional grid encoding device provided in an embodiment of the present application, such as Fig.15 As shown, the three-dimensional grid encoding device 1500 includes:

[0293] A first encoding module 1501 is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded;

[0294] A first reconstruction module 1502 is used to perform three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid;

[0295] A first processing module 1503 is configured to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates;

[0296] A conversion module 1504 is used to convert the first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is the texture map corresponding to the three-dimensional mesh to be encoded;

[0297] The second encoding module 1505 is used to encode the second texture map to obtain a texture map bit stream.

[0298] Optionally, the first processing module is specifically configured to:

[0299] An orthogonal projection algorithm is used to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh.

[0300] Optionally, the first processing module is specifically configured to:

[0301] Dividing the first reconstructed mesh into at least two connected regions, wherein each of the connected regions includes at least one face of the first reconstructed mesh, and faces having the same projection face and being connected in three-dimensional space belong to the same connected region;

[0302] Merging the at least two connected regions to obtain at least one first connected region;

[0303] Performing a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions;

[0304] Merging the connected regions obtained after the first processing to obtain at least one second connected region;

[0305] The projection area of ​​the at least one second connected area is two-dimensionally arranged to obtain a second reconstructed grid.

[0306] Optionally, the first processing module is specifically configured to:

[0307] Acquire a candidate region set of a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region in the connected regions obtained after the first processing, and the third connected region is any connected region in the connected regions obtained after the first processing;

[0308] Acquire at least one target candidate region of the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection region of the third connected region does not overlap with a projection region of the target candidate region;

[0309] The third connected region is merged into one of the at least one target candidate region.

[0310] Optionally, the first processing module is specifically configured to:

[0311] Merging the third connected region into the first target candidate region;

[0312] Among them, the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.

[0313] Optionally, there are at least two overlapping vertices between the candidate region and the third connected region.

[0314] Optionally, the first grid is determined according to a simplified grid, and the simplified grid is a grid obtained by performing grid simplification processing on the three-dimensional grid to be encoded;

[0315] The number of vertices of the simplified mesh is smaller than the number of vertices of the three-dimensional mesh to be encoded, or the number of faces of the simplified mesh is smaller than the number of faces of the three-dimensional mesh to be encoded.

[0316] Optionally, the device further comprises:

[0317] A first subdivision deformation module, used for performing subdivision deformation on the simplified mesh to obtain a second mesh;

[0318] The conversion module is specifically used for:

[0319] Calculating the displacement of each vertex of the second mesh according to the second reconstructed mesh and the second mesh;

[0320] Encoding the displacement of each vertex of the second mesh to obtain a displacement code stream;

[0321] Performing three-dimensional grid reconstruction according to the first reconstruction grid and the displacement code stream to obtain a third reconstruction grid;

[0322] The first texture map is converted according to the three-dimensional grid to be encoded and the third reconstructed grid to obtain a second texture map.

[0323] Optionally, the device further comprises:

[0324] The first execution module is used to perform a first operation on first coded information, wherein the first coded information includes the grid code stream and the texture map code stream, and the first operation includes at least one of the following: storage and transmission.

[0325] Optionally, the first coding information further includes at least one of the following:

[0326] Displacement code stream;

[0327] The first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

[0328] The three-dimensional grid coding device in the embodiment of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or a network side device, or it can be other devices other than a terminal or a network side device. Exemplarily, the terminal can include but is not limited to the types of terminals 11 listed above, and the network side device can include but is not limited to the types of network side devices 12 listed above. Other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiment of the present application.

[0329] The three-dimensional grid encoding device provided in the embodiment of the present application can achieve Figure 3 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0330] It should be noted that the 3D mesh decoding method provided in the embodiment of the application can be executed by a 3D mesh decoding device, or a control module in the 3D mesh decoding device for executing the 3D mesh decoding method. In the embodiment of the application, the 3D mesh decoding device executing the 3D mesh decoding method is taken as an example to illustrate the 3D mesh decoding device provided in the embodiment of the application.

[0331] See also Fig.16 , Fig.16 is a structural diagram of a three-dimensional grid decoding device provided in an embodiment of the present application, such as Fig.16 As shown, the three-dimensional grid decoding device 1600 includes:

[0332] The first acquisition module 1601 is used to acquire a grid code stream and a texture map code stream;

[0333] A decoding module 1602, configured to decode the mesh code stream to obtain a third mesh, and decode the texture map code stream to obtain a third texture map;

[0334] The second processing module 1603 is used to perform mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

[0335] Optionally, the second processing module is specifically configured to:

[0336] An orthogonal projection algorithm is used to perform grid parameterization processing on the third grid to obtain a fourth grid.

[0337] Optionally, the second processing module is specifically configured to:

[0338] Dividing the third grid into at least two connected regions, wherein each of the connected regions includes at least one face of the third grid, and faces having the same projection face and being connected in three-dimensional space belong to the same connected region;

[0339] Merging the at least two connected regions to obtain at least one first connected region;

[0340] Performing a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions;

[0341] Merging the connected regions obtained after the first processing to obtain at least one second connected region;

[0342] The projection area of ​​the at least one second connected area is two-dimensionally arranged to obtain a fourth grid.

[0343] Optionally, the second processing module is specifically configured to:

[0344] Acquire a candidate region set of a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region in the connected regions obtained after the first processing, and the third connected region is any connected region in the connected regions obtained after the first processing;

[0345] Acquire at least one target candidate region of the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection region of the third connected region does not overlap with a projection region of the target candidate region;

[0346] The third connected region is merged into one of the at least one target candidate region.

[0347] Optionally, the second processing module is specifically configured to:

[0348] Merging the third connected region into the first target candidate region;

[0349] Among them, the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.

[0350] Optionally, there are at least two overlapping vertices between the candidate region and the third connected region.

[0351] Optionally, the device further comprises:

[0352] The second acquisition module is used to acquire the displacement code stream;

[0353] A second subdivision deformation module, used for performing subdivision deformation on the fourth mesh to obtain a fifth mesh;

[0354] The second reconstruction module is used to perform three-dimensional grid reconstruction according to the fifth grid and the displacement obtained by decoding the displacement code stream to obtain a fourth reconstructed grid.

[0355] Optionally, the device further comprises:

[0356] The third acquisition module is used to obtain first auxiliary information, wherein the first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

[0357] The three-dimensional grid decoding device in the embodiment of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a network-side device or a terminal, or can be other devices other than network-side devices or terminals. Exemplarily, the network-side device can include but is not limited to the types of network-side devices 12 listed above, the terminal can include but is not limited to the types of terminals 11 listed above, and other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiment of the present application.

[0358] The three-dimensional grid decoding device provided in the embodiment of the present application can achieve Fig.12 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0359] Optional, such as Fig.17As shown, the embodiment of the present application further provides a communication device 1700, including a processor 1701 and a memory 1702, and the memory 1702 stores a program or instruction that can be run on the processor 1701. For example, when the communication device 1700 is an encoding end, the program or instruction is executed by the processor 1701 to implement the various steps of the above-mentioned three-dimensional mesh encoding method embodiment, and can achieve the same technical effect. When the communication device 1700 is a decoding end, the program or instruction is executed by the processor 1701 to implement the various steps of the above-mentioned three-dimensional mesh decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0360] The embodiment of the present application also provides a decoding end, including a processor and a communication interface, the processor is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded; reconstruct the three-dimensional grid based on the grid code stream to obtain a first reconstructed grid; perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; convert a first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional grid to be encoded; encode the second texture map to obtain a texture map code stream. This encoding end embodiment corresponds to the above-mentioned encoding end side method embodiment, and each implementation process and implementation method of the above-mentioned method embodiment can be applied to this encoding end embodiment, and can achieve the same technical effect. Specifically, Fig.18 A schematic diagram of the hardware structure of an encoding end to implement an embodiment of the present application.

[0361] The encoding end 1800 includes but is not limited to: a radio frequency unit 1801, a network module 1802, an audio output unit 1803, an input unit 1804, a sensor 1805, a display unit 1806, a user input unit 1807, an interface unit 1808, a memory 1809 and at least some of the components of the processor 1810.

[0362] Those skilled in the art will appreciate that the encoding end 1800 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1810 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.18 The encoding end structure shown in the figure does not constitute a limitation on the encoding end. The encoding end may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0363] It should be understood that in the embodiment of the present application, the input unit 1804 may include a graphics processing unit (GPU) 18041 and a microphone 18042, and the graphics processor 18041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1806 may include a display panel 18061, and the display panel 18061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1807 includes a touch panel 18071 and at least one of other input devices 18072. The touch panel 18071 is also called a touch screen. The touch panel 18071 may include two parts: a touch detection device and a touch controller. Other input devices 18072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0364] In the embodiment of the present application, after receiving downlink data from the network side device, the RF unit 1801 can transmit the data to the processor 1810 for processing; in addition, the RF unit 1801 can send uplink data to the network side device. Generally, the RF unit 1801 includes but is not limited to an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0365] The memory 1809 can be used to store software programs or instructions and various data. The memory 1809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1809 may include a volatile memory or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1809 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0366] The processor 1810 may include one or more processing units; optionally, the processor 1810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1810.

[0367] Among them, processor 1810 is used to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded; reconstruct a three-dimensional grid based on the grid code stream to obtain a first reconstructed grid; perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, wherein the second reconstructed grid is a first reconstructed grid including texture coordinates; convert a first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional grid to be encoded; encode the second texture map to obtain a texture map code stream.

[0368] It can be understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the aforementioned method embodiment, and achieve the same or corresponding technical effect. To avoid repetition, it will not be repeated here.

[0369] The embodiment of the present application also provides a decoding end, including a processor and a communication interface, the processor is used to obtain a grid code stream and a texture map code stream; decode the grid code stream to obtain a third grid, and decode the texture map code stream to obtain a third texture map; perform grid parameterization processing on the third grid to obtain a fourth grid, wherein the fourth grid is a grid including texture coordinates. This decoding end embodiment corresponds to the above-mentioned decoding end method embodiment, and each implementation process and implementation method of the above-mentioned method embodiment can be applied to this decoding end embodiment, and can achieve the same technical effect.

[0370] Specifically, the embodiment of the present application also provides a decoding end. Fig.19 As shown, the decoding end 1900 includes: an antenna 1901, a radio frequency device 1902, a baseband device 1903, a processor 1904 and a memory 1905. The antenna 1901 is connected to the radio frequency device 1902. In the uplink direction, the radio frequency device 1902 receives information through the antenna 1901 and sends the received information to the baseband device 1903 for processing. In the downlink direction, the baseband device 1903 processes the information to be sent and sends it to the radio frequency device 1902, and the radio frequency device 1902 processes the received information and sends it out through the antenna 1901.

[0371] The method executed by the decoding end in the above embodiment may be implemented in the baseband device 1903, which includes a baseband processor.

[0372] The baseband device 1903 may include, for example, at least one baseband board on which a plurality of chips are arranged. Fig.19As shown, one of the chips is, for example, a baseband processor, which is connected to the memory 1905 through a bus interface to call the program in the memory 1905 to execute the network device operations shown in the above method embodiment.

[0373] The decoding end may also include a network interface 1906, which is, for example, a Common Public Radio Interface (CPRI).

[0374] Specifically, the decoding end 1900 of the embodiment of the present application further includes: instructions or programs stored in the memory 1905 and executable on the processor 1904, and the processor 1904 calls the instructions or programs in the memory 1905 to execute. Fig.16 The methods executed by the modules shown achieve the same technical effects, and therefore will not be described here in detail to avoid repetition.

[0375] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned three-dimensional grid coding method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0376] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[0377] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned three-dimensional grid coding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0378] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0379] The embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the various processes of the above-mentioned three-dimensional grid coding method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0380] The embodiment of the present application also provides a three-dimensional mesh encoding and decoding system, including: an encoding end and a decoding end, wherein the encoding end is used to perform the following Figure 3And each process of each method embodiment above, the decoding end is used to perform the following steps: Fig.12 And the various processes of the above-mentioned method embodiments can achieve the same technical effect. To avoid repetition, they will not be described here.

[0381] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0382] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, disk, CD, etc.), including several instructions to enable a terminal or a network-side device to execute the methods described in each embodiment of the present application.

[0383] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of the present application and the scope of protection of the claims, and these implementation methods are all within the protection of the present application.

Claims

1. A three-dimensional grid coding method, characterized in that: include: The encoding end compresses and encodes the first grid to obtain a grid code stream, wherein the first grid is a grid determined according to the three-dimensional grid to be encoded; The encoding end performs three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid; The encoder performs mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates; The encoding end converts the first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is the texture map corresponding to the three-dimensional mesh to be encoded; The encoding end encodes the second texture map to obtain a texture map bit stream.

2. The method according to claim 1, characterized in that The encoder performs grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid, including: The encoding end uses an orthogonal projection algorithm to perform grid parameterization processing on the first reconstructed grid to obtain a second reconstructed grid.

3. The method according to claim 2, characterized in that The encoding end performs mesh parameterization processing on the first reconstructed mesh by using an orthogonal projection algorithm to obtain a second reconstructed mesh, including: The encoding end divides the first reconstructed grid into at least two connected regions, wherein each of the connected regions includes at least one face of the first reconstructed grid, and faces having the same projection face and being connected in three-dimensional space belong to the same connected region; The encoding end merges the at least two connected regions to obtain at least one first connected region; The encoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions; The encoding end merges the connected regions obtained after the first processing to obtain at least one second connected region; The encoding end arranges the projection area of ​​the at least one second connected area in two dimensions to obtain a second reconstructed grid.

4. The method according to claim 3, characterized in that The encoder performs a merging process on the connected regions obtained after the first process to obtain at least one second connected region, including: The encoder end obtains a candidate region set for a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region among the connected regions obtained after the first processing, and the third connected region is any connected region among the connected regions obtained after the first processing; The encoder obtains at least one target candidate region of the merged candidate region set, wherein when the third connected region is merged into the target candidate region, a projection region of the third connected region does not overlap with a projection region of the target candidate region; The encoder merges the third connected region into one of the at least one target candidate region.

5. The method according to claim 4, characterized in that The encoder merges the third connected region into one of the at least one target candidate region, including: The encoder merges the third connected region into the first target candidate region; Among them, the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.

6. The method according to claim 4 or 5, characterized in that: There are at least two overlapping vertices between the candidate region and the third connected region.

7. The method according to any one of claims 1 to 6, characterized in that The first grid is determined according to a simplified grid, where the simplified grid is a grid obtained by performing grid simplification processing on the three-dimensional grid to be encoded; The number of vertices of the simplified mesh is smaller than the number of vertices of the three-dimensional mesh to be encoded, or the number of faces of the simplified mesh is smaller than the number of faces of the three-dimensional mesh to be encoded.

8. The method according to claim 7, characterized in that The method further comprises: The encoding end subdivides and deforms the simplified grid to obtain a second grid; The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the second reconstructed grid to obtain a second texture map, including: The encoding end calculates the displacement of each vertex of the second mesh according to the second reconstructed mesh and the second mesh; The encoding end encodes the displacement of each vertex of the second mesh to obtain a displacement code stream; The encoding end performs three-dimensional grid reconstruction according to the first reconstruction grid and the displacement code stream to obtain a third reconstruction grid; The encoding end converts the first texture map according to the three-dimensional grid to be encoded and the third reconstructed grid to obtain a second texture map.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: The encoding end performs a first operation on first encoding information, wherein the first encoding information includes the grid code stream and the texture map code stream, and the first operation includes at least one of the following: storage, transmission.

10. The method according to claim 9, characterized in that The first coding information also includes at least one of the following: Displacement code stream; The first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

11. A three-dimensional grid decoding method, characterized in that: include: The decoding end obtains the grid code stream and the texture map code stream; The decoding end decodes the grid code stream to obtain a third grid, and decodes the texture map code stream to obtain a third texture map; The decoding end performs mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

12. The method according to claim 11, characterized in that The decoding end performs grid parameterization processing on the third grid to obtain a fourth grid, including: The decoding end performs grid parameterization processing on the third grid by using an orthogonal projection algorithm to obtain a fourth grid.

13. The method according to claim 12, characterized in that The decoding end performs grid parameterization processing on the third grid by using an orthogonal projection algorithm to obtain a fourth grid, including: The decoding end divides the third grid into at least two connected regions, wherein each of the connected regions includes at least one face of the third grid, and faces having the same projection face and being connected in three-dimensional space belong to the same connected region; The decoding end merges the at least two connected regions to obtain at least one first connected region; The decoding end performs a first process on each of the at least one first connected regions, wherein the first process includes projecting each surface of each of the first connected regions to detect overlapping regions, and splitting the first connected regions with overlapping regions into at least two non-overlapping connected regions; The decoding end merges the connected regions obtained after the first processing to obtain at least one second connected region; The decoding end arranges the projection area of ​​the at least one second connected area in two dimensions to obtain a fourth grid.

14. The method according to claim 13, characterized in that The decoding end performs a merging process on the connected regions obtained after the first process to obtain at least one second connected region, including: The decoding end obtains a candidate region set of a third connected region, wherein the candidate region set includes at least one candidate region, the candidate region is a connected region adjacent to the third connected region in the connected regions obtained after the first processing, and the third connected region is any connected region in the connected regions obtained after the first processing; The decoding end obtains at least one target candidate region of the merged candidate region set, wherein, when the third connected region is merged into the target candidate region, a projection region of the third connected region does not overlap with a projection region of the target candidate region; The decoding end merges the third connected region into one of the at least one target candidate region.

15. The method according to claim 14, characterized in that The decoding end merges the third connected region into one of the at least one target candidate region, including: The decoding end merges the third connected region into the first target candidate region; Among them, the first target candidate area is the target candidate area in which the angle between the projection plane and the normal vector of the third connected area is the smallest among the at least one target candidate area, or the first target candidate area is the target candidate area with the largest number of faces contained in the at least one target candidate area.

16. The method according to claim 14 or 15, characterized in that There are at least two overlapping vertices between the candidate region and the third connected region.

17. The method according to any one of claims 11 to 16, characterized in that The method further comprises: The decoding end obtains the displacement code stream; The decoding end subdivides and deforms the fourth grid to obtain a fifth grid; The decoding end reconstructs a three-dimensional grid according to the fifth grid and the displacement obtained by decoding the displacement code stream to obtain a fourth reconstructed grid.

18. The method according to any one of claims 11 to 17, characterized in that The method further comprises: The decoding end obtains first auxiliary information, wherein the first auxiliary information includes at least one of the following: information related to mesh coding, information related to texture map coding, information related to mesh subdivision and deformation, information related to mesh displacement, and information related to mesh parameterization processing.

19. A three-dimensional grid encoding device, characterized in that: include: A first encoding module, configured to compress and encode a first grid to obtain a grid code stream, wherein the first grid is a grid determined according to a three-dimensional grid to be encoded; A first reconstruction module, used for performing three-dimensional grid reconstruction based on the grid code stream to obtain a first reconstructed grid; A first processing module, configured to perform mesh parameterization processing on the first reconstructed mesh to obtain a second reconstructed mesh, wherein the second reconstructed mesh is the first reconstructed mesh including texture coordinates; A conversion module, configured to convert a first texture map according to the three-dimensional mesh to be encoded and the second reconstructed mesh to obtain a second texture map, wherein the first texture map is a texture map corresponding to the three-dimensional mesh to be encoded; The second encoding module is used to encode the second texture map to obtain a texture map code stream.

20. A three-dimensional grid decoding device, characterized in that: include: A first acquisition module is used to acquire a grid code stream and a texture map code stream; A decoding module, used for decoding the grid code stream to obtain a third grid, and decoding the texture map code stream to obtain a third texture map; The second processing module is used to perform mesh parameterization processing on the third mesh to obtain a fourth mesh, wherein the fourth mesh is a mesh including texture coordinates.

21. An electronic device, characterized in that: It includes a processor and a memory, the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, it implements the steps of the three-dimensional grid encoding method as described in any one of claims 1 to 10, or implements the steps of the three-dimensional grid decoding method as described in any one of claims 11 to 18.

22. A chip, characterized in that: The chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the steps of the three-dimensional grid encoding method as described in any one of claims 1 to 10, or to implement the steps of the three-dimensional grid decoding method as described in any one of claims 11 to 18.

23. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the three-dimensional grid encoding method as described in any one of claims 1 to 10 are implemented, or the steps of the three-dimensional grid decoding method as described in any one of claims 11 to 18 are implemented.

Images