Encoding method, decoding method and related equipment

Abstract

This application discloses an encoding method, a decoding method, and related devices, which belong to the technical field of 3D mesh encoding. The encoding method of an embodiment of this application includes the steps of: determining a base mesh bitstream based on a mesh to be encoded; and generating a target bitstream based on a texture map to be encoded corresponding to the mesh to be encoded and the base mesh bitstream, wherein the target bitstream includes target identification information for indicating a method of obtaining target texture coordinates, and the obtaining method includes obtaining the target texture coordinates from the base mesh bitstream or generating the target texture coordinates by a decoding side based on mesh parameterization.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority from Chinese Patent Application No. 202310262481.2 filed in China on March 17, 2023, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of 3D mesh coding, and more particularly to an encoding process method, a decoding process method and related equipment. [Background technology]

[0003] In the process of 3D mesh coding, mesh coding is usually performed using a fixed coding method. For example, the encoding device performs mesh simplification, mesh parameterization, subdivision, and transformation on the mesh to be coded to obtain a base mesh, and then compresses and codes the base mesh to obtain a base mesh bitstream. However, the bitstream obtained by coding is large. Therefore, the conventional technology has a problem of low flexibility in 3D mesh coding. Summary of the Invention

[0004] The embodiments of the present application provide an encoding method, a decoding method and related devices that can solve the problem of low flexibility in 3D mesh coding.

[0005] In a first aspect, there is provided an encoding processing method applied to an encoding side, comprising: determining a base mesh bitstream based on a mesh to be encoded; and generating a target bitstream based on a texture map to be coded corresponding to the mesh to be coded and the base mesh bitstream, wherein the target bitstream includes target identification information for indicating a method for obtaining target texture coordinates, and the obtaining method includes obtaining the target texture coordinates from the base mesh bitstream or generating the target texture coordinates by a decoding side based on mesh parameterization.

[0006] In a second aspect, a decoding processing method applied to a decoding side, receiving a target bitstream, the target bitstream including a base mesh bitstream and target identification information for indicating how to obtain target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; decoding the base mesh bitstream to obtain a third mesh; generating a target decoded mesh based on the third mesh and the acquisition method; A decoding method is provided, wherein the target decoded mesh includes the target texture coordinates.

[0007] In a third aspect, there is provided a coding processing device applied to a coding side, a processing module for determining a base mesh bitstream based on a mesh to be encoded; and a generation module for generating a target bitstream based on a texture map to be coded corresponding to the mesh to be coded and the base mesh bitstream, wherein the target bitstream includes target identification information for indicating a method for obtaining target texture coordinates, and the obtaining method includes obtaining the target texture coordinates from the base mesh bitstream or generating the target texture coordinates by a decoding side based on mesh parameterization.

[0008] In a fourth aspect, a decoding processing device applied to a decoding side, a receiving module for receiving a target bitstream, the target bitstream including a base mesh bitstream and target identification information for indicating a method of obtaining target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; a decoding module for decoding the base mesh bitstream to obtain a third mesh, and generating a target decoded mesh based on the third mesh and the obtaining method; A decoding processor is provided, wherein the target decoded mesh includes the target texture coordinates.

[0009] In a fifth aspect, there is provided an electronic device comprising a processor and a memory for storing a program or command executable by the processor, wherein when the program or command is executed by the processor, the steps of the method described in the first aspect are realized, or when the program or command is executed by the processor, the steps of the method described in the second aspect are realized.

[0010] In a sixth aspect, there is provided an electronic device comprising a processor and a communication interface, When the electronic device is an encoding side, the processor is used to determine a base mesh bitstream based on a mesh to be encoded, and generate a target bitstream based on a texture map to be encoded corresponding to the mesh to be encoded and the base mesh bitstream, the target bitstream includes target identification information for indicating a method of obtaining target texture coordinates, and the obtaining method includes obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; When the electronic device is a decoding side, the communication interface is used to receive a target bitstream including a base mesh bitstream and target identification information for instructing a method for obtaining target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by the decoding side based on mesh parameterization, and the processor is used to decode the base mesh bitstream to obtain a third mesh and generate a target decoded mesh based on the third mesh and the obtaining method, and the target decoded mesh includes the target texture coordinates.

[0011] In a seventh aspect, there is provided a video encoding / decoding system comprising an encoding side device that can be used to perform steps of the encoding processing method described in the first aspect, and a decoding side device that can be used to perform steps of the decoding processing method described in the second aspect.

[0012] In an eighth aspect, there is provided a readable storage medium having a program or command stored thereon, the program or command being executed by a processor to achieve the steps of the method according to the first aspect or to achieve the steps of the method according to the second aspect.

[0013] In a ninth aspect, there is provided a chip comprising a processor and a communication interface, wherein the communication interface and the processor are coupled together, and the processor executes a program or command to implement steps of the method according to the first aspect, or is used to implement steps of the method according to the second aspect.

[0014] In a tenth aspect, there is provided a computer program / program product stored on a storage medium and executed by at least one processor to implement the steps of the method according to the first aspect or to implement the steps of the method according to the second aspect.

[0015] In the embodiment of the present application, target identification information is set in the target bitstream to indicate how to obtain the target texture coordinates, so that 3D mesh encoding can be performed in different ways according to different needs, thereby improving the flexibility of 3D mesh encoding. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram of a conventional coding framework. [Figure 2] FIG. 1 is a conventional decoding framework diagram. [Figure 3] FIG. 1 is a flow diagram of an encoding processing method provided in an embodiment of the present application. [Figure 4] FIG. 2 is an exemplary framework diagram of encoding in the encoding processing method provided in the embodiment of the present application. [Figure 5] FIG. 10 is an illustrative diagram of a mesh simplification operation in the encoding processing method provided in the embodiment of the present application. [Figure 6] FIG. 10 is an exemplary diagram of a subdivision process in the encoding method provided in the embodiment of the present application. [Figure 7] FIG. 10 is an exemplary framework diagram of another encoding in the encoding processing method provided in the embodiments of the present application. [Figure 8] FIG. 2 is a flow diagram of a decoding processing method provided in an embodiment of the present application. [Figure 9] FIG. 2 is an exemplary framework diagram of decoding in the decoding processing method provided in the embodiment of the present application. [Figure 10] FIG. 10 is an exemplary framework diagram of another decoding process in the decoding processing method provided in the embodiment of the present application. [Figure 11] 1 is a block diagram of an encoding processing device provided in an embodiment of the present application. [Figure 12] 1 is a block diagram of a decoding processing device provided in an embodiment of the present application; [Figure 13] FIG. 1 is a configuration diagram of a communication device provided in an embodiment of the present application. [Figure 14]1 is a block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, the technical solutions in the embodiments of the present application will be clearly explained with reference to the drawings in the embodiments of the present application, and it should be understood that the described embodiments are only a part of the embodiments of the present application, not all of the embodiments, and all other embodiments obtained by those skilled in the art based on the embodiments in the present application fall within the scope of protection of the present application.

[0018] The terms "first," "second," etc., used in the specification and claims of this application are not intended to describe a particular order or sequence, but rather to distinguish between similar objects. It should be understood that terms used in this manner may be interchanged where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein. It should also be understood that the objects distinguished by "first" and "second" generally belong to a single category and do not limit the number of objects; for example, the first object may be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, "A or B" includes three situations: including A but not B, including B but not A, and including both A and B. The symbol " / " generally indicates that the related objects are in an "or" relationship.

[0019] The term "instruction" in the specification and claims of this application may be an explicit instruction or an implicit instruction. Here, an explicit instruction can be understood as an instruction sent by a sender that clearly informs a receiver of an operation to be performed or a requested result. An implicit instruction can be understood as an instruction sent by a sender that the receiver makes a decision based on the instruction sent by the sender and determines an operation to be performed or a requested result, etc., based on the decision result.

[0020] The encoding / decoding side corresponding to the mesh encoding / decoding method in the embodiments of the present application may be a terminal, which may be called a terminal device or user terminal (UE), and the terminal may be a terminal-side device such as a mobile phone, a tablet personal computer, a laptop computer (also called a notebook computer), a personal digital assistant (PDA), a palmtop computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), a wearable device or a vehicle user equipment (VUE), a pedestrian user equipment (PUE), etc., and wearable devices include smart watches, bracelets, earphones, glasses, etc. It should be noted that the specific type of terminal is not limited in the embodiments of the present application.

[0021] For ease of understanding, the following describes the relevant contents of the examples of the present application.

[0022] 1. Visual Volumetric Video-based Coding (V3C) standard.

[0023] The V3C standard provides methods for encoding and decoding various 3D media using video or image coding technologies. Specifically, before encoding, 3D media content is converted from its 3D representation into multiple 2D representations (called V3C components) using methods such as projection, and then the 2D representations are encoded using existing video or image coding technologies. V3C components primarily include occupancy components, geometry components, and attribute components. The occupancy components can indicate which areas of the 2D representation are associated with the data in the given 3D representation. The geometry components represent information about the spatial location of the 3D data, and the attribute components can provide attribute information corresponding to vertices, such as material and texture. The components also include information on how to reconstruct a 3D model from these components, known as atlas information.

[0024] The atlas information is what connects all the components together, and the atlas components also contain additional information to reconstruct the 2D back to 3D. The atlas is made up of basic units called patches. Each patch represents a region in the available 2D components and contains the information needed to project that region back into 3D space.

[0025] Second, video-based dynamic mesh coding (VDMC).

[0026] VDMC is a standard for compressing 3D meshes developed by the Moving Picture Experts Group (MPEG). Its main idea is to compress 3D meshes using the existing V3C standard. Because 3D meshes contain connectivity information that must be encoded, the specific encoding flow differs slightly from V3C. The V3C standard's decoding semantics and decoding operations must be extended to support the decoding and reconstruction of 3D meshes. Figures 1 and 2 show the VDMC-related encoding and decoding framework.

[0027] The overall coding framework is shown in Figure 1. The input mesh is first simplified by a simplification module, then mesh parameterization generates new texture coordinates for the mesh. The parameterized mesh is then subdivision and deformation processed, i.e., new vertices are inserted into the mesh according to a specific subdivision method. The distances from the mesh vertices after subdivision to the nearest points on the input mesh are calculated, which is called displacement information. The parameterized mesh, i.e., the vertex positions of the mesh before subdivision and deformation, are then adjusted based on the displacement information. The adjusted mesh, called the base mesh, is sent to a base mesh encoding module and compressed using an existing base mesh encoding module and existing mesh encoder. In interframe mode, motion vectors for each vertex of the base mesh can also be generated based on a reference frame, and the base mesh module only needs to compress the motion vectors. After encoding, the base mesh is reconstructed, and the displacement sequence is then adjusted based on the vertex sequence of the reconstructed base mesh. The sequence-adjusted vertex displacement information is then first wavelet transformed, and the transformed coefficients (also called wavelet coefficients) are quantized. The quantized coefficients are then arranged into a two-dimensional image according to a specific scan sequence, and the two-dimensional image is encoded using a video encoder. The reconstructed displacement information is then applied to the subdivided base mesh to obtain a reconstructed subdivision-deformed mesh. The mesh, the original input mesh, and its corresponding texture map are input to a corresponding texture map conversion module to obtain a texture map corresponding to the reconstructed mesh, which is also encoded by a video encoder. Parameters used in the encoding process, such as the type of video encoder used, the type of mesh encoder, transformation parameters, and quantization parameters, are transmitted to the decoding side via auxiliary information.

[0028] The overall framework of the decoding side is shown in Figure 2. For a received bitstream, the decoding side first demultiplexes the bitstreams of each part to obtain base mesh bitstreams, displacement bitstreams, texture map bitstreams, and auxiliary information bitstreams. The base mesh is obtained by decoding the base mesh bitstream using a mesh decoder indicated by the auxiliary information. The displacement bitstreams and texture map bitstreams are decoded by the video decoder. For the displacement part, after decoding the video, the displacement is extracted from the image by a displacement decoding module, which also needs to perform steps such as inverse quantization and inverse transformation. This is then applied to the subdivided base mesh to obtain a reconstructed deformed mesh by the decoding side. The texture map corresponds to the reconstructed deformed mesh after decoding. The application or rendering module then processes the reconstructed deformed mesh and the decoded texture map as input.

[0029] Next, the encoding processing method provided in the embodiments of the present application will be described in detail with reference to the drawings through several embodiments and their application scenarios.

[0030] Referring to FIG. 3, an embodiment of the present application provides an encoding method, as shown in FIG. 3, the encoding method includes: A step 301 of determining a base mesh bitstream based on the mesh to be coded; and step 302 of generating a target bitstream based on the target texture map corresponding to the target mesh and the base mesh bitstream, wherein the target bitstream includes target identification information for indicating a method of obtaining target texture coordinates, and the obtaining method includes obtaining them from the base mesh bitstream or generating them by a decoding side based on mesh parameterization.

[0031] In the embodiment of the present application, the target bitstream carries target identification information, which can explicitly or implicitly indicate how to obtain the target texture coordinates.

[0032] Optionally, in some embodiments, the step of determining the base mesh bitstream based on the mesh to be encoded specifically comprises: performing mesh simplification on the mesh to be encoded to obtain a first mesh; performing a subdivision process and a deformation process on the first mesh to obtain a second mesh by adjusting displacement information and vertex positions of the first mesh; The method may further include a step of performing a compression encoding process on the second mesh to obtain a basic mesh bit stream.

[0033] In the embodiment of the present application, the encoding side can determine the encoding method according to the actual situation. For example, if it is necessary to reduce the transmission of the bitstream, the encoding side may adopt an operation of not encoding texture coordinates, i.e., not performing mesh parameterization on the first mesh before performing subdivision and deformation processing on the first mesh. Otherwise, the encoding side may adopt an encoding method of performing texture coordinate encoding.

[0034] Optionally, if the obtaining method is to obtain from the base mesh bitstream, after obtaining the first mesh, the first mesh may be subjected to mesh parameterization, and the mesh-parameterized first mesh may be used as the input for subdivision processing. The second mesh may be understood as a base mesh containing target texture coordinates. That is, before the step of performing subdivision processing and deformation processing on the first mesh, the method may include: performing mesh parameterization on the first mesh; Here, the input of the subdivision process is a mesh-parameterized first mesh.

[0035] Optionally, if the obtaining method is to generate by the decoding side based on mesh parameterization, the second mesh may be understood as a base mesh that does not contain target texture coordinates.

[0036] In the embodiment of the present application, target identification information is set in the target bitstream to indicate how to obtain the target texture coordinates, so that 3D mesh encoding can be performed in different ways according to different needs, thereby improving the flexibility of 3D mesh encoding.

[0037] It should be noted that before encoding a texture map, texture map transformation based on texture coordinates is required, so if the texture coordinates are not encoded at the encoding side, texture coordinates need to be generated based on mesh parameterization before texture map transformation. In the embodiments of the present application, mesh parameterization can be performed at different positions, and when mesh parameterization is performed at different positions, the corresponding encoding flow is also different, which will be described in detail below.

[0038] Optionally, in some embodiments, the step of generating a target bitstream based on a texture map to be coded corresponding to the mesh to be coded and the base mesh bitstream comprises: performing mesh reconstruction based on the basic mesh bitstream to obtain a first reconstructed mesh; performing mesh parameterization on the first reconstructed mesh to obtain a second reconstructed mesh; blending the base mesh bitstream, the displacement bitstream, the side information bitstream, and the texture map bitstream to obtain the target bitstream; Here, the displacement bitstream is obtained based on displacement information and the second reconstructed mesh, the displacement information is obtained based on the mesh to be encoded, the auxiliary information bitstream is obtained based on auxiliary information for assisting decoding by the decoding side, and the texture map bitstream is obtained based on the first reconstructed mesh, the texture map to be encoded, and the displacement bitstream.

[0039] Optionally, in some embodiments, the method further comprises: adjusting a displacement sequence of the second reconstructed mesh based on the displacement information to obtain a first vertex displacement; performing displacement encoding on the first vertex displacement to obtain a displacement bitstream; reconstructing a deformed mesh based on the first reconstructed mesh and first reconstructed vertex displacements obtained based on displacement reconstruction for the displacement bitstream to obtain a first reconstructed deformed mesh; determining the texture map bitstream based on the first reconstructed mesh, a texture map to be coded, and the first reconstructed deformed mesh.

[0040] In the embodiment of the present application, the encoding side does not encode texture coordinates, but performs mesh parameterization before displacement encoding, that is, the encoding side performs mesh parameterization before reconstruction of the deformed mesh, which can reduce the complexity of mesh parameterization. Correspondingly, the decoding side needs to perform mesh parameterization before subdivision processing to generate target texture coordinates.

[0041] In the embodiment of the present application, the encoding framework on the encoding side specifically includes the following flow, as shown in FIG.

[0042] 1. Mesh simplification Mesh simplification involves simplifying the current input mesh to be coded into a base mesh with a relatively small number of points and faces, while preserving the shape of the original mesh as much as possible. The focus of mesh simplification is the simplification operation and the corresponding error metric. One possible mesh simplification operation, as shown in Figure 5, is to merge the vertices at both ends of an edge into a single vertex and remove the connection between the two vertices. This process is repeated over the entire mesh according to a set rule, until the number of faces and vertices of the mesh is reduced to a target value.

[0043] During the simplification process, an error metric can be selected to optimize the simplification result. For example, the sum of the equation coefficients of all the adjacent faces of a vertex can be selected as the error metric of the vertex, and the error metric of the corresponding edge is the sum of the error metrics of the two vertices on the edge. That is, the error caused by merging an edge is the sum of the distances from the merged vertex to all the faces adjacent to the two original vertices of the edge.

[0044] After the simplification operation and the corresponding error metric are determined, the mesh simplification is performed iteratively. First, the errors of the vertices of the initial mesh are calculated to obtain the error of each edge. Next, the edges are sorted from smallest to largest error, and the edge with the smallest error is selected and merged each time. The position of the merged vertex is also calculated, and the errors of all edges associated with the merged vertex are updated. In other words, the order of the edge array is updated, ensuring that each iteration is based on the overall error metric. The mesh faces are simplified iteratively to the number required for lossy encoding.

[0045] Mesh parameterization mainly involves regenerating the texture coordinates of the simplified mesh to obtain the original mesh. The mesh parameterization algorithm can be specifically configured according to actual needs. For example, the Isocharts algorithm uses spectral analysis to achieve 3D mesh parameterization by stretching, and then UV unwraps the 3D mesh to split it into patches and pack them into 2D texture regions.

[0046] 2. Mesh parameterization Mesh parameterization can be performed at different positions according to different coding frameworks. If texture coordinates are coded on the coding side, coding framework 1 can be used, for example, the coding framework that performs mesh parameterization after mesh simplification in Fig. 4. If texture coordinates are not coded on the coding side, the coding side and the decoding side regenerate the texture coordinates of the mesh in the same mesh parameterization step, i.e., coding framework 2 can be used, for example, the coding framework that performs mesh parameterization after the reconstruction of the base mesh in Fig. 4 (i.e., the step of performing mesh reconstruction based on the base mesh bitstream to obtain the first reconstructed mesh).

[0047] Mesh parameterization primarily involves regenerating the texture coordinates of the simplified mesh to obtain the original mesh. The mesh parameterization algorithm can be tailored to specific needs. For example, the Isocharts algorithm uses spectral analysis to achieve 3D mesh parameterization through stretching, UV unwrapping the 3D mesh, dividing it into patches, and packing it into 2D texture regions. A stretch threshold is set, and the algorithm is summarized as follows: a) Compute a surface spectral analysis to provide an initial parameterization. b) Performing stretch optimization iterations. c) If the stretch of the derived parameterization is below a threshold, stop. d) Perform surface spectral clustering to divide the surface into charts. e) Optimize the chart boundary using the graph cut algorithm. f) Iterate over the charts division until the stretch criteria is met.

[0048] Optionally, surface spectral analysis is parameterized based on the isometric feature mapping (IsoMap) dimensionality reduction method. Given a high-dimensional point cloud, IsoMap computes geodesic distances along the manifold as a sequence of jumps between adjacent points. It then applies a multidimensional scaling (MDS) algorithm to these geodesic distances to find points with similar pairwise distances embedded in a lower-dimensional space. The computational process for a surface given N points is as follows:

[0049] a) a symmetric matrix of squared geodesic distances between surface points

number

[0050] b)

number

number

[0051]

number

[0052] During the ceremony,

number

[0053] c)B N The eigenvalue λ of i and the corresponding eigenvectors

number

[0054] d) For each point \(i\) on the original surface, the embedding into the new space is the \(N\)-dimensional vector

Number

[0055]

Number

[0056] B N The eigenvalues \(\lambda\) of i and the corresponding eigenvectors

Number

[0057] To completely represent a surface with \(N\) vertices, \(N\) eigenvalues are required, but only a small part of their energy usually occupies most of the total energy. Therefore, only calculate \(n (n\ll N)\) largest eigenvalues and the corresponding eigenvectors to generate the \(N\)-dimensional embedding of all points.

[0058] Since the mapping from the high-dimensional space to the low-dimensional space is not equidistant, this parameterization causes distortion. Such distortion of the \(N\)-dimensional embedding can be calculated as the sum of the geodesic distance distortions at all vertices. For each vertex \(i\), define the geodesic distance distortion (GDD) due to the embedding as in Equation 3.

[0059]

number

[0060] During the ceremony,

number

number

[0061] For n=2, surface spectral analysis produces a surface parameterization that minimizes the sum of squares of the GDD over all vertices.

[0062] It should be noted that the Isomap algorithm calculates the geodesic distance along the manifold, but if there are some non-manifolds in the input 3D mesh, this method will perform corresponding preprocessing to eliminate the existence of these non-manifolds.

[0063] Optionally, for stretch optimization, the parameterization introduces distortions because the distances are not equidistant from 3D space to 2D space. Distortion can be measured in various ways, such as preserving angles or areas, or how much the parameter distances are stretched or shrunk across the surface. The algorithm focuses on distance distortion, specifically the definition of geometric stretch, and calculates the average stretch L of local distances on the surface. 2 and worst case stretch L ∞ Consider a triangle T with 2D texture coordinates p1, p2, p3, where p i =(s i ,t i ) and the corresponding three-dimensional coordinates are expressed as q1, q2, q3. The calculation process of affine mapping S(p) = S(s, t) = q is shown in Equation 4.

[0064]

number

[0065] During the ceremony,

number

number

[0066]

number

[0067]

number

[0068] During the ceremony,

number

[0069] Then calculate the Jacobian matrix

number

[0070]

number

[0071]

number

[0072] During the ceremony,

number

number

number

number

number

[0073]

number

[0074]

number

[0075] The entire 3D mesh

number

[0076]

number

[0077]

number

[0078] During the ceremony,

number

number

[0079] L ∞ depends only on one worst-case point in the domain, so L ∞ Stretching is difficult to control, and 2 Several iterations of stretch minimization can significantly improve the results.

[0080] Optionally, for surface spectral clustering, if the parameterization generated by spectral analysis does not meet the stretch threshold, it is split into smaller charts. Since the global characteristics of the model correspond to large eigenvalues, the split is performed using the global characteristics. Using the results of spectral analysis, several representative vertices are calculated, and simultaneously grow charts are centered around these representative points. This method is called surface spectral clustering. The specific algorithm process is as follows:

[0081] a) Arrange the eigenvalues ​​and corresponding eigenvectors from the spectral analysis in order from largest to smallest, i.e., λ1 ≥ λ2 ≥ ... ≥ λ N Let's say.

[0082] b)λ n / λ n+1 Obtain the first N eigenvalues ​​and eigenvectors (n≦10) that maximize

[0083] c) For each vertex i in the 3D mesh, calculate its N-dimensional embedding coordinates;

number

[0084] d) For each of the N embedded coordinates, find the two points with the maximum and minimum coordinates and set them as 2n representative points.

[0085] e) Remove representative points that are too close to each other, generating m≦2n representative points (using a distance threshold of 10 times the average edge length of the input mesh).

[0086] f) Using the geodesic distances calculated by the surface spectral analysis, the 3D mesh is divided into m parts by simultaneously growing charts around the representative points. Each triangle is assigned to the chart with the representative point closest to it (the geodesic distance from a triangle to the representative point is calculated as the average of the geodesic distances from the three vertices of the triangle to the representative vertex).

[0087] Optionally, for boundary optimization, after dividing the charts, we use a graph cut algorithm to optimize the boundaries between the charts. The chart boundaries should satisfy two goals: 1) passing through high-curvature regions without too much roughness, and 2) minimizing distortion of the charts' embedding at the boundaries. This algorithm formulates the boundary optimization problem as a graph cutting problem. For simplicity, we consider a binary division, which divides the surface into two halves. If the subdivision is into two or more charts, we consider each pair of adjacent charts in turn.

[0088] Suppose we find the optimal boundary between chart A and chart B, and the initial partition is generated by surface spectral clustering. Then, we generate an intermediate region C by extending one region on both sides of the initial partition boundary. The size of the intermediate region is proportional to the total area of ​​the patch that has not been peeled. We extend the graph cut algorithm technique to construct an undirected flow network graph from C. Here, the two adjacent triangles f in the graph cut algorithm are i and f j The definition of “capacity” between is modified as in Equation 13.

[0089]

number

[0090] The first term in Equation 13 corresponds to the first goal of cutting without large irregularities along edges with high dihedral angles, and the calculation process is shown in Equation 14.

[0091]

number

[0092] During the ceremony,

number

number

number

[0093] The embedding distortion is measured by the second term of Equation 13, and the calculation process is shown in Equations 15 and 16.

[0094]

number

[0095]

number

[0096] During the ceremony,

number

number

number

number

number

number

[0097] The weight parameters in Equation 13

number

[0098] The simple implementation of this stretch-based chartification and parameterization algorithm is costly, especially as the number of model vertices increases. Therefore, to speed up the calculation, in practical applications, the Iso-charts algorithm uses landmark Isomap, an extension of Isomap. To further reduce embedding distortion, the landmark Isomap algorithm is also used to calculate the embedding coordinates of intermediate region vertices during boundary optimization.

[0099] Finally, the charts generated in the above process are packed into a 2D texture region using the chart packing algorithm used in the MCGIM algorithm, resulting in a 3D mesh of regenerated texture coordinates.

[0100] 3. Subdivision and transformation processing The subdivision process is applied to an input 3D mesh to generate displacement vector information. The input 2D curve (represented by a 2D crease), called the "original" curve, is first downsampled to generate a base curve / crease line, called the "simplified" curve. The subdivision process is then applied to the resulting multi-segment line segments to generate a "subdivision" curve. The resulting multi-segment line segments are then deformed to obtain a better approximation of the original curve. That is, a geometric displacement vector is calculated for each vertex of the subdivision mesh, so that the shape of the subdivision curve is as close as possible to the shape of the original curve. These geometric displacement vectors are the geometric displacement vector information output by the module. A similar deformation process is also applied to the attribute information corresponding to the vertices to obtain the corresponding attribute displacement vectors.

[0101] The subdivision process takes a parameterized mesh as input. In this step, the input mesh is first subdivided. The subdivision scheme can be selected arbitrarily. One possible scheme is the midpoint subdivision scheme, which subdivides each triangle into four subtriangles at each subdivision iteration, as shown in Figure 6. A new vertex is introduced at the center of each edge. Since the connection relationships between geometry information and attribute information are usually different, the subdivision processes for geometry information and attribute information are performed independently.

[0102] Selectable, newly introduced Edge

number

number

number

[0103]

number

[0104] During the ceremony,

number

number

number

number

[0105] For the subdivision processed mesh, the module searches for the nearest neighbors of each point in the original input mesh (including points on the original mesh surface), and data structures such as kdTree can be used to speed up the search. The module calculates the distance between each vertex in the subdivision mesh and the geometric coordinates of the nearest point in the original input mesh to obtain a displacement vector for the geometric coordinates of each vertex of the subdivision mesh. The module transmits the generated displacement vector to subsequent modules for encoding.

[0106] The generated displacement vectors are in the same global coordinate system as the input mesh, and one possible optimization method is to transform them into a local coordinate system, where the local coordinate system of each vertex is defined by the normal vector of the vertex in the subdivision mesh. The advantage of this method is that the normal component of the geometric displacement vector has a greater impact on the quality of the reconstructed mesh than the two tangential components, so a larger quantization parameter can be set for the tangential component.

[0107] 4. Compression coding of basic mesh The base mesh may be understood as the second mesh. Compression coding has two main modes: intraframe mode and interframe mode. In intraframe mode, the base mesh compression module encodes and reconstructs the input 3D mesh using an existing static mesh encoder. In interframe mode, the base mesh compression module calculates motion vectors between the input mesh vertices and the reference frame mesh vertices, then encodes the motion vectors, and reconstructs the base mesh for the current frame based on the reconstructed motion vectors and the reference frame mesh. The base mesh compression module outputs a compressed bitstream and transmits the reconstructed reconstructed base mesh as output to the displacement sequence adjustment module, which adjusts the vertex displacements of the reconstructed base mesh based on the displacement information to obtain a first vertex displacement.

[0108] 5. Wavelet transform A transform can be applied to the displacement vectors to reduce correlation between the data. Selectable transforms include, for example, a linear wavelet transform, and the prediction process is defined as in Equation 18.

[0109]

number

[0110] During the ceremony,

number

number

number

number

number

number

number

number

number

[0111]

number

[0112] During the ceremony,

number

number

[0113] 6. Coefficient quantization The wavelet coefficients, which are transformed displacement vectors, may be quantized. There are various methods for quantization, but one method is shown in Equations 20 and 21.

[0114]

number

[0115]

number

[0116] During the ceremony,

number

number

number

number

number

number

number

number

[0117] Also, depending on the characteristics of the wavelet transform, different quantization parameters may be used for the newly generated vertices by the subdivision process and the original vertices. That is, for the subdivision-processed vertices, the quantization parameters are updated as shown in Equation 7.

[0118]

number

[0119] During the ceremony,

number

[0120] 7. Displacement coding The method may include video coding or quotient coding. Optionally, taking video coding as an example, the displacement coding unit takes quantized wavelet coefficients as input to perform video coding, and the quantized wavelet coefficients need to be arranged into a two-dimensional image. One arrangement method is to traverse the wavelet coefficients in order from low frequency to high frequency.

[0121] Optionally, for each coefficient,

number

[0122] Selectable according to Morton order

number

[0123] It should be understood that the embodiments of the present application are not limited to the ordering scheme, and other ordering schemes may also be used, such as zigzag ordering, raster ordering, etc. The encoder may explicitly specify the corresponding ordering scheme in the bitstream.

[0124] It should be noted that in the embodiments of the present application, displacement processing may be performed, or displacement coding may be performed directly without displacement processing, where displacement processing can improve the compression efficiency of the subsequent displacement coding and reduce the size of the target bitstream.

[0125] 8. Reconstruction of deformed mesh The displacement coding module obtains the reconstructed displacement, that is, obtains the corresponding displacement vector on the decoding side through the methods of inverse quantization and inverse transformation. After obtaining the reconstructed geometric displacement vector, the reconstructed base mesh is subjected to subdivision processing, and a reconstructed subdivision-deformed mesh is obtained according to the corresponding displacement vector, and is transmitted to the texture map conversion module.

[0126] 9. Texture map conversion The texture map conversion module performs texture map conversion based on the input original mesh (i.e., the mesh to be coded), the input original texture map (i.e., the texture map to be coded), and the reconstructed deformed mesh. Specifically, the texture map conversion may include the following steps:

[0127] The texture coordinates of each pixel in the target texture map to be generated are calculated, and for example, the texture coordinates corresponding to pixel A(i,j) are P(u,v).

[0128] It is determined whether the texture coordinate lies within a certain triangular surface after the subdivision transformation and mesh parameterization. If the texture coordinate does not belong to any triangle, the pixel may be marked as a null pixel and then filled with the filling algorithm. If the texture coordinate belongs to one triangle, perform the target operation.

[0129] Here, the target operation may include: Mark the pixel as filled. Based on the texture coordinates, the barycentric coordinates within the current triangular surface are calculated. Based on the barycentric coordinates and the corresponding triangular faces, the two-dimensional texture coordinates are mapped to three-dimensional geometric coordinates, i.e., to the points in the subdivision deformed mesh that correspond to the texture coordinates (denoted as M(x,y,z) in the figure). In the input original mesh, the point closest to the 3D coordinates (in the figure

number

number

number

[0130] 10. Texture map compression After obtaining the transformed texture map, for null pixels therein, a related filling algorithm (e.g., a Push-Pull algorithm) can be used to fill these null pixels. Then, the transformed texture map can be encoded using an existing video encoder such as H.264 / AVC, H.265 / HEVC, or H.266 / VVC to obtain an output texture map bitstream. Also, operations such as color space conversion (e.g., from RGB 444 to YUV 420) and chroma subsampling can be optionally used to improve the rate-distortion performance of the video encoding.

[0131] Optionally, in some embodiments, the step of generating a target bitstream based on a texture map to be coded corresponding to the mesh to be coded and the base mesh bitstream comprises: performing mesh reconstruction based on the basic mesh bitstream to obtain a first reconstructed mesh; reconstructing a deformed mesh based on the first reconstructed mesh and second reconstructed vertex displacements obtained based on displacement reconstruction for the displacement bitstream to obtain a second reconstructed deformed mesh; performing mesh parameterization on the second reconstructed deformed mesh to obtain a third reconstructed deformed mesh; blending the base mesh bitstream, the displacement bitstream, the side information bitstream, and the texture map bitstream to obtain the target bitstream; The displacement bitstream is obtained based on displacement information and the first reconstructed mesh, the displacement information is obtained based on the mesh to be encoded, the auxiliary information bitstream is obtained based on auxiliary information for assisting decoding by the decoding side, and the texture map bitstream is obtained based on the first reconstructed mesh, the texture map to be encoded, and the third reconstructed deformed mesh.

[0132] Optionally, in some embodiments, the method further comprises: adjusting a displacement sequence of the first reconstructed mesh based on the displacement information to obtain second vertex displacements; and performing displacement encoding on the second vertex displacement to obtain a displacement bitstream.

[0133] In the embodiment of the present application, the coding framework on the encoding side is specifically shown in FIG. 7. Here, FIG. 7 can be understood as two coding frameworks due to the different positions of mesh parameterization. If texture coordinates are coded on the encoding side, coding framework 1 can be used, for example, the coding framework in which mesh parameterization is performed after mesh simplification in FIG. 7. If texture coordinates are not coded on the encoding side, the coding side and the decoding side regenerate the texture coordinates of the mesh using the same mesh parameterization steps, i.e., coding framework 3 can be used, for example, the coding framework in which mesh parameterization is performed after the deformed mesh reconstruction in FIG. 7 (i.e., the deformed mesh is reconstructed based on the first reconstructed mesh and the second reconstructed vertex displacements to obtain the second reconstructed deformed mesh). In the embodiment of the present application, the implementation flow of coding framework 3 is similar to that of coding framework 2, with only the position of mesh parameterization being different. For the specific implementation flow, please refer to the above embodiment, and a detailed description will be omitted here.

[0134] It should be noted that the setting position of the target identification information in the target bitstream can be set according to actual needs, for example, in some embodiments, the target identification information is located in an auxiliary information bitstream in the target bitstream, and the auxiliary information bitstream is obtained based on the auxiliary information, that is, the target bitstream includes auxiliary information, and the auxiliary information includes the target identification information.

[0135] Optionally, in some embodiments, the target identification information is: First field information for indicating the number of the target texture coordinates; second field information for indicating whether the target texture coordinates are generated by the decoding side based on mesh parameterization; and third field information for indicating index information of a method for generating the target texture coordinates.

[0136] Optionally, in some embodiments, the target identification information is information for indicating the functions and algorithms that the decoding side needs to support. In the embodiments of the present application, the target identification information can be understood as profile identification (ID), that is, different profile IDs can implicitly indicate how to obtain texture coordinates.

[0137] In an embodiment of the present application, based on the V3C syntax structure, a V3C parameter set for specifying parameters common to sequences may be extended, and parameters indicating a texture coordinate acquisition method may be defined in the parameter set, specifically as shown in Table 1 below.

[0138] [Table 1]

[0139] Here, asps_vmc_ext_num_attribute_video represents the number of attribute maps corresponding to the mesh that references the parameter set, that is, the number of texture coordinates corresponding to the mesh.

[0140] asps_vmc_ext_derive_textcoord_flag[i] indicates whether the texture coordinates of the i-th attribute map corresponding to the mesh that references the parameter set are generated from the decoding side. If it is 0, it indicates that the texture coordinates corresponding to the attribute are transmitted through the bitstream, and if it is 1, it indicates that the texture coordinates corresponding to the attribute are regenerated on the decoding side.

[0141] asps_vmc_ext_derive_textcoord_method[i] represents the method index used when the texture coordinates of the ith attribute map corresponding to the mesh that references the parameter set are regenerated. The decoding side obtains the corresponding method according to the corresponding index from the supported method list, and then obtains the corresponding parameters based on the corresponding method index.

[0142] In the embodiment of the present application, asps_vmc_ext_num_attribute_video may be understood as the first field information, asps_vmc_ext_derive_textcoord_flag[i] may be understood as the second field information, and asps_vmc_ext_derive_textcoord_method[i] may be understood as the third field information.

[0143] In addition to specifying parameters related to the method for obtaining texture coordinates in a parameter set, a corresponding profile may be set to support different encoding / decoding algorithms and tools. Specifically, the different methods for processing texture coordinates supported by different profiles may be specified in the profile. Table 2 shows an example of one profile. Here, two profiles are defined, and different encoding parameters are specified in the two profiles to restrict the corresponding functions.

[0144] [Table 2]

[0145] Here, ptl_profile_toolset_idc is defined in profile_tier_level() of the V3C parameter set and represents the decoding tools supported by the bitstream that references the V3C parameter set. Taking the above table as an example, if the value is 0, it means that asps_vmc_ext_derive_textcoord_flag must be 0 for all attributes, and if the value is 1, it means that asps_vmc_ext_derive_textcoord_flag may be 1.

[0146] Optionally, in some embodiments, the above coding framework 1, coding framework 2, and coding framework 3 may be combined, in which case, in addition to the above acquisition method, position information of mesh parameterization may be further indicated. For example, in some embodiments, the target identification information further includes fourth field information for indicating position information where a decoding side performs the mesh parameterization.

[0147] In an embodiment of the present application, the target identification information is in the auxiliary information bitstream. The fourth field information can explicitly or implicitly indicate the position information where the decoding side performs the mesh parameterization. Here, the fourth field information for indicating the position information where the decoding side performs the mesh parameterization may be understood or replaced with the fourth field information for indicating the position information where the encoding side performs the mesh parameterization. For example, an optional syntax structure may be as shown in Table 3.

[0148] [Table 3]

[0149] Here, if the value of asps_vmc_ext_derive_tex_from_base_flag[i] is 1, when the decoding side regenerates the texture coordinates of the attribute map with index i, mesh parameterization is performed based on the reconstructed basic mesh, and if the value is 0, mesh parameterization is performed based on the reconstructed deformed mesh.

[0150] In the embodiment of the present application, the asps_vmc_ext_derive_tex_from_base_flag[i] may be understood as the fourth field information.

[0151] Optionally, in some embodiments, the target identification information is further used to indicate location information at which the decoding side performs the mesh parameterization.

[0152] In the embodiment of the present application, the target identification information may be a profile identifier (ID), that is, the target identification information is carried in the header field of the target bitstream. For example, three different profiles may be defined, and different syntax parameter values ​​may be specified for each profile to restrict the capabilities of the decoding side. Specifically, as shown in Table 4:

[0153] [Table 4]

[0154] Here, in a profile corresponding to Extended VDMC (VDMC Extended), asps_vmc_ext_derive_tex_from_base_flag must be 1, and the corresponding function is that the profile only supports parameterization of the base mesh reconstructed on the decoding side. The function corresponding to High VDMC (VDMC High) indicates that the deformed mesh reconstructed on the decoding side can be parameterized.

[0155] Optionally, the embodiment of the present application further provides a decoding processing method, as shown in FIG. 8, the decoding processing method includes: Step 801: receiving a target bitstream, the target bitstream including a base mesh bitstream and target identification information for indicating how to obtain target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; Step 802: decoding the base mesh bitstream to obtain a third mesh; Step 803 of generating a target decoded mesh based on the third mesh and the acquisition method; Here, the target decoded mesh includes the target texture coordinates.

[0156] In the embodiment of the present application, the target identification information can explicitly or implicitly indicate the acquisition method. Since the acquisition method is indicated by the target identification information, the decoding side can use the corresponding acquisition method to perform mesh decoding, thus improving the flexibility of mesh decoding.

[0157] Optionally, in some embodiments, when the obtaining method is obtaining texture coordinates from the base mesh bitstream, the third mesh includes the target texture coordinates, and the step of generating a target decoded mesh based on the third mesh and the obtaining method includes: subdividing the third mesh to obtain a third subdivided mesh; reconstructing a deformed mesh based on the third subdivision mesh and the third vertex displacements to obtain the target decoded mesh; Here, the third vertex displacement is obtained based on a displacement bitstream in the target bitstream.

[0158] Optionally, when the obtaining method is to generate by a decoding side based on mesh parameterization, the step of generating a target decoded mesh based on the third mesh and the obtaining method comprises: performing mesh parameterization on the third mesh to obtain a fourth mesh, the fourth mesh including the target texture coordinates; subdividing the fourth mesh to obtain a first subdivision mesh; reconstructing a deformed mesh based on the first subdivision mesh and the third vertex displacements to obtain the target decoded mesh; Here, the third vertex displacement is obtained based on a displacement bitstream in the target bitstream.

[0159] Optionally, said step of performing mesh parameterization on said third mesh to obtain a fourth mesh comprises: If the target identification information instructs the decoding side to perform the mesh parameterization before performing subdivision processing, the method includes a step of performing mesh parameterization on the third mesh to obtain a fourth mesh.

[0160] Optionally, when the obtaining method is to generate by a decoding side based on mesh parameterization, the step of generating a target decoded mesh based on the third mesh and the obtaining method comprises: subdividing the third mesh to obtain a second subdivision mesh; reconstructing a deformed mesh based on the second subdivision mesh and the third vertex displacements to obtain a fourth reconstructed deformed mesh; performing mesh parameterization on the fourth reconstructed deformed mesh to obtain the target decoded mesh; Here, the third vertex displacement is obtained based on a displacement bitstream in the target bitstream.

[0161] Optionally, the step of performing mesh parameterization on the fourth reconstructed deformed mesh to obtain the target decoded mesh comprises: If the target identification information instructs the decoding side to perform the mesh parameterization after reconstructing the deformed mesh, the method includes a step of performing mesh parameterization on the fourth reconstructed deformed mesh to obtain the target decoded mesh.

[0162] Optionally, the target identification information is located in a supplementary information bitstream in the target bitstream, and the supplementary information bitstream is derived based on supplementary information.

[0163] Optionally, the target identification information is: First field information for indicating the number of the target texture coordinates; second field information for indicating whether the target texture coordinates are generated by the decoding side based on mesh parameterization; and third field information for indicating index information of a method for generating the target texture coordinates.

[0164] Optionally, the target identification information further includes fourth field information for indicating position information where a decoding side performs the mesh parameterization.

[0165] Optionally, the target identification information is information for indicating the functions and algorithms that the decoding side needs to support.

[0166] Optionally, the target identification information is further used to indicate location information at which the decoding side performs the mesh parameterization.

[0167] It should be noted that in the embodiments of the present application, the decoding framework of the decoding side will be different depending on the position where the decoding side performs mesh parameterization, and may include, for example, the decoding frameworks of Figures 9 and 10. The decoding flow will be described in detail below using Figure 9 as an example.

[0168] 1. Decoding auxiliary information The decoding side first determines a decoding method based on the auxiliary information. This mainly includes the identification of the texture coordinate acquisition method, the static mesh encoder type, the video encoder type, and the subdivision processing method. The identification of the texture coordinate acquisition method, i.e., whether the texture coordinates are regenerated on the decoding side, may optionally include the location where the texture coordinates are generated. The static mesh encoder type instructs the decoding side to use the corresponding static mesh decoder. The video encoder type instructs the decoding side to use the corresponding video decoder. The subdivision processing method, i.e., the subdivision processing method for the base mesh of the reconstructed deformed mesh, should be the same between the encoding side and the decoding side. Optionally, there are also airspace displacement conversion methods, coefficient array methods, etc.

[0169] 2. Decoding the basic mesh The decoding of the base mesh can be divided into intra-frame mode and inter-frame mode. In the intra-frame mode, the module decodes the input base mesh bitstream through a decoder corresponding to the static mesh encoder indicated by the auxiliary information, and the output of the decoding is a 3D mesh, including geometric information, connectivity, texture coordinate information, etc. In the inter-frame mode, the module decodes the motion vector corresponding to the vertex, and then reconstructs the base mesh of the current frame based on the reference frame.

[0170] 3. Mesh parameterization If the auxiliary information indicates that the texture coordinates are to be regenerated at the decoding side, the module parameterizes the input mesh based on the corresponding parameters. The parameterization operation is consistent with that at the encoding side. The auxiliary information may also indicate the location of the module and whether the module parameterizes the reconstructed base mesh or the reconstructed deformed mesh.

[0171] 4. Displacement decoding During the disparity decoding process, the corresponding video encoder needs to be determined based on the auxiliary information identification, and the decoding side calls the corresponding video decoder to decode the disparity bitstream. For the decoded disparity information, the displacement vector corresponding to the subdivision mesh vertex reconstructed by the displacement reconstruction module needs to be obtained. The operation of the displacement reconstruction module mainly involves performing inverse quantization and inverse transformation on the decoded disparity information, i.e., the wavelet coefficients, according to the quantization parameters and transformation parameters indicated by the auxiliary information. For the video decoding information, the corresponding disparity information needs to be extracted from the 2D image based on the encoding side's arrangement method.

[0172] 5. Subdivision processing The subdivision process is the same as that on the encoding side, and the subdivision method of the basic mesh, the number of iterations, etc. are specified by auxiliary information.

[0173] 6. Reconstruction of deformed mesh After the decoding and reconstruction of the base mesh and displacement vectors are completed, the deformed mesh is reconstructed based on these two parts. For each vertex of the subdivision processed mesh, the corresponding displacement vector is added, i.e., as shown in Equation 23.

[0174]

number

[0175] During the ceremony,

number

number

number

number

number

number

number

[0176] 7. Decoding the displacement map The texture map decoder decodes the texture map bitstream, which is then decoded by a video decoder indicated by the side information, and optionally performs color space conversion to obtain an image format that matches the texture map input from the encoding side, and finally obtains the decoded and output texture map.

[0177] When the above flow is completed, the target decoded mesh reconstructed by the decoding side and the corresponding attribute map are finally input and the corresponding processing is performed.

[0178] The encoding method provided in the embodiments of the present application may be executed by an encoding processing device. In the embodiments of the present application, the encoding processing device provided in the embodiments of the present application will be described as an example in which the encoding processing method is executed by an encoding processing device.

[0179] Referring to FIG. 11, an embodiment of the present application further provides an encoding processing device, as shown in FIG. 11, the encoding processing device 1100 includes: a processing module 1101 for determining a base mesh bitstream based on a mesh to be coded; a generation module 1102 that generates a target bitstream based on a target texture map corresponding to the target mesh and the base mesh bitstream, wherein the target bitstream includes target identification information for indicating a method of obtaining target texture coordinates, and the obtaining method includes obtaining them from the base mesh bitstream or generating them by a decoding side based on mesh parameterization.

[0180] Optionally, the processing module 1101 is specifically used to perform mesh simplification on a mesh to be encoded to obtain a first mesh, perform subdivision processing and deformation processing on the first mesh to obtain a second mesh by adjusting the displacement information and vertex positions of the first mesh, and perform compression encoding processing on the second mesh to obtain a basic mesh bit stream.

[0181] Optionally, the generating module 1104: a first reconstruction unit for performing mesh reconstruction based on the basic mesh bitstream to obtain a first reconstructed mesh; a first processing unit for performing mesh parameterization on the first reconstructed mesh to obtain a second reconstructed mesh; a first mixing unit for mixing the base mesh bitstream, the displacement bitstream, the auxiliary information bitstream, and the texture map bitstream to obtain the target bitstream; Here, the displacement bitstream is obtained based on displacement information and the second reconstructed mesh, the displacement information is obtained based on the mesh to be encoded, the auxiliary information bitstream is obtained based on auxiliary information for assisting decoding by the decoding side, and the texture map bitstream is obtained based on the first reconstructed mesh, the texture map to be encoded, and the displacement bitstream.

[0182] Optionally, the generating module 1104: a first adjusting unit for adjusting a displacement sequence of the second reconstructed mesh based on the displacement information to obtain a first vertex displacement; an encoding unit for performing displacement encoding on the first vertex displacement to obtain a displacement bitstream; a second reconstruction unit for reconstructing a deformed mesh based on the first reconstructed mesh and first reconstructed vertex displacements obtained based on displacement reconstruction for the displacement bitstream, to obtain a first reconstructed deformed mesh; and a second processing unit for determining the texture map bitstream based on the first reconstructed mesh, the texture map to be coded, and the first reconstructed deformed mesh.

[0183] Optionally, the generating module 1104: a third reconstruction unit for performing mesh reconstruction based on the basic mesh bitstream to obtain a first reconstructed mesh; a fourth reconstruction unit for reconstructing a deformed mesh based on the first reconstructed mesh and a second reconstructed vertex displacement obtained based on displacement reconstruction for the displacement bit stream, to obtain a second reconstructed deformed mesh; a third processing unit for performing mesh parameterization on the second reconstructed deformed mesh to obtain a third reconstructed deformed mesh; a second mixing unit for mixing the base mesh bitstream, the displacement bitstream, the auxiliary information bitstream, and the texture map bitstream to obtain the target bitstream; Here, the displacement bitstream is obtained based on displacement information and the first reconstructed mesh, the displacement information is obtained based on the mesh to be encoded, the auxiliary information bitstream is obtained based on auxiliary information for assisting decoding by the decoding side, and the texture map bitstream is obtained based on the first reconstructed mesh, the texture map to be encoded, and the third reconstructed deformed mesh.

[0184] Optionally, the generation module 1104 further includes a fourth processing unit for adjusting the displacement sequence of the first reconstructed mesh based on the displacement information to obtain a second vertex displacement, performing displacement encoding on the second vertex displacement, and obtaining a displacement bitstream.

[0185] Optionally, the processing module 1101 is further adapted to perform mesh parameterization on the first mesh; Here, the input of the subdivision process is a mesh-parameterized first mesh.

[0186] Optionally, the target identification information is located in a supplementary information bitstream in the target bitstream, and the supplementary information bitstream is derived based on supplementary information.

[0187] Optionally, the target identification information is: First field information for indicating the number of the target texture coordinates; second field information for indicating whether the target texture coordinates are generated by the decoding side based on mesh parameterization; and third field information for indicating index information of a method for generating the target texture coordinates.

[0188] Optionally, the target identification information further includes fourth field information for indicating position information where a decoding side performs the mesh parameterization.

[0189] Optionally, the target identification information is information for indicating the functions and algorithms that the decoding side needs to support.

[0190] Optionally, the target identification information is further used to indicate location information at which the decoding side performs the mesh parameterization.

[0191] The decoding method provided in the embodiments of the present application may be executed by a decoding processing device. In the embodiments of the present application, the decoding processing device provided in the embodiments of the present application will be described as an example in which the decoding processing method is executed by a decoding processing device.

[0192] Referring to FIG. 12, an embodiment of the present application further provides a decoding processing device, and as shown in FIG. 12, the decoding processing device 1200 includes: a receiving module 1201 for receiving a target bitstream, the target bitstream including a base mesh bitstream and target identification information for indicating a method of obtaining target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; a decoding module 1202 for decoding the base mesh bitstream to obtain a third mesh, and generating a target decoded mesh based on the third mesh and the obtaining method; Here, the target decoded mesh includes the target texture coordinates.

[0193] Optionally, if the acquisition method is to acquire texture coordinates from the base mesh bitstream, the third mesh includes the target texture coordinates, and the decoding module 1002 is specifically used to subdivide the third mesh to obtain a third subdivision mesh, and reconstruct a deformed mesh based on the third subdivision mesh and third vertex displacements to obtain the target decoded mesh, where the third vertex displacements are obtained based on the displacement bitstream.

[0194] Optionally, if the acquisition method is to generate by the decoding side based on mesh parameterization, the decoding module 1002 is specifically used to perform mesh parameterization on the third mesh to obtain a fourth mesh including the target texture coordinates, perform subdivision processing on the fourth mesh to obtain a first subdivision mesh, and reconstruct a deformed mesh based on the first subdivision mesh and third vertex displacements to obtain the target decoded mesh, where the third vertex displacements are obtained based on the displacement bitstream.

[0195] Optionally, the decoding module 1202 is specifically used to perform mesh parameterization on the third mesh to obtain a fourth mesh if the target identification information instructs the decoding side to perform the mesh parameterization before performing the subdivision process.

[0196] Optionally, if the acquisition method is to generate by the decoding side based on mesh parameterization, the decoding module 1002 is specifically used to subdivide the third mesh to obtain a second subdivision mesh, reconstruct a deformed mesh based on the second subdivision mesh and a third vertex displacement, obtain a fourth reconstructed deformed mesh, perform mesh parameterization on the fourth reconstructed deformed mesh, and obtain the target decoded mesh, wherein the third vertex displacement is obtained based on the displacement bitstream.

[0197] Optionally, the decoding module 1202 is specifically used to perform mesh parameterization on the fourth reconstructed deformed mesh to obtain the target decoded mesh when the target identification information instructs the decoding side to perform the mesh parameterization after reconstructing the deformed mesh.

[0198] Optionally, the target identification information is located in a supplementary information bitstream in the target bitstream, and the supplementary information bitstream is derived based on supplementary information.

[0199] Optionally, the target identification information is: First field information for indicating the number of the target texture coordinates; second field information for indicating whether the target texture coordinates are generated by the decoding side based on mesh parameterization; and third field information for indicating index information of a method for generating the target texture coordinates.

[0200] Optionally, the target identification information further includes fourth field information for indicating position information where a decoding side performs the mesh parameterization.

[0201] Optionally, the target identification information is information for indicating the functions and algorithms that the decoding side needs to support.

[0202] Optionally, the target identification information is further used to indicate location information at which the decoding side performs the mesh parameterization.

[0203] The encoding processing device and decoding processing device provided in the embodiments of the present application can realize each process realized in the method embodiments of Figures 3 to 10 and achieve similar technical effects, and detailed descriptions thereof will be omitted here to avoid repetition.

[0204] Optionally, as shown in Fig. 13, an embodiment of the present application further provides a communication device 1300 including a processor 1301 and a memory 1302 storing a program or command executable by the processor 1301. For example, when the communication device 1300 is an encoding device, the program or command is executed by the processor 1301 to realize each step of the embodiment of the encoding method, thereby achieving the same technical effect. When the communication device 1300 is a decoding device, the program or command is executed by the processor 1301 to realize each step of the embodiment of the decoding method, thereby achieving the same technical effect. In order to avoid repetition, detailed description will be omitted here.

[0205] An embodiment of the present application further provides an electronic device comprising a processor and a communication interface, When the electronic device is an encoding side, the processor is used to determine a base mesh bitstream based on a mesh to be encoded, and generate a target bitstream based on a texture map to be encoded corresponding to the mesh to be encoded and the base mesh bitstream, the target bitstream includes target identification information for indicating a method of obtaining target texture coordinates, and the obtaining method includes obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; When the electronic device is a decoding side, the communication interface is used to receive a target bitstream including a base mesh bitstream and target identification information for instructing a method for obtaining target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by the decoding side based on mesh parameterization, and the processor is used to decode the base mesh bitstream to obtain a third mesh and generate a target decoded mesh based on the third mesh and the obtaining method, wherein the target decoded mesh includes the target texture coordinates.

[0206] The embodiment of the electronic device corresponds to the method embodiment on the encoding / decoding side device side, and the respective implementation steps and realization forms of the above method embodiments can be applied to the embodiment of the electronic device, and the same technical effects can be achieved. Specifically, Figure 14 is a schematic diagram of the hardware structure of the electronic device implementing the embodiment of the present application.

[0207] The electronic device 1400 includes at least some components such as, but not limited to, a radio frequency unit 1401, a network module 1402, an audio output unit 1403, an input unit 1404, a sensor 1405, a display unit 1406, a user input unit 1407, an interface unit 1408, a memory 1409, and a processor 1410.

[0208] As will be understood by those skilled in the art, the electronic device 1400 may further include a power source (e.g., a battery) that supplies power to each component, and the power source may be logically connected to the processor 1410 via a power management system, which may further realize functions such as charge / discharge management and power consumption management. The configuration of the electronic device shown in Figure 14 is not intended to limit the electronic device, and the electronic device may include more or fewer components than those shown, or a combination of some components, or a different component arrangement, and detailed description thereof will be omitted here.

[0209] It should be understood that in an embodiment of the present application, the input unit 1404 may include a graphics processing unit (GPU) 14041 and a microphone 14042 for processing image data of still or video images captured by an image capture device (e.g., a camera) in a video capture mode or an image capture mode. The display unit 1406 may include a display panel 14061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1407 includes at least one of a touch panel 14071 and other input devices 14072. The touch panel 14071 is also called a touch screen. The touch panel 14071 may include two parts: a touch detection device and a touch controller. The other input devices 14072 may include, but are not limited to, a physical keyboard, function buttons (e.g., volume control buttons, switch buttons, etc.), a trackball, a mouse, and a control lever. Detailed descriptions thereof are omitted here.

[0210] In the embodiment of the present application, the radio frequency unit 1401 can receive downlink data from the network side device and then transmit the data to the processor 1410 for processing. The radio frequency unit 1401 can also transmit uplink data to the network side device. Typically, the radio frequency unit 1401 includes, but is not limited to, an antenna, an amplifier, a receiver / transmitter, a coupler, a low-noise amplifier, a duplexer, etc.

[0211] The memory 1409 can be used to store software programs or commands and various data. The memory 1409 may mainly include a first storage area for storing programs or commands, and a second storage area for storing data, which can store an operating system, an application or command required for at least one function (e.g., an audio playback function, an image playback function, etc.). The memory 1409 may include volatile memory or nonvolatile memory, or may include both volatile memory and nonvolatile memory. Here, the nonvolatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synch link dynamic random access memory (SLDRAM), and direct Rambus random access memory (DRRAM). Memory 1409 in embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.

[0212] The processor 1410 may include one or more processing units, and optionally integrates an application processor that mainly processes operations related to an operating system, a user interface, and applications, and a modem processor that mainly processes wireless communication signals, such as a baseband processor, into the processor 1410. It is understood that the modem processor may not be integrated into the processor 1410.

[0213] Wherein, when the electronic device is an encoding side, the processor 1410 is used to determine a base mesh bitstream based on a mesh to be encoded, and generate a destination bitstream based on a texture map to be encoded corresponding to the mesh to be encoded and the base mesh bitstream, the destination bitstream includes destination identification information for indicating a method of obtaining destination texture coordinates, and the obtaining method includes obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; When the electronic device is a decoding side, the high frequency unit 1401 is used to receive a target bitstream including a base mesh bitstream and target identification information for instructing a method for obtaining target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by the decoding side based on mesh parameterization, and the processor 1410 is used to decode the base mesh bitstream to obtain a third mesh and generate a target decoded mesh based on the third mesh and the obtaining method, wherein the target decoded mesh includes the target texture coordinates.

[0214] An embodiment of the present application further provides a readable storage medium in which a program or command is stored, and when the program or command is executed by a processor, each step of the embodiment of the encoding processing method or decoding processing method described above is realized, and similar technical effects can be achieved. In order to avoid repetition, detailed description will be omitted here.

[0215] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0216] An embodiment of the present application further provides a chip including a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor executes a program or command to realize each step of the embodiment of the encoding processing method or decoding processing method described above, thereby achieving similar technical effects. Detailed description will be omitted here to avoid repetition.

[0217] It should be understood that the chips referred to in the embodiments of this application may also be referred to as system level chips, system chips, chip systems, or system-on-chips, etc.

[0218] The embodiments of the present application further provide a computer program / program product that is stored in a storage medium and is executed by at least one processor to realize each step of the embodiments of the encoding processing method or decoding processing method described above, and can achieve similar technical effects. Detailed descriptions thereof will be omitted here to avoid repetition.

[0219] The embodiments of the present application further provide a video encoding / decoding system that includes an encoding side device for performing each step of each method embodiment of the encoding side device in Figure 3 and the above, and a decoding side device for performing each step of each method embodiment of the decoding side device in Figure 8 and the above, and can achieve similar technical effects, and detailed descriptions thereof will be omitted here to avoid repetition.

[0220] It should be noted that, as used herein, terms such as "comprises," "consists of," or any other variation thereof are intended to include a non-exclusive inclusion, such that a process, method, article, or apparatus comprising a set of elements includes not only those elements but also other elements not expressly specified or inherent in such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element. It should also be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may include performing functions substantially simultaneously or in the reverse order, depending on such functionality. For example, the described method may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to one example may be combined in other examples.

[0221] From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be realized in the form of a combination of software and a necessary common hardware platform, and of course, they can also be realized by hardware, but in many cases the former is a more preferred embodiment. Based on this view, the technical solutions of the present application can be substantially embodied in the form of a software product, and the computer software product is stored in a storage medium (e.g., ROM / RAM, magnetic disk, optical disk) and includes a plurality of commands that cause a terminal (which may be a mobile phone, a computer, a server, a network device, etc.) to execute the methods described in each embodiment of the present application.

[0222] Although the examples of the present application have been described above with reference to the drawings, the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative and not limiting. Based on the suggestions of the present application, many forms that a person skilled in the art can make without departing from the spirit of the present application and the scope of protection of the claims are all within the scope of protection of the present application.

Claims

1. An encoding processing method applied to an encoding side, determining a base mesh bitstream based on a mesh to be encoded; generating a target bitstream based on a texture map to be coded corresponding to the mesh to be coded and the base mesh bitstream, wherein the target bitstream includes target identification information for indicating a method of obtaining target texture coordinates, and the obtaining method includes obtaining them from the base mesh bitstream or generating them by a decoding side based on mesh parameterization.

2. The step of determining a base mesh bitstream based on a mesh to be encoded comprises: performing mesh simplification on the mesh to be encoded to obtain a first mesh; performing a subdivision process and a deformation process on the first mesh to obtain a second mesh by adjusting displacement information and vertex positions of the first mesh; and performing a compression encoding process on the second mesh to obtain a base mesh bitstream.

3. generating a target bitstream based on a texture map to be coded corresponding to the mesh to be coded and the base mesh bitstream, performing mesh reconstruction based on the basic mesh bitstream to obtain a first reconstructed mesh; performing mesh parameterization on the first reconstructed mesh to obtain a second reconstructed mesh; blending the base mesh bitstream, the displacement bitstream, the side information bitstream, and the texture map bitstream to obtain the target bitstream; 3. The method of claim 1 or 2, wherein the displacement bitstream is obtained based on displacement information and the second reconstructed mesh, the displacement information is obtained based on the mesh to be encoded, the auxiliary information bitstream is obtained based on auxiliary information for assisting decoding by a decoding side, and the texture map bitstream is obtained based on the first reconstructed mesh, the texture map to be encoded, and the displacement bitstream.

4. generating a target bitstream based on a texture map to be coded corresponding to the mesh to be coded and the base mesh bitstream, performing mesh reconstruction based on the basic mesh bitstream to obtain a first reconstructed mesh; reconstructing a deformed mesh based on the first reconstructed mesh and second reconstructed vertex displacements obtained based on displacement reconstruction for the displacement bitstream to obtain a second reconstructed deformed mesh; performing mesh parameterization on the second reconstructed deformed mesh to obtain a third reconstructed deformed mesh; blending the base mesh bitstream, the displacement bitstream, the side information bitstream, and the texture map bitstream to obtain the target bitstream; 3. The method of claim 1 or 2, wherein the displacement bitstream is obtained based on displacement information and the first reconstructed mesh, the displacement information is obtained based on the mesh to be encoded, the auxiliary information bitstream is obtained based on auxiliary information for assisting decoding by a decoding side, and the texture map bitstream is obtained based on the first reconstructed mesh, the texture map to be encoded, and the third reconstructed deformed mesh.

5. The method according to claim 1 , wherein the target identification information is located in a supplementary information bitstream in the target bitstream, and the supplementary information bitstream is obtained based on supplementary information.

6. The target identification information is first field information for indicating the number of the target texture coordinates; second field information for indicating whether the target texture coordinates are generated by the decoding side based on mesh parameterization; and third field information for indicating index information of a method for generating the target texture coordinates.

7. The method of claim 6 , wherein the target identification information further includes fourth field information for indicating position information at which a decoding side performs the mesh parameterization.

8. The method according to claim 1 , wherein the target identification information is information for indicating functions and algorithms that a decoding side needs to support.

9. The method according to claim 1 , wherein the target identification information is further used to indicate position information at which a decoding side performs the mesh parameterization.

10. A decoding processing method applied to a decoding side, receiving a target bitstream, the target bitstream including a base mesh bitstream and target identification information for indicating how to obtain target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; decoding the base mesh bitstream to obtain a third mesh; generating a target decoded mesh based on the third mesh and the acquisition method; The decoding method, wherein the target decoded mesh includes the target texture coordinates.

11. When the acquisition method is to generate a target decoded mesh by a decoding side based on mesh parameterization, the step of generating a target decoded mesh based on the third mesh and the acquisition method includes: performing mesh parameterization on the third mesh to obtain a fourth mesh, the fourth mesh including the target texture coordinates; subdividing the fourth mesh to obtain a first subdivision mesh; reconstructing a deformed mesh based on the first subdivision mesh and the third vertex displacements to obtain the target decoded mesh; The method of claim 10 , wherein the third vertex displacement is obtained based on a displacement bitstream in the target bitstream.

12. said step of performing mesh parameterization on said third mesh to obtain a fourth mesh comprises: The method of claim 11, further comprising the step of performing mesh parameterization on the third mesh to obtain a fourth mesh if the target identification information instructs the decoding side to perform the mesh parameterization before performing subdivision processing.

13. When the acquisition method is to generate a target decoded mesh by a decoding side based on mesh parameterization, the step of generating a target decoded mesh based on the third mesh and the acquisition method includes: subdividing the third mesh to obtain a second subdivision mesh; reconstructing a deformed mesh based on the second subdivision mesh and the third vertex displacements to obtain a fourth reconstructed deformed mesh; performing mesh parameterization on the fourth reconstructed deformed mesh to obtain the target decoded mesh; The method of claim 10 , wherein the third vertex displacement is obtained based on a displacement bitstream in the target bitstream.

14. the step of performing mesh parameterization on the fourth reconstructed deformed mesh to obtain the target decoded mesh comprises:

14. The method of claim 13, further comprising, if the target identification information instructs the decoding side to perform the mesh parameterization after reconstructing a deformed mesh, performing mesh parameterization on the fourth reconstructed deformed mesh to obtain the target decoded mesh.

15. 15. The method according to claim 10, wherein the target identification information is located in a supplementary information bitstream in the target bitstream, and the supplementary information bitstream is obtained based on supplementary information.

16. The target identification information is first field information for indicating the number of the target texture coordinates; second field information for indicating whether the target texture coordinates are generated by the decoding side based on mesh parameterization; and third field information for indicating index information of a method for generating the target texture coordinates.

17. The method of claim 16 , wherein the target identification information further includes fourth field information for indicating position information at which a decoding side performs the mesh parameterization.

18. The method according to claim 10 , wherein the target identification information is information for indicating functions and algorithms that a decoding side needs to support.

19. 19. The method according to any one of claims 10 to 15 and 18, wherein the target identification information is further used to indicate position information at which a decoding side performs the mesh parameterization.

20. An encoding processing device applied to an encoding side, a processing module for determining a base mesh bitstream based on a mesh to be encoded; a generation module for generating a target bitstream based on a texture map to be coded corresponding to the mesh to be coded and the base mesh bitstream, wherein the target bitstream includes target identification information for indicating a method of obtaining target texture coordinates, and the obtaining method includes obtaining them from the base mesh bitstream or generating them by a decoding side based on mesh parameterization.

21. A decoding processing device applied to a decoding side, a receiving module for receiving a target bitstream, the target bitstream including a base mesh bitstream and target identification information for indicating a method of obtaining target texture coordinates, the obtaining method including obtaining from the base mesh bitstream or generating by a decoding side based on mesh parameterization; a decoding module for decoding the base mesh bitstream to obtain a third mesh, and generating a target decoded mesh based on the third mesh and the obtaining method; A decoding processing device, wherein the target decoded mesh includes the target texture coordinates.

22. 20. An electronic device comprising: a processor; and a memory for storing a program or command executable by the processor, wherein, when the program or command is executed by the processor, steps of the encoding method according to any one of claims 1 to 9 are realized; or, when the program or command is executed by the processor, steps of the decoding method according to any one of claims 10 to 19 are realized.

23. A readable storage medium on which a program or command is stored, and when the program or command is executed by a processor, the steps of the encoding method of any one of claims 1 to 9 are realized, or when the program or command is executed by a processor, the steps of the decoding method of any one of claims 10 to 19 are realized.

Images