Encoding method, decoding method, encoder, decoder, code stream, and storage medium

Abstract

Embodiments of the present application provide a decoding method, comprising: at a decoding end, a decoder decoding a code stream, determining a shift coefficient corresponding to the current LOD layer, and on the basis of the shift coefficient, determining initial geometric information of a reconstructed mesh corresponding to the current LOD layer; decoding the code stream, and determining first identification information corresponding to the current LOD layer; and when the first identification information indicates that the current LOD layer uses a first mode, determining geometric information of the reconstructed mesh of the current LOD layer on the basis of fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

Description

Coding and decoding method, encoder, decoder, code stream and storage medium Technical Field

[0001] The embodiments of the present application relate to the technical field of grid compression coding, and in particular to a coding and decoding method, an encoder, a decoder, a code stream, and a storage medium. Background Art

[0002] In the standard reference software for Dynamic Mesh Coding (DMC) provided by the Moving Picture Experts Group (MPEG), encoding and decoding geometric information of a mesh mainly involves compressing shift coefficients corresponding to the original mesh.

[0003] However, in the current compressed transmission of shift coefficients, both the lifting transformation and the conversion processing are lossy processes, which limits the quality of the geometric information of the grid, reduces the coding efficiency of the grid geometric information, and affects the grid compression performance.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, an encoder, a decoder, a code stream and a storage medium, which can improve the coding efficiency of the geometric information while ensuring the quality of the geometric information of the grid, thereby improving the compression performance of the grid.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Decoding the code stream, determining a shift coefficient corresponding to the current LOD layer, and determining initial geometric information of a reconstructed mesh corresponding to the current LOD layer according to the shift coefficient;

[0009] Decoding the code stream to determine the first identification information corresponding to the current LOD layer;

[0010] When the first identification information indicates that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0011] In a second aspect, an embodiment of the present application provides a decoding method applied to a decoder, wherein the decoder includes a video decoder and a grid decoder, and the grid decoder and the video decoder are used to perform the decoding method as described in the first aspect.

[0012] In a third aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0013] Determine a shift coefficient corresponding to a current LOD layer, and determine initial geometric information of a reconstructed mesh corresponding to the current LOD layer according to the shift coefficient;

[0014] Determining whether the current LOD layer uses the first mode according to the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer;

[0015] When it is determined that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, the first identification information corresponding to the current LOD layer is set, and the first identification information is written into the bitstream.

[0016] In a fourth aspect, an embodiment of the present application provides an encoding method applied to an encoder, wherein the encoder includes a video encoder and a grid encoder, and the grid encoder and the video encoder are used to execute the encoding method as described in the third aspect.

[0017] In a fifth aspect, an embodiment of the present application provides a code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:

[0018] First identification information, second identification information, third identification information, shift coefficient, geometric upper limit information, geometric lower limit information, fitting information, and geometric information of the basic grid.

[0019] In a sixth aspect, an embodiment of the present application provides an encoder, comprising: a first determining unit and an encoding unit; wherein,

[0020] The first determining unit is configured to determine a shift coefficient corresponding to the current LOD layer, and determine initial geometric information of a reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determine whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; and if it is determined that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh;

[0021] The encoding unit is configured to set first identification information corresponding to the current LOD layer and write the first identification information into the bitstream.

[0022] In a seventh aspect, an embodiment of the present application provides an encoder, comprising: a first memory and a first processor; wherein,

[0023] a first memory for storing a computer program capable of running on the first processor;

[0024] The first processor is configured to execute the encoding method described in the third aspect when running a computer program.

[0025] In an eighth aspect, an embodiment of the present application provides a decoder, comprising: a decoding unit, a second determining unit; wherein,

[0026] The decoding unit is configured to decode the code stream;

[0027] The second determination unit is configured to determine the shift coefficient corresponding to the current LOD layer, and determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determine the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0028] In a ninth aspect, an embodiment of the present application provides a decoder, comprising: a second memory and a second processor; wherein:

[0029] a second memory for storing a computer program capable of running on the second processor;

[0030] The second processor is configured to execute the method according to the first aspect when running a computer program.

[0031] In the tenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, it implements the method described in the first aspect or the second aspect, or implements the method described in the third aspect or the fourth aspect.

[0032] The embodiment of the present application provides a coding and decoding method, an encoder, a decoder, a bitstream, and a storage medium. At the decoding end, the decoder decodes the bitstream, determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; decodes the bitstream, determines the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh. At the encoding end, the encoder determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determines whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; when it is determined that the current LOD layer uses the first mode, determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, sets the first identification information corresponding to the current LOD layer, and writes the first identification information into the bitstream. Thus, it can be seen that in the embodiment of the present application, if the first identification information corresponding to the current LOD layer indicates that the current LOD layer uses the first mode, then after the reconstruction of the mesh of the current LOD layer is completed, the initial geometric information of the reconstructed mesh can be fitted, thereby obtaining geometric information of the reconstructed mesh with better quality. That is to say, in the embodiment of the present application, based on the first mode, after the preliminary reconstruction of the mesh is completed, the quality of the geometric information can be improved by fitting the reconstructed geometric information, thereby ensuring the quality of the geometric information of the mesh while improving the encoding efficiency of the geometric information, thereby improving the compression performance of the mesh. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG1A is a schematic diagram of a three-dimensional grid image 1;

[0034] FIG1B is a partially enlarged schematic diagram of a three-dimensional grid image;

[0035] Figure 2 is a schematic diagram of the connection method of the three-dimensional grid;

[0036] FIG3A is a second schematic diagram of a three-dimensional grid image;

[0037] FIG3B is a schematic diagram of a grid data storage format;

[0038] FIG3C is a schematic diagram of properties of a three-dimensional grid image;

[0039] FIG4 is a schematic diagram showing the composition of the overall framework of grid coding;

[0040] FIG5A is a schematic diagram of preprocessing of a two-dimensional curve;

[0041] FIG5B is a schematic diagram of generating a shift coefficient;

[0042] FIG6A is a first schematic diagram of quantization processing of grid geometric position information;

[0043] FIG6B is a second schematic diagram of quantization processing of grid geometric position information;

[0044] FIG7A is a schematic diagram of coding of the connection relationship of triangular facets;

[0045] FIG7B is a schematic diagram of encoding geometric position information;

[0046] FIG7C is a schematic diagram of texture coordinate encoding;

[0047] FIG8 is a schematic diagram showing the basic principle of the shift coefficient;

[0048] FIG9 is a schematic diagram of encoding of a shift coefficient mapped to a two-dimensional image;

[0049] FIG10 is a schematic diagram of encoding of inter-frame geometric position information;

[0050] FIG11A is a schematic diagram showing the composition of an intra-frame coding framework;

[0051] FIG11B is a schematic diagram showing the composition of an inter-frame coding framework;

[0052] FIG12A is a schematic diagram showing the composition of an intra-frame decoding framework;

[0053] FIG12B is a schematic diagram showing the composition of an inter-frame decoding framework;

[0054] FIG13A is a schematic diagram of iterative subdivision of a basic grid;

[0055] FIG13B is a schematic diagram of an LOD space structure;

[0056] FIG14 is a schematic diagram of coefficient reorganization for quantized coefficients;

[0057] FIG15 is a schematic diagram showing the basic principle of reconstructing and restoring geometric position information;

[0058] FIG16 is a schematic diagram of a mesh architecture of a codec provided in an embodiment of the present application;

[0059] FIG17 is a first schematic diagram of a decoding method proposed in an embodiment of the present application;

[0060] FIG18 is a schematic diagram showing a basic principle of determining fitting information provided by an embodiment of the present application;

[0061] FIG19 is a second schematic diagram of the decoding method proposed in an embodiment of the present application;

[0062] FIG20 is a third schematic diagram of the decoding method proposed in an embodiment of the present application;

[0063] FIG21 is a schematic diagram of geometric information after fitting processing;

[0064] FIG22 is a first schematic diagram of the encoding method proposed in an embodiment of the present application;

[0065] FIG23 is a second schematic diagram of the encoding method proposed in an embodiment of the present application;

[0066] FIG24 is a third schematic diagram of the encoding method proposed in an embodiment of the present application;

[0067] FIG25 is a schematic diagram of the composition structure of the encoder;

[0068] FIG26 is a second schematic diagram of the encoder structure;

[0069] FIG27 is a schematic diagram of the first structure of a decoder;

[0070] FIG28 is a second schematic diagram of the decoder structure. DETAILED DESCRIPTION

[0071] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0073] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0074] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0075] It should be noted that it is possible to decode and synthesize different data format bitstreams within the same video scene. These can include at least image format, point cloud format, and mesh format. In this way, real-time immersive video interaction services can be provided for multiple data formats (e.g., mesh, point cloud, image, etc.) from different sources.

[0076] In embodiments of the present application, the data format-based approach allows for independent processing at the bitstream level of the data format. This means that, similar to tiles or slices in video encoding, different data formats in this scenario can be encoded independently, enabling independent encoding and decoding based on the data format.

[0077] Generally speaking, 3D animation content uses a keyframe-based representation method, that is, each frame is a static mesh. Static meshes at different times have the same topological structure and different geometric structures. However, the amount of data of 3D dynamic meshes represented based on keyframes is extremely large, so how to effectively store, transmit and draw them has become a problem faced by the development of 3D dynamic meshes. In addition, the spatial scalability of the mesh needs to be supported for different user terminals (computers, notebooks, portable devices, mobile phones); different network bandwidths (broadband, narrowband, wireless) need to support the quality scalability of the mesh. Therefore, 3D dynamic mesh compression is a very critical issue.

[0078] A 3D mesh is the surface of a 3D object composed of countless polygons in space. Polygons are composed of vertices and edges. Figure 1A shows a 3D mesh image, and Figure 1B shows a partially enlarged schematic diagram of the 3D mesh image. Figures 1A and 1B show that the mesh surface is composed of closed polygons.

[0079] A two-dimensional image has information expressed at every pixel point and is distributed regularly, so there is no need to record its position information separately. However, the distribution of vertices in the mesh in three-dimensional space is random and irregular, and the way polygons are formed requires additional regulations. Therefore, it is necessary to record the position of each vertex in space and the connection information of each polygon to fully express a mesh image. As shown in Figure 2, the same number of vertices and vertex positions will form completely different surfaces due to different connection methods.

[0080] In addition to the above information, since 3D mesh images are usually encoded using existing 2D image / video encoding methods, the 3D mesh needs to be converted from 3D space to 2D images. The UV coordinates define this conversion process.

[0081] Similar to 2D images, each position in the image may have corresponding attribute information, typically RGB color values, which reflect the object's color. For 3D meshes, in addition to color, each vertex often has reflectance values, which reflect the surface material. 3D mesh attribute information is stored in 2D images, and the mapping from 2D to 3D is defined by UV coordinates.

[0082] Therefore, 3D mesh data typically includes 3D geometric position information (x, y, z), geometric connectivity, UV coordinates, and an attribute map. Figure 3A shows a 3D mesh image, Figure 3B shows the mesh data storage format, which includes 3D geometric position information, UV coordinates, and connectivity information, and Figure 3C shows the corresponding attribute diagram.

[0083] Current 3D dynamic mesh compression methods include space-time prediction methods, which improve compression efficiency by eliminating spatial and temporal correlations; principal component analysis (PCA)-based technology, which projects in the eigenvector space to concentrate energy; and wavelet-based methods, which support spatial scalability and quality scalability.

[0084] It should be noted that in the dynamic mesh coding provided by the Moving Picture Experts Group (MPEG), Figure 4 shows a schematic diagram of the overall mesh coding framework, Figure 5A shows a schematic diagram of the preprocessing of a 2D curve, and Figure 5B shows a schematic diagram of the generation of shift coefficients. The preprocessing process for a 3D mesh is similar, and on the encoding side, it is mainly divided into two parts: preprocessing and encoder. Preprocessing first generates a base mesh and shift coefficients. The preprocessing process includes: first, downsampling the original mesh to generate a simplified mesh (decimated mesh) with a significantly reduced number of vertices, or the base mesh. The base mesh is then subdivided and algorithmically generated, with newly generated vertices inserted along the edges of the base mesh to form a subdivided mesh. Finally, for each vertex in the subdivided mesh, the nearest vertex in the original mesh is found. The vector between the vertex in the subdivided mesh and the nearest vertex in the original mesh is the shift coefficient. Since the subdivision grid can be automatically generated at the codec end as long as the subdivision algorithm and the number of subdivision iterations are determined, after preprocessing, the original grid only needs to be represented as a simple basic grid and a series of shift coefficients. This can greatly reduce the amount of data without affecting the reconstruction at the decoding end.

[0085] Video Dynamic Mesh Coding (V-DMC) based on video coding can be broadly categorized into two main categories: geometric position information encoding and attribute information encoding. As shown in Figure 5, each frame in the basketball_player sequence includes two files: basketball_player_fr0001_qp12_qt12.obj and basketball_player_fr0002.png. basketball_player_fr0001_qp12_qt12.obj contains four types of information: geometric position information (x, y, z), the connectivity of geometric position triangles, texture coordinates (u, v), and the connectivity of texture coordinates. basketball_player_fr0002.png represents the texture attribute information of the current frame (e.g., the current image). In current V-DMC encoders, geometric position information is jointly encoded using the Dynamic Range Arithmetic Coding (DRACO) and the video codec, while texture information is encoded directly using the video codec. Among them, Video Codec can include H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC / VV-enC), etc. Therefore, the following will introduce the mesh geometric information encoding in detail.

[0086] Geometric information can be divided into the encoding of position information (geometric position information and texture position information) and the encoding of connectivity relationships (geometric position information triangle patch connectivity, texture position information connectivity). Currently, V-DMC coding is mainly divided into two coding test conditions: intra-frame coding and inter-frame coding (low latency, currently no RA test environment).

[0087] (1) Intra-frame geometric information coding (Intra coding).

[0088] 1. Mesh preprocessing.

[0089] a) As shown in Figures 5A and 5B, using a two-dimensional connection relationship as an example, the original mesh's connection relationship contains a large number of points. Before encoding the mesh's geometric information, the mesh's geometric information is first quantized or simplified, ultimately resulting in a corresponding decimated mesh as the base mesh.

[0090] b) As shown in FIG6A and FIG6B , the quantization processing of the grid is performed based on the coordinates of the triangle patch. According to the connection relationship between the quantization points, the quantization processing can be divided into the following two cases:

[0091] When two vertices share a common edge, that is, two vertices that belong to the same edge before quantization, then after quantization, all the triangles connected by the two vertices need to be connected together, which involves the vanishing of the previous triangles as shown in Figure 6A;

[0092] Otherwise, if the two vertices do not share a common edge, that is, the two vertices do not belong to the same edge, then after quantization, it is only necessary to merge the boundaries of the two vertices, as shown in FIG6B , which does not affect the number of triangles.

[0093] c) In the whole process of mesh quantization based on triangle coordinates, the core problem is how to get the best vertex based on the previous vertex coordinates. The current V-DMC will get the best quantization point in the following four modes. Assuming that the vertex distribution before quantization is V1 and V2, and the vertex coordinate after quantization is V', there are the following: V1, V2, (V1+V2) / 2 and Q -1 (V1+V2), where Q is the quantization matrix corresponding to the vertex coordinates of V1 and V2. The distortion measure D before and after basic quantization is used to select the optimal quantization point.

[0094] 2.Base mesh encoding.

[0095] a) After obtaining the base mesh, the DRACO encoder is used to encode its geometric information. This geometric information primarily includes connectivity relationships and geometric position information. The DRACO encoding process is as follows: first, the connectivity relationships are encoded. Then, the geometric position information of the points is encoded based on the connectivity relationships. Finally, the texture position information is encoded based on the connectivity relationships and geometric position information.

[0096] b) Encoding of connection relationships. DRACO uses the "Edgebreaker Coding" scheme to encode the connection relationships of the mesh. See Figure 7A for details. In Figure 7A, v represents the current vertex. Before encoding the connection relationships of the mesh, the vertices of the mesh are divided into five types: C, L, R, S, and E. The physical meaning of each symbol is as follows:

[0097] C: None of the triangles connected to the current vertex have been encoded;

[0098] L: The triangle on the left connected to the current vertex completes the encoding;

[0099] R: The triangle on the right side connected to the current vertex completes the encoding;

[0100] S: The left and right triangles connected to the current vertex have not been encoded;

[0101] E: The left and right triangles connected to the current vertex have been encoded.

[0102] Finally, the type of each vertex and the processing order of the vertices are encoded in a certain order, and the decoding end restores the geometric connection relationship of the mesh according to the processing order and type of the vertices.

[0103] c) Coding of geometric position information. After completing the coding of the vertex connection relationship, the geometric position information of each vertex is predictively coded based on the vertex connection relationship. The idea adopted by predictive coding is the "Parallelograms algorithm", as shown in Figure 7B. A simple linear fit is performed using the three vertices adjacent to the current point to be coded: the left vertex, the right vertex, and the opposite vertex: pred pos =(left+right)-opposite (1)

[0104] d) After completing the point connection relationship and geometric position information, the texture coordinates are predictively encoded based on the decoding and reconstruction of these two, as shown in Figure 7C. Similarly, assuming the current vertex is C, the left and right vertices of the current point can be obtained based on the point connection relationship. Then, the texture coordinates of the left and right vertices are used to predict the texture coordinates of the current vertex C.

[0105] 3. Displacement coefficient encoding.

[0106] a) First, after the base mesh is encoded and reconstructed, a partitioning algorithm is used to partition the base mesh to obtain the initial reconstructed mesh. Specifically, the curve corresponding to the subdivided mesh in Figure 5A is used to obtain the subdivided mesh (also called the "initial mesh") through simple linear interpolation. The coordinates of the newly inserted points are obtained by linear interpolation based on the two vertices on the current boundary:

[0107] b) Secondly, calculate the error Delta between the points in the subdivided mesh and the original mesh after division. The error Delta can be a point error in the world coordinate system. Finally, the displacement (i.e., displacement coefficient) of each point is calculated using the error Delta between each point and the normal vector Norm of each point. See Figure 8 for details. In Figure 8, the bold solid line represents the error Delta, and N and T represent the normal vector Norm. Thus, the specific calculation method is as follows: Displacement = Delta × Norm (3)

[0108] c) After calculating the displacement of each point, the spatial domain residual coefficient can be transformed into the frequency domain using the lifting transform to obtain the corresponding frequency domain residual coefficient.

[0109] d) Finally, a coefficient packing algorithm is used to map the frequency domain residual coefficients of each point into a two-dimensional image in a certain order. The current V-DMC can be arranged according to the Morton Code Order, as shown in Figure 9.

[0110] e) Finally, a traditional Video Codec is used to encode the two-dimensional image.

[0111] 4. Recoloring.

[0112] Recoloring is an algorithm on the encoder side. After the reconstruction of the encoder's geometric information is completed, the original geometric information, the original texture attribute information, and the reconstructed mesh geometric information are used to recolor the texture attribute information of the reconstructed mesh.

[0113] (2) Inter-frame geometric information coding (Inter coding).

[0114] a) Similar to the encoding above, the geometric position information includes the geometric connection relationship and the geometric position information encoding. However, it should be noted that the geometric position information encoding between frames only needs to encode the geometric position information (x, y, z) of the current base mesh, and does not need to encode the connection relationship and texture position information (u, v). The specific reason is as follows: If the current frame can use inter-frame encoding, then the base mesh of the current frame's reference frame (such as the reference image) will be used at the encoder to obtain the mesh information of the current frame. Therefore, the current frame and the reference frame have the same connection relationship and UV texture coordinates, only the geometric position information is different.

[0115] b) Based on a), it can be known that there is only an error in the geometric position information between the current frame and the reference frame, so the current V-DMC performs predictive coding on the geometric position information of the current frame.

[0116] As shown in Figure 10, the black dot is the point to be coded. The current point is used in the reference frame to get the corresponding predicted point (similar to the same-position block in video coding). Then, the neighborhood point of the current point (MV of the coded vertex) is used to predict the motion vector (MV) of the current point. The specific steps are as follows: assuming that the coordinates of the current point are pos and the coordinates of the corresponding same-position point are Pred pos , then the MV of the current point is calculated as: MV = Pos-Pred pos (4)

[0117] There are two predictive coding modes in the current V-DMC:

[0118] i. Directly encode the MV of the current point;

[0119] ii. Use the neighborhood to perform predictive coding on the MV of the current point.

[0120] At the encoding end, the rate-distortion optimization algorithm is used to obtain the optimal coding mode for each coding group (CG). The current V-DMC sets the number of points for each CG to be at most 16.

[0121] Coding of texture attribute information: Current V-DMC encodes texture attribute information directly using a video codec (Video-Codec), such as AVC, HEVC, VVC, or VV-enC.

[0122] Figure 11A is a schematic diagram of the framework of an intra-frame encoder. As shown in Figure 11A, in the intra-frame encoder, a common static mesh encoder (Static Mesh Encoder) can be used to encode the simplified mesh to generate the corresponding bitstream (Compressed base mesh bitstream). Next, the reconstructed simplified mesh is used to update the displacement coefficients (Update Displacements). The updated displacement coefficients are subjected to wavelet transform (Wavelet Transform) and quantization (Quantization) to obtain the displacement coefficients. After being packaged into images and videos (Image Packing, Video Packing), they are encoded using HEVC to generate a bitstream (Compressed displacements bitstream) of the displacement coefficients. For attribute map encoding, the feature map is first transformed (Texture Transfer) according to the difference between the reconstructed geometric information and the original geometric information, and then padded (Padding) and packaged (Video Packing) and encoded using a video encoder to form an attribute bitstream (Compressed attribute bitstream).

[0123] Figure 11B is a schematic diagram of an inter-frame encoder framework. As shown in Figure 11B, the inter-frame encoder and intra-frame encoder process are roughly the same, but the inter-frame encoder does not directly encode the simplified grid. Instead, it encodes the motion vector MV between the simplified grid of the current frame and the simplified grid of the reference frame and generates the corresponding motion vector bitstream (Compressed motion bitstream).

[0124] Correspondingly, during the decoding process, the decoder can also be divided into an intra-frame decoder and an inter-frame decoder according to the type of the frame it operates on, which are used to perform intra-frame decoding and inter-frame decoding respectively.

[0125] FIG12A is a schematic diagram of intra-frame decoding. As shown in FIG12A , in the intra-frame decoder, a static mesh decoder can be used to decode the simplified mesh. A video decoder is used to decode the shift coefficient video, and the shift coefficient is obtained through video unpacking and inverse wavelet transform. The decoded simplified mesh and shift coefficient are used to obtain the decoded mesh geometry information. The attribute map is decoded directly through the video decoder.

[0126] FIG12B is a schematic diagram of inter-frame decoding. As shown in FIG12B , for an inter-frame decoder, the process is basically the same as that of an intra-frame decoder, except that the simplified grid is not directly decoded, but the motion vector is decoded and the simplified grid of the current frame is calculated using the simplified grid of the previous frame (reference frame).

[0127] In summary, in the dynamic mesh coding (Dynamic Mesh Coding) currently provided by MPEG, the dynamic mesh coding process is divided into the following steps: at the encoding end, the basic mesh generated by preprocessing is quantized and then encoded using Google's open source DRACO encoder, and the shift coefficients are encoded using HEVC after wavelet transform, quantization, and two-dimensional mapping. The two-dimensional attribute map is also directly transmitted to the HEVC encoder for encoding; at the decoding end, the basic mesh code stream is decoded by DRACO to generate a decoded basic mesh, and the shift coefficients are decoded by HEVC decoding, inverse two-dimensional mapping, inverse quantization, and inverse transformation to generate decoded shift coefficients. Then, the decoded basic mesh and the decoded shift coefficients are used together to reconstruct the three-dimensional mesh geometry, and the attribute code stream is decoded by HEVC to generate a reconstructed attribute map.

[0128] (3) General test conditions for MPEG DMC.

[0129] a. There are 2 test conditions:

[0130] Condition 1: all intra geometry is lossy and attributes are lossy;

[0131] Condition 2: Random access is lossy in geometry and attributes;

[0132] b. The general test sequence may include five categories, namely Cat1-A, Cat1-B and Cat1-C, all of which contain geometric information and color attribute information.

[0133] The following is a detailed introduction to the Displacement coefficient encoding of V-DMC.

[0134] In one specific implementation, the encoder first iteratively partitions the base mesh using a specific algorithm to obtain the corresponding mesh position information. The specific partitioning algorithm is consistent with the previous description, using linear interpolation of vertices on each boundary to obtain the corresponding geometric position information. Assuming that the entire partitioning is iterated N times, the LOD partitioning is performed based on the displacement coefficients obtained from different iterative partitioning, as shown in Figure 13A.

[0135] As shown in Figure 13A, the base mesh is linearly interpolated to obtain the corresponding mesh geometric position information. The initial geometric position information is used to calculate the error with the original mesh to obtain the displacement coefficient of each point. The LOD division can be divided into four layers: level 0, level 1, level 2, and level 3. The specific LOD spatial structure is shown in Figure 13B.

[0136] Secondly, a lifting wavelet transform is performed based on the LOD spatial structure, which can include two steps: prediction and update. The prediction algorithm is as follows:

[0137] Here, v represents the vertex to be predicted, and v1 and v2 represent the vertices at both ends of the boundary where the vertex to be predicted is located.

[0138] Again, the update steps are as follows:

[0139] Finally, the transformed coefficients are quantized and reorganized. As shown in Figure 14, this reorganization is performed on a block-by-block basis, where they are grouped into Block 0, Block 1, Block 2, and Block 3. In other words, the current V-DMC performs coefficient reorganization on a block-by-block basis, with each block being 16×16 in size. The coefficients within each block are arranged according to the Morton code to produce the corresponding 2D image. After completing this series of operations, the 2D image can be encoded using the Video Codec.

[0140] In summary, the dynamic grid encoding process can be divided into the following steps:

[0141] 1. Preprocess the original mesh by reducing the number of vertices in the mesh and simplifying the connection relationship.

[0142] 2. Subdivide the simplified mesh in step 1. For any two connected vertices in step 1, add a new point at the midpoint of the connecting line, and repeat this process twice.

[0143] 3. For each vertex in step 2, find the point in the original mesh that is closest to it and calculate the displacement coefficient of these two points.

[0144] 4. Use an encoder such as Draco to quantize the simplified grid in step 1 and then encode it.

[0145] 5. Adjust the shift coefficients in step 3 based on the reconstructed simplified grid obtained in step 4.

[0146] 6. Perform wavelet transform on the shift coefficients in step 5, and quantize the shift coefficients after wavelet transform to obtain quantized transform coefficients.

[0147] 7. Map the quantized transform coefficients from three-dimensional space to a two-dimensional image (or "image packing") to generate a two-dimensional image of shifted coefficients.

[0148] 8. Use a standard video encoder such as H.265 to encode the shift coefficient two-dimensional image in step 6.

[0149] In another specific implementation, for the decoding end, first, the Video-Codec is used to decode and reconstruct the two-dimensional image to restore the corresponding two-dimensional image. Secondly, according to the coefficient reorganization method, the lifting transform coefficient corresponding to each point can be restored. Finally, the inverse transform of the lifting wavelet transform can be used to restore the displacement coefficient of each point. After obtaining the displacement coefficient of each point, the geometric position information of the base mesh and the displacement coefficient are used to reconstruct and restore the geometric position information corresponding to the current mesh. As shown in Figure 15, the geometric position information of the level 0 layer in the base mesh and the displacement coefficient of the level 0 layer can reconstruct and restore the geometric position information of the level 0 layer in the reconstructed mesh. The geometric position information of the level 1 layer in the base mesh and the displacement coefficient of the level 1 layer can reconstruct and restore the geometric position information of the level 1 layer in the reconstruct mesh. And so on, the geometric position information of the level 3 layer in the base mesh and the displacement coefficient of the level 3 layer can reconstruct and restore the geometric position information of the level 3 layer in the reconstruct mesh.

[0150] In summary, the dynamic grid decoding process can be divided into the following steps:

[0151] 1. The basic grid code stream is decoded by a decoder such as draco to generate a decoded basic grid.

[0152] 2. The shift coefficient bit stream is decoded using a standard video encoder such as H.265 to obtain a shift coefficient two-dimensional image.

[0153] 3. Map the shift coefficient two-dimensional image from the two-dimensional image to the three-dimensional space (or "image unpacking") to obtain quantized transform coefficients.

[0154] 4. Inverse quantize the quantized transform coefficients and perform inverse wavelet transform to obtain decoded displacement coefficients.

[0155] 5. Generate reconstructed three-dimensional mesh geometric information by combining the decoded base mesh and the decoded displacement coefficients.

[0156] 6. The attribute bitstream generates a reconstructed attribute map after being decoded by HEVC.

[0157] In common techniques, such as V-DMC coding, generally, the initial reconstructed geometric position information of the mesh obtained by dividing the Base Mesh and the Displacement value are used to reconstruct and recover the geometric position information of the current mesh. However, since the Displacement value uses the lifting wavelet transform to transform the Displacement coefficients from the spatial domain to the frequency domain, and the frequency domain coefficients are quantized. The lifting wavelet transform coding is a lossy coding, which leads to a large difference between the original Displacement of the Displacement coefficients obtained by inverse quantization reconstruction. Therefore, based on such a reconstruction scheme, the quality of the geometric position information of the reconstructed mesh is low. Specifically, the calculation method of the Displacement value in V-DMC is analyzed in detail below:

[0158] Assume that the coordinates of the current point are pos(x, y, z), the normal vector of the current point is normal(n_x, n_y, n_z), and the original mesh coordinate position information corresponding to the current point is orgPos(x_org, y_org, z_org).

[0159] Then: First, calculate the error delta between them: delta[i] = pos[i] - orgPos[i] (7)

[0160] Where i = 0, 1, 2 represent x, y, z respectively.

[0161] After obtaining delta, calculate the normal vector of the current point in each dimension. If normal is the normal vector in the Z dimension, the corresponding normal vectors in the X and Y axes are: E0 = (1, 0, 0), E1 = (0, 1, 0), E2 = (0, 0, 1)

[0162] Then d0 = normal * E0, d1 = normal * E(0), d2 = normal * E(0)

[0163] When abs(d0) < abs(d1) && abs(d0) < abs(d2): normal x= E0 - normal * d0 (8)

[0164] Otherwise, if abs(d1) < abs(d2): normal x = E1 - normal * d1 (9)

[0165] Otherwise normal x = E2 - normal * d2 (10)

[0166] Finally, normal y = normal x × normal (11)

[0167] Then the Displacement coefficient of this point: disp x = delta * normal (12) disp y = delta * normal x (13) disp z = delta * normal y (14)

[0168] That is: disp x = delta x * normal x + delta y * normal y + delta z * normal z (15) disp y = delta x * normal xx + delta y * normal xy + delta z * normal xz (16) disp z = delta x * normal yx + delta y * normal yy + delta z * normal yz (17)

[0169] According to the above formula, we can get the linear relationship between disp and delta. Then, according to the formula, we can also use disp to get delta.

[0170] However, the current V-DMC takes into account the coding efficiency of geometric information and only encodes the displacement when encoding the displacement. x One component, the final delta is calculated as follows: delat x =disp x *normal x +disp y *normal y +disp z *normal z (18)

[0171] This causes the linear conversion of delta to become a lossy process.

[0172] It can be seen that the reconstruction process of mesh geometric position information based on the existing V-DMC reconstruction has the following two lossy processing methods: first, the displacement component adopts the lifting transform encoding idea, which is itself a lossy encoding scheme; second, the linear conversion relationship between disp and delta is itself lossy.

[0173] Then, the quality of the reconstructed mesh geometric information obtained by dividing the initial mesh geometric information using the Displacement component and the Base mesh has certain limitations, which leads to a reduction in the quality of the existing mesh geometric information reconstruction.

[0174] To solve the above problems, the embodiments of the present application provide a coding and decoding method, an encoder, a decoder, a bitstream, and a storage medium. At the decoding end, the decoder decodes the bitstream, determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; decodes the bitstream, determines the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh. At the encoding end, the encoder determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determines whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; when it is determined that the current LOD layer uses the first mode, determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, sets the first identification information corresponding to the current LOD layer, and writes the first identification information into the bitstream. Thus, it can be seen that in the embodiment of the present application, if the first identification information corresponding to the current LOD layer indicates that the current LOD layer uses the first mode, then after the reconstruction of the mesh of the current LOD layer is completed, the initial geometric information of the reconstructed mesh can be fitted, thereby obtaining geometric information of the reconstructed mesh with better quality. That is to say, in the embodiment of the present application, based on the first mode, after the preliminary reconstruction of the mesh is completed, the quality of the geometric information can be improved by fitting the reconstructed geometric information, thereby ensuring the quality of the geometric information of the mesh while improving the encoding efficiency of the geometric information, thereby improving the compression performance of the mesh.

[0175] In other words, to address the challenges of common techniques and to ensure a balance between the displacement coding component and the quality of the reconstructed mesh's geometric information, this solution proposes fitting the relationship between the reconstructed mesh's geometric information and the original mesh's geometric information at different LOD levels to ensure the quality of the reconstructed mesh's geometric information. This approach only requires encoding the fitting parameters for each LOD level, while existing encoding schemes only encode one component, thereby improving the quality of the mesh's geometric information and increasing the efficiency of geometric information coding.

[0176] The embodiment of the present application provides a network architecture of a codec system including a decoding method and an encoding method. Figure 16 is a schematic diagram of a network architecture of a codec provided by an embodiment of the present application. As shown in Figure 16, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During the implementation process, the electronic device can be various types of devices with codec functions. For example, the electronic device can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensor device, a server, etc., and the embodiment of the present application is not limited. Among them, the decoder or encoder in the embodiment of the present application can be the above-mentioned electronic device.

[0177] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0178] An embodiment of the present application provides a decoding method. FIG17 is a schematic diagram 1 of the decoding method provided in the embodiment of the present application. As shown in FIG17 , in the embodiment of the present application, the method for performing decoding processing by the decoder may include the following steps:

[0179] Step 101: Decode the code stream, determine the shift coefficient corresponding to the current LOD layer, and determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer according to the shift coefficient.

[0180] In an embodiment of the present application, the decoder may first decode the code stream to determine the shift coefficient corresponding to the current LOD layer, and then further determine the initial geometric information of the reconstructed grid corresponding to the current LOD layer based on the shift coefficient.

[0181] It should be noted that the decoding method in the embodiment of the present application may refer to a method for decoding geometric information, more specifically, a method for decoding geometric information in a dynamic mesh. The decoding method may be applied to a decoder in a V-DMC, but is not limited thereto.

[0182] Furthermore, in an embodiment of the present application, the encoding of V-DMC can be mainly divided into two categories: geometric position information encoding and attribute information encoding. As shown in Figure 5, each frame file of the sequence basketball_player includes two files: basketball_player_fr0001_qp12_qt12.obj and basketball_player_fr0002.png. Among them, basketball_player_fr0001_qp12_qt12.obj contains four types of information: geometric position information (x, y, z), the connection relationship of geometric position information triangles, texture coordinates (u, v) and the connection relationship of texture coordinates. basketball_player_fr0002.png represents the texture attribute information of the current image. In the current V-DMC encoder, the geometric position information is jointly encoded using DRACO and Video Codec (AVC, HEVC or VVC), and the texture information encoding is directly encoded using Video Codec.

[0183] It should be noted that in the embodiments of the present application, the encoding of geometric information can be divided into: encoding of position information (geometric position information and texture position information) and encoding of connection relationships (connection relationships of triangle patches of geometric position information and connection relationships of texture position information).

[0184] For intra-frame mode, after obtaining the base mesh, the DRACO encoder is used to encode the base mesh's geometric information. The entire DRACO encoding process mainly includes: first, completing the encoding of the connection relationship, then encoding the geometric position information of the points based on the geometric connection relationship, and finally encoding the texture position information based on the connection relationship and geometric position information.

[0185] For the inter-frame mode, after obtaining the base mesh, you can choose to use the base mesh of the reference image of the current image to obtain the mesh information of the current image. Since the current image and the reference image have the same connection relationship and uv texture coordinates, only the geometric position information is different. Therefore, the geometric position information encoding between frames only needs to encode the geometric position information (x, y, z) of the current base mesh, and does not need to encode the connection relationship and texture position information (u, v).

[0186] It can be understood that, in the embodiments of the present application, the current image can be understood as the current frame, and the reference image can be understood as the reference frame.

[0187] Furthermore, in an embodiment of the present application, when decoding the code stream and determining the shift coefficient corresponding to the current LOD layer, the code stream can be decoded first to determine the two-dimensional image of the current LOD layer; then the coefficients of the two-dimensional image can be reorganized to determine the lifting transformation coefficient of the current LOD layer; finally, the lifting transformation coefficient of the current LOD layer can be inversely transformed to determine the shift coefficient of the current LOD layer.

[0188] In a specific embodiment, performing coefficient reorganization processing on a two-dimensional image to determine the lifting transformation coefficient of the current LOD layer may include: performing coefficient reorganization processing on the two-dimensional image to determine the quantization coefficient of the current LOD layer; performing inverse quantization processing on the quantization coefficient of the current LOD layer to determine the lifting transformation coefficient of the current LOD layer.

[0189] Specifically, the bitstream here can refer to the shift coefficient bitstream. By decoding the shift coefficient bitstream through a video decoder (e.g., Video-Codec), the corresponding two-dimensional image can be reconstructed. Then, by using the coefficient reorganization method, the quantization coefficients corresponding to each point in the current LOD layer are recovered. Dequantization is then used to recover the lifting transform coefficients corresponding to each point in the current LOD layer. Finally, the inverse of the lifting wavelet transform is used to recover the shift coefficients corresponding to each point in the current LOD layer.

[0190] Furthermore, in an embodiment of the present application, after determining the shift coefficient corresponding to each point in the current LOD layer, the preliminary reconstruction of the current LOD layer can be completed according to the shift coefficient, thereby determining the initial geometric information of the reconstructed grid corresponding to the current LOD layer.

[0191] It should be noted that in an embodiment of the present application, when determining the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient, it is possible to choose to determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient and the geometric information of the initial mesh corresponding to the current LOD layer.

[0192] It is understandable that, in the embodiment of the present application, the geometric information of the initial mesh corresponding to the current LOD layer may be determined first.

[0193] Furthermore, in an embodiment of the present application, when determining the geometric information of the initial mesh corresponding to the current LOD layer, the code stream can be decoded first to determine the basic mesh corresponding to the current LOD layer; and then the basic mesh can be subdivided to determine the geometric information of the initial mesh corresponding to the current LOD layer.

[0194] It should also be noted that, in the embodiment of the present application, the basic grid can also be called a "simplified grid".

[0195] In some embodiments, for the intra mode, determining the base grid of the current LOD layer may include: decoding a code stream, and determining the base grid of the current LOD layer.

[0196] For example, the code stream here may refer to a basic grid code stream. Then, by decoding the basic grid code stream with a dynamic grid decoder (eg, DRACO), the basic grid of the current image may be obtained.

[0197] In some embodiments, for the inter-frame mode, determining the base grid of the current LOD layer may include: determining the base grid of the current LOD layer of the current image based on the base grid of the reference LOD layer of the reference image.

[0198] It should also be noted that, in the embodiment of the present application, the reference image is a decoded frame before the current image. For example, the reference image can be the frame before the current image, but this is not specifically limited.

[0199] Furthermore, when performing subdivision processing on the basic grid to determine the geometric information of the initial grid corresponding to the current image, it is possible to iteratively divide the basic grid according to a grid subdivision mode to determine the geometric information of the initial grid.

[0200] It should be noted that, in the embodiment of the present application, the grid subdivision mode may include: a subdivision algorithm and a subdivision iteration number, wherein the subdivision algorithm is a linear interpolation algorithm.

[0201] It should be noted that, in the embodiment of the present application, by dividing the current image into levels of detail (LOD), at least one LOD layer can be determined; wherein, the at least one LOD layer can include the current LOD layer.

[0202] For example, as shown in FIG13A , the geometric information of the initial mesh obtained by subdividing the base mesh three times is considered to be the 0th iteration corresponding to the 0th layer (level 0). The newly added vertices in the first iteration constitute the 1st layer (level 1), the newly added vertices in the second iteration constitute the 2nd layer (level 2), and the newly added vertices in the third iteration constitute the 3rd layer (level 3). The specific LOD division structure is shown in FIG13B . The top layer is the base mesh. As the iteration proceeds, the number of newly added vertices in each iteration increases successively, forming a pyramid structure. In this way, by subdividing the base mesh, the geometric information of the initial mesh of the current LOD layer in the current image can be determined.

[0203] In some embodiments, subdividing the base grid to determine the geometric information of the initial grid of the current LOD layer in the current image may include: iteratively dividing the base grid according to a grid subdivision mode to determine the geometric information of the initial grid of the current LOD layer in the current image.

[0204] Specifically, in the embodiments of the present application, the mesh subdivision mode can be understood as upsampling the vertices on each boundary of the base mesh, or can also be understood as interpolating the vertices on each boundary of the base mesh. Exemplarily, the mesh subdivision mode includes a subdivision algorithm and a number of subdivision iterations. In some embodiments, the subdivision algorithm can be an interpolation algorithm, for example, a linear interpolation algorithm, or a nonlinear interpolation algorithm, which is not specifically limited herein.

[0205] Here, after the base mesh is decoded and reconstructed, a subdivision algorithm is used to subdivide the base mesh to obtain an initial mesh. For example, the base mesh can be used to obtain an initial mesh (also called a "subdivided mesh") through a linear interpolation algorithm. In each iteration, the coordinates of the newly inserted points are obtained by linear interpolation based on the two vertices on the current boundary, as shown in formula (7).

[0206] Among them, pos1 and pos2 are the geometric position coordinates of the two end vertices on the current boundary participating in this iteration, pos new The geometric position coordinates of the new vertices added for this iteration.

[0207] It should be noted that, in the embodiments of the present application, the decoder may be a video decoder, or any decoding device including a video decoder and a trellis decoder.

[0208] Furthermore, in an embodiment of the present application, the code stream transmitted to the decoder may be a code stream including the first identification information, or may be code stream data including the code stream including the first identification information and a code stream of a simplified grid (basic grid) (or a code stream of a motion vector).

[0209] It is understandable that, in the embodiments of the present application, the current image may be a current image frame or a current video frame, which is not specifically limited in the present application.

[0210] Step 102: Decode the code stream to determine the first identification information corresponding to the current LOD layer.

[0211] In an embodiment of the present application, the decoder may further decode the bitstream to determine first identification information corresponding to the current LOD layer, wherein the first identification information may be used to determine whether the current LOD layer in the current image uses the first mode.

[0212] It should be noted that, in the embodiment of the present application, the first mode may represent a prediction mode for optimizing and correcting geometric information of the reconstructed grid.

[0213] Accordingly, in the embodiments of the present application, the first mode may be understood as a prediction mode for acquiring geometric information of the reconstructed mesh of the current LOD layer by fitting parameters.

[0214] It should be noted that, in the embodiment of the present application, after the first identification information is determined by decoding the code stream, whether the current LOD layer uses the first mode can be determined by the value of the first identification information.

[0215] Exemplarily, in some embodiments, if the value of the first identification information is a first value, then it can be determined that the current LOD layer corresponding to the current image uses the first mode; if the value of the first identification information is a second value, then it can be determined that the current LOD layer corresponding to the current image does not use the first mode.

[0216] It should be noted that, in the embodiment of the present application, the first value and the second value may be in parameter form or in digital form. For example, the first value may be 1 and the second value may be 0.

[0217] Furthermore, in an embodiment of the present application, before determining the first identification information, the code stream may be decoded to determine the second identification information corresponding to the current sequence, wherein the second identification information may be used to determine whether the current sequence allows the use of the first mode, that is, whether the prediction mode corresponding to the current sequence includes the first mode. If the second identification information corresponding to the current sequence indicates that the current sequence allows the use of the first mode, then it can be considered that the prediction mode used when predicting the current sequence frame may include the first mode, that is, for any image frame or video frame in the current sequence, a prediction mode that optimizes and corrects the geometric information of the reconstructed grid may be used.

[0218] It should be noted that, in the embodiment of the present application, after the second identification information corresponding to the current sequence is determined by decoding the code stream, whether the current sequence allows the use of the first mode can be determined by the value of the second identification information.

[0219] Exemplarily, in some embodiments, if the value of the second identification information is the third value, it can be determined that the current sequence allows the use of the first mode; if the value of the second identification information is the fourth value, it can be determined that the current sequence does not allow the use of the first mode.

[0220] It should be noted that, in the embodiment of the present application, the third value and the fourth value may be in parameter form or in digital form. For example, the third value may be 1 and the fourth value may be 0.

[0221] It is understandable that, in an embodiment of the present application, when the second identification information indicates that the current sequence allows the use of the first mode, the code stream can continue to be decoded to determine the third identification information corresponding to the current image in the current sequence; wherein the third identification information can be used to determine whether the current image allows the use of the first mode, that is, whether the prediction mode corresponding to the current image includes the first mode. If the third identification information corresponding to the current image indicates that the current image allows the use of the first mode, then it can be considered that the prediction mode used when predicting the current image can include the first mode, that is, for any image frame or video frame in the current image, a prediction mode that optimizes and corrects the geometric information of the reconstructed grid can be used.

[0222] It should be noted that, in the embodiment of the present application, after the third identification information corresponding to the current image is determined by decoding the code stream, whether the current image is allowed to use the first mode can be determined by the value of the third identification information.

[0223] Exemplarily, in some embodiments, if the value of the third identification information is the fifth value, it can be determined that the current image allows the use of the first mode; if the value of the third identification information is the sixth value, it can be determined that the current image does not allow the use of the first mode.

[0224] It should be noted that, in the embodiment of the present application, the fifth value and the sixth value may be in parameter form or in digital form. For example, the fifth value may be 1 and the sixth value may be 0.

[0225] That is, in an embodiment of the present application, the identification information for whether the first mode can be used can be determined by parsing layer by layer, that is, the encoding end can choose to set the identification information indicating whether the first mode can be used layer by layer from the upper layer to the lower layer. For example, first, the second identification information corresponding to the VFPS (Vmesh Frames Parameters Set) indicates whether the first mode is allowed to be used in the current sequence; secondly, after the FPS (Frame Parameters Set) inherits the VFPS, it determines whether the first mode is allowed to be used for the current image.

[0226] For example, in some embodiments, for a GOF->Frame->LOD encoding structure, the corresponding syntax elements are VFPS->FPS->LODPS. First, the corresponding syntax element of the GOF (Group of Frame) determines whether skip coding is enabled for the current GOF. Then, the Frame inherits the GOF syntax and can also be updated. Finally, syntax is added at the LOD level to determine whether the current LOD layer is skipped, and the syntax elements can also be updated. Ultimately, the shift coefficient decoding of each LOD layer proceeds layer by layer through VFPS->FPS->LODPS.

[0227] Step 103 : When the first identification information indicates that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0228] In an embodiment of the present application, after decoding the code stream and determining the first identification information corresponding to the current LOD layer, when the first identification information indicates that the current LOD layer uses the first mode, it is possible to select the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0229] It should be noted that in the embodiment of the present application, based on the first mode, the initial geometric information of the reconstructed mesh that has been reconstructed in the current LOD layer needs to be fitted using the fitting information corresponding to the current LOD layer, so as to complete the correction and optimization of the initial geometric information of the reconstructed mesh and obtain the geometric information of the reconstructed mesh of the current LOD layer with better quality.

[0230] Furthermore, in an embodiment of the present application, after decoding the code stream and determining the first identification information corresponding to the current LOD layer, if the first identification information indicates that the current LOD layer uses the first mode, you can also choose to continue decoding the code stream to determine the fitting information corresponding to the current LOD layer.

[0231] That is to say, in the embodiment of the present application, the encoding end can write the fitting parameters into the code stream and transmit it to the decoding end.

[0232] It should be noted that, in an embodiment of the present application, the fitting information may include a fitting type and fitting parameters corresponding to the fitting type; wherein the fitting type includes linear fitting, nonlinear fitting, neural grid-based fitting, curve fitting, and convolution parameter fitting.

[0233] For example, when the fitting type includes linear fitting, the fitting parameters may include a slope and / or an intercept of the linear function.

[0234] Exemplarily, when the fitting type includes nonlinear fitting, the fitting parameter may include at least one constant in the nonlinear function. For example, if the nonlinear function is an exponential function, the fitting parameter may include the constant a of the nonlinear function; if the nonlinear function is a polynomial function, the fitting parameter may include the coefficients a_0, a_1, a_2, ... of the polynomial function; and if the nonlinear function is a logarithmic function, the fitting parameter may include the base a of the nonlinear function.

[0235] Illustratively, in the embodiment of the present application, the fitting type may also include a linear function type, an exponential function type, a logarithmic function type, a polynomial function type, etc., which are not specifically limited here.

[0236] Furthermore, in an embodiment of the present application, at the encoding end, fitting information such as the fitting type and fitting parameters may be determined through the correlation between the original grid and the reconstructed grid.

[0237] For example, in some embodiments, FIG18 is a schematic diagram of the basic principle of determining fitting information provided by an embodiment of the present application. As shown in FIG18 , at the encoder, after determining the geometric information of the reconstructed mesh, fitting information for constructing a mapping relationship between the initial geometric information and the predicted geometric information can be determined based on parameter correlations between the geometric position information of the midpoints of the reconstructed mesh and the geometric position information of the midpoints of the original mesh.

[0238] Furthermore, in an embodiment of the present application, when determining the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, it is possible to choose to determine the geometric information of the reconstructed mesh of the current LOD layer based on the fitting type, fitting parameters and the initial geometric information of the reconstructed mesh.

[0239] It should be noted that in an embodiment of the present application, when determining the geometric information of the reconstructed mesh of the current LOD layer based on the fitting type, fitting parameters and the initial geometric information of the reconstructed mesh, the mapping relationship between the initial geometric information and the predicted geometric information can be determined first based on the fitting type and fitting parameters, and then the geometric information of the reconstructed mesh corresponding to the initial geometric information of the reconstructed mesh can be determined based on the mapping relationship between the initial geometric information and the predicted geometric information.

[0240] For example, in some embodiments, a mapping relationship between the initial geometric information and the predicted geometric information is constructed based on the fitting information including the fitting type and the fitting parameters, wherein the mapping relationship is shown in the following formula: mesh =f(reconMesh, lvl) (19)

[0241] Among them, lvl represents different LOD layers, reconMesh represents the initial geometric information, that is, the initial mesh information, predict mesh It represents the use of a certain functional relationship to obtain the corresponding predicted geometric position information, that is, the predicted geometric information.

[0242] For example, in the embodiment of the present application, considering the balance between coding complexity and coding efficiency, the relationship between the reconstructed mesh geometric position information and the initial mesh position information can be obtained by using a simple linear fitting, as shown below: mesh =k*(reconMesh, lvl)+b (20)

[0243] Where * and + represent vector multiplication and addition. This means the fit type is linear, and the fit parameters are k and b.

[0244] Furthermore, in an embodiment of the present application, after determining the mapping relationship between the initial geometric information and the predicted geometric information based on the fitting type and fitting parameters, the mapping relationship can be used to determine the geometric information of the reconstructed grid after fitting corresponding to the initial geometric information of the reconstructed grid.

[0245] That is to say, in an embodiment of the present application, based on the spatial structure of LOD, after the position information of the reconstruction point of the current LOD layer needs to be fitted according to the corresponding first identification information, the final reconstruction point can be obtained through fitting processing, that is, the fitting parameters will be used to post-process the position information of the current LOD layer point, and the position information of the current LOD layer point will be reconstructed and restored.

[0246] Exemplarily, in some embodiments, the linear fitting error and the error of the original coding scheme are first compared at the encoding end. When the former is smaller than the latter, it is determined to use the first mode. At this time, the first identification information needs to be transmitted to the decoding end. For example, a flag is transmitted to the current LOD layer: posFittingFlag. When the symbol is true, it means that linear fitting is required, that is, the first mode is used. Otherwise, linear fitting is not performed, that is, the first mode is not used. When the symbol is true, since linear fitting is used, it is necessary to transmit fitting information including the fitting type (linear fitting) and fitting parameters (such as parameters k and b) at the current LOD layer. Finally, the decoding end uses the fitting parameter flag of each LOD layer, that is, the first identification information to determine whether the current LOD layer needs to be fitted.

[0247] Furthermore, in an embodiment of the present application, FIG19 is a second schematic diagram of a decoding method proposed in an embodiment of the present application. As shown in FIG19 , after decoding the bitstream and determining the first identification information corresponding to the current LOD layer, that is, after step 102, the method for the decoder to perform decoding processing may further include the following steps:

[0248] Step 104 : When the first identification information indicates that the current LOD layer does not use the first mode, the initial geometric information of the reconstructed mesh is determined as the geometric information of the reconstructed mesh of the current LOD layer.

[0249] In an embodiment of the present application, after decoding the code stream and determining the first identification information corresponding to the current LOD layer, if the first identification information indicates that the current LOD layer does not use the first mode, the initial geometric information of the reconstructed mesh can be directly determined as the geometric information of the reconstructed mesh of the current LOD layer instead of being fitted.

[0250] That is to say, in an embodiment of the present application, based on the spatial structure of LOD, after the position information of the reconstruction point of the current LOD layer according to the corresponding first identification information does not need to be fitted, the reconstruction point of the current LOD layer can be directly used as the position information of the current LOD layer point.

[0251] Furthermore, in an embodiment of the present application, FIG20 is a third schematic diagram of a decoding method proposed in an embodiment of the present application. As shown in FIG20 , after decoding the bitstream and determining the first identification information corresponding to the current LOD layer, that is, after step 102, the method for the decoder to perform decoding processing may further include the following steps:

[0252] Step 105: When the first identification information indicates that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer according to the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0253] In an embodiment of the present application, after decoding the code stream and determining the first identification information corresponding to the current LOD layer, when the first identification information indicates that the current LOD layer uses the first mode, it is also possible to further determine the geometric information of the reconstructed mesh of the current LOD layer based on the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, as well as the initial geometric information of the reconstructed mesh.

[0254] It should be noted that, in an embodiment of the present application, when determining the geometric information of the reconstructed mesh of the current LOD layer based on the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, and the initial geometric information of the reconstructed mesh, on the one hand, when the initial geometric information of the reconstructed mesh is greater than the geometric lower limit information and the initial geometric information of the reconstructed mesh is less than the geometric upper limit information, the initial geometric information of the reconstructed mesh is determined as the geometric information of the reconstructed mesh; on the other hand, when the initial geometric information of the reconstructed mesh is less than or equal to the geometric lower limit information, the first preset geometric information is determined as the geometric information of the reconstructed mesh; on the further hand, when the initial geometric information of the reconstructed mesh is greater than or equal to the geometric upper limit information, the second preset geometric information is determined as the geometric information of the reconstructed mesh.

[0255] It should be noted that in an embodiment of the present application, the first preset geometric information and the second preset geometric information can be pre-set or transmitted through decoding, wherein the first preset geometric information and the second preset geometric information are both greater than or equal to the mean of the geometric lower limit and less than or equal to the mean of the geometric upper limit.

[0256] That is to say, in an embodiment of the present application, the first mode can be a prediction mode that improves the quality of the geometric information of the reconstructed network through the fitting information corresponding to the current LOD layer, or it can be a prediction mode that improves the quality of the geometric information of the reconstructed network through the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer.

[0257] It can be understood that in an embodiment of the present application, based on the first mode, for the initial geometric information of the reconstructed network of the current LOD layer, the corresponding geometric upper limit information and geometric lower limit information can be used to perform a clip operation on the geometric information of the reconstructed mesh, so that the geometric information of the reconstructed mesh finally obtained can be controlled to be within the range of the geometric upper limit information and the geometric lower limit information.

[0258] It is understood that in the embodiments of the present application, based on the first mode, the initial geometric information of the reconstructed mesh of the current LOD layer that has completed the preliminary reconstruction can, on the one hand, be fitted using fitting parameters and fitting types to improve the reconstruction quality of the mesh geometric information; on the other hand, the reconstructed mesh information can also be clipped. That is, for each LOD layer, a maximum value (geometric upper limit information) and a minimum value (geometric lower limit value) of each dimension of the current LOD layer are transmitted, and the final reconstructed value of the current LOD layer must be within the minimum and maximum values.

[0259] It is understandable that in the embodiments of the application, the above two solutions for improving the quality of reconstructed geometric information based on the first mode have no restrictions or order of precedence and can be used separately or in combination.

[0260] Furthermore, in an embodiment of the present application, when the first identification information indicates that the current LOD layer uses the first mode, the code stream can also be decoded to determine the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer.

[0261] That is to say, in the embodiment of the present application, the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer can be transmitted to the decoding end through the code stream.

[0262] Furthermore, in an embodiment of the present application, it is also possible to choose to perform correction processing on the geometric information of the reconstructed mesh of the current LOD layer according to the geometric upper limit information and the geometric lower limit information.

[0263] It can be understood that in the embodiments of the present application, regardless of whether the geometric information of the reconstructed mesh of the current LOD layer is determined through fitting processing, the corresponding geometric upper limit information and geometric lower limit information can be used to correct the geometric information of the reconstructed mesh of the current LOD layer, thereby further improving the quality of the reconstructed geometric information.

[0264] Exemplarily, in some embodiments, when the first identification information indicates that the current LOD layer uses the first mode, after fitting the initial set information of the reconstructed network using the fitting parameters corresponding to the current LOD layer, it is also possible to choose to clip the geometric information of the reconstructed mesh again according to the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, so as to further improve the quality of the reconstructed geometric information.

[0265] It should be noted that, in the embodiments of the present application, the decoder for performing decoding processing may include a video decoder and a grid decoder.

[0266] For example, in some embodiments, the bitstream may be decoded by a mesh decoder to determine the basic mesh corresponding to the current LOD layer.

[0267] It should be noted that, in the embodiment of the present application, for intra-frame encoding and decoding, the grid decoder can decode the code stream to obtain the basic grid corresponding to the current LOD layer.

[0268] Accordingly, in an embodiment of the present application, for intra-frame encoding and decoding, the grid decoder can receive the code stream of the basic grid corresponding to the current LOD layer transmitted by the encoding end, and determine the basic grid corresponding to the current LOD layer by decoding the code stream of the basic grid corresponding to the current LOD layer.

[0269] Exemplarily, in some embodiments, the bitstream may be decoded by a grid decoder to determine the motion vector information corresponding to the first basic grid.

[0270] For example, in some embodiments, the bitstream may be decoded by a video decoder to determine the shift coefficient and / or fitting information corresponding to the current LOD layer.

[0271] To summarize, through the decoding method proposed in the above steps 101 to 105, it is possible to choose to reconstruct the mesh geometric information (initial geometric information of the reconstructed mesh) using the geometric position information after the base mesh (Base Mesh) division and the displacement coefficient (Displacement component), and then use the correlation with the original mesh geometric position information to post-process the geometric reconstruction information of the mesh (initial geometric information of the reconstructed mesh), that is, use the corresponding fitting parameters to fit the initial geometric information of the reconstructed mesh, so that the reconstruction quality of the geometric information of each LOD layer point can be improved according to the relationship between the geometric position information of each LOD layer point and the geometric position information of the original point, thereby achieving the improvement of the coding efficiency of the mesh geometric position information under the premise of encoding one component.

[0272] Figure 21 is a schematic diagram of the geometric information after fitting processing. As shown in Figure 21, taking a part of the frames of the soldier_voxelized sequence as an example, the error between the reconstructed points and the original points of each LOD layer after linear fitting shows that the error between the geometric information of the reconstructed points and the geometric information of the original points after linear fitting is reduced relative to the error of the original encoding scheme, that is, the quality of the geometric information of the reconstructed mesh is improved.

[0273] The embodiment of the present application provides a decoding method, at the decoding end, the decoder decodes the code stream, determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; decodes the code stream, determines the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh. It can be seen that in the embodiment of the present application, if the first identification information corresponding to the current LOD layer indicates that the current LOD layer uses the first mode, then after completing the reconstruction of the mesh of the current LOD layer, the initial geometric information of the reconstructed mesh can be fitted, thereby obtaining geometric information of the reconstructed mesh with better quality. That is, in the embodiment of the present application, based on the first mode, after completing the preliminary reconstruction of the mesh, the quality of the geometric information can be improved by fitting the reconstructed geometric information, thereby ensuring the quality of the geometric information of the mesh while improving the coding efficiency of the geometric information, thereby improving the compression performance of the mesh.

[0274] An embodiment of the present application proposes an encoding method. FIG22 is a schematic diagram 1 of the encoding method proposed in the embodiment of the present application. As shown in FIG22 , in the embodiment of the present application, the encoding method of the encoder may include the following steps:

[0275] Step 201: Determine a shift coefficient corresponding to the current LOD layer, and determine initial geometric information of a reconstructed mesh corresponding to the current LOD layer according to the shift coefficient.

[0276] In an embodiment of the present application, the encoder may first determine a shift coefficient corresponding to the current LOD layer, and then may further determine initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient.

[0277] It should be noted that the encoding method in the embodiment of the present application may refer to an encoding method for geometric information, more specifically, an encoding method for geometric information in a dynamic mesh, wherein the encoding method may be applied to an encoder in a V-DMC, but is not limited thereto.

[0278] Furthermore, in an embodiment of the present application, the encoding of V-DMC can be mainly divided into two categories: geometric position information encoding and attribute information encoding. As shown in Figure 5, each frame file of the sequence basketball_player includes two files: basketball_player_fr0001_qp12_qt12.obj and basketball_player_fr0002.png. Among them, basketball_player_fr0001_qp12_qt12.obj contains four types of information: geometric position information (x, y, z), the connection relationship of geometric position information triangles, texture coordinates (u, v) and the connection relationship of texture coordinates. basketball_player_fr0002.png represents the texture attribute information of the current image. In the current V-DMC encoder, the geometric position information is jointly encoded using DRACO and Video Codec (AVC, HEVC or VVC), and the texture information encoding is directly encoded using Video Codec.

[0279] It should be noted that in the embodiments of the present application, the encoding of geometric information can be divided into: encoding of position information (geometric position information and texture position information) and encoding of connection relationships (connection relationships of triangle patches of geometric position information and connection relationships of texture position information).

[0280] For intra-frame mode, after obtaining the base mesh, the DRACO encoder is used to encode the base mesh's geometric information. The entire DRACO encoding process mainly includes: first, completing the encoding of the connection relationship, then encoding the geometric position information of the points based on the geometric connection relationship, and finally encoding the texture position information based on the connection relationship and geometric position information.

[0281] For the inter-frame mode, after obtaining the base mesh, you can choose to use the base mesh of the reference image of the current image to obtain the mesh information of the current image. Since the current image and the reference image have the same connection relationship and uv texture coordinates, only the geometric position information is different. Therefore, the geometric position information encoding between frames only needs to encode the geometric position information (x, y, z) of the current base mesh, and does not need to encode the connection relationship and texture position information (u, v).

[0282] It can be understood that, in the embodiments of the present application, the current image can be understood as the current frame, and the reference image can be understood as the reference frame.

[0283] Furthermore, in an embodiment of the present application, a lifting transformation can be performed on the shift coefficient corresponding to the current LOD layer to determine the lifting transformation coefficient; the lifting transformation coefficient can be quantized to determine the quantization coefficient of the current LOD layer; the quantization coefficient of the current LOD layer can be reorganized to determine the two-dimensional image of the current LOD layer; the two-dimensional image of the current LOD layer can be encoded, and the obtained encoding bits can be written into the bitstream.

[0284] Exemplarily, the implementation steps of the shift coefficient encoding method provided in the embodiment of the present application may include:

[0285] First, the base mesh is iteratively partitioned using a specific algorithm to obtain the corresponding mesh position information. The specific partitioning algorithm is consistent with the previous content, using linear interpolation of vertices on each boundary to obtain the corresponding geometric position information. Assuming that the entire partitioning is iterated N times, the displacements obtained from different iterative partitioning are then divided into Level of Dimensions (LODs) to obtain the corresponding mesh geometric position information. The error between the initial geometric position information and the original mesh is calculated to obtain the displacement of each point.

[0286] Secondly, the shift coefficients are subjected to lifting wavelet transform based on the LOD spatial structure, which includes two steps: prediction and update. The prediction step is shown in formula (5).

[0287] The update steps are shown in formula (6). Finally, the transformed coefficients are quantized and reorganized. The current V-DMC performs coefficient reorganization based on blocks, each of which is 16×16 in size. The coefficients within each block are arranged according to the Morton code to obtain the corresponding two-dimensional image. After completing a series of operations, the two-dimensional image is finally encoded using the Video Codec.

[0288] Exemplarily, the shift coefficients may be directly encoded using an entropy encoder to obtain a shift coefficient code stream.

[0289] Furthermore, in an embodiment of the present application, after determining the shift coefficient corresponding to each point in the current LOD layer, the preliminary reconstruction of the current LOD layer can be completed according to the shift coefficient, thereby determining the initial geometric information of the reconstructed grid corresponding to the current LOD layer.

[0290] It should be noted that in an embodiment of the present application, when determining the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient, it is possible to choose to determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient and the geometric information of the initial mesh corresponding to the current LOD layer.

[0291] It is understandable that, in the embodiment of the present application, the geometric information of the initial mesh corresponding to the current LOD layer may be determined first.

[0292] Furthermore, in an embodiment of the present application, when determining the geometric information of the initial mesh corresponding to the current LOD layer, the basic mesh corresponding to the current LOD layer can be determined first; and then the basic mesh is subdivided to determine the geometric information of the initial mesh corresponding to the current LOD layer.

[0293] It should also be noted that, in the embodiment of the present application, the basic grid can also be called a "simplified grid".

[0294] In some embodiments, for the intra mode, determining the base mesh of the current LOD layer may include: determining the base mesh of the current LOD layer.

[0295] In some embodiments, for the inter-frame mode, determining the base grid of the current LOD layer may include: determining the base grid of the current LOD layer of the current image based on the base grid of the reference LOD layer of the reference image.

[0296] It should also be noted that, in the embodiment of the present application, the reference image is an encoded frame before the current image. For example, the reference image can be the frame before the current image, but this is not specifically limited.

[0297] Furthermore, when performing subdivision processing on the basic grid to determine the geometric information of the initial grid corresponding to the current image, it is possible to iteratively divide the basic grid according to a grid subdivision mode to determine the geometric information of the initial grid.

[0298] It should be noted that, in the embodiment of the present application, the grid subdivision mode may include: a subdivision algorithm and a subdivision iteration number, wherein the subdivision algorithm is a linear interpolation algorithm.

[0299] It should be noted that, in the embodiment of the present application, by dividing the current image into levels of detail (LOD), at least one LOD layer can be determined; wherein, the at least one LOD layer can include the current LOD layer.

[0300] For example, as shown in FIG13A , the geometric information of the initial mesh obtained by subdividing the base mesh three times is considered to be the 0th iteration corresponding to the 0th layer (level 0). The newly added vertices in the first iteration constitute the 1st layer (level 1), the newly added vertices in the second iteration constitute the 2nd layer (level 2), and the newly added vertices in the third iteration constitute the 3rd layer (level 3). The specific LOD division structure is shown in FIG13B . The top layer is the base mesh. As the iteration proceeds, the number of newly added vertices in each iteration increases successively, forming a pyramid structure. In this way, by subdividing the base mesh, the geometric information of the initial mesh of the current LOD layer in the current image can be determined.

[0301] In some embodiments, subdividing the base grid to determine the geometric information of the initial grid of the current LOD layer in the current image may include: iteratively dividing the base grid according to a grid subdivision mode to determine the geometric information of the initial grid of the current LOD layer in the current image.

[0302] Specifically, in the embodiments of the present application, the mesh subdivision mode can be understood as upsampling the vertices on each boundary of the base mesh, or can also be understood as interpolating the vertices on each boundary of the base mesh. Exemplarily, the mesh subdivision mode includes a subdivision algorithm and a number of subdivision iterations. In some embodiments, the subdivision algorithm can be an interpolation algorithm, for example, a linear interpolation algorithm, or a nonlinear interpolation algorithm, which is not specifically limited herein.

[0303] Here, after the base mesh is reconstructed, a subdivision algorithm is used to subdivide the base mesh to obtain an initial mesh. For example, the base mesh can be used to obtain an initial mesh (also called a "subdivided mesh") through a linear interpolation algorithm. In each iteration, the coordinates of the newly inserted points are obtained by linear interpolation based on the two vertices on the current boundary, as shown in Formula (7).

[0304] Among them, pos1 and pos2 are the geometric position coordinates of the two end vertices on the current boundary participating in this iteration, pos new The geometric position coordinates of the new vertices added for this iteration.

[0305] It should be noted that, in the embodiments of the present application, the encoder may be a video encoder, or any encoding device including a video encoder and a grid encoder.

[0306] Furthermore, in an embodiment of the present application, the code stream transmitted from the encoder to the decoder may be a code stream including the first identification information, or may be code stream data including the code stream including the first identification information and a code stream of a simplified grid (basic grid) (or a code stream of a motion vector).

[0307] It is understandable that, in the embodiments of the present application, the current image may be a current image frame or a current video frame, which is not specifically limited in the present application.

[0308] Step 202: Determine whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer.

[0309] In an embodiment of the present application, after the encoder determines the shift coefficient corresponding to the current LOD layer and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient, it can further determine whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer.

[0310] Furthermore, in an embodiment of the present application, when judging whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer, the initial geometric information of the reconstructed mesh and the geometric information of the original mesh can be fitted first to determine the geometric information of the fitted predicted mesh; then the first error can be determined based on the geometric information of the predicted mesh and the geometric information of the original mesh, and the second error can be determined based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh. Finally, it can be judged whether the current LOD layer uses the first mode based on the first error and the second error.

[0311] It should be noted that, in an embodiment of the present application, when determining whether the current LOD layer uses the first mode based on the first error and the second error, the error parameters of the first error and the second error can be determined first; when the error parameter is less than the preset threshold value, it is determined that the current LOD layer uses the first mode; when the error parameter is greater than or equal to the preset threshold value, it is determined that the current LOD layer does not use the first mode.

[0312] It can be understood that in the embodiments of the present application, the error parameter can be used to further determine the correlation between the first error and the second error, wherein the error parameter can be obtained by calculating the first error and the second error, for example, the error parameter can be the difference between the first error and the second error, or it can be the ratio between the first error and the second error.

[0313] For example, in some embodiments, the first error and the second error can be calculated by a certain distortion. Here, the mean square error (MSE) algorithm can be used to obtain the error of the geometric position information of the mesh at different LOD layers, as shown below:

[0314] That is to say, the error between the reconstructed mesh obtained by the original encoding method and the original mesh can be calculated first, that is, the second error, which is represented by Dist_org; secondly, the method of the embodiment of the present application is used: the geometric position information of the initial mesh obtained by dividing the Base mesh is used and the displacement coefficient (Displacement coefficient) is used to reconstruct the geometric information of the predicted mesh and the original mesh, that is, the first error, which is represented by Dist_skip.

[0315] In this way, the error ratio can be expressed as Dist_skip / Dist_org. If Dist_skip / Dist_org is less than a preset threshold, it can be determined that the current LOD layer uses the first mode; if Dist_skip / Dist_org is greater than or equal to the preset threshold, it can be determined that the current LOD layer does not use the first mode.

[0316] It should be noted that, in an embodiment of the present application, at the encoding end, the error (i.e., the first error) between the geometric position information of the reconstructed mesh (the geometric information of the predicted mesh) and the original position information (the geometric information of the original mesh) obtained after parameter fitting of different LOD layers can be compared, and the error (second error) between the original encoding scheme and the geometric position information of the reconstructed mesh can be calculated. When the error after fitting (the first error) is less than the error of the original encoding scheme (the second error), it is considered that the reconstructed geometric information of the current LOD layer point can be fitted, that is, it is determined that the current LOD layer uses the first mode. Otherwise, there is no need to further fit the geometric information of the reconstructed mesh point of the current LOD layer, that is, it is determined that the current LOD layer does not use the first mode.

[0317] It should be noted that, in the embodiment of the present application, the first mode may represent a prediction mode for optimizing and correcting geometric information of the reconstructed grid.

[0318] Accordingly, in the embodiments of the present application, the first mode may be understood as a prediction mode for acquiring geometric information of the reconstructed mesh of the current LOD layer by fitting parameters.

[0319] Step 203: When it is determined that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, the first identification information corresponding to the current LOD layer is set, and the first identification information is written into the bitstream.

[0320] In an embodiment of the present application, when it is determined that the current LOD layer uses the first mode, on the one hand, the geometric information of the reconstructed mesh of the current LOD layer can be determined based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh. At the same time, the first identification information corresponding to the current LOD layer can be set, and the first identification information can be written into the bitstream.

[0321] It should be noted that, in the embodiment of the present application, the first identification information can be used to determine whether the current LOD layer in the current image uses the first mode.

[0322] It should be noted that, in the embodiment of the present application, the first mode may represent a prediction mode for optimizing and correcting geometric information of the reconstructed grid.

[0323] Accordingly, in the embodiments of the present application, the first mode may be understood as a prediction mode for acquiring geometric information of the reconstructed mesh of the current LOD layer by fitting parameters.

[0324] It should be noted that, in an embodiment of the present application, after determining whether the current LOD layer uses the first mode, the value of the first identification information can be set to indicate whether the current LOD layer uses the first mode.

[0325] Exemplarily, in some embodiments, if the value of the first identification information is a first value, then it can be determined that the current LOD layer corresponding to the current image uses the first mode; if the value of the first identification information is a second value, then it can be determined that the current LOD layer corresponding to the current image does not use the first mode.

[0326] It should be noted that, in the embodiment of the present application, the first value and the second value may be in parameter form or in digital form. For example, the first value may be 1 and the second value may be 0.

[0327] Furthermore, in an embodiment of the present application, before determining the first identification information, the second identification information corresponding to the current sequence may be determined first, wherein the second identification information may be used to determine whether the current sequence allows the use of the first mode, that is, whether the prediction mode corresponding to the current sequence includes the first mode. If the second identification information corresponding to the current sequence indicates that the current sequence allows the use of the first mode, then it can be considered that the prediction mode used when predicting the current sequence frame may include the first mode, that is, for any image frame or video frame in the current sequence, a prediction mode that optimizes and corrects the geometric information of the reconstructed grid may be used.

[0328] It should be noted that, in the embodiment of the present application, after determining whether the current sequence allows the use of the first mode, the value of the second identification information can be set to indicate whether the current sequence allows the use of the first mode.

[0329] Exemplarily, in some embodiments, if the value of the second identification information is the third value, it can be determined that the current sequence allows the use of the first mode; if the value of the second identification information is the fourth value, it can be determined that the current sequence does not allow the use of the first mode.

[0330] It should be noted that, in the embodiment of the present application, the third value and the fourth value may be in parameter form or in digital form. For example, the third value may be 1 and the fourth value may be 0.

[0331] It is understandable that, in an embodiment of the present application, when the second identification information indicates that the current sequence allows the use of the first mode, the third identification information corresponding to the current image in the current sequence can be further determined; wherein the third identification information can be used to determine whether the current image allows the use of the first mode, that is, whether the prediction mode corresponding to the current image includes the first mode. If the third identification information corresponding to the current image indicates that the current image allows the use of the first mode, then it can be considered that the prediction mode used when performing prediction processing on the current image can include the first mode, that is, for any image frame or video frame in the current image, a prediction mode that optimizes and corrects the geometric information of the reconstructed grid can be used.

[0332] It should be noted that, in the embodiment of the present application, after determining whether the current image allows the use of the first mode, the value of the third identification information can be set to indicate whether the current image allows the use of the first mode.

[0333] Exemplarily, in some embodiments, if the value of the third identification information is the fifth value, it can be determined that the current image allows the use of the first mode; if the value of the third identification information is the sixth value, it can be determined that the current image does not allow the use of the first mode.

[0334] It should be noted that, in the embodiment of the present application, the fifth value and the sixth value may be in parameter form or in digital form. For example, the fifth value may be 1 and the sixth value may be 0.

[0335] Furthermore, in an embodiment of the present application, after the encoder completes the determination of the first identification information, the second identification information, and the third identification information, it may further write the first identification information, the second identification information, and the third identification information into the code stream and transmit it to the decoding end.

[0336] That is, in an embodiment of the present application, the identification information for whether the first mode can be used can be determined by parsing layer by layer, that is, the encoding end can choose to set the identification information indicating whether the first mode can be used layer by layer from the upper layer to the lower layer. For example, first, the second identification information corresponding to the VFPS (Vmesh Frames Parameters Set) indicates whether the first mode is allowed to be used in the current sequence; secondly, after the FPS (Frame Parameters Set) inherits the VFPS, it determines whether the first mode is allowed to be used for the current image.

[0337] For example, in some embodiments, for a GOF->Frame->LOD encoding structure, the corresponding syntax elements are VFPS->FPS->LODPS. First, the corresponding syntax element of the GOF (Group of Frame) determines whether skip coding is enabled for the current GOF. Then, the Frame inherits the GOF syntax and can also be updated. Finally, syntax is added at the LOD level to determine whether the current LOD layer is skipped, and the syntax elements can also be updated. Ultimately, the shift coefficient decoding of each LOD layer proceeds layer by layer through VFPS->FPS->LODPS.

[0338] Furthermore, in an embodiment of the present application, when it is determined that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer can be determined based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0339] It should be noted that in the embodiment of the present application, based on the first mode, the initial geometric information of the reconstructed mesh that has been reconstructed in the current LOD layer needs to be fitted using the fitting information corresponding to the current LOD layer, so as to complete the correction and optimization of the initial geometric information of the reconstructed mesh and obtain the geometric information of the reconstructed mesh of the current LOD layer with better quality.

[0340] Furthermore, in an embodiment of the present application, when it is determined that the current LOD layer uses the first mode, the fitting information corresponding to the current LOD layer can also be selected to be written into the bitstream. The fitting information is determined during the fitting process of the initial geometric information of the reconstructed mesh and the geometric information of the original mesh.

[0341] That is to say, in the embodiment of the present application, the encoding end can write the fitting parameters into the code stream and transmit it to the decoding end.

[0342] It should be noted that, in an embodiment of the present application, the fitting information may include a fitting type and fitting parameters corresponding to the fitting type; wherein the fitting type includes linear fitting, nonlinear fitting, neural grid-based fitting, curve fitting, and convolution parameter fitting.

[0343] For example, when the fitting type includes linear fitting, the fitting parameters may include a slope and / or an intercept of the linear function.

[0344] Exemplarily, when the fitting type includes nonlinear fitting, the fitting parameter may include at least one constant in the nonlinear function. For example, if the nonlinear function is an exponential function, the fitting parameter may include the constant a of the nonlinear function; if the nonlinear function is a polynomial function, the fitting parameter may include the coefficients a_0, a_1, a_2, ... of the polynomial function; and if the nonlinear function is a logarithmic function, the fitting parameter may include the base a of the nonlinear function.

[0345] Illustratively, in the embodiment of the present application, the fitting type may also include a linear function type, an exponential function type, a logarithmic function type, a polynomial function type, etc., which are not specifically limited here.

[0346] Furthermore, in an embodiment of the present application, at the encoding end, fitting information such as the fitting type and fitting parameters may be determined through the correlation between the original grid and the reconstructed grid.

[0347] Exemplarily, in some embodiments, as shown in Figure 18, at the encoding end, after determining the geometric information of the reconstructed grid, it is possible to select fitting information for constructing a mapping relationship between the initial geometric information and the predicted geometric information based on the parameter correlation between the geometric position information of the midpoint of the reconstructed grid and the geometric position information of the midpoint of the original grid.

[0348] Furthermore, in an embodiment of the present application, when determining the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, it is possible to choose to determine the geometric information of the reconstructed mesh of the current LOD layer based on the fitting type, fitting parameters and the initial geometric information of the reconstructed mesh.

[0349] It should be noted that in an embodiment of the present application, when determining the geometric information of the reconstructed mesh of the current LOD layer based on the fitting type, fitting parameters and the initial geometric information of the reconstructed mesh, the mapping relationship between the initial geometric information and the predicted geometric information can be determined first based on the fitting type and fitting parameters, and then the geometric information of the reconstructed mesh corresponding to the initial geometric information of the reconstructed mesh can be determined based on the mapping relationship between the initial geometric information and the predicted geometric information.

[0350] Exemplarily, in some embodiments, a mapping relationship between initial geometric information and predicted geometric information is constructed based on fitting information including fitting type and fitting parameters, wherein the mapping relationship is as shown in formula (19).

[0351] Among them, lvl represents different LOD layers, reconMesh represents the initial geometric information, that is, the initial mesh information, predict mesh It represents the use of a certain functional relationship to obtain the corresponding predicted geometric position information, that is, the predicted geometric information.

[0352] For example, in an embodiment of the present application, considering the balance between coding complexity and coding efficiency, a simple linear fit can be used to obtain the relationship between the reconstructed mesh geometric position information and the initial mesh position information, as shown in formula (20). Where * and + represent vector multiplication and addition. That is, the fitting type is linear fitting, and the fitting parameters are k and b.

[0353] Furthermore, in an embodiment of the present application, after determining the mapping relationship between the initial geometric information and the predicted geometric information based on the fitting type and fitting parameters, the mapping relationship can be used to determine the geometric information of the reconstructed grid after fitting corresponding to the initial geometric information of the reconstructed grid.

[0354] That is to say, in an embodiment of the present application, based on the spatial structure of LOD, after the position information of the reconstruction point of the current LOD layer needs to be fitted according to the corresponding first identification information, the final reconstruction point can be obtained through fitting processing, that is, the fitting parameters will be used to post-process the position information of the current LOD layer point, and the position information of the current LOD layer point will be reconstructed and restored.

[0355] Exemplarily, in some embodiments, the linear fitting error and the error of the original coding scheme are first compared at the encoding end. When the former is smaller than the latter, it is determined to use the first mode. At this time, the first identification information needs to be transmitted to the decoding end. For example, a flag is transmitted to the current LOD layer: posFittingFlag. When the symbol is true, it means that linear fitting is required, that is, the first mode is used. Otherwise, linear fitting is not performed, that is, the first mode is not used. When the symbol is true, since linear fitting is used, it is necessary to transmit fitting information including the fitting type (linear fitting) and fitting parameters (such as parameters k and b) at the current LOD layer. Finally, the decoding end uses the fitting parameter flag of each LOD layer, that is, the first identification information to determine whether the current LOD layer needs to be fitted.

[0356] Furthermore, in an embodiment of the present application, FIG23 is a second schematic diagram of the encoding method proposed in an embodiment of the present application. As shown in FIG23 , after determining whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer, that is, after step 202, the method for the encoder to perform encoding processing may further include the following steps:

[0357] Step 204: If it is determined that the current LOD layer does not use the first mode, the initial geometric information of the reconstructed mesh is determined as the geometric information of the reconstructed mesh of the current LOD layer, the first identification information corresponding to the current LOD layer is set, and the first identification information is written into the bitstream.

[0358] In an embodiment of the present application, when it is determined that the current LOD layer does not use the first mode, on the one hand, the initial geometric information of the reconstructed mesh may not be fitted, but the initial geometric information of the reconstructed mesh may be directly determined as the geometric information of the reconstructed mesh of the current LOD layer. At the same time, the first identification information corresponding to the current LOD layer may be set, and the first identification information may be written into the bitstream.

[0359] That is to say, in an embodiment of the present application, based on the spatial structure of LOD, after determining that the position information of the reconstruction point of the current LOD layer does not need to be fitted, the reconstruction point of the current LOD layer can be directly used as the position information of the current LOD layer point.

[0360] Furthermore, in an embodiment of the present application, FIG24 is a third schematic diagram of the encoding method proposed in an embodiment of the present application. As shown in FIG24 , after determining the shift coefficient corresponding to the current LOD layer and determining the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient, that is, after step 201, the method for the decoder to perform decoding processing may further include the following steps:

[0361] Step 205: Determine whether the current LOD layer uses the first mode based on the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0362] In an embodiment of the present application, after completing the preliminary reconstruction of the mesh of the current LOD layer, it is also possible to first determine whether the current LOD layer uses the first mode based on the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, as well as the initial geometric information of the reconstructed mesh.

[0363] Furthermore, in the example of the present application, when judging whether the current LOD layer uses the first mode based on the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, and the initial geometric information of the reconstructed mesh, if the initial geometric information of the reconstructed mesh is less than or equal to the geometric lower limit information, or the initial geometric information of the reconstructed mesh is greater than or equal to the geometric upper limit information, it can be determined whether the current LOD layer uses the first mode.

[0364] Step 206: When it is determined that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined based on the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, and the initial geometric information of the reconstructed mesh, the first identification information corresponding to the current LOD layer is set, and the first identification information is written into the bitstream.

[0365] In an embodiment of the present application, when it is determined that the current LOD layer uses the first mode, it is also possible to further determine the geometric information of the reconstructed mesh of the current LOD layer based on the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, as well as the initial geometric information of the reconstructed mesh.

[0366] It should be noted that, in an embodiment of the present application, when determining the geometric information of the reconstructed mesh of the current LOD layer based on the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, and the initial geometric information of the reconstructed mesh, on the one hand, when the initial geometric information of the reconstructed mesh is greater than the geometric lower limit information and the initial geometric information of the reconstructed mesh is less than the geometric upper limit information, the initial geometric information of the reconstructed mesh is determined as the geometric information of the reconstructed mesh; on the other hand, when the initial geometric information of the reconstructed mesh is less than or equal to the geometric lower limit information, the first preset geometric information is determined as the geometric information of the reconstructed mesh; on the further hand, when the initial geometric information of the reconstructed mesh is greater than or equal to the geometric upper limit information, the second preset geometric information is determined as the geometric information of the reconstructed mesh.

[0367] It should be noted that in the embodiment of the present application, the first preset geometric information and the second preset geometric information are both greater than or equal to the mean of the geometric lower limit and less than or equal to the mean of the geometric upper limit. The first preset geometric information and the second preset geometric information can also be transmitted to the decoding end through the code stream.

[0368] That is to say, in an embodiment of the present application, the first mode can be a prediction mode that improves the quality of the geometric information of the reconstructed network through the fitting information corresponding to the current LOD layer, or it can be a prediction mode that improves the quality of the geometric information of the reconstructed network through the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer.

[0369] It can be understood that in an embodiment of the present application, based on the first mode, for the initial geometric information of the reconstructed network of the current LOD layer, the corresponding geometric upper limit information and geometric lower limit information can be used to perform a clip operation on the geometric information of the reconstructed mesh, so that the geometric information of the reconstructed mesh finally obtained can be controlled to be within the range of the geometric upper limit information and the geometric lower limit information.

[0370] It is understood that in the embodiments of the present application, based on the first mode, the initial geometric information of the reconstructed mesh of the current LOD layer that has completed the preliminary reconstruction can, on the one hand, be fitted using fitting parameters and fitting types to improve the reconstruction quality of the mesh geometric information; on the other hand, the reconstructed mesh information can also be clipped. That is, for each LOD layer, a maximum value (geometric upper limit information) and a minimum value (geometric lower limit value) of each dimension of the current LOD layer are transmitted, and the final reconstructed value of the current LOD layer must be within the minimum and maximum values.

[0371] It is understandable that in the embodiments of the application, the above two solutions for improving the quality of reconstructed geometric information based on the first mode have no restrictions or order of precedence and can be used separately or in combination.

[0372] Furthermore, in an embodiment of the present application, when it is determined that the current LOD layer uses the first mode, the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer may also be written into the bitstream.

[0373] That is to say, in the embodiment of the present application, the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer can be transmitted to the decoding end through the code stream.

[0374] Furthermore, in an embodiment of the present application, it is also possible to choose to perform correction processing on the geometric information of the reconstructed mesh of the current LOD layer according to the geometric upper limit information and the geometric lower limit information.

[0375] It can be understood that in the embodiments of the present application, regardless of whether the geometric information of the reconstructed mesh of the current LOD layer is determined through fitting processing, the corresponding geometric upper limit information and geometric lower limit information can be used to correct the geometric information of the reconstructed mesh of the current LOD layer, thereby further improving the quality of the reconstructed geometric information.

[0376] Exemplarily, in some embodiments, when the first identification information indicates that the current LOD layer uses the first mode, after fitting the initial set information of the reconstructed network using the fitting parameters corresponding to the current LOD layer, it is also possible to choose to clip the geometric information of the reconstructed mesh again according to the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, so as to further improve the quality of the reconstructed geometric information.

[0377] It should be noted that, in the embodiments of the present application, the encoder used to perform encoding processing may include a video encoder and a grid encoder.

[0378] Exemplarily, in some embodiments, the base mesh corresponding to the current LOD layer may be written into the bitstream via a mesh encoder.

[0379] It should be noted that, in the embodiment of the present application, for intra-frame encoding and decoding, the grid encoder can write the basic grid corresponding to the current LOD layer into the code stream and transmit it to the decoding end.

[0380] Accordingly, in an embodiment of the present application, for intra-frame encoding and decoding, the grid decoder can receive the code stream of the basic grid corresponding to the current LOD layer transmitted by the encoding end, and determine the basic grid corresponding to the current LOD layer by decoding the code stream of the basic grid corresponding to the current LOD layer.

[0381] Exemplarily, in some embodiments, the shift coefficient and / or fitting information corresponding to the current LOD layer may be written into the bitstream through a video encoder.

[0382] Exemplarily, in some embodiments, the shift coefficient and / or fitting information corresponding to the current LOD layer may be written into the bitstream through a video encoder.

[0383] In summary, through the encoding method proposed in the above steps 201 to 206, it is possible to choose to reconstruct the mesh geometric information (initial geometric information of the reconstructed mesh) using the geometric position information after the base mesh (Base Mesh) division and the displacement coefficient (Displacement component), and then use the correlation with the original mesh geometric position information to post-process the geometric reconstruction information of the mesh (initial geometric information of the reconstructed mesh), that is, use the corresponding fitting parameters to fit the initial geometric information of the reconstructed mesh, so that the reconstruction quality of the geometric information of each LOD layer point can be improved according to the relationship between the geometric position information of each LOD layer point and the geometric position information of the original point, thereby achieving the improvement of the encoding efficiency of the mesh geometric position information under the premise of encoding one component.

[0384] The embodiment of the present application provides an encoding method, in which, at the encoding end, the encoder determines a shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determines whether the current LOD layer uses a first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; if it is determined that the current LOD layer uses the first mode, the encoder determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, sets the first identification information corresponding to the current LOD layer, and writes the first identification information into the bitstream. It can be seen that in the embodiment of the present application, if the first identification information corresponding to the current LOD layer indicates that the current LOD layer uses the first mode, then after completing the reconstruction of the mesh of the current LOD layer, the initial geometric information of the reconstructed mesh can be fitted, thereby obtaining geometric information of a reconstructed mesh with better quality. That is, in the embodiment of the present application, based on the first mode, after completing the preliminary reconstruction of the mesh, the quality of the geometric information can be improved by fitting the reconstructed geometric information, thereby ensuring the quality of the geometric information of the mesh while improving the coding efficiency of the geometric information, thereby improving the compression performance of the mesh.

[0385] Based on the above embodiment, another embodiment of the present application proposes a coding and decoding method, which is applied to an encoder and a decoder, wherein the encoder includes a video encoder, a trellis encoder, and a preprocessor, and the decoder includes a video decoder and a trellis decoder.

[0386] It should be noted that, in the embodiments of the present application, the coding and decoding method can be used for intra-frame coding and decoding, and can also be used for inter-frame coding and decoding, which is not specifically limited in the present application.

[0387] It should be noted that, in the embodiment of the present application, the preprocessor can be used to generate a base grid and a shift vector according to the original grid of the current image.

[0388] It is understood that in the embodiments of the present application, during the preprocessing process, the original mesh of the current image can be simplified to obtain a simplified mesh (decimated mesh), or a base mesh. The base mesh can then be subdivided to obtain a subdivided mesh. Finally, for each vertex in the subdivided mesh, the point in the original mesh closest to it is found, and the displacement vector of the two points is calculated.

[0389] Furthermore, in an embodiment of the present application, after the preprocessor generates a corresponding basic grid based on the original grid, the grid encoder can be used to encode the basic grid and then generate a code stream of the basic grid.

[0390] It should be noted that, in the embodiment of the present application, for intra-frame coding, the grid encoder can encode the basic grid of the current image to obtain a code stream of the basic grid.

[0391] Accordingly, in an embodiment of the present application, for intra-frame coding, the video encoder may first determine the reconstructed basic network of the current image using the bitstream of the generated basic grid; and then may update the shift vector by reconstructing the basic grid.

[0392] That is to say, in an embodiment of the present application, after the grid encoder completes encoding of the basic grid of the current image and generates a code stream of the basic grid, the video encoder can use the code stream of the basic grid to complete the reconstruction of the basic grid, and use the reconstructed basic grid to complete the update of the shift vector.

[0393] It should be noted that, in the embodiment of the present application, for inter-frame coding, the grid encoder can encode the motion vector of the current image to obtain a code stream of the motion vector.

[0394] Accordingly, in an embodiment of the present application, for inter-frame coding, the video encoder can first use the code stream of the generated motion vector to determine the motion vector of the current image, and then further determine the reconstructed basic network of the current image based on the motion vector; finally, the shift vector can be updated by reconstructing the basic grid.

[0395] That is to say, in an embodiment of the present application, after the grid encoder completes encoding of the motion vector of the current image and generates a code stream of the motion vector, the video encoder can use the code stream of the motion vector to complete the reconstruction of the basic grid, and use the reconstructed basic grid to complete the update of the shift vector.

[0396] It can be understood that, in the embodiment of the present application, after the grid encoder generates the code stream of the basic grid or the code stream of the motion vector, it can transmit the code stream of the basic grid or the code stream of the motion vector to the decoding end.

[0397] The common V-DMC encoding process for reconstructing mesh geometry suffers from two lossy aspects: first, the displacement component uses a lifting transform, inherently a lossy encoding scheme; second, the linear conversion relationship between disp and delta is inherently lossy. Consequently, the quality of the reconstructed mesh geometry, derived from the initial mesh geometry obtained by partitioning the displacement component with the base mesh, is limited, resulting in a reduction in the quality of the existing mesh geometry.

[0398] Furthermore, the encoding and decoding method proposed in the embodiment of the present application can use the correlation between the initial set information of the reconstructed mesh and the original mesh geometric position information (geometric information of the original mesh) to post-process the geometric reconstruction information of the mesh after reconstructing the mesh geometric information (initial set information of the reconstructed mesh) based on the geometric position information after the Base Mesh division (geometric information of the initial mesh) and the Displacement component (displacement coefficient) to obtain the geometric information of the final reconstructed mesh, so as to improve the reconstruction quality of the geometric information of each LOD layer point according to the relationship between the geometric position information of each LOD layer point and the geometric position information of the original point, and also achieve the improvement of the coding efficiency of the mesh geometric position information under the premise of encoding only one component.

[0399] It can be understood that in an embodiment of the present application, as shown in Figure 18, at the encoding end, after determining the geometric information of the reconstructed grid, it is possible to select the fitting information for constructing the mapping relationship between the initial geometric information and the predicted geometric information based on the parameter correlation between the geometric position information of the midpoint of the reconstructed grid and the geometric position information of the midpoint of the original grid.

[0400] That is to say, in an embodiment of the present application, a certain algorithm can be used to determine the fitting information used for fitting processing in the first mode by fitting the relationship between the geometric position information of the initial mesh reconstruction points (the initial set information of the reconstructed mesh) and the geometric position information of the original mesh (the geometric information of the original mesh), wherein the fitting information includes the fitting type and the fitting parameters corresponding to the fitting type.

[0401] It should be noted that in an embodiment of the present application, the mapping relationship between the initial geometric information and the predicted geometric information can be determined based on the fitting type and the fitting parameters, and then the geometric information of the reconstructed grid corresponding to the initial geometric information of the reconstructed grid can be determined based on the mapping relationship between the initial geometric information and the predicted geometric information.

[0402] Exemplarily, in some embodiments, a mapping relationship between initial geometric information and predicted geometric information is constructed based on fitting information including fitting type and fitting parameters, wherein the mapping relationship is as shown in formula (19).

[0403] Among them, lvl represents different LOD layers, reconMesh represents the initial geometric information, that is, the initial mesh information, predict mesh It represents the use of a certain functional relationship to obtain the corresponding predicted geometric position information, that is, the predicted geometric information.

[0404] For example, in an embodiment of the present application, considering the balance between coding complexity and coding efficiency, a simple linear fit can be used to obtain the relationship between the reconstructed mesh geometric position information and the initial mesh position information, as shown in formula (20). Where * and + represent vector multiplication and addition. That is, the fitting type is linear fitting, and the fitting parameters are k and b.

[0405] In this way, through a certain distortion, for example, the MSE algorithm can be used to obtain the error of the grid geometric position information corresponding to different LOD layers, as shown in formula (21).

[0406] It should be noted that in an embodiment of the present application, the error between the geometric position information of the reconstructed mesh and the original position information is obtained after parameter fitting of different LOD layers at the encoding end, and the error between the original encoding scheme and the geometric position information of the reconstructed mesh is calculated. When the error after fitting is less than the error of the original encoding scheme, it is considered that the reconstructed geometric information of the current LOD layer point can be fitted, otherwise there is no need to further fit the geometric information of the reconstructed mesh point of the current LOD layer.

[0407] Exemplarily, in some embodiments, the linear fitting error and the error of the original coding scheme are first compared at the encoding end. When the former is smaller than the latter, it is determined to use the first mode. At this time, the first identification information needs to be transmitted to the decoding end. For example, a flag is transmitted to the current LOD layer: posFittingFlag. When the symbol is true, it means that linear fitting is required, that is, the first mode is used. Otherwise, linear fitting is not performed, that is, the first mode is not used. When the symbol is true, since linear fitting is used, it is necessary to transmit fitting information including the fitting type (linear fitting) and fitting parameters (such as parameters k and b) at the current LOD layer. Finally, the decoding end uses the fitting parameter flag of each LOD layer, that is, the first identification information to determine whether the current LOD layer needs to be fitted.

[0408] Furthermore, in an embodiment of the present application, the maximum and minimum values ​​of the geometric position information of the original mesh points of each LOD layer can also be obtained at the encoding end in each dimension, so the parameter value can also be transmitted to the decoding end. When the geometric information of the mesh points is reconstructed at the decoding end, the minimum and maximum values ​​can be used to perform a clip operation on the reconstructed position information of the current LOD layer point.

[0409] Correspondingly, at the decoding end, Video-Codec is used to parse and obtain a two-dimensional image, and the two-dimensional image is used to restore the displacement coefficient of each point. Finally, the geometric information of the mesh reconstruction points obtained after the Base Mesh division (the geometric information of the initial mesh) and the Displacement reconstruction are used to restore the position information of the reconstruction points of the current mesh. Secondly, based on the spatial structure of the LOD, first determine whether to use the first mode based on the identification information obtained by decoding, that is, determine whether the position information of the reconstruction point of the current LOD layer (the initial geometric information of the reconstructed mesh) needs to be fitted.

[0410] It can be understood that in the embodiments of the present application, based on the spatial structure of LOD, after the position information of the reconstruction point of the current LOD layer needs to be fitted according to the corresponding first identification information, the final reconstruction point can be obtained through fitting processing, that is, the fitting parameters will be used to post-process the position information of the current LOD layer point, and the position information of the current LOD layer point will be reconstructed and restored.

[0411] It can be understood that based on the spatial structure of LOD, after the position information of the reconstruction point of the current LOD layer does not need to be fitted according to the corresponding first identification information, the reconstruction point of the current LOD layer can be directly used as the position information of the current LOD layer point.

[0412] It can be seen that the encoding and decoding method proposed in the embodiment of the present application performs adaptive post-processing on the geometric position information of different LOD layer points by utilizing the relationship between the position information of different LOD layer points after the Base Mesh is divided and the original Mesh position information, thereby ensuring that the geometric information quality of the mesh is improved without the need to encode additional components of the Displacement coefficient.

[0413] On the one hand, at the encoding end, the LOD spatial structure is first obtained in a certain way. Secondly, based on the LOD spatial structure, the relationship between the initial mesh position information and the original mesh position information is fitted, and the error between the position information after fitting and the original position information is calculated. When the error after fitting is less than the error of the original encoding scheme, it is considered that the point geometric information in the current LOD layer can be fitted. Finally, the fitting parameters are used to post-process the geometric position information of the points in the current LOD layer, so that the reconstructed geometric position information after post-processing can be used as the final reconstructed position information of the current LOD layer.

[0414] It should be noted that, in the embodiments of the present application, there is no limitation on the fitting method or the use of linear fitting, curve fitting or convolution parameter fitting.

[0415] It can be understood that the encoding and decoding method proposed in this application is more about establishing the relationship between the geometric position information of the original points of the Base Mesh and the position information of the original mesh points, so that the reconstructed mesh geometric information can be linearly fitted based on the relationship between the reconstructed mesh and the original mesh, thereby improving the quality of the mesh geometric information and further improving the mesh geometric information encoding efficiency.

[0416] The embodiment of the present application provides a coding and decoding method. At the decoding end, the decoder decodes the bitstream, determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; decodes the bitstream, determines the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh. At the encoding end, the encoder determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determines whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; when it is determined that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, sets the first identification information corresponding to the current LOD layer, and writes the first identification information into the bitstream. Thus, it can be seen that in the embodiment of the present application, if the first identification information corresponding to the current LOD layer indicates that the current LOD layer uses the first mode, then after the reconstruction of the mesh of the current LOD layer is completed, the initial geometric information of the reconstructed mesh can be fitted, thereby obtaining geometric information of the reconstructed mesh with better quality. That is to say, in the embodiment of the present application, based on the first mode, after the preliminary reconstruction of the mesh is completed, the quality of the geometric information can be improved by fitting the reconstructed geometric information, thereby ensuring the quality of the geometric information of the mesh while improving the encoding efficiency of the geometric information, thereby improving the compression performance of the mesh.

[0417] Based on the above embodiment, in another embodiment of the present application, based on the same inventive concept as the above embodiment, FIG25 is a schematic diagram of the composition structure of the encoder. As shown in FIG25 , the encoder 100 may include: a first determining unit 111, an encoding unit 112, wherein:

[0418] The first determining unit 111 is configured to determine a shift coefficient corresponding to the current LOD layer, and determine initial geometric information of a reconstructed mesh corresponding to the current LOD layer according to the shift coefficient; determine whether the current LOD layer uses the first mode according to the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; and if it is determined that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh;

[0419] The encoding unit 112 is configured to set first identification information corresponding to the current LOD layer and write the first identification information into the bitstream.

[0420] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0421] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0422] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 100. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0423] Based on the composition of the above-mentioned encoder 100 and the computer-readable storage medium, Figure 26 is a second schematic diagram of the composition structure of the encoder. As shown in Figure 26, the encoder 100 may include: a first memory 121 and a first processor 122, a first communication interface 123 and a first bus system 124. The first memory 121, the first processor 122, and the first communication interface 123 are coupled together through the first bus system 124. It can be understood that the first bus system 124 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 124 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 124 in Figure 10. Among them,

[0424] The first communication interface 123 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0425] The first memory 121 is used to store computer programs that can be run on the first processor;

[0426] The first processor 122 is used to determine the shift coefficient corresponding to the current LOD layer when running the computer program, and determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; judge whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; when it is determined that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, set the first identification information corresponding to the current LOD layer, and write the first identification information into the bitstream.

[0427] It is understood that the first memory 121 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 121 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0428] The first processor 122 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 122. The above-mentioned first processor 122 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 121 , and the first processor 122 reads the information in the first memory 121 and completes the steps of the above method in combination with its hardware.

[0429] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0430] Optionally, as another embodiment, the first processor 122 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0431] FIG27 is a schematic diagram of the first structure of a decoder. As shown in FIG27 , the decoder 200 may include: a decoding unit 211 and a second determining unit 212; wherein,

[0432] The decoding unit 211 is configured to decode the code stream;

[0433] The second determination unit 212 is configured to determine the shift coefficient corresponding to the current LOD layer, and determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determine the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0434] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0435] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0436] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the decoder 200. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0437] Based on the composition of the above-mentioned decoder 200 and the computer-readable storage medium, Figure 28 is a second schematic diagram of the composition structure of the decoder. As shown in Figure 28, the decoder 200 may include: a second memory 221 and a second processor 222, a second communication interface 223 and a second bus system 224. The second memory 221 and the second processor 222, and the second communication interface 223 are coupled together through the second bus system 224. It can be understood that the second bus system 224 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 224 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as the second bus system 224 in Figure 12. Among them,

[0438] The second communication interface 223 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0439] The second memory 221 is used to store computer programs that can be run on the second processor;

[0440] The second processor 222 is used to decode the code stream when running the computer program, determine the shift coefficient corresponding to the current LOD layer, and determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determine the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

[0441] It is understood that the second memory 221 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The second memory 221 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0442] The second processor 222 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the second processor 222. The above-mentioned second processor 222 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the second memory 221 , and the second processor 222 reads the information in the second memory 221 and completes the steps of the above method in combination with its hardware.

[0443] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0444] Optionally, as another embodiment, the second processor 222 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0445] The embodiment of the present application provides a codec. At the decoding end, the decoder decodes the bitstream, determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; decodes the bitstream, determines the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh. At the encoding end, the encoder determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determines whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; when it is determined that the current LOD layer uses the first mode, determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, sets the first identification information corresponding to the current LOD layer, and writes the first identification information into the bitstream. Thus, it can be seen that in the embodiment of the present application, if the first identification information corresponding to the current LOD layer indicates that the current LOD layer uses the first mode, then after the reconstruction of the mesh of the current LOD layer is completed, the initial geometric information of the reconstructed mesh can be fitted, thereby obtaining geometric information of the reconstructed mesh with better quality. That is to say, in the embodiment of the present application, based on the first mode, after the preliminary reconstruction of the mesh is completed, the quality of the geometric information can be improved by fitting the reconstructed geometric information, thereby ensuring the quality of the geometric information of the mesh while improving the encoding efficiency of the geometric information, thereby improving the compression performance of the mesh.

[0446] Furthermore, an embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:

[0447] First identification information, second identification information, third identification information, shift coefficient, geometric upper limit information, geometric lower limit information, fitting information, and geometric information of the basic grid.

[0448] It should be noted that, in the embodiments of the present application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0449] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0450] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0451] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0452] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0453] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0454] The embodiment of the present application provides a coding and decoding method, an encoder, a decoder, a bitstream, and a storage medium. At the decoding end, the decoder decodes the bitstream, determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; decodes the bitstream, determines the first identification information corresponding to the current LOD layer; when the first identification information indicates that the current LOD layer uses the first mode, determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh. At the encoding end, the encoder determines the shift coefficient corresponding to the current LOD layer, and determines the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determines whether the current LOD layer uses the first mode based on the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; when it is determined that the current LOD layer uses the first mode, determines the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, sets the first identification information corresponding to the current LOD layer, and writes the first identification information into the bitstream. Thus, it can be seen that in the embodiment of the present application, if the first identification information corresponding to the current LOD layer indicates that the current LOD layer uses the first mode, then after the reconstruction of the mesh of the current LOD layer is completed, the initial geometric information of the reconstructed mesh can be fitted, thereby obtaining geometric information of the reconstructed mesh with better quality. That is to say, in the embodiment of the present application, based on the first mode, after the preliminary reconstruction of the mesh is completed, the quality of the geometric information can be improved by fitting the reconstructed geometric information, thereby ensuring the quality of the geometric information of the mesh while improving the encoding efficiency of the geometric information, thereby improving the compression performance of the mesh.

Claims

1. A decoding method, applied to a decoder, the method comprising: Decoding the bitstream, determining a shift coefficient corresponding to the current LOD layer, and determining initial geometric information of a reconstructed mesh corresponding to the current LOD layer according to the shift coefficient; Decode the code stream to determine the first identification information corresponding to the current LOD layer; When the first identification information indicates that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

2. The method according to claim 1, wherein: The method further comprises: Decoding the bitstream to determine the second identification information corresponding to the current sequence; When the second identification information indicates that the current sequence allows the use of the first mode, decoding the bitstream to determine third identification information corresponding to the current image in the current sequence; When the third identification information indicates that the current image allows the use of the first mode, a process for determining the first identification information corresponding to the current LOD layer is executed.

3. The method according to claim 1 or 2, wherein: The method further comprises: When the first identification information indicates that the current LOD layer uses the first mode, the bitstream is decoded to determine the fitting information corresponding to the current LOD layer.

4. The method according to claim 3, wherein: The fitting information includes a fitting type and fitting parameters corresponding to the fitting type; The fitting types include linear fitting, nonlinear fitting, neural grid-based fitting, curve fitting, and convolution parameter fitting.

5. The method according to claim 4, wherein: The step of determining the geometric information of the reconstructed mesh of the current LOD layer according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh comprises: Based on the fitting type and the fitting parameters, determining a mapping relationship between initial geometric information and predicted geometric information; According to the mapping relationship between the initial geometric information and the predicted geometric information, the geometric information of the reconstructed mesh corresponding to the initial geometric information of the reconstructed mesh is determined.

6. The method according to claim 1 or 2, wherein: The method further comprises: When the first identification information indicates that the current LOD layer does not use the first mode, the initial geometric information of the reconstructed mesh is determined as the geometric information of the reconstructed mesh of the current LOD layer.

7. The method according to claim 6, wherein: After the decoded code stream determines the first identification information corresponding to the current LOD layer, the method further includes: When the first identification information indicates that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined according to the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, and the initial geometric information of the reconstructed mesh.

8. The method according to claim 7, wherein: The step of determining the geometric information of the reconstructed mesh of the current LOD layer according to the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh comprises: When the initial geometric information of the reconstructed mesh is greater than the geometric lower limit information and the initial geometric information of the reconstructed mesh is less than the geometric upper limit information, determining the initial geometric information of the reconstructed mesh as the geometric information of the reconstructed mesh; When the initial geometric information of the reconstructed grid is less than or equal to the geometric lower limit information, determining the first preset geometric information as the geometric information of the reconstructed grid; When the initial geometric information of the reconstructed mesh is greater than or equal to the geometric upper limit information, the second preset geometric information is determined as the geometric information of the reconstructed mesh.

9. The method according to claim 8, wherein: The method further comprises: When the first identification information indicates that the current LOD layer uses the first mode, the code stream is decoded to determine the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer.

10. The method according to any one of claims 7 to 9, wherein: The method further comprises: The geometric information of the reconstructed mesh of the current LOD layer is corrected according to the geometric upper limit information and the geometric lower limit information.

11. The method according to claim 1, wherein: The determining, according to the shift coefficient, the initial geometric information of the reconstructed mesh corresponding to the current LOD layer comprises: The initial geometric information of the reconstructed mesh corresponding to the current LOD layer is determined according to the shift coefficient and the geometric information of the initial mesh corresponding to the current LOD layer.

12. The method according to claim 11, wherein: The method further comprises: Decode the code stream to determine the base grid corresponding to the current LOD layer; The basic mesh is subdivided to determine the geometric information of the initial mesh corresponding to the current LOD layer.

13. The method according to claim 12, wherein: The subdividing the basic grid to determine the geometric information of the initial grid corresponding to the current image includes: The basic grid is iteratively divided according to a grid subdivision mode to determine geometric information of the initial grid.

14. The method according to claim 13, wherein: The grid subdivision mode includes: a subdivision algorithm and a subdivision iteration number.

15. The method according to claim 14, wherein: The subdivision algorithm is a linear interpolation algorithm.

16. The method according to claim 1, wherein: The decoding bit stream determines the shift coefficient corresponding to the current LOD layer, including: Decoding the code stream to determine the two-dimensional image of the current LOD layer; Performing coefficient reorganization processing on the two-dimensional image to determine the lifting transformation coefficient of the current LOD layer; An inverse transformation process is performed on the lifting transformation coefficient of the current LOD layer to determine the shift coefficient of the current LOD layer.

17. The method according to claim 16, wherein: The performing coefficient reorganization processing on the two-dimensional image to determine the lifting transformation coefficient of the current LOD layer includes: Performing coefficient reorganization processing on the two-dimensional image to determine the quantization coefficient of the current LOD layer; Dequantization is performed on the quantization coefficients of the current LOD layer to determine the lifting transformation coefficients of the current LOD layer.

18. The method according to claim 3, wherein: The method further comprises: The bit stream is decoded by a mesh decoder to determine the basic mesh corresponding to the current LOD layer.

19. The method according to claim 1, wherein: The method further comprises: The bitstream is decoded by a video decoder to determine the shift coefficient and / or the fitting information corresponding to the current LOD layer.

20. A decoding method, applied to a decoder, wherein: The decoder comprises a video decoder and a trellis decoder, The grid decoder and the video decoder are used to perform the decoding method as described in any one of claims 1-19.

21. A coding method, applied to an encoder, the method comprising: Determine a shift coefficient corresponding to the current LOD layer, and determine initial geometric information of a reconstructed mesh corresponding to the current LOD layer according to the shift coefficient; Determining whether the current LOD layer uses the first mode according to the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; When it is determined that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, the first identification information corresponding to the current LOD layer is set, and the first identification information is written into the bitstream.

22. The method according to claim 21, wherein: The method further comprises: Determine second identification information corresponding to the current sequence; When the second identification information indicates that the current sequence allows the use of the first mode, determining third identification information corresponding to a current image in the current sequence; When the third identification information indicates that the current image allows the use of the first mode, a process for determining the first identification information corresponding to the current LOD layer is executed.

23. The method according to claim 21, wherein: The determining, according to the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer, whether the current LOD layer uses the first mode includes: Performing fitting processing on the initial geometric information of the reconstructed grid and the geometric information of the original grid to determine the geometric information of the fitted predicted grid; A first error is determined based on the geometric information of the predicted grid and the geometric information of the original grid, and a second error is determined based on the initial geometric information of the reconstructed grid and the geometric information of the original grid. Whether the current LOD layer uses the first mode is determined according to the first error and the second error.

24. The method according to claim 23, wherein: The determining, according to the first error and the second error, whether the current LOD layer uses the first mode includes: determining error parameters of the first error and the second error; When the error parameter is less than a preset threshold, determining that the current LOD layer uses the first mode; When the error parameter is greater than or equal to a preset threshold, it is determined that the current LOD layer does not use the first mode.

25. The method according to claim 23 or 24, wherein: The method further comprises: When it is determined that the current LOD layer uses the first mode, the fitting information corresponding to the current LOD layer is written into a bitstream.

26. The method according to claim 25, wherein: The fitting information includes a fitting type and fitting parameters corresponding to the fitting type; The fitting types include linear fitting, nonlinear fitting, neural grid-based fitting, curve fitting, and convolution parameter fitting.

27. The method according to claim 26, wherein: The step of determining the geometric information of the reconstructed mesh of the current LOD layer according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh comprises: Based on the fitting type and the fitting parameters, determining a mapping relationship between initial geometric information and predicted geometric information; According to the mapping relationship between the initial geometric information and the predicted geometric information, the geometric information of the reconstructed mesh corresponding to the initial geometric information of the reconstructed mesh is determined.

28. The method according to claim 23 or 24, wherein: The method further comprises: When it is determined that the current LOD layer does not use the first mode, the initial geometric information of the reconstructed mesh is determined as the geometric information of the reconstructed mesh of the current LOD layer, the first identification information corresponding to the current LOD layer is set, and the first identification information is written into the bitstream.

29. The method according to claim 28, wherein: The method further comprises: Determine whether the current LOD layer uses the first mode according to the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh; When it is determined that the current LOD layer uses the first mode, the geometric information of the reconstructed mesh of the current LOD layer is determined according to the geometric upper limit information and geometric lower limit information corresponding to the current LOD layer, and the initial geometric information of the reconstructed mesh, the first identification information corresponding to the current LOD layer is set, and the first identification information is written into the bitstream.

30. The method of claim 29, wherein: The determining, according to the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh, whether the current LOD layer uses the first mode includes: When the initial geometric information of the reconstructed mesh is less than or equal to the geometric lower limit information, or when the initial geometric information of the reconstructed mesh is greater than or equal to the geometric upper limit information, it is determined whether the current LOD layer uses the first mode.

31. The method of claim 29, wherein: The step of determining the geometric information of the reconstructed mesh of the current LOD layer according to the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh comprises: When the initial geometric information of the reconstructed mesh is greater than the geometric lower limit information and the initial geometric information of the reconstructed mesh is less than the geometric upper limit information, determining the initial geometric information of the reconstructed mesh as the geometric information of the reconstructed mesh; When the initial geometric information of the reconstructed grid is less than or equal to the geometric lower limit information, determining the first preset geometric information as the geometric information of the reconstructed grid; When the initial geometric information of the reconstructed mesh is greater than or equal to the geometric upper limit information, the second preset geometric information is determined as the geometric information of the reconstructed mesh.

32. The method of claim 30, wherein: The method further comprises: When it is determined that the current LOD layer uses the first mode, the geometric upper limit information and the geometric lower limit information corresponding to the current LOD layer are written into the bitstream.

33. The method according to any one of claims 29 to 32, wherein: The method further comprises: The geometric information of the reconstructed mesh of the current LOD layer is corrected according to the geometric upper limit information and the geometric lower limit information.

34. The method of claim 21, wherein: The determining, according to the shift coefficient, the initial geometric information of the reconstructed mesh corresponding to the current LOD layer comprises: The initial geometric information of the reconstructed mesh corresponding to the current LOD layer is determined according to the shift coefficient and the geometric information of the initial mesh corresponding to the current LOD layer.

35. The method of claim 34, wherein: The method further comprises: Determine the base mesh corresponding to the current LOD layer; The basic mesh is subdivided to determine the geometric information of the initial mesh corresponding to the current LOD layer.

36. The method of claim 35, wherein: The subdividing the basic grid to determine the geometric information of the initial grid corresponding to the current image includes: The basic grid is iteratively divided according to a grid subdivision mode to determine geometric information of the initial grid.

37. The method of claim 36, wherein: The grid subdivision mode includes: a subdivision algorithm and a subdivision iteration number.

38. The method of claim 37, wherein: The subdivision algorithm is a linear interpolation algorithm.

39. The method of claim 21, wherein: The method further comprises: Performing a lifting transformation on the shift coefficient of the current LOD layer to determine a lifting transformation coefficient; Quantizing the lifting transformation coefficient to determine the quantization coefficient of the current LOD layer; Performing coefficient reorganization processing on the quantization coefficients of the current LOD layer to determine a two-dimensional image of the current LOD layer; The two-dimensional image of the current LOD layer is coded, and the obtained coded bits are written into a bit stream.

40. The method of claim 35, wherein: The method further comprises: The basic mesh corresponding to the current LOD layer is written into the bitstream through the mesh encoder.

41. The method of claim 21, wherein: The method further comprises: The shift coefficient and / or the fitting information corresponding to the current LOD layer is written into a bitstream through a video encoder.

42. A coding method, applied to an encoder, wherein: The encoder includes a video encoder and a trellis encoder, The grid encoder and the video encoder are used to perform the decoding method as described in any one of claims 21-41.

43. A code stream, the code stream is generated by bit encoding according to information to be encoded; wherein, The information to be encoded includes at least one of the following: First identification information, second identification information, third identification information, shift coefficient, geometric upper limit information, geometric lower limit information, fitting information, and geometric information of the basic grid.

44. An encoder, comprising: The first determining unit is a coding unit; wherein, The first determination unit is configured to determine a shift coefficient corresponding to the current LOD layer, and determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer according to the shift coefficient; determine whether the current LOD layer uses the first mode according to the initial geometric information of the reconstructed mesh and the geometric information of the original mesh corresponding to the current LOD layer; if it is determined that the current LOD layer uses the first mode, determine the geometric information of the reconstructed mesh of the current LOD layer according to the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh; The encoding unit is configured to set first identification information corresponding to the current LOD layer and write the first identification information into the bitstream.

45. An encoder, comprising: a first memory and a first processor; wherein, A first memory, for storing a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 21 to 41 when running a computer program.

46. ​​A decoder, the decoder comprising: A decoding unit, a second determining unit; wherein, The decoding unit is configured to decode the code stream; The second determination unit is configured to determine a shift coefficient corresponding to a current LOD layer, and determine the initial geometric information of the reconstructed mesh corresponding to the current LOD layer based on the shift coefficient; determine the first identification information corresponding to the current LOD layer; and when the first identification information indicates that the current LOD layer uses a first mode, determine the geometric information of the reconstructed mesh of the current LOD layer based on the fitting information corresponding to the current LOD layer and the initial geometric information of the reconstructed mesh.

47. A decoder, the decoder comprising: A second memory and a second processor; wherein, A second memory for storing a computer program that can be run on a second processor; The second processor is configured to execute the method according to any one of claims 1 to 20 when running a computer program.

48. A computer-readable storage medium storing a computer program, wherein the computer program, when executed, implements the method according to any one of claims 1 to 20, or implements the method according to any one of claims 21 to 41.

Images