Coding method, code stream, encoder, decoder and storage medium

Abstract

This application discloses an encoding / decoding method, a bitstream, an encoder, a decoder, and a storage medium. The method includes: determining a first basic grid for the current frame based on the original grid of the current frame; selecting matching points from the original grid of the current frame based on a second basic grid of a reference frame to obtain a third basic grid; wherein the vertices of the third basic grid are matched with those of the second basic grid; determining a target encoding mode for the current frame based on the error metric difference between the first and third basic grids; and encoding the current frame according to the target encoding mode. This approach can improve grid encoding efficiency while ensuring grid reconstruction quality.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of dynamic grid coding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] In the standard reference software for Dynamic Mesh Coding provided by the Moving Picture Experts Group (MPEG), most of them use the geometric information of the initial mesh obtained by basic mesh division and the shift coefficient to reconstruct and restore the geometric position information of the current mesh.

[0003] In the inter-frame coding mode, the motion vector between the base grid of the current frame and the base grid of the reference frame is calculated and encoded using arithmetic coding, thereby avoiding directly using a static encoder to compress the base grid. However, in related technologies, since only a few frames in a sequence with large motion can use the inter-frame coding mode, a static encoder needs to be used to compress a large number of base grids. This reduces the number of frames that can use inter-frame prediction and reduces coding efficiency.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium, which can improve the coding efficiency of the grid while ensuring the quality of grid reconstruction.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0008] Determining a first basic grid of the current frame according to the original grid of the current frame;

[0009] Based on the second basic mesh of the reference frame, selecting matching points from the original mesh of the current frame to obtain a third basic mesh; wherein the third basic mesh matches vertices in the second basic mesh;

[0010] determining a target coding mode for the current frame according to an error metric difference between the first basic grid and the third basic grid;

[0011] The current frame is encoded according to the target encoding mode.

[0012] In a second aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0013] Parse the code stream to obtain syntax identification information;

[0014] When the syntax identification information indicates that the coding mode used by the current frame is an inter-frame coding mode, parsing to obtain a motion vector corresponding to a third basic mesh of the current frame and shift information corresponding to a subdivided mesh of the current frame; wherein the third basic mesh is obtained by selecting matching points from an original mesh of the current frame based on a second basic mesh of a reference frame; and the third basic mesh matches vertices in the second basic mesh;

[0015] A reconstruction grid of the current frame is determined according to the motion vector and the shift information.

[0016] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:

[0017] The value of the syntax identification information, the motion vector corresponding to the third basic grid, the first basic grid of the current frame, and the shift information corresponding to the subdivided grid of the current frame; wherein the syntax identification information is used to indicate the target coding mode adopted by the current frame.

[0018] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining part, a matching part, and an encoding part, wherein:

[0019] The first determining part is configured to determine a first basic grid of the current frame according to the original grid of the current frame;

[0020] The matching part is configured to select matching points from the original mesh of the current frame based on the second basic mesh of the reference frame to obtain a third basic mesh; wherein the third basic mesh matches vertices in the second basic mesh;

[0021] The first determining portion is further configured to determine a target coding mode for the current frame based on an error metric difference between the first base grid and the third base grid;

[0022] The encoding part is configured to perform encoding processing on the current frame according to the target encoding mode.

[0023] In a fifth aspect, an embodiment of the present application provides a decoder, comprising a decoding part and a second determining part, wherein:

[0024] The decoding portion is configured to parse a bitstream to obtain syntax identification information; if the syntax identification information indicates that the coding mode used by the current frame is an inter-frame coding mode, parse the bitstream to obtain a motion vector corresponding to a third basic grid of the current frame and shift information corresponding to a subdivided grid of the current frame; wherein the third basic grid is obtained by selecting matching points from an original grid of the current frame based on a second basic grid of a reference frame; and the third basic grid matches vertices in the second basic grid;

[0025] The second determining part is configured to determine a reconstruction grid of the current frame according to the motion vector and the shift information.

[0026] In a sixth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor, wherein:

[0027] The first memory is configured to store a computer program that can be executed on the first processor;

[0028] The first processor is configured to execute the method according to the first aspect when running the computer program.

[0029] In a seventh aspect, an embodiment of the present application provides a decoder, comprising a second memory and a second processor, wherein:

[0030] The second memory is configured to store a computer program that can be executed on the second processor;

[0031] The second processor is configured to execute the method according to the second aspect when running the computer program.

[0032] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the method as described in the first aspect is implemented, or the method as described in the second aspect is implemented.

[0033] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium. At the encoding end, based on the original grid of the current frame, a first basic grid of the current frame is determined; based on the second basic grid of the reference frame, matching points are selected for the original grid of the current frame to obtain a third basic grid; wherein the third basic grid matches the vertices in the second basic grid; based on the error metric difference between the first basic grid and the third basic grid, a target coding mode of the current frame is determined; and according to the target coding mode, the current frame is encoded. At the decoding end, the code stream is parsed to obtain syntax identification information; when the syntax identification information indicates that the coding mode adopted by the current frame is the inter-frame coding mode, the motion vector corresponding to the third basic grid of the current frame and the displacement information corresponding to the subdivided grid of the current frame are obtained by parsing; wherein the third basic grid is obtained by selecting matching points for the original grid of the current frame based on the second basic grid of the reference frame; the third basic grid matches the vertices in the second basic grid; and based on the motion vector and the displacement information, a reconstructed grid of the current frame is determined. This approach, on the one hand, generates a third base grid based on the second base grid of the reference frame by selecting the best matching point on the original grid of the current frame. This expands the applicability of inter-frame coding modes, significantly reduces the bit rate of the encoded base grid, and ensures the quality of the reconstructed grid. On the other hand, selecting the appropriate target coding mode for the current frame based on the error metric difference allows for adaptive selection of the appropriate target coding mode in different scenarios, ensuring grid reconstruction quality while improving grid encoding and decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1A is a schematic diagram of a three-dimensional grid image 1;

[0035] FIG1B is a partially enlarged schematic diagram of a three-dimensional grid image;

[0036] Figure 2 is a schematic diagram of the connection method of the three-dimensional grid;

[0037] FIG3A is a second schematic diagram of a three-dimensional grid image;

[0038] FIG3B is a schematic diagram of a grid data storage format;

[0039] FIG3C is a schematic diagram of properties of a three-dimensional grid image;

[0040] FIG4 is a schematic diagram showing the composition of the overall framework of grid coding;

[0041] FIG5A is a schematic diagram of preprocessing of a two-dimensional curve;

[0042] FIG5B is a schematic diagram of generating a shift coefficient;

[0043] FIG6A is a first schematic diagram of quantization processing of grid geometric position information;

[0044] FIG6B is a second schematic diagram of quantization processing of grid geometric position information;

[0045] FIG7A is a schematic diagram showing the composition of an intra-frame coding framework;

[0046] FIG7B is a schematic diagram showing the composition of an inter-frame coding framework;

[0047] FIG8A is a schematic diagram showing the composition of an intra-frame decoding framework;

[0048] FIG8B is a schematic diagram showing the composition of an inter-frame decoding framework;

[0049] FIG9A is a second schematic diagram of the composition of an inter-frame coding framework;

[0050] FIG9B is a second schematic diagram of the composition of the inter-frame decoding framework;

[0051] FIG10 is a schematic diagram of a mesh architecture of a codec provided in an embodiment of the present application;

[0052] FIG11 is a flowchart diagram 1 of an encoding method provided in an embodiment of the present application;

[0053] FIG12 is a second flow chart of an encoding method provided in an embodiment of the present application;

[0054] FIG13 is a third flow chart of an encoding method provided in an embodiment of the present application;

[0055] FIG14 is a schematic diagram of a flow chart of a decoding method provided in an embodiment of the present application;

[0056] FIG15 is a fourth flow chart of an encoding method provided in an embodiment of the present application;

[0057] FIG16 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0058] FIG17 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0059] FIG18 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0060] FIG19 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;

[0061] FIG20 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0064] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0065] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0066] It should be noted that it is possible to decode and synthesize different data format bitstreams within the same video scene. These can include at least image format, point cloud format, and mesh format. In this way, real-time immersive video interaction services can be provided for multiple data formats (e.g., mesh, point cloud, image, etc.) from different sources.

[0067] In embodiments of the present application, the data format-based approach allows for independent processing at the bitstream level of the data format. This means that, similar to tiles or slices in video encoding, different data formats in this scenario can be encoded independently, enabling independent encoding and decoding based on the data format.

[0068] Generally speaking, 3D animation content uses a keyframe-based representation method, that is, each frame is a static mesh. Static meshes at different times have the same topological structure and different geometric structures. However, the amount of data of 3D dynamic meshes represented based on keyframes is extremely large, so how to effectively store, transmit and draw them has become a problem faced by the development of 3D dynamic meshes. In addition, the spatial scalability of the mesh needs to be supported for different user terminals (computers, notebooks, portable devices, mobile phones); different mesh bandwidths (broadband, narrowband, wireless) need to support the quality scalability of the mesh. Therefore, 3D dynamic mesh compression is a very critical issue.

[0069] A 3D mesh is the surface of a 3D object composed of countless polygons in space. Polygons are composed of vertices and edges. Figure 1A shows a 3D mesh image, and Figure 1B shows a partially enlarged schematic diagram of the 3D mesh image. Figures 1A and 1B show that the mesh surface is composed of closed polygons.

[0070] A two-dimensional image has information expressed at every pixel point and is distributed regularly, so there is no need to record its position information separately. However, the distribution of vertices in the mesh in three-dimensional space is random and irregular, and the way polygons are formed requires additional regulations. Therefore, it is necessary to record the position of each vertex in space and the connection information of each polygon to fully express a mesh image. As shown in Figure 2, the same number of vertices and vertex positions will form completely different surfaces due to different connection methods.

[0071] In addition to the above information, since 3D mesh images are usually encoded using existing 2D image / video encoding methods, the 3D mesh needs to be converted from 3D space to 2D images. The UV coordinates define this conversion process.

[0072] Similar to 2D images, each position in the image may have corresponding attribute information, typically RGB color values, which reflect the object's color. For 3D meshes, in addition to color, each vertex often has reflectance values, which reflect the surface material. 3D mesh attribute information is stored in 2D images, and the mapping from 2D to 3D is defined by UV coordinates.

[0073] Therefore, 3D mesh data typically includes 3D geometric position information (x, y, z), geometric connectivity, UV coordinates, and an attribute map. Figure 3A shows a 3D mesh image, Figure 3B shows the mesh data storage format, which includes 3D geometric position information, UV coordinates, and connectivity information, and Figure 3C shows the corresponding attribute diagram.

[0074] Current 3D dynamic mesh compression methods include space-time prediction methods, which improve compression efficiency by eliminating spatial and temporal correlations; principal component analysis (PCA)-based technology, which projects in the eigenvector space to concentrate energy; and wavelet-based methods, which support spatial scalability and quality scalability.

[0075] It should be noted that in the dynamic mesh coding provided by the Moving Picture Experts Group (MPEG), Figure 4 shows the overall mesh coding framework, Figure 5A shows the preprocessing of a 2D curve, and Figure 5B shows the generation of the shift coefficients. The preprocessing process for a 3D mesh is similar, and on the encoding side, it is mainly divided into two parts: preprocessing and encoder. Preprocessing first generates a base mesh and shift coefficients. The preprocessing process involves downsampling the original mesh to generate a simplified mesh (decimated mesh) with a significantly reduced number of vertices, or the base mesh. The base mesh is then subdivided and algorithmically generated, with newly generated vertices inserted along the edges of the base mesh to create subdivided meshes. Finally, for each vertex in the subdivided mesh, the nearest vertex in the original mesh is found. The vector between the vertex in the subdivided mesh and the nearest vertex in the original mesh is the shift coefficient. Since the subdivision grid can be automatically generated at the codec end as long as the subdivision algorithm and the number of subdivision iterations are determined, after preprocessing, the original grid only needs to be represented as a simple basic grid and a series of shift coefficients. This can greatly reduce the amount of data without affecting the reconstruction at the decoding end.

[0076] Video Dynamic Mesh Coding (V-DMC) based on video coding can be broadly categorized into two main categories: geometric position information encoding and attribute information encoding. As shown in Figure 5, each frame of the basketball_player sequence includes two files: basketball_player_fr0001_qp12_qt12.obj and basketball_player_fr0002.png. basketball_player_fr0001_qp12_qt12.obj contains four types of information: geometric position information (x, y, z), the connectivity of geometric position triangles, texture coordinates (u, v), and the connectivity of texture coordinates. basketball_player_fr0002.png represents the texture attribute information of the current image. In current V-DMC encoders, geometric position information is jointly encoded using the Dynamic Range Arithmetic Coding (DRACO) and the video codec, while texture information is encoded directly using the video codec. Among them, Video Codec can include H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC / VV-enC), etc. Therefore, the following will introduce the mesh geometric information encoding in detail.

[0077] Geometric information can be divided into the encoding of position information (geometric position information and texture position information) and the encoding of connectivity relationships (geometric position information triangle patch connectivity, texture position information connectivity). Currently, V-DMC coding is mainly divided into two coding test conditions: intra-frame coding and inter-frame coding (low latency, currently no RA test environment).

[0078] (1) Intra-frame geometric information coding (Intra coding).

[0079] 1. Mesh preprocessing.

[0080] a) As shown in Figures 5A and 5B, using a two-dimensional connection relationship as an example, the original mesh's connection relationship contains a large number of points. Before encoding the mesh's geometric information, the mesh's geometric information is first quantized or simplified, ultimately resulting in a corresponding decimated mesh as the base mesh.

[0081] b) As shown in FIG6A and FIG6B , the quantization processing of the grid is performed based on the coordinates of the triangle patch. According to the connection relationship between the quantization points, the quantization processing can be divided into the following two cases:

[0082] When two vertices share a common edge, that is, two vertices that belong to the same edge before quantization, then after quantization, all the triangles connected by the two vertices need to be connected together, which involves the vanishing of the previous triangles as shown in Figure 6A;

[0083] Otherwise, if the two vertices do not share a common edge, that is, the two vertices do not belong to the same edge, then after quantization, it is only necessary to merge the boundaries of the two vertices, as shown in FIG6B , which does not affect the number of triangles.

[0084] c) In the whole process of mesh quantization based on triangle coordinates, the core problem is how to get the best vertex based on the previous vertex coordinates. The current V-DMC will get the best quantization point in the following four modes. Assuming that the vertex distribution before quantization is V1 and V2, and the vertex coordinate after quantization is V', there are the following: V1, V2, (V1+V2) / 2 and Q -1 (V1+V2), where Q is the quantization matrix corresponding to the vertex coordinates of V1 and V2. The distortion measure D before and after basic quantization is used to select the optimal quantization point.

[0085] 2.Base mesh encoding.

[0086] a) After obtaining the base mesh, the DRACO encoder is used to encode its geometric information. This geometric information primarily includes connectivity relationships and geometric position information. The DRACO encoding process is as follows: first, the connectivity relationships are encoded. Then, the geometric position information of the points is encoded based on the connectivity relationships. Finally, the texture position information is encoded based on the connectivity relationships and geometric position information.

[0087] b) Connection relationship encoding: DRACO uses the "Edgebreaker Coding" scheme to encode the connection relationship of the mesh.

[0088] Finally, the type of each vertex and the processing order of the vertices are encoded in a certain order, and the decoding end restores the geometric connection relationship of the mesh according to the processing order and type of the vertices.

[0089] c) Coding of geometric position information: After the coding of the vertex connectivity relationship is completed, the geometric position information of each vertex is predictively coded based on the vertex connectivity relationship.

[0090] d) After completing the point connection relationship and geometric position information, the texture coordinates are predictively encoded based on the decoding and reconstruction of the two.

[0091] 3. Displacement coefficient encoding.

[0092] a) First, after completing the encoding reconstruction of the base mesh, a certain partitioning algorithm will be used to divide the base mesh to obtain the initial reconstructed mesh.

[0093] b) Next, the point-to-point error Delta between the subdivided mesh and the original mesh is calculated. This error Delta can be expressed as a point-to-point error in the world coordinate system. Finally, the point-to-point displacement (i.e., displacement coefficient) is calculated using the point-to-point error Delta and the point normal vector Norm.

[0094] c) After calculating the displacement of each point, the spatial domain residual coefficient can be transformed into the frequency domain using the lifting transform to obtain the corresponding frequency domain residual coefficient.

[0095] d) Finally, a coefficient packing algorithm is used to map the frequency domain residual coefficients of each point into a two-dimensional image in a certain order. The current V-DMC can be arranged according to the Morton Code Order.

[0096] e) Finally, a traditional Video Codec is used to encode the two-dimensional image.

[0097] 4. Recoloring.

[0098] Recoloring is an algorithm on the encoder side. After the reconstruction of the encoder's geometric information is completed, the original geometric information, the original texture attribute information, and the reconstructed mesh geometric information are used to recolor the texture attribute information of the reconstructed mesh.

[0099] (2) Inter-frame geometric information coding (Inter coding).

[0100] a) Similar to the encoding above, the geometric position information includes the geometric connection relationship and the geometric position information encoding. However, it should be noted that the inter-frame geometric position information encoding only needs to encode the geometric position information (x, y, z) of the current base mesh, and does not need to encode the connection relationship and texture position information (u, v). The specific reason is as follows: if the current image can be inter-coded, then the base mesh of the reference image of the current image will be used at the encoder to obtain the mesh information of the current image. Therefore, the current image and the reference image have the same connection relationship and UV texture coordinates, only the geometric position information is different.

[0101] b) Based on a), it can be known that the only difference between the current image and the reference image is the geometric position information. Therefore, the current V-DMC performs predictive coding on the geometric position information of the current image.

[0102] Figure 7A is a schematic diagram of the composition of the intra-frame coding framework. As shown in Figure 7A, in the intra-frame encoder, a common static mesh encoder (Static Mesh Encoder) can be used to encode the simplified mesh to generate the corresponding bitstream (Compressed base mesh bitstream). Next, the reconstructed simplified mesh is used to update the displacement coefficients (Update Displacements). The updated displacement coefficients are subjected to wavelet transform (Wavelet Transform) and quantization (Quantization) to obtain the displacement coefficients. They are then packaged into images and videos (Image Packing, Video Packing) and encoded using HEVC to generate a bitstream of displacement coefficients (Compressed displacements bitstream). For attribute map encoding, the feature map is first transformed (Texture Transfer) according to the difference between the reconstructed geometric information and the original geometric information, and then padded (Padding) and packaged (Video Packing) and encoded using a video encoder to form an attribute bitstream (Compressed attribute bitstream).

[0103] Figure 7B is a schematic diagram of the inter-frame coding framework. As shown in Figure 7B , the inter-frame encoder follows a similar process to the intra-frame encoder. However, the inter-frame encoder does not directly encode the simplified grid. Instead, it encodes the motion vector MV between the simplified grid of the current image and the simplified grid of the reference image, generating a compressed motion bitstream.

[0104] Correspondingly, during the decoding process, the decoder can also be divided into an intra-frame decoder and an inter-frame decoder according to the type of the frame it operates on, which are used to perform intra-frame decoding and inter-frame decoding respectively.

[0105] FIG8A is a schematic diagram of the composition of the intra-frame decoding framework. As shown in FIG8A , in the intra-frame decoder, a static mesh decoder can be used to decode the simplified mesh. A video decoder is used to decode the shift coefficient video, and the shift coefficient is obtained through video unpacking and inverse wavelet transform. The decoded simplified mesh and shift coefficient are used to obtain the decoded mesh geometry information. The attribute graph is decoded directly through the video decoder.

[0106] FIG8B is a schematic diagram of the inter-frame decoding framework. As shown in FIG8B , the inter-frame decoder follows a similar process to the intra-frame decoder, except that the simplified grid is not directly decoded. Instead, the motion vector is decoded and the simplified grid of the current image is calculated using the simplified grid of the previous image (e.g., the reference image).

[0107] Figure 9A is a second schematic diagram of the inter-frame coding framework. As shown in Figure 9A , motion vector encoding is performed based on the reference frame base grid and the current frame base grid to obtain a motion vector code stream. The base grid is reconstructed based on the motion vectors to obtain a reconstructed grid. Furthermore, vertex displacements are updated based on the reconstructed grid and vertex displacements to obtain updated vertex displacements. The updated vertex displacements are further subjected to wavelet transformation and transform coefficient encoding to obtain a transform coefficient code stream. Furthermore, a deformed grid is reconstructed based on the transform coefficients, and the deformed grid and the attribute map are converted into an attribute map. The attribute map is encoded to obtain an attribute code stream.

[0108] Figure 9B is a second schematic diagram of the inter-frame decoding framework. As shown in Figure 9B , the motion vector bitstream is parsed and decoded to obtain the motion vector of the current frame. The transform coefficient bitstream is parsed to obtain the transform coefficients of the current frame. The attribute map bitstream is parsed, and the decoded attribute map of the current frame is obtained through the video decoder. Furthermore, the base grid is reconstructed based on the reference frame base grid and the motion vector of the current frame to obtain a reconstructed grid. Furthermore, the deformed grid is reconstructed based on the transform coefficients and the reconstructed grid to obtain a decoded grid. Finally, the original grid of the current frame is restored based on the decoded attribute map and the decoded grid.

[0109] For intra-frame coding, the original input mesh contains a large amount of geometric and connectivity information, requiring compression and simplification. MPEG-VDMC simplifies the original input mesh using the Quadric Error Metrics algorithm to obtain a base mesh (i.e., a simplified mesh). Then, using midpoint subdivision, a new point is inserted at the midpoint of each edge. The coordinates and normal of this point are the average of the two vertices on that edge. This subdivision results in a subdivided mesh. For each vertex in the subdivided mesh, the coordinates of the nearest point on the surface of the original input mesh are found. The distance between these two points is the displacement of the vertex in the subdivided mesh. This mesh subdivision and displacement calculation process is repeated (by default three times) to obtain the simplified base mesh and displacement of the original input mesh. The base mesh is then compressed and encoded using a static mesh encoder. The displacements are then wavelet transformed, quantized, and packed into 2D images before being compressed and encoded using a video encoder. This algorithm not only leverages the static encoder's ability to preserve the 3D mesh's structure, but also avoids static encoding of the entire 3D mesh by adjusting vertex positions, reducing the required encoding bitrate and demonstrating excellent rate-distortion performance.

[0110] In summary, in the dynamic mesh coding (Dynamic Mesh Coding) currently provided by MPEG, the dynamic mesh coding process is divided into the following steps: at the encoding end, the basic mesh generated by preprocessing is quantized and then encoded using Google's open source DRACO encoder, and the shift coefficients are encoded using HEVC after wavelet transform, quantization, and two-dimensional mapping. The two-dimensional attribute map is also directly transmitted to the HEVC encoder for encoding; at the decoding end, the basic mesh code stream is decoded by DRACO to generate a decoded basic mesh, and the shift coefficients are decoded by HEVC decoding, inverse two-dimensional mapping, inverse quantization, and inverse transformation to generate decoded shift coefficients. Then, the decoded basic mesh and the decoded shift coefficients are used together to reconstruct the three-dimensional mesh geometry, and the attribute code stream is decoded by HEVC to generate a reconstructed attribute map.

[0111] In summary, the dynamic grid encoding process can be divided into the following steps:

[0112] 1. Preprocess the original mesh by reducing the number of vertices in the mesh and simplifying the connection relationship.

[0113] 2. Subdivide the simplified mesh in step 1. For any two connected vertices in step 1, add a new point at the midpoint of the connecting line, and repeat this process twice.

[0114] 3. For each vertex in step 2, find the point in the original mesh that is closest to it and calculate the displacement coefficient of these two points.

[0115] 4. Use an encoder such as Draco to quantize the simplified grid in step 1 and then encode it.

[0116] 5. Adjust the shift coefficients in step 3 based on the reconstructed simplified grid obtained in step 4.

[0117] 6. Perform wavelet transform on the shift coefficients in step 5, and quantize the shift coefficients after wavelet transform to obtain quantized transform coefficients.

[0118] 7. Map the quantized transform coefficients from three-dimensional space to a two-dimensional image (or "image packing") to generate a two-dimensional image of shifted coefficients.

[0119] 8. Use a standard video encoder such as H.265 to encode the shift coefficient two-dimensional image in step 6.

[0120] In another specific implementation, for the decoding end, first, the Video-Codec is used to decode and reconstruct the two-dimensional image to restore the corresponding two-dimensional image. Secondly, according to the coefficient reorganization method, the lifting transform coefficient corresponding to each point can be restored. Finally, the inverse transform of the lifting wavelet transform can be used to restore the displacement coefficient of each point. After obtaining the displacement coefficient of each point, the geometric position information of the base mesh and the displacement coefficient are used to reconstruct and restore the geometric position information corresponding to the current mesh. Specifically, the geometric position information of the level 0 layer in the base mesh and the displacement coefficient of the level 0 layer can reconstruct and restore the geometric position information of the level 0 layer in the reconstructed mesh, the geometric position information of the level 1 layer in the base mesh and the displacement coefficient of the level 1 layer can reconstruct and restore the geometric position information of the level 1 layer in the reconstruct mesh, and so on. The geometric position information of the level 3 layer in the base mesh and the displacement coefficient of the level 3 layer can reconstruct and restore the geometric position information of the level 3 layer in the reconstruct mesh.

[0121] In summary, the dynamic grid decoding process can be divided into the following steps:

[0122] 1. The basic grid code stream is decoded by a decoder such as draco to generate a decoded basic grid.

[0123] 2. The shift coefficient bit stream is decoded using a standard video encoder such as H.265 to obtain a shift coefficient two-dimensional image.

[0124] 3. Map the shift coefficient two-dimensional image from the two-dimensional image to the three-dimensional space (or "image unpacking") to obtain quantized transform coefficients.

[0125] 4. Dequantize and inverse wavelet transform the quantized transform coefficients to obtain the decoded shift coefficients.

[0126] 5. The decoded base grid and the decoded shift coefficients are combined to generate the reconstructed 3D grid geometric information.

[0127] 6. After the attribute code stream is decoded by HEVC, a reconstructed attribute graph is generated.

[0128] In the related art, inter-frame coding is mainly reflected in: 1. Coding of the base grid: When two consecutive frames meet the conditions of having the same number of points, the same connectivity, the same texture coordinates, and the same texture connectivity, the base grid generated by simplifying the input grid has the same number of points and connectivity as the base grid of the previous frame, so the corresponding point residual is the motion vector of the base grid vertex of this frame. Furthermore, intra-frame prediction is used based on the encoded base grid vertices, and arithmetic coding is used to encode the prediction residual, thereby avoiding the use of a static encoder to compress the vertices and geometric relationships of the base grid. 2. Coding of displacement: The inter-frame mode in the video encoding tool can be used to perform inter-frame prediction on the video generated by vertex displacement, thereby saving bit rate.

[0129] In the current V-DMC grid coding, in inter-frame prediction mode, the motion vector between the base grid of the current frame and the base grid of the reference frame is calculated and encoded using arithmetic coding, thereby avoiding the direct use of a static encoder to compress the base grid. Inter-frame prediction is allowed when the following conditions are met:

[0130] 1. The number of vertices between the reference base mesh and the current base mesh is the same;

[0131] 2. The vertex connections between the reference base mesh and the current base mesh are the same;

[0132] 3. The texture coordinates between the reference base mesh and the current base mesh are the same;

[0133] 4. The connectivity of the texture coordinates between the reference base mesh and the current base mesh is the same,

[0134] Meeting these conditions means that inter-frame coding can only be performed on frames where the base grid differs only in geometric position from the reference grid base frame. Therefore, in practice, only a small number of frames can use inter-frame prediction. This limitation reduces the number of frames that can use inter-frame prediction and requires the use of a static encoder to compress a large number of base grids, reducing coding efficiency.

[0135] Based on this, the present invention provides a coding method.

[0136] The embodiment of the present application also provides a grid architecture of a codec system including a decoding method and an encoding method. Figure 10 is a schematic diagram of a grid architecture of a codec provided by the embodiment of the present application. As shown in Figure 10, the grid architecture includes one or more electronic devices 13 to 1N and a communication grid 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication grid 01. During the implementation process, the electronic device can be various types of devices with codec functions. For example, the electronic device can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensor device, a server, etc., which is not specifically limited in the embodiment of the present application. Here, the decoder or encoder described in the embodiment of the present application can be the above-mentioned electronic device.

[0137] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0138] In one embodiment of the present application, referring to FIG11 , a flow chart of a coding method provided by the embodiment of the present application is shown. As shown in FIG11 , the method may include S101 to S104:

[0139] S101: Determine a first basic grid of the current frame according to the original grid of the current frame.

[0140] It should be noted that the encoding method of the embodiment of the present application may refer to an inter-frame encoding method, more specifically, an inter-frame encoding method for a matching grid in a dynamic grid. The encoding method can be applied to an encoder in a V-DMC, but is not limited thereto.

[0141] It should also be noted that, in the embodiment of the present application, the basic grid can also be called a "simplified grid".

[0142] In the embodiment of the present application, the first basic grid is a basic grid corresponding to the intra-frame coding mode.

[0143] In some embodiments of the present application, the implementation of determining the first basic grid of the current frame based on the original grid of the current frame may include: performing downsampling processing on the original grid of the current frame to determine the first basic grid.

[0144] In the embodiment of the present application, a mesh simplification algorithm, such as Quadric Error Metric (QEM) and Edge Collapse, is used to simplify the original mesh of the current frame. The mesh simplification algorithm usually deletes some unimportant vertices to reduce the complexity of the mesh while preserving the key features of the original mesh as much as possible.

[0145] In the embodiment of the present application, the downsampling methods include: uniform downsampling, random downsampling, feature-based downsampling, etc. The embodiment of the present application does not impose any restrictions on this, and the specific selection can be made according to the actual application scenario.

[0146] In an embodiment of the present application, a selected downsampling operation is performed to reduce the number of vertices in the original mesh to the number required by the first base mesh. This may involve deleting or merging vertices, edges, or faces to reduce the complexity of the mesh. It should be noted that the number of vertices in the first base mesh is less than the number of vertices in the original mesh of the current frame.

[0147] S102 : Based on the second basic mesh of the reference frame, select matching points for the original mesh of the current frame to obtain a third basic mesh; wherein the third basic mesh matches vertices in the second basic mesh.

[0148] In the embodiment of the present application, the vertices of the first base mesh are traversed and matched in the second base mesh. This can be achieved by comparing properties such as distance, normal vector, and color between the vertices. When a preset condition is met, the vertex of the first base mesh is associated with the matching vertex in the second base mesh.

[0149] In the embodiment of the present application, the second basic mesh of the reference frame is matched with vertices in the third basic mesh.

[0150] In some embodiments of the present application, the reference frame is an encoded frame before the current frame. For example, the reference image can be the frame before the current image, but this is not specifically limited.

[0151] In some embodiments of the present application, the third base mesh has the same number of vertices and connectivity as the second base mesh.

[0152] In the embodiment of the present application, the connectivity of the vertices includes at least one of the following: connectivity of vertex coordinates and connectivity of texture coordinates.

[0153] Among them, the connectivity of vertex coordinates refers to the relationship between vertices, that is, how to connect the vertices in the mesh to form edges and faces. For example, a rectangular model has 8 vertices, and the connectivity between these vertices forms 12 edges and 6 faces. The correct connection relationship ensures the correct shape of the rectangular model. The connectivity of texture coordinates refers to how the texture coordinates are associated with the vertex coordinates of the three-dimensional model during the texture mapping process to determine how the texture map on the surface of the model is mapped to the model. The connectivity of texture coordinates is often used to correctly attach texture images to the surface of a three-dimensional model to increase the appearance details and realism of the model.

[0154] In some embodiments of the present application, when the first basic grid and the second basic grid of the reference frame meet preset conditions, in some embodiments of the present application, matching points are selected for the original grid of the current frame based on the second basic grid of the reference frame to obtain a third basic grid.

[0155] In some embodiments of the present application, the preset conditions include: the number of vertices between the first base mesh and the second base mesh is the same; and / or the connectivity of vertex coordinates and / or the connectivity of texture coordinates between the first base mesh and the second base mesh is the same.

[0156] In the embodiment of the present application, S102 is implemented in the following situations:

[0157] Case 1: For any current frame, based on the second basic grid of the reference frame, matching points are selected for the original grid of the current frame to obtain a third basic grid.

[0158] Case 2: When the first basic mesh and the second basic mesh of the reference frame have the same number of vertices, matching points are selected from the original mesh of the current frame to obtain a third basic mesh.

[0159] Case 3: When the connectivity of the vertex coordinates of the first basic mesh and the second basic mesh of the reference frame is the same, matching points are selected for the original mesh of the current frame to obtain a third basic mesh.

[0160] Case 4: When the connectivity of the texture coordinates of the first basic mesh and the second basic mesh of the reference frame is the same, matching points are selected for the original mesh of the current frame to obtain a third basic mesh.

[0161] Case 5: When the number of vertices, connectivity of vertex coordinates, and connectivity of texture coordinates of the first basic mesh and the second basic mesh of the reference frame are the same, matching points are selected for the original mesh of the current frame to obtain a third basic mesh.

[0162] Case 6: When the number of vertices and the connectivity of vertex coordinates of the first basic mesh and the second basic mesh of the reference frame are the same, matching points are selected from the original mesh of the current frame to obtain a third basic mesh.

[0163] Case 7: When the number of vertices and the connectivity of texture coordinates of the first basic mesh and the second basic mesh of the reference frame are the same, matching points are selected for the original mesh of the current frame to obtain a third basic mesh.

[0164] Case 7: When the connectivity of vertex coordinates and texture coordinates of the first basic mesh and the second basic mesh of the reference frame are the same, matching points are selected for the original mesh of the current frame to obtain a third basic mesh.

[0165] S103: Determine a target coding mode for the current frame according to an error metric difference between the first basic grid and the third basic grid.

[0166] In the embodiment of the present application, the error metric difference includes: peak signal-to-noise ratio, mean square error, and mean absolute error, etc., and the present application does not impose any limitation on this.

[0167] In the embodiment of the present application, the error metric value of the first basic grid is subtracted from the error metric value of the third basic grid to obtain an error metric difference.

[0168] In some embodiments of the present application, the implementation of determining the target coding mode of the current frame according to the error metric difference between the first base grid and the third base grid in S103 may include S1031 to S1033:

[0169] S1031. Determine an error metric difference value based on error metric values ​​of the first basic grid and the third basic grid; wherein the error metric value includes: a peak signal-to-noise ratio;

[0170] S1032: If the error metric difference is less than or equal to a first preset threshold, determine that the target coding mode is an inter-frame coding mode;

[0171] S1033: If the error metric difference is greater than a first preset threshold, determine that the target coding mode is an intra-frame coding mode.

[0172] In the embodiment of the present application, the first preset threshold is a pre-set value. For example, the first preset threshold is 1.

[0173] In an embodiment of the present application, when the peak signal-to-noise ratio difference (i.e., error metric parameter) between the base grid generated by intra-frame coding (i.e., the first base grid) and the matching grid generated by inter-frame coding mode (i.e., the third base grid) is less than a threshold (i.e., the first preset threshold), the matching grid generated by the inter-frame coding mode is selected as the base grid of the current frame, and inter-frame coding is performed on the current frame; otherwise, the base grid generated by intra-frame coding is selected as the base grid of the current frame, and intra-frame coding is performed.

[0174] It is understandable that by comparing the error metric value with the preset threshold, an appropriate encoding mode can be selected according to different situations to achieve higher encoding efficiency or image quality.

[0175] S104: Encode the current frame according to the target encoding mode.

[0176] In the embodiment of the present application, the target coding mode is any one of the intra-frame coding mode and the inter-frame coding mode.

[0177] In some embodiments of the present application, the implementation of encoding the current frame according to the target encoding mode in S104 may include: S1041 or S1042:

[0178] S1041 : When the target coding mode is the intra-frame coding mode, perform coding processing on the first basic grid of the current frame, and write the obtained coding bits into a bitstream.

[0179] S1042: When the target coding mode is the inter-frame coding mode, perform coding processing on the motion vector corresponding to the third basic grid of the current frame, and write the obtained coding bits into the bitstream.

[0180] It should be noted that either S1041 or S1042 is executed selectively, that is, the encoder can execute S1041 or S1042.

[0181] In some embodiments of the present application, the method further comprises:

[0182] When the target coding mode is the inter-frame coding mode, for at least one second vertex in the second basic mesh of the reference frame, determining, in the third basic mesh, fifth vertices respectively corresponding to the at least one second vertex;

[0183] determining a motion vector corresponding to the third basic mesh according to a difference between coordinate information of a fifth vertex corresponding to the at least one second vertex and coordinate information of the at least one second vertex;

[0184] The motion vector is coded and the resulting coded bits are written into the bitstream.

[0185] In an embodiment of the present application, traversing each second vertex in the second basic mesh of the reference frame can be implemented by looping or iterating to process each second vertex. For each second vertex, it is necessary to determine the fifth vertex corresponding to it in the third basic mesh. This usually requires the use of a matching or mapping relationship to ensure that the second vertex corresponds to the correct vertex in the third basic mesh. In order to achieve correct mapping, a mapping table or association table can be created to associate each second vertex in the second basic mesh with the corresponding fifth vertex in the third basic mesh. This can be a data structure, such as a dictionary or an array, which makes looking up the mapping relationship more efficient.

[0186] Furthermore, when traversing each second vertex of the second base mesh, the corresponding fifth vertex is found by searching a mapping table or association table. The coordinates of this fifth vertex are used to construct the third base mesh, ensuring correct connectivity and geometric correspondence. The third base mesh is constructed using the found fifth vertex coordinates and the connectivity information of the second base mesh. This ensures that the third base mesh has the same number of vertices and connectivity as the second base mesh.

[0187] In an embodiment of the present application, for each second vertex, the difference between the coordinates of the fifth vertex and the coordinates of the second vertex is calculated. This will generate a displacement vector representing the movement of the second vertex relative to the fifth vertex. The difference can be calculated on the x, y, and z axes respectively. For each second vertex, the calculated difference is constructed into a motion vector. Generally, a motion vector is a three-dimensional vector that contains displacement information on each axis.

[0188] For example, the process of determining the motion vector corresponding to the third basic mesh according to the difference between the coordinate information of the fifth vertex corresponding to each second vertex and the coordinate information of each second vertex can be described by formula (1): mv = M – R (1)

[0189] In formula (1), mv represents the motion vector (also called motion vector) corresponding to the third basic grid, R represents the coordinate information of each second vertex (i.e., the point coordinates of the reference frame basic grid), and M represents the coordinate information of the fifth vertex corresponding to each second vertex (i.e., the corresponding target point coordinates).

[0190] In some embodiments of the present application, the method further comprises:

[0191] When the motion vector is greater than or equal to a second preset threshold, determining a neighbor point of the at least one fifth vertex according to connection information of the at least one fifth vertex;

[0192] Adjusting the at least one fifth vertex according to relative position information between a neighbor point of the at least one fifth vertex and the at least one fifth vertex to obtain adjusted vertices corresponding to the at least one fifth vertex;

[0193] An adjusted third basic mesh is obtained according to the adjusted vertex corresponding to the at least one fifth vertex.

[0194] In the embodiment of the present application, the second preset threshold is a preset value.

[0195] In an embodiment of the present application, for each fifth vertex, its neighboring points are determined based on the connection information. The connection information typically includes the indices or identifiers of the other vertices connected to each vertex. By considering the neighbor information of each point, the accuracy of matching point selection can be improved. Neighboring points are usually located in similar contexts, so their matching points are more likely to be accurate. By considering the matching information of neighboring points, it can help determine the matching position of the current point.

[0196] In this embodiment of the present application, vertex adjustment is performed for each fifth vertex and its neighboring points based on relative position information. The relative position information may include an offset vector or displacement information describing how to adjust the vertex position. The relative position information is applied to each fifth vertex to adjust it to its new position. This results in the adjusted position of each fifth vertex.

[0197] It can be understood that by comparing the matching positions of neighboring points and their positions relative to the current point, the deviation of the matching points can be detected and corrected, which helps to adjust the position of the matching points to a more accurate position to reflect the actual movement or change.

[0198] In the embodiment of the present application, the adjusted position of the fifth vertex and the connectivity information are used to construct the adjusted third basic mesh, ensuring that the updated vertex position and connectivity information are consistent to maintain the connectivity and geometric correctness of the mesh.

[0199] It is understood that by adjusting the fifth vertex and generating the adjusted mesh, a more natural and smoother deformation or fluidity can be achieved. In addition, it can be ensured that the adjusted mesh still maintains correct connectivity to prevent mesh breakage or unreasonable geometry.

[0200] In some embodiments of the present application, the method further comprises:

[0201] Iteratively divide the first basic grid or the third basic grid according to a grid subdivision algorithm to determine a subdivision grid of the current frame; the grid subdivision algorithm includes: a linear interpolation algorithm;

[0202] For at least one sixth vertex in the subdivided mesh of the current frame, determining a seventh vertex corresponding to the at least one fifth vertex in the first base mesh;

[0203] determining, according to a difference between coordinate information of at least one sixth vertex and coordinate information of a seventh vertex corresponding to at least one sixth vertex, shift information corresponding to the subdivided grid of the current frame;

[0204] The shift information is encoded and the obtained encoded bits are written into the bit stream.

[0205] In the embodiment of the present application, the network subdivision algorithm may include: a linear interpolation algorithm, a Catmull-Clark subdivision algorithm, or a Loop subdivision algorithm, and the present application does not impose any limitation on this.

[0206] In the embodiment of the present application, the first base mesh or the third base mesh is iteratively divided using a selected mesh subdivision algorithm. In each iteration, each face in the mesh is subdivided into more sub-faces, and each vertex is subdivided into more sub-vertices. This process can be repeated multiple times until the desired subdivision level is reached.

[0207] In this embodiment of the present application, for each sixth vertex in the subdivided mesh of the current frame, the subdivided mesh is traversed. The fifth vertex corresponding to each sixth vertex is determined, and this process requires some matching or mapping method to ensure correct matching. The seventh vertex corresponding to the fifth vertex in the first base mesh is found, and this process requires a lookup or mapping method to achieve this.

[0208] Furthermore, for each sixth vertex, the difference between the coordinates of the sixth vertex and the coordinates of the corresponding seventh vertex is calculated, which will generate a displacement vector representing the movement of the sixth vertex relative to the seventh vertex. The difference can be calculated on the x, y, and z axes respectively. For each sixth vertex, the calculated difference is constructed into a displacement information. Generally, the displacement information is a three-dimensional vector that contains the displacement information on each axis.

[0209] For example, the process of determining the displacement information corresponding to the subdivided grid of the current frame according to the difference between the coordinate information of each sixth vertex and the coordinate information of the first vertex corresponding to each sixth vertex can be described by formula (2): disp = M′ - I (2)

[0210] In formula (2), disp represents the displacement information corresponding to the subdivided mesh of the current frame, M′ represents the coordinate information of each sixth vertex (i.e., after multiple subdivisions, the vertex coordinates of the subdivided mesh are obtained after each midpoint subdivision), and I represents the coordinate information of the seventh vertex corresponding to each sixth vertex (i.e., the coordinates of the point closest to the corresponding M′ vertex on the input original mesh surface).

[0211] In some embodiments of the present application, the method further comprises:

[0212] Determine a value of syntax identification information; wherein the syntax identification information is used to indicate a target coding mode used for the current frame;

[0213] The value of the syntax identification information is encoded, and the obtained encoded bits are written into the code stream.

[0214] In the embodiment of the present application, if the value of the syntax identification information is the first value, it is determined that the syntax identification information indicates that the target coding mode used by the current frame is the intra-frame coding mode;

[0215] If the value of the syntax identification information is the second value, it is determined that the syntax identification information indicates that the target coding mode used by the current frame is the inter-frame coding mode.

[0216] In the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the first syntax identification information can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0217] For example, for the first value and the second value, the first value can be set to 1 and the second value can be set to 0; or the first value can be set to 0 and the second value can be set to 1; or the first value can be set to true and the second value can be set to false; or the first value can be set to false and the second value can be set to true. In the embodiment of the present application, the first value is set to 1 and the second value is set to 0, but this is not a specific limitation.

[0218] In some embodiments of the present application, as shown in FIG12 , the selection of matching points from the original grid of the current frame in S102 to obtain the third basic grid may include S1021 to S1024:

[0219] S1021. Determine, from at least one first vertex in the original mesh of the current frame, a matching vertex that matches at least one second vertex in the second basic mesh; wherein the at least one first vertex includes a matching vertex.

[0220] In this embodiment of the present application, each second vertex in the second base mesh is traversed. Each second vertex typically has a unique identifier or index. For each second vertex, a matching vertex is selected in the original mesh of the current frame. The matching method can be based on various criteria, such as minimum distance, feature matching, etc., which are not limited in this application.

[0221] In some embodiments of the present application, determining, from at least one first vertex in the original mesh of the current frame, a matching vertex that matches each second vertex in the second base mesh includes:

[0222] Construct the octree structure of the original mesh of the current frame;

[0223] For each second vertex in the second basic mesh, at least one first vertex in the octree structure is traversed, and a first vertex in the octree structure that is closest to each second vertex is used as a matching vertex for each second vertex.

[0224] In an embodiment of the present application, the process of constructing an octree structure of the original mesh of the current frame may include: recursively dividing the space occupied by the original mesh of the current frame to construct an octree structure. Each node represents a spatial area, the root node represents the entire space, and the child nodes represent eight equal subspaces of the space contained in the parent node. Further, vertex information in the original mesh corresponding to the node is allocated or stored for each node. This can be information such as the coordinates and index of the vertex within the node. Further, appropriate subdivision rules are defined to decide when to stop splitting the tree, usually based on factors such as the number of vertices contained in the spatial area and the size of the spatial area.

[0225] In this embodiment of the present application, for each second vertex, by traversing at least one first vertex in the octree structure, finding the first vertex closest to the second vertex, this can be achieved by recursively traversing the octree structure. Once a matching first vertex is found, it is associated with the corresponding second vertex to indicate the matching relationship between them. Repeat the above steps until you have traversed all second vertices in the second base mesh and found a matching vertex for each second vertex.

[0226] It is understood that an octree structure is constructed to represent the spatial distribution of the original mesh, and then each second vertex in the second base mesh is matched by traversing the octree structure. The matching process is generally based on a nearest neighbor search to determine a matching vertex for each second vertex, thereby improving the matching and association between meshes.

[0227] For example, the process of using the first vertex closest to each second vertex in the octree structure as the matching vertex for each second vertex can be described by formula (3):

[0228] In formula (3), dist represents the Euclidean distance between the second vertex and the first vertex, R represents the second vertex in the second base mesh, and I represents the first vertex in the original mesh of the current frame.

[0229] S1022: Determine an edge error of at least one connecting edge associated with at least one matching vertex based on coordinate information of two connecting vertices of at least one connecting edge associated with at least one matching vertex.

[0230] In the embodiment of the present application, the edge error is usually caused by the distance between connected vertices or other geometric properties.

[0231] In some embodiments of the present application, determining an edge error of at least one connecting edge associated with at least one matching vertex based on coordinate information of two connected vertices associated with at least one connecting edge associated with at least one matching vertex includes:

[0232] For any one of the at least one connecting edge associated with at least one matching vertex, based on coordinate information of the two connecting vertices associated with the any one connecting edge, determining a vertex error between the two connecting vertices associated with the any one connecting edge;

[0233] The edge error of any connecting edge is determined according to the vertex errors of the two connected vertices associated with any connecting edge.

[0234] In some embodiments of the present application, a weighted sum of vertex errors of two connected vertices associated with any connecting edge is performed to obtain an edge error of any connecting edge.

[0235] For example, the vertex errors of two connected vertices associated with any connecting edge can be directly added to obtain the edge error of any connecting edge, or the vertex errors of two connected vertices associated with any connecting edge can be multiplied by corresponding weighting coefficients and then added to obtain the edge error of any connecting edge. This application is not limited to this.

[0236] In some embodiments of the present application, determining the vertex error between two connected vertices associated with any connecting edge based on the coordinate information of the two connected vertices associated with any connecting edge may include:

[0237] For any of the two connected vertices associated with any connecting edge, the vertex error of any connected vertex is determined based on the plane set corresponding to any connected vertex and the coordinate information of any connected vertex; wherein the plane set includes at least one intersecting plane associated with any connected vertex.

[0238] In the embodiment of the present application, the plane set corresponding to any connected vertex refers to all planes associated with the connected vertex, and any connected vertex is associated with multiple planes. The plane set corresponding to any connected vertex can be represented by p∈planes(v).

[0239] In an embodiment of the present application, the process of determining the plane set corresponding to any connected vertex may include: obtaining information about all planes associated with the connecting edge of any connected vertex. These planes are generally defined by the coordinate information of the connected vertex and attributes such as the normal vector. By determining whether the connected vertex is located on or close to a plane, the plane associated with it is selected. All planes associated with the connected vertex that meet the conditions are constructed into a plane set, which contains all planes associated with the connected vertex.

[0240] For example, the process of determining the vertex error of any connected vertex according to the plane set corresponding to any connected vertex and the coordinate information of any connected vertex can be described by formula (4):

[0241] In formula (4), represents the vertex error of any connected vertex v, planes(v) represents the plane set corresponding to any connected vertex v, and p represents a plane in the plane set.

[0242] In the embodiment of the present application, determining the vertex error of any connected vertex according to the plane set corresponding to any connected vertex and the coordinate information of any connected vertex includes:

[0243] According to the plane equation corresponding to each plane in the plane set corresponding to any connected vertex, determine the coefficient vector corresponding to each plane; wherein the coefficient vector is used to represent the coefficient of the plane equation;

[0244] The vertex error of any connected vertex is determined based on the coefficient vector of each plane and the coordinate information of any connected vertex.

[0245] In the embodiment of the present application, the plane equation corresponding to each plane in the plane set corresponding to any connected vertex can be expressed as ax+by+cz+d=0. Where a, b, c, and d are the coefficients of the plane equation, which can be calculated based on the coordinate information in the plane equation. The coefficient vector corresponding to each plane can be expressed as [abcd] T It should be noted that a, b and c satisfy a 2 +b 2 +c 2 =1 relationship.

[0246] In some embodiments of the present application, as shown in FIG13 , in S1022, optimizing the matching vertex corresponding to the second vertex based on the edge error of at least one connecting edge respectively associated with at least one matching vertex to obtain the optimized matching vertex respectively corresponding to the at least one matching vertex may include S201 to S203:

[0247] S201. For a current matching vertex corresponding to a current second vertex in a second basic mesh, determine a current edge to be collapsed of the current matching vertex based on an edge error of at least one connecting edge associated with the current matching vertex; wherein the edge to be collapsed is a connecting edge associated with the current matching vertex in the current original mesh; and the current original mesh is obtained by optimizing the original mesh of the current frame.

[0248] In some embodiments of the present application, a connecting edge corresponding to a minimum error among edge errors of at least one connecting edge associated with the current matching vertex is determined as the current edge to be collapsed corresponding to the current matching vertex.

[0249] S202. Optimize the edge error of the current edge to be collapsed according to the coefficient vector corresponding to the plane where the two points to be collapsed are located, obtain the current optimized matching vertex corresponding to the current matching vertex, and obtain the next original mesh after the collapse process.

[0250] In some embodiments of the present application, the error matrix of the two points to be collapsed associated with the current edge to be collapsed is determined based on the coefficient vector corresponding to the plane where the two points to be collapsed associated with the current edge to be collapsed are located; the target error matrix of the current edge to be collapsed is determined based on the error matrix of the two points to be collapsed associated with the current edge to be collapsed; through the error measurement model, the edge error of the current edge to be collapsed is optimized according to the target error matrix to obtain the target edge error, and the collapsed vertex corresponding to the target edge error is used as the current optimized matching vertex corresponding to the current matching vertex.

[0251] In the embodiment of the present application, the target error matrix and the error metric model are used to optimize the edge error of the edge to be collapsed. This optimization process may involve minimizing the error or optimizing according to a specific optimization goal.

[0252] In the embodiment of the present application, the error matrix of the two points to be collapsed associated with the current edge to be collapsed can be expressed by formula (5):

[0253] In the embodiment of the present application, the error matrices of the two points to be collapsed associated with the current edge to be collapsed are weighted summed to obtain the target error matrix of the current edge to be collapsed. The above process can be described by formula (6): Q E =q v +q v′ (6)

[0254] In formula (6), Q E Represents the target error matrix of the current edge to be collapsed, q v Represents the error matrix of a point to be collapsed associated with the current edge to be collapsed, q v′ Represents the error matrix of another point to be collapsed associated with the current edge to be collapsed.

[0255] In the embodiment of the present application, Q E It can be expressed by formula (7):

[0256] In an embodiment of the present application, after obtaining the target error matrix of the current edge to be collapsed, the edge error of the current edge to be collapsed is optimized according to the target error matrix through the error measurement model to obtain the target edge error.

[0257] In some embodiments of the present application, the error metric model is used to characterize the relationship between the target error matrix, the edge error of the edge to be collapsed, and the collapsed vertex; the collapsed vertex is the vertex obtained after collapsing the edge to be collapsed, and the next original mesh after the collapse processing is obtained.

[0258] In the embodiment of the present application, the error measurement model can be expressed by formula (8):

[0259] In formula (8), Error represents the edge error of the edge to be collapsed, Indicates the coordinate information of the collapsed vertex, Q E Represents the target error matrix of the current edge to be collapsed. x, y, and z represent points coordinate information.

[0260] Based on formula (7) and formula (8), we can get:

[0261] In the embodiment of the present application, the partial derivative of Error can be obtained to obtain the minimum time point of the error metric. After calculating and comparing the error metrics for all edges containing point v, the edge with the smallest error is selected for collapse. The coordinates of the collapsed point are the coordinates of the target point. Repeat the nearest point selection and target point calculation process until each point in the reference frame base mesh has a corresponding target point. During this process, the original input mesh is continuously collapsed and simplified, with each collapse based on the previous collapse.

[0262] In the embodiment of the present application, for each target edge error corresponding to the coordinate component (x, y, z) of the collapsed vertex, the Error is calculated relative to This can be done by calculating the partial derivative of Error. Set each partial derivative to zero and then solve the equation to find the minimum error point. This will produce a system of three equations, one for each coordinate component (x, y, z). Solve this system of equations to find the equation that minimizes Error. 's coordinates.

[0263] S203. Continue to determine the next edge to be collapsed of the next matching vertex based on the edge error of at least one connecting edge associated with the next matching vertex, and obtain the next optimized matching vertex corresponding to the next matching vertex, until the matching vertices corresponding to the second vertex in the second basic mesh are traversed, thereby obtaining the optimized matching vertices corresponding to at least one matching vertex; wherein, the next second vertex has a connection relationship with the current second vertex, and the vertex identification order of the current second vertex is before the next second vertex.

[0264] In the embodiment of the present application, the vertex identification order is a pre-set order, for example, the identifier or index of each vertex has a certain order. For example, the vertex identification order is 0, 1, 2, ..., n, the current second vertex is identified as 5, and the next second vertex is identified as 6.

[0265] S1023 . Optimize the matching vertex corresponding to the at least one second vertex based on the edge error of the at least one connecting edge respectively associated with the at least one matching vertex, to obtain an optimized matching vertex respectively corresponding to the at least one matching vertex.

[0266] S1024: Determine a third basic mesh according to the optimized matching vertex corresponding to the at least one matching vertex.

[0267] In some embodiments of the present application, for any one of the first basic grid and the third basic grid, the error metric value of any one of the basic grids is determined based on the geometric distance peak of the original grid of the current frame and the target distortion result corresponding to any one of the basic grids; wherein the target distortion result is used to characterize the distortion of any one of the basic grids relative to the original grid of the current frame.

[0268] In the embodiment of the present application, the geometric distance peak of the original mesh of the current frame refers to the maximum value of the geometric distances between all vertices in the original mesh of the current frame, which represents the distance between the two farthest points in the mesh.

[0269] In an embodiment of the present application, all first vertices in the original mesh of the current frame are traversed, and for each pair of first vertices, the geometric distance between them is calculated. This can be accomplished by calculating the Euclidean distance between the two points. During the traversal process, a variable is maintained to store the maximum geometric distance. Whenever a larger distance is found, this variable is updated. After completing the traversal of all vertices, the maintained maximum distance is the peak geometric distance of the original mesh of the current frame.

[0270] In some embodiments of the present application, the method further includes: determining a target distortion result based on a first distortion result and / or a second distortion result corresponding to any basic grid; wherein the first distortion result is used to characterize the distortion result of any basic grid relative to the original grid of the current frame; and the second distortion result is used to characterize the distortion result of the original grid of the current frame relative to any basic grid.

[0271] In some embodiments of the present application, the maximum distortion result between the first distortion result and the second distortion result is used as the target distortion result.

[0272] Exemplarily, the step of determining the error metric value of any base grid according to the first distortion result and the second distortion result of any base grid can be described by formula (9):

[0273] In formula (9), PSNR represents the error metric of any basic grid, p G Indicates the geometric distance peak of the original grid of the current frame (that is, the geometric distance peak of the input grid), Represents the distortion result of any base grid relative to the original grid of the current frame (i.e., the distortion of grid B relative to grid A). Indicates the distortion result of the original mesh of the current frame relative to any base mesh (i.e., the distortion of mesh A relative to mesh B).

[0274] In some embodiments of the present application, the first distortion result includes: a first distortion value; the first distortion value represents a point-to-point distortion error of any base grid relative to an original grid of the current frame; and the method further includes:

[0275] Upsampling any base mesh and the original mesh of the current frame to obtain an updated original mesh and any updated base mesh; wherein the updated original mesh and any updated base mesh have the same number of vertices;

[0276] For at least one third vertex in any updated base mesh, determining a Euclidean distance between the at least one third vertex and corresponding fourth vertices in the updated original mesh;

[0277] determining a first distortion value according to the number of vertices in any updated base mesh and the Euclidean distance corresponding to at least one third vertex;

[0278] A first distortion value is determined as a first distortion result.

[0279] In an embodiment of the present application, for each base mesh and the original mesh of the current frame, new vertices need to be interpolated to generate. The interpolation method can be selected according to needs, such as linear interpolation, Catmull-Clark algorithm, etc. These new vertices will fill the blank areas of the original mesh. As new vertices are added, the connection information needs to be updated to ensure that the connection relationship between the new vertices and the old vertices is maintained. This may involve updating information such as edges, faces, and normal vectors. Ensure that after upsampling, the updated original mesh and the updated base mesh have the same number of vertices, which can be achieved by controlling the upsampling multiple and the interpolation method. After the upsampling is completed, you will get an updated original mesh and an updated base mesh, which have the same number of vertices.

[0280] In the embodiment of the present application, the process of determining the first distortion value according to the number of vertices in any updated base mesh and the Euclidean distance corresponding to each third vertex can be described by formula (10) and formula (11):

[0281] In formula (10) and formula (11), represents the first distortion value of each third vertex in any updated base mesh, E(i, j) represents the Euclidean distance between each third vertex in any updated base mesh and the fourth vertex j corresponding to each third vertex i in the updated original mesh, C B Indicates the number of vertices in any base mesh that is updated, Indicates the first distortion value.

[0282] In the embodiment of the present application, for formula (10), after upsampling any base grid and the original grid of the current frame, the square of the Euclidean distance between any point in any base grid and the corresponding point in the original grid of the current frame is numerically the square of the norm of the error vector E(i, j) between the two.

[0283] In some embodiments of the present application, the first distortion result further includes: a second distortion value; the second distortion value represents a point-to-surface distortion error of any base grid relative to the original grid of the current frame;

[0284] After determining the first distortion value based on the updated number of vertices in any one of the base meshes and the Euclidean distance corresponding to at least one third vertex, the method further includes:

[0285] For at least one third vertex in any updated base mesh, determining an error vector between the at least one third vertex and corresponding fourth vertices in the updated original mesh;

[0286] Projecting the error vectors corresponding to the at least one third vertex along the direction of the normal vector of the fourth vertex corresponding to the at least one third vertex to obtain a projection error vector;

[0287] determining a second distortion value according to the number of vertices in any updated base mesh and the projection error vector corresponding to at least one third vertex;

[0288] A first distortion result is determined according to the first distortion value and the second distortion value.

[0289] In an embodiment of the present application, for each third vertex and its corresponding fourth vertex, an error vector is calculated between them. The error vector generally represents the difference between the two points and can be calculated by subtracting the coordinates of the third vertex (in the updated original mesh) and the coordinates of the fourth vertex (in the updated base mesh). For example, if the coordinates of the third vertex are (x3, y3, z3) and the coordinates of the fourth vertex are (x4, y4, z4), the error vector can be expressed as: E(i, j) = (x3-x4, y3-y4, z3-z4).

[0290] Furthermore, for each third vertex and its corresponding fourth vertex, a normal vector needs to be calculated. The normal vector is usually the difference between two vertices and is normalized to ensure that it represents a unit vector. Project the error vector corresponding to each third vertex along the direction of the normal vector of its corresponding fourth vertex. This can be achieved by dot multiplication (inner product) of the normal vector. The projected error vector can be expressed as: P(i,j) = E(i,j) N(i,j), where P(i,j) is the projected error vector, E(i,j) is the original error vector, and N(i,j) is the normal vector.

[0291] In the embodiment of the present application, the second distortion value is determined according to the number of vertices in any updated basic mesh and the projection error vector corresponding to each third vertex, which can be described by formula (12) and formula (13):

[0292] In formula (12) and formula (13), represents the second distortion value of each third vertex in any updated base mesh, E(i, j) represents the Euclidean distance between each third vertex in any updated base mesh and the fourth vertex j corresponding to each third vertex i in the updated original mesh, C B Indicates the number of vertices in any base mesh that is updated, Represents the first distortion value, N j represents the normal vector of the fourth vertex corresponding to each third vertex, Represents the projection error vector corresponding to each third vertex.

[0293] In the embodiment of the present application, for formula (12), after upsampling any base grid and the original grid of the current frame, the distance between any point in any base grid and the corresponding point a in the original grid of the current frame is j , a j The error vector E(i,j) along a j Normal vector N j Directional projection, get the projection error vector

[0294] In the embodiment of the present application, determining the first distortion result according to the first distortion value and the second distortion value includes: performing a weighted summation of the first distortion value and the second distortion value to obtain the first distortion result.

[0295] In some embodiments of the present application, the second distortion result includes: a third distortion value; the third distortion value represents a point-to-point distortion error of the original grid of the current frame relative to any basic grid;

[0296] The method also includes:

[0297] Upsampling any base mesh and the original mesh of the current frame to obtain an updated original mesh and any updated base mesh; wherein the updated original mesh and any updated base mesh have the same number of vertices;

[0298] For at least one fourth vertex in the updated original mesh, determining a Euclidean distance between the at least one fourth vertex and corresponding third vertices in any updated base mesh;

[0299] determining a third distortion value according to the number of vertices in the updated original mesh and the Euclidean distance corresponding to at least one fourth vertex;

[0300] A third distortion value is determined as the second distortion result.

[0301] In some embodiments of the present application, the second distortion result further includes: a fourth distortion value; the fourth distortion value represents a point-to-surface distortion error of the original grid of the current frame relative to any base grid;

[0302] After determining the third distortion value based on the number of vertices in the updated original mesh and the Euclidean distance corresponding to at least one fourth vertex, the method further includes:

[0303] For at least one fourth vertex in the updated original mesh, determining an error vector between the at least one fourth vertex and the third vertices respectively corresponding to the at least one fourth vertex in any updated base mesh;

[0304] Projecting the error vector corresponding to the at least one fourth vertex along the direction of the normal vector of the third vertex corresponding to the at least one fourth vertex to obtain a projection error vector;

[0305] determining a fourth distortion value according to the number of vertices in the updated original mesh and a projection error vector corresponding to at least one fourth vertex;

[0306] A second distortion result is determined according to the third distortion value and the fourth distortion value.

[0307] In the embodiment of the present application, determining the second distortion result according to the third distortion value and the fourth distortion value includes: performing a weighted summation of the third distortion value and the fourth distortion value to obtain the second distortion result.

[0308] An embodiment of the present application provides a coding method, which determines the first basic grid of the current frame based on the original grid of the current frame; when the first basic grid and the second basic grid of the reference frame meet the preset conditions, the original grid of the current frame is matched with the points to obtain a third basic grid; wherein the third basic grid matches the vertices in the second basic grid; the target coding mode of the current frame is determined based on the error metric difference between the first basic grid and the third basic grid; and the current frame is coded according to the target coding mode. In this way, on the one hand, when the first basic grid and the second basic grid of the reference frame meet the preset conditions, the third basic grid is generated by selecting the best matching points on the original grid of the current frame, thereby expanding the scope of application of the inter-frame coding mode, greatly reducing the bit rate of the coding basic grid, and ensuring the quality of the reconstructed grid. On the other hand, by selecting the appropriate target coding mode of the current frame based on the error metric difference, the appropriate target coding mode can be adaptively selected in different scenarios, which can improve the coding efficiency of the grid while ensuring the quality of grid reconstruction.

[0309] In another embodiment of the present application, referring to FIG14 , a schematic flow chart of a decoding method provided by an embodiment of the present application is shown. As shown in FIG14 , the method may include S301 to S303:

[0310] S301: Parse the code stream to obtain syntax identification information.

[0311] It should be noted that the decoding method of the embodiment of the present application may refer to an inter-frame coding method, more specifically, an inter-frame decoding method for a matching grid in a dynamic grid. The decoding method can be applied to a decoder in a V-DMC, but is not limited thereto.

[0312] It should also be noted that, in the embodiment of the present application, the basic grid can also be called a "simplified grid".

[0313] In the embodiment of the present application, the first basic grid is a basic grid corresponding to the intra-frame decoding mode.

[0314] In the embodiment of the present application, if the value of the syntax identification information is the first value, it is determined that the syntax identification information indicates that the target coding mode used by the current frame is the intra-frame coding mode;

[0315] If the value of the syntax identification information is the second value, it is determined that the syntax identification information indicates that the target coding mode used by the current frame is the inter-frame coding mode.

[0316] In the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the first syntax identification information can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0317] For example, for the first value and the second value, the first value can be set to 1 and the second value can be set to 0; or the first value can be set to 0 and the second value can be set to 1; or the first value can be set to true and the second value can be set to false; or the first value can be set to false and the second value can be set to true. In the embodiment of the present application, the first value is set to 1 and the second value is set to 0, but this is not a specific limitation.

[0318] S302. When the syntax identification information indicates that the coding mode adopted by the current frame is the inter-frame coding mode, parse and obtain the motion vector corresponding to the third basic grid of the current frame and the shift information corresponding to the subdivided grid of the current frame; wherein the third basic grid is based on the second basic grid of the reference frame and is obtained by selecting matching points on the original grid of the current frame; the third basic grid matches the vertices in the second basic grid.

[0319] In some embodiments of the present application, the reference frame is a decoded frame before the current frame. For example, the reference image can be the frame before the current image, but this is not specifically limited.

[0320] In some embodiments of the present application, the third base mesh has the same number of vertices and connectivity as the second base mesh.

[0321] In the embodiment of the present application, the connectivity of the vertices includes at least one of the following: connectivity of vertex coordinates and connectivity of texture coordinates.

[0322] Among them, the connectivity of vertex coordinates refers to the relationship between vertices, that is, how to connect the vertices in the mesh to form edges and faces. For example, a rectangular model has 8 vertices, and the connectivity between these vertices forms 12 edges and 6 faces. The correct connection relationship ensures the correct shape of the rectangular model. The connectivity of texture coordinates refers to how the texture coordinates are associated with the vertex coordinates of the three-dimensional model during the texture mapping process to determine how the texture map on the surface of the model is mapped to the model. The connectivity of texture coordinates is often used to correctly attach texture images to the surface of a three-dimensional model to increase the appearance details and realism of the model.

[0323] In some embodiments of the present application, the preset conditions include: the number of vertices between the first base mesh and the second base mesh is the same; and / or the connectivity of vertex coordinates and / or the connectivity of texture coordinates between the first base mesh and the second base mesh is the same.

[0324] S303: Determine a reconstruction grid for the current frame according to the motion vector and the shift information.

[0325] In some embodiments of the present application, the implementation of determining the reconstruction grid of the current frame according to the motion vector and the shift information in S303 includes:

[0326] Determine a third basic grid according to the motion vector and the second basic grid;

[0327] A reconstructed grid for the current frame is determined according to the shift information and the third basic grid.

[0328] In some embodiments of the present application, determining a third base grid according to the motion vector and the second base grid includes:

[0329] For at least one second vertex in the second basic mesh of the reference frame, determining a motion offset corresponding to the at least one second vertex in the motion vector;

[0330] A third basic mesh of the current frame is determined according to the sum of the motion offset corresponding to the at least one second vertex and the coordinate information of the at least one second vertex.

[0331] In some embodiments of the present application, determining the reconstructed grid according to the displacement information and the third basic grid includes:

[0332] Iteratively divide the third basic grid according to a grid subdivision algorithm to determine a subdivision grid of the current frame; the grid subdivision algorithm includes: a linear interpolation algorithm;

[0333] For at least one sixth vertex in the subdivided mesh of the current frame, determine a shift offset corresponding to the at least one sixth vertex in the shift information;

[0334] determining, according to the coordinate information of the at least one sixth vertex and the difference between the shift offset corresponding to the at least one sixth vertex, a seventh vertex in the reconstructed mesh corresponding to the at least one sixth vertex;

[0335] A reconstructed mesh of the current frame is determined according to a seventh vertex corresponding to the at least one sixth vertex.

[0336] In some embodiments of the present application, the method further comprises:

[0337] When the syntax identification information indicates that the coding mode adopted by the current frame is the intra-frame coding mode, parsing the code stream to obtain the first basic grid of the current frame and the shift information corresponding to the subdivided grid of the current frame;

[0338] A reconstructed grid of the current frame is determined according to the first basic grid and the shift information.

[0339] The embodiment of the present application provides a decoding method, which parses a bitstream to obtain syntax identification information; when the syntax identification information indicates that the coding mode adopted by the current frame is the inter-frame coding mode, the method parses and obtains the motion vector corresponding to the third basic grid of the current frame and the displacement information corresponding to the subdivided grid of the current frame; wherein the third basic grid is obtained by selecting matching points on the original grid of the current frame when the first basic grid and the second basic grid of the reference frame meet a preset condition; the third basic grid matches the vertices in the second basic grid; and the reconstructed grid of the current frame is determined based on the motion vector and the displacement information. In this way, since the third basic grid is determined by selecting the best matching point on the original grid of the current frame when the first basic grid and the second basic grid of the reference frame meet a preset condition, the applicable scope of the inter-frame coding mode can be expanded, the code rate of the decoded basic grid can be greatly reduced, and the quality of the reconstructed grid can be guaranteed. While ensuring the quality of the grid reconstruction, the decoding efficiency of the grid can be improved.

[0340] In another embodiment of the present application, the embodiment of the present application takes into account that it is difficult to apply the inter-frame coding mode to sequences with larger motion vectors. According to its restrictive conditions, it is proposed to ensure that the basic grid of the previous frame has the same number of points and connectivity, and combine the quadratic error surface algorithm to select the best matching points on the original grid of the current frame input to generate a matching grid, thereby expanding the scope of application of the inter-frame coding mode, greatly reducing the bit rate of the encoded basic grid, and ensuring the quality of the reconstructed grid.

[0341] Figure 15 is a fourth flow chart of a coding method provided by an embodiment of the present application. As shown in Figure 15, the method includes: simplifying the original grid of the current frame to obtain the first basic grid of the current frame, and calculating the motion of two adjacent frames of the original grid of the current frame to obtain the third basic grid of the current frame. Further, according to the error metric difference between the first basic grid and the third basic grid, the target coding mode of the current frame is determined. When the target coding mode is the intra-frame coding mode, the motion vector corresponding to the third basic grid is encoded; when the target coding mode is the inter-frame coding mode, the first basic grid pair is encoded. Further, the displacement information of the subdivided grid corresponding to the first basic grid or the third basic grid is encoded.

[0342] In the embodiment of the present application, since the first input frame is intra-coded, a matching mesh (equivalent to the third base mesh) is generated for the current input frame (equivalent to the current frame) starting from the second frame. Since the octree structure is a good way to find neighbor points and the nearest point, an octree structure is constructed for the input original mesh of the current frame (equivalent to the first base mesh). All points in the base mesh of the reference frame (previous frame) (equivalent to the second base mesh) are traversed, and according to formula (3), the coordinates of the corresponding nearest point in the input original mesh (equivalent to the matching vertex) are calculated.

[0343] In the embodiment of the present application, after obtaining the coordinates of the corresponding points of all reference frames, the optimized point coordinates are calculated based on the corresponding point coordinates using the quadratic error surface algorithm. The algorithm is specifically described as follows: based on the input corresponding point coordinates, all edges containing the point (equivalent to connecting edges) are obtained, the errors of all edges (equivalent to connecting edges) are calculated, and the edge with the smallest error is selected as the object of collapse (equivalent to the edge to be collapsed). The calculation of the edge error is determined by the error between the two vertices. The error of vertex v (equivalent to the vertex error) can be expressed by formula (4).

[0344] In the embodiment of the present application, it is assumed that Represents the coordinates of the minimum error point after collapsing the edge E. At this time, the error can be expressed by formula (8). Taking the partial derivative of Error (equivalent to the error of the edge to be collapsed) can obtain the minimum error metric time point After calculating and comparing the error metrics for all edges containing point v, the edge with the smallest error is selected for collapse. The coordinates of the collapsed point are the coordinates of the target point. Repeat the nearest point selection and target point calculation process until each point in the reference frame base mesh has a corresponding target point. During this process, the original input mesh is continuously collapsed and simplified, with each collapse based on the previous collapse.

[0345] In the embodiment of the present application, the number of points and connectivity of the obtained matching mesh are the same as those of the base mesh of the reference frame, and the calculation of the motion vector can be expressed by formula (1). The calculation of the vertex displacement can be expressed by formula (2).

[0346] In the embodiment of the present application, since the matching grid generated by inter-frame coding has a certain quality degradation as the basic grid of the current frame, in order to ensure the quality of the final reconstructed grid, the peak signal-to-noise ratio (equivalent to the error metric value) of the basic grid generated by intra-frame coding and the matching grid generated by the inter-frame coding mode is calculated respectively based on the input original grid. The calculation method can be expressed by formula (9).

[0347] In an embodiment of the present application, when the difference in peak signal-to-noise ratio (equivalent to the error metric difference) between the basic grid generated by intra-frame coding and the matching grid generated by inter-frame coding mode is less than a threshold (equivalent to the first preset threshold) (the default is 1), the matching grid generated by the inter-frame coding mode (equivalent to the third basic grid) is selected as the basic grid of the current frame, and inter-frame coding is performed on the current frame; otherwise, the basic grid generated by intra-frame coding (equivalent to the first basic grid) is selected as the basic grid of the current frame, and intra-frame coding is performed.

[0348] In an embodiment of the present application, for the intra-frame coding mode, a static mesh encoder is required to compress the base mesh and then compress the vertex displacement; for the inter-frame coding mode, only the motion vector and vertex displacement need to be compressed, which greatly reduces the bit rate of the encoded base mesh.

[0349] In an embodiment of the present application, when the motion vector (equivalent to the motion vector) is large, the selection of matching points may be inaccurate. The use of the neighbor information of the point can solve this problem well. The relative position information of the matching point of the neighbor point and the current point can help determine the accuracy of the current matching point position and correct it, thereby improving the quality of the generated matching grid.

[0350] In the embodiment of the present application, the scope of application of the inter-frame coding mode can be greatly expanded, avoiding the situation where sequences with large displacements can hardly use inter-frame coding. At the same time, by calculating the error and selecting the minimum error edge for collapse, the point coordinates of the matching grid are obtained, which improves the generation quality of the matching grid, and then improves the quality of the final reconstructed grid, and obtains a greater BD-Rate gain from a lower bit rate at the same peak signal-to-noise ratio.

[0351] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, see FIG16 , which shows a schematic diagram of the composition structure of an encoder provided in an embodiment of the present application. As shown in FIG16 , the encoder 20 includes a first determination part 21, a matching part 22, and an encoding part 23, wherein:

[0352] The first determining part 21 is configured to determine a first basic grid of the current frame according to the original grid of the current frame;

[0353] The matching part 22 is configured to select matching points from the original mesh of the current frame based on the second basic mesh of the reference frame to obtain a third basic mesh; wherein the third basic mesh matches vertices in the second basic mesh;

[0354] The first determining part 21 is further configured to determine a target coding mode of the current frame according to an error metric difference between the first basic grid and the third basic grid;

[0355] The encoding part 23 is configured to perform encoding processing on the current frame according to the target encoding mode.

[0356] In some embodiments of the present application, the third basic mesh has the same number of vertices and connectivity as the second basic mesh.

[0357] In some embodiments of the present application, the matching part 22 is further configured to determine, from at least one first vertex in the original mesh of the current frame, a matching vertex that matches at least one second vertex in the second basic mesh; wherein the at least one first vertex includes the matching vertex; based on the coordinate information of two connecting vertices of at least one connecting edge associated with the at least one matching vertex, determine the edge error of at least one connecting edge associated with the at least one matching vertex; based on the edge error of at least one connecting edge associated with the at least one matching vertex, optimize the matching vertex corresponding to the at least one second vertex to obtain the optimized matching vertex corresponding to the at least one matching vertex; and determine the third basic mesh based on the optimized matching vertex corresponding to the at least one matching vertex.

[0358] In some embodiments of the present application, the first determining part 21 is further configured to determine at least one connecting edge respectively associated with the at least one matching vertex in the original mesh of the current frame based on the coordinate information of the at least one matching vertex.

[0359] In some embodiments of the present application, the matching part 22 is further configured to construct an octree structure of the original mesh of the current frame; for the at least one second vertex in the second basic mesh, by traversing at least one first vertex in the octree structure, the first vertex in the octree structure that is closest to the at least one second vertex is used as a matching vertex for matching the at least one second vertex.

[0360] In some embodiments of the present application, the matching part 22 is further configured to determine, for a current matching vertex corresponding to a current second vertex in the second basic mesh, a current edge to be collapsed of the current matching vertex based on an edge error of at least one connecting edge associated with the current matching vertex; wherein the edge to be collapsed is a connecting edge associated with the current matching vertex in the current original mesh; the current original mesh is obtained by optimizing the original mesh of the current frame; optimizing the edge error of the current edge to be collapsed based on a coefficient vector corresponding to a plane where two points to be collapsed associated with the current edge to be collapsed are located, obtaining a current optimized matching vertex corresponding to the current matching vertex, and obtaining a next original mesh after collapse processing; continuing to determine a next edge to be collapsed of the next matching vertex based on the edge error of at least one connecting edge associated with the next matching vertex, obtaining a next optimized matching vertex corresponding to the next matching vertex, until all matching vertices corresponding to the second vertex in the second basic mesh are traversed, thereby obtaining optimized matching vertices corresponding to the at least one matching vertex; wherein the next second vertex is connected to the current second vertex, and the vertex identifier order of the current second vertex is before that of the next second vertex.

[0361] In some embodiments of the present application, the matching part 22 is further configured to determine the connecting edge corresponding to the minimum error among the edge errors of at least one connecting edge associated with the current matching vertex as the current edge to be collapsed corresponding to the current matching vertex.

[0362] In some embodiments of the present application, the matching part 22 is further configured to determine, for any one of the at least one connecting edges respectively associated with the at least one matching vertex, the vertex error of the two connecting vertices associated with the any one connecting edge based on the coordinate information of the two connecting vertices associated with the any one connecting edge; and determine the edge error of the any one connecting edge based on the vertex error of the two connecting vertices associated with the any one connecting edge.

[0363] In some embodiments of the present application, the matching part 22 is further configured to determine the vertex error of any one of the two connecting vertices associated with any one of the connecting edges based on a plane set corresponding to any one of the connecting vertices and the coordinate information of any one of the connecting vertices; wherein the plane set includes at least one intersecting plane associated with any one of the connecting vertices.

[0364] In some embodiments of the present application, the matching part 22 is further configured to determine a coefficient vector corresponding to at least one plane based on a plane equation corresponding to at least one plane in the plane set corresponding to any one of the connected vertices; wherein the coefficient vector is used to represent the coefficient of the plane equation; and determine the vertex error of any one of the connected vertices based on the coefficient vector of the at least one plane and the coordinate information of the any one of the connected vertices.

[0365] In some embodiments of the present application, the matching part 22 is further configured to perform a weighted sum of vertex errors of two connecting vertices associated with any connecting edge to obtain an edge error of the any connecting edge.

[0366] In some embodiments of the present application, the matching part 22 is further configured to determine the error matrix of the two points to be collapsed associated with the current edge to be collapsed based on the coefficient vector corresponding to the plane where the two points to be collapsed associated with the current edge to be collapsed are located; determine the target error matrix of the current edge to be collapsed based on the error matrix of the two points to be collapsed associated with the current edge to be collapsed; optimize the edge error of the current edge to be collapsed according to the target error matrix through the error measurement model to obtain the target edge error, and use the collapsed vertex corresponding to the target edge error as the currently optimized matching vertex corresponding to the current matching vertex; wherein, the error measurement model is used to characterize the relationship between the target error matrix, the edge error of the edge to be collapsed and the collapsed vertex; the collapsed vertex is the vertex obtained after the edge to be collapsed is collapsed, and the next original mesh after the collapse is obtained.

[0367] In some embodiments of the present application, the first determination part 21 is further configured to determine the error metric difference value based on the error metric values ​​of the first basic grid and the third basic grid; wherein the error metric value includes: peak signal-to-noise ratio; if the error metric difference value is less than or equal to a first preset threshold value, the target coding mode is determined to be an inter-frame coding mode; if the error metric difference value is greater than the first preset threshold value, the target coding mode is determined to be an intra-frame coding mode.

[0368] In some embodiments of the present application, the first determination part 21 is further configured to determine, for any one of the first basic grid and the third basic grid, an error metric value of the any one basic grid based on the geometric distance peak of the original grid of the current frame and the target distortion result corresponding to the any one basic grid; wherein the target distortion result is used to characterize the distortion of the any one basic grid with respect to the original grid of the current frame.

[0369] In some embodiments of the present application, the first determination part 21 is further configured to determine the target distortion result based on the first distortion result and / or the second distortion result corresponding to any one of the basic grids; wherein the first distortion result is used to characterize the distortion result of any one of the basic grids relative to the original grid of the current frame; and the second distortion result is used to characterize the distortion result of the original grid of the current frame relative to any one of the basic grids.

[0370] In some embodiments of the present application, the first determining part 21 is further configured to use the maximum distortion result between the first distortion result and the second distortion result as the target distortion result.

[0371] In some embodiments of the present application, the first distortion result includes: a first distortion value; the first distortion value represents the point-to-point distortion error of any one of the basic meshes relative to the original mesh of the current frame; the first determination part 21 is also configured to upsample the any one of the basic meshes and the original mesh of the current frame to obtain an updated original mesh and an updated any one of the basic meshes; wherein the updated original mesh and the updated any one of the basic meshes have the same number of vertices; for at least one third vertex in any one of the updated basic meshes, determine the Euclidean distance between the fourth vertices corresponding to the at least one third vertex in the updated original mesh; determine the first distortion value based on the number of vertices in any one of the updated basic meshes and the Euclidean distance corresponding to the at least one third vertex; and determine the first distortion value as the first distortion result.

[0372] In some embodiments of the present application, the first distortion result also includes: a second distortion value; the second distortion value represents the distortion error of the point-to-surface of any basic mesh relative to the original mesh of the current frame; the first determination part 21 is also configured to determine, for the at least one third vertex in any updated basic mesh, the error vector between the at least one third vertex and the corresponding fourth vertices in the updated original mesh; project the error vectors corresponding to the at least one third vertex along the direction of the normal vector of the fourth vertex corresponding to the at least one third vertex to obtain a projection error vector; determine the second distortion value based on the number of vertices in any updated basic mesh and the projection error vectors corresponding to the at least one third vertex; determine the first distortion result based on the first distortion value and the second distortion value.

[0373] In some embodiments of the present application, the second distortion result includes: a third distortion value; the third distortion value represents the point-to-point distortion error of the original mesh of the current frame relative to any one of the basic meshes; the first determination part 21 is further configured to upsample the any one of the basic meshes and the original mesh of the current frame to obtain an updated original mesh and an updated any one of the basic meshes; wherein the updated original mesh and the updated any one of the basic meshes have the same number of vertices; for at least one fourth vertex in the updated original mesh, determine the Euclidean distance between the third vertices corresponding to the at least one fourth vertex in the updated any one of the basic meshes; determine the third distortion value based on the number of vertices in the updated original mesh and the Euclidean distance corresponding to the at least one fourth vertex; and determine the third distortion value as the second distortion result.

[0374] In some embodiments of the present application, the second distortion result also includes: a fourth distortion value; the fourth distortion value represents the distortion error of the original mesh of the current frame relative to the point-to-surface of any one of the basic meshes; the first determination part 21 is also configured to determine, for at least one fourth vertex in the updated original mesh, an error vector between the third vertices corresponding to the at least one fourth vertex in any one of the updated basic meshes; project the error vector corresponding to the at least one fourth vertex along the direction of the normal vector of the third vertex corresponding to the at least one fourth vertex to obtain a projection error vector; determine the fourth distortion value based on the number of vertices in the updated original mesh and the projection error vector corresponding to the at least one fourth vertex; and determine the second distortion result based on the third distortion value and the fourth distortion value.

[0375] In some embodiments of the present application, the encoding part 23 is further configured to, when the target encoding mode is an intra-frame encoding mode, encode the first basic grid of the current frame and write the obtained encoding bits into the bitstream; or, when the target encoding mode is an inter-frame encoding mode, encode the motion vector corresponding to the third basic grid of the current frame and write the obtained encoding bits into the bitstream.

[0376] In some embodiments of the present application, the encoding part 23 is further configured to, when the target encoding mode is the inter-frame encoding mode, determine, for at least one second vertex in the second basic mesh of the reference frame, a fifth vertex corresponding to the at least one second vertex in the third basic mesh; determine the motion vector corresponding to the third basic mesh based on the difference between the coordinate information of the fifth vertex corresponding to the at least one second vertex and the coordinate information of the at least one second vertex; encode the motion vector and write the obtained encoding bits into the bitstream.

[0377] In some embodiments of the present application, the first determination part 21 is further configured to determine the neighbor points of the at least one fifth vertex based on the connection information of the at least one fifth vertex when the motion vector is greater than or equal to a second preset threshold; adjust the at least one fifth vertex based on the relative position information between the neighbor points of the at least one fifth vertex and the at least one fifth vertex to obtain an adjusted vertex corresponding to the at least one fifth vertex; and obtain an adjusted third basic mesh based on the adjusted vertices corresponding to the at least one fifth vertex.

[0378] In some embodiments of the present application, the encoding part 23 is further configured to iteratively divide the first basic grid or the third basic grid according to a grid subdivision algorithm to determine the subdivided grid of the current frame; the grid subdivision algorithm includes: a linear interpolation algorithm; for at least one sixth vertex in the subdivided grid of the current frame, determining a seventh vertex corresponding to the at least one fifth vertex in the first basic grid; determining the shift information corresponding to the subdivided grid of the current frame based on the difference between the coordinate information of the at least one sixth vertex and the coordinate information of the seventh vertex corresponding to the at least one sixth vertex; encoding the shift information and writing the obtained encoding bits into the bitstream.

[0379] In some embodiments of the present application, the encoding part 23 is further configured to determine the value of syntax identification information; wherein the syntax identification information is used to indicate the target encoding mode adopted by the current frame; encode the value of the syntax identification information, and write the obtained encoded bits into the bitstream.

[0380] In some embodiments of the present application, the reference frame is an encoded frame before the current frame.

[0381] In some embodiments of the present application, the preset conditions include: the number of vertices between the first base mesh and the second base mesh is the same; and / or the connectivity of vertex coordinates and / or the connectivity of texture coordinates between the first base mesh and the second base mesh is the same.

[0382] In some embodiments of the present application, the first determining part 21 is further configured to perform downsampling processing on the original grid of the current frame to determine the first basic grid.

[0383] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0384] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or grid device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0385] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 20. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0386] Based on the composition of the encoder 20 and the computer-readable storage medium, refer to Figure 17, which shows a specific hardware structure diagram of an encoder provided by an embodiment of the present application. As shown in Figure 17, the encoder 20 may include: a first communication interface 2001, a first memory 2002 and a first processor 2003; each component is coupled together through a first bus system 2004. It can be understood that the first bus system 2004 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 2004 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 2004 in Figure 17. Among them,

[0387] The first communication interface 2001 is used to receive and send signals when sending and receiving information with other external network elements;

[0388] A first memory 2002 is used to store a computer program that can be run on the first processor 2003;

[0389] The first processor 2003 is configured to, when running the computer program, execute:

[0390] Determine a first basic grid of the current frame according to the original grid of the current frame;

[0391] Based on the second basic mesh of the reference frame, matching points are selected from the original mesh of the current frame to obtain a third basic mesh; wherein the third basic mesh matches vertices in the second basic mesh;

[0392] determining a target coding mode for a current frame according to an error metric difference between the first base grid and the third base grid;

[0393] Encode the current frame according to the target encoding mode.

[0394] It is understood that the first memory 2002 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 2002 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0395] The first processor 2003 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 2003. The above-mentioned first processor 2003 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 2002 , and the first processor 2003 reads the information in the first memory 2002 and completes the steps of the above method in combination with its hardware.

[0396] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0397] Optionally, as another embodiment, the first processor 2003 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0398] This embodiment provides an encoder that, on the one hand, generates a third base grid based on the second base grid of the reference frame by selecting the best matching point on the original grid of the current frame. This expands the applicability of the inter-frame coding mode, significantly reduces the bit rate of the encoded base grid, and ensures the quality of the reconstructed grid. On the other hand, by selecting an appropriate target coding mode for the current frame based on the error metric difference, the appropriate target coding mode can be adaptively selected in different scenarios, thereby improving the coding efficiency of the grid while ensuring the quality of grid reconstruction.

[0399] Based on the same inventive concept as the above embodiment, referring to FIG18 , a schematic diagram of the structure of a decoder provided by an embodiment of the present application is shown. As shown in FIG18 , the decoder 30 may include a decoding part 31 and a second determining part 32, wherein:

[0400] The decoding part 31 is configured to parse the code stream to obtain syntax identification information; when the syntax identification information indicates that the coding mode adopted by the current frame is the inter-frame coding mode, parse to obtain the motion vector corresponding to the third basic grid of the current frame and the displacement information corresponding to the subdivided grid of the current frame; wherein the third basic grid is based on the second basic grid of the reference frame and is obtained by selecting matching points on the original grid of the current frame; the third basic grid matches the vertices in the second basic grid

[0401] The second determining part 32 is configured to determine a reconstruction grid of the current frame according to the motion vector and the shift information.

[0402] In some embodiments of the present application, the second determining part 32 is further configured to determine the third basic grid based on the motion vector and the second basic grid; and determine the reconstructed grid of the current frame based on the shift information and the third basic grid.

[0403] In some embodiments of the present application, the second determination part 32 is further configured to determine, in the motion vector, a motion offset corresponding to at least one second vertex in the second basic mesh of the reference frame; and determine the third basic mesh of the current frame based on the sum of the motion offset corresponding to the at least one second vertex and the coordinate information of the at least one second vertex.

[0404] In some embodiments of the present application, the second determination part 32 is further configured to iteratively divide the third basic grid according to a grid subdivision algorithm to determine the subdivided grid of the current frame; the grid subdivision algorithm includes: a linear interpolation algorithm; for at least one sixth vertex in the subdivided grid of the current frame, determining the shift offset corresponding to the at least one sixth vertex in the shift information; determining the seventh vertex corresponding to the at least one sixth vertex in the reconstructed grid according to the difference between the coordinate information of the at least one sixth vertex and the shift offset corresponding to the at least one sixth vertex; and determining the reconstructed grid of the current frame according to the seventh vertex corresponding to the at least one sixth vertex.

[0405] In some embodiments of the present application, the second determination part 32 is further configured to parse the code stream when the syntax identification information indicates that the coding mode adopted by the current frame is the intra-frame coding mode, obtain the first basic grid of the current frame and the shift information corresponding to the subdivided grid of the current frame; and determine the reconstructed grid of the current frame based on the first basic grid and the shift information.

[0406] In some embodiments of the present application, the reference frame is a decoded frame before the current frame.

[0407] In some embodiments of the present application, the number of vertices between the first base mesh and the second base mesh is the same; and / or the connectivity of vertex coordinates and / or the connectivity of texture coordinates between the first base mesh and the second base mesh are the same.

[0408] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0409] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium, which is applied to the decoder 30 and stores a computer program. When the computer program is executed by the second processor, it implements any of the methods in the aforementioned embodiments.

[0410] Based on the composition of the decoder 30 and the computer-readable storage medium, refer to Figure 19, which shows a specific hardware structure diagram of a decoder provided by an embodiment of the present application. As shown in Figure 19, the decoder 30 may include: a second communication interface 3001, a second memory 3002 and a second processor 3003; each component is coupled together through a second bus system 3004. It can be understood that the second bus system 3004 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 3004 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 3004 in Figure 19. Among them,

[0411] The second communication interface 3001 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0412] The second memory 3002 is used to store computer programs that can be run on the second processor 3003;

[0413] The second processor 3003 is configured to, when running the computer program, execute:

[0414] Parse the code stream to obtain syntax identification information;

[0415] When the syntax identification information indicates that the coding mode used by the current frame is the inter-frame coding mode, the motion vector corresponding to the third basic grid of the current frame and the shift information corresponding to the subdivided grid of the current frame are parsed; wherein the third basic grid is obtained by selecting matching points from the original grid of the current frame based on the second basic grid of the reference frame; and the third basic grid matches vertices in the second basic grid.

[0416] The reconstruction grid of the current frame is determined based on the motion vector and shift information.

[0417] Optionally, as another embodiment, the second processor 3003 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0418] It can be understood that the hardware functions of the second memory 3002 are similar to those of the first memory 2002, and the hardware functions of the second processor 3003 are similar to those of the first processor 2003; they will not be described in detail here.

[0419] This embodiment provides a decoder, in which, since the third basic grid is determined by selecting the best matching point on the original grid of the current frame when the first basic grid and the second basic grid of the reference frame meet preset conditions, the application scope of the inter-frame coding mode can be expanded, the bit rate of the decoded basic grid is greatly reduced, and the quality of the reconstructed grid is guaranteed. While ensuring the quality of the grid reconstruction, the decoding efficiency of the grid can be improved.

[0420] In yet another embodiment of the present application, a code stream is provided, wherein the code stream is generated by bit encoding according to information to be encoded; wherein the information to be encoded includes at least one of the following:

[0421] The value of the syntax identification information, the motion vector corresponding to the first basic grid and the third basic grid of the current frame, and the shift information corresponding to the subdivided grid of the current frame; wherein the syntax identification information is used to indicate the target coding mode adopted by the current frame.

[0422] In yet another embodiment of the present application, referring to FIG20 , a schematic diagram of the structure of a coding and decoding system provided by an embodiment of the present application is shown. As shown in FIG20 , the coding and decoding system 40 may include an encoder 401 and a decoder 402 .

[0423] In the embodiment of the present application, the encoder 401 may be the encoder 20 described in any one of the aforementioned embodiments, and the decoder 402 may be the decoder 30 described in any one of the aforementioned embodiments.

[0424] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0425] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0426] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0427] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0428] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0429] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0430] In an embodiment of the present application, at the encoding end, the first basic grid of the current frame is determined based on the original grid of the current frame; based on the second basic grid of the reference frame, matching points are selected for the original grid of the current frame to obtain a third basic grid; wherein the third basic grid matches the vertices in the second basic grid; the target coding mode of the current frame is determined based on the error metric difference between the first basic grid and the third basic grid; and the current frame is encoded according to the target coding mode. At the decoding end, the code stream is parsed to obtain syntax identification information; when the syntax identification information indicates that the coding mode adopted by the current frame is the inter-frame coding mode, the motion vector corresponding to the third basic grid of the current frame and the displacement information corresponding to the subdivided grid of the current frame are obtained by parsing; wherein the third basic grid is obtained by selecting matching points for the original grid of the current frame based on the second basic grid of the reference frame; the third basic grid matches the vertices in the second basic grid; and the reconstructed grid of the current frame is determined based on the motion vector and the displacement information. This approach, on the one hand, generates a third base grid based on the second base grid of the reference frame by selecting the best matching point on the original grid of the current frame. This expands the applicability of inter-frame coding modes, significantly reduces the bit rate of the encoded base grid, and ensures the quality of the reconstructed grid. On the other hand, selecting the appropriate target coding mode for the current frame based on the error metric difference allows for adaptive selection of the appropriate target coding mode in different scenarios, ensuring grid reconstruction quality while improving grid encoding and decoding efficiency.

Claims

1. A coding method, applied to an encoder, the method comprising: Determine a first basic grid of the current frame according to an original grid of the current frame; Based on the second basic mesh of the reference frame, matching points are selected for the original mesh of the current frame to obtain a third basic mesh; wherein the third basic mesh matches vertices in the second basic mesh; Determining a target coding mode of the current frame according to an error metric difference between the first basic grid and the third basic grid; The current frame is encoded according to the target encoding mode.

2. The method according to claim 1, wherein: The third base mesh has the same number of vertices and connectivity as the second base mesh.

3. The method according to claim 1 or 2, wherein: The second basic grid based on the reference frame selects matching points from the original grid of the current frame to obtain a third basic grid, including: Determine, among at least one first vertex in the original mesh of the current frame, a matching vertex that matches at least one second vertex in the second basic mesh, respectively; wherein the at least one first vertex includes the matching vertex; Determine, based on coordinate information of two connected vertices of at least one connected edge associated with at least one matching vertex, an edge error of at least one connected edge associated with the at least one matching vertex; Based on the edge error of at least one connecting edge respectively associated with the at least one matching vertex, optimizing the matching vertex corresponding to the second vertex to obtain an optimized matching vertex respectively corresponding to the at least one matching vertex; The third basic mesh is determined according to the optimized matching vertex corresponding to the at least one matching vertex.

4. The method according to claim 3, wherein: Before determining the edge error of at least one connecting edge respectively associated with at least one matching vertex based on the coordinate information of two connecting vertices of at least one connecting edge associated with at least one matching vertex, the method further includes: At least one connecting edge respectively associated with the at least one matching vertex is determined in the original mesh of the current frame according to the coordinate information of the at least one matching vertex.

5. The method according to claim 3 or 4, wherein: The step of determining, from at least one first vertex in the original mesh of the current frame, a matching vertex that matches at least one second vertex in the second basic mesh, respectively, comprises: Constructing an octree structure of the original grid of the current frame; For the second vertex in the second basic mesh, by traversing at least one first vertex in the octree structure, a first vertex in the octree structure that is closest to the second vertex is used as a matching vertex for matching the second vertex.

6. The method according to any one of claims 3 to 5, wherein: The step of optimizing the matching vertex corresponding to the second vertex based on the edge error of at least one connecting edge respectively associated with the at least one matching vertex to obtain the optimized matching vertex respectively corresponding to the at least one matching vertex includes: For a current matching vertex corresponding to the current second vertex in the second basic mesh, determine a current edge to be collapsed of the current matching vertex according to an edge error of at least one connecting edge associated with the current matching vertex; wherein the edge to be collapsed is a connecting edge associated with the current matching vertex in the current original mesh; and the current original mesh is obtained by optimizing the original mesh of the current frame; According to the coefficient vector corresponding to the plane where the two points to be collapsed associated with the current edge to be collapsed are located, the edge error of the current edge to be collapsed is optimized to obtain the current optimized matching vertex corresponding to the current matching vertex, and obtain the next original mesh after the collapse process; Continue to determine the next edge to be collapsed of the next matching vertex according to the edge error of at least one connecting edge associated with the next matching vertex, and obtain the next optimized matching vertex corresponding to the next matching vertex, until the matching vertices corresponding to the second vertex in the second basic mesh are traversed, thereby obtaining the optimized matching vertices corresponding to the at least one matching vertex; wherein the next second vertex is connected to the current second vertex, and the vertex identification order of the current second vertex is before the next second vertex.

7. The method according to claim 6, wherein: The step of determining the current edge to be collapsed of the current matching vertex according to the edge error of at least one connecting edge associated with the current matching vertex comprises: A connecting edge corresponding to a minimum error among edge errors of at least one connecting edge associated with the current matching vertex is determined as a current edge to be collapsed corresponding to the current matching vertex.

8. The method according to any one of claims 3 to 7, wherein: The step of determining the edge error of at least one connecting edge respectively associated with at least one matching vertex based on the coordinate information of two connecting vertices of at least one connecting edge associated with at least one matching vertex comprises: For any one of the at least one connecting edge respectively associated with the at least one matching vertex, based on the coordinate information of the two connecting vertices associated with the any one connecting edge, determine the vertex error of the two connecting vertices associated with the any one connecting edge; The edge error of any connecting edge is determined according to the vertex errors of two connecting vertices associated with any connecting edge.

9. The method according to claim 8, wherein: The step of determining the vertex error of the two connected vertices associated with any one of the connected edges based on the coordinate information of the two connected vertices associated with any one of the connected edges comprises: For any one of the two connection vertices associated with any one of the connection edges, the vertex error of the any one connection vertex is determined based on a plane set corresponding to the any one connection vertex and the coordinate information of the any one connection vertex; wherein the plane set includes at least one intersecting plane associated with the any one connection vertex.

10. The method according to claim 9, wherein: The step of determining the vertex error of any of the connection vertices according to the plane set corresponding to any of the connection vertices and the coordinate information of any of the connection vertices comprises: Determine the coefficient vectors corresponding to at least one plane in the plane set corresponding to any one of the connected vertices according to the plane equations corresponding to the at least one plane; wherein the coefficient vectors are used to represent the coefficients of the plane equations; A vertex error of any one of the connected vertices is determined according to the coefficient vector of the at least one plane and the coordinate information of any one of the connected vertices.

11. The method according to any one of claims 8 to 10, wherein: The step of determining the edge error of any connecting edge according to the vertex errors of two connecting vertices associated with any connecting edge comprises: The vertex errors of the two connection vertices associated with any connection edge are weighted summed to obtain the edge error of any connection edge.

12. The method according to claim 6 or 7, wherein: The step of optimizing the edge error of the current edge to be collapsed according to the coefficient vector corresponding to the plane where the two points to be collapsed associated with the current edge to be collapsed are located, obtaining the current optimized matching vertex corresponding to the current matching vertex, and obtaining the next original mesh after the collapse process, comprises: Determine the error matrix of the two to-be-collapsed points associated with the current to-be-collapsed edge according to the coefficient vector corresponding to the plane where the two to-be-collapsed points associated with the current to-be-collapsed edge are located; Determine a target error matrix of the current edge to be collapsed according to the error matrices of two points to be collapsed associated with the current edge to be collapsed; Through the error measurement model, the edge error of the current edge to be collapsed is optimized according to the target error matrix to obtain the target edge error, and the collapsed vertex corresponding to the target edge error is used as the current optimized matching vertex corresponding to the current matching vertex; wherein, The error metric model is used to characterize the relationship between the target error matrix, the edge error of the edge to be collapsed, and the collapsed vertex; the collapsed vertex is the vertex obtained after collapsing the edge to be collapsed, and the next original mesh after the collapse is obtained.

13. The method according to any one of claims 1 to 12, wherein: The determining, according to the error metric difference between the first basic grid and the third basic grid, the target encoding mode of the current frame comprises: Determining the error metric difference value according to the error metric values ​​of the first basic grid and the third basic grid; wherein the error metric value includes: a peak signal-to-noise ratio; If the error metric difference is less than or equal to a first preset threshold, determining that the target coding mode is an inter-frame coding mode; If the error metric difference is greater than a first preset threshold, the target coding mode is determined to be an intra-frame coding mode.

14. The method according to claim 13, wherein: Before determining the error metric difference value according to the error metric values ​​of the first basic grid and the third basic grid, the method further includes: For any one of the first basic grid and the third basic grid, determine the error metric value of any one of the basic grids according to the geometric distance peak of the original grid of the current frame and the target distortion result corresponding to any one of the basic grids; wherein the target distortion result is used to characterize the distortion of any one of the basic grids with respect to the original grid of the current frame.

15. The method according to claim 14, wherein: The method further comprises: The target distortion result is determined according to the first distortion result and / or the second distortion result corresponding to any one of the basic grids; wherein the first distortion result is used to characterize the distortion result of any one of the basic grids relative to the original grid of the current frame; and the second distortion result is used to characterize the distortion result of the original grid of the current frame relative to any one of the basic grids.

16. The method according to claim 15, wherein: The determining the target distortion result according to the first distortion result and / or the second distortion result corresponding to any one of the basic grids includes: The maximum distortion result between the first distortion result and the second distortion result is used as the target distortion result.

17. The method according to claim 15 or 16, wherein: The first distortion result includes: a first distortion value; the first distortion value represents a point-to-point distortion error of any one of the basic grids relative to the original grid of the current frame; The method further comprises: Upsampling the any one of the base meshes and the original mesh of the current frame to obtain an updated original mesh and an updated any one of the base meshes; wherein the updated original mesh and the updated any one of the base meshes have the same number of vertices; For at least one third vertex in any updated basic mesh, determining the Euclidean distance between the at least one third vertex and the fourth vertices respectively corresponding to the at least one third vertex in the updated original mesh; Determine the first distortion value according to the number of vertices in any one of the updated basic meshes and the Euclidean distances respectively corresponding to the at least one third vertex; The first distortion value is determined as the first distortion result.

18. The method according to claim 17, wherein: The first distortion result further includes: a second distortion value; the second distortion value represents a point-to-surface distortion error of any one of the basic grids relative to the original grid of the current frame; After determining the first distortion value according to the number of vertices in any updated basic mesh and the Euclidean distances respectively corresponding to the at least one third vertex, the method further includes: For the at least one third vertex in any one of the updated basic meshes, determining an error vector between the at least one third vertex and corresponding fourth vertices in the updated original mesh; Projecting the error vectors corresponding to the at least one third vertex along the direction of the normal vector of the fourth vertex corresponding to the at least one third vertex to obtain a projection error vector; Determine the second distortion value according to the number of vertices in any one of the updated basic meshes and the projection error vectors respectively corresponding to the at least one third vertex; The first distortion result is determined according to the first distortion value and the second distortion value.

19. The method according to claim 15 or 16, wherein: The second distortion result includes: a third distortion value; the third distortion value represents a point-to-point distortion error of the original grid of the current frame relative to any one of the basic grids; The method further comprises: Upsampling the any one of the base meshes and the original mesh of the current frame to obtain an updated original mesh and an updated any one of the base meshes; wherein the updated original mesh and the updated any one of the base meshes have the same number of vertices; For at least one fourth vertex in the updated original mesh, determining a Euclidean distance between the at least one fourth vertex and third vertices respectively corresponding to the at least one fourth vertex in any one of the updated basic meshes; Determining the third distortion value according to the number of vertices in the updated original mesh and the Euclidean distances respectively corresponding to the at least one fourth vertex; The third distortion value is determined as the second distortion result.

20. The method according to claim 19, wherein the second distortion result further comprises: fourth distortion value; The fourth distortion value represents a distortion error of the original grid of the current frame relative to a point-to-surface of any one of the basic grids; After determining the third distortion value according to the number of vertices in the updated original mesh and the Euclidean distances respectively corresponding to the at least one fourth vertex, the method further includes: For at least one fourth vertex in the updated original mesh, determining an error vector between the at least one fourth vertex and the third vertices respectively corresponding to the at least one fourth vertex in any one of the updated basic meshes; Projecting the error vector corresponding to the at least one fourth vertex along the direction of the normal vector of the third vertex respectively corresponding to the at least one fourth vertex to obtain a projection error vector; Determining the fourth distortion value according to the number of vertices in the updated original mesh and the projection error vectors respectively corresponding to the at least one fourth vertex; The second distortion result is determined according to the third distortion value and the fourth distortion value.

21. The method according to any one of claims 1 to 20, wherein: The encoding process of the current frame according to the target encoding mode includes: When the target coding mode is an intra-frame coding mode, encoding the first basic grid of the current frame, and writing the obtained coding bits into a bitstream; or When the target coding mode is the inter-frame coding mode, the motion vector corresponding to the third basic grid of the current frame is coded, and the obtained coded bits are written into the bitstream.

22. The method according to claim 21, wherein: The method further comprises: When the target coding mode is the inter-frame coding mode, for at least one second vertex in the second basic mesh of the reference frame, determining, in the third basic mesh, fifth vertices respectively corresponding to the at least one second vertex; Determine the motion vector corresponding to the third basic mesh according to a difference between coordinate information of a fifth vertex respectively corresponding to the at least one second vertex and the coordinate information of the at least one second vertex; The motion vector is coded and the obtained coded bits are written into a bit stream.

23. The method according to claim 22, wherein: The method further comprises: When the motion vector is greater than or equal to a second preset threshold, determining a neighbor point of the at least one fifth vertex according to connection information of the at least one fifth vertex; According to the relative position information between the neighbor point of the at least one fifth vertex and the at least one fifth vertex, the at least one fifth vertex is adjusted to obtain adjusted vertices respectively corresponding to the at least one fifth vertex; An adjusted third basic mesh is obtained according to the adjusted vertex corresponding to the at least one fifth vertex.

24. The method according to any one of claims 1 to 20, wherein: The method further comprises: Iteratively divide the first basic grid or the third basic grid according to a grid subdivision algorithm to determine a subdivided grid of the current frame; the grid subdivision algorithm includes: a linear difference algorithm; For at least one sixth vertex in the subdivided mesh of the current frame, determining a seventh vertex corresponding to the at least one sixth vertex in the first basic mesh; Determine, according to a difference between the coordinate information of the at least one sixth vertex and the coordinate information of a seventh vertex corresponding to the at least one sixth vertex, the displacement information corresponding to the subdivided grid of the current frame; The shift information is coded and the obtained coded bits are written into a bit stream.

25. The method according to any one of claims 1 to 20, wherein: The method further comprises: Determine a value of syntax identification information; wherein the syntax identification information is used to indicate the target coding mode adopted by the current frame; The value of the syntax identification information is encoded, and the obtained encoded bits are written into a bit stream.

26. The method according to any one of claims 1 to 20, wherein: The reference frame is an encoded frame before the current frame.

27. The method according to any one of claims 1 to 20, wherein: The second basic grid based on the reference frame selects matching points from the original grid of the current frame to obtain a third basic grid, including: When the first basic grid and the second basic grid of the reference frame meet a preset condition, matching points are selected for the original grid of the current frame based on the second basic grid of the reference frame to obtain the third basic grid; wherein, The preset conditions include: The first basic mesh and the second basic mesh have the same number of vertices; and / or, Connectivity of vertex coordinates and / or connectivity of texture coordinates between the first base mesh and the second base mesh are the same.

28. The method according to any one of claims 1 to 20, wherein: The step of determining the first basic grid of the current frame according to the original grid of the current frame comprises: Down-sampling is performed on the original grid of the current frame to determine the first basic grid.

29. A decoding method, applied to a decoder, the method comprising: Parse the code stream to obtain syntax identification information; When the syntax identification information indicates that the coding mode adopted by the current frame is the inter-frame coding mode, a motion vector corresponding to a third basic grid of the current frame and displacement information corresponding to a subdivided grid of the current frame are obtained by parsing; wherein the third basic grid is obtained by selecting matching points from an original grid of the current frame based on a second basic grid of a reference frame; and the third basic grid matches vertices in the second basic grid; A reconstruction grid of the current frame is determined according to the motion vector and the shift information.

30. The method of claim 29, wherein: The step of determining a reconstruction grid of the current frame according to the motion vector and the shift information includes: Determine the third basic grid according to the motion vector and the second basic grid; The reconstructed grid of the current frame is determined according to the shift information and the third basic grid.

31. The method according to claim 30, wherein: The step of determining the third basic grid according to the motion vector and the second basic grid comprises: For at least one second vertex in the second basic mesh of the reference frame, determining in the motion vector a motion offset respectively corresponding to the at least one second vertex; The third basic mesh of the current frame is determined according to the sum of the motion offsets respectively corresponding to the at least one second vertex and the coordinate information of the at least one second vertex.

32. The method according to claim 30 or 31, wherein: The step of determining the reconstructed grid of the current frame according to the shift information and the third basic grid comprises: Iteratively divide the third basic grid according to a grid subdivision algorithm to determine a subdivided grid of the current frame; the grid subdivision algorithm includes: a linear difference algorithm; For at least one sixth vertex in the subdivided mesh of the current frame, determining in the shift information a shift offset respectively corresponding to the at least one sixth vertex; Determine, according to the coordinate information of the at least one sixth vertex and the difference between the displacement offset corresponding to the at least one sixth vertex, the seventh vertex corresponding to the at least one sixth vertex; The reconstructed mesh of the current frame is determined according to a seventh vertex corresponding to the at least one sixth vertex.

33. The method of claim 29, wherein: The method further comprises: When the syntax identification information indicates that the coding mode adopted by the current frame is the intra-frame coding mode, parsing the bitstream to obtain the first basic grid of the current frame and the shift information corresponding to the subdivided grid of the current frame; A reconstructed grid of the current frame is determined according to the first basic grid and the shift information.

34. The method according to any one of claims 29 to 33, wherein: The reference frame is a decoded frame before the current frame.

35. The method according to any one of claims 29 to 33, wherein: The third base mesh has the same number of vertices and connectivity as the second base mesh.

36. A code stream, the code stream is generated by bit encoding according to information to be encoded; wherein, The information to be encoded includes at least one of the following: The value of the syntax identification information, the motion vectors corresponding to the first and third basic grids of the current frame, and the shift information corresponding to the subdivided grids of the current frame; wherein the syntax identification information is used to indicate the target encoding mode adopted by the current frame.

37. An encoder, comprising a first determining part, a matching part and an encoding part, wherein: The first determining part is configured to determine a first basic grid of the current frame according to an original grid of the current frame; The matching part is configured to select matching points for the original mesh of the current frame based on the second basic mesh of the reference frame to obtain a third basic mesh; wherein the third basic mesh matches vertices in the second basic mesh; The first determining part is further configured to determine a target encoding mode of the current frame according to an error metric difference between the first basic grid and the third basic grid; The encoding part is configured to perform encoding processing on the current frame according to the target encoding mode.

38. A decoder, comprising a decoding part and a second determining part, wherein: The decoding part is configured to parse the code stream to obtain syntax identification information; when the syntax identification information indicates that the coding mode adopted by the current frame is the inter-frame coding mode, parse to obtain the motion vector corresponding to the third basic grid of the current frame and the displacement information corresponding to the subdivided grid of the current frame; wherein the third basic grid is obtained by selecting matching points for the original grid of the current frame based on the second basic grid of the reference frame; and the third basic grid matches the vertices in the second basic grid; The second determining part is configured to determine a reconstruction grid of the current frame according to the motion vector and the shift information.

39. An encoder, comprising a first memory and a first processor, wherein: The first memory is configured to store a computer program that can be executed on the first processor; The first processor is configured to perform the method according to any one of claims 1 to 28 when running the computer program.

40. A decoder, comprising a second memory and a second processor, wherein: The second memory is configured to store a computer program that can be executed on the second processor; The second processor is configured to perform the method according to any one of claims 29 to 35 when running the computer program.

41. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 28 is implemented, or the method according to any one of claims 29 to 35 is implemented.

Images