Coding method, code stream, encoder, decoder and storage medium

Abstract

The embodiment of the application discloses a coding and decoding method, a decoder decodes a code stream, determines a reconstructed base grid of a current image, partition information and partition arrangement information; determines a texture coordinate according to the partition information, the partition arrangement information and the reconstructed base grid, so as to determine a reconstructed grid of the current image based on the texture coordinate.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The present application relates to the field of video coding and decoding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] Video-based Dynamic Mesh Coding (VDMC) is a standard for compressing 3D meshes. It primarily compresses 3D meshes by leveraging the existing Visual Volumetric Video-based Coding (V3C) standard. However, due to the encoding of connectivity information in 3D meshes, the specific encoding process differs slightly from V3C.

[0003] However, in the encoding and decoding process of three-dimensional meshes, there is a problem that the results of the preprocessing stage do not correspond to the reconstructed basic mesh, which reduces the encoding and decoding efficiency and affects the compression performance.

[0004] Summary of the Invention

[0005] The present application provides a coding and decoding method, a code stream, an encoder, a decoder and a storage medium, which can improve coding and decoding efficiency and enhance compression performance.

[0006] The technical solution of this application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Decode the code stream to determine the reconstruction base grid, partition information, and partition arrangement information of the current image;

[0009] Texture coordinates are determined according to the partition information, the partition arrangement information, and the reconstructed basic mesh, so as to determine a reconstructed mesh of the current image based on the texture coordinates.

[0010] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0011] Determine a base grid of the current image and a reconstructed base grid, and write the reconstructed base grid into a bitstream;

[0012] Texture coordinates are determined according to the partition information, the partition arrangement information, and the reconstructed basic grid, so as to determine a reconstructed grid of the current image based on the texture coordinates; and the partition information and the partition arrangement information are written into a bitstream.

[0013] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding based on the information to be encoded; wherein the information to be encoded includes at least one of the following: a reconstructed basic grid, displacement information, partition information, partition arrangement information, subdivision method, number of subdivision iterations, and a texture map.

[0014] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit,

[0015] The first determination unit is configured to determine a base grid of the current image and a reconstructed base grid, and write the reconstructed base grid into a bitstream; determine texture coordinates based on the partition information, the partition arrangement information, and the reconstructed base grid, so as to determine the reconstructed grid of the current image based on the texture coordinates; and write the partition information and the partition arrangement information into the bitstream.

[0016] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor, wherein:

[0017] a first memory for storing a computer program capable of running on the first processor;

[0018] The first processor is configured to execute the method according to the second aspect when running a computer program.

[0019] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a second determining unit,

[0020] The second determination unit is configured to decode the code stream, determine the reconstructed basic grid, partition information and partition arrangement information of the current image; determine the texture coordinates based on the partition information, the partition arrangement information and the reconstructed basic grid, so as to determine the reconstructed grid of the current image based on the texture coordinates.

[0021] In a seventh aspect, an embodiment of the present application provides a decoder, comprising a second memory and a second processor, wherein:

[0022] a second memory for storing a computer program capable of running on the second processor;

[0023] The second processor is configured to execute the method according to the first aspect when running a computer program.

[0024] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed, implements the method described in the first aspect or the method described in the second aspect.

[0025] The embodiment of the present application provides a coding and decoding method, a code stream, an encoder, a decoder and a storage medium. The codec can determine the texture coordinates based on the partition information, the partition arrangement information and the reconstructed basic grid, so that the reconstructed grid of the current image can be obtained based on the texture coordinates. Specifically, the reconstructed subdivision grid of the current image can be determined based on the basic grid (the first basic grid with texture coordinates); then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid. That is to say, in the embodiment of the present application, at the encoding end, the reconstructed basic grid written into the code stream does not have texture coordinates, that is, the reconstructed basic grid obtained by the decoding end does not have texture coordinates, and the decoding end can complete the parameterization of the reconstructed basic grid at the decoding end based on the partition information and partition arrangement information corresponding to the encoding end transmitted in the code stream, that is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation result of the preprocessing stage corresponds to the result of the reconstruction stage, thereby improving the coding and decoding efficiency and improving the compression performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] FIG1A is a schematic diagram of a three-dimensional grid image 1;

[0027] FIG1B is a partially enlarged schematic diagram of a three-dimensional grid image;

[0028] Figure 2 is a schematic diagram of the connection method of the three-dimensional grid;

[0029] FIG3 is a schematic diagram of a three-dimensional grid-based coding framework;

[0030] FIG4 is a schematic diagram of a three-dimensional grid-based decoding framework;

[0031] FIG5 is a schematic diagram of a mesh architecture of a codec proposed in an embodiment of the present application;

[0032] FIG6 is a schematic diagram of an implementation flow of a decoding method proposed in an embodiment of the present application;

[0033] FIG7 is a schematic diagram of the subdivision process proposed in an embodiment of the present application;

[0034] FIG8 is a first schematic diagram of a decoding process implementation proposed in an embodiment of the present application;

[0035] FIG9 is a second schematic diagram of the decoding process implementation proposed in an embodiment of the present application;

[0036] FIG10 is a schematic diagram of an implementation flow of an encoding method proposed in an embodiment of the present application;

[0037] FIG11 is a schematic diagram of a mesh simplification operation proposed in an embodiment of the present application;

[0038] FIG12 is a first schematic diagram of an implementation of the encoding process proposed in an embodiment of the present application;

[0039] FIG13 is a second schematic diagram of the implementation of the encoding process proposed in an embodiment of the present application;

[0040] FIG14 is a schematic diagram of a subdivision deformation process proposed in an embodiment of the present application;

[0041] FIG15 is a schematic diagram of texture map conversion proposed in an embodiment of the present application;

[0042] FIG16 is a schematic diagram of the first structure of the encoder proposed in an embodiment of the present application;

[0043] FIG17 is a second schematic diagram of the structure of the encoder proposed in an embodiment of the present application;

[0044] FIG18 is a first schematic diagram of the structure of a decoder according to an embodiment of the present application;

[0045] FIG19 is a second schematic diagram of the composition structure of the decoder proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0048] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0049] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0050] It should be noted that it is possible to decode and synthesize different data format bitstreams within the same video scene. These can include at least image format, point cloud format, and mesh format. In this way, real-time immersive video interaction services can be provided for multiple data formats (e.g., mesh, point cloud, image, etc.) from different sources.

[0051] In embodiments of the present application, the data format-based approach allows for independent processing at the bitstream level of the data format. This means that, similar to tiles or slices in video encoding, different data formats in this scenario can be encoded independently, enabling independent encoding and decoding based on the data format.

[0052] Generally speaking, three-dimensional animation content adopts a keyframe-based representation method, that is, each frame is a static mesh. Static meshes at different times have the same topological structure and different geometric structures. However, the amount of data of the three-dimensional dynamic mesh represented based on keyframes is particularly large, so how to effectively store, transmit and draw it has become a problem faced by the development of three-dimensional dynamic meshes. In addition, the spatial scalability of the mesh needs to be supported for different user terminals (computers, notebooks, portable devices, mobile phones); different mesh bandwidths (broadband, narrowband, wireless) need to support the quality scalability of the mesh. Therefore, three-dimensional dynamic mesh compression is a very critical issue. Among them, "one frame" can be understood as an image. For example, a keyframe can be understood as a key image in a three-dimensional animation.

[0053] A 3D mesh is the surface of a 3D object composed of countless polygons in space. Polygons are composed of vertices and edges. Figure 1A shows a 3D mesh image, and Figure 1B shows a partially enlarged schematic diagram of the 3D mesh image. Figures 1A and 1B show that the mesh surface is composed of closed polygons.

[0054] A two-dimensional image has information expressed at every pixel point and is distributed regularly, so there is no need to record its position information separately. However, the distribution of vertices in the mesh in three-dimensional space is random and irregular, and the way polygons are formed requires additional regulations. Therefore, it is necessary to record the position of each vertex in space and the connection information of each polygon to fully express a mesh image. As shown in Figure 2, the same number of vertices and vertex positions will form completely different surfaces due to different connection methods.

[0055] In addition to the above information, since 3D mesh images are usually encoded using existing 2D image / video encoding methods, the 3D mesh needs to be converted from 3D space to 2D images. The UV coordinates define this conversion process.

[0056] Similar to 2D images, each location in the acquisition process may have corresponding attribute information, usually RGB color values, which reflect the color of the object. For 3D meshes, in addition to color, each vertex's attribute information also commonly includes reflectance values, which reflect the surface material of the object. The attribute information of a 3D mesh is stored in a 2D image, and its mapping from 2D to 3D is specified by UV coordinates. 3D mesh data typically includes 3D geometric coordinate information (x, y, z), geometric connection relationships, UV coordinates, and attribute maps.

[0057] It should also be noted that Video-based Dynamic Mesh Coding (VDMC) is a standard developed by the Moving Pictures Experts Group (MPEG) for compressing three-dimensional meshes. Its main idea is to compress three-dimensional meshes by leveraging the existing Visual Volumetric Video-based Coding (V3C) standard. Since three-dimensional meshes contain connection information that needs to be encoded, its specific encoding process is slightly different from V3C. The syntax, semantics, and decoding operations of the V3C standard decoding end need to be expanded to support the decoding and reconstruction of three-dimensional meshes.

[0058] FIG3 is a schematic diagram of a three-dimensional grid encoding framework, and FIG4 is a schematic diagram of a three-dimensional grid decoding framework.

[0059] In an embodiment of the present application, the overall framework of the encoding end is shown in Figure 3. For the input mesh, the basic mesh and the corresponding subdivided deformed mesh are first obtained through the basic mesh generation module. Among them, the input mesh first obtains a simplified mesh through the simplification module, and then generates new texture coordinates for the simplified mesh through mesh parameterization, and then subdivides and deforms the parameterized mesh, that is, inserts new vertices on the mesh according to a specific subdivision method and calculates the distance from the subdivided mesh vertex to the nearest neighbor of the input mesh - called displacement, and obtains the corresponding subdivided deformed mesh. Subsequently, the parameterized mesh is adjusted according to the displacement information, that is, the vertex position of the mesh before subdividing the deformation, and the adjusted mesh is called the basic mesh, which is sent to the basic mesh encoding module and compressed using the existing mesh encoder. In the inter-frame mode, a motion vector can also be generated for each vertex of the basic mesh according to the reference frame, and the basic mesh module only needs to compress the motion vector.

[0060] After encoding, the base mesh is reconstructed, and displacements are calculated based on the reconstructed base mesh and the subdivided deformed mesh obtained during the base mesh generation phase. The displacement information is then transformed and quantized, and then encoded using a video encoder or entropy encoder. The reconstructed displacement information is then applied to the subdivided base mesh to produce a reconstructed subdivided deformed mesh. This mesh, along with the original input mesh and its corresponding texture map, is input into a texture map conversion module to produce a texture map corresponding to the reconstructed mesh. This texture map is also encoded using a video encoder.

[0061] In an embodiment of the present application, the overall framework of the decoding end is shown in Figure 4. For the received code stream, the decoding end first demultiplexes the various code streams to obtain the basic grid code stream, the displacement video code stream, and the texture map video code stream respectively. For the basic grid code stream, the basic grid is obtained by decoding using the grid decoder corresponding to the encoding end. The displacement video code stream and the texture map video code stream are decoded by the video decoder. For the displacement part, after the video is decoded, the displacement needs to be taken out from the image through the displacement decoding module, and the steps such as inverse quantization and inverse transformation are performed, and then it is applied to the subdivided basic grid to obtain the deformed grid reconstructed by the decoding end. After decoding, the texture map is the texture map corresponding to the reconstructed deformed grid. The subsequent application or rendering module processes the reconstructed deformed grid and the texture map obtained by decoding as input.

[0062] However, in the above encoding and decoding scheme, since mesh parameterization and subdivision deformation are both preprocessing, the vertex positions of the mesh input to the mesh parameterization module and the basic mesh output by the subdivision deformation module are offset after the basic mesh encoding and decoding. This will cause the results calculated in the preprocessing stage to be inconsistent with the reconstructed basic mesh, thereby reducing the encoding and decoding efficiency and affecting the compression performance.

[0063] An embodiment of the present application provides a coding and decoding method. At the encoding end, the reconstructed basic grid written into the code stream does not have texture coordinates, that is, the reconstructed basic grid obtained by the decoding end does not have texture coordinates. The decoding end can complete the parameterization of the reconstructed basic grid at the decoding end based on the partition information and partition arrangement information corresponding to the encoding end transmitted in the code stream. That is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation results of the preprocessing stage correspond to the results of the reconstruction stage, thereby improving the coding and decoding efficiency and improving the compression performance.

[0064] The present application also provides a grid architecture of a codec system including a decoding method and an encoding method. FIG5 is a schematic diagram of a grid architecture of a codec proposed in the present application. As shown in FIG5 , the grid architecture includes one or more electronic devices 13 to 1N and a communication grid 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication grid 01. During implementation, the electronic devices can be various types of devices with codec functions. For example, the electronic devices can include mobile phones, tablet computers, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensor devices, servers, etc., which are not specifically limited here.

[0065] Here, the decoder or encoder described in the embodiment of the present application can be the above-mentioned electronic device.

[0066] The following describes in detail the various embodiments of the present application with reference to the accompanying drawings.

[0067] In one embodiment of the present application, FIG6 is a schematic diagram of a decoding method implementation process proposed in the embodiment of the present application. As shown in FIG6, the method for the decoder to perform decoding processing may include the following steps:

[0068] Step 101: Decode the code stream to determine the reconstructed basic grid, partition information, and partition arrangement information of the current image.

[0069] In an embodiment of the present application, the basic grid for reconstruction of the current image may be determined by first decoding the code stream, and the partition information and partition arrangement information corresponding to the current image may also be determined.

[0070] It should be noted that the decoding method in the embodiment of the present application may refer to a 3D mesh reconstruction method. In addition, the decoding method may be applied to a decoder, specifically a decoder in a V-DMC, but is not limited thereto.

[0071] Furthermore, in an embodiment of the present application, the decoder may decode the basic grid code stream to determine the reconstructed basic grid of the current image.

[0072] It should be noted that in an embodiment of the present application, the code stream can also be decoded to determine the auxiliary information, and the basic grid decoder indicated by the auxiliary information can be used to decode the obtained basic grid code stream, so as to determine the reconstructed basic grid of the current image.

[0073] It will be appreciated that in the embodiments of the present application, the encoder can generate corresponding auxiliary information based on the encoding information, then write the auxiliary information into the bitstream for transmission to the decoder. Accordingly, after decoding and obtaining the auxiliary information, the decoder can select the base grid decoder corresponding to the encoder based on the base grid encoder identifier indicated by the auxiliary information to decode the base grid bitstream, thereby obtaining a reconstructed base grid.

[0074] For example, in some embodiments, the basic mesh decoder may include but is not limited to DRACO, MEB, etc. DRACO is an open source static mesh decoder (Google DRACO) launched by Google, and MEB is a static mesh decoder (MPEG EdgeBreaker) implemented by MPEG.

[0075] Furthermore, in an embodiment of the present application, the decoder may decode the texture coordinate code stream to determine corresponding partition information and partition arrangement information.

[0076] It should be noted that, in the embodiments of the present application, the partition information and the partition arrangement information can be used to perform grid parameterization processing.

[0077] It can be understood that in the embodiments of the present application, when performing mesh parameterization, it is necessary to first partition the mesh, then perform independent parameterization on each partition, obtain the parameterization results of each partition, and finally arrange them into a texture map of a preset size according to specific rules.

[0078] It should be noted that, in the embodiment of the present application, the partition information may be a partition index that identifies each face in the mesh during the mesh parameterization process.

[0079] It should be noted that, in the embodiment of the present application, the partition arrangement information is used to determine the position of each partition in the two-dimensional texture map. The partition arrangement information may include at least: parameterization scheme identification parameters, parameterization mode of the current partition, angle index of the current partition, bounding box position parameters of the current partition, bounding box size parameters of the current partition, scaling parameters of the current partition, etc.

[0080] Illustratively, in some embodiments, the parameterization method of the current partition may be used to determine the method for parameterizing the mesh of the current partition.

[0081] For example, in some embodiments, the angle index of the current partition may include an angle index for determining the rotation of the current partition when it is arranged on the image.

[0082] Exemplarily, in some embodiments, the position parameters of the bounding box of the current partition may include the position posX and posY of the upper left corner of the bounding box of the current partition in the image when the current partition is arranged on the image.

[0083] For example, in some embodiments, the bounding box size parameters of the current partition may include the width (sizeX) and height (sizeY) of the bounding box of the current partition when the partition is arranged on the image.

[0084] For example, in some embodiments, the scaling parameter of the current partition indicates whether scaling is performed when the current partition is arranged and the corresponding scaling coefficient.

[0085] Furthermore, in an embodiment of the present application, the syntax elements determined by decoding the texture coordinate code stream may be as follows:

[0086] Among them, base_mesh_texcoord_information() represents the texture coordinate information in the base mesh code stream;

[0087] base_mesh_parameterazation_method indicates the parameterization scheme used by the base mesh (parameterization scheme identification parameter), 0 represents the use of the isochart algorithm, 1 represents the use of the orthogonal projection algorithm, and the default value is 0;

[0088] base_mesh_uv_charts_number_minus1 means the number of partitions after parameterization minus 1;

[0089] uv_chart_parameterazation_type[i] represents the projection plane corresponding to the parameterization of the i-th partition. This value is only valid when the parameterization scheme is orthogonal projection parameterization;

[0090] uv_chart_orientation_id[i] represents the rotation angle index when the i-th partition is arranged;

[0091] uv_chart_posX[i] represents the position of the horizontal coordinate of the upper left corner of the partition bounding box in the two-dimensional texture map when the i-th partition is arranged;

[0092] uv_chart_posY[i] represents the position of the ordinate of the upper left corner of the partition bounding box in the two-dimensional texture map when the i-th partition is arranged;

[0093] uv_chart_sizeX[i] represents the width of the partition bounding box when the i-th partition is arranged;

[0094] uv_chart_sizeY[i] represents the height of the partition bounding box when the i-th partition is arranged;

[0095] uv_chart_scale_flag[i] indicates whether the i-th partition is scaled when arranged;

[0096] uv_chart_scale_factor[i] represents the scaling factor of the i-th partition.

[0097] Step 102: Determine texture coordinates according to the partition information, the partition arrangement information, and the reconstructed basic mesh, so as to determine a reconstructed mesh of the current image based on the texture coordinates.

[0098] In an embodiment of the present application, after determining the reconstructed basic grid, partition information and partition arrangement information of the current image, the texture coordinates can be further determined based on the partition information, partition arrangement information and the reconstructed basic grid, and then the reconstructed grid of the current image can be determined based on the texture coordinates.

[0099] Furthermore, in an embodiment of the present application, when determining texture coordinates based on partition information, partition arrangement information, and a reconstructed base mesh, the texture coordinates corresponding to the reconstructed base mesh may be determined based on the partition information and partition arrangement information.

[0100] Furthermore, in an embodiment of the present application, after determining the texture coordinates based on the partition information, the partition arrangement information, and the reconstructed base mesh, reconstruction can be further performed based on the texture coordinates to obtain a first base mesh, wherein the first base mesh can be a base mesh with texture coordinates.

[0101] It should be noted that in an embodiment of the present application, after determining the reconstructed basic grid, partition information and partition arrangement information, the partition information and partition arrangement information can be used to perform mesh parameterization on the reconstructed basic grid, thereby completing the reconstruction of the texture coordinates and obtaining the texture coordinates corresponding to the reconstructed basic grid.

[0102] Furthermore, in an embodiment of the present application, when determining the texture coordinates corresponding to the reconstructed basic mesh based on the partition information and the partition arrangement information, the reconstructed basic mesh can be partitioned according to the partition information to determine at least one partition corresponding to the reconstructed basic mesh; then the texture coordinates of at least one partition can be determined based on the partition arrangement information; finally, the texture coordinates of at least one partition can be arranged based on the partition arrangement information to determine the texture coordinates corresponding to the reconstructed basic mesh.

[0103] It should be noted that, in the examples of the present application, the reconstructed basic grid of the current image may be partitioned first according to the partition information to obtain at least one corresponding partition.

[0104] It is understood that in the embodiments of the present application, for any of the at least one partition obtained, the pre-arrangement texture coordinates of the partition, i.e., the texture coordinates of the partition, can be first determined according to the partition arrangement information. The texture coordinates of the partition can then be transformed according to the partition arrangement information to obtain the final reconstructed texture coordinates.

[0105] Furthermore, in an embodiment of the present application, the partition arrangement information may include a parameterization scheme identification parameter. When determining the texture coordinates of at least one partition based on the partition arrangement information, if the parameterization scheme identification parameter indicates that the current image uses a first parameterization scheme, all partitions can be parameterized according to the first parameterization method to determine the texture coordinates of at least one partition.

[0106] It is understood that in the embodiment of the present application, the parameterization scheme identification parameter is used to determine the parameterization scheme used for the current image, wherein the parameterization scheme includes but is not limited to the isochart algorithm and the orthogonal projection algorithm.

[0107] For example, in some embodiments, the parameterization scheme used for the current image can be determined based on the value of the parameterization scheme identification parameter. If the value of the parameterization scheme identification parameter is a first value, the parameterization scheme used for the current image can be determined to be the first parameterization scheme, such as the isochart algorithm.

[0108] It should be noted that, in the embodiment of the present application, for the isochart algorithm, the parameterization method of each partition can be the same. Therefore, each partition can be parameterized according to the same first parameterization method, and finally the texture coordinates of each partition can be determined.

[0109] Furthermore, in an embodiment of the present application, the partition arrangement information may include a parameterization scheme identification parameter and a second parameterization method corresponding to the current partition. When determining the texture coordinates of at least one partition based on the partition arrangement information, if the parameterization scheme identification parameter indicates that the current image uses the second parameterization scheme, the current partition can be parameterized separately according to the second parameterization method to determine the texture coordinates of the current partition.

[0110] For example, in some embodiments, the parameterization scheme used for the current image indicated by the parameterization scheme identification parameter may be determined based on the value of the parameterization scheme identification parameter. If the value of the parameterization scheme identification parameter is the second value, the parameterization scheme used for the current image may be determined to be the second parameterization scheme, such as the orthogonal projection algorithm.

[0111] It is understood that in the embodiments of the present application, the first value and the second value are different. For example, if the parameterized scheme identification parameter has a value of 0, it indicates that the isochart algorithm is used, and if the parameterized scheme identification parameter has a value of 1, it indicates that the orthogonal projection algorithm is used.

[0112] It should be noted that, in an embodiment of the present application, for the orthogonal projection algorithm, for the current partition in at least one partition, when the parameterization scheme identification parameter indicates that the current image uses the second parameterization scheme, the current partition can be parameterized according to the second parameterization method corresponding to the current partition in the partition arrangement information, so that the texture coordinates of the current partition can be determined.

[0113] Accordingly, in the embodiment of the present application, after each partition is parameterized in turn according to the second parameterization method corresponding to each partition, the texture coordinates of each partition can be determined.

[0114] Furthermore, in an embodiment of the present application, in addition to the parameterized scheme identification parameters, the partition arrangement information may include the bounding box position parameters, angle index, bounding box size parameters, and scaling parameters corresponding to the current partition; wherein the scaling parameters include scaling identification parameters and scaling coefficients.

[0115] It can be understood that in an embodiment of the present application, the position parameters of the bounding box of the current partition can represent the position coordinates of the upper left corner of the bounding box of the current partition in the image when arranged on the image, where posX represents the position of the horizontal coordinate and posY represents the position of the vertical coordinate.

[0116] For example, in some embodiments, after decoding the bitstream, the syntax elements uv_chart_posX[i] and uv_chart_posY[i] may be determined, and thus the bounding box position parameters may be determined based on these two syntax elements. uv_chart_posX[i] represents the position of the abscissa of the upper left corner of the partition bounding box in the two-dimensional texture image when the i-th partition is arranged; uv_chart_posY[i] represents the position of the ordinate of the upper left corner of the partition bounding box in the two-dimensional texture image when the i-th partition is arranged.

[0117] It can be understood that, in the embodiment of the present application, the angle index of the current partition is used to determine the rotation angle of the current partition when it is arranged on the image.

[0118] For example, in some embodiments, when decoding the code stream, the syntax element uv_chart_orientation_id[i] may be determined, and the angle index may be determined based on the syntax element, where uv_chart_orientation_id[i] represents the rotation angle index when the i-th partition is arranged.

[0119] It is understood that in the embodiment of the present application, the bounding box size parameter of the current partition may include the size of the bounding box of the current partition when arranged on the image. The bounding box size parameter may include the width sizeX and height sizeY of the bounding box.

[0120] For example, in some embodiments, after decoding the code stream, the syntax elements uv_chart_sizeX[i] and uv_chart_sizeY[i] may be determined, and the bounding box size parameters may be determined based on these two syntax elements. uv_chart_sizeX[i] represents the width of the partition bounding box when the i-th partition is arranged; uv_chart_sizeY[i] represents the height of the partition bounding box when the i-th partition is arranged.

[0121] It is understood that in an embodiment of the present application, the scaling parameters of the current partition may include a scaling indicator parameter corresponding to the current partition and a scaling factor corresponding to the current partition. The scaling indicator parameter may be used to determine whether scaling is performed when the current partition is arranged, and the scaling factor may represent the scaling scale during scaling.

[0122] For example, in some embodiments, when decoding a bitstream, the syntax elements uv_chart_scale_flag[i] and uv_chart_scale_factor[i] may be determined, and thus a scaling parameter may be determined based on these two syntax elements. uv_chart_scale_flag[i] indicates whether scaling is performed when arranging the i-th partition, i.e., the corresponding scaling flag parameter; and uv_chart_scale_factor[i] indicates the scaling factor of the i-th partition.

[0123] Furthermore, in an embodiment of the present application, when arranging the texture coordinates of at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed basic mesh, when the scaling identification parameter indicates that the current partition is not to be scaled, the texture coordinates of the current partition can be converted according to the bounding box position parameter, the angle index and the bounding box size parameter to determine the arranged texture coordinates corresponding to the current partition, so as to determine the texture coordinates corresponding to the reconstructed basic mesh.

[0124] It should be noted that in an embodiment of the present application, if the scaling identification parameter indicates that the current partition is not to be scaled, then the texture coordinates of the current partition can be directly converted according to the bounding box position parameters, angle index and bounding box size parameters of the current partition to complete the arrangement of the texture coordinates of the current partition and obtain the arranged texture coordinates.

[0125] It can be understood that in an embodiment of the present application, when the texture coordinates of the current partition are converted according to the bounding box position parameters, angle index and bounding box size parameters of the current partition, the rotation angle corresponding to the current partition can be determined according to the angle index, and the height and width of the bounding box of the current partition can be determined according to the bounding box size parameters; then, the matrix used for conversion processing can be further determined according to the rotation angle, width and height of the bounding box corresponding to the current partition; finally, the arranged texture coordinates corresponding to the current partition are further determined according to the bounding box position parameters and the matrix used for conversion processing.

[0126] For example, in some embodiments, the texture coordinates may be converted according to the following formula to obtain the arranged texture coordinates:

[0127] Among them, u0 and v0 are unarranged texture coordinates, u and v are arranged texture coordinates, width and height represent the width and height of the texture image, the bounding box position parameters of the current partition (i-th partition), that is, the position of the upper left corner of the bounding box is posX(i) and posY(i), and R(i) is a matrix defined according to the rotation angle and the width and height of the bounding box.

[0128] Furthermore, in an embodiment of the present application, when arranging the texture coordinates of at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed basic mesh, when the scaling identification parameter indicates that the current partition is scaled, the scaling factor can be determined first; then the texture coordinates of the current partition are converted according to the bounding box position parameter, the angle index, the bounding box size parameter and the scaling factor, and the arranged texture coordinates corresponding to the current partition are determined to determine the texture coordinates corresponding to the reconstructed basic mesh.

[0129] It should be noted that in an embodiment of the present application, if the scaling identification parameter indicates that the current partition is to be scaled, then the scaling coefficient of the current partition can be combined to convert the texture coordinates of the current partition according to the bounding box position parameters, angle index and bounding box size parameters of the current partition to complete the arrangement of the texture coordinates of the current partition and obtain the arranged texture coordinates.

[0130] It can be understood that in an embodiment of the present application, when the texture coordinates of the current partition are converted according to the bounding box position parameters, angle index, bounding box size parameters and scaling coefficient of the current partition, the rotation angle corresponding to the current partition can be determined according to the angle index, and the height and width of the bounding box of the current partition can be determined according to the bounding box size parameters; then, the matrix used for conversion processing can be further determined according to the rotation angle, width and height of the bounding box corresponding to the current partition; finally, according to the bounding box position parameters, scaling coefficient and the matrix used for conversion processing, the arranged texture coordinates corresponding to the current partition are further determined.

[0131] For example, in some embodiments, the texture coordinates may be converted according to the following formula to obtain the arranged texture coordinates:

[0132] Among them, u0 and v0 are unarranged texture coordinates, u and v are arranged texture coordinates, width and height represent the width and height of the texture image, the bounding box position parameters of the current partition (i-th partition), that is, the position of the upper left corner of the bounding box is posX(i) and posY(i), R(i) is a matrix defined according to the rotation angle and the width and height of the bounding box, and scale(i) is the scaling factor of the current partition.

[0133] Furthermore, in an embodiment of the present application, after completing the reconstruction of the texture coordinates and obtaining the first basic grid having the texture coordinates, the reconstructed subdivision grid of the current image can be further determined based on the first basic grid.

[0134] It should be noted that, in the embodiment of the present application, after the texture coordinates of the reconstructed basic mesh are generated, the first basic mesh may be further subdivided and deformed to obtain a reconstructed subdivided mesh of the current image.

[0135] Furthermore, in an embodiment of the present application, when determining a subdivided grid for reconstructing the current image based on the first basic grid, the first basic grid may be subdivided to determine a corresponding subdivided grid for reconstructing the current image.

[0136] It should be noted that, in the embodiment of the present application, when the first basic grid is subdivided to determine the reconstructed subdivided grid, the first grid subdivision parameters of the current image can be determined first; and then the first basic grid is iteratively divided according to the first grid subdivision parameters, so that the reconstructed subdivided grid can be determined.

[0137] Furthermore, in an embodiment of the present application, when determining the reconstructed subdivision grid of the current image based on the first basic grid, you can choose to first clean the first basic grid, determine the cleaned grid, and then subdivide the cleaned grid to determine the corresponding reconstructed subdivision grid.

[0138] It should be noted that in an embodiment of the present application, when subdividing the cleaned grid and determining the corresponding reconstructed subdivided grid, the second grid subdivision parameter of the current image can be determined first; and then the cleaned grid is iteratively divided according to the second grid subdivision parameter, so that the reconstructed subdivided grid can be determined.

[0139] That is to say, in an embodiment of the present application, when executing the subdivision deformation process, you can choose to directly perform subdivision deformation processing on the first basic grid to obtain the corresponding reconstructed subdivision grid; you can also choose to first clean the first basic grid, and then perform subdivision deformation processing on the cleaned grid to obtain the corresponding reconstructed subdivision grid.

[0140] It should be noted that, in the embodiment of the present application, the first grid subdivision parameter may include: a first subdivision method and a first subdivision iteration number.

[0141] Furthermore, in the embodiment of the present application, the first subdivision mode and the first subdivision iteration number can be determined by decoding the bitstream. In other words, both the first subdivision mode and the first subdivision iteration number can be determined based on syntax elements transmitted in the bitstream.

[0142] It should be noted that, in the embodiment of the present application, the second grid subdivision parameters may include: a second subdivision method and a second subdivision iteration number.

[0143] Furthermore, in the embodiment of the present application, the second subdivision mode and the second subdivision iteration number can be determined by decoding the bitstream. In other words, both the second subdivision mode and the second subdivision iteration number can be determined based on syntax elements transmitted in the bitstream.

[0144] It should be noted that, in the embodiment of the present application, the first subdivision method of the subdivision deformation processing may include midpoint subdivision, loop subdivision, etc., which is not limited here.

[0145] It should be noted that, in the embodiment of the present application, the second subdivision method of the subdivision deformation processing may include midpoint subdivision, loop subdivision, etc., which is not limited here.

[0146] It is understood that in the embodiments of the present application, the first grid subdivision parameters and the second grid subdivision parameters may be the same or different. The first subdivision method and the second subdivision method may be the same or different, and the first subdivision iteration number and the second subdivision iteration number may be the same or different, and this application does not specifically limit these.

[0147] For example, in some embodiments, FIG7 is a schematic diagram of the subdivision process proposed in an embodiment of the present application. As shown in FIG7 , taking the midpoint subdivision scheme as an example, each triangle is subdivided into four subtriangles in each subdivision iteration. A new vertex is introduced in the middle of each edge, and the subdivision of geometric information and attribute information is performed independently because the connection relationship between geometric information and attribute information is generally different.

[0148] Furthermore, in an embodiment of the present application, after determining the reconstructed subdivision grid of the current image based on the first basic grid, the reconstructed displacement information of the current image can be further determined, and then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid and the reconstructed displacement information.

[0149] Furthermore, in an embodiment of the present application, when determining the reconstructed grid of the current image based on the reconstructed subdivision grid, reconstructed displacement information of the current image can be determined, and the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid and the reconstructed displacement information.

[0150] It should be noted that, in the embodiment of the present application, after completing the subdivision deformation of the reconstructed basic grid and obtaining the corresponding reconstructed subdivision grid, the deformed grid can be further reconstructed in combination with the reconstructed displacement information to obtain the reconstructed grid of the current image.

[0151] Furthermore, in an embodiment of the present application, the reconstructed displacement information of the current image may be determined first.

[0152] It should be noted that in an embodiment of the present application, when determining the reconstructed displacement information of the current image, the code stream can be decoded first to determine the displacement information corresponding to the current image; then the displacement information can be reconstructed to determine the reconstructed displacement information.

[0153] It is understood that, in the embodiment of the present application, the corresponding displacement information can be determined by decoding the displacement code stream, wherein the displacement code stream can be decoded by a displacement decoder.

[0154] Accordingly, in an embodiment of the present application, after the displacement information is decoded, the displacement information can be further reconstructed to obtain corresponding reconstructed displacement information. In the process of reconstructing the displacement information, the displacement information can be optionally subjected to inverse transformation, inverse quantization, and other processing to obtain the reconstructed displacement information.

[0155] For example, in some embodiments, if the encoder compresses the displacement information through video coding, the decoder can decode the information using a corresponding video decoder and restore the displacement information from the two-dimensional image in the corresponding order according to the arrangement scheme. The displacement information is then subjected to inverse transformation, inverse quantization, and other operations to restore the reconstructed displacement information consistent with that obtained by the encoder.

[0156] For example, in some embodiments, if the encoding end adopts entropy coding, then at the decoding end, the displacement information can be directly entropy decoded, and then the displacement information can be subjected to subsequent reconstruction steps such as inverse transformation and inverse quantization to obtain reconstructed displacement information.

[0157] Furthermore, when determining the reconstructed mesh of the current image based on the reconstructed subdivided mesh and the reconstructed displacement information, a corresponding displacement operation may be performed on at least one vertex of the reconstructed subdivided mesh based on the reconstructed displacement information, thereby determining the reconstructed mesh of the current image.

[0158] It can be understood that in an embodiment of the present application, when reconstructing a deformed mesh based on the reconstructed displacement information and the reconstructed subdivided mesh, it is possible to choose to add the reconstructed displacement (reconstructed displacement information) to each vertex of the subdivided base mesh (reconstructed subdivided mesh) in sequence, thereby obtaining a reconstructed deformed mesh (reconstructed mesh), that is, the mesh finally output by the decoding end.

[0159] Furthermore, in the embodiment of the present application, the texture map of the current image can also be determined by decoding the code stream.

[0160] It should be noted that, in the embodiment of the present application, the texture map code stream may be decoded to obtain the corresponding texture map, wherein the texture map code stream may be decoded by a texture map decoder.

[0161] For example, in some embodiments, the texture map may be decoded using a video decoder, wherein an optional color space conversion may be performed to obtain an image format consistent with the texture map input by the encoder, and finally a decoded output texture map may be obtained.

[0162] Furthermore, in an embodiment of the present application, after the reconstructed grid of the current image and the texture map of the current image are determined, image application can be performed based on the reconstructed grid and the texture map.

[0163] It should be noted that in an embodiment of the present application, the texture map obtained by decoding may be the texture map corresponding to the reconstructed grid. After obtaining the reconstructed grid and the corresponding texture map, it may be further applied or rendered based on the reconstructed grid and the corresponding texture map.

[0164] That is, in an embodiment of the present application, the texture map (or attribute map) of the current image can also be determined. Here, the code stream can also be decoded to determine the texture map. Specifically, the texture map code stream can be decoded by a texture map decoder, or the texture map can also be decoded using a video decoder. An optional color space conversion is performed to obtain an image format consistent with the input texture map of the encoding end, thereby obtaining the texture map of the final decoded output.

[0165] In this way, at the decoding end, a reconstructed deformed mesh (reconstructed mesh) and a corresponding texture map are finally obtained, and subsequent applications can process the reconstructed deformed mesh and texture map as input.

[0166] In summary, through the decoding method proposed in the above steps 101 to 102, at the decoding end, the reconstructed basic grid obtained by decoding the code stream does not have texture coordinates, and then the parameterization of the reconstructed basic grid is completed at the decoding end according to the partition information and partition arrangement information corresponding to the encoding end transmitted in the code stream. That is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation results of the preprocessing stage correspond to the results of the reconstruction stage, thereby improving the encoding and decoding efficiency.

[0167] Furthermore, in an embodiment of the present application, FIG8 is a schematic diagram of a decoding process implementation proposed in an embodiment of the present application. As shown in FIG8 , auxiliary information can be transmitted in the code stream, wherein the auxiliary information, after being decoded, can guide each module to decode in accordance with a method preset by the encoding end. Specifically, the basic grid code stream is decoded using the basic grid decoder corresponding to the encoding end, and at the same time, the texture coordinates corresponding to the basic grid are reconstructed according to the texture coordinate information (partition information and partition arrangement information) in the texture coordinate code stream to obtain a first basic grid. At the same time, the first basic grid can be further subdivided and deformed to obtain a corresponding subdivided grid, i.e., a reconstructed subdivided grid. The displacement code stream is decoded by a displacement decoding module, and the displacement information is reconstructed, specifically including steps such as inverse transformation and inverse quantization. The reconstructed displacement information is then applied to the subdivided basic grid (reconstructed subdivided grid) to obtain a reconstructed deformed grid, i.e., a reconstructed grid. The texture map code stream is decoded by the texture map to obtain a reconstructed texture map.

[0168] Furthermore, in an embodiment of the present application, FIG9 is a second schematic diagram of the decoding process implementation proposed in an embodiment of the present application. As shown in FIG9 , auxiliary information can be transmitted in the bitstream, wherein, after decoding, the auxiliary information can guide each module to decode according to a method preset by the encoder. Specifically, the base grid bitstream is decoded using the base grid decoder corresponding to the encoder. Simultaneously, the texture coordinates corresponding to the base grid are reconstructed based on the texture coordinate information (partition information and partition arrangement information) in the texture coordinate bitstream to obtain a first base grid. Simultaneously, the first base grid can be further cleaned to obtain a cleaned grid, which is then subdivided and deformed to obtain a corresponding subdivided grid, i.e., a reconstructed subdivided grid. The displacement bitstream is decoded by the displacement decoding module, and the displacement information is reconstructed, specifically including steps such as inverse transformation and inverse quantization. The reconstructed displacement information is then applied to the subdivided base grid (reconstructed subdivided grid) to obtain a reconstructed deformed grid, i.e., a reconstructed grid. The texture map bitstream is decoded through the texture map to obtain a reconstructed texture map.

[0169] That is, in an embodiment of the present application, before subdividing the reconstructed base mesh (first base mesh), the first base mesh may be cleaned up to obtain a subdivided deformed mesh that better matches the first base mesh.

[0170] The embodiment of the present application provides a decoding method, and the codec can determine the texture coordinates based on the partition information, the partition arrangement information and the reconstructed basic grid, so that the reconstructed grid of the current image can be obtained based on the texture coordinates. Specifically, the reconstructed subdivision grid of the current image can be determined based on the basic grid (the first basic grid with texture coordinates); then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid. That is to say, in the embodiment of the present application, at the encoding end, the reconstructed basic grid written into the code stream does not have texture coordinates, that is, the reconstructed basic grid obtained by the decoding end does not have texture coordinates, and the decoding end can complete the parameterization of the reconstructed basic grid at the decoding end based on the partition information and partition arrangement information corresponding to the encoding end transmitted in the code stream, that is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation result of the preprocessing stage corresponds to the result of the reconstruction stage, thereby improving the encoding and decoding efficiency and improving the compression performance.

[0171] In one embodiment of the present application, FIG10 is a schematic diagram of an implementation flow of an encoding method proposed in the embodiment of the present application. As shown in FIG10 , the encoding method of the encoder may include the following steps:

[0172] Step 201: Determine a base grid of the current image and a reconstructed base grid, and write the reconstructed base grid into a bitstream.

[0173] In an embodiment of the present application, for a current image, the encoder may first determine a base grid corresponding to the current image and a reconstructed base grid, and then write the reconstructed base grid into a bitstream for transmission to a decoding end.

[0174] It should be noted that the encoding method in the embodiment of the present application may refer to a 3D mesh reconstruction method. In addition, the encoding method may be applied to an encoder, specifically an encoder in a V-DMC, but is not limited thereto.

[0175] Furthermore, in an embodiment of the present application, when determining the base grid of the current image, the input grid of the current image may be determined first; then the input grid may be downsampled to determine the base grid of the current image.

[0176] It should be noted that, in the embodiments of the present application, the base grid may also be referred to as a “simplified grid.” In some embodiments, the input grid of the current image may be subjected to downsampling processing to determine the base grid of the current image.

[0177] It should also be noted that, in the embodiment of the present application, the input grid of the current image may also be referred to as the original grid. By downsampling the input grid, that is, simplifying the input grid, a basic grid may be obtained.

[0178] It should be noted that in the embodiments of the present application, mesh simplification is to simplify the current input mesh into a base mesh with relatively fewer points and faces, while maintaining the original shape as much as possible. The focus of mesh simplification is the simplification operation and the corresponding merge error energy function.

[0179] For example, in some embodiments, FIG11 is a schematic diagram of a mesh simplification operation proposed in an embodiment of the present application. As shown in FIG11 , the vertices at both ends of an edge are merged into a single vertex, and the connection between the two vertices is deleted. This process is repeated throughout the entire mesh according to a certain rule to reduce the number of faces and vertices in the mesh to the target value.

[0180] It should be noted that in the embodiments of the present application, after the simplification operation and the corresponding error metric are determined, the mesh simplification is iteratively performed. First, the vertex errors of the original mesh are calculated to obtain the error of each edge. Each edge is then arranged from smallest to largest error, and the edge with the smallest error is selected for merging each time. At the same time, the position of the merged vertex is calculated, and the errors of all edges related to the merged vertex are updated. That is, the order of edge arrangement is updated to ensure that each iteration is based on a global error metric. The mesh surface is simplified to the number required to meet lossy coding through iteration.

[0181] Furthermore, in an embodiment of the present application, after determining the basic grid corresponding to the current image, a reconstructed basic grid corresponding to the current image may be further determined based on the basic grid.

[0182] Furthermore, in some embodiments, after determining the base grid of the current image, the base grid of the current image may be coded to obtain a reconstructed base grid of the current image, and the obtained coded bits may be written into a bitstream.

[0183] It should be noted that, in the embodiments of the present application, the obtained base grid can be encoded to obtain the corresponding reconstructed base grid. The base grid can be encoded using a base grid encoder. The base grid compression module is flexible, meaning the base grid encoder used by the module is replaceable. The encoder only needs to specify the identifier of the base grid encoder to be used, so that the decoder can use the corresponding base grid decoder for decoding.

[0184] That is, in embodiments of the present application, the encoder can generate corresponding auxiliary information based on the encoding information, then write the auxiliary information into the bitstream for transmission to the decoder. Accordingly, after decoding and obtaining the auxiliary information, the decoder can select the base grid decoder corresponding to the encoder based on the base grid encoder identifier indicated by the auxiliary information to decode the base grid bitstream, thereby obtaining the corresponding reconstructed base grid.

[0185] For example, the codestream here can refer to the basic grid codestream. A grid encoder (such as DRACO or MEB) can then write the current image's basic grid into the basic grid codestream. On the encoding side, DRACO is an open-source static grid encoder (Google DRACO) released by Google, and MEB is a static grid encoder (MPEG EdgeBreaker) implemented by MPEG.

[0186] Step 202: Determine texture coordinates according to the partition information, the partition arrangement information, and the reconstructed basic grid, determine the reconstructed grid of the current image based on the texture coordinates, and write the partition information and the partition arrangement information into a bitstream.

[0187] In an embodiment of the present application, after determining the basic grid that can be used to reconstruct the current image, the texture coordinates can be further determined based on the partition information, the partition arrangement information and the reconstructed basic grid, and then the reconstructed grid of the current image can be determined based on the texture coordinates. At the same time, the partition information and the partition arrangement information can also be written into the code stream and transmitted to the decoding end.

[0188] It should be noted that in an embodiment of the present application, the partition information and partition arrangement information can be encoded to generate a corresponding texture coordinate code stream, so that the texture coordinate code stream can be transmitted to the decoding end. Accordingly, the decoder can decode the texture coordinate code stream to determine the corresponding partition information and partition arrangement information.

[0189] It should be noted that, in the embodiments of the present application, the partition information and the partition arrangement information can be used to perform grid parameterization processing.

[0190] It can be understood that in the embodiments of the present application, when performing mesh parameterization, it is necessary to first partition the mesh, then perform independent parameterization on each partition, obtain the parameterization results of each partition, and finally arrange them into a texture map of a preset size according to specific rules.

[0191] It should be noted that, in the embodiment of the present application, the partition information may be a partition index that identifies each face in the mesh during the mesh parameterization process.

[0192] It should be noted that, in the embodiment of the present application, the partition arrangement information is used to determine the position of each partition in the two-dimensional texture map. The partition arrangement information may include at least: parameterization scheme identification parameters, parameterization mode of the current partition, angle index of the current partition, bounding box position parameters of the current partition, bounding box size parameters of the current partition, scaling parameters of the current partition, etc.

[0193] Illustratively, in some embodiments, the parameterization method of the current partition may be used to determine the method for parameterizing the mesh of the current partition.

[0194] For example, in some embodiments, the angle index of the current partition may include an angle index for determining the rotation of the current partition when it is arranged on the image.

[0195] Exemplarily, in some embodiments, the position parameters of the bounding box of the current partition may include the position posX and posY of the upper left corner of the bounding box of the current partition in the image when the current partition is arranged on the image.

[0196] For example, in some embodiments, the bounding box size parameters of the current partition may include the width (sizeX) and height (sizeY) of the bounding box of the current partition when the partition is arranged on the image.

[0197] For example, in some embodiments, the scaling parameter of the current partition indicates whether scaling is performed when the current partition is arranged and the corresponding scaling coefficient.

[0198] Furthermore, in an embodiment of the present application, the decoder may write the following syntax elements into the texture coordinate code stream and transmit it to the decoding end:

[0199] Among them, base_mesh_texcoord_information() represents the texture coordinate information in the base mesh code stream;

[0200] base_mesh_parameterazation_method indicates the parameterization scheme used by the base mesh (parameterization scheme identification parameter), 0 represents the use of the isochart algorithm, 1 represents the use of the orthogonal projection algorithm, the default is 0;

[0201] base_mesh_uv_charts_number_minus1 means the number of partitions after parameterization minus 1;

[0202] uv_chart_parameterazation_type[i] represents the projection plane corresponding to the parameterization of the i-th partition. This value is only valid when the parameterization scheme is orthogonal projection parameterization;

[0203] uv_chart_orientation_id[i] represents the rotation angle index when the i-th partition is arranged;

[0204] uv_chart_posX[i] represents the position of the horizontal coordinate of the upper left corner of the partition bounding box in the two-dimensional texture map when the i-th partition is arranged;

[0205] uv_chart_posY[i] represents the position of the ordinate of the upper left corner of the partition bounding box in the two-dimensional texture map when the i-th partition is arranged;

[0206] uv_chart_sizeX[i] represents the width of the partition bounding box when the i-th partition is arranged;

[0207] uv_chart_sizeY[i] represents the height of the partition bounding box when the i-th partition is arranged;

[0208] uv_chart_scale_flag[i] indicates whether the i-th partition is scaled when arranged;

[0209] uv_chart_scale_factor[i] represents the scaling factor of the i-th partition.

[0210] Furthermore, in an embodiment of the present application, when determining texture coordinates based on partition information, partition arrangement information, and a reconstructed base mesh, the texture coordinates corresponding to the reconstructed base mesh may be determined based on the partition information and partition arrangement information.

[0211] Furthermore, in an embodiment of the present application, after determining the texture coordinates based on the partition information, the partition arrangement information, and the reconstructed base mesh, reconstruction can be further performed based on the texture coordinates to obtain a first base mesh, wherein the first base mesh can be a base mesh with texture coordinates.

[0212] It should be noted that in an embodiment of the present application, after determining the reconstructed basic grid, partition information and partition arrangement information, the partition information and partition arrangement information can be used to perform mesh parameterization on the reconstructed basic grid, thereby completing the reconstruction of the texture coordinates and obtaining the texture coordinates corresponding to the reconstructed basic grid.

[0213] Furthermore, in an embodiment of the present application, when determining the texture coordinates corresponding to the reconstructed basic mesh based on the partition information and the partition arrangement information, the reconstructed basic mesh can be partitioned according to the partition information to determine at least one partition corresponding to the reconstructed basic mesh; then the texture coordinates of at least one partition can be determined based on the partition arrangement information; finally, the texture coordinates of at least one partition can be arranged based on the partition arrangement information to determine the texture coordinates corresponding to the reconstructed basic mesh.

[0214] It should be noted that, in the examples of the present application, the reconstructed basic grid of the current image may be partitioned first according to the partition information to obtain at least one corresponding partition.

[0215] It is understood that in the embodiments of the present application, for any of the at least one partition obtained, the pre-arrangement texture coordinates of the partition, i.e., the texture coordinates of the partition, can be first determined according to the partition arrangement information. The texture coordinates of the partition can then be transformed according to the partition arrangement information to obtain the final reconstructed texture coordinates.

[0216] Furthermore, in an embodiment of the present application, the partition arrangement information may include a parameterization scheme identification parameter. When determining the texture coordinates of at least one partition based on the partition arrangement information, if the parameterization scheme identification parameter indicates that the current image uses a first parameterization scheme, all partitions can be parameterized according to the first parameterization method to determine the texture coordinates of at least one partition.

[0217] It is understood that in the embodiment of the present application, the parameterization scheme identification parameter is used to determine the parameterization scheme used for the current image, wherein the parameterization scheme includes but is not limited to the isochart algorithm and the orthogonal projection algorithm.

[0218] For example, in some embodiments, the parameterization scheme used for the current image can be determined based on the value of the parameterization scheme identification parameter. If the value of the parameterization scheme identification parameter is a first value, the parameterization scheme used for the current image can be determined to be the first parameterization scheme, such as the isochart algorithm.

[0219] It should be noted that, in the embodiment of the present application, for the isochart algorithm, the parameterization method of each partition can be the same. Therefore, each partition can be parameterized according to the same first parameterization method, and finally the texture coordinates of each partition can be determined.

[0220] Furthermore, in an embodiment of the present application, the partition arrangement information may include a parameterization scheme identification parameter and a second parameterization method corresponding to the current partition. When determining the texture coordinates of at least one partition based on the partition arrangement information, if the parameterization scheme identification parameter indicates that the current image uses the second parameterization scheme, the current partition can be parameterized separately according to the second parameterization method to determine the texture coordinates of the current partition.

[0221] For example, in some embodiments, the parameterization scheme used for the current image indicated by the parameterization scheme identification parameter may be determined based on the value of the parameterization scheme identification parameter. If the value of the parameterization scheme identification parameter is the second value, the parameterization scheme used for the current image may be determined to be the second parameterization scheme, such as the orthogonal projection algorithm.

[0222] It is understood that in the embodiments of the present application, the first value and the second value are different. For example, if the parameterized scheme identification parameter has a value of 0, it indicates that the isochart algorithm is used, and if the parameterized scheme identification parameter has a value of 1, it indicates that the orthogonal projection algorithm is used.

[0223] It should be noted that, in an embodiment of the present application, for the orthogonal projection algorithm, for the current partition in at least one partition, when the parameterization scheme identification parameter indicates that the current image uses the second parameterization scheme, the current partition can be parameterized according to the second parameterization method corresponding to the current partition in the partition arrangement information, so that the texture coordinates of the current partition can be determined.

[0224] Accordingly, in the embodiment of the present application, after each partition is parameterized in turn according to the second parameterization method corresponding to each partition, the texture coordinates of each partition can be determined.

[0225] Furthermore, in an embodiment of the present application, in addition to the parameterized scheme identification parameters, the partition arrangement information may include the bounding box position parameters, angle index, bounding box size parameters, and scaling parameters corresponding to the current partition; wherein the scaling parameters include scaling identification parameters and scaling coefficients.

[0226] It can be understood that in an embodiment of the present application, the position parameters of the bounding box of the current partition can represent the position coordinates of the upper left corner of the bounding box of the current partition in the image when arranged on the image, where posX represents the position of the horizontal coordinate and posY represents the position of the vertical coordinate.

[0227] For example, in some embodiments, the bounding box position parameters may be determined, and then the syntax elements uv_chart_posX[i] and uv_chart_posY[i] may be determined based on the bounding box position parameters. uv_chart_posX[i] represents the position of the horizontal coordinate of the upper left corner of the partition bounding box in the two-dimensional texture image when the i-th partition is arranged; and uv_chart_posY[i] represents the position of the vertical coordinate of the upper left corner of the partition bounding box in the two-dimensional texture image when the i-th partition is arranged.

[0228] It can be understood that, in the embodiment of the present application, the angle index of the current partition is used to determine the rotation angle of the current partition when it is arranged on the image.

[0229] For example, in some embodiments, an angle index may be determined, and then a syntax element uv_chart_orientation_id[i] may be determined according to the angle index, where uv_chart_orientation_id[i] represents the rotation angle index when the i-th partition is arranged.

[0230] It is understood that in the embodiment of the present application, the bounding box size parameter of the current partition may include the size of the bounding box of the current partition when arranged on the image. The bounding box size parameter may include the width sizeX and height sizeY of the bounding box.

[0231] For example, in some embodiments, a bounding box size parameter may be determined, and then the syntax elements uv_chart_sizeX[i] and uv_chart_sizeY[i] may be determined based on the bounding box size parameter, where uv_chart_sizeX[i] represents the width of the partition bounding box when the i-th partition is arranged; and uv_chart_sizeY[i] represents the height of the partition bounding box when the i-th partition is arranged.

[0232] It is understood that in an embodiment of the present application, the scaling parameters of the current partition may include a scaling indicator parameter corresponding to the current partition and a scaling factor corresponding to the current partition. The scaling indicator parameter may be used to determine whether scaling is performed when the current partition is arranged, and the scaling factor may represent the scaling scale during scaling.

[0233] For example, in some embodiments, it is possible to determine whether the current partition is scaled, and further determine the corresponding scaling factor when scaling is performed, so that the syntax elements uv_chart_scale_flag[i] and uv_chart_scale_factor[i] can be set. uv_chart_scale_flag[i] indicates whether the i-th partition is scaled when arranged, i.e., the corresponding scaling flag parameter; uv_chart_scale_factor[i] indicates the scaling factor of the i-th partition.

[0234] Furthermore, in an embodiment of the present application, when arranging the texture coordinates of at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed basic mesh, when it is determined that the current partition is not scaled, the texture coordinates of the current partition can be converted according to the bounding box position parameters, angle index and bounding box size parameters to determine the arranged texture coordinates corresponding to the current partition, so as to determine the texture coordinates corresponding to the reconstructed basic mesh.

[0235] Accordingly, in an embodiment of the present application, when it is determined that the current partition is not to be scaled, a scaling flag parameter may be set to indicate that the current partition is not to be scaled.

[0236] It should be noted that in an embodiment of the present application, if the current partition is not scaled, the texture coordinates of the current partition can be directly converted according to the bounding box position parameters, angle index and bounding box size parameters of the current partition to complete the arrangement of the texture coordinates of the current partition and obtain the arranged texture coordinates.

[0237] It can be understood that in an embodiment of the present application, when the texture coordinates of the current partition are converted according to the bounding box position parameters, angle index and bounding box size parameters of the current partition, the rotation angle corresponding to the current partition can be determined according to the angle index, and the height and width of the bounding box of the current partition can be determined according to the bounding box size parameters; then, the matrix used for conversion processing can be further determined according to the rotation angle, width and height of the bounding box corresponding to the current partition; finally, the arranged texture coordinates corresponding to the current partition are further determined according to the bounding box position parameters and the matrix used for conversion processing.

[0238] For example, in some embodiments, the texture coordinates may be converted according to formula (1) to obtain arranged texture coordinates.

[0239] Among them, u0 and v0 are unarranged texture coordinates, u and v are arranged texture coordinates, width and height represent the width and height of the texture image, the bounding box position parameters of the current partition (i-th partition), that is, the position of the upper left corner of the bounding box is posX(i) and posY(i), and R(i) is a matrix defined according to the rotation angle and the width and height of the bounding box.

[0240] Furthermore, in an embodiment of the present application, when arranging the texture coordinates of at least one partition according to the partition arrangement information and determining the texture coordinates corresponding to the reconstructed basic mesh, when determining that the current partition is scaled, the scaling factor can be determined first; then the texture coordinates of the current partition are converted according to the bounding box position parameters, angle index, bounding box size parameters and scaling factor, and the arranged texture coordinates corresponding to the current partition are determined to determine the texture coordinates corresponding to the reconstructed basic mesh.

[0241] Accordingly, in an embodiment of the present application, when it is determined that the current partition is to be scaled, a scaling identification parameter may be set to indicate that the current partition is to be scaled.

[0242] It should be noted that in an embodiment of the present application, if the current partition is scaled, the texture coordinates of the current partition can be converted according to the bounding box position parameters, angle index and bounding box size parameters of the current partition in combination with the scaling coefficient of the current partition to complete the arrangement of the texture coordinates of the current partition and obtain the arranged texture coordinates.

[0243] It can be understood that in an embodiment of the present application, when the texture coordinates of the current partition are converted according to the bounding box position parameters, angle index, bounding box size parameters and scaling coefficient of the current partition, the rotation angle corresponding to the current partition can be determined according to the angle index, and the height and width of the bounding box of the current partition can be determined according to the bounding box size parameters; then, the matrix used for conversion processing can be further determined according to the rotation angle, width and height of the bounding box corresponding to the current partition; finally, according to the bounding box position parameters, scaling coefficient and the matrix used for conversion processing, the arranged texture coordinates corresponding to the current partition are further determined.

[0244] For example, in some embodiments, the texture coordinates may be converted according to formula (2) to obtain arranged texture coordinates.

[0245] Among them, u0 and v0 are unarranged texture coordinates, u and v are arranged texture coordinates, width and height represent the width and height of the texture image, the bounding box position parameters of the current partition (i-th partition), that is, the position of the upper left corner of the bounding box is posX(i) and posY(i), R(i) is a matrix defined according to the rotation angle and the width and height of the bounding box, and scale(i) is the scaling factor of the current partition.

[0246] Furthermore, in an embodiment of the present application, after completing the reconstruction of the texture coordinates and obtaining the first basic grid having the texture coordinates, the reconstructed subdivision grid of the current image can be further determined based on the first basic grid.

[0247] It should be noted that, in the embodiment of the present application, after the texture coordinates of the reconstructed basic mesh are generated, the first basic mesh may be further subdivided and deformed to obtain a reconstructed subdivided mesh of the current image.

[0248] Furthermore, in an embodiment of the present application, when determining a subdivided grid for reconstructing the current image based on the first basic grid, the first basic grid may be subdivided to determine a corresponding subdivided grid for reconstructing the current image.

[0249] It should be noted that, in an embodiment of the present application, when subdividing the first basic grid and determining the reconstructed subdivided grid, the first grid subdivision parameters of the current image can be determined first; and then the first basic grid is iteratively divided according to the first grid subdivision parameters, so that the reconstructed subdivided grid can be determined.

[0250] Furthermore, in an embodiment of the present application, when determining the reconstructed subdivision grid of the current image based on the first basic grid, you can choose to first clean the first basic grid, determine the cleaned grid, and then subdivide the cleaned grid to determine the corresponding reconstructed subdivision grid.

[0251] It should be noted that in an embodiment of the present application, when subdividing the cleaned grid and determining the corresponding reconstructed subdivided grid, the second grid subdivision parameter of the current image can be determined first; and then the cleaned grid is iteratively divided according to the second grid subdivision parameter, so that the reconstructed subdivided grid can be determined.

[0252] That is to say, in an embodiment of the present application, when executing the subdivision deformation process, you can choose to directly perform subdivision deformation processing on the first basic grid to obtain the corresponding reconstructed subdivision grid; you can also choose to first clean the first basic grid, and then perform subdivision deformation processing on the cleaned grid to obtain the corresponding reconstructed subdivision grid.

[0253] It should be noted that, in the embodiment of the present application, the first grid subdivision parameter may include: a first subdivision method and a first subdivision iteration number.

[0254] Furthermore, in an embodiment of the present application, the first subdivision method and the first subdivision iteration number can be written into the bitstream. That is, at the decoding end, both the first subdivision method and the first subdivision iteration number can be determined based on the syntax elements transmitted in the bitstream.

[0255] It should be noted that, in the embodiment of the present application, the second grid subdivision parameters may include: a second subdivision method and a second subdivision iteration number.

[0256] Furthermore, in an embodiment of the present application, the second subdivision mode and the second subdivision iteration number can be written into the bitstream. That is, at the decoding end, both the second subdivision mode and the second subdivision iteration number can be determined based on the syntax elements transmitted in the bitstream.

[0257] It should be noted that, in the embodiment of the present application, the first subdivision method of the subdivision deformation processing may include midpoint subdivision, loop subdivision, etc., which is not limited here.

[0258] It should be noted that, in the embodiment of the present application, the second subdivision method of the subdivision deformation processing may include midpoint subdivision, loop subdivision, etc., which is not limited here.

[0259] It is understood that in the embodiments of the present application, the first grid subdivision parameters and the second grid subdivision parameters may be the same or different. The first subdivision method and the second subdivision method may be the same or different, and the first subdivision iteration number and the second subdivision iteration number may be the same or different, and this application does not specifically limit these.

[0260] For example, in some embodiments, as shown in FIG7 , a midpoint subdivision scheme is used as an example. In each subdivision iteration, each triangle is subdivided into four subtriangles. A new vertex is introduced in the middle of each edge, and the subdivision of geometric information and attribute information is performed independently because the connection relationship between geometric information and attribute information is generally different.

[0261] Furthermore, in an embodiment of the present application, after determining the reconstructed subdivision grid of the current image based on the first basic grid, the reconstructed displacement information of the current image can be further determined, and then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid and the reconstructed displacement information.

[0262] Furthermore, in an embodiment of the present application, when determining the reconstructed grid of the current image based on the reconstructed subdivision grid, reconstructed displacement information of the current image can be determined, and the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid and the reconstructed displacement information.

[0263] It should be noted that, in the embodiment of the present application, after completing the subdivision deformation of the reconstructed basic grid and obtaining the corresponding reconstructed subdivision grid, the deformed grid can be further reconstructed in combination with the reconstructed displacement information to obtain the reconstructed grid of the current image.

[0264] Furthermore, in an embodiment of the present application, the reconstructed displacement information of the current image may be determined first.

[0265] It should be noted that, in the embodiment of the present application, when determining the reconstructed displacement information of the current image, the displacement information corresponding to the current image may be determined first, and then the displacement information may be reconstructed to determine the reconstructed displacement information.

[0266] It should be noted that, in the embodiment of the present application, when determining the displacement information corresponding to the current image, the subdivision grid of the current image may be determined first; and then the displacement information may be determined based on the subdivision grid.

[0267] Furthermore, in an embodiment of the present application, when determining the subdivision network of the current image, the basic grid may be subdivided, thereby determining the subdivision grid of the current image.

[0268] It can be understood that in an embodiment of the present application, when subdividing the basic grid to determine the subdivided grid of the current image, the third grid subdivision parameter of the current image can be determined first; and then the basic grid is iteratively divided according to the third grid subdivision parameter to determine the subdivided grid of the current image.

[0269] It should be noted that, in the embodiment of the present application, the third grid subdivision parameter may include: a third subdivision method and a third subdivision iteration number.

[0270] Furthermore, in an embodiment of the present application, the third subdivision method and the third subdivision iteration number can be written into the bitstream. That is, at the decoding end, both the third subdivision method and the third subdivision iteration number can be determined based on the syntax elements transmitted in the bitstream.

[0271] It should be noted that, in the embodiment of the present application, the third subdivision method of the subdivision deformation processing may include midpoint subdivision, loop subdivision, etc., which is not limited here.

[0272] It is understood that in the embodiments of the present application, the first grid subdivision parameter, the second grid subdivision parameter, and the third grid subdivision parameter may be the same or different. Furthermore, the first subdivision method, the second subdivision method, and the third subdivision method may be the same or different; and the first subdivision iteration number, the second subdivision iteration number, and the third subdivision iteration number may be the same or different, and this application does not specifically limit these.

[0273] It should be noted that, in the embodiments of the present application, generally, the third subdivision iteration number is greater than or equal to the first subdivision iteration number and the second subdivision iteration number.

[0274] Furthermore, in the implementation of the present application, when determining displacement information based on a subdivided grid, multiple segments of the subdivided grid can be deformed first to determine the deformed grid; then corresponding displacement calculations can be performed on at least one vertex of the deformed grid and at least one vertex of the subdivided grid to determine the displacement information of at least one vertex.

[0275] It is understandable that, in the embodiment of the present application, after the subdivision operation is performed, the displacement information of the subdivided grid can be further determined based on the obtained subdivided grid.

[0276] It should be noted that, in the embodiment of the present application, a corresponding deformed grid may be determined based on the subdivided grid. When determining the deformed grid, multiple segments of the subdivided grid may be deformed to determine the deformed grid.

[0277] It can be understood that, in the embodiment of the present application, corresponding displacement calculation can be performed between the subdivided grid and the deformed grid, so as to determine the displacement information.

[0278] For example, in some embodiments, corresponding displacement calculations may be performed on at least one vertex of the deformed mesh and at least one vertex of the subdivided mesh to determine displacement information of the at least one vertex, thereby obtaining displacement information of the subdivided mesh.

[0279] It can be understood that in the embodiment of the present application, after determining the displacement information corresponding to the current image, the displacement information can be written into the code stream; accordingly, in the embodiment of the present application, at the decoding end, the code stream can be decoded to determine the displacement information corresponding to the current image.

[0280] That is, in the embodiment of the present application, at the decoding end, the corresponding displacement information can be determined by decoding the displacement code stream, wherein the displacement code stream can be decoded by a displacement decoder.

[0281] Furthermore, in an embodiment of the present application, after the displacement information is determined, the displacement information can be further reconstructed to obtain corresponding reconstructed displacement information. In the process of reconstructing the displacement information, the displacement information can be optionally subjected to inverse transformation, inverse quantization, and other processing to obtain the reconstructed displacement information.

[0282] For example, in some embodiments, if the encoder compresses the displacement information through video coding, the decoder can decode the information using a corresponding video decoder and restore the displacement information from the two-dimensional image in the corresponding order according to the arrangement scheme. The displacement information is then subjected to inverse transformation, inverse quantization, and other operations to restore the reconstructed displacement information consistent with that obtained by the encoder.

[0283] For example, in some embodiments, if the encoding end adopts entropy coding, then at the decoding end, the displacement information can be directly entropy decoded, and then the displacement information can be subjected to subsequent reconstruction steps such as inverse transformation and inverse quantization to obtain reconstructed displacement information.

[0284] Furthermore, when determining the reconstructed mesh of the current image based on the reconstructed subdivided mesh and the reconstructed displacement information, a corresponding displacement operation may be performed on at least one vertex of the reconstructed subdivided mesh based on the reconstructed displacement information, thereby determining the reconstructed mesh of the current image.

[0285] It can be understood that in an embodiment of the present application, when reconstructing a deformed mesh based on the reconstructed displacement information and the reconstructed subdivided mesh, it is possible to choose to add the reconstructed displacement (reconstructed displacement information) to each vertex of the subdivided base mesh (reconstructed subdivided mesh) in sequence, thereby obtaining a reconstructed deformed mesh (reconstructed mesh), that is, the mesh finally output by the decoding end.

[0286] Furthermore, in an embodiment of the present application, after determining the reconstructed subdivision grid of the current image, a texture map of the current image may be further determined based on the reconstructed subdivision grid of the current image, and the texture map may be written into the bitstream.

[0287] It should be noted that in the embodiment of the present application, the texture map encoding module first performs texture map conversion based on the input original mesh, the input original texture map, and the mesh after subdivision and deformation (reconstructed subdivision mesh). Then, the converted texture map is encoded and written into the texture map code stream.

[0288] In summary, through the decoding method proposed in the above steps 201 to 202, at the encoding end, the reconstructed basic grid written into the bitstream does not have texture coordinates. At the same time, the partition information and partition arrangement information used by the encoding end in the parameterization process can be written into the bitstream and transmitted to the decoding end, so that the decoding end completes the parameterization of the reconstructed basic grid based on the partition information and partition arrangement information obtained by decoding. That is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation result of the preprocessing stage corresponds to the result of the reconstruction stage, thereby improving the encoding and decoding efficiency.

[0289] Furthermore, in an embodiment of the present application, FIG12 is a schematic diagram of a first embodiment of the encoding process proposed in an embodiment of the present application. As shown in FIG12 , after the input mesh passes through the base mesh generation module, a base mesh with a smaller amount of data and a corresponding subdivided deformed mesh (subdivided mesh) are obtained. The base mesh can then be encoded using a static mesh encoder. The base mesh only contains geometric coordinates and connection relationships, not texture coordinates. The reconstructed base mesh is then mesh parameterized to obtain a base mesh containing texture coordinates, i.e., a first base mesh. The partition information and partition arrangement information in the mesh parameterization are transmitted to the decoder as a texture coordinate code stream. Subsequently, displacement information is calculated based on the subdivided deformed mesh obtained in the base mesh generation phase and the reconstructed first base mesh. The displacement is then transformed, quantized, and other operations are performed to obtain reconstructed displacement information. The displacement information can also be encoded and transmitted to the decoder. Furthermore, a reconstructed deformed mesh (reconstructed mesh) can be obtained based on the reconstructed displacement and the reconstructed base mesh (first base mesh). The reconstructed mesh is input into a texture map conversion module to obtain a texture map corresponding to the reconstructed deformed mesh. The relevant encoding information required by the decoder is transmitted to the decoder via auxiliary information. After the auxiliary information is decoded, it can guide each module to decode according to the method preset by the encoding end.

[0290] Furthermore, in an embodiment of the present application, FIG13 is a second schematic diagram of an implementation of the encoding process proposed in an embodiment of the present application. As shown in FIG13 , after the input mesh passes through the base mesh generation module, a base mesh with a smaller data volume is obtained. This base mesh can then be encoded using a static mesh encoder. This base mesh only contains geometric coordinates and connectivity relationships, not texture coordinates. The reconstructed base mesh is then mesh parameterized to obtain a base mesh containing texture coordinates, namely, the first base mesh. The partition information and partition arrangement information in the mesh parameterization are transmitted to the decoder as a texture coordinate code stream. The position of the subdivision deformation module has changed, and the input of the subdivision deformation module is the cleaned base mesh. Subsequently, displacement information is calculated based on the obtained subdivision deformed mesh and the reconstructed first base mesh. The displacement is then transformed, quantized, and other operations are performed to obtain reconstructed displacement information. The displacement information can also be encoded and transmitted to the decoder. Furthermore, a reconstructed deformed mesh (reconstructed mesh) can be obtained based on the reconstructed displacement and the reconstructed base mesh (first base mesh). The reconstructed mesh is then input into the texture map conversion module to obtain a texture map corresponding to the reconstructed deformed mesh. The relevant encoding information required by the decoder is transmitted to the decoder via auxiliary information.

[0291] That is, in an embodiment of the present application, before subdividing the reconstructed base mesh (first base mesh), the first base mesh may be cleaned up to obtain a subdivided deformed mesh that better matches the first base mesh.

[0292] The embodiment of the present application provides an encoding method, and the codec can determine the texture coordinates based on the partition information, the partition arrangement information and the reconstructed basic grid, so that the reconstructed grid of the current image can be obtained based on the texture coordinates. Specifically, the reconstructed subdivision grid of the current image can be determined based on the basic grid (the first basic grid with texture coordinates); then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid. That is to say, in the embodiment of the present application, at the encoding end, the reconstructed basic grid written into the code stream does not have texture coordinates, that is, the reconstructed basic grid obtained by the decoding end does not have texture coordinates, and the decoding end can complete the parameterization of the reconstructed basic grid at the decoding end based on the partition information and partition arrangement information corresponding to the encoding end transmitted in the code stream, that is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation result of the preprocessing stage corresponds to the result of the reconstruction stage, thereby improving the encoding and decoding efficiency and improving the compression performance.

[0293] Based on the above embodiment, another embodiment of the present application proposes a coding and decoding method, wherein the coding and decoding method may include a reparameterizable three-dimensional mesh coding and decoding framework. Specifically, the coding framework first encodes the simplified input mesh to obtain a reconstructed base mesh, and then performs mesh parameterization on it to obtain texture coordinates corresponding to the reconstructed base mesh. The encoding end passes the partition information and partition arrangement information during the mesh parameterization process to the decoding end to guide the decoding end to regenerate the texture coordinates.

[0294] Furthermore, in an embodiment of the present application, at the encoding end, after the input mesh passes through the basic mesh generation module, a basic mesh with less data volume and a corresponding subdivided deformed mesh are obtained, and then the basic mesh is encoded using a static mesh encoder, and the basic mesh only contains geometric coordinates and connection relationships. The reconstructed basic mesh is then mesh parameterized to obtain a basic mesh containing texture coordinates, and the partition information and partition arrangement information in the mesh parameterization are passed to the decoding end as a texture coordinate code stream. The displacement is then calculated based on the subdivided deformed mesh obtained in the basic mesh generation stage and the reconstructed basic mesh. The displacement is then transformed, quantized, and other operations are performed, and the processed displacement is encoded and reconstructed. A reconstructed deformed mesh is obtained based on the reconstructed displacement and the reconstructed basic mesh, and is input into a texture map conversion module to obtain a texture map corresponding to the reconstructed deformed mesh. The relevant coding information required by the decoding end is passed to the decoding end through auxiliary information.

[0295] (1) Basic mesh generation

[0296] The basic mesh generation module includes mesh simplification and subdivision deformation modules.

[0297] Mesh simplification involves reducing the input mesh to a base mesh with relatively few points and faces, while preserving the original shape as much as possible. The key to mesh simplification lies in the simplification operation and the corresponding merge error energy function. A possible mesh simplification operation is shown in Figure 11. This operation merges the vertices at the ends of an edge into a single vertex and deletes the connection between them. This process is repeated throughout the mesh according to a specific rule to reduce the number of faces and vertices to the target value.

[0298] During the simplification process, you can choose an error metric to optimize the simplified results. For example, the error metric for a vertex can be the sum of the coefficients of the equations of all adjacent faces. The error metric for an edge can be the sum of the error metrics of the two vertices on the edge. In other words, the error resulting from merging an edge is the sum of the squared distances from the merged vertex to all adjacent faces of the original two vertices on the edge. Furthermore, the error metric can also take into account the displacement between the base mesh and the subdivided mesh.

[0299] After determining the simplification operation and the corresponding error metric, the mesh simplification process begins iteratively. First, the vertex errors of the initial mesh are calculated to obtain the error for each edge. Edges are then sorted from smallest to largest error, and the edge with the smallest error is merged each time. Simultaneously, the positions of the merged vertices are calculated, and the errors of all edges associated with the merged vertices are updated. This means that the order of edge arrangement is updated to ensure that each iteration is based on a global error metric. Through iteration, the mesh faces are simplified to the number required for lossy encoding.

[0300] Figure 14 is a schematic diagram of the subdivision deformation processing proposed in an embodiment of the present application. The basic idea of ​​the subdivision deformation module is shown in Figure 14, and the same concept is applied to the input 3D mesh to generate displacement. In Figure 14, the input 2D curve (represented by a 2D polyline), called the "original" curve, is first downsampled to generate a basic curve / polyline, called a "simplified" curve. The subdivision scheme is then applied to the simplified polyline to generate a "subdivision" curve. The subdivided polyline is then deformed to obtain a better approximation of the original curve. That is, a displacement vector is calculated for each vertex of the subdivided mesh so that the shape of the displacement curve is as close as possible to the shape of the original curve. These displacement vectors are the displacement information output by the module.

[0301] This step first subdivides the input mesh. Any subdivision scheme can be chosen. One possible scheme is the midpoint subdivision scheme, which subdivides each triangle into four subtriangles in each subdivision iteration, as shown in Figure 7. A new vertex is introduced in the middle of each edge. The subdivision of geometric information and attribute information is performed independently, as the connection between geometric information and attribute information is usually different.

[0302] For the subdivided mesh, we find the nearest neighbor of each point on the original input mesh (including points on the original mesh surface). This can be accelerated using data structures such as kdTree. The displacement of the geometric coordinates of each vertex on the subdivided mesh is obtained by calculating the distance between the geometric coordinates of each vertex on the subdivided mesh and its nearest neighbor on the original input mesh. This module can generate a base mesh and the corresponding subdivided, deformed mesh.

[0303] (2) Basic mesh compression

[0304] In this framework, the basic grid compression module is flexible, that is, the basic grid encoder used by the module is replaceable. The encoding end only needs to specify the identifier of the basic grid encoder to be used so that the decoding end can use the corresponding basic grid decoder for decoding.

[0305] (3) Mesh parameterization

[0306] Mesh parameterization is used to generate texture coordinates for a mesh. Numerous algorithms exist for this purpose, including the Isochart algorithm and the orthogonal projection algorithm. Within this encoding framework, both of the aforementioned schemes can be used to parameterize the reconstructed base mesh. The following briefly describes both algorithms.

[0307] The Isochart algorithm uses spectral analysis to implement stretch-driven 3D mesh parameterization, UV-unwrapping the 3D mesh, slicing it, and packing it into a 2D texture domain. A stretch threshold is set, and the algorithm is outlined as follows:

[0308] a) Compute surface spectrum analysis to provide an initial parameterization

[0309] b) Perform stretch optimization iterations

[0310] c) If the stretch of this derived parameterization is less than a threshold, stop

[0311] d) Perform surface spectral clustering to divide the surface into charts

[0312] e) Use graph cut algorithm to optimize chart boundaries

[0313] f) Iteratively split the charts until the stretching criteria are met

[0314] Surface spectral analysis is parameterized based on the dimensionality reduction method IsoMap (isometric feature mapping). Given a set of high-dimensional points, IsoMap computes the geodesic distances along the manifold as a sequence of hops between adjacent points. The MDS (multidimensional scaling) algorithm is then applied to these geodesic distances to find a set of points with similar pairwise distances embedded in a low-dimensional space. Given a surface with N points, the computation proceeds as follows:

[0315] a) Calculate the symmetric matrix D of the squared geodesic distances between surface points N ;

[0316] b) For D N Perform double centering and normalization to get D N , and its calculation process is shown in the formula:

[0317] in, I is the N-dimensional identity matrix and 1 is the unit vector of length N.

[0318] c) Calculate B N The eigenvalue λ i and the corresponding eigenvectors

[0319] d) For each point i on the original surface, its embedding in the new space is an N-dimensional vector The calculation process of its j-th element is as shown in the formula:

[0320] B N The eigenvalue λ of i and the corresponding eigenvector constitute the spectral decomposition of the surface shape. The eigenvectors corresponding to large eigenvalues represent the global low-frequency features on the surface, and the eigenvectors corresponding to small eigenvalues represent high-frequency details. The high-energy and low-frequency components are used as the basis for chartification and parameterization.

[0321] Although N eigenvalues are required to fully represent a surface with N vertices, a small portion of them usually occupies most of the energy. Therefore, only n << N largest eigenvalues and the corresponding eigenvectors are calculated to generate an n-dimensional embedding of all points.

[0322] Since the mapping from a high-dimensional space to a low-dimensional space is not isometric, this parameterization will cause distortion. The distortion of this n-dimensional embedding can be calculated as the sum of the geodesic distance distortions at all vertices. For each vertex i, its geodesic distance distortion (GDD) under the embedding is defined as shown in the formula:

[0323] where, are the n-dimensional embedding coordinates of vertex i, and d geo (i, j) is the geodesic distance between point i and point j.

[0324] When n = 2, the surface spectral analysis produces a surface parameterization that minimizes the sum of the squares of GDD for all vertices.

[0325] It should be noted that although the Isomap algorithm calculates the geodesic distance along the manifold, for some non-manifold cases in the input three-dimensional mesh, this scheme will perform corresponding preprocessing to eliminate the existence of these non-manifolds.

[0326] Since the three-dimensional space to two-dimensional space is not isometric, the parameterization will cause distortion. The distortion can be measured in many ways, including the preservation of angles or regions, or how much the parameter distance is stretched or shrunk on the surface. What this algorithm focuses on is the distance distortion, especially the definition of geometric stretching, which defines the average stretching L of the local distance on the surface 2 and the stretching L in the worst case ∞ Two measures. Assume a triangle T with two-dimensional texture coordinates p1, p2, p3, where p i =(s i , ti ), the corresponding three-dimensional coordinates are expressed as q1, q2, q3, and the calculation process of the affine mapping S(p) = S(s, t) = q is shown as follows:

[0327] in,<a,b,c> Represents the area of ​​triangle abc. Since the mapping is affine, its partial derivative is constant on (s, t), and its calculation process is shown as follows:

[0328] Where A=<p1,p2,p3> =((s2-s1)(t3-t1)-(s3-s1)(t2-t1)) / 2

[0329] Then calculate the Jacobian matrix [S S ,S t The calculation process of the larger and smaller singular values ​​of ] is as follows:

[0330] Where a = S s ·S s ,b=S s ·S t ,c=S t ·S t . Singular value γ max , γ min Represents the maximum and minimum lengths obtained when a unit length vector is mapped from a two-dimensional texture domain to a three-dimensional surface, that is, the maximum and minimum local "stretching". The two stretching measures on triangle T are defined as follows: L ∞ (T) = γ max (12)

[0331] In the entire three-dimensional grid M={T i The definition of the upper stretch measure is as shown in the formula:

[0332] Among them, A ′ (T i ) is a triangle T i The surface area in three-dimensional space.

[0333] Due to L ∞ Relying only on one worst-case point in the domain, L ∞ Stretching is difficult to control by any method, but after a few L 2 Iterations of stretch minimization can significantly improve the results.

[0334] If the parameterization generated by spectral analysis fails to meet the stretch threshold, it will be divided into smaller charts. Since the global features of the model correspond to larger eigenvalues, they are used for division. The results of spectral analysis are used to calculate several representative vertices, and then charts are grown simultaneously around these representative points. This method is called surface spectral clustering. The specific algorithm process is as follows:

[0335] a) Sort the eigenvalues ​​and corresponding eigenvectors from spectral analysis from large to small, i.e. λ1≥λ2≥...≥λ N .

[0336] b) Get λ n / λ n+1 Maximize the top n eigenvalues ​​and eigenvectors (n≤10).

[0337] c) For each vertex i in the 3D mesh, calculate its n-dimensional embedding coordinates

[0338] d) For each of the n embedded coordinates, find the two points with the largest and smallest coordinates and set them as 2n representative points.

[0339] e) Remove those representative points that are too close to each other, generating m ≤ 2n representative points (using a distance threshold of 10 times the average edge length of the input grid).

[0340] f) Using the geodesic distances calculated in the surface spectrum analysis, the 3D mesh is divided into m parts by simultaneously growing charts around the representative points. Each triangle is assigned to the chart with the closest representative point to the triangle (the geodesic distance from the triangle to the representative point is calculated as the average of the geodesic distances from the three vertices of the triangle to the representative vertex).

[0341] After splitting the charts, a graph cut algorithm is used to optimize the boundaries between them. Chart boundaries should meet two objectives: 1) they should pass through areas of high curvature without being too jagged, and 2) they should minimize the embedding distortion of the bounding charts. This algorithm formulates the optimal boundary problem as a grap cutting problem. For simplicity, the following discusses the binary case of splitting the surface in two. When subdividing into more than two charts, each pair of adjacent charts is considered sequentially.

[0342] Suppose we are looking for an optimal boundary between charts A and B. The initial partition is generated using surface spectral clustering. We then generate an intermediate region C by extending a region to both sides of the initial partition boundary. The size of the intermediate region is proportional to the total area of ​​the unpeeled patches. We now construct an undirected flow network graph from C using an extension of the method in the graph cut algorithm. Here, we transform two adjacent triangles f in the graph cut algorithm into i and f j The definition of "capacity" between is modified as shown in the formula: c(f i ,f j )=αc ang (f i ,f j )+(1-α)c distort (f i ,f j ) (15)

[0343] The first term in Equation (15) corresponds to the first goal of non-ragged cutting along edges with high dihedral angles. The calculation process is shown in the following equation:

[0344] Among them, d ang (f I ,f j )=1-cosα ij , α ij is a triangle f i and f j The angle between normals, avg(d ang ) is the average angular distance between adjacent triangles.

[0345] The second term in (15) measures the embedding distortion, which is calculated as follows: d distort (f i ,f j )=|GDD A (f i )-GDD B (f i )|+|GDD A (f j )-GDD B (f j )| (18)

[0346] Among them, GDD A (f i ) and GDD B (f i ) are triangles f iGDD embedded under the induction of chart A or chart B, avg(d distort ) is the d on all pairs of adjacent triangles distort (f I ,f j ) average. c distort (f I ,f j ) prefers that its neighboring triangles balance the boundary edges of the GDD between the embeddings determined by chart A and chart B. In other words, the cut should avoid placing triangles on the wrong side, which would produce unnecessary deformation.

[0347] The weight parameter α in formula (15) is a trade-off between the above two objectives.

[0348] This stretch-driven charting and parameterization algorithm is expensive to implement in its simplest form, especially as the number of model vertices increases. Therefore, to speed up computation, the Iso-charts algorithm uses the Landmark Isomap algorithm, an extension of the Isomap algorithm. The Landmark Isomap algorithm is also used during boundary optimization to calculate the embedding coordinates of vertices in the intermediate region, further minimizing embedding distortion.

[0349] Finally, the charts generated by the above process are packed into the 2D texture domain using the chart packing algorithm used in the MCGIM algorithm. Finally, a 3D mesh with regenerated UV coordinates is obtained.

[0350] The orthogonal projection algorithm is a projection-based mesh parameterization method that generates texture coordinates for the mesh through orthogonal projection. Its main process includes:

[0351] a) Calculate mesh properties, including the neighboring faces of each face and the area and normal vector of each face;

[0352] b) Determine the projection plane of each face based on the normal vector;

[0353] c) Start clustering all faces according to the projection plane to form a connected region, first selecting the starting face of the cluster;

[0354] d) Iterating from the starting face, determining whether adjacent faces of the face added to the connected region can be added to the connected region;

[0355] e) After each connected region is iterated, multiple connected regions are obtained;

[0356] f) determining whether to merge adjacent connected regions based on the error metric;

[0357] g) Check whether there are overlapping areas during projection, remove the overlapping areas and regenerate connected areas;

[0358] h) Arrange all the projected regions into a two-dimensional image.

[0359] This stage first calculates some properties of the mesh, including the adjacency list of the triangles and the normal vectors and areas of the triangles; triangle adjacency can be defined as sharing a single vertex or sharing an edge; the normal vector and area of ​​the triangle can be derived from the triangles after its projection (later used to calculate the distortion after projection).

[0360] The triangles are then clustered, and the area formed by the same type of triangles is called a connected area. There are two methods for selecting the starting point of the clustering algorithm: The projection direction is obtained based on the statistical results of all face normal vectors. The triangle face whose normal vector is closest to the projection direction can be selected as the starting point of the clustering algorithm and it has not yet been added to the connected area.

[0361] Select the first available face as the starting point of the clustering algorithm (that is, the face has not yet been added to the connected area), and determine whether to add adjacent faces according to the criteria.

[0362] Added to connected regions; criteria include:

[0363] ■Has the same projection direction as the connected region;

[0364] The angle between the normal vector of the last face added to the connected region and the normal vector of the last face added to the connected region is less than the threshold;

[0365] ■A threshold value where the area is less than the total area;

[0366] ■The number of triangles is less than the threshold;

[0367] After clustering is completed, there may be many small partitions that need to be merged. When merging, a cost function is used to determine whether two partitions can be merged. The function is defined by the perimeter and stretch degree of the connected area. The detailed steps are as follows:

[0368] First, the connected regions are sorted in a specific order. The sorting criteria can be based on the number of triangles contained and the proximity between the average normal vector of the connected region and the corresponding projection direction. The cost function of each connected region is COST(M i )=L 2 (M i )+λPERIMETER(M i ); Then, take the first connected region after sorting each time, obtain the connected regions adjacent to the connected region to form a candidate list, and then for each candidate region, first calculate the cost function after the two regions are merged: COST(M i +M j )=L 2 (M i +M j )+λPERIMETER(M i +M j ), for all (COST(M i +M j ) <COST(M i )+COST(M j )) area, select the area that minimizes the cost function, merge the extracted areas into this area and update the ranking of connected areas;

[0369] The stretching function in the cost function measures the degree of distortion after projection, which is obtained by calculating the area of ​​the triangle after projection in each direction.

[0370] Finally, we need to check whether there are overlapping vertices or faces in each connected region, splitting the overlapping regions from the connected regions to form new connected regions. Detecting overlapping regions means determining whether the currently projected face overlaps with the projected region. If so, the current face is removed from the connected region, and this face and the other removed faces form a new connected region. Finally, within the removed connected regions, we check whether there are disconnected regions and split these disconnected regions into multiple connected regions.

[0371] After the above steps, the grid is divided into multiple partitions, and the result of each partition after projection has been determined. The two-dimensional arrangement step is to arrange the projected partitions onto a texture map of a predetermined size. This step will first determine the overall scaling factor of the current frame, thereby scaling the vertex coordinates to ensure that all partitions of the current grid can occupy the texture map to the maximum extent; then sort each partition from large to small according to the projected area, and then calculate the width and height of the bounding box of each partition after being projected onto the two-dimensional image; then start pixel-by-pixel traversal, taking the current pixel as the position of the upper left corner of the bounding box after the current partition is projected onto the two-dimensional plane, and then determine whether the current partition can be placed at this position. If it can be placed, its occupied area will be marked as occupied, and other partitions cannot overlap with the occupied area; traverse all pixels to determine the position of each partition projection, thereby determining the transformation scheme of the texture coordinates of each partition vertex.

[0372] (4) Texture coordinate encoding

[0373] The texture coordinate encoding is used to encode the texture coordinates of the basic mesh generated by the mesh parameterization module to obtain a texture coordinate code stream and a basic mesh for reconstructing the texture coordinates.

[0374] In both algorithms, the mesh is partitioned, and each partition is independently parameterized. After obtaining the parameterized results for each partition, they are arranged into a texture map of a preset size according to specific rules. The partition information in the mesh parameterization is the partition index that identifies each face in the mesh. The partition arrangement information is the position of each partition in the 2D texture map, which mainly includes the following information:

[0375] ●The parameterization method of the current partition; for the isochart algorithm, the parameterization method of each partition is the same; for the orthogonal projection algorithm, this information identifies the projection plane corresponding to each partition.

[0376] ●The angle index of the current partition when it is arranged on the image;

[0377] ● The position posX, posY of the upper left corner of the bounding box of the current partition when it is arranged on the image;

[0378] ●The width and height (sizeX and sizeY) of the bounding box of the current partition when it is arranged on the image;

[0379] ●Whether to scale the current partition arrangement and the corresponding scaling factor;

[0380] The parameterized partition information of the current base grid and the arrangement information of each partition are also transmitted to the decoding end as a texture coordinate code stream, and the decoder recovers the texture coordinates corresponding to the base grid based on this information.

[0381] (5) Displacement processing and encoding

[0382] The base mesh compression module compresses and reconstructs the base mesh, and calculates the corresponding vertex displacement using the reconstructed base mesh and the subdivided deformed mesh obtained in the subdivision deformation step.

[0383] Displacement encoding can be implemented in a variety of ways. One approach involves transforming the coordinate system of the displacements. Specifically, the coordinate system of each vertex's displacement is converted to a coordinate system constructed using the vertex's normal vector and two components tangent to that normal vector. The displacements are then transformed using techniques such as wavelet transforms. The transformed coefficients are quantized and arranged in the image in scan order, and video encoding is applied to the image. Alternatively, entropy coding can be used to directly encode the generated or processed displacements.

[0384] (6) Reconstruct the deformed mesh

[0385] Because the displacement encoding phase causes some loss in the quantization process, the displacement needs to be reconstructed on the encoder side to keep it consistent with the decoder side. After obtaining the reconstructed displacement, the reconstructed base mesh is subdivided and the reconstructed subdivided mesh is obtained based on the corresponding displacement.

[0386] (7) Texture image conversion and encoding

[0387] The texture map encoding module first performs texture map conversion based on the input original mesh, the input original texture map, and the subdivided and deformed mesh. FIG15 is a schematic diagram of the texture map conversion proposed in an embodiment of the present application. As shown in FIG15 , the steps of texture map conversion are as follows:

[0388] a) Calculate the texture coordinates of each pixel on the texture map to be generated, for example, the texture coordinates corresponding to pixel A(i,j) are P(u,v).

[0389] b) Determine whether the texture coordinate is within a certain triangular face after the parameterization of the subdivided deformed mesh.

[0390] c) If the texture coordinate does not belong to any triangle, the pixel is marked as an empty pixel and can be filled with a filling algorithm.

[0391] d) If the texture coordinate belongs to a triangle, then

[0392] ●Mark the pixel as filled.

[0393] ●Calculate the center of gravity coordinates of the texture in the current triangle based on the texture coordinates.

[0394] ●Based on the barycentric coordinates and the corresponding triangular face, the two-dimensional texture coordinates are mapped to three-dimensional geometric coordinates, that is, mapped to the points on the subdivided deformed grid corresponding to the texture coordinates, as shown by M(x,y,z) in the figure.

[0395] ●Find the point on the input original grid that is closest to the three-dimensional coordinate, as shown in the figure as M'(x,y,z).

[0396] ●Calculate the centroid coordinates of the three-dimensional coordinates based on the triangle surface they are on and map them to two dimensions to calculate their texture coordinates, namely P'(u',v').

[0397] ●Sampling is performed on the input original texture map using the texture coordinates to obtain the value A'(i',j') of the corresponding pixel position.

[0398] ●Assign this value to the corresponding pixel A(i,j) on the texture map to be generated.

[0399] For empty pixels, existing filling algorithms (such as Push-Pull algorithm) can be used to fill these empty pixels.

[0400] Furthermore, in an embodiment of the present application, at the decoding end, after the auxiliary information is decoded, it guides each module to decode according to the method preset by the encoding end. The basic grid code stream is decoded using the basic grid decoder corresponding to the encoding end, and the texture coordinates corresponding to the basic grid are reconstructed according to the texture coordinate information in the texture coordinate code stream. The displacement code stream is decoded by the displacement decoding module, and the displacement is reconstructed, including inverse transformation, inverse quantization and other steps. The reconstructed displacement is then applied to the subdivided basic grid to obtain a reconstructed deformed grid. The texture map code stream is decoded by the texture map to obtain a reconstructed texture map. Next, the main modules of the three-dimensional grid encoding and decoding framework proposed in this patent are described in detail.

[0401] (1) Basic grid decoding

[0402] The basic grid decoding module selects a corresponding basic grid decoder according to the basic grid encoder identifier indicated by the auxiliary information to decode the basic grid code stream to obtain a reconstructed basic grid.

[0403] (2) Texture coordinate reconstruction

[0404] The texture coordinate reconstruction step reconstructs the texture coordinates corresponding to the current base grid based on the partition information and partition arrangement information of the encoding end in the texture coordinate bitstream. The corresponding syntax elements obtained by decoding are as follows:

[0405] base_mesh_texcoord_information() represents the texture coordinate information in the base mesh code stream;

[0406] base_mesh_parameterazation_method represents the parameterization scheme used by the base mesh. 0 represents the use of the isochart algorithm, 1 represents the use of the orthogonal projection algorithm, and the default value is 0.

[0407] base_mesh_uv_charts_number_minus1 means the number of partitions after parameterization minus 1;

[0408] uv_chart_parameterazation_type[i] represents the projection plane corresponding to the parameterization of the i-th partition. This value is only valid when the parameterization scheme is orthogonal projection parameterization;

[0409] uv_chart_orientation_id[i] represents the rotation angle index when the i-th partition is arranged;

[0410] uv_chart_posX[i] represents the position of the horizontal coordinate of the upper left corner of the partition bounding box in the two-dimensional texture map when the i-th partition is arranged;

[0411] uv_chart_posY[i] represents the position of the ordinate of the upper left corner of the partition bounding box in the two-dimensional texture map when the i-th partition is arranged;

[0412] uv_chart_sizeX[i] represents the width of the partition bounding box when the i-th partition is arranged;

[0413] uv_chart_sizeY[i] represents the height of the partition bounding box when the i-th partition is arranged;

[0414] uv_chart_scale_flag[i] indicates whether the i-th partition is scaled when arranged;

[0415] uv_chart_scale_factor[i] represents the scaling factor of the i-th partition.

[0416] First, the decoded base mesh is partitioned according to the partition information. The texture coordinates of each partition before arrangement are obtained according to the parameterization method in the partition arrangement information. Then, the texture coordinates of each partition are converted according to the arrangement information of each partition to obtain the final reconstructed texture coordinates. The conversion formula is as shown in (2) above.

[0417] Where width and height represent the width and height of the texture image, the top-left corner of the current partition's bounding box is posX(i) and posY(i), R(i) is a matrix defined by the rotation angle and the bounding box's width and height, and scale(i) is the scaling factor for the current partition.

[0418] (3) Displacement decoding and reconstruction

[0419] The displacement code stream is decoded by a displacement decoder. If the encoder compressed the displacement using video coding, the decoder decodes it using the corresponding video decoder and restores it from the 2D image in the corresponding order according to the permutation scheme. Then, inverse transformation and dequantization are performed to restore the displacement consistent with the encoder. If entropy coding is used, entropy decoding is performed directly, followed by subsequent reconstruction steps such as inverse transformation and dequantization.

[0420] (4) Deformed mesh reconstruction

[0421] The deformed mesh reconstruction step is to sequentially add the reconstructed displacement to each vertex of the subdivided base mesh to obtain the reconstructed deformed mesh, which is the mesh finally output by the decoding end.

[0422] (5) Texture map decoding

[0423] The texture decoder is responsible for decoding the texture stream. The texture is decoded by the video decoder. An optional color space conversion is performed on it to obtain an image format consistent with the input texture of the encoder, and the final decoded texture output is obtained.

[0424] It can be seen that the encoding and decoding scheme proposed in the embodiment of the present application may include a three-dimensional grid encoding and decoding framework that can regenerate texture coordinates at the decoding end; it may also include a grammatical result and a decoding scheme for regenerating texture coordinates at the decoding end.

[0425] Furthermore, the encoding and decoding scheme proposed in the embodiment of the present application can also be applied to the extended framework of VDMC, and its encoding end is shown in Figure 13, wherein the position of the subdivision deformation module has changed, and the input of the subdivision deformation module is the basic grid after grid cleaning. This method can obtain a subdivision deformation grid that is more matched with the reconstructed basic grid, and the other modules are the same as the original framework.

[0426] Accordingly, the corresponding decoding framework is shown in FIG9 , wherein the base grid needs to be cleaned before the reconstructed base grid is subdivided.

[0427] The embodiment of the present application provides a coding and decoding method, and the codec can determine the texture coordinates based on the partition information, the partition arrangement information and the reconstructed basic grid, so that the reconstructed grid of the current image can be obtained based on the texture coordinates. Specifically, the reconstructed subdivision grid of the current image can be determined based on the basic grid (the first basic grid with texture coordinates); then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid. That is to say, in the embodiment of the present application, at the encoding end, the reconstructed basic grid written into the code stream does not have texture coordinates, that is, the reconstructed basic grid obtained by the decoding end does not have texture coordinates, and the decoding end can complete the parameterization of the reconstructed basic grid at the decoding end based on the partition information and partition arrangement information corresponding to the encoding end transmitted in the code stream, that is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation result of the preprocessing stage corresponds to the result of the reconstruction stage, thereby improving the coding and decoding efficiency and improving the compression performance.

[0428] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, FIG16 is a schematic diagram of a first structure of an encoder proposed in an embodiment of the present application. As shown in FIG16 , the encoder 100 may include: a first determining unit 111; wherein,

[0429] The first determination unit 111 is configured to determine a base grid of the current image and a reconstructed base grid, and write the reconstructed base grid into a bitstream; determine texture coordinates based on the partition information, the partition arrangement information, and the reconstructed base grid, so as to determine the reconstructed grid of the current image based on the texture coordinates; and write the partition information and the partition arrangement information into the bitstream.

[0430] It should be noted that, in the embodiment of the present application, the encoder 100 can also be regarded as a data processing mode (or "entropy encoder"), which is used to encode the values ​​of the syntax elements to be encoded.

[0431] It is understood that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular device. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0432] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0433] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 100. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the encoding method described in any one of the aforementioned embodiments.

[0434] Based on the composition of the above-mentioned encoder 100 and the computer-readable storage medium, Figure 17 is a second schematic diagram of the composition structure of the encoder proposed in an embodiment of the present application. As shown in Figure 17, the encoder 100 may include: a first memory 121 and a first processor 122, a first communication interface 123 and a first bus system 124. The first memory 121, the first processor 122, and the first communication interface 123 are coupled together through the first bus system 124. It can be understood that the first bus system 124 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 124 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are marked as the first bus system 124 in the figure. Among them,

[0435] The first communication interface 123 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0436] The first memory 121 is used to store computer programs that can be run on the first processor;

[0437] The first processor 122 is configured to, when running the computer program, determine a base grid of the current image and a reconstructed base grid, and write the reconstructed base grid into a bitstream; determine texture coordinates based on the partition information, the partition arrangement information, and the reconstructed base grid, so as to determine the reconstructed grid of the current image based on the texture coordinates; and write the partition information and the partition arrangement information into the bitstream.

[0438] It is understood that the first memory 121 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 121 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0439] The first processor 122 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 122. The above-mentioned first processor 122 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 121 , and the first processor 122 reads the information in the first memory 121 and completes the steps of the above method in combination with its hardware.

[0440] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0441] Optionally, as another embodiment, the first processor 122 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0442] The present embodiment provides an encoder that can determine texture coordinates based on partition information, partition arrangement information, and a reconstructed base grid, thereby obtaining a reconstructed grid of the current image based on the texture coordinates. Specifically, the reconstructed subdivision grid of the current image can be determined based on the base grid (the first base grid with texture coordinates); then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid. That is, in an embodiment of the present application, at the encoding end, the reconstructed base grid written into the bitstream does not have texture coordinates, that is, the reconstructed base grid obtained by the decoding end does not have texture coordinates, and the decoding end can complete the parameterization of the reconstructed base grid at the decoding end based on the partition information and partition arrangement information corresponding to the encoding end transmitted in the bitstream, that is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation result of the preprocessing stage corresponds to the result of the reconstruction stage, thereby improving the encoding and decoding efficiency and improving the compression performance.

[0443] In yet another embodiment of the present application, based on the same inventive concept as the above embodiment, FIG18 is a schematic diagram of the first structure of a decoder proposed in an embodiment of the present application. As shown in FIG18 , the decoder 200 may include: a second determining unit 211; wherein,

[0444] The second determination unit 211 is configured to decode the code stream, determine the reconstructed basic grid, partition information and partition arrangement information of the current image; determine the texture coordinates based on the partition information, the partition arrangement information and the reconstructed basic grid, so as to determine the reconstructed grid of the current image based on the texture coordinates.

[0445] It should be noted that, in the embodiment of the present application, the decoder 200 can also be regarded as a data processing mode (or "entropy decoder"), which is used to decode the values ​​of the syntax elements to be decoded.

[0446] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0447] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium, which is applied to the decoder 200 and stores a computer program. When the computer program is executed by the second processor, it implements any of the methods in the aforementioned embodiments.

[0448] Based on the composition of the above-mentioned decoder 200 and the computer-readable storage medium, Figure 19 is a second schematic diagram of the composition structure of the decoder proposed in an embodiment of the present application. As shown in Figure 19, the decoder 200 may include: a second memory 221 and a second processor 222, a second communication interface 223 and a second bus system 224. The second memory 221 and the second processor 222, and the second communication interface 223 are coupled together through the second bus system 224. It can be understood that the second bus system 224 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 224 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as the second bus system 224 in the figure. Among them,

[0449] The second communication interface 223 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0450] The second memory 221 is used to store computer programs that can be run on the second processor;

[0451] The second processor 222 is used to decode the code stream when running the computer program, determine the reconstructed basic grid, partition information and partition arrangement information of the current image; determine the texture coordinates based on the partition information, the partition arrangement information and the reconstructed basic grid, so as to determine the reconstructed grid of the current image based on the texture coordinates.

[0452] It is understood that the second memory 221 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The second memory 221 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0453] The second processor 222 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the second processor 222. The above-mentioned second processor 222 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the second memory 221 , and the second processor 222 reads the information in the second memory 221 and completes the steps of the above method in combination with its hardware.

[0454] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0455] Optionally, as another embodiment, the second processor 222 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0456] The present embodiment provides a decoder that can determine texture coordinates based on partition information, partition arrangement information, and a reconstructed base grid, thereby obtaining a reconstructed grid of the current image based on the texture coordinates. Specifically, the reconstructed subdivision grid of the current image can be determined based on the base grid (the first base grid with texture coordinates); then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid. That is, in an embodiment of the present application, at the encoding end, the reconstructed base grid written into the bitstream does not have texture coordinates, that is, the reconstructed base grid obtained by the decoding end does not have texture coordinates, and the decoding end can complete the parameterization of the reconstructed base grid at the decoding end based on the partition information and partition arrangement information corresponding to the encoding end transmitted in the bitstream, that is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation result of the preprocessing stage corresponds to the result of the reconstruction stage, thereby improving the encoding and decoding efficiency and improving the compression performance.

[0457] Furthermore, an embodiment of the present application also proposes a code stream, wherein the code stream is generated by bit encoding based on the information to be encoded; wherein the information to be encoded includes at least: a reconstructed basic grid, displacement information, partition information, partition arrangement information, subdivision method, number of subdivision iterations, and a texture map.

[0458] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0459] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0460] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0461] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0462] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0463] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0464] The embodiment of the present application provides a coding and decoding method, a code stream, an encoder, a decoder and a storage medium. The codec can determine the texture coordinates based on the partition information, the partition arrangement information and the reconstructed basic grid, so that the reconstructed grid of the current image can be obtained based on the texture coordinates. Specifically, the reconstructed subdivision grid of the current image can be determined based on the basic grid (the first basic grid with texture coordinates); then the reconstructed grid of the current image can be determined based on the reconstructed subdivision grid. That is to say, in the embodiment of the present application, at the encoding end, the reconstructed basic grid written into the code stream does not have texture coordinates, that is, the reconstructed basic grid obtained by the decoding end does not have texture coordinates, and the decoding end can complete the parameterization of the reconstructed basic grid at the decoding end based on the partition information and partition arrangement information corresponding to the encoding end transmitted in the code stream, that is, the parameterization information of the encoding end can be used to guide the parameterization processing of the decoding end, thereby ensuring the consistency of the parameterization process, so that the calculation result of the preprocessing stage corresponds to the result of the reconstruction stage, thereby improving the coding and decoding efficiency and improving the compression performance.

Claims

1. A decoding method, applied to a decoder, the method comprising: Decoding a bitstream, determining a reconstructed base grid, partition information, and partition arrangement information of a current image; Determining texture coordinates based on the partition information, the partition arrangement information, and the reconstructed base grid, to determine a reconstructed grid of the current image based on the texture coordinates.

2. The method according to claim 1, wherein, The method further comprises: Performing reconstruction based on the texture coordinates to obtain a first base grid; Determining a reconstructed subdivision grid of the current image according to the first base grid; Determining a reconstructed grid of the current image according to the reconstructed subdivision grid.

3. The method according to claim 2, wherein, The determining texture coordinates based on the partition information, the partition arrangement information, and the reconstructed base grid comprises: Determining the texture coordinates corresponding to the reconstructed base grid according to the partition information and the partition arrangement information.

4. The method according to claim 3, wherein, The determining the texture coordinates corresponding to the reconstructed base grid according to the partition information and the partition arrangement information comprises: Partitioning the reconstructed base grid according to the partition information to determine at least one partition corresponding to the reconstructed base grid; Determining the texture coordinates of the at least one partition according to the partition arrangement information; Arranging the texture coordinates of the at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed base grid.

5. The method according to claim 4, wherein The partition arrangement information includes a parameterization scheme identification parameter, and the determining the texture coordinates of the at least one partition according to the partition arrangement information comprises: When the parameterization scheme identification parameter indicates that the current image uses a first parameterization scheme, parameterizing all partitions in a first parameterization manner to determine the texture coordinates of the at least one partition.

6. The method according to claim 4, wherein, The partition arrangement information includes a parameterization scheme identification parameter and a second parameterization manner corresponding to the current partition, and the determining the texture coordinates of the at least one partition according to the partition arrangement information comprises: When the parameterization scheme identification parameter indicates that the current image uses a second parameterization scheme, parameterizing the current partition according to the second parameterization manner to determine the texture coordinates of the current partition.

7. The method according to any one of claims 4 - 6, wherein The partition arrangement information includes a bounding box position parameter, an angle index, a bounding box size parameter, and a scaling parameter corresponding to the current partition; wherein the scaling parameter includes a scaling identification parameter and a scaling factor.

8. The method according to claim 7, wherein The arranging the texture coordinates of the at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed base grid comprises: When the scaling identification parameter indicates that the current partition is not scaled, converting the texture coordinates of the current partition according to the bounding box position parameter, the angle index, and the bounding box size parameter to determine the arranged texture coordinates corresponding to the current partition, so as to determine the texture coordinates corresponding to the reconstructed base grid.

9. The method according to claim 7, wherein The arranging the texture coordinates of the at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed base grid comprises: When the scaling identification parameter indicates that the current partition is to be scaled, determine the scaling factor; Convert the texture coordinates of the current partition according to the bounding box position parameter, the angle index, the bounding box size parameter, and the scaling factor to determine the arranged texture coordinates corresponding to the current partition, so as to determine the texture coordinates corresponding to the reconstructed basic mesh.

10. The method according to claim 2, wherein The determining the reconstructed subdivision mesh of the current image according to the first basic mesh includes: Subdivide the first basic mesh to determine the reconstructed subdivision mesh.

11. The method according to claim 2, wherein, The determining the reconstructed subdivision mesh of the current image according to the first basic mesh includes: Perform mesh cleaning on the first basic mesh to determine the cleaned mesh; Subdivide the cleaned mesh to determine the reconstructed subdivision mesh.

12. The method according to claim 2, wherein, The determining the reconstructed mesh of the current image according to the reconstructed subdivision mesh includes: Determine the displacement information of the reconstruction of the current image; Determine the reconstructed mesh of the current image according to the reconstructed subdivision mesh and the displacement information of the reconstruction.

13. The method according to claim 12, wherein, The determining the displacement information of the reconstruction of the current image includes: Decode the bitstream to determine the displacement information corresponding to the current image; Reconstruct the displacement information to determine the displacement information of the reconstruction.

14. The method according to claim 12 or 13, wherein, The determining the reconstructed mesh of the current image according to the reconstructed subdivision mesh and the displacement information of the reconstruction includes: Perform corresponding displacement operations on at least one vertex of the reconstructed subdivision mesh according to the displacement information of the reconstruction to determine the reconstructed mesh of the current image.

15. The method according to claim 10, wherein The subdividing the first basic mesh to determine the reconstructed subdivision mesh includes: Determine the first mesh subdivision parameter of the current image; Iteratively divide the first basic mesh according to the first mesh subdivision parameter to determine the reconstructed subdivision mesh.

16. The method according to claim 11, wherein, The subdividing the cleaned mesh to determine the reconstructed subdivision mesh includes: Determine the second mesh subdivision parameter of the current image; Iteratively divide the cleaned mesh according to the second mesh subdivision parameter to determine the reconstructed subdivision mesh.

17. The method according to claim 15, wherein, The first mesh subdivision parameter includes: the first subdivision method and the first subdivision iteration times, and the method further includes: Decode the bitstream to determine the first subdivision method and the first subdivision iteration times.

18. The method according to claim 16, wherein, The second mesh subdivision parameter includes: the second subdivision method and the second subdivision iteration times, and the method further includes: Decode the bitstream to determine the second subdivision method and the second subdivision iteration times.

19. The method according to claim 1, wherein The method further includes: Decode the bitstream to determine the texture map of the current image.

20. The method according to claim 19, wherein, The method further includes: Perform image application based on the reconstructed mesh and the texture map.

21. An encoding method, applied to an encoder, the method includes: Determine the basic mesh and the reconstructed basic mesh of the current image, and write the reconstructed basic mesh into the bitstream; Determine texture coordinates based on the partition information, the partition arrangement information, and the reconstructed basic grid, so as to determine the reconstructed grid of the current image based on the texture coordinates; and write the partition information and the partition arrangement information into the bitstream.

22. The method according to claim 21, wherein The method further includes: Perform reconstruction based on the texture coordinates to obtain a first basic grid; Determine the reconstructed subdivision grid of the current image according to the first basic grid; Determine the reconstructed grid of the current image according to the reconstructed subdivision grid.

23. The method according to claim 22, wherein, The determining the texture coordinates according to the partition information, the partition arrangement information, and the reconstructed basic grid includes: Determine the texture coordinates corresponding to the reconstructed basic grid according to the partition information and the partition arrangement information.

24. The method according to claim 23, wherein, The determining the texture coordinates corresponding to the reconstructed basic grid according to the partition information and the partition arrangement information includes: Partition the reconstructed basic grid according to the partition information to determine at least one partition corresponding to the reconstructed basic grid; Determine the texture coordinates of the at least one partition according to the partition arrangement information; Arrange the texture coordinates of the at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed basic grid.

25. The method according to claim 24, wherein, The partition arrangement information includes a parameterization scheme identification parameter, and the determining the texture coordinates of the at least one partition according to the partition arrangement information includes: When the parameterization scheme identification parameter indicates that the first parameterization scheme is used for the current image, parameterize all partitions according to the first parameterization method to determine the texture coordinates of the at least one partition.

26. The method according to claim 24, wherein, The partition arrangement information includes a parameterization scheme identification parameter and a second parameterization method corresponding to the current partition. The determining the texture coordinates of the at least one partition according to the partition arrangement information includes: When the parameterization scheme identification parameter indicates that the second parameterization scheme is used for the current image, parameterize the current partition according to the second parameterization method to determine the texture coordinates of the current partition.

27. The method according to any one of claims 24-26, wherein The partition arrangement information includes a bounding box position parameter, an angle index, a bounding box size parameter, and a scaling parameter corresponding to the current partition; wherein, the scaling parameter includes a scaling identification parameter and a scaling factor.

28. The method according to claim 27, wherein, The arranging the texture coordinates of the at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed basic grid includes: When it is determined that the current partition is not scaled, convert the texture coordinates of the current partition according to the bounding box position parameter, the angle index, and the bounding box size parameter to determine the arranged texture coordinates corresponding to the current partition, so as to determine the texture coordinates corresponding to the reconstructed basic grid; Set the scaling identification parameter to indicate that the current partition is not scaled.

29. The method according to claim 27, wherein, The arranging the texture coordinates of the at least one partition according to the partition arrangement information to determine the texture coordinates corresponding to the reconstructed basic grid includes: When it is determined that the current partition is scaled, determine the scaling factor; Convert the texture coordinates of the current partition according to the bounding box position parameter, the angle index, the bounding box size parameter, and the scaling factor to determine the rearranged texture coordinates corresponding to the current partition, so as to determine the texture coordinates corresponding to the reconstructed base mesh; Set the scaling flag parameter to indicate that the current partition is scaled.

30. The method according to claim 22, wherein, The determining the reconstructed subdivision mesh of the current image according to the first base mesh includes: Subdivide the first base mesh to determine the reconstructed subdivision mesh.

31. The method according to claim 22, wherein, The determining the reconstructed subdivision mesh of the current image according to the first base mesh includes: Perform mesh cleaning on the first base mesh to determine the cleaned mesh; Subdivide the cleaned mesh to determine the reconstructed subdivision mesh.

32. [Corrected according to Rule 26 on 23.01.2024] The method according to claim 22, wherein, The determining the reconstructed mesh of the current image according to the reconstructed subdivision mesh includes: Determine the displacement information of the reconstruction of the current image; Determine the reconstructed mesh of the current image according to the reconstructed subdivision mesh and the displacement information of the reconstruction.

33. The method according to claim 32, wherein, The determining the displacement information of the reconstruction of the current image includes: Determine the displacement information corresponding to the current image and write the displacement information into the bitstream; Reconstruct the displacement information to determine the displacement information of the reconstruction.

34. The method according to claim 33, wherein, The determining the displacement information corresponding to the current image includes: Determine the subdivision network of the current image; Determine the displacement information based on the subdivision mesh.

35. The method according to claim 34, wherein, The determining the subdivision network of the current image includes: Subdivide the base mesh to determine the subdivision mesh of the current image.

36. The method according to claim 35, wherein, The subdividing the base mesh to determine the subdivision mesh of the current image includes: Determine the third mesh subdivision parameter of the current image; Iteratively divide the base mesh according to the third mesh subdivision parameter to determine the subdivision mesh of the current image.

37. The method according to any one of claims 24 - 36, wherein, The determining the displacement information based on the subdivision mesh includes: Deform multiple segments of the subdivision mesh to determine the deformed mesh; Perform corresponding displacement calculations on at least one vertex of the deformed mesh and at least one vertex of the subdivision mesh to determine the displacement information of the at least one vertex, so as to determine the displacement information.

38. The method according to claim 30, wherein, The subdividing the first base mesh to determine the reconstructed subdivision mesh includes: Determine the first mesh subdivision parameter of the current image; Iteratively divide the first base mesh according to the first mesh subdivision parameter to determine the reconstructed subdivision mesh.

39. The method according to claim 31, wherein, The subdividing the cleaned mesh to determine the reconstructed subdivision mesh includes: Determine the second mesh subdivision parameter of the current image; Iteratively divide the cleaned mesh according to the second mesh subdivision parameter to determine the reconstructed subdivision mesh.

40. The method according to claim 36, wherein, The third mesh subdivision parameter includes: a third subdivision method and a third subdivision iteration count, and the method further includes: Write the third subdivision method and the third subdivision iteration count into the bitstream.

41. The method according to claim 38, wherein, The first mesh subdivision parameter includes: a first subdivision method and a first subdivision iteration count, and the method further includes: Write the first subdivision method and the first subdivision iteration count into the bitstream.

42. The method according to claim 39, wherein, The second mesh subdivision parameters include: a second subdivision method and a second subdivision iteration count, and the method further includes: Write the second subdivision method and the second subdivision iteration count into the bitstream.

43. The method according to any one of claims 32-36, wherein The determining the reconstructed mesh of the current image according to the reconstructed subdivision mesh and the reconstructed displacement information includes: Performing corresponding displacement operations on at least one vertex of the reconstructed subdivision mesh according to the reconstructed displacement information to determine the reconstructed mesh of the current image.

44. The method according to claim 21, wherein, The determining the base mesh of the current image includes: Determine the input mesh of the current image; Perform downsampling processing on the input mesh to determine the base mesh of the current image.

45. The method according to claim 21, wherein The method further includes: Determine the texture map of the current image according to the reconstructed subdivision mesh of the current image, and write the texture map into the bitstream.

46. A bitstream, wherein, The bitstream is generated by performing bit encoding on the information to be encoded; wherein, the information to be encoded includes at least one of the following: the reconstructed base mesh, displacement information, partition information, partition arrangement information, subdivision method, subdivision iteration count, texture map.

47. An encoder, including a first determination unit, wherein: The first determination unit is configured to determine the base mesh and the reconstructed base mesh of the current image, and write the reconstructed base mesh into the bitstream; Determine a first base mesh according to the partition information, the partition arrangement information, and the reconstructed base mesh, and write the partition information and the partition arrangement information into the bitstream; determine the reconstructed subdivision mesh of the current image according to the first base mesh; determine the reconstructed mesh of the current image according to the reconstructed subdivision mesh.

48. An encoder, including a first memory and a first processor, wherein: The first memory is used to store a computer program that can run on the first processor; The first processor is configured to execute the method according to any one of claims 21 to 45 when running the computer program.

49. A decoder, including a second determination unit, wherein: The second determination unit is configured to decode the bitstream, determine the reconstructed base mesh, partition information, and partition arrangement information of the current image; Determine a first base mesh according to the partition information, the partition arrangement information, and the reconstructed base mesh; determine the reconstructed subdivision mesh of the current image according to the first base mesh; determine the reconstructed mesh of the current image according to the reconstructed subdivision mesh.

50. A decoder, the decoder includes a second memory and a second processor, wherein: The second memory is used to store a computer program that can run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 20 when running the computer program.

51. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program, and when the computer program is executed, it implements the method according to any one of claims 1 to 20, or implements the method according to any one of claims 21 to 45.

Images