Coding method and electronic device

Abstract

The application provides a coding method and an electronic device. The coding method comprises: obtaining data of a 3D mesh, the data of the 3D mesh comprising vertex coordinates of a plurality of vertices; obtaining a scaling parameter corresponding to a current vertex; scaling the vertex coordinates of the current vertex based on the scaling parameter corresponding to the current vertex to obtain scaled vertex coordinates of the current vertex; predicting and compensating based on the scaled vertex coordinates of the current vertex to obtain residual information corresponding to the current vertex; and encoding the residual information corresponding to the current vertex to obtain a bitstream. The scaling parameter corresponding to the current vertex is positively correlated with the importance of the current vertex on the 3D mesh. In this way, for a vertex with low importance, a smaller scaling parameter is used for scaling, which can reduce the code rate overhead. For a vertex with high importance, a larger scaling parameter is used for scaling, which can improve the reconstruction quality.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of coding and decoding, and in particular to a coding and decoding method and an electronic device. BACKGROUND

[0002] Volumetric Video (VV) is different from two-dimensional plane video plus camera operation to simulate three-dimensional video space and panoramic video. VV is a three-dimensional video synthesized by multiple perspectives, which can better restore the three-dimensional world and bring stronger immersion to users. VV is widely used in education, medical treatment, entertainment and other scenarios.

[0003] Dynamic three-dimensional mesh (3D mesh) is a main form of VV. The code rate required for transmitting the data of 3D mesh without compression is large (more than 3 Gbps). Therefore, the data of 3D mesh can be compressed before transmission to reduce the transmission pressure. Generally, the prior art first scales the same scaling multiple for the same frame of 3D mesh, and then compresses the data of the scaled 3D mesh, but the compression performance is poor. SUMMARY

[0004] To solve the above technical problems, the present application provides a coding and decoding method and an electronic device. In the method, the compression performance of the data compression for 3D mesh can be improved.

[0005] In a first aspect, an embodiment of the present application provides an encoding method, which comprises: first, obtaining data of a three-dimensional mesh 3D mesh, the 3D mesh being composed of a plurality of triangular facets, each triangular facet including three vertices, the data of the 3D mesh including vertex coordinates of the plurality of vertices; then, obtaining a scaling parameter corresponding to a current vertex; subsequently, scaling the vertex coordinates of the current vertex based on the scaling parameter corresponding to the current vertex to obtain scaled vertex coordinates of the current vertex; thereafter, predicting and compensating based on the scaled vertex coordinates of the current vertex to obtain residual information corresponding to the current vertex; and encoding the residual information corresponding to the current vertex to obtain a bitstream. Wherein the scaling parameter corresponding to the current vertex is positively correlated with the importance of the current vertex on the 3D mesh, that is, the higher the importance of the current vertex on the 3D mesh, the larger the scaling parameter corresponding to the current vertex; conversely, the lower the importance of the current vertex on the 3D mesh, the smaller the scaling parameter corresponding to the current vertex. In this way, for vertices with low importance, a smaller scaling parameter is used for scaling, which can obtain smaller residuals, and thus fewer bytes are used for encoding; thereby reducing the code rate overhead under certain distortion conditions. For vertices with high importance, a larger scaling parameter is used for scaling, which can obtain larger residuals, and the quantization error in the subsequent encoding process is smaller, thereby improving the reconstruction quality. In the present application, the scaling parameter of each vertex is determined, which can optimize the encoding process of the 3D mesh from the vertex level, thereby achieving better encoding effect.

[0006] Illustratively, the importance of a vertex on the 3D mesh is related to the importance of the region on the 3D mesh where the vertex is located; the importance of the vertex on the 3D mesh can be determined according to the importance of the region where the vertex is located. Wherein the importance of the vertex on the 3D mesh is positively correlated with the importance of the region on the 3D mesh where the vertex is located: that is, the higher the importance of the region on the 3D mesh where the vertex is located, the higher the importance of the vertex on the 3D mesh; conversely, the lower the importance of the region on the 3D mesh where the vertex is located, the lower the importance of the vertex on the 3D mesh. Wherein the importance of different regions in the 3D mesh can be different, for example, the 3D mesh is a 3D mesh of a person, then the importance of the facial region is higher than that of the body region; correspondingly, the scaling parameter of the vertex located in the facial region is greater than that of the vertex located in the body region.

[0007] Illustratively, the scaling parameter can also be referred to as a scaling factor.

[0008] Illustratively, the scaling of the vertex coordinates of the current vertex can be implemented by multiplying the vertex coordinates of the current vertex by the scaling parameter to obtain the scaled vertex coordinates of the current vertex.

[0009] It should be noted that each step of the first aspect is a step in the encoding process. The "encoding" in the first aspect can include multiple steps such as transformation, quantization, and entropy encoding, and the present application does not limit this.

[0010] Exemplarily, the 3D mesh can be a dynamic 3D mesh, and the generation process of the dynamic 3D mesh and the data of the dynamic 3D mesh can be as follows: the multi-path video data (wherein the multi-path video data has different shooting angles) can be angle-synthesized (for example, angle-synthesized by using images with the same timestamp in different paths of video data), and the synthesized video data can be obtained; wherein the synthesized video data includes multiple frames of images. Then, each frame of image of the synthesized video data is subjected to point cloud processing to obtain a 3D point cloud corresponding to each frame of image of the synthesized video data. Subsequently, the 3D point cloud corresponding to each frame of image of the synthesized video data is subjected to mesh processing, and multiple frames of 3D mesh (one frame of 3D mesh corresponds to one frame of image in the synthesized video data) and the data of each frame of 3D mesh can be obtained.

[0011] Exemplarily, the 3D mesh can be a static 3D mesh, and the generation process of the static 3D mesh and the data of the static 3D mesh can be as follows: multiple images (each image corresponds to a different shooting angle) are subjected to angle synthesis, point cloud processing, and mesh processing, and one frame of 3D mesh and the data of the frame of 3D mesh are obtained. The one frame of 3D mesh can be referred to as a static 3D mesh.

[0012] Exemplarily, the data of the 3D mesh can further include texture coordinates, texture maps, and vertex connection relationships.

[0013] Exemplarily, each frame of 3D mesh can include at least one 3D mesh (one 3D mesh corresponds to one target object in one frame of image of the synthesized video data), and each 3D mesh is composed of multiple triangular facets; each triangular facet includes three vertices, and the coordinates of each vertex can be referred to as vertex coordinates, which are three-dimensional coordinates (for example, represented by (x, y, z)).

[0014] Exemplarily, each triangular facet in each 3D mesh has a corresponding texture map; wherein the texture map corresponding to each triangular facet in each 3D mesh can be the same or different, and the present application does not limit this.

[0015] Exemplarily, each vertex of each triangle patch in each 3D mesh can be mapped to a pixel point in a corresponding texture map, and a coordinate of the pixel point is a texture coordinate; wherein, the texture coordinate is a two-dimensional coordinate (e.g., which can be represented by (u, v)).

[0016] Exemplarily, the vertex connection relationship refers to a connection relationship among three vertices included in each triangle patch; wherein, the vertex connection relationship of each 3D mesh can include multiple groups, and each group of vertex connection relationship can include index values of three vertices belonging to a same triangle patch.

[0017] Exemplarily, in addition to encoding the vertex coordinates in the data of the 3D mesh, the texture coordinates, the vertex connection relationship and the texture map in the data of the 3D mesh can also be encoded.

[0018] Exemplarily, the texture coordinates in the data of the 3D mesh can be encoded according to the encoding method for the vertex coordinates described above.

[0019] Exemplarily, the EdgeBreaker algorithm, TFAN algorithm or the like can be used to encode the vertex connection relationship in the data of the 3D mesh. Exemplarily, when a plurality of continuous frames of 3D mesh include a same triangle patch, the data of the plurality of continuous frames of 3D mesh includes a same group of vertex connection relationship; in the process of encoding the vertex connection relationship of the plurality of continuous frames of 3D mesh, the vertex connection relationship corresponding to the triangle patch can be encoded once; in this way, the encoding efficiency of the vertex connection relationship can be improved.

[0020] Exemplarily, a video / image encoding algorithm can be used to encode / decode the texture map.

[0021] It should be understood that the encoding / decoding method of the texture coordinates, the vertex connection relationship and the texture map in the data of the 3D mesh is not limited in the present application.

[0022] According to the first aspect, the scaling parameter corresponding to the current vertex is obtained, including: obtaining the importance information of the current vertex on the 3D mesh; determining the scaling parameter corresponding to the current vertex based on the importance information of the current vertex on the 3D mesh and a preset selection condition. Wherein, the preset selection condition can be preset, and the preset selection condition can include a mapping relationship between the preset importance information and the preset scaling parameter; in this way, the scaling parameter of the current vertex can be quickly obtained.

[0023] According to a first aspect, or any possible implementation mode of the above first aspect, the obtaining the scaling parameter corresponding to the current vertex comprises: obtaining vertex density information of a region where the current vertex is located; and determining the scaling parameter corresponding to the current vertex based on the vertex density information of the region where the current vertex is located.

[0024] In the process of generating the data of the 3D mesh, in order to reduce the data amount corresponding to the data of the 3D mesh, the point cloud granularity corresponding to different image regions in the image of the collected video data can be adjusted according to the importance of the different image regions. In this way, after the point cloud is generated, the image regions with high importance in the image of the collected video data (such as the face region of a person) can correspond to more point clouds; and after the mesh is generated, the regions corresponding to the image regions with high importance in the image of the collected video data on the 3D mesh (such as the face region on the 3D mesh of a person) can correspond to smaller and denser triangular facets. In addition, after the point cloud is generated, the image regions with low importance in the image of the collected video data (such as the body region of a person) can correspond to fewer and sparser point clouds, and after the mesh is generated, the regions corresponding to the image regions with low importance in the image of the collected video data on the 3D mesh (such as the body region on the 3D mesh of a person) can correspond to larger triangular facets. In this way, for the 3D mesh, the higher the vertex density of the region where the vertex is located, the higher the importance of the vertex; on the contrary, the lower the vertex density of the region where the vertex is located, the lower the importance of the vertex. Further, the vertex density information can accurately represent the importance of the vertex on the 3D mesh, and the scaling parameter corresponding to the current vertex can be accurately determined subsequently.

[0025] In a possible mode, the encoder can also use a Region Of Interest (ROI) detection algorithm to detect the importance information of each vertex in the 3D mesh. In this way, the encoder can obtain the importance information of the current vertex on the 3D mesh determined by the ROI detection algorithm.

[0026] It should be understood that the application does not limit the way of obtaining the importance information of the current vertex on the 3D mesh.

[0027] According to a first aspect, or any possible implementation mode of the first aspect, the obtaining the vertex density information of the region where the current vertex is located comprises: determining one or more first neighboring triangular patches, the first neighboring triangular patch being a triangular patch with the current vertex as a vertex; determining one or more second neighboring triangular patches based on the one or more first neighboring triangular patches, the second neighboring triangular patch being a triangular patch sharing the reconstructed vertex with the first neighboring triangular patch of the current vertex; determining the area of the one or more second neighboring triangular patches based on the reconstructed coordinates of the vertices included in the one or more second neighboring triangular patches; and determining the vertex density information based on the area of the one or more second neighboring triangular patches.

[0028] As can be known from the above description, the area of the triangular patch of the region with a higher importance degree on the 3D mesh is smaller, and therefore the area of the second neighboring triangular patch of the current vertex can be used to accurately determine the vertex density information of the region where the current vertex is located. In addition, the vertex density information is determined based on the reconstructed coordinates of the vertices, and therefore the decoding end can also determine the vertex density information of the current vertex in this manner, and the encoding end does not need to write the scaling parameter / vertex density information into the code stream, thereby reducing the code rate overhead.

[0029] For example, the vertex reconstruction process of the encoding end can be as follows: performing prediction and compensation based on the residual information corresponding to the current vertex to obtain the first reconstructed coordinates of the current vertex; and performing scaling on the first reconstructed coordinates based on the scaling parameter corresponding to the current vertex to obtain the second reconstructed coordinates of the current vertex. In the above manner, the reconstructed coordinates of the vertices included in the second neighboring triangular patch are the second reconstructed coordinates of the vertices included in the second neighboring triangular patch.

[0030] According to a first aspect, or any possible implementation mode of the first aspect, the obtaining the vertex density information of the region where the current vertex is located comprises: determining one or more first neighboring triangular patches, the first neighboring triangular patch being a triangular patch with the current vertex as a vertex; determining the area of the one or more first neighboring triangular patches based on the vertex coordinates of the vertices included in the one or more first neighboring triangular patches; and determining the vertex density information based on the area of the one or more first neighboring triangular patches.

[0031] As can be known from the above description, the area of the triangular patch of the region with a higher importance degree on the 3D mesh is smaller, and therefore the area of the triangular patch with the current vertex as a vertex can be used to accurately determine the vertex density information of the region where the current vertex is located.

[0032] In addition, the vertex density information is determined based on vertex coordinates of the vertex, i.e., based on original vertex coordinates, so that the scaling parameter / vertex density information can be written into a bitstream by an encoding end, and a decoding end can determine the scaling parameter corresponding to the vertex.

[0033] According to the first aspect, or any of the implementations of the first aspect, the vertex density information is determined based on the area of the one or more second adjacent triangular patches, including: determining an area of a largest reconstructed triangular patch and an area of a smallest reconstructed triangular patch based on reconstructed coordinates of the reconstructed vertex; and determining the vertex density information based on the area of the one or more second adjacent triangular patches, the area of the largest reconstructed triangular patch, and the area of the smallest reconstructed triangular patch.

[0034] For example, in one possible manner, the vertex density information p can be determined with reference to the following formula (1):

[0035] p = (S-S1 min ) / (S1 max -S1 min ) (1)

[0036] wherein S is the area of the second adjacent triangular patch when the number of the second adjacent triangular patches is one, and S is an average of the areas of the second adjacent triangular patches when the number of the second adjacent triangular patches is more than one. S1 max is the area of the largest reconstructed triangular patch, and S1 min is the area of the smallest reconstructed triangular patch.

[0037] According to the first aspect, or any of the implementations of the first aspect, the vertex density information is determined based on the area of the one or more second adjacent triangular patches, including: determining an area of a largest reconstructed triangular patch and an area of a smallest reconstructed triangular patch based on reconstructed coordinates of the reconstructed vertex; and determining the vertex density information based on the area of the one or more second adjacent triangular patches, the area of the largest reconstructed triangular patch, and the area of the smallest reconstructed triangular patch.

[0038] Specifically, S2 max and S2 min may be updated based on the area of the second adjacent triangular patch, and the area S2 max of the largest triangular patch and the area S2 min of the smallest triangular patch included in the 3D mesh, with reference to the following formula (2):

[0039] S3 max = max(S2max , S), S3 min = min(S2 min , S) (2)

[0040] wherein, when the number of the second adjacent triangular patches is one, S is the area of the second adjacent triangular patch; when the second adjacent triangular patches are multiple, S is the average of the areas of the second adjacent triangular patches.

[0041] Then, based on the area of the second adjacent triangular patch, S3 max and S3 min , the vertex density information p is determined; in one possible implementation manner, the vertex density information p can be determined by referring to the following formula (3):

[0042] p = (S - S3 min ) / (S3 max - S3 min ) (3)

[0043] wherein, when the number of the second adjacent triangular patches is one, S is the area of the second adjacent triangular patch; when the second adjacent triangular patches are multiple, S is the average of the areas of the second adjacent triangular patches.

[0044] Since the number of reconstructed vertices and reconstructed triangular patches is small at the beginning of encoding, the vertex density information calculated by using the formula (3) is more accurate and has smaller error compared with the formula (1).

[0045] It should be noted that when the formula (3) is used to calculate the vertex density information of the current vertex, S2 max and S2 min may be written into the bitstream; so that the decoding end can also determine the vertex density information of the current vertex based on S2 max and S2 min parsed from the bitstream.

[0046] According to the first aspect, or any one of the implementation manners of the first aspect, the vertex density information is determined based on the area of the one or more first adjacent triangular patches, including: determining the area of the triangular patch with the largest area and the area of the triangular patch with the smallest area based on the vertex coordinates of the plurality of vertices included in the data of the 3D mesh; and determining the vertex density information based on the area of the one or more first adjacent triangular patches, the area of the triangular patch with the largest area and the area of the triangular patch with the smallest area. Specifically, the formula (2) and the formula (3) can be used.

[0047] It should be noted that when the formula (3) is used to calculate the vertex density information of the current vertex, S2 max and S2min write into the bitstream; so that the decoding end can also determine S2 max and S2 min determine the vertex density information of the current vertex.

[0048] According to the first aspect, or any one of the implementation forms of the first aspect, the prediction and compensation based on the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex comprises: performing the prediction and compensation based on the scaling parameter and the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex.

[0049] The prediction and compensation can be performed based on the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex of the current vertex. Since the scaling parameter corresponding to the reference vertex of the current vertex can be the same as or different from the scaling parameter corresponding to the current vertex, when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex of the current vertex, the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex are not in the same order of magnitude. When the current vertex is a point with high importance, if the residual is directly calculated based on the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex, the residual will be reduced, the subsequent quantization error will be increased, and then the reconstruction quality will be reduced. When the current vertex is a point with low importance, if the residual is directly calculated based on the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex, the residual will be increased, and then the compression code rate will be increased. Therefore, in the process of prediction and compensation, the scaling parameter of the current vertex and the scaling parameter of the reference vertex can be adjusted based on the scaling parameter, the difference in the order of magnitude of the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex can be reduced, and the reconstruction quality or the compression code rate can be improved.

[0050] According to the first aspect, or any one of the implementation forms of the first aspect, the prediction and compensation based on the scaling parameter and the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex comprises: predicting the reference vertex of the current vertex; when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex, determining a normalized scaling parameter based on the scaling parameter corresponding to the current vertex and / or the scaling parameter corresponding to the reference vertex; determining the normalized coordinates corresponding to the current vertex and the normalized coordinates corresponding to the reference vertex based on the normalized scaling parameter; and performing compensation based on the normalized coordinates corresponding to the current vertex and the normalized coordinates corresponding to the reference vertex to determine the residual information corresponding to the current vertex. In this way, by normalizing the scaling parameter of the current vertex and the scaling parameter of the reference vertex, the difference in the order of magnitude of the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex can be reduced to the greatest extent.

[0051] For example, the compensation in the encoding process can refer to residual calculation.

[0052] According to the first aspect, or any one of the implementations of the above first aspect, the normalized scaling parameter is determined based on the scaling parameter corresponding to the current vertex or the scaling parameter corresponding to the reference vertex, including: determining the scaling parameter corresponding to the current vertex as the normalized scaling parameter.

[0053] In this way, when the current vertex is a vertex with high importance, the scaling parameter corresponding to the current vertex is relatively large, and normalizing the scaling parameter corresponding to the reference vertex to the scaling parameter of the current vertex can obtain larger residual information, thereby reducing the quantization error of subsequent quantization, improving the prediction accuracy, and preserving more details, thereby improving the image quality of the region with high importance. When the current vertex is a vertex with low importance, the scaling parameter corresponding to the current vertex is relatively small, and normalizing the scaling parameter corresponding to the reference vertex to the scaling parameter of the current vertex can obtain smaller residual information, thereby reducing the number of bytes required for encoding, and reducing the code rate under certain distortion conditions.

[0054] According to the first aspect, or any one of the implementations of the above first aspect, the normalized scaling parameter is determined based on the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex, including: determining the maximum scaling parameter in the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex as the normalized scaling parameter.

[0055] In this way, if the scaling parameter of the current vertex is the maximum scaling parameter, the scaled vertex coordinates of the reference vertex need to be scaled, thereby reducing the quantization error of the vertex coordinates of the current vertex and improving the accuracy of the vertex coordinates of the current vertex. If the scaling parameter of the reference vertex is the maximum scaling parameter, it indicates that the current vertex and the reference vertex are at the boundary of the important region of the 3D mesh, and scaling the scaled vertex coordinates of the current vertex can also reduce the quantization error of the vertex coordinates of the current vertex and improve the accuracy of the vertex coordinates of the current vertex, thereby improving the reconstruction quality at the boundary of the important region.

[0056] According to the first aspect, or any one of the implementations of the above first aspect, the scaling parameter corresponding to the current vertex is written into the code stream.

[0057] It should be understood that no matter which way is used to determine the vertex density information, the scaling parameter corresponding to the current vertex can be encoded and written into the code stream.

[0058] It should be noted that when the vertex density information is determined based on the area of the one or more second adjacent triangular patches, the scaling parameter corresponding to the current vertex can not be written into the bitstream. When the vertex density information is determined based on the area of the one or more second adjacent triangular patches, the scaling parameter corresponding to the current vertex needs to be written into the bitstream.

[0059] In a possible manner, the scaling parameter corresponding to the current vertex can not be encoded, but the importance information of the current vertex on the 3D mesh can be encoded.

[0060] According to the first aspect, or any one of the implementation forms of the first aspect, the method further includes: writing the area of the triangular patch with the largest area and the area of the triangular patch with the smallest area into the bitstream.

[0061] In the second aspect, an embodiment of the present application provides a decoding method, which includes: first, receiving a bitstream; then, parsing the residual information corresponding to the current vertex from the bitstream; and obtaining the scaling parameter corresponding to the current vertex, which is positively correlated with the importance of the current vertex on the 3D mesh; and then, performing prediction and compensation based on the residual information corresponding to the current vertex to obtain the first reconstructed coordinates of the current vertex; and then, performing scaling on the first reconstructed coordinates based on the scaling parameter corresponding to the current vertex to obtain the second reconstructed coordinates of the current vertex.

[0062] According to the second aspect, the obtaining of the scaling parameter corresponding to the current vertex includes: obtaining the importance information of the current vertex on the 3D mesh; and determining the scaling parameter corresponding to the current vertex based on the importance information of the current vertex on the 3D mesh and a preset selection condition.

[0063] According to the second aspect, or any one of the implementation forms of the second aspect, the obtaining of the scaling parameter corresponding to the current vertex includes: obtaining the vertex density information of the region where the current vertex is located; and determining the scaling parameter corresponding to the current vertex based on the vertex density information of the region where the current vertex is located.

[0064] According to a second aspect, or any possible implementation mode of the second aspect, the obtaining the vertex density information of the region where the current vertex is located comprises: determining one or more first neighboring triangular patches, the first neighboring triangular patch being a triangular patch having the current vertex as a vertex; determining one or more second neighboring triangular patches based on the one or more first neighboring triangular patches, the second neighboring triangular patch being a triangular patch sharing the reconstructed vertex with the first neighboring triangular patch of the current vertex; determining areas of the one or more second neighboring triangular patches based on second reconstructed coordinates of vertices included in the one or more second neighboring triangular patches; and determining the vertex density information based on the areas of the one or more second neighboring triangular patches.

[0065] According to the second aspect, or any possible implementation mode of the second aspect, the determining the vertex density information based on the areas of the one or more second neighboring triangular patches comprises: parsing, from the bitstream, an area of a triangular patch with the largest area and an area of a triangular patch with the smallest area; and determining the vertex density information based on the areas of the one or more second neighboring triangular patches, the area of the triangular patch with the largest area and the area of the triangular patch with the smallest area.

[0066] According to the second aspect, or any possible implementation mode of the second aspect, the determining the vertex density information based on the areas of the one or more second neighboring triangular patches comprises: determining, based on the second reconstructed coordinates of the reconstructed vertices, an area of a reconstructed triangular patch with the largest area and an area of a reconstructed triangular patch with the smallest area; and determining the vertex density information based on the areas of the one or more second neighboring triangular patches, the area of the reconstructed triangular patch with the largest area and the area of the reconstructed triangular patch with the smallest area.

[0067] According to the second aspect, or any possible implementation mode of the second aspect, the obtaining the scaling parameter corresponding to the current vertex comprises: parsing, from the bitstream, the scaling parameter corresponding to the current vertex.

[0068] According to the second aspect, or any possible implementation mode of the second aspect, the performing prediction and compensation based on the residual information corresponding to the current vertex to obtain the first reconstructed coordinate of the current vertex comprises: performing prediction and compensation based on the scaling parameter corresponding to the current vertex and the residual information corresponding to the current vertex to obtain the first reconstructed coordinate of the current vertex.

[0069] According to a second aspect, or any possible implementation mode of the second aspect, the predicting and the compensating to obtain the first reconstructed coordinate of the current vertex based on the scaling parameter corresponding to the current vertex and the residual information corresponding to the current vertex comprises: predicting a reference vertex of the current vertex; when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex, determining a normalized scaling parameter based on the scaling parameter corresponding to the current vertex and / or the scaling parameter corresponding to the reference vertex; scaling the first reconstructed coordinate corresponding to the reference vertex based on the normalized scaling parameter to obtain a normalized coordinate corresponding to the reference vertex; compensating the normalized coordinate corresponding to the current vertex based on the normalized coordinate corresponding to the reference vertex and the residual information corresponding to the current vertex; and scaling the normalized coordinate corresponding to the current vertex based on the normalized scaling parameter to obtain the first reconstructed coordinate corresponding to the current vertex.

[0070] For example, the compensating in the decoding process can be a superposition operation.

[0071] According to the second aspect, or any possible implementation mode of the second aspect, the determining the normalized scaling parameter based on the scaling parameter corresponding to the current vertex or the scaling parameter corresponding to the reference vertex comprises: determining the scaling parameter corresponding to the current vertex as the normalized scaling parameter.

[0072] According to the second aspect, or any possible implementation mode of the second aspect, the determining the normalized scaling parameter based on the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex comprises: determining the maximum scaling parameter in the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex as the normalized scaling parameter.

[0073] The second aspect and any possible implementation mode of the second aspect correspond to the first aspect and any possible implementation mode of the first aspect respectively. The technical effects of the second aspect and any possible implementation mode of the second aspect can refer to the technical effects of the first aspect and any possible implementation mode of the first aspect, which will not be described herein.

[0074] In a third aspect, an embodiment of the present application provides an encoding device, which comprises:

[0075] a data obtaining module, configured to obtain data of a three-dimensional mesh (3D mesh), the 3D mesh being composed of a plurality of triangular facets, each triangular facet including three vertices, and the data of the 3D mesh including vertex coordinates of the plurality of vertices, the vertex coordinates being three-dimensional coordinates;

[0076] an information obtaining module, configured to obtain a scaling parameter corresponding to a current vertex, the scaling parameter corresponding to the current vertex being positively correlated with an importance of the current vertex on the 3D mesh;

[0077] a scaling module, configured to scale the vertex coordinate of the current vertex based on the scaling parameter corresponding to the current vertex, to obtain a scaled vertex coordinate of the current vertex;

[0078] a prediction and compensation module, configured to perform prediction and compensation based on the scaled vertex coordinate of the current vertex, to obtain residual information corresponding to the current vertex;

[0079] an encoding module, configured to encode the residual information corresponding to the current vertex, to obtain the bitstream.

[0080] For example, the encoding apparatus can be used to perform the encoding method in the first aspect or any possible implementation manner of the first aspect.

[0081] The third aspect and any possible implementation manner of the third aspect correspond to the first aspect and any possible implementation manner of the first aspect respectively. For details, refer to the descriptions of the first aspect and any possible implementation manner of the first aspect, which are not described here.

[0082] In a fourth aspect, an embodiment of the present application provides a decoding apparatus, which comprises:

[0083] a bitstream receiving module, configured to receive a bitstream;

[0084] a decoding module, configured to parse the residual information corresponding to the current vertex from the bitstream;

[0085] an information obtaining module, configured to obtain the scaling parameter corresponding to the current vertex, wherein the scaling parameter corresponding to the current vertex is positively correlated with the importance of the current vertex on the 3D mesh;

[0086] a prediction and compensation module, configured to perform prediction and compensation based on the residual information corresponding to the current vertex, to obtain a first reconstructed coordinate of the current vertex;

[0087] a scaling module, configured to scale the first reconstructed coordinate based on the scaling parameter corresponding to the current vertex, to obtain a second reconstructed coordinate of the current vertex.

[0088] For example, the decoding apparatus can be used to perform the encoding method in the second aspect or any possible implementation manner of the second aspect.

[0089] The fourth aspect and any possible implementation manner of the fourth aspect correspond to the second aspect and any possible implementation manner of the second aspect respectively. For details, refer to the descriptions of the second aspect and any possible implementation manner of the second aspect, which are not described here.

[0090] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor, the memory being coupled with the processor; the memory storing program instructions, when the program instructions are executed by the processor, causing the electronic device to perform the encoding method in the first aspect or any possible implementation manner of the first aspect.

[0091] The fifth aspect and any possible implementation manner of the fifth aspect correspond to the first aspect and any possible implementation manner of the first aspect respectively. For reference to the technical effects of the first aspect and any possible implementation manner of the first aspect, details are not repeated here.

[0092] In a sixth aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor, the memory being coupled with the processor; the memory storing program instructions, when the program instructions are executed by the processor, causing the electronic device to perform the decoding method in the second aspect or any possible implementation manner of the second aspect.

[0093] The sixth aspect and any possible implementation manner of the sixth aspect correspond to the second aspect and any possible implementation manner of the second aspect respectively. For reference to the technical effects of the second aspect and any possible implementation manner of the second aspect, details are not repeated here.

[0094] In a seventh aspect, an embodiment of the present application provides a chip, including one or more interface circuits and one or more processors; the interface circuit is used for receiving a signal from a memory of an electronic device, and sending a signal to the processor, the signal including computer instructions stored in the memory; when the processor executes the computer instructions, causing the electronic device to perform the encoding method in the first aspect or any possible implementation manner of the first aspect.

[0095] The seventh aspect and any possible implementation manner of the seventh aspect correspond to the first aspect and any possible implementation manner of the first aspect respectively. For reference to the technical effects of the first aspect and any possible implementation manner of the first aspect, details are not repeated here.

[0096] In an eighth aspect, an embodiment of the present application provides a chip, including one or more interface circuits and one or more processors; the interface circuit is used for receiving a signal from a memory of an electronic device, and sending a signal to the processor, the signal including computer instructions stored in the memory; when the processor executes the computer instructions, causing the electronic device to perform the decoding method in the second aspect or any possible implementation manner of the second aspect.

[0097] The eighth aspect and any possible implementation of the eighth aspect correspond to the second aspect and any possible implementation of the second aspect respectively. The technical effects of the eighth aspect and any possible implementation of the eighth aspect can refer to the technical effects of the second aspect and any possible implementation of the second aspect, which will not be repeated here.

[0098] In a ninth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. When the computer program is executed on a computer or a processor, the computer or the processor performs the encoding method in the first aspect or any possible implementation of the first aspect.

[0099] The ninth aspect and any possible implementation of the ninth aspect correspond to the first aspect and any possible implementation of the first aspect respectively. The technical effects of the ninth aspect and any possible implementation of the ninth aspect can refer to the technical effects of the first aspect and any possible implementation of the first aspect, which will not be repeated here.

[0100] In a tenth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. When the computer program is executed on a computer or a processor, the computer or the processor performs the encoding and decoding method in the second aspect or any possible implementation of the second aspect.

[0101] The tenth aspect and any possible implementation of the tenth aspect correspond to the second aspect and any possible implementation of the second aspect respectively. The technical effects of the tenth aspect and any possible implementation of the tenth aspect can refer to the technical effects of the second aspect and any possible implementation of the second aspect, which will not be repeated here.

[0102] In an eleventh aspect, an embodiment of the present application provides a computer program product, which includes a software program. When the software program is executed on a computer or a processor, the computer or the processor performs the encoding method in the first aspect or any possible implementation of the first aspect.

[0103] The eleventh aspect and any possible implementation of the eleventh aspect correspond to the first aspect and any possible implementation of the first aspect respectively. The technical effects of the eleventh aspect and any possible implementation of the eleventh aspect can refer to the technical effects of the first aspect and any possible implementation of the first aspect, which will not be repeated here.

[0104] In a twelfth aspect, an embodiment of the present application provides a computer program product, which comprises a software program. When the software program is executed by a computer or a processor, the computer or the processor performs the decoding method in the second aspect or any possible implementation manner of the second aspect.

[0105] The twelfth aspect and any possible implementation manner of the twelfth aspect correspond to the second aspect and any possible implementation manner of the second aspect respectively. For details, refer to the technical effects of the second aspect and any possible implementation manner of the second aspect, which will not be repeated here.

[0106] In a thirteenth aspect, an embodiment of the present application provides a code stream generation method. The code stream generation method generates a code stream according to the first aspect and any possible implementation manner of the first aspect.

[0107] The thirteenth aspect and any possible implementation manner of the thirteenth aspect correspond to the first aspect and any possible implementation manner of the first aspect respectively. For details, refer to the technical effects of the first aspect and any possible implementation manner of the first aspect, which will not be repeated here.

[0108] In a fourteenth aspect, an embodiment of the present application provides a device for storing a code stream. The device comprises a receiver and at least one storage medium,

[0109] The receiver is configured to receive the code stream.

[0110] The at least one storage medium is configured to store the code stream.

[0111] The code stream is generated according to the first aspect and any possible implementation manner of the first aspect.

[0112] The fourteenth aspect and any possible implementation manner of the fourteenth aspect correspond to the first aspect and any possible implementation manner of the first aspect respectively. For details, refer to the technical effects of the first aspect and any possible implementation manner of the first aspect, which will not be repeated here.

[0113] In a fifteenth aspect, an embodiment of the present application provides a device for transmitting a code stream. The device comprises a transmitter and at least one storage medium,

[0114] The at least one storage medium is configured to store the code stream. The code stream is generated according to the first aspect and any possible implementation manner of the first aspect.

[0115] The transmitter is configured to acquire the code stream from the storage medium and send the code stream to the terminal device through a transmission medium.

[0116] The fifteenth aspect and any kind of implementation manner of the fifteenth aspect correspond to the first aspect and any kind of implementation manner of the first aspect respectively. The technical effects corresponding to the fifteenth aspect and any kind of implementation manner of the fifteenth aspect can refer to the technical effects corresponding to the first aspect and any kind of implementation manner of the first aspect, which will not be described here.

[0117] The sixteenth aspect, the embodiment of the application provides a system for distributing code stream, the system comprises:

[0118] At least one storage medium is configured to store at least one code stream, and the at least one code stream is generated according to the first aspect and any kind of implementation manner of the first aspect,

[0119] The streaming media device is configured to acquire the target code stream from the at least one storage medium and send the target code stream to the terminal device, wherein the streaming media device comprises a content server or a content distribution server.

[0120] The sixteenth aspect and any kind of implementation manner of the sixteenth aspect correspond to the first aspect and any kind of implementation manner of the first aspect respectively. The technical effects corresponding to the sixteenth aspect and any kind of implementation manner of the sixteenth aspect can refer to the technical effects corresponding to the first aspect and any kind of implementation manner of the first aspect, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0121] Figure 1 The application scenario schematic diagram is exemplarily shown;

[0122] Figure 2a The encoding process schematic diagram is exemplarily shown;

[0123] Figure 2b The framework schematic diagram of the encoding module is exemplarily shown;

[0124] Figure 3a The decoding process schematic diagram is exemplarily shown;

[0125] Figure 3b The framework schematic diagram of the decoding module is exemplarily shown;

[0126] Figure 4a The 3D mesh schematic diagram is exemplarily shown;

[0127] Figure 4b The encoding process schematic diagram is exemplarily shown;

[0128] Figure 4cA schematic diagram of part of the triangular facets in a 3D mesh shown for illustration purposes;

[0129] Figure 4d A schematic diagram of part of the triangular facets in a 3D mesh shown for illustration purposes;

[0130] Figure 4e A schematic diagram of an encoding process for vertex coordinates shown for illustration purposes;

[0131] Figure 5a A schematic diagram of a decoding process shown for illustration purposes;

[0132] Figure 5b A schematic diagram of a decoding process shown for illustration purposes;

[0133] Figure 5c A schematic diagram of a vertex coordinate compression process shown for illustration purposes;

[0134] Figure 6a A schematic diagram of an encoding process shown for illustration purposes;

[0135] Figure 6b A schematic diagram of an encoding process for vertex coordinates shown for illustration purposes;

[0136] Figure 7a A schematic diagram of a decoding process shown for illustration purposes;

[0137] Figure 7b A schematic diagram of a decoding process shown for illustration purposes;

[0138] Figure 8 A schematic diagram of an encoding apparatus shown for illustration purposes;

[0139] Figure 9 A schematic diagram of a decoding apparatus shown for illustration purposes;

[0140] Figure 10 A schematic diagram of the structure of an apparatus shown for illustration purposes. DETAILED DESCRIPTION

[0141] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0142] The term “and / or” in the present document is merely used to describe an association relationship of associated objects, and indicates that there can be three relationships, for example, A and / or B can represent the three cases of existence of A alone, existence of A and B simultaneously, and existence of B alone.

[0143] The terms “first” and “second” and the like in the description and claims of the present application are used to distinguish different objects, and are not used to describe a specific order of the objects. For example, the first target object and the second target object are used to distinguish different target objects, and are not used to describe a specific order of the target objects.

[0144] In the embodiments of the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as “exemplary” or “for example” in the embodiments of the present application should not be construed as preferred or advantageous over other embodiments or design solutions. Rather, the use of “exemplary” or “for example” is intended to present concepts in a particular manner.

[0145] In the description of the embodiments of the present application, unless otherwise specified, the meaning of “a plurality of” is two or more. For example, a plurality of processing units refers to two or more processing units; a plurality of systems refers to two or more systems.

[0146] Figure 1 The application scenario shown is exemplary.

[0147] Referring to Figure 1 (1), exemplary, a plurality of cameras can be arranged around the target object; wherein each camera can be deployed at different angles relative to the target object. It should be understood that, Figure 1 (1) is only one example of the deployment of multiple cameras, and the number of cameras deployed around the target object by the present application can be more than Figure 1 (1) or less, which is not limited by the present application.

[0148] In one possible scenario, for example, artistic dance performance, sports competition, film and television variety show, net red performance, scenic spot photo, live streaming, exhibition drainage, action teaching, etc. After the multiple cameras work, each camera can collect one way of video data; and multiple ways of video data can be obtained. Among them, multiple cameras can shoot pictures of the target object at different angles at the same time.

[0149] Referring to Figure 1(2), exemplary, after obtaining the multi-path video data, the multi-path video data can be perspective synthesized (for example, perspective synthesis is performed on images with the same timestamp in different path video data), and the synthesized video data can be obtained; wherein the synthesized video data includes multiple frames of images. Then, the point cloud processing is performed on each frame of image of the synthesized video data, and the 3D point cloud corresponding to each frame of image of the synthesized video data is obtained. Subsequently, the mesh processing is performed on the 3D point cloud corresponding to each frame of image of the synthesized video data, and multiple frames of 3D mesh (one frame of 3D mesh corresponds to one frame of image of the synthesized video data) and data of each frame of 3D mesh can be obtained. Wherein, the multiple frames of 3D mesh can constitute a volume video (also can be called dynamic 3D mesh); the data of each frame of 3D mesh can include vertex coordinates, texture coordinates, texture map and vertex connection relationship.

[0150] Exemplary, each frame of 3D mesh can include at least one 3D mesh (one 3D mesh corresponds to one target object in the image of the synthesized video data), and each 3D mesh is composed of multiple triangular facets; each triangular facet includes three vertices, and the coordinates of each vertex can be called vertex coordinates, and the vertex coordinates are three-dimensional coordinates (for example, which can be represented by (x, y, z)).

[0151] Exemplary, each triangular facet in each 3D mesh has a corresponding texture map; wherein the corresponding texture map of each triangular facet in each 3D mesh can be the same or different, which is not limited in the present application.

[0152] Exemplary, each vertex of each triangular facet in each 3D mesh can be mapped to a pixel point in the corresponding texture map, and the coordinates of the pixel point are also the texture coordinates; wherein the texture coordinates are two-dimensional coordinates (for example, which can be represented by (u, v)).

[0153] Exemplary, the vertex connection relationship refers to the connection relationship between the three vertices included in each triangular facet; wherein the vertex connection relationship of each 3D mesh can include multiple groups, and each group of vertex connection relationship can include the index values of the three vertices belonging to the same triangular facet.

[0154] It should be noted that, in order to facilitate the improvement of the coding efficiency of the vertex connection relationship, for each triangular patch in each frame of 3D mesh, tracking can be performed to determine the position of the triangular patch in subsequent adjacent frames of 3D mesh. In this way, when a plurality of continuous frames of 3D mesh contain the same triangular patch, the data of the plurality of continuous frames of 3D mesh include the same set of vertex connection relationships; in the process of subsequently encoding the vertex connection relationships of the plurality of continuous frames of 3D mesh, the vertex connection relationship corresponding to the triangular patch can be encoded once.

[0155] In a possible scenario, such as a digital exhibition scenario, after a plurality of cameras work, each camera can capture an image, and then a plurality of images can be obtained; each image corresponds to a different angle of shooting. Then, the plurality of images can be used for perspective synthesis, point cloudization, and meshing, and a frame of 3D mesh and data of the frame of 3D mesh can be obtained. The frame of 3D mesh can be referred to as a static 3D mesh.

[0156] With reference to Figure 1 (3), for example, after the first electronic device obtains the data of the 3D mesh, the first electronic device can input the data of the 3D mesh to the encoding module, and the encoding module encodes the data of the 3D mesh and outputs a code stream to the communication module; then, the communication module can send the code stream to the second electronic device.

[0157] With reference to Figure 1 (3), for example, after the communication module of the second electronic device receives the code stream, the communication module of the second electronic device can output the code stream to the decoding module, and the decoding module decodes the code stream to obtain the data of the reconstructed 3D mesh.

[0158] It should be understood that, Figure 1 (2) The electronic device that generates the data of the 3D mesh can be the first electronic device, or a third electronic device other than the first electronic device and the second electronic device, and the present application does not limit this.

[0159] For example, when Figure 1 (2) When the electronic device that generates the data of the 3D mesh is the third electronic device, the first electronic device can be a server, and the second electronic device and the third electronic device can be terminal devices. The server can refer to a single server, or a server cluster, and the present application does not limit this. The terminal device includes, but is not limited to, for example, a personal computer, a computer workstation, a smart phone, a tablet computer, a server, a smart camera, a smart car, or other types of cellular phones, a media consumption device, a wearable device, a set-top box, a game console, and the like, and the present application does not limit this.

[0160] Exemplarily, when Figure 1 (2) the electronic device generating the data of the 3D mesh is the first electronic device, the first electronic device and the second electronic device can both be terminal devices.

[0161] The following is described by taking the data of a dynamic 3D mesh as an example.

[0162] Exemplarily, to improve the compression performance of the compressed 3D mesh data, the application can scale and then compress the vertex coordinates in the data of each frame of 3D mesh based on the importance of each vertex in each frame of 3D mesh. Wherein, for the vertices with low importance, a smaller scaling parameter can be used for scaling, and smaller residual information can be obtained; and then fewer bytes are used for encoding, so that the code rate overhead can be reduced under certain distortion conditions. For the vertices with high importance, a larger scaling parameter can be used for scaling, and larger residual information can be obtained; and then the quantization error in the subsequent encoding process is small, and the reconstruction quality can be improved.

[0163] Based on this, the encoding and decoding process of the data of the 3D mesh is described below.

[0164] Figure 2a The encoding process is exemplarily shown in the schematic diagram. Figure 2a The encoding process shown can be the encoding process performed by the encoding module of the first electronic device (also referred to as the encoding end) in Figure 1 (3); wherein, Figure 2a The encoding process described is the encoding process of the vertex coordinates in the data of the 3D mesh.

[0165] S201, obtaining the data of a three-dimensional mesh 3D mesh, the 3D mesh being composed of a plurality of triangular facets, each triangular facet including three vertices, and the data of the 3D mesh including vertex coordinates of a plurality of vertices.

[0166] In one possible implementation, the first electronic device can obtain the data of the 3D mesh to be encoded from the third electronic device; wherein the data of the 3D mesh is generated by the third electronic device according to the process shown in Figure 1 (2).

[0167] In one possible way, the data of the 3D mesh to be encoded can be generated by the first electronic device according to the process shown in Figure 1 (2).

[0168] S202, obtaining a scaling parameter corresponding to the current vertex, the scaling parameter corresponding to the current vertex being positively correlated with the importance of the current vertex on the 3D mesh.

[0169] Exemplarily, different scaling parameters can be set in advance for different importance degrees of the vertex to obtain preset selection conditions. The scaling parameter is positively correlated with the importance degree of the vertex, that is, the higher the importance degree of the vertex, the larger the scaling parameter; otherwise, the lower the importance degree of the vertex, the smaller the scaling parameter. Exemplarily, the scaling parameter can also be referred to as a scaling factor, such as 20, 50, and 100, etc.

[0170] Exemplarily, the importance degree of the vertex on the 3D mesh is related to the importance degree of the region on the 3D mesh where the vertex is located; the importance degree of the vertex on the 3D mesh can be determined according to the importance degree of the region on the 3D mesh where the vertex is located. The importance degree of the vertex on the 3D mesh is positively correlated with the importance degree of the region on the 3D mesh where the vertex is located, that is, the higher the importance degree of the region on the 3D mesh where the vertex is located, the higher the importance degree of the vertex on the 3D mesh; otherwise, the lower the importance degree of the region on the 3D mesh where the vertex is located, the lower the importance degree of the vertex on the 3D mesh. The importance degrees of different regions in the 3D mesh can be different, for example, the importance degree of the facial region is higher than that of the body region when the 3D mesh is a 3D mesh of a person; correspondingly, the scaling parameter of the vertex located in the facial region is larger than that of the vertex located in the body region.

[0171] Exemplarily, the vertex coordinates of each vertex in the data of the 3D mesh can be encoded in sequence, wherein each vertex to be encoded is referred to as a current vertex. Exemplarily, the importance degree of the current vertex can be obtained in the process of encoding the current vertex; then the scaling parameter corresponding to the current vertex is determined based on the importance degree of the current vertex and the preset selection condition; which will be described in detail later.

[0172] S203, based on the scaling parameter corresponding to the current vertex, the vertex coordinates of the current vertex are scaled to obtain the scaled vertex coordinates of the current vertex.

[0173] Exemplarily, after obtaining the scaling parameter corresponding to the current vertex, the vertex coordinates of the current vertex can be multiplied by the scaling parameter corresponding to the current vertex to scale the vertex coordinates of the current vertex, and then the scaled vertex coordinates of the current vertex can be obtained.

[0174] For example, assuming that the vertex coordinates of the current vertex are (x1, y1, z1) and the scaling parameter is k; then the vertex coordinates of the current vertex are scaled based on the scaling parameter k: x2 = x1 * k, y2 = y1 * k, and z2 = z1 * k, and the scaled vertex coordinates of the current vertex can be (x2, y2, z2).

[0175] S204, prediction and compensation are performed based on the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex.

[0176] For example, after obtaining the scaled vertex coordinates of the current vertex, prediction and compensation can be performed based on the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex. It should be understood that prediction and compensation in the encoding process are two steps; wherein, prediction is used to determine the reference vertex matching the current vertex; and compensation is used to determine the residual information, and compensation can refer to residual calculation.

[0177] For example, prediction can include spatial prediction and spatio-temporal dual prediction; wherein, spatial prediction can refer to intra prediction, and spatio-temporal dual prediction can include inter prediction and intra prediction.

[0178] In one possible manner, spatial prediction can be differential prediction. At this time, the previous encoded vertex can be taken as the reference vertex matching the current vertex; and then, residual calculation can be performed on the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex to obtain the residual information.

[0179] In one possible manner, spatial prediction can be neighborhood weighted prediction. At this time, the neighboring vertex of the current vertex can be taken as the reference vertex matching the current vertex; wherein, the neighboring vertex refers to the vertex of the encoded vertex sharing the triangular facet with the current vertex. When the neighboring vertex is one, residual calculation can be performed on the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the neighboring vertex to obtain the residual information; and when the neighboring vertex is multiple, residual calculation can be performed on the coordinate average of the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the multiple neighboring vertices to obtain the residual information.

[0180] In one possible manner, when the prediction is spatio-temporal dual prediction, inter prediction can be performed first and then intra prediction. Specifically, the reference vertex matching the current vertex can be determined from the reference 3D mesh; and then, residual calculation can be performed on the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex to obtain the first MV of the current vertex. Next, the neighboring vertex of the current vertex is determined, and for one neighboring vertex, the reference vertex matching the neighboring vertex (hereinafter referred to as the neighboring reference vertex) is determined from the reference 3D mesh. Then, residual calculation is performed based on the scaled vertex coordinates of the neighboring vertex and the scaled vertex coordinates of the neighboring reference vertex to obtain the second MV of the neighboring vertex. When the neighboring vertex is one, residual calculation can be performed on the second MV of the neighboring vertex and the first MV of the current vertex to obtain the residual information. When the neighboring vertex is multiple, the MV average of the second MVs of the multiple neighboring vertices can be calculated first, and then residual calculation can be performed on the first MV of the current vertex and the average MV to obtain the residual information.

[0181] It should be understood that other spatial prediction or spatio-temporal dual prediction manners can also be included, and the present application does not limit this.

[0182] S205, the residual information corresponding to the current vertex is encoded to obtain a code stream.

[0183] For example, the residual information can be transformed (such as discrete Fourier transform) to obtain first data; then, the first data is quantized according to a preset quantization parameter to obtain second data; and then, the second data can be entropy encoded to obtain a code stream.

[0184] It should be noted that S201-S205 are steps performed in the encoding process.

[0185] Figure 2b A block diagram of an exemplary framework of the encoding module is shown.

[0186] Referring to Figure 2b For example, the encoding module includes a pre-processing module, a prediction and compensation module, a transformation module, a quantization module, and an entropy encoding module. The prediction and compensation module can be implemented as one module or can be split into a prediction module and a compensation module. It should be understood that Figure 2b This is only one example of the present application. The encoding module of the present application can include more or fewer modules than Figure 2b shown, and the present application does not limit this.

[0187] In combination with Figure 2a and Figure 2b For example, after the data of the 3D mesh is input to the pre-processing module, the pre-processing module can obtain the scaling parameter corresponding to the current vertex, and then scale the vertex coordinates of the current vertex based on the scaling parameter corresponding to the current vertex to obtain the scaled vertex coordinates of the current vertex. Then, the pre-processing module can output the scaled vertex coordinates of the current vertex to the prediction and compensation module. After receiving the scaled vertex coordinates of the current vertex, the prediction and compensation module can perform prediction and compensation based on the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex and output the residual information to the transformation module. After receiving the residual information, the transformation module can transform the residual information (such as discrete Fourier transform) to obtain first data and output the first data to the quantization module. After receiving the first data, the quantization module can quantize the first data according to a preset quantization parameter to obtain second data and output the second data to the entropy encoding module. After receiving the second data, the entropy encoding module can entropy encode the second data to obtain a code stream.

[0188] Thus, for vertices of low importance, using smaller scaling parameters results in smaller residuals, requiring fewer bytes for encoding; thereby reducing bitrate overhead while maintaining a certain level of distortion. For vertices of high importance, using larger scaling parameters results in larger residuals, leading to smaller quantization errors in subsequent encoding processes and thus improving reconstruction quality.

[0189] For example, the first electronic device may include means for transmitting a code stream; the means for transmitting the code stream may include: a transmitter and at least one storage medium, the at least one storage medium being used to store the above-mentioned... Figure 2a The example generates a bitstream; the transmitter is used to obtain the bitstream from the storage medium and send the bitstream to the end-side device through the transmission medium.

[0190] For example, after generating a stream, the first electronic device can send the stream to a system that distributes the stream. The system that distributes the stream may include at least one storage medium and a streaming media device; the storage medium is used to store the aforementioned... Figure 2a The embodiment generates at least one bitstream; a streaming media device is used to obtain a target bitstream from at least one storage medium and send the target bitstream to an end-side device, wherein the streaming media device includes a content server or a content distribution server.

[0191] Figure 3a This is a schematic diagram illustrating the decoding process as an example. Figure 3a The decoding process shown is the same as described above. Figure 2a The encoding process shown corresponds to, Figure 3a The decoding process shown can be as described above. Figure 1 (3) The encoding process performed by the decoding module of the second electronic device (also known as the decoding end). Figure 3a The decoding process shown is the decoding process of vertex coordinates.

[0192] For example, the second electronic device may include a means for storing a bitstream. The means for storing the bitstream may include: a receiver for receiving the bitstream; and at least one storage medium for storing the bitstream. The bitstream can then be decoded with reference to S302-S305.

[0193] S301, receive bitstream.

[0194] For example, after the communication module of the first electronic device sends the bitstream, the communication module of the second electronic device can receive the bitstream; then the communication module of the second electronic device can output the bitstream to the decoding module, which decodes the bitstream. The following explanation uses decoding the vertex coordinates of a single vertex as an example, where the vertex to be decoded can be referred to as the current vertex.

[0195] S302, parse the residual information corresponding to the current vertex from the bitstream.

[0196] For example, the bitstream can be parsed to parse the residual information corresponding to the current vertex from the bitstream. For example, the bitstream can be first entropy decoded to obtain second data; then, the second data can be inverse quantized according to a preset quantization parameter to obtain third data; and then, the third data can be inverse transformed to parse the residual information corresponding to the current vertex.

[0197] S303, obtain the scaling parameter corresponding to the current vertex, which is positively correlated with the importance of the current vertex on the 3D mesh.

[0198] In a possible manner, the second electronic device can obtain the scaling parameter corresponding to the current vertex in the decoding process in a manner similar to that of the first electronic device in the encoding process, that is, S303 can refer to the description of S202, which will not be repeated here.

[0199] In a possible manner, the first electronic device can further write the scaling parameter corresponding to the current vertex into the bitstream, so that the second electronic device can parse the scaling parameter corresponding to the current vertex from the bitstream.

[0200] S304, predict and compensate based on the residual information corresponding to the current vertex to obtain the first reconstructed coordinates of the current vertex.

[0201] For example, the first reconstructed coordinates of the current vertex can be obtained by predicting and compensating based on the residual information corresponding to the current vertex. It should be understood that the prediction and compensation in the decoding process are also two steps; wherein, the prediction is used to determine the reference vertex matched with the current vertex; and the compensation is used to determine the first reconstructed coordinates of the current vertex, and the compensation can mean superposition.

[0202] For example, the bitstream can include reference description information, which can be used to describe the information referenced by the current vertex in encoding, for example, can include prediction mode identifier, reference vertex identifier, etc., which is not limited in the present application. When it is determined according to the reference description information parsed from the bitstream that difference prediction is needed, the previous decoded vertex can be determined as the reference vertex of the current vertex (or, the reference vertex of the current vertex is determined according to the reference description information; or, the reference vertex of the current vertex is determined according to the pre-agreed rule); and then the first reconstructed coordinates of the reference vertex are superposed with the residual information to obtain the first reconstructed coordinates of the current vertex.

[0203] For example, when it is determined according to the reference description information parsed from the bitstream that the neighbor weighted prediction needs to be performed, the reference vertex of the current vertex can be determined according to the reference description information (wherein the reference vertex of the current vertex is a neighboring vertex of the current vertex; of course, the reference vertex of the current vertex can also be determined according to a pre-agreed rule). When the reference vertex is one, the first reconstruction coordinate of the reference vertex can be superimposed with the residual information to obtain the first reconstruction coordinate of the current vertex; when the reference vertex is multiple, the coordinate mean of the first reconstruction coordinates of the multiple reference vertices can be superimposed with the residual information to obtain the first reconstruction coordinate of the current vertex.

[0204] For example, when it is determined according to the reference description information parsed from the bitstream that the spatio-temporal dual prediction needs to be performed, the reference vertex of the current vertex and the neighboring vertex of the current vertex can be determined based on the reference description information. Then, for one neighboring vertex, the residual calculation is performed on the first reconstruction coordinate of the neighboring vertex and the first reconstruction coordinate of the neighboring reference vertex to obtain the second MV of the neighboring vertex. When the neighboring vertex is one, the second MV of the neighboring vertex, the residual information and the first reconstruction coordinate of the reference vertex of the current vertex can be superimposed to obtain the first reconstruction coordinate of the current vertex. When the neighboring vertex is multiple, the MV mean of the second MVs of the multiple neighboring vertices can be calculated first, and then the residual information, the mean MV and the first reconstruction coordinate of the reference vertex of the current vertex can be superimposed to obtain the first reconstruction coordinate of the current vertex.

[0205] It should be understood that other spatial prediction or spatio-temporal dual prediction modes can also be included, which correspond to the encoding process, and the present application does not limit this.

[0206] S305, scaling the first reconstruction coordinate based on the scaling parameter corresponding to the current vertex to obtain the second reconstruction coordinate of the current vertex.

[0207] For example, after the scaling parameter corresponding to the current vertex and the first reconstruction coordinate of the current vertex are obtained, the first reconstruction coordinate of the current vertex can be divided by the scaling parameter corresponding to the current vertex to realize the scaling of the first reconstruction coordinate of the current vertex, and thus the second reconstruction coordinate can be obtained.

[0208] For example, assuming that the first reconstruction coordinate is (x3, y3, z3) and the scaling parameter is k; then the vertex coordinate is scaled based on the scaling parameter k: x4=x3 / k, y4=y3 / k, z4=z3 / k, and the obtained second reconstruction coordinate can be (x4, y4, z4).

[0209] It should be understood that S301-S305 are steps performed in the decoding process. After S301-S305 are executed, the decoding of the vertex coordinate of the current vertex is completed.

[0210] Figure 3b A block diagram of a framework of the decoding module is shown for illustration.

[0211] With reference to Figure 3b , the encoding module includes, for example, an entropy decoding module, an inverse quantization module, an inverse transform module, a prediction and compensation module, and a post-processing module. The prediction and compensation module can be implemented as one module or can be split into a prediction module and a compensation module, i.e., implemented as two separate modules. It should be understood that Figure 3b This is only one example of the present application, and the decoding module of the present application can include more or fewer modules than Figure 3b shown, which is not limited in the present application.

[0212] In combination Figure 3a and Figure 3b , for example, after receiving the code stream, the entropy decoding module can entropy decode the code stream to obtain second data and output the second data to the inverse quantization module. Then, the inverse quantization module performs inverse quantization on the second data according to a preset quantization parameter to obtain third data and output the third data to the inverse transform module. Subsequently, the inverse transform module performs inverse transform on the third data to parse out residual information corresponding to the current module and output the residual information corresponding to the current module to the prediction and compensation module. Then, the prediction and compensation module can perform prediction and compensation based on the residual information corresponding to the current vertex to obtain first reconstructed coordinates of the current vertex and output the first reconstructed coordinates to the post-processing module. Subsequently, the post-processing module can obtain a scaling parameter corresponding to the current vertex, and then scale the first reconstructed coordinates based on the scaling parameter corresponding to the current vertex to obtain second reconstructed coordinates of the current vertex.

[0213] It should be noted that in the encoding process of the data of the 3D mesh, in addition to encoding the vertex coordinates in the data of the 3D mesh, the texture coordinates, the vertex connection relationship, and the texture map in the data of the 3D mesh can also be encoded. Correspondingly, in the decoding process of the data of the 3D mesh, in addition to decoding the vertex coordinates in the data of the 3D mesh to obtain the second reconstructed coordinates, the texture coordinates in the data of the 3D mesh can also be decoded to obtain reconstructed texture coordinates, and the vertex connection relationship can be decoded, and the texture map can be decoded to obtain a reconstructed texture map.

[0214] For example, the texture coordinates in the data of the 3D mesh can be encoded and decoded according to the encoding and decoding method of the vertex coordinates described above.

[0215] Exemplarily, an EdgeBreaker algorithm, a TFAN algorithm, or the like can be used to encode and decode the vertex connection relationship in the 3D mesh data.

[0216] Exemplarily, a video / image encoding algorithm can be used to encode and decode the texture map.

[0217] It should be understood that the application does not limit the encoding and decoding methods of the texture coordinates, the vertex connection relationship, and the texture map in the 3D mesh data.

[0218] In this way, after the second reconstructed coordinates of the vertices, the reconstructed texture coordinates, the vertex connection relationship, and the reconstructed texture map are decoded, a reconstructed 3D mesh without texture can be generated based on the vertex connection relationship and the reconstructed coordinates of the vertices. Then, texture mapping can be performed based on the reconstructed texture coordinates and the reconstructed texture map, and the texture map of each triangular facet in the reconstructed 3D mesh without texture can be performed, so that a reconstructed 3D mesh with texture can be obtained. After a plurality of reconstructed 3D meshes with texture are obtained, the plurality of reconstructed 3D meshes with texture can constitute a volumetric video, that is, the second electronic device can decode the volumetric video sent by the first electronic device.

[0219] Exemplarily, the importance information of the current vertex on the 3D mesh can be obtained, and the scaling parameter corresponding to the current vertex can be determined based on the importance information of the current vertex on the 3D mesh and a preset selection condition.

[0220] In a possible manner, a Region Of Interest (ROI) detection algorithm can be used to detect the importance information of each vertex in the 3D mesh. In this way, the importance information of the current vertex on the 3D mesh determined by the ROI detection algorithm can be obtained by the encoder.

[0221] Exemplarily, the above Figure 1 (2) During the generation of the 3D mesh data, the point cloud granularity corresponding to different image regions in the image of the collected video data can be adjusted according to the importance of the different image regions. In this way, after the point cloud is generated, the image region with high importance (for example, the face region of a person) in the image of the collected video data can correspond to more point clouds, and after the mesh is generated, the region corresponding to the image region with high importance (for example, the face region of a person on the 3D mesh, such as Figure 4a as shown in Figure 4aFor the face region of the 3D mesh (as shown in the elliptical region), the corresponding triangular patches can be smaller and denser. After point cloudization, the image regions of the captured video data that are less important (such as the body region of the person) can correspond to a less important point cloud, and after meshing, the regions of the 3D mesh corresponding to the image regions of the captured video data that are less important (such as the body region of the 3D mesh of the person, as shown in the rectangular region) can correspond to larger triangular patches. In this way, the amount of data corresponding to the 3D mesh can be reduced, and the code rate can be reduced to a certain extent. Figure 4a Figure 4a For the face region of the 3D mesh (as shown in the elliptical region), the corresponding triangular patches can be smaller and denser. After point cloudization, the image regions of the captured video data that are less important (such as the body region of the person) can correspond to a less important point cloud, and after meshing, the regions of the 3D mesh corresponding to the image regions of the captured video data that are less important (such as the body region of the 3D mesh of the person, as shown in the rectangular region) can correspond to larger triangular patches. In this way, the amount of data corresponding to the 3D mesh can be reduced, and the code rate can be reduced to a certain extent.

[0222] That is, for the 3D mesh, the higher the vertex density of the region where the vertex is located, the more important the vertex is; conversely, the lower the vertex density of the region where the vertex is located, the less important the vertex is. In one possible manner, the vertex density information of the region where the current vertex is located can be obtained, and the vertex density information can be used to represent the importance of the vertex on the 3D mesh; subsequently, the scaling parameter corresponding to the current vertex can be determined based on the vertex density of the region where the current vertex is located.

[0223] It should be understood that the present application does not limit the manner of obtaining the importance information of the current vertex on the 3D mesh. The present application takes obtaining the vertex density information of the region where the current vertex is located and determining the importance information of the current vertex on the 3D mesh based on the vertex density information of the region where the current vertex is located as an example for illustration.

[0224] The manner of obtaining the vertex density information can include various manners. The different manners of obtaining the vertex density information of the region where the current vertex is located by the encoding process will be described below.

[0225] Figure 4b An example encoding process is shown in the figure. In the embodiment of the present application, the vertex density information is determined based on the area of the reconstructed triangular patch. Figure 4b

[0226] S401, obtaining the data of a three-dimensional mesh 3D mesh, the 3D mesh being composed of a plurality of triangular patches, each triangular patch including three vertices, and the data of the 3D mesh including the vertex coordinates of a plurality of vertices.

[0227] For example, after obtaining the data of the 3D mesh, the connection relationship of the vertices in the data of the 3D mesh can be encoded first.

[0228] ​​For example, the vertex connection relationship in the data of the 3D mesh is encoded by using the Edgebreaker algorithm: an arbitrary triangle is selected as an initial boundary, and an arbitrary edge is selected as a current edge; then, five operators (also referred to as state symbols) C, L, E, R and S are used to record the topological relationship between the current triangle and the boundary. C represents that the third vertex is not on the boundary; L and R represent that the third vertex is on the boundary, and the current triangle has an edge (e) on the boundary in addition to the current edge; L and R represent that e is in different directions of the current edge; S is used to divide the graph into two parts, and additional offset or other operations are needed to record the branch information; and E represents that the three edges of the triangle are on the boundary. In this way, after all the triangles are traversed, a traversal result (for example, CCRCRSERCSCRRCRRRERCRCRCRRRLLRL (the last E operation can be omitted)) can be obtained; then, the traversal result is compressed by using Huffman coding to obtain a code stream. In the process of traversing the triangles, all the vertices are also traversed, and thus a vertex traversal order can be obtained.

[0229] It should be understood that, regardless of the algorithm used to encode the vertex connection relationship, all the vertices can be traversed to obtain a vertex traversal order.

[0230] Then, the encoding order of the plurality of vertices in the data of the 3D mesh can be determined according to the vertex traversal order determined in the process of encoding the vertex connection relationship; the encoding order is the same as the vertex traversal order. Subsequently, each vertex of the plurality of vertices in the data of the 3D mesh can be sequentially determined as a current vertex according to the encoding order, and then the vertex coordinates of the current vertex are encoded.

[0231] It should be understood that the encoding order of the texture coordinates, the vertex coordinates and the texture map in the data of the 3D mesh can be set according to requirements, and the application does not limit the encoding order of the three.

[0232] S402, one or more first adjacent triangles are determined, the first adjacent triangle being a triangle with the current vertex as a vertex.

[0233] S403, one or more second adjacent triangles are determined based on the one or more first adjacent triangles, the second adjacent triangle being a triangle sharing the reconstructed vertex with the first adjacent triangle of the current vertex.

[0234] Exemplarily, after encoding the vertex coordinate of each vertex, the encoding module can also reconstruct the first reconstructed coordinate of the vertex; then, based on the scaling parameter corresponding to the vertex, the first reconstructed coordinate is scaled to reconstruct the second reconstructed coordinate of the vertex. After the second reconstructed coordinates of the vertices of the triangular patch are reconstructed, the triangular patch is also reconstructed.

[0235] Exemplarily, the triangular patch with the current vertex as the vertex (hereinafter referred to as the first adjacent triangular patch) can be determined based on the indexes of other vertices, the index of the current vertex and the vertex connection relationship. Then, the first adjacent triangular patch containing the reconstructed vertex can be selected from the first adjacent triangular patch; then, based on the second reconstructed coordinates of the reconstructed vertices contained in the first adjacent triangular patch and the vertex connection relationship, the second adjacent triangular patch is determined from the reconstructed triangular patch; wherein the second adjacent triangular patch refers to the triangular patch sharing the reconstructed vertex with the first adjacent triangular patch of the current vertex.

[0236] Figure 4c A schematic diagram of part of the triangular patches in the exemplarily shown 3D mesh. Referring to Figure 4c , assuming that the current vertex is A, if △BOC, △COD, △DOE and △EOF are reconstructed triangular patches, the first adjacent triangular patch includes △AOB and △AOF, then the second adjacent triangular patch includes: △BOC, △COD, △DOE and △EOF. If △BOC and △COD are reconstructed triangular patches, the first adjacent triangular patch includes △AOB and △AOF, then the second adjacent triangular patch includes: △BOC and △COD.

[0237] S404, based on the second reconstructed coordinates of the vertices contained in one or more second adjacent triangular patches, the area of the one or more second adjacent triangular patches is determined.

[0238] S405, based on the area of one or more second adjacent triangular patches, the vertex density information is determined.

[0239] In one possible way, the area of all reconstructed triangular patches can be determined based on the second reconstructed coordinates of the reconstructed vertices; and the area S1 max of the reconstructed triangular patch with the largest area and the area S1 min of the reconstructed triangular patch with the smallest area. Then, based on the second reconstructed coordinates of the reconstructed vertices, the area of the second adjacent triangular patch is determined; then, based on the area of the second adjacent triangular patch, the area S1 max of the reconstructed triangular patch with the largest area and the area S1 min of the reconstructed triangular patch with the smallest area, the vertex density information is determined.

[0240] In one possible implementation, the vertex density information p can be determined according to the following formula (1):

[0241] p = (S - S1 min ) / (S1 max - S1 min ) (1)

[0242] wherein S is the area of the second adjacent triangle when the number of the second adjacent triangle is one; and S is the average of the areas of the second adjacent triangles when the number of the second adjacent triangles is more than one.

[0243] In one possible implementation, the areas of all the triangles included in the 3D mesh can be determined based on the vertex coordinates of the vertices included in the data of the 3D mesh; then the area S2 max of the triangle with the largest area and the area S2 min of the triangle with the smallest area can be selected based on the areas of all the triangles included in the 3D mesh. Next, S2 max and S2 min are updated based on the areas of the second adjacent triangles and S2 max and S2 min , which can be determined according to the following formula (2):

[0244] S3 max = max(S2 max , S), S3 min = min(S2 min , S) (2)

[0245] wherein S is the area of the second adjacent triangle when the number of the second adjacent triangle is one; and S is the average of the areas of the second adjacent triangles when the number of the second adjacent triangles is more than one.

[0246] Next, the vertex density information is determined based on the areas of the second adjacent triangles, S3 max and S3 min .

[0247] In one possible implementation, the vertex density information p can be determined according to the following formula (3):

[0248] p = (S - S3 min ) / (S3 max - S3 min ) (3)

[0249] When the number of the second adjacent triangle patches is one, S is the area of the second adjacent triangle patch; when the second adjacent triangle patches are multiple, S is the average value of the areas of the second adjacent triangle patches.

[0250] Since the number of the reconstructed vertex and the reconstructed triangle patch is small at the beginning of encoding, the vertex density information calculated by formula (3) is more accurate and has smaller error compared with formula (1).

[0251] It should be noted that when formula (3) is used to calculate the vertex density information of the current vertex, S2 max and S2 min are written into the bitstream; so that the decoding end can also determine the vertex density information of the current vertex based on S2 max and S2 min parsed from the bitstream.

[0252] S406, based on the vertex density information and the preset selection condition, determines the scaling parameter corresponding to the current vertex.

[0253] For example, the preset selection condition can include a mapping relationship between the preset vertex density information and the corresponding preset scaling parameter; wherein the mapping relationship between the preset vertex density information and the corresponding preset scaling parameter can be set according to requirements, which is not limited in the present application. For example, the preset selection condition can include: {py<0.2, SF=100} (py is the preset vertex density information, SF is the preset scaling parameter; wherein when the preset vertex density information is less than 0.2, the preset scaling parameter is 100), {0.2≤py<0.8, SF=50} (when the preset vertex density information is greater than or equal to 0.2 and less than 0.8, the preset scaling parameter is 50), {0.8≤py, SF=25} (when the preset vertex density information is less than or equal to 0.8, the preset scaling parameter is 25).

[0254] In this way, the preset scaling parameter matched with the vertex density information can be selected as the scaling parameter corresponding to the current vertex according to the preset selection condition.

[0255] In one possible way, if the frame 3D mesh where the current vertex is located is a key frame, the scaling parameter of the current vertex can be determined in the manner of S402-S406. If the frame 3D mesh where the current vertex is located is a non-key frame, the scaling parameter of the vertex corresponding to the current vertex (which can be determined according to the tracking result) in the last encoded 3D mesh can be determined as the scaling parameter of the current vertex. In this way, the calculation amount can be reduced.

[0256] It should be understood that whether the frame 3D mesh in which the current vertex is located is a key frame or not, the scaling parameter of the current vertex can be determined in the manner of S402-S406.

[0257] S407, based on the scaling parameter corresponding to the current vertex, scaling the vertex coordinates of the current vertex to obtain the scaled vertex coordinates of the current vertex.

[0258] For example, since the vertex coordinates are floating-point numbers, after S407 is executed, a floating-point rounding operation can be performed to convert the floating-point numbers to integers, facilitating subsequent encoding of the vertex coordinates. It should be understood that the floating-point rounding operation is an optional parameter and can be performed by the pre-processing module in S201. Figure 2b

[0259] S408, based on the scaling parameter and the scaled vertex coordinates of the current vertex, performing prediction and compensation to obtain the residual information corresponding to the current vertex.

[0260] For example, the reference vertex of the current vertex can be predicted; for details, reference can be made to the description of S204 above, which will not be repeated here.

[0261] For example, the scaling parameter corresponding to the reference vertex of the current vertex can be the same as or different from the scaling parameter corresponding to the current vertex; when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex of the current vertex, the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex are not in the same order of magnitude. When the current vertex is a point with high importance, if the residual is directly calculated based on the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex, the residual will be reduced, the subsequent quantization error will be increased, and the reconstruction quality will be reduced. When the current vertex is a point with low importance, if the residual is directly calculated based on the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex, the residual will be increased, and the compression code rate will be increased.

[0262] Further, in the process of compensation, when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex of the current vertex, the scaling parameters corresponding to the current vertex and the reference vertex can be normalized, that is, the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex are normalized to be consistent. In this way, the order of magnitude difference between the scaled vertex coordinates of the current vertex and the scaled vertex coordinates of the reference vertex can be reduced, and the reconstruction quality or the code rate overhead can be improved.

[0263] ​Exemplarily, the normalized scaling parameter can be determined based on the scaling parameter corresponding to the current vertex and / or the scaling parameter corresponding to the reference vertex; then, the normalized coordinate of the current vertex and the normalized coordinate of the reference vertex can be determined based on the normalized scaling parameter; and then, the residual information corresponding to the current vertex can be determined based on the normalized coordinate of the current vertex and the normalized coordinate of the reference vertex.

[0264] In a possible manner, the scaling parameter corresponding to the current vertex can be determined as the normalized scaling parameter; at this time, the scaled vertex coordinate of the current vertex does not need to be scaled (or scaled by 1 times), and only the scaled vertex coordinate of the reference vertex needs to be scaled. At this time, the normalized coordinate of the current vertex determined based on the normalized scaling parameter is the scaled vertex coordinate of the current vertex; and the normalized coordinate of the reference vertex can be obtained by scaling the scaled vertex coordinate of the reference vertex based on the normalized scaling parameter.

[0265] Figure 4d A schematic diagram of part of the triangular facets in the exemplary 3D mesh is shown in FIG. 1. Figure 4d In FIG. 1, A is a current vertex, the scaling parameter of A is 50, the scaled vertex coordinate of A is (x1, y1, z1), B is a neighboring vertex of A, the scaling parameter of B is 25, and the scaled vertex coordinate of B is (x2, y2, z2). Assuming that the difference prediction is used and B is the last coded vertex of A, the scaled vertex coordinate of B can be enlarged by 2 times to obtain the normalized coordinate of B, which is (2*x2, 2*y2, 2*z2).

[0266] Then, the residual information corresponding to the current vertex is determined based on the scaled vertex coordinate of the current vertex and the normalized coordinate of the reference vertex; for details, refer to the description above, which will not be repeated here.

[0267] It should be noted that when the spatio-temporal dual prediction manner is used for prediction, the scaling parameters corresponding to the neighboring vertex and the scaling parameters corresponding to the neighboring reference vertex can also be normalized in the above manner. Then, the normalized scaling parameters are determined based on the scaling parameters corresponding to the neighboring vertex and the scaling parameters corresponding to the neighboring reference vertex, and the normalized coordinates of the neighboring vertex and the normalized coordinates of the neighboring reference vertex are determined based on the normalized scaling parameters. Then, the second MV of the neighboring vertex is determined. Details are not described herein again. In addition, after the first MV of the current vertex and the second MV of the neighboring vertex are obtained, the first MV of the current vertex and the second MV of the neighboring vertex can be scaled based on the normalized scaling parameters determined based on the scaling parameters corresponding to the current vertex and the scaling parameters corresponding to the neighboring vertex, and then residual calculation is performed to obtain residual information. In addition, after the MV average of the first MV of the current vertex and the second MV of the neighboring vertex is obtained, the MV average of the first MV of the current vertex and the second MV of the neighboring vertex can be scaled based on the normalized scaling parameters determined based on the scaling parameters corresponding to the current vertex and the scaling parameters corresponding to the neighboring vertex, and then residual calculation is performed to obtain residual information.

[0268] In one possible manner, the maximum scaling parameter in the scaling parameters corresponding to the current vertex and the scaling parameters corresponding to the reference vertex can be determined as the normalized scaling parameter.

[0269] In this way, if the scaling parameter of the current vertex is the maximum scaling parameter, the scaled vertex coordinates of the reference vertex need to be scaled, and thus the quantization error of the vertex coordinates of the current vertex can be reduced, and the accuracy of the vertex coordinates of the current vertex can be improved. If the scaling parameter of the reference vertex is the maximum scaling parameter, it indicates that the current vertex and the reference vertex are located at the boundary of the important region of the 3D mesh, and scaling the scaled vertex coordinates of the current vertex can also reduce the quantization error of the vertex coordinates of the current vertex, improve the accuracy of the vertex coordinates of the current vertex, and thus improve the reconstruction quality at the boundary of the important region.

[0270] S409, the residual information of the current vertex is encoded to obtain a bitstream.

[0271] For example, S409 can refer to the description of S205 above, and details are not described herein again.

[0272] Figure 4e An example of a vertex coordinate encoding process is shown in the schematic diagram.

[0273] In combination Figure 4a and Figure 4eFor example, the prior information can be vertex coordinates of vertices included in data of the 3D mesh, or second reconstruction coordinates of the reconstructed vertices. In the encoding process, the scaling parameter can be determined according to the prior information (for reference can be made to S402-S406), and then the vertex coordinates of the vertices are scaled based on the scaling parameter to obtain scaled vertex coordinates of the vertices; then, the scaled vertex coordinates are predicted and compensated based on the scaling parameter to obtain residual information; subsequently, the residual information is processed in sequence, such as transformation, quantization, entropy encoding, and the like, and the code stream can be obtained.

[0274] It should be noted that the step of writing the scaling parameter into the code stream is an optional step.

[0275] Figure 5a The decoding process is schematically shown. Figure 5a The decoding process corresponds to the encoding process of Figure 4a The decoding process corresponds to the encoding process of

[0276] S501, receiving a code stream.

[0277] S502, parsing residual information of a current vertex from the code stream.

[0278] For example, S501-S502 can refer to the description of S301-S302 described above, and will not be described here.

[0279] For example, the received code stream can include code stream corresponding to the vertex connection relationship, code stream corresponding to the vertex coordinates, code stream corresponding to the texture map, and code stream corresponding to the texture coordinates.

[0280] For example, the vertex connection relationship can be parsed from the code stream corresponding to the vertex connection relationship, and the vertex traversal order can be determined in the process of decoding the vertex connection relationship. Then, the vertex traversal order can be determined as the decoding order; after that, the code stream corresponding to the vertex coordinates of each vertex can be decoded in sequence according to the decoding order to obtain the second reconstruction coordinates of each vertex.

[0281] In addition, the reconstructed texture map can be parsed from the code stream corresponding to the texture map, and the reconstructed texture coordinates can be parsed from the code stream corresponding to the texture coordinates. It should be understood that the order of decoding the code stream corresponding to the vertex coordinates, the code stream corresponding to the texture map, and the code stream corresponding to the texture coordinates is not limited in the present application.

[0282] S503, determining one or more first adjacent triangular facets, the first adjacent triangular facet being a triangular facet with the current vertex as a vertex.

[0283] S504, determining one or more second neighboring triangles based on the one or more first neighboring triangles, the second neighboring triangles referring to the triangles that share the reconstructed vertex with the first neighboring triangles of the current vertex.

[0284] S505, determining areas of the one or more second neighboring triangles based on the second reconstructed coordinates of the vertices comprised in the one or more second neighboring triangles.

[0285] Exemplarily, S503-S505 can refer to the descriptions of S402-S404 above, and will not be repeated here.

[0286] S506, determining the vertex density information based on the areas of the one or more second neighboring triangles.

[0287] In one possible manner, when the encoding end determines the vertex density information of the current vertex according to the formula (1) above, the decoding end can also determine the vertex density information of the current vertex according to the formula (1) above, and can refer to the descriptions above, and will not be repeated here.

[0288] In one possible manner, when the encoding end determines the vertex density information according to the formula (3) above, and writes S2 max and S2 min into the bitstream, the decoding end can parse S2 max and S2 min from the bitstream. Then, the area S of the second neighboring triangle (when the second neighboring triangle is multiple, S is the average of the areas of the multiple second neighboring triangles) can be determined; and then, S2 max and S2 min can be updated based on the area S of the second neighboring triangle to determine S3 max and S3 min , which can refer to the descriptions of the formula (2) above. Subsequently, the vertex density information p of the current vertex can be determined according to the formula (3) above, and will not be repeated here.

[0289] S507, determining the scaling parameter corresponding to the current vertex based on the vertex density information and a preset selection condition.

[0290] Exemplarily, S507 can refer to the descriptions of S406 above, and will not be repeated here.

[0291] S508, predicting and compensating the residual information of the current vertex based on the scaling parameter corresponding to the current vertex to obtain the first reconstructed coordinates of the current vertex.

[0292] Exemplarily, when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex of the current vertex, S508 can include: S5081-S5085.

[0293] S5081, predicting the reference vertex of the current vertex.

[0294] Exemplarily, S5081 can refer to the description of S304 described above, and details are not described herein again.

[0295] S5082, determining a normalized scaling parameter based on the scaling parameter corresponding to the current vertex and / or the scaling parameter corresponding to the reference vertex.

[0296] In a possible manner, the scaling parameter corresponding to the current vertex can be determined as the normalized scaling parameter.

[0297] In a possible manner, the maximum scaling parameter in the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex can be determined as the normalized scaling parameter.

[0298] It should be understood that the manner of determining the normalized scaling parameter in the decoding process is the same as the manner of determining the normalized scaling parameter in the encoding process. Exemplarily, the first electronic device and the second electronic device can agree on the manner of determining the normalized scaling parameter in advance; or the first electronic device can write the manner of determining the normalized scaling parameter into the code stream, so that the second electronic device can parse the manner of determining the normalized scaling parameter from the code stream.

[0299] S5083, scaling the second reconstructed coordinate corresponding to the reference vertex based on the normalized scaling parameter to obtain the normalized coordinate corresponding to the reference vertex.

[0300] For example, A is the current vertex, the scaling parameter of A is 50, B is a neighboring vertex of A, the scaling parameter of B is 25, and the second reconstructed coordinate of B is (x2, y2, z2). Assuming that the difference prediction is used, and B is the last encoded vertex of A, the second vertex coordinate of B can be scaled by 50 times to obtain the normalized coordinate of B, which is (50*x2, 50*y2, 50*z2).

[0301] S5084, compensating the normalized coordinate corresponding to the reference vertex based on the residual information corresponding to the current vertex to obtain the normalized coordinate corresponding to the current vertex.

[0302] Exemplarily, S5084 can refer to the description of S304 in the above embodiment, and details are not described herein again. Figure 3a

[0303] ​S5085, scale the normalized coordinate corresponding to the current vertex based on the normalized scaling parameter, to obtain the first reconstruction coordinate corresponding to the current vertex.

[0304] For example, if the scaling parameter corresponding to the current vertex is determined as the normalized scaling parameter in the encoding process, the normalized coordinate corresponding to the current vertex is scaled by 1 times, and the normalized coordinate is the first reconstruction coordinate at this time. Of course, S5084 can also not be performed.

[0305] If the scaling parameter corresponding to the reference vertex is determined as the normalized scaling parameter in the encoding process, the normalized coordinate corresponding to the current vertex can be divided by the normalized scaling parameter to obtain the first reconstruction coordinate of the current vertex.

[0306] S509, scale the first reconstruction coordinate based on the scaling parameter corresponding to the current vertex, to obtain the second reconstruction coordinate of the current vertex.

[0307] For example, S509 can refer to the description of S305 described above, and will not be described here.

[0308] It should be noted that, optionally, the first electronic device can encode the scaling parameter corresponding to the current vertex, and write the corresponding encoded data into the bitstream; subsequently, the second electronic device can directly parse the scaling parameter corresponding to the current vertex from the bitstream, so that the decoding power of the second electronic device can be saved.

[0309] It should be noted that, optionally, the first electronic device can encode the vertex density information of the region where the current vertex is located, and write the corresponding encoded data into the bitstream; subsequently, the second electronic device can directly parse the vertex density information of the region where the current vertex is located from the bitstream, so that the decoding power of the second electronic device can also be saved.

[0310] Figure 5b For example, the decoding process is schematically shown.

[0311] In combination with Figure 5a and Figure 5b For example, the prior information can refer to the second reconstruction coordinate of the reconstructed vertex. In the decoding process, the bitstream can be entropy decoded, dequantized and inverse transformed to obtain the residual information. In one possible way, the scaling parameter can be parsed from the bitstream; in one possible way, the scaling parameter can be determined based on the prior information. Then, the first reconstruction coordinate of the vertex can be obtained based on the scaling parameter and the residual information. Subsequently, the first reconstruction coordinate is scaled based on the scaling parameter to obtain the second reconstruction coordinate of the vertex.

[0312] Figure 5cAn exemplary vertex coordinate compression process diagram.

[0313] In combination Figure 4a and Figure 5a , the parameter Figure 5c , in an exemplary encoding process at the encoding end, the vertex coordinates of the vertices can be scaled, predicted and compensated, transformed, quantized, entropy encoded, etc., and the code stream can be obtained and sent. In a decoding process at the decoding end, the code stream can be entropy decoded, dequantized, inverse transformed, predicted and compensated, and scaled, etc., and the second reconstructed coordinates of the vertices can be obtained.

[0314] Figure 6a An exemplary encoding process diagram is shown. In Figure 6a , the vertex density information is determined based on the areas of the original triangular patches (i.e., the triangular patches included in the 3D mesh to be encoded).

[0315] S601, obtain data of a three-dimensional mesh 3D mesh, the 3D mesh is composed of multiple triangular patches, each triangular patch includes three vertices, and the data of the 3D mesh includes vertex coordinates of multiple vertices.

[0316] Exemplarily, S601 can refer to the description of S201 described above, and will not be repeated here.

[0317] S602, determine one or more first adjacent triangular patches, the first adjacent triangular patch refers to a triangular patch with the current vertex as a vertex.

[0318] Exemplarily, the triangular patch with the current vertex as a vertex (hereinafter referred to as the first adjacent triangular patch) can be determined based on the index of the current vertex, the index of the other vertex, and the vertex connection relationship.

[0319] Again referring to Figure 4c , assuming that the current vertex is O, then △BOC, △COD, △EOF, △BOC, △COD, △EOF are all first adjacent triangular patches.

[0320] S603, determine the area of one or more first adjacent triangular patches based on the vertex coordinates of the vertices included in the one or more first adjacent triangular patches.

[0321] Exemplarily, the vertex coordinates of the vertices included in the one or more first adjacent triangular patches can be determined from the data of the 3D mesh (i.e., the original data); then, the area of the one or more first adjacent triangular patches based on the vertex coordinates of the vertices included in the one or more first adjacent triangular patches.

[0322] S604, determine vertex density information based on the area of ​​one or more first neighboring triangular facets.

[0323] For example, the area of ​​all triangular faces contained in the 3D mesh can be determined based on the vertex coordinates of the vertices contained in the 3D mesh; then, based on the areas of all triangular faces contained in the 3D mesh, the area S2 of the triangular face with the largest area is selected. max The area S2 of the smallest triangular facet. min Next, based on the area of ​​the first neighboring triangular facet, the area S2 of the largest triangular facet is calculated. max The area S2 of the smallest triangular facet. min This determines the vertex density information. One possible approach is to refer to the description in formula (4) below:

[0324] p = (S - S2) min ) / (S2 max -S2 min (4)

[0325] Wherein, when there is one first neighboring triangular facet, S is the area of ​​the first neighboring triangular facet; when there are multiple first neighboring triangular facets, S is the average area of ​​the first neighboring triangular facets.

[0326] S605 determines the scaling parameters corresponding to the current vertex based on vertex density information and preset selection conditions.

[0327] S606: Based on the scaling parameters corresponding to the current vertex, scale the vertex coordinates of the current vertex to obtain the scaled vertex coordinates of the current vertex.

[0328] S607, based on the scaling parameters and the scaled vertex coordinates of the current vertex, performs prediction and compensation to obtain the residual information corresponding to the current vertex.

[0329] S608 encodes the residual information of the current vertex to obtain the bitstream.

[0330] For example, S605 to S608 can be referred to the description of S405 to S408 above, and will not be repeated here.

[0331] S609 writes the scaling parameters corresponding to the current vertex into the bitstream.

[0332] For example, due to Figure 6aThe data used for determining the vertex density information in the embodiment of the present application comprises the vertex coordinates of the uncoded vertex, and therefore, in order to enable the second electronic device to obtain the scaling parameter corresponding to the current vertex, the scaling parameter corresponding to the current vertex can be coded; subsequently, the second electronic device can directly parse the scaling parameter corresponding to the current vertex from the bitstream.

[0333] In a possible manner, the scaling parameter corresponding to the current vertex can be variable-length coded. For example, the scaling parameter is (100, 50, 25), and the variable-length coding can represent (10, 0, 11) respectively.

[0334] In a possible manner, the scaling parameter corresponding to the current vertex can be fixed-length coded. For example, the scaling parameter is (100, 50, 25), and the fixed-length coding can represent (00, 10, 11) respectively.

[0335] It should be understood that the present application does not limit the manner of coding the scaling parameter corresponding to the current vertex.

[0336] It should be noted that the first electronic device can also code the vertex density information of the region where the current vertex is located, and write the corresponding coded data into the bitstream; subsequently, the second electronic device can directly parse the vertex density information of the region where the current vertex is located from the bitstream, so that the decoding computing power of the second electronic device can also be saved.

[0337] Figure 6b The encoding process of the vertex coordinates is schematically shown as an example.

[0338] In combination with Figure 6a and Figure 6b , the prior information can be the vertex coordinates of the vertices included in the data of the 3D mesh. In the encoding process, the scaling parameter can be determined according to the prior information (which can be referred to as S602-S605), and then the vertex coordinates of the vertices are scaled based on the scaling parameter to obtain the scaled vertex coordinates of the vertices; then, the scaled vertex coordinates are predicted and compensated based on the scaling parameter to obtain the residual information; subsequently, the residual information is sequentially processed by transformation, quantization, entropy coding and the like, and the bitstream can be obtained; and the scaling parameter can be written into the bitstream.

[0339] Figure 7a The decoding process is schematically shown as an example. Figure 7a The decoding process of the present application corresponds to the encoding process of the present application. Figure 6a

[0340] S701, receiving a bitstream.

[0341] S702, parsing the residual information corresponding to the current vertex and the scaling parameter corresponding to the current vertex from the bitstream.​

[0342] For example, after the first electronic device encodes the residual information of the current vertex and the scaling parameter corresponding to the current vertex, obtains the code stream, and sends the code stream to the second electronic device, the decoding module of the second electronic device can parse the residual information corresponding to the current vertex and the scaling parameter corresponding to the current vertex from the code stream.

[0343] In S703, the residual information of the current vertex is predicted and compensated based on the scaling parameter corresponding to the current vertex, to obtain a first reconstructed coordinate of the current vertex.

[0344] In S704, the first reconstructed coordinate is scaled based on the scaling parameter corresponding to the current vertex, to obtain a second reconstructed coordinate of the current vertex.

[0345] For example, S703-S704 can refer to the description of S508-S509 described above, and will not be described here.

[0346] Figure 7b The decoding process is schematically shown.

[0347] In combination with Figure 7a and Figure 7b For example, in the decoding process, the code stream can be entropy decoded, dequantized, and inverse transformed to obtain the residual information, and the scaling parameter can be parsed from the code stream. Then, the first reconstructed coordinate of the vertex can be obtained by predicting and compensating based on the scaling parameter and the residual information. Subsequently, the second reconstructed coordinate of the vertex can be obtained by scaling the first reconstructed coordinate based on the scaling parameter.

[0348] Figure 8 The encoding apparatus is schematically shown. The encoding apparatus can be used to execute the encoding method in the foregoing embodiments, and thus the beneficial effects that can be achieved by the encoding apparatus can refer to the beneficial effects provided in the corresponding method, which will not be described here.

[0349] In reference to Figure 8 For example, the encoding apparatus can include:

[0350] The data acquisition module 801 is configured to acquire data of a three-dimensional mesh 3D mesh, the 3D mesh being composed of a plurality of triangular facets, each triangular facet including three vertices, and the data of the 3D mesh including vertex coordinates of the plurality of vertices, the vertex coordinates being three-dimensional coordinates.

[0351] The information acquisition module 802 is configured to acquire a scaling parameter corresponding to a current vertex, the scaling parameter corresponding to the current vertex being positively correlated with an importance of the current vertex on the 3D mesh.

[0352] The scaling module 803 is configured to scale the vertex coordinates of the current vertex based on the scaling parameter corresponding to the current vertex, to obtain scaled vertex coordinates of the current vertex.

[0353] The prediction and compensation module 804 is configured to perform prediction and compensation based on the scaled vertex coordinates of the current vertex, to obtain residual information corresponding to the current vertex.

[0354] The encoding module 805 is configured to encode the residual information corresponding to the current vertex, to obtain a bitstream.

[0355] For example, the information acquisition module 802 is specifically configured to acquire importance information of the current vertex on the 3D mesh; and determine the scaling parameter corresponding to the current vertex based on the importance information of the current vertex on the 3D mesh and a preset selection condition.

[0356] For example, the information acquisition module 802 is specifically configured to acquire vertex density information of a region in which the current vertex is located; and determine the scaling parameter corresponding to the current vertex based on the vertex density information of the region in which the current vertex is located.

[0357] For example, the information acquisition module 802 is specifically configured to determine one or more first adjacent triangular facets, the first adjacent triangular facet being a triangular facet with the current vertex as a vertex; determine one or more second adjacent triangular facets based on the one or more first adjacent triangular facets, the second adjacent triangular facet being a triangular facet that shares a reconstructed vertex with the first adjacent triangular facet of the current vertex; determine areas of the one or more second adjacent triangular facets based on reconstructed coordinates of vertices included in the one or more second adjacent triangular facets; and determine the vertex density information based on the areas of the one or more second adjacent triangular facets.

[0358] For example, the information acquisition module 802 is specifically configured to determine one or more first adjacent triangular facets, the first adjacent triangular facet being a triangular facet with the current vertex as a vertex; determine areas of the one or more first adjacent triangular facets based on vertex coordinates of vertices included in the one or more first adjacent triangular facets; and determine the vertex density information based on the areas of the one or more first adjacent triangular facets.

[0359] For example, the information acquisition module 802 is specifically configured to determine an area of a triangular facet with a largest area and an area of a triangular facet with a smallest area based on vertex coordinates of a plurality of vertices included in data of the 3D mesh; and determine the vertex density information based on the areas of the one or more second adjacent triangular facets, the area of the triangular facet with the largest area, and the area of the triangular facet with the smallest area.

[0360] For example, the information obtaining module 802 is specifically configured to determine the area of the triangle patch with the largest area and the area of the triangle patch with the smallest area based on the reconstructed coordinates of the reconstructed vertex; and determine the vertex density information based on the area of the one or more second adjacent triangle patches, the area of the triangle patch with the largest area and the area of the triangle patch with the smallest area.

[0361] For example, the information obtaining module 802 is specifically configured to determine the area of the triangle patch with the largest area and the area of the triangle patch with the smallest area based on the vertex coordinates of the plurality of vertices included in the data of the 3D mesh; and determine the vertex density information based on the area of the one or more first adjacent triangle patches, the area of the triangle patch with the largest area and the area of the triangle patch with the smallest area.

[0362] For example, the prediction and compensation module 804 is specifically configured to perform prediction and compensation based on the scaling parameter and the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex.

[0363] For example, the prediction and compensation module 804 is specifically configured to predict a reference vertex of the current vertex; when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex, determine a normalized scaling parameter based on the scaling parameter corresponding to the current vertex and / or the scaling parameter corresponding to the reference vertex; determine the normalized coordinates corresponding to the current vertex and the normalized coordinates corresponding to the reference vertex based on the normalized scaling parameter; and perform compensation based on the normalized coordinates corresponding to the current vertex and the normalized coordinates corresponding to the reference vertex to determine the residual information corresponding to the current vertex.

[0364] For example, the prediction and compensation module 804 is specifically configured to determine the scaling parameter corresponding to the current vertex as the normalized scaling parameter.

[0365] For example, the prediction and compensation module 804 is specifically configured to determine the maximum scaling parameter in the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex as the normalized scaling parameter.

[0366] For example, the encoding module 805 is further configured to write the scaling parameter corresponding to the current vertex into the bitstream.

[0367] For example, the encoding module 805 is further configured to write the area of the triangle patch with the largest area and the area of the triangle patch with the smallest area into the bitstream.

[0368] Figure 9 A schematic diagram of a decoding apparatus is shown for example. The decoding apparatus can be used to perform the decoding method in the foregoing embodiments, and therefore, the beneficial effects that can be achieved by the decoding apparatus can refer to the beneficial effects provided in the corresponding method, which will not be described herein again.

[0369] With reference to Figure 9 , the decoding apparatus can include, for example:

[0370] The code stream receiving module 901 is configured to receive a code stream.

[0371] The decoding module 902 is configured to parse residual information corresponding to a current vertex from the code stream.

[0372] The information obtaining module 903 is configured to obtain a scaling parameter corresponding to the current vertex, the scaling parameter corresponding to the current vertex being positively correlated with an importance of the current vertex on the 3D mesh.

[0373] The prediction and compensation module 904 is configured to perform prediction and compensation based on the residual information corresponding to the current vertex to obtain a first reconstructed coordinate of the current vertex.

[0374] The scaling module 905 is configured to scale the first reconstructed coordinate based on the scaling parameter corresponding to the current vertex to obtain a second reconstructed coordinate of the current vertex.

[0375] For example, the information obtaining module 903 is specifically configured to obtain importance information of the current vertex on the 3D mesh; and determine the scaling parameter corresponding to the current vertex based on the importance information of the current vertex on the 3D mesh and a preset selection condition.

[0376] For example, the information obtaining module 903 is specifically configured to obtain vertex density information of a region in which the current vertex is located; and determine the scaling parameter corresponding to the current vertex based on the vertex density information of the region in which the current vertex is located.

[0377] For example, the information obtaining module 903 is specifically configured to determine one or more first adjacent triangular facets, the first adjacent triangular facet being a triangular facet with the current vertex as a vertex; determine one or more second adjacent triangular facets based on the one or more first adjacent triangular facets, the second adjacent triangular facet being a triangular facet that shares a reconstructed vertex with the first adjacent triangular facet of the current vertex; determine areas of the one or more second adjacent triangular facets based on second reconstructed coordinates of vertices included in the one or more second adjacent triangular facets; and determine the vertex density information based on the areas of the one or more second adjacent triangular facets.

[0378] For example, the information obtaining module 903 is specifically configured to parse an area of a triangular facet with a largest area and an area of a triangular facet with a smallest area from the code stream; and determine the vertex density information based on the areas of the one or more second adjacent triangular facets, the area of the triangular facet with the largest area, and the area of the triangular facet with the smallest area.

[0379] The information obtaining module 903 is configured to determine the area of the triangle patch with the largest area and the area of the triangle patch with the smallest area based on the second reconstructed coordinates of the reconstructed vertex.

[0380] The information obtaining module 903 is configured to parse the scaling parameter corresponding to the current vertex from the bitstream.

[0381] The prediction and compensation module 904 is configured to perform prediction and compensation based on the scaling parameter corresponding to the current vertex and the residual information corresponding to the current vertex to obtain the first reconstructed coordinate of the current vertex.

[0382] The prediction and compensation module 904 is configured to predict a reference vertex of the current vertex, determine a normalized scaling parameter based on the scaling parameter corresponding to the current vertex and / or the scaling parameter corresponding to the reference vertex when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex, scale the second reconstructed coordinate of the reference vertex based on the normalized scaling parameter to obtain a normalized coordinate of the reference vertex, perform compensation based on the normalized coordinate of the reference vertex and the residual information corresponding to the current vertex to obtain a normalized coordinate of the current vertex, and scale the normalized coordinate of the current vertex based on the normalized scaling parameter to obtain the first reconstructed coordinate of the current vertex.

[0383] The prediction and compensation module 904 is configured to determine the scaling parameter corresponding to the current vertex as the normalized scaling parameter.

[0384] The prediction and compensation module 904 is configured to determine the maximum scaling parameter between the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex as the normalized scaling parameter.

[0385] In one example, Figure 10 A schematic block diagram of an apparatus 1000 is shown, which can be used to implement the embodiments of the present application. The apparatus 1000 can include a processor 1001 and a transceiver / transmitting / receiving pin 1002, and optionally further include a memory 1003.

[0386] The various components of the apparatus 1000 are coupled each other through a bus 1004, which includes not only a data bus, but also a power supply bus, a control bus and a status signal bus. However, for the sake of clarity, all the buses are referred to as the bus 1004 in the figure.

[0387] Optionally, the memory 1003 can be configured to store instructions in the foregoing method embodiments. The processor 1001 can be configured to execute the instructions in the memory 1003, and control the receiving pin to receive the signal and the sending pin to send the signal.

[0388] The apparatus 1000 can be an electronic device or a chip of an electronic device in the foregoing method embodiments.

[0389] All relevant contents of each step involved in the foregoing method embodiments can be referred to the function description of the corresponding function module, and will not be repeated here.

[0390] The embodiment of the present application provides a chip, which comprises one or more interface circuits and one or more processors; the interface circuit is configured to receive a signal from a memory of an electronic device and send the signal to the processor, and the signal comprises computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the related method steps to implement the coding and decoding method in the foregoing embodiment. The interface circuit can be the transceiver / transceiver pin 1002.

[0391] The embodiment also provides a computer readable storage medium, which stores computer instructions, and when the computer instructions run on an electronic device, the electronic device executes the related method steps to implement the coding and decoding method in the foregoing embodiment.

[0392] The embodiment also provides a computer program product, and when the computer program product runs on a computer, the computer executes the related steps to implement the coding and decoding method in the foregoing embodiment.

[0393] In addition, the embodiment of the present application also provides an apparatus, which can be a chip, a component or a module in particular, and the apparatus can comprise a processor and a memory connected to each other; the memory is configured to store computer execution instructions, and when the apparatus runs, the processor can execute the computer execution instructions stored in the memory to make the chip execute the coding and decoding method in the foregoing method embodiments.

[0394] The electronic device, the computer readable storage medium, the computer program product or the chip provided in the embodiment can be used to execute the corresponding method provided in the foregoing, and the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding method provided in the foregoing, and will not be repeated here.

[0395] Through the description of the above embodiments, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0396] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another device, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0397] The units described as separate components can or can not be physically separated, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place or distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0398] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0399] Any content of each embodiment of the present application, and any content of the same embodiment, can be freely combined. Any combination of the above is within the scope of the present application.

[0400] If the integrated unit is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, includes several instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0401] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are only illustrative, not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope of protection of the claims, and all of them belong to the protection of the present application.

[0402] The steps of the method or algorithm described in combination with the disclosure of the embodiments of the present application can be implemented in hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a compact disc (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.

[0403] Those skilled in the art can understand that the functions described in the embodiments of the present application in the one or more examples above can be implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium. The computer readable medium includes computer readable storage medium and communication medium, wherein the communication medium includes any medium that facilitates transfer of a computer program from one place to another. The storage medium can be any available medium accessible by a general purpose or special purpose computer.

[0404] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the specific embodiments described above, which are merely illustrative rather than restrictive, and those of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims.

Claims

1. An encoding method characterized by comprising: The method comprises: obtaining data of a three-dimensional mesh 3D mesh; wherein the 3D mesh is composed of a plurality of triangular facets, each triangular facet comprises three vertices, and the data of the 3D mesh comprises vertex coordinates of a plurality of vertices, the vertex coordinates being three-dimensional coordinates; obtaining a scaling parameter corresponding to a current vertex, the scaling parameter corresponding to the current vertex being positively correlated with an importance degree of the current vertex on the 3D mesh; scaling the vertex coordinates of the current vertex based on the scaling parameter corresponding to the current vertex to obtain scaled vertex coordinates of the current vertex; performing prediction and compensation based on the scaled vertex coordinates of the current vertex to obtain residual information corresponding to the current vertex; encoding the residual information corresponding to the current vertex to obtain a bitstream.

2. The method of claim 1, wherein, The importance degree of the current vertex on the 3D mesh is represented by vertex density information of a region where the current vertex is located, and the obtaining of the scaling parameter corresponding to the current vertex comprises: obtaining vertex density information of a region where the current vertex is located; determining the scaling parameter corresponding to the current vertex based on the vertex density information of the region where the current vertex is located.

3. The method of claim 2, wherein, The obtaining of the vertex density information of the region where the current vertex is located comprises: determining one or more first adjacent triangular facets, the first adjacent triangular facet being a triangular facet with the current vertex as a vertex; determining one or more second adjacent triangular facets based on the one or more first adjacent triangular facets, the second adjacent triangular facet being a triangular facet sharing a reconstructed vertex with the first adjacent triangular facet of the current vertex; determining areas of the one or more second adjacent triangular facets based on reconstructed coordinates of vertices included in the one or more second adjacent triangular facets; determining the vertex density information based on the areas of the one or more second adjacent triangular facets.

4. The method of claim 2, wherein, The obtaining of the vertex density information of the region where the current vertex is located comprises: determining one or more first adjacent triangular facets, the first adjacent triangular facet being a triangular facet with the current vertex as a vertex; determining areas of the one or more first adjacent triangular facets based on vertex coordinates of vertices included in the one or more first adjacent triangular facets; determining the vertex density information based on the areas of the one or more first adjacent triangular facets.

5. The method of claim 3, wherein, The determining of the vertex density information based on the areas of the one or more second adjacent triangular facets comprises: determining an area of a triangular facet with a largest area and an area of a triangular facet with a smallest area based on the vertex coordinates of the plurality of vertices included in the data of the 3D mesh; determining the vertex density information based on the areas of the one or more second adjacent triangular facets, the area of the triangular facet with the largest area, and the area of the triangular facet with the smallest area.

6. The method of claim 3, wherein, The determining of the vertex density information based on the areas of the one or more second adjacent triangular facets comprises: determine, based on the area of the one or more first neighboring triangular patches, the area of the largest triangular patch, and the area of the smallest triangular patch, the vertex density information. The determining the vertex density information based on the area of the one or more first neighboring triangular patches includes:

7. The method of claim 4, wherein, determine, based on vertex coordinates of a plurality of vertices included in data of the 3D mesh, an area of a largest triangular patch and an area of a smallest triangular patch; determine, based on the area of the one or more first neighboring triangular patches, the area of the largest triangular patch, and the area of the smallest triangular patch, the vertex density information. The predicting and compensating based on the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex includes:

8. The method according to any one of claims 1 to 7, characterized in that, The predicting and compensating based on the scaling parameter and the scaled vertex coordinates of the current vertex to obtain the residual information corresponding to the current vertex includes: predict a reference vertex of the current vertex; 9. The method of claim 8, wherein, when the scaling parameter corresponding to the current vertex is different from the scaling parameter corresponding to the reference vertex, determine a normalized scaling parameter based on the scaling parameter corresponding to the current vertex and / or the scaling parameter corresponding to the reference vertex; determine normalized coordinates corresponding to the current vertex and normalized coordinates corresponding to the reference vertex based on the normalized scaling parameter; determine the residual information corresponding to the current vertex based on the normalized coordinates corresponding to the current vertex and the normalized coordinates corresponding to the reference vertex. The determining the normalized scaling parameter based on the scaling parameter corresponding to the current vertex or the scaling parameter corresponding to the reference vertex includes: determine the scaling parameter corresponding to the current vertex as the normalized scaling parameter.

10. The method of claim 9, wherein, The determining the normalized scaling parameter based on the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex includes: determine the maximum scaling parameter in the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex as the normalized scaling parameter.

11. The method of claim 9, wherein, The method further includes: write the scaling parameter corresponding to the current vertex into the bitstream.

12. The method according to any one of claims 1 to 7, characterized in that, The method further includes: write the area of the largest triangular patch and the area of the smallest triangular patch into the bitstream.

13. The method of claim 5 or 7, wherein, The method includes: receive a bitstream; 14. A decoding method, comprising: parse, from the bitstream, residual information corresponding to a current vertex; obtain a scaling parameter corresponding to the current vertex, the scaling parameter corresponding to the current vertex being positively correlated with importance of the current vertex on a 3D mesh; predict and compensate based on the residual information corresponding to the current vertex to obtain a first reconstructed coordinate of the current vertex; ​ ​ scaling the first reconstructed coordinate based on the scaling parameter corresponding to the current vertex to obtain a second reconstructed coordinate of the current vertex.

15. The method of claim 14, wherein, The importance of the current vertex on the 3D mesh is represented by vertex density information of a region where the current vertex is located, and the obtaining of the scaling parameter corresponding to the current vertex comprises: obtaining vertex density information of a region where the current vertex is located; determining the scaling parameter corresponding to the current vertex based on the vertex density information of the region where the current vertex is located.

16. The method of claim 15, wherein, The obtaining of the vertex density information of the region where the current vertex is located comprises: determining one or more first neighboring triangular facets, the first neighboring triangular facet being a triangular facet with the current vertex as a vertex; determining one or more second neighboring triangular facets based on the one or more first neighboring triangular facets, the second neighboring triangular facet being a triangular facet sharing a reconstructed vertex with the first neighboring triangular facet of the current vertex; determining areas of the one or more second neighboring triangular facets based on second reconstructed coordinates of vertices included in the one or more second neighboring triangular facets; determining the vertex density information based on the areas of the one or more second neighboring triangular facets.

17. The method of claim 16, wherein, The determining of the vertex density information based on the areas of the one or more second neighboring triangular facets comprises: parsing, from the code stream, an area of a triangular facet with a largest area and an area of a triangular facet with a smallest area; determining the vertex density information based on the areas of the one or more second neighboring triangular facets, the area of the triangular facet with the largest area, and the area of the triangular facet with the smallest area.

18. The method of claim 16, wherein, The determining of the vertex density information based on the areas of the one or more second neighboring triangular facets comprises: determining an area of a reconstructed triangular facet with a largest area and an area of a reconstructed triangular facet with a smallest area based on second reconstructed coordinates of reconstructed vertices; determining the vertex density information based on the areas of the one or more second neighboring triangular facets, the area of the reconstructed triangular facet with the largest area, and the area of the reconstructed triangular facet with the smallest area.

19. The method of claim 14, wherein, The obtaining of the scaling parameter corresponding to the current vertex comprises: parsing, from the code stream, the scaling parameter corresponding to the current vertex.

20. The method according to any one of claims 14 to 19, characterized in that, The predicting and compensating based on the residual information corresponding to the current vertex to obtain the first reconstructed coordinate of the current vertex comprises: predicting and compensating based on the scaling parameter corresponding to the current vertex and the residual information corresponding to the current vertex to obtain the first reconstructed coordinate of the current vertex.

21. The method of claim 20, wherein, The predicting and compensating based on the scaling parameter corresponding to the current vertex and the residual information corresponding to the current vertex to obtain the first reconstructed coordinate of the current vertex comprises: predicting a reference vertex of the current vertex; when the scaling parameter corresponding to the current vertex is different from a scaling parameter corresponding to the reference vertex, determining a normalized scaling parameter based on the scaling parameter corresponding to the current vertex and / or the scaling parameter corresponding to the reference vertex; scaling the first reconstructed coordinate corresponding to the reference vertex based on the normalization scaling parameter to obtain a normalized coordinate corresponding to the reference vertex; compensating the normalized coordinate corresponding to the reference vertex based on the residual information corresponding to the current vertex to obtain a normalized coordinate corresponding to the current vertex; scaling the normalized coordinate corresponding to the current vertex based on the normalization scaling parameter to obtain the first reconstructed coordinate corresponding to the current vertex.

22. The method of claim 21, wherein, The normalization scaling parameter is determined based on the scaling parameter corresponding to the current vertex or the scaling parameter corresponding to the reference vertex, including: The normalization scaling parameter is determined as the scaling parameter corresponding to the current vertex.

23. The method of claim 21, wherein, The normalization scaling parameter is determined based on the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex, including: The normalization scaling parameter is determined as the maximum scaling parameter in the scaling parameter corresponding to the current vertex and the scaling parameter corresponding to the reference vertex.

24. An encoding apparatus, comprising: The apparatus includes: a data obtaining module configured to obtain data of a three-dimensional mesh (3D mesh), wherein the 3D mesh is composed of a plurality of triangular facets, each triangular facet includes three vertices, and the data of the 3D mesh includes vertex coordinates of a plurality of vertices, the vertex coordinates being three-dimensional coordinates; an information obtaining module configured to obtain a scaling parameter corresponding to a current vertex, the scaling parameter corresponding to the current vertex being positively correlated with an importance of the current vertex on the 3D mesh; a scaling module configured to scale a vertex coordinate of the current vertex based on the scaling parameter corresponding to the current vertex to obtain a scaled vertex coordinate of the current vertex; a prediction and compensation module configured to perform prediction and compensation based on the scaled vertex coordinate of the current vertex to obtain residual information corresponding to the current vertex; an encoding module configured to encode the residual information corresponding to the current vertex to obtain a bitstream.

25. The apparatus of claim 24, wherein, The apparatus is further configured to perform the encoding method in any one of the preceding claims 2 to 13.

26. A decoding apparatus, comprising: The apparatus includes: a bitstream receiving module configured to receive a bitstream; a decoding module configured to parse residual information corresponding to a current vertex from the bitstream; an information obtaining module configured to obtain a scaling parameter corresponding to the current vertex, the scaling parameter corresponding to the current vertex being positively correlated with an importance of the current vertex on the 3D mesh; a prediction and compensation module configured to perform prediction and compensation based on the residual information corresponding to the current vertex to obtain a first reconstructed coordinate of the current vertex; a scaling module configured to scale the first reconstructed coordinate based on the scaling parameter corresponding to the current vertex to obtain a second reconstructed coordinate of the current vertex.

27. The apparatus of claim 26, wherein, The apparatus is further configured to perform the decoding method in any one of the preceding claims 15 to 23.

28. A method of generating a bitstream, the method comprising: A bitstream is generated according to the encoding method in any one of the preceding claims 1 to 13.

29. An electronic device, comprising: includes: a memory and a processor, the memory being coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-23.

30. A chip, characterized by An electronic device comprising one or more interface circuits and one or more processors; the interface circuits configured to receive signals from a memory of the electronic device and send the signals to the processors, the signals comprising computer instructions stored in the memory; when the processors execute the computer instructions, cause the electronic device to perform the method of any one of claims 1-23.

31. A computer readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when running on a computer or a processor, causes the computer or the processor to perform the method of any one of claims 1-23.

32. A computer program product, characterised in that, The computer program product contains computing instructions that, when executed by a computer or a processor, cause the steps of the method of any one of claims 1-23 to be performed.

Images