Dynamic grid encoding and decoding method, device, encoder, decoder and system

Abstract

The invention discloses a dynamic grid encoding and decoding method and device, an encoder, a decoder and a system, and relates to the field of multimedia. The method comprises the steps that a grid obtained by filtering a reconstructed grid of an encoded grid is used as a reference grid, and due to the fact that the reference grid obtained by filtering the reconstructed grid is closer to the encoded grid, the error between the reconstructed grid and the encoded grid corresponding to the reconstructed grid is reduced; the reference grid obtained by filtering the reconstructed grid is utilized to predict the dynamic grid, the prediction result is more accurate, the dynamic grid to be coded is coded according to the prediction result to obtain the code stream, the grid can be represented by using as little data as possible, and thus the code rate is reduced. The grid obtained by filtering the reconstructed grid of the decoded grid is used as the reference grid, and the reference grid obtained by filtering the reconstructed grid is used for decoding and reconstructing the dynamic grid, so that the obtained reconstructed grid is more accurate.

Description

Technical Field

[0001] This application relates to the field of multimedia, and more particularly to a dynamic grid encoding and decoding method, apparatus, encoder, decoder and system. Background Technology

[0002] Currently, dynamic meshes are generated based on dynamic mesh generation technology. A dynamic mesh includes vertex coordinates, connectivity relationships, and attribute information. Due to the large data volume of dynamic meshes, they are typically encoded to reduce this data size for transmission or storage. However, to ensure the quality of the reconstructed mesh, a high bitrate is required. Summary of the Invention

[0003] This application provides a dynamic trellis coding and decoding method, apparatus, encoder, decoder and system, thereby reducing the bit rate of dynamic trellis coding.

[0004] Firstly, a dynamic mesh coding method is provided. This method can be applied to an encoder, or to a device that supports the encoder in implementing the method, such as a chip system. The method includes: acquiring a dynamic mesh to be encoded; predicting the dynamic mesh to be encoded based on a reference mesh to obtain a prediction result; and encoding the dynamic mesh to be encoded based on the prediction result to obtain a bitstream. The reference mesh is obtained by filtering a reconstructed mesh of an already encoded mesh.

[0005] The dynamic grid coding method provided in this application uses a grid obtained by filtering the reconstructed grid of the encoded grid as a reference grid during the encoding process. Since the reference grid obtained by filtering the reconstructed grid is closer to the encoded grid, the error between the reconstructed grid and the corresponding encoded grid is reduced. The reference grid obtained by filtering the reconstructed grid is used to predict the dynamic grid, and the prediction result is more accurate. This results in a smaller bitstream encoded based on the prediction result, thereby improving coding efficiency and effectively reducing the bit rate.

[0006] It should be understood that a coded mesh can be a mesh that has been encoded in time and / or space. A coded mesh can be obtained by encoding a corresponding mesh to be encoded. For example, a coded mesh can be obtained by encoding a corresponding mesh to be encoded using the encoding method of this application.

[0007] In one possible implementation, the encoded mesh includes the preceding mesh.

[0008] The preceding grid can include the already encoded complete grid. After the grid to be encoded is completed, the reconstructed grid of the already encoded complete grid is filtered to reduce the error between the reconstructed grid and the corresponding already encoded grid. The reference grid obtained by filtering the reconstructed grid is used to predict the dynamic grid. For example, an inter-frame prediction method can be used to predict the dynamic grid. The prediction result is more accurate, and the grid can be represented with as little data as possible, thereby reducing the bit rate.

[0009] In another possible implementation, the encoded mesh includes a portion of the encoded region in the mesh to be encoded, and the portion of the encoded region includes one or more encoded vertex regions, encoded blocks, or encoded specified regions.

[0010] During the encoding process of the grid to be encoded, a portion of the encoded region within the grid is filtered, and the reconstructed region of this portion is then filtered to obtain a reference grid. This reference grid is used to predict dynamic grids; for example, intra-frame prediction methods can be employed to predict dynamic grids, resulting in more accurate predictions and further improving the quality of the predictions. This approach uses as little data as possible to represent the grid, thereby reducing the bitrate.

[0011] In another possible implementation, the reference mesh is obtained by filtering the reconstructed mesh of the encoded mesh, including: the reference mesh is obtained by filtering the vertex positions and / or attribute information of the reconstructed mesh of the encoded mesh.

[0012] The dynamic mesh encoding method provided in this application can be used not only to encode the vertex positions of dynamic meshes, but also to encode the attribute information of dynamic meshes. This reduces the bitrate of the attributes corresponding to the vertices and improves the reconstruction quality of the attribute information of the dynamic mesh.

[0013] In another possible implementation, the bitstream includes auxiliary parameters, which include at least one of a grid filter identifier, a region filter identifier, filter coefficients, compensation information, or a reference grid identifier; wherein, the grid filter identifier is used to indicate filtering of the reconstructed grid; the region filter identifier is used to indicate filtering of regions in the reconstructed grid; the filter coefficients include at least one of an identifier of the filtering algorithm, a filter weight, or a number of filtering iterations; the compensation information includes compensation values ​​for the reconstructed grid and / or compensation values ​​for the regions; and the reference grid identifier is used to indicate a reference grid.

[0014] The bitstream contains auxiliary parameters, which enable the decoder to decode the bitstream based on the auxiliary parameters, thereby improving the accuracy and flexibility of the reconstructed grid obtained from the decoding.

[0015] In another possible implementation, the auxiliary parameters are determined based on the mesh to be encoded and its reconstructed mesh, and / or based on the mesh to be encoded and the reference mesh.

[0016] The auxiliary parameters indicate information for filtering the reconstructed grid. The auxiliary parameters are determined based on the difference between the grid to be encoded and the reconstructed grid, or the difference between the grid to be encoded and the reference grid, to improve the accuracy of the auxiliary parameters. This allows the decoder to decode the bitstream based on the auxiliary parameters, thereby improving the accuracy of the reconstructed grid obtained from the decoder.

[0017] Secondly, a dynamic grid decoding method is provided. This method can be applied to a decoder, or to a device that supports the decoder in implementing the method, such as a chip system. The method includes: acquiring a bitstream including a first dynamic grid; processing the bitstream to obtain the first dynamic grid; and reconstructing the first dynamic grid based on a reference grid to obtain a first reconstructed grid, wherein the reference grid is obtained by filtering the reconstructed grid of the decoded grid.

[0018] In the encoding process of dynamic grids, the grid obtained by filtering the reconstructed grid of the encoded grid is used as the reference grid. Since the reference grid obtained by filtering the reconstructed grid is closer to the encoded grid, the error between the reconstructed grid and the corresponding encoded grid is reduced. The dynamic grid is predicted using the reference grid obtained by filtering the reconstructed grid. The prediction result is more accurate, so the grid can be represented with as little data as possible. Therefore, the bitstream obtained by encoding based on the prediction result is smaller, which effectively reduces the bit rate.

[0019] Because the prediction results are relatively accurate during the encoding process of dynamic grids, the bitstream contains accurate information that can represent the grid. During the decoding process, decoding and reconstructing the bitstream yields a high-quality reconstructed grid. Furthermore, filtering the reconstructed grid of the decoded grid, based on the filtering method used for the reconstructed grid of the encoded grid, reduces the error between the reconstructed grid and its corresponding encoded grid, improving the quality of the reconstructed grid. Using the filtered reconstructed grid of the decoded grid as a reference grid, and then decoding and reconstructing the bitstream containing data that can represent the grid using this reference grid, results in a more accurate reconstructed grid that is closer to the encoded grid. Thus, while reducing the error in the reconstructed grid, the system minimizes the loss of the grid's inherent detail information, reconstructing the dynamic grid and improving the overall reconstruction quality.

[0020] In one possible implementation, the decoded mesh includes the preceding mesh.

[0021] After decoding the bitstream, the reconstructed grid of the decoded complete grid is filtered to reduce the error between the reconstructed grid and the corresponding encoded grid. The reference grid obtained by filtering the reconstructed grid is used to decode and reconstruct the dynamic grid. For example, the inter-frame prediction method is used to decode and reconstruct the dynamic grid, making the reconstructed grid more accurate.

[0022] In another possible implementation, the decoded mesh includes a portion of the decoded region in the first dynamic mesh, which includes one or more decoded vertex regions, decoded blocks, or decoded designated regions.

[0023] During the bitstream decoding process, a portion of the decoded regions within the decoded grid is filtered, and the reconstructed regions of these partially decoded regions are then filtered to obtain a reference grid. This reference grid is used to decode and reconstruct the dynamic grid; for example, intra-frame prediction methods can be employed to decode and reconstruct the dynamic grid, further improving the quality of the reconstructed grid.

[0024] In another possible implementation, the reference mesh is obtained by filtering the reconstructed mesh of the decoded mesh, including: the reference mesh is obtained by filtering the vertex positions and / or attribute information of the reconstructed mesh of the decoded mesh.

[0025] The dynamic mesh decoding method provided in this application can be used not only to decode the vertex positions of dynamic meshes, but also to decode the attribute information of dynamic meshes. Since the vertex positions of the dynamic mesh shift after encoding, filtering the decoded attribute information allows it to match the reconstructed mesh, thereby improving the reconstruction quality of the attribute information of the dynamic mesh.

[0026] In another possible implementation, the method further includes filtering the first reconstructed mesh to obtain a filtered mesh.

[0027] Filtering the reconstructed grid can further reduce the error of the reconstructed grid. The filtered grid is closer to the grid to be encoded at the encoding end corresponding to the filtered grid. The filtered grid is used as a reference grid for the decoding and reconstruction of subsequent grids, which further improves the reconstruction quality.

[0028] In another possible implementation, the bitstream includes auxiliary parameters, including a grid filtering identifier, which indicates filtering of the reconstructed grid; filtering the first reconstructed grid to obtain a filtered grid includes: filtering the first reconstructed grid to obtain a filtered grid when the grid filtering identifier indicates filtering of the first reconstructed grid.

[0029] The decoder decodes the bitstream based on auxiliary parameters, improving the accuracy and flexibility of the reconstructed grid obtained from the decoding.

[0030] In another possible implementation, the bitstream includes auxiliary parameters, including a region filtering identifier, which indicates that regions in the reconstructed mesh are filtered; filtering the first reconstructed mesh to obtain a filtered mesh includes: filtering one or more vertices in the first reconstructed mesh to obtain a filtered mesh, provided that the region filtering identifier indicates that one or more vertices in the first reconstructed mesh are filtered.

[0031] In another possible implementation, the bitstream includes auxiliary parameters, which further include compensation information used to determine the compensation value for the first reconstructed grid; filtering the first reconstructed grid to obtain a filtered grid includes: filtering the first reconstructed grid according to the compensation information to obtain a filtered grid.

[0032] In another possible implementation, the auxiliary parameters may include at least one of the following: filter coefficients, filter rules, or reference grid identifiers.

[0033] In another possible implementation, the method further includes: adjusting the attribute information of the first reconstructed mesh according to the vertex positions of the first reconstructed mesh to obtain the adjusted attribute information of the first reconstructed mesh.

[0034] In another possible implementation, the method further includes: adjusting the attribute information of the filtered mesh based on the vertex positions of the filtered mesh to obtain the adjusted attributes of the filtered mesh.

[0035] Since the vertex positions of the dynamic mesh will shift after reconstruction and / or filtering, the attribute information corresponding to the vertex positions can be adjusted to match the attribute information with the reconstructed mesh, thereby improving the reconstruction quality of the attribute information of the dynamic mesh.

[0036] Thirdly, an encoding apparatus is provided, comprising modules for performing the first aspect or any possible design of the first aspect. For example, the encoding apparatus includes a communication module and an encoding module.

[0037] The communication module is used to acquire the dynamic grid to be encoded; the encoding module is used to predict the dynamic grid to be encoded based on the reference grid to obtain the prediction result of the dynamic grid to be encoded. The reference grid is obtained by filtering the reconstructed grid of the encoded grid; the encoding module is also used to encode the dynamic grid to be encoded based on the prediction result to obtain the bit stream.

[0038] In one possible implementation, the encoded mesh includes the preceding mesh.

[0039] In another possible implementation, the encoded mesh includes a portion of the encoded region in the mesh to be encoded, and the portion of the encoded region includes one or more encoded vertex regions, encoded blocks, or encoded specified regions.

[0040] In another possible implementation, the reference mesh is obtained by filtering the reconstructed mesh of the encoded mesh, including: the reference mesh is obtained by filtering the vertex positions and / or attribute information of the reconstructed mesh of the encoded mesh.

[0041] In another possible implementation, the bitstream includes auxiliary parameters, which include at least one of a grid filter identifier, a region filter identifier, filter coefficients, compensation information, or a reference grid identifier; wherein, the grid filter identifier is used to indicate filtering of the reconstructed grid; the region filter identifier is used to indicate filtering of regions in the reconstructed grid; the filter coefficients include at least one of an identifier of the filtering algorithm, a filter weight, or a number of filtering iterations; the compensation information includes compensation values ​​for the reconstructed grid and / or compensation values ​​for the regions; and the reference grid identifier is used to indicate a reference grid.

[0042] In another possible implementation, the auxiliary parameters are determined based on the mesh to be encoded and its reconstructed mesh, and / or based on the mesh to be encoded and the reference mesh.

[0043] Fourthly, a decoding apparatus is provided, comprising modules for performing the methods of the second aspect or any possible design of the second aspect. For example, the encoding apparatus includes a communication module and a decoding module.

[0044] The communication module is used to acquire a bitstream, which includes a first dynamic grid; the decoding module is used to decode the bitstream to obtain the first dynamic grid; the decoding module is also used to process the first dynamic grid according to a reference grid to obtain a first reconstructed grid, wherein the reference grid is obtained by filtering the reconstructed grid of the decoded grid.

[0045] In another possible implementation, the decoded mesh includes the preceding mesh.

[0046] In another possible implementation, the decoded mesh includes a portion of the decoded region in the first dynamic mesh, which includes one or more decoded vertex regions, decoded blocks, or decoded designated regions.

[0047] In another possible implementation, the reference mesh is obtained by filtering the reconstructed mesh of the decoded mesh, including: the reference mesh is obtained by filtering the vertex positions and / or attribute information of the reconstructed mesh of the decoded mesh.

[0048] In another possible implementation, the decoding module is also used to: filter the first reconstructed mesh to obtain a filtered mesh.

[0049] In another possible implementation, the bitstream includes auxiliary parameters, including a grid filter identifier, which indicates filtering of the reconstructed grid. When the decoding module filters the first reconstructed grid to obtain a filtered grid, it specifically filters the first reconstructed grid to obtain a filtered grid when the grid filter identifier indicates filtering of the first reconstructed grid.

[0050] In another possible implementation, the bitstream includes auxiliary parameters, including a region filtering identifier, which is used to indicate filtering of regions in the reconstructed grid. When the decoding module filters the first reconstructed grid to obtain a filtered grid, it specifically filters one or more vertices in the first reconstructed grid to obtain a filtered grid when the region filtering identifier indicates filtering of one or more vertices in the first reconstructed grid.

[0051] In another possible implementation, the bitstream includes auxiliary parameters, which further include compensation information used to determine the compensation value for the first reconstructed grid; when the decoding module filters the first reconstructed grid to obtain the filtered grid, it specifically filters the first reconstructed grid according to the compensation information to obtain the filtered grid.

[0052] In another possible implementation, the auxiliary parameters may include at least one of the following: filter coefficients, filter rules, or reference grid identifiers.

[0053] In another possible implementation, the decoding module is also used to: adjust the attribute information of the first reconstructed mesh according to the vertex position of the first reconstructed mesh to obtain the adjusted attribute information of the first reconstructed mesh.

[0054] In another possible implementation, the decoding module is also used to adjust the attribute information of the filtered mesh according to the vertex positions of the filtered mesh to obtain the adjusted attributes of the filtered mesh.

[0055] Fifthly, an encoder is provided, the encoder including at least one processor and a memory, wherein the memory is used to store a computer program such that when the computer program is executed by at least one processor, it implements the method described in the first aspect or any possible design of the first aspect.

[0056] In one possible implementation, the encoder also includes a filter for filtering the reconstructed mesh of the encoded mesh.

[0057] In a sixth aspect, a decoder is provided, the decoder comprising at least one processor and a memory, wherein the memory is used to store a computer program such that when the computer program is executed by at least one processor, it implements the method described in the second aspect or any possible design of the second aspect.

[0058] In one possible implementation, the decoder also includes a filter for filtering the reconstructed grid of the decoded grid.

[0059] In a seventh aspect, a coding and decoding system is provided, the coding and decoding system comprising an encoder as described in the fifth aspect and a decoder as described in the sixth aspect.

[0060] Eighthly, a chip is provided, comprising: a processor and a power supply circuit; wherein the power supply circuit is used to supply power to the processor; the processor is used to perform operational steps of the method in the first aspect or any possible implementation of the first aspect, and to perform operational steps of the method in the second aspect or any possible implementation of the second aspect.

[0061] Ninth aspect, a computer program product is provided, the computer program product including a computer program or instructions, which, when the computer program product is run on a processor, causes the processor to perform the operation steps of the method in the first aspect or any possible implementation of the first aspect, or to perform the operation steps of the method in the second aspect or any possible implementation of the second aspect.

[0062] A tenth aspect provides a computer-readable storage medium comprising: computer software instructions; which, when executed in a computing device, cause the computing device to perform operational steps of the method in the first aspect or any possible implementation thereof, or to perform operational steps of the method in the second aspect or any possible implementation thereof.

[0063] Eleventhly, a bitstream is provided, said bitstream being obtained by the first aspect or any possible implementation thereof.

[0064] In a twelfth aspect, a bitstream is provided, the bitstream including auxiliary parameters, the auxiliary parameters including at least one of grid filter identifier, region filter identifier, filter coefficients, compensation information, or reference grid identifier; wherein...

[0065] The grid filter identifier is used to indicate the filtering applied to the reconstructed grid;

[0066] The region filtering identifier is used to indicate regions in the reconstructed grid to be filtered;

[0067] The filtering coefficients include at least one of the following: the identifier of the filtering algorithm, the filtering weight, or the number of filtering iterations;

[0068] The compensation information includes compensation values ​​for the reconstructed grid and / or compensation values ​​for the region;

[0069] The reference grid identifier is used to indicate the reference grid.

[0070] In a thirteenth aspect, a method for storing a bitstream is provided, the method comprising: receiving a bitstream generated according to the first aspect or any possible implementation thereof, or a bitstream as described in the twelfth aspect; and storing the bitstream into a storage medium.

[0071] In a fourteenth aspect, an apparatus for storing a bitstream is provided, the apparatus being used to store a bitstream generated according to the first aspect or any possible implementation thereof, or for storing a bitstream as described in the twelfth aspect. Exemplarily, the apparatus may be a computer-readable storage medium.

[0072] In a fifteenth aspect, a method for transmitting a bitstream is provided, the method comprising: acquiring a bitstream, the bitstream being generated by the first aspect or any possible implementation thereof, or the bitstream being the bitstream described in the twelfth aspect; and transmitting the bitstream.

[0073] In a sixteenth aspect, an apparatus for transmitting a bitstream is provided, the apparatus being used to acquire and transmit a bitstream generated by the first aspect or any possible implementation thereof, or to acquire and transmit a bitstream as described in the twelfth aspect.

[0074] The technical effects of any of the implementation methods in aspects three through sixteen can be found in the technical effects of the corresponding implementation methods in aspects one or two, and will not be repeated here.

[0075] It should be noted that any of the possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0076] Figure 1 A schematic diagram of the structure of an encoding / decoding system provided in this application;

[0077] Figure 2 A schematic diagram of another encoding / decoding system provided in this application;

[0078] Figure 3 A schematic diagram of the structure of an encoder and decoder provided in this application;

[0079] Figure 4 A flowchart illustrating a dynamic mesh encoding and decoding method provided in this application;

[0080] Figure 5A schematic diagram of a reference vertex provided for this application;

[0081] Figure 6 A flowchart illustrating another dynamic mesh encoding and decoding method provided in this application;

[0082] Figure 7 A schematic diagram of a reference grid for a decoding end provided in this application;

[0083] Figure 8 A schematic diagram of the decoding of an attribute graph provided in this application;

[0084] Figure 9 A schematic diagram illustrating the decoding of texture coordinates provided in this application;

[0085] Figure 10 A schematic diagram of normal direction decoding provided for this application;

[0086] Figure 11 A schematic diagram of the structure of an encoding device provided in this application;

[0087] Figure 12 A schematic diagram of the structure of a decoding device provided in this application;

[0088] Figure 13 A schematic diagram of an encoder provided in this application;

[0089] Figure 14 This is a schematic diagram of the structure of a decoder provided in this application. Detailed Implementation

[0090] To facilitate understanding, the main terms used in this application will be explained first.

[0091] A mesh, also known as a static mesh, contains multiple polygons used to describe the boundary surfaces of a volume object. A mesh can contain one or more types of polygons to describe the boundary surfaces of a volume object. For example, a mesh might contain two types of polygons (such as triangles and quadrilaterals) to describe the boundary surfaces of a volume object; or, for instance, a mesh might contain triangles to describe the boundary surfaces of a volume object.

[0092] A polygon comprises multiple vertices and multiple edges. In this embodiment, a polygon is defined by vertices in three-dimensional (3D) space and the way these vertices are connected. The mesh data includes the vertex positions and connections of multiple polygons. Vertex positions indicate the location information of the vertices constituting the polygon in 3D space. For example, vertex positions can be the coordinates of the vertices constituting the polygon in 3D space. If the polygon is a triangle, the vertex positions can include the coordinates of the three vertices of the triangle, such as (0, 2, 1.3, 2.5), representing the positions of the (x, y, z) axes in the coordinate system. Connections indicate the connection relationships between the vertices of the polygons in the mesh.

[0093] Optionally, the mesh data also includes attribute information. Attribute information includes at least one of an attribute graph, texture coordinates, or normals. An attribute graph includes at least one of a texture graph, normal graph, or material graph. Normals are vectors perpendicular to the polygons and are continuous. Normals affect the rendering quality of the mesh; for example, rendering quality includes lighting quality. Normals can be calculated using a normal graph, or they can be calculated using connectivity and vertex coordinates. Normals can also be encoded, and a bitstream containing the encoded normals can be transmitted.

[0094] Attribute information includes the properties of each vertex in the mesh. In some examples, attribute information can be represented by mesh parameterization, showing a mapping from a 3D region to a planar 2D region. This mapping is typically described by a set of parametric coordinates, called UV coordinates or texture coordinates.

[0095] Dynamic meshes: These include multi-frame static meshes. Dynamic meshes can be used to represent volumetric video, digital humans (or meta humans), and computer graphics (CG) content.

[0096] For example, cameras from different perspectives can capture multiple frames of images, and dynamic mesh generation technology can be used to generate a dynamic mesh of the multiple frames. This dynamic mesh can be displayed to the user, or the dynamic mesh can be rendered so that the user can watch videos from different perspectives.

[0097] Because dynamic meshes contain a large amount of data, storing them requires a significant amount of storage space, and transmitting them requires a high bitrate. This puts immense pressure on both transmission and storage. Therefore, it is necessary to encode dynamic meshes to reduce the storage space and bitrate they require.

[0098] For example, the Moving Picture Expert Group (MPEG) Video-based Dynamic Mesh Coding (MPEG V-DMC) technology can be used to encode and decode dynamic meshes.

[0099] The dynamic mesh data includes vertex positions and attribute information. For example, lossy compression, such as quantization and / or simplification, can be applied to vertex positions (e.g., vertex coordinates) and / or attribute information. This will lead to errors between the reconstructed mesh at the decoding end and the mesh to be encoded at the encoding end. For instance, the reconstructed mesh may be uneven, reducing the continuity of normals and thus affecting the quality of the final displayed image.

[0100] For example, during the encoding process, vertex coordinates and attribute information can be quantized; or, vertex coordinates and attribute information can be simplified; or vertex coordinates and attribute information can be both quantized and simplified.

[0101] For example, the encoding process for vertex positions may also include prediction, and this application does not limit the order of quantization, simplification, and prediction. Alternatively, lossy compression such as quantization and / or simplification may be performed only on vertex positions, or lossy compression such as quantization and / or simplification may be performed only on attribute information.

[0102] Quantization refers to changing the precision of mesh data. For example, discretizing continuous mesh data. For instance, if the mesh data consists of vertex positions, the floating-point values ​​of the vertex positions can be changed to integers. For example, if the mesh contains multiple triangles and the vertex coordinates include the coordinates of the three vertices of a triangle (2.2, 6.3, 9.5), the vertex coordinates can be modified to (2, 6, 9). By quantizing the mesh data, the precision of the mesh data (e.g., vertex positions) can be reduced, decreasing the amount of mesh data and thus lowering the bitrate. However, because the vertex positions change, the shape of the polygons also changes, leading to errors in mesh reconstruction. For example, quantizing the vertex positions can cause abrupt changes in the mesh, resulting in distortion in the final rendered video or image, uneven objects, and discontinuous motion.

[0103] Simplification refers to reducing the amount of data in the mesh, thereby reducing the bitrate. For example, reducing the number of vertices in a polygon will change the shape of the polygon, leading to errors in mesh reconstruction.

[0104] The data in a grid can also include the connection relationships between vertices.

[0105] Connection relationships can include the index values ​​of multiple vertices of a polygon. Algorithms such as EdgeBreaker or TFAN can be used to encode these connections using either lossy or lossless methods. For example, the EdgeBreaker algorithm uses a finite set of state symbols ("CLERS") to identify each polygon based on its relative position to the currently encoded polygon set, thus completing the traversal encoding of all polygons. During the traversal of all polygons, the traversal order of all vertices can be further obtained for subsequent encoding of vertex coordinates and other attribute information.

[0106] Filtering (or smoothing): Used to alter vertex positions, attribute information, or connectivity in a mesh. For example, by applying weighted averages or other mathematical operations to the vertex positions, attribute information, or connectivity in the mesh, noise, blur, or other unwanted features can be reduced, while retaining or enhancing useful information in the mesh. Filtering can be replaced with terms like smoothing, polishing, or finishing.

[0107] In existing technologies, to ensure the quality of the reconstructed mesh and the rendering effect, errors can be reduced by increasing the precision of vertex positions. However, higher precision requires a higher bitrate. Additionally, the reconstructed mesh obtained during decoding can be filtered to reduce the error between the reconstructed mesh and the encoded mesh caused by quantization and / or simplification operations during encoding. However, filtering not only reduces the error of the reconstructed mesh but also loses the mesh's inherent detail.

[0108] Furthermore, adding a filtering module at the decoding end cannot directly reduce the bitrate of encoding dynamic grids at the encoding end; it can only reduce grid noise.

[0109] Therefore, in order to reduce the bit rate problem of dynamic grid encoding and decoding, this application provides a dynamic grid encoding and decoding method, that is, the encoder uses the reconstructed grid of the encoded grid as a reference grid, predicts the dynamic grid to be encoded based on the reference grid to obtain the prediction result of the dynamic grid to be encoded, and encodes the dynamic grid to be encoded based on the prediction result to obtain the bit stream.

[0110] Compared to predicting dynamic grids directly using the reconstructed grid as a reference grid, the dynamic grid coding process employs a lossy coding method, resulting in a significant error between the reconstructed grid and its corresponding encoded grid. When predicting the next grid to be encoded based on this large-error reconstructed grid, the prediction result is inaccurate, leading to a large discrepancy between the predicted result and the actual grid, resulting in a large encoded bitstream. The dynamic grid coding method provided in this application uses a reference grid obtained by filtering the reconstructed grid, which is closer to the encoded grid. This reduces the error between the reconstructed grid and its corresponding encoded grid. Using this filtered reference grid to predict the dynamic grid results in a more accurate prediction, leading to a smaller bitstream encoded based on the prediction result, thereby improving coding efficiency and reducing the bitrate.

[0111] The decoder uses the reconstructed grid obtained by filtering the decoded grid as a reference grid, and reconstructs the first dynamic grid obtained from the bitstream based on the reference grid to obtain the first reconstructed grid.

[0112] Compared to reconstructing the first dynamic grid obtained from the bitstream based on the reconstructed grid as a reference grid, the reconstructed grid has a large error with the corresponding encoded grid due to the lossy coding method used in the dynamic grid encoding process. Decoding and reconstructing the first dynamic grid obtained from the bitstream based on the reconstructed grid with a large error as a reference grid results in an inaccurate reconstructed grid. Filtering the reconstructed grid further reduces the error, but also loses the necessary detail information of the grid itself. In the dynamic grid decoding method provided in this application, at the encoding end, the reference grid obtained by filtering the reconstructed grid is closer to the encoded grid, reducing the error between the reconstructed grid and the corresponding encoded grid. Using the reference grid obtained by filtering the reconstructed grid to predict the dynamic grid results in more accurate predictions. The grid can be represented using as little data as possible, and the bitstream contains data that can represent the grid. At the decoding end, the reconstructed grid of the decoded grid is filtered according to the method of filtering the reconstructed grid of the encoded grid, reducing the error between the reconstructed grid and the corresponding encoded grid, improving the quality of the reconstructed grid. The grid obtained by filtering the reconstructed grid of the decoded grid is used as the reference grid. The bitstream is decoded and reconstructed using the reference grid, resulting in a more accurate reconstructed grid, that is, the reconstructed grid is closer to the encoded grid. In other words, while reducing the error of the reconstructed grid, the inherent detail information of the grid itself is not lost as much as possible, thus reconstructing the dynamic grid and improving the reconstruction quality.

[0113] The embodiments of this application will now be described with reference to the accompanying drawings.

[0114] Figure 1This is a schematic diagram of an encoding / decoding system provided in this application. The encoding / decoding system 100 includes a source device 110 and a destination device 120. The source device 110 encodes the grid to obtain a bitstream and sends the bitstream to the destination device 120. The destination device 120 decodes and reconstructs the bitstream to obtain a reconstructed grid. Optionally, the destination device can also display the reconstructed grid or a rendered grid of the reconstructed grid.

[0115] Specifically, the source device 110 includes an image acquisition unit 111, a preprocessor 112, an encoder 113, and a communication interface 114.

[0116] Image acquisition device 111 is used to acquire raw images. Image acquisition device 111 may include or be any type of image capture device for, for example, capturing real-world images, and / or any type of image or commentary (for screen content encoding, some text on the screen is also considered as an image to be encoded or part of an image) generation device, such as a computer graphics processor for generating computer-animated images, or any type of device for acquiring and / or providing real-world images, computer-animated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). Image acquisition device 111 may be a camera for capturing images or a memory for storing images. Image acquisition device 111 may also include any type of (internal or external) interface for storing previously captured or generated images and / or acquiring or receiving images. When the image acquisition device 111 is a camera, it may be, for example, a local camera or an integrated camera integrated into the source device; when the image acquisition device 111 is a memory, it may be a local memory or an integrated memory integrated into the source device. When the image acquisition device 111 includes an interface, the interface may be, for example, an external interface for receiving images from an external video source, such as an external image capture device, like a camera, external memory, or an external image generation device, such as an external computer graphics processor, computer, or server. The interface may be any type of interface according to any proprietary or standardized interface protocol, such as a wired or wireless interface, or an optical interface.

[0117] An image can be viewed as a two-dimensional array or matrix of pixels (picture elements). Pixels in an array are also called sample points. The number of sample points in the array or image along the horizontal and vertical directions (or axes) defines the image's size and / or resolution. To represent color, three color components are typically used; that is, an image can be represented as or contain three sample arrays. For example, in RBG format or color space, an image includes corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luma / chroma format or color space. For example, for a YUV format image, this includes a luma component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The luma component Y represents the brightness or grayscale level intensity (e.g., both are the same in a grayscale image), while the two chroma components U and V represent chroma or color information components. Accordingly, a YUV format image includes a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(U and V). An RGB format image can be converted or transformed to YUV format, and vice versa; this process is also called color transformation or conversion. If the image is black and white, it may only include a luminance sampling array. In this application, the image transmitted from the image acquisition unit 111 to the encoder 113 can also be referred to as raw image data.

[0118] The preprocessor 112 receives the raw image acquired by the image acquisition unit 111 and preprocesses the raw image to obtain a preprocessed image. For example, the preprocessing performed by the preprocessor 112 includes retouching, color format conversion (e.g., from RGB format to YUV format), color adjustment, or noise reduction.

[0119] In this application, the preprocessor 112 can also be used to generate dynamic meshes for multiple frames of images using dynamic mesh generation technology.

[0120] This application does not limit the source of the dynamic mesh. The source device 110 can generate the dynamic mesh, or generate the dynamic mesh based on multiple frames of images, or obtain the dynamic mesh from outside the source device.

[0121] Encoder 113 is used to acquire a dynamic mesh and encode the dynamic mesh to obtain a bitstream. For example, encoder 113 may include a filtering unit 1131 and an encoding unit 1132. Filtering unit 1131 is used to reconstruct the encoded mesh to obtain a reconstructed mesh, and to filter the reconstructed mesh to obtain a reference mesh, so that encoding unit 1132 can perform prediction, quantization, and entropy encoding on the dynamic mesh to be encoded based on the reference mesh to obtain a bitstream. For example, for each mesh in the dynamic mesh, encoder 113 can predict the mesh to be encoded based on the reference mesh to obtain a prediction result for the mesh to be encoded, and then perform residual encoding on the mesh to be encoded based on the prediction result to obtain a bitstream. The reference mesh is obtained by filtering the reconstructed mesh of the encoded mesh.

[0122] It should be understood that the filtering unit 1131 described above can also filter only the reconstructed mesh to obtain the reference mesh. The process of reconstructing the encoded mesh to obtain the reconstructed mesh can be implemented by other units of the encoder. This application does not specifically limit this.

[0123] The communication interface 114 is used to receive the code stream generated by the encoder 113 and send the code stream through the communication channel 130.

[0124] The target device 120 includes a display 121, a post-processor 122, a decoder 123, and a communication interface 124.

[0125] The communication interface 124 is used to receive the bit stream and transmit it to the decoder 123 so that the decoder 123 can reconstruct the dynamic grid based on the bit stream.

[0126] Communication interfaces 114 and 124 can be used to communicate via a communication link between source device 110 and destination device 120, such as through wired or wireless connections, or through any type of mesh, such as wired mesh, wireless mesh or any combination thereof, any type of private network and public network or any combination thereof, to send or receive relevant data from the dynamic mesh.

[0127] Both communication interface 114 and communication interface 124 can be configured as follows: Figure 1 The arrow pointing from the source device 110 to the corresponding communication channel 130 of the destination device 120 indicates a one-way or two-way communication interface, which can be used to send and receive messages, establish connections, acknowledge and exchange any other information related to the communication link and / or data transmission such as encoded bitstream transmission, etc.

[0128] Communication interface 114 can also transmit the bitstream to a storage device. Communication interface 124 can also communicate with the storage device to obtain the bitstream. For example, the storage device can be a readable storage medium, a storage server, a storage gateway, an edge server, a content delivery network (CDN), etc. The CDN can receive and store the bitstream and then send it.

[0129] Decoder 123 is used to decode the bitstream and reconstruct a dynamic grid. For example, decoder 123 performs entropy decoding, inverse quantization, and prediction reconstruction on the bitstream to obtain a reconstructed grid. Decoder 123 may include a filtering unit 1231 and a decoding unit 1232. The filtering unit 1231 filters the reconstructed grid of the decoded grid to obtain a reference grid, so that the decoding unit 1232 can decode and reconstruct the bitstream based on the reference grid to obtain the reconstructed grid. Decoder 123 may decode the bitstream to obtain a first dynamic grid, and reconstruct the first dynamic grid based on the reference grid to obtain a first reconstructed grid. The reference grid is obtained by filtering the reconstructed grid of the decoded grid.

[0130] The post-processor 122 is used to receive the reconstructed mesh generated by the decoder 123 and perform post-processing on the reconstructed mesh. For example, the post-processing performed by the post-processor 122 includes color format conversion (e.g., from YUV format to RGB format), color correction, retouching or resampling, or any other processing.

[0131] Display 121 is used to display the reconstructed mesh. Display 121 can be or can include any class of display devices for presenting the reconstructed mesh, such as integrated or external displays or monitors. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other class of display.

[0132] Both encoder 113 and decoder 123 can be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology is implemented in part in software, the device can store software instructions in a suitable non-transitory computer-readable storage medium, and one or more processors can be used to execute the instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) can be considered as one or more processors.

[0133] The image acquisition unit 111 and the encoder 113 can be integrated into a single physical device or located on different physical devices; there is no limitation on this. For example, such as... Figure 1 The source device 110 shown includes an image acquisition unit 111 and an encoder 113, indicating that the image acquisition unit 111 and the encoder 113 are integrated into a single physical device. Therefore, the source device 110 can also be referred to as an acquisition device. The source device 110 can be, for example, a mobile phone, tablet computer, computer, laptop computer, camera, wearable device, in-vehicle device, terminal device, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, extended reality (XR) device, or other image acquisition device. If the source device 110 does not include the image acquisition unit 111, it means that the image acquisition unit 111 and the encoder 113 are two different physical devices, and the source device 110 can acquire raw images from other devices (such as image acquisition devices or image storage devices).

[0134] In this application, the filter unit 1131 and the encoder 113 can be integrated into one physical device or disposed on different physical devices, without limitation. If the encoder 113 does not include the filter unit 1131, it means that the filter unit 1131 and the encoder 113 are two different physical devices.

[0135] Furthermore, the display 121 and the decoder 123 can be integrated into a single physical device or located on different physical devices; there is no limitation on this. For example, such as... Figure 1The destination device 120 shown includes a display 121 and a decoder 123, indicating that the display 121 and decoder 123 are integrated into a single physical device. Therefore, the destination device 120 can also be referred to as a playback device. The destination device 120 has the function of decoding and displaying the reconstructed image. The destination device 120 can be, for example, a monitor, television, digital media player, video game console, in-vehicle computer, or other image display device. If the destination device 120 does not include the display 121, it means that the display 121 and decoder 123 are two different physical devices. After decoding the bitstream and reconstructing the dynamic mesh, the destination device 120 transmits the reconstructed mesh to other display devices (such as a television or digital media player) for display.

[0136] also, Figure 1 It is shown that the source device 110 and the destination device 120 can be integrated into one physical device or set on different physical devices, without limitation.

[0137] For example, such as Figure 2 The diagram (a) shows a schematic of an encoding / decoding system. Source device 110 can be a server, which may be a server in a cloud system. Destination device 120 can be a display of various possible forms. Source device 110 can acquire video of a first scene and generate a dynamic mesh based on multiple frames of original images from the video; alternatively, source device 110 can generate a dynamic mesh; or, source device 110 can obtain a dynamic mesh from a storage device. Source device 110 includes an encoding module and an encapsulation module. The encoding module encodes the dynamic mesh to obtain a bitstream, and the encapsulation module encapsulates the bitstream and other information to obtain a message, which is then transmitted through a channel.

[0138] The destination device 120 includes a decapsulation module and a decoding module. The decapsulation module receives packets from the channel and parses the bitstream from the packets. The decoding module decodes and reconstructs the bitstream to obtain the reconstructed mesh. Optionally, the destination device 120 can also render, store, or display the reconstructed mesh.

[0139] For example, such as Figure 2The diagram in (b) shows a schematic of an encoding / decoding system where source device 110 and destination device 120 are integrated into the same device, such as a smartphone, tablet, computer, laptop, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, or extended reality (XR) device. In this case, the device has the function of encoding and decoding dynamic meshes. For example, source device 110 can acquire images of the user's real-world scene, generate a dynamic mesh based on the images, encode the dynamic mesh, and store the bitstream. Destination device 120 can then display the reconstructed mesh of the real-world scene in a virtual environment.

[0140] In these embodiments, the source device 110 or its corresponding functions and the destination device 120 or its corresponding functions may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof. As described, Figure 1 The presence and division of different units or functions in the source device 110 and / or destination device 120 shown may vary depending on the actual device and application, which is obvious to those skilled in the art.

[0141] The structure of the above-described encoding / decoding system is only illustrative. In some possible implementations, the encoding / decoding system may also include other devices, such as end-side devices or cloud-side devices. After the source device 110 acquires the original image, it preprocesses the original image to obtain a preprocessed image; and then transmits the preprocessed image to the end-side device or cloud-side device, which performs encoding / decoding on the preprocessed image.

[0142] The image encoding / decoding method provided in this application is applied to both the encoder and decoder. Figure 3 The structure of the encoder and decoder is described in detail. For example... Figure 3 As shown, the encoder 300 includes a prediction unit 310, a quantization unit 320, a residual coding unit 330, a decoding and reconstruction unit 340, and a filtering unit 350.

[0143] The prediction unit 310 is used to perform intra-frame prediction and / or inter-frame prediction based on the vertex position and / or attribute information according to the reference grid to obtain the prediction result, and outputs the dynamic grid and the residual of the prediction result. For an explanation of intra-frame prediction and inter-frame prediction, please refer to the description of dynamic grid prediction in MPEG V-DMC.

[0144] Intra-frame prediction is a common method for removing spatial redundancy within a frame. For example, reconstructed regions of adjacent coded regions within the frame are used as reference values ​​to predict the current coded region. Since the coded region within the frame is correlated with its surrounding coded regions, the surrounding reconstructed coded regions can be used to estimate the current coded region, resulting in a predicted value. The residual between the predicted value and the original value of the current coded region is quantized and entropy-encoded. The predicted residual can be transmitted instead of the original value. The decoding end performs a reconstruction process corresponding to the prediction at the encoding end to obtain the predicted value of the current coded region. Then, the reconstructed value of the current coded region is obtained by adding the predicted value to the decoded residual.

[0145] Inter-frame prediction is a common method for removing redundant information between frames. For example, using the reconstructed grid from the inter-frame grid as a reference value to predict the current grid is equivalent to the inter-frame grid having a correlation with the current grid. The vertex positions of the current grid can be estimated using the inter-frame grid, and the estimated vertex positions are the predicted values. The residual between the predicted values ​​and the original values ​​of the current grid is then quantized and entropy-encoded. Since the changes between adjacent frames are small, this similarity can be used to predict the content of the current frame, thus requiring only the transmitted data of the changed parts instead of the entire frame. This method can improve coding efficiency and ensure reconstruction quality.

[0146] In this application, inter-frame prediction is performed on the vertex positions and / or attribute information of each grid cell to be encoded in the dynamic mesh. Alternatively, intra-frame prediction is performed on the vertex positions and / or attribute information of each grid cell in the dynamic mesh. Alternatively, both intra-frame and inter-frame prediction are performed on the vertex positions and / or attribute information of each grid cell in the dynamic mesh.

[0147] Optionally, the encoder 300 further includes a block partitioning unit 360. The block partitioning unit 360 is used to divide the dynamic mesh into multiple encoding regions. In this embodiment, the encoding region may refer to a portion of the dynamic mesh. The encoding region contains vertex positions and attribute information of a portion of the dynamic mesh. The reference mesh includes a portion of the encoded region in the mesh to be encoded. Block partitioning methods include clustering and mesh segmentation, etc.

[0148] The prediction unit 310 is used to perform intra-frame prediction and / or inter-frame prediction on the coding region output by the block partitioning unit 360 according to the reference grid to obtain the prediction result and output the residual of the coding region.

[0149] The quantization unit 320 is used to quantize the residual output by the prediction unit 310 to obtain the quantized residual.

[0150] The residual coding unit 330 is used to encode the quantization residual output by the quantization unit 320 to obtain the dynamic grid bitstream. For example, entropy coding is performed on the quantization residual output by the quantization unit 320.

[0151] The decoding and reconstruction unit 340 is used to decode and reconstruct the encoded mesh to obtain the reconstructed mesh.

[0152] The filtering unit 350 is used to filter the reconstructed grid of the encoded grid to obtain a reference grid. For example, by adjusting the vertex positions of the reconstructed grid, the reference grid obtained by filtering the reconstructed grid is made closer to the encoded grid, reducing the error between the reconstructed grid and the corresponding encoded grid. Furthermore, the reference grid obtained by filtering the reconstructed grid is used to predict the dynamic grid. The prediction result is more accurate, so the grid can be represented with as little data as possible, thereby improving coding efficiency and reducing the code rate.

[0153] The units included in the encoder 300 described above are merely examples; in some embodiments, it may contain more or fewer units. For example, the encoder may not include a residual encoding unit and may directly transmit the quantization residual.

[0154] Combination Figure 3 The structure of the decoder is described in detail. For example... Figure 3 As shown, the decoder 400 includes a residual decoding unit 410, an inverse quantization unit 420, a prediction reconstruction unit 430, and a filtering unit 440.

[0155] The residual decoding unit 410 is used to perform residual decoding on the bitstream to obtain the content of the dynamic grid, for example, the content of the dynamic grid includes quantization residuals.

[0156] The dequantization unit 420 is used to dequantize the quantized residual output by the residual decoding unit 410 to obtain the residual.

[0157] The prediction reconstruction unit 430 is used to predict and reconstruct the residual output by the inverse quantization unit 420 based on the reference grid to obtain the reconstructed grid, so that the reconstructed grid can be displayed on the display.

[0158] The filtering unit 440 is used to filter the reconstructed grid of the decoded grid to obtain a reference grid. For example, by adjusting the vertex positions of the reconstructed grid, the reference grid obtained by filtering the reconstructed grid is made closer to the encoded grid, reducing the error between the reconstructed grid and the corresponding encoded grid. Furthermore, the reference grid obtained by filtering the reconstructed grid is used to decode the bitstream, making the reconstructed grid more accurate.

[0159] To address the high bitrate issue of dynamic lattice coding, this application provides a dynamic lattice encoding and decoding method. This method involves adding a filtering module at the encoding end to filter the reconstructed lattice of the encoded lattice, making the resulting lattice closer to the encoded lattice. The filtered lattice is then used as a reference lattice for predicting the dynamic lattice, resulting in more accurate predictions. The dynamic lattice to be encoded is then encoded based on these predictions to obtain the bitstream. This allows for the representation of the lattice with minimal data, reducing the bitrate. Similarly, at the decoding end, a filtering module filters the reconstructed lattice of the decoded lattice, making the resulting lattice closer to the encoded lattice. This filtered lattice is then used as a reference lattice for decoding and reconstructing the dynamic lattice, resulting in a more accurate reconstructed lattice, closer to the encoded lattice. This reduces reconstruction errors and minimizes the loss of essential details in the lattice, thus improving the overall reconstruction quality.

[0160] Next, the dynamic mesh encoding and decoding process will be explained with reference to the accompanying diagram. Figure 4 Figure (a) shows a flowchart of a dynamic mesh coding method provided in this application. Figure 1 The following explanation uses the dynamic mesh encoding / decoding process performed by source device 110 and destination device 120 as an example. Source device 110 may contain an encoder, and destination device 120 may contain a decoder. Figure 4 As shown in (a) of the diagram, the method includes the following steps.

[0161] Step 410: Obtain the dynamic mesh to be encoded.

[0162] The source device 110 can acquire multiple frames of images and generate a dynamic mesh of the multiple frames using dynamic mesh generation technology. The images may include at least one of the following: real-time acquired real-world images, images stored in the device, and images synthesized from multiple images. This embodiment does not limit the method of image acquisition or the type of images.

[0163] Optionally, the source device 110 may also receive a dynamic mesh generated by another device; or, it may obtain the dynamic mesh from the memory in the source device 110 or other memory. This embodiment does not limit the method of obtaining the dynamic mesh.

[0164] A dynamic mesh can refer to the dynamic mesh input to the encoder. A dynamic mesh contains multiple meshes. The encoder can encode multiple meshes within the dynamic mesh to generate a bitstream. The multiple meshes contained in the dynamic mesh input to the encoder can also be called the input mesh or the mesh to be encoded.

[0165] Step 420: Predict the dynamic grid to be encoded based on the reference grid to obtain the prediction result of the dynamic grid to be encoded.

[0166] Step 430: Encode the dynamic grid to be encoded according to the prediction results to obtain the bitstream.

[0167] The data in a dynamic mesh includes vertex positions, attribute information, and vertex connectivity. The encoding method for vertex positions in a dynamic mesh will be explained below.

[0168] The encoder makes predictions on multiple grids contained in the dynamic grid to obtain the prediction result for each grid; the prediction result for each grid is then residual-coded to obtain the bitstream.

[0169] The prediction can include inter-frame prediction and / or intra-frame prediction. For example, the encoder can perform inter-frame prediction on multiple grids contained in a dynamic mesh to obtain the prediction result for each grid. Alternatively, the encoder can perform intra-frame prediction on multiple grids contained in a dynamic mesh to obtain the prediction result for each grid. Or, the encoder can perform inter-frame prediction on some grids and intra-frame prediction on others to obtain the prediction result for each grid. The encoder can also perform intra-frame and inter-frame prediction on a single grid to obtain the prediction result for that grid.

[0170] Prediction methods may include motion vectors, predicted motion vectors, motion vector residuals, or residual prediction, etc.

[0171] The encoding of the dynamic grid to be encoded based on the prediction results described in this application may refer to residual encoding of the dynamic grid to be encoded based on the prediction results.

[0172] Residual coding can include quantization and entropy coding. For example, the encoder quantizes the residuals of the grid contained in the prediction result, and then entropy codes the quantized residuals to obtain the bitstream. Quantization operations can be performed in various ways, such as scalar quantization or vector quantization.

[0173] The encoder completes encoding of a grid, performs residual decoding and reconstruction on the encoded grid to obtain a reconstructed grid, and filters the reconstructed grid of the encoded grid to obtain a set of decoded grids. The grids in the set of decoded grids are used as reference grids.

[0174] Residual decoding is the inverse process of residual coding, and reconstruction is the inverse process of prediction. For example, residual coding includes quantization and entropy coding. Residual decoding includes inverse quantization and entropy decoding. When the encoder performs residual decoding and reconstruction on the encoded results of the encoded mesh, it performs entropy decoding on the encoded results of the mesh to obtain the quantized residual, then performs inverse quantization on the quantized residual to obtain the residual, and finally performs prediction and reconstruction on the residual to obtain the reconstructed mesh.

[0175] Filtering the reconstructed mesh can reduce its noise, allowing the reference mesh obtained by filtering the reconstructed mesh of the already encoded mesh to be used for prediction and encoding of other meshes to be encoded, thus improving the accuracy of the prediction results. Noise can be features that the encoded mesh itself does not possess, such as the unevenness of the edges of the reconstructed mesh; or it can be the difference between the vertices of the encoded mesh and the vertices of the reconstructed mesh.

[0176] In some embodiments, the encoder filters the reconstructed grid based on the encoded grid, making the filtered grid closer to the encoded grid. Using the filtered grid obtained by filtering the reconstructed grid as a reference grid, more accurate prediction results can be obtained when predicting other grids based on the reference grid. This allows for the use of as little data as possible to represent the grid, thereby ensuring the quality of the reconstructed grid, improving coding efficiency, and reducing the bit rate.

[0177] For example, assume the dynamic grid comprises a first grid and a second grid. After prediction and residual coding of the first grid, an encoded result of the first grid is obtained. Residual decoding and reconstruction of the encoded result of the first grid are then performed to obtain a first reconstructed grid. The first reconstructed grid is then filtered based on the first grid to make the first filtered grid more closely resemble the first grid. The reference grid includes the first filtered grid. After prediction and residual coding of the second grid based on the reference grid (which may be the first filtered grid), an encoded result of the second grid is obtained. Residual decoding and reconstruction of the encoded result of the second grid are then performed to obtain a second reconstructed grid. The second reconstructed grid is then filtered based on the second grid to make the second filtered grid more closely resemble the second grid. The reference grid also includes the second filtered grid.

[0178] This application provides multiple options for reconstructing the mesh by performing filtering.

[0179] In the first possible implementation, the encoder and decoder filter the reconstructed mesh for each mesh in the dynamic mesh by default.

[0180] The encoder encodes each grid in the dynamic mesh and then filters the reconstructed grid of each encoded grid. The reference grid consists of the filtered grids of multiple reconstructed grids.

[0181] For example, the encoder and decoder are pre-programmed to filter each vertex position in the reconstructed mesh. The encoder can adjust the vertex positions of the reconstructed mesh based on the vertex positions of the corresponding encoded mesh.

[0182] In the second possible implementation, the encoder may not need to filter the reconstructed mesh for each mesh in the dynamic mesh. Instead, the encoder can first determine whether to filter the reconstructed mesh. For example, the encoder compares the differences between the encoded mesh and the reconstructed mesh, and determines whether to filter the reconstructed mesh based on these differences.

[0183] If the encoder determines that filtering of the reconstructed mesh is necessary, it means that the difference between the reconstructed mesh and the encoded mesh is large, and the encoder filters the reconstructed mesh based on the encoded mesh. If the encoder determines that filtering of the reconstructed mesh is not necessary, it means that the difference between the reconstructed mesh and the encoded mesh is small, and the encoder does not filter the reconstructed mesh.

[0184] In some embodiments, if the difference between the reconstructed mesh and the encoded mesh is greater than a threshold, the encoder determines that filtering of the reconstructed mesh is necessary. If the difference between the reconstructed mesh and the encoded mesh is less than a threshold, the encoder determines that filtering of the reconstructed mesh is not necessary.

[0185] In some embodiments, the encoder may determine whether to filter the reconstructed mesh based on the difference between the vertex positions of the encoded mesh and the vertex positions of the reconstructed mesh. Alternatively, the encoder may determine whether to filter the reconstructed mesh based on the difference between the normals of the encoded mesh and the normals of the reconstructed mesh.

[0186] For example, the above differences can be measured using methods such as mean square error between vertex positions or peak signal-to-noise ratio.

[0187] Optionally, this application does not limit the number of times a reconstructed mesh is filtered. The encoder can perform multiple filtering operations on a reconstructed mesh to reduce noise in the reconstructed mesh. For example, after the encoder performs filtering on the reconstructed mesh once, it can also compare the difference between the encoded mesh and the filtered mesh to determine whether to filter the filtered mesh again.

[0188] In a third possible implementation, the encoder does not need to filter every vertex position in the reconstructed mesh. Instead, the encoder can first determine whether to filter the vertex positions in the reconstructed mesh, and then filter the vertex positions of the determined subset of vertices in the reconstructed mesh.

[0189] For example, if the vertex positions of the same vertex differ between the encoded mesh and the reconstructed mesh, the encoder compares the differences between the vertex positions in the encoded mesh and the reconstructed mesh to determine whether to filter the vertex positions in the reconstructed mesh. For instance, a preset threshold can be used to determine whether to filter the vertex positions in the reconstructed mesh.

[0190] If the encoder determines that filtering of the vertex positions of the reconstructed mesh is necessary, it means that the difference between the vertex positions of the reconstructed mesh and the coded mesh is large. In this case, the encoder filters the vertex positions of the reconstructed mesh based on the vertex positions of the coded mesh. If the encoder determines that filtering of the vertex positions of the reconstructed mesh is not necessary, it means that the difference between the vertex positions of the reconstructed mesh and the coded mesh is small. In this case, the encoder does not filter the reconstructed mesh.

[0191] In a fourth possible implementation, the encoder can divide the mesh into multiple patches, and determine whether to filter multiple patches in the reconstructed mesh. A patch can contain multiple vertex positions.

[0192] For example, if the same block in the encoded grid and the reconstructed grid differs, the encoder compares the differences between the blocks in the encoded grid and the reconstructed grid to determine whether to filter the blocks in the reconstructed grid. For example, comparing block 1 in the encoded grid and block 1 in the reconstructed grid, since block 1 has been reconstructed, the position of the reconstructed block 1 may have changed relative to block 1 in the encoded grid, so the reconstructed block 1 needs to be filtered.

[0193] If the encoder determines that filtering is needed for blocks of the reconstructed mesh, it means that the difference between the reconstructed mesh blocks and the already encoded mesh blocks is large. In this case, the encoder filters the reconstructed mesh blocks based on the already encoded mesh blocks. If the encoder determines that filtering is not needed for blocks of the reconstructed mesh, it means that the difference between the reconstructed mesh blocks and the already encoded mesh blocks is small.

[0194] Filtering a block of the reconstructed mesh can refer to filtering the vertex positions of multiple vertices contained in that block as a whole. For example, if we treat the multiple vertex positions contained in a block of the reconstructed mesh as a whole, and take one vertex position contained in that block as the vertex position of the block, then filtering the block of the reconstructed mesh can refer to filtering the vertex positions of the block as a whole.

[0195] In a fifth possible implementation, the encoder can filter a specified region. For example, the specified region could be a boundary region within a grid, such as the fingertips of a hand.

[0196] The encoder can also first determine whether to filter a specified region of the reconstructed mesh. For example, if there is a difference between a specified region of the encoded mesh and the reconstructed mesh, the encoder compares the differences between the specified regions of the encoded mesh and the reconstructed mesh to determine whether to filter the specified region of the reconstructed mesh.

[0197] It should be understood that the methods for determining the differences and determining whether to perform filtering in the above implementation methods can be referenced from each other. To avoid redundancy, they have not been described in detail.

[0198] In some embodiments, after filtering the reconstructed mesh, the encoder can further compensate for some or all vertices in the filtered mesh to restore more details and make the filtered mesh closer to the encoded mesh. Using the filtered mesh as a reference mesh, when predicting other meshes based on the reference mesh, the mesh is represented using as little data as possible. This ensures the quality of the reconstructed mesh, improves coding efficiency, and reduces the bit rate.

[0199] For example, the encoder can determine the compensation value based on the difference between the encoded mesh and the filtered mesh obtained by filtering the reconstructed mesh.

[0200] For example, the encoder can determine the compensation value based on the difference between the encoded mesh and the filtered mesh obtained by filtering the reconstructed mesh, and the constraint value. For instance, if the constraint value is 10, and the difference between the encoded mesh and the filtered mesh obtained by filtering the reconstructed mesh is 9, the compensation value is 0 because the difference is less than the constraint value; if the difference between the encoded mesh and the filtered mesh obtained by filtering the reconstructed mesh is 20, the compensation value is 10 because the difference is greater than the constraint value.

[0201] The reference grid used for predicting the dynamic grid to be encoded may be obtained by filtering the reconstructed grid of the encoded grid or by filtering and compensating the reconstructed grid of the encoded grid.

[0202] The filtering algorithm used by the encoder to filter the reconstructed mesh is not limited in this application. For example, the filtering algorithm includes at least one of Laplace smoothing, Taubin smoothing, or depth smoothing.

[0203] In this embodiment, the encoder can use a filtering algorithm to filter the reconstructed meshes of multiple meshes contained in the dynamic mesh. Alternatively, the encoder can use different filtering algorithms to filter the reconstructed meshes of different meshes. For example, the encoder can use a first filtering algorithm to filter the reconstructed meshes of a portion of the meshes contained in the dynamic mesh, and the encoder can use a second filtering algorithm to filter the reconstructed meshes of another portion of the meshes contained in the dynamic mesh. As another example, the encoder can use different filtering algorithms for different regions in the first mesh of the dynamic mesh. Furthermore, the encoder can perform multiple filtering operations on the first mesh, using different filtering algorithms, the same filtering algorithm, or the same filtering algorithm with different filtering parameters, etc. The method for selecting the filtering algorithm can include selection based on region or selection based on differences in vertex positions, etc.

[0204] In some embodiments, the encoded mesh includes a preceding mesh. A preceding mesh can also be referred to as a prior mesh. A prior mesh is a mesh that has been encoded at a time prior to the current time. A prior mesh can be a mesh from the time before the current time or from several previous time points. The current mesh is the mesh being encoded at the current time. The current mesh can indicate the mesh to be encoded that currently needs to be encoded.

[0205] After the encoder finishes encoding a grid, it filters the reconstructed grid of the encoded grid and uses the filtered grid as a reference grid for predicting subsequent grids.

[0206] At the encoding end, the reference grid includes at least one grid obtained by filtering the reconstructed grid of the encoded grid.

[0207] In other embodiments, the encoded mesh includes a portion of the encoded region within the mesh to be encoded. The portion of the encoded region includes one or more encoded vertex regions, encoded blocks, or encoded designated regions.

[0208] After the encoder encodes a portion of a grid, it filters the reconstructed region of the partially encoded region and uses the filtered region to predict other unencoded regions of the current grid.

[0209] For example, after the encoder has encoded one or more vertex regions in the grid, it filters the reconstructed vertices of those one or more vertices and uses the filtered vertices to predict other vertices in the current grid.

[0210] For example, after the encoder encodes a block in the grid, it filters the reconstructed block of that block. The filtered block is then used to predict other blocks in the current grid. This filtering process involves filtering the reconstructed vertices of one or more vertices, filtering each vertex individually, and applying different levels of filtering to each vertex. Finally, filtering is applied to the reconstructed block of a block, and then filtering is performed on all vertices contained within that block. By encoding blocks in the grid and filtering their reconstructed blocks, the encoder reduces its computational load.

[0211] For example, after the encoder finishes encoding a specified region in the grid, it filters the reconstructed region of the specified region and uses the filtered region to predict other regions in the current grid.

[0212] Optionally, the source device 110 can divide the grid to be encoded into multiple regions and encode the multiple regions.

[0213] At the encoding end, the reference grid includes at least one reference information obtained by filtering a portion of the encoded region in the grid to be encoded.

[0214] For example, when the encoder predicts the first grid in a dynamic grid, the reference grid includes reference information obtained by filtering at least one portion of the encoded region in the grid to be encoded. The encoder can predict the first grid using an intra-frame prediction method based on the reference grid to obtain the prediction result of the first grid. The encoder then encodes the first grid based on the prediction result to obtain the encoded result of the first grid. Furthermore, the encoder decodes and reconstructs the encoded result of the first grid to obtain a reconstructed grid. The reconstructed grid is then filtered to obtain a first filtered grid, which is used as the reference grid. Optionally, the encoder can also compensate for the filtering result of the reconstructed grid to obtain the first filtered grid.

[0215] Optionally, the encoded mesh includes the preceding mesh and a portion of the encoded region in the mesh to be encoded. For example, the encoded mesh includes the encoded and filtered vertices in the mesh to be encoded and the filtered vertices in the encoded mesh.

[0216] The reference grid includes at least one grid obtained by filtering the reconstructed grid of the encoded grid, and at least one reference information obtained by filtering a portion of the encoded region in the grid to be encoded.

[0217] For example, such as Figure 5 As shown, when predicting the vertex C2 to be encoded, the reference vertices include C1, A2, B2, G2, and D2. Among them, C1 is a vertex in the already encoded grid, and A2, B2, G2, and D2 are all derived from the filtered vertices in the grid to be encoded.

[0218] Therefore, the reference vertex for predicting the vertex to be encoded may include the reference vertex corresponding to the previous frame, and may also include at least one vertex in a partially encoded region of the grid to be encoded that is connected to the vertex to be encoded.

[0219] For example, when the encoder predicts the second and subsequent grids in a dynamic grid, the reference grid includes at least one grid obtained by filtering the reconstructed grid of the already encoded grid, and at least one reference information obtained by filtering a portion of the encoded region in the grid to be encoded. The grid to be encoded is predicted based on the reference grid to obtain the prediction result, and the grid to be encoded is encoded based on the prediction result to obtain the bitstream. The reference grid is obtained by filtering the vertex positions of the reconstructed grid of the already encoded grid.

[0220] After the encoder encodes the dynamic mesh to be encoded to obtain a bitstream, it sends the bitstream so that the decoder can receive the bitstream, decode and reconstruct the mesh to obtain the reconstructed mesh.

[0221] In some embodiments, the encoder can send a bitstream after encoding multiple grids contained in the dynamic grid, the bitstream containing the encoding information of the dynamic grid.

[0222] In other embodiments, the encoder may send a bitstream containing the encoded information of a grid after encoding it.

[0223] In other embodiments, the encoder may send a bitstream containing the encoded information of a grid after encoding a portion of a grid.

[0224] In other embodiments, the bitstream includes auxiliary parameters to enable the decoder to decode and reconstruct the bitstream based on the auxiliary parameters, thereby improving the accuracy of the reconstructed grid obtained from the decoder.

[0225] Auxiliary parameters may include at least one of the following: grid filter identifier, region filter identifier, filter coefficient, compensation information, or reference grid identifier.

[0226] The grid filtering flag is used to indicate whether grid filtering is applied to the reconstructed grid. The grid filtering flag can be 1 bit. Different bit values ​​indicate whether grid filtering is applied to the reconstructed grid. For example, a value of 1 indicates that grid filtering is applied to the reconstructed grid; a value of 0 indicates that grid filtering is not applied to the reconstructed grid. Similarly, a value of 1 indicates that grid filtering is not applied to the reconstructed grid; a value of 0 indicates that grid filtering is applied to the reconstructed grid.

[0227] A region filtering identifier is used to indicate whether to filter regions within the reconstructed grid. The region filtering identifier can be 1 bit. Different bit values ​​indicate whether filtering is applied to regions within the reconstructed grid. For example, a region filtering identifier value of 1 indicates filtering of the reconstructed region; a value of 0 indicates no filtering. Similarly, a region filtering identifier value of 1 indicates no filtering of the reconstructed region; a value of 0 indicates filtering of the reconstructed region. The encoder and decoder can pre-define the region division method. For example, the bitstream can contain region identifiers and region filtering identifiers, allowing the decoder to determine regions based on the region identifiers and whether to filter the reconstructed region based on the region filtering identifiers. Alternatively, only the region filtering identifier can be used to filter the region indicated by it; that is, the region filtering identifier indicates one or more regions within the reconstructed grid.

[0228] A region can include one or more vertex regions, encoded blocks, or encoded specified regions.

[0229] Region filtering identifiers include region filtering identifiers for one or more vertex regions, region filtering identifiers for encoded blocks, or region filtering identifiers for encoded specified regions.

[0230] The filtering coefficients include at least one of the following: the identifier of the filtering algorithm, the filtering weights, the filtering parameters, or the number of filtering iterations.

[0231] The compensation information may include compensation values ​​for the reconstructed mesh and / or for the region. Optionally, the compensation information may include indications as to whether to compensate the filtered reconstructed mesh.

[0232] The reference grid identifier is used to indicate the reference grid. The reference grid can be a specified filtered grid, such as the previous grid of the grid to be encoded. Alternatively, the reference grid can be selected as either a filtered grid or a reconstructed grid by an identifier. For example, in one implementation, an identifier of 1 indicates that the reference grid is the filtered grid of the previous grid, and an identifier of 0 indicates that the reference grid is the reconstructed grid of the previous grid.

[0233] In this approach, the encoder and decoder can pre-agree on auxiliary parameters, eliminating the need to transmit them. The bitstream may also omit these auxiliary parameters. For example, if the encoder and decoder agree to filter each grid in the dynamic grid, the bitstream does not need to include a grid filtering identifier. Similarly, if the encoder and decoder pre-agree on the calculation rules for compensation values ​​and the filtering algorithm, the bitstream does not need to include compensation information and filtering coefficients. Furthermore, if the encoder and decoder pre-agree that the reference grid is the previous grid of the current grid, the bitstream does not need to include a reference grid identifier.

[0234] The bitstream may also contain one or more auxiliary parameters. For example, if the encoder filters a portion of the grid in a dynamic grid, the bitstream contains a grid filtering identifier; or, if the encoder filters a portion of the grid while other portions are not filtered, the bitstream contains a region filtering identifier; or, if the encoder compensates for the grid or a portion of the grid, the bitstream contains compensation values ​​for the filtered vertices in the grid.

[0235] The auxiliary parameters are determined based on the grid to be encoded and its reconstructed grid, and / or based on the grid to be encoded and the reference grid.

[0236] For example, the encoder can determine whether to filter the reconstructed grid based on the difference between the grid to be encoded and the reconstructed grid, and filter a portion of the reconstructed grid of the grid to be encoded. The explanation of this portion of the grid can be found above.

[0237] For example, the encoder can determine whether to filter the reconstructed grid based on the difference between the grid to be encoded and the reference grid, and filter a portion of the reconstructed grid of the grid to be encoded.

[0238] The above embodiment uses the encoding of vertex positions in a dynamic mesh as an example for illustration. The dynamic mesh data can also include attribute information. The encoder can further predict and encode the attribute information, transmitting the attribute information bitstream. The method for encoding the attribute information of a dynamic mesh can be referred to the above description of encoding the vertex positions in a dynamic mesh, and will not be repeated here.

[0239] Source device 110 can send a bitstream to destination device 120 after encoding a dynamic mesh or a dynamic mesh. Alternatively, source device 110 can encode a mesh in real time, sending a bitstream for each mesh after encoding. Or, source device 110 can encode regions within a mesh in real time, sending a bitstream for each region after encoding.

[0240] The specific method for sending the bitstream can refer to conventional techniques and the descriptions of communication interfaces 114 and 124 in the above embodiments. Optionally, the source device 110 can also encapsulate the bitstream to obtain a message and send the message.

[0241] The dynamic mesh encoding and decoding method provided in this application mainly encodes and decodes vertex positions and attribute information. The encoding and decoding method for connection relationships can refer to traditional methods and is not limited here.

[0242] The above embodiments illustrate the encoding method for dynamic meshes. The decoding method for dynamic meshes will be described below. Figure 4 As shown in (b) of this application, this is a flowchart illustrating a dynamic mesh decoding method. The dynamic mesh decoding method includes the following steps.

[0243] Step 440: Obtain the bitstream.

[0244] Destination device 120 receives a bitstream from source device 110. Alternatively, destination device 120 generates a bitstream or retrieves a bitstream from its own memory or other memory. The bitstream may include a dynamic grid bitstream, a single grid bitstream, or a bitstream of a region within a grid.

[0245] Step 450: Process the bitstream to obtain the first dynamic grid.

[0246] Step 460: Reconstruct the first dynamic mesh based on the reference mesh to obtain the first reconstructed mesh.

[0247] The decoder performs residual decoding on the bitstream to obtain the first dynamic grid, and reconstructs the first dynamic grid to obtain the first reconstructed grid. The first reconstructed grid can refer to the reconstructed grid of the first dynamic grid.

[0248] Residual decoding is the inverse process of residual coding, and reconstruction is the inverse process of prediction. Residual decoding includes entropy decoding and dequantization. For example, the decoder performs entropy decoding on the bitstream to obtain the quantized residual of the dynamic grid, dequantizes the quantized residual to obtain the residual, and reconstructs the residual to obtain the reconstructed grid.

[0249] The decoder decodes and reconstructs the bitstream to obtain a reconstructed grid. It then filters the reconstructed grid to obtain a filtered grid, which is used as a reference grid. For example, filtering the reconstructed grid of a decoded grid yields a set of decoded grids. The grids in this set are then used as the reference grid.

[0250] Filtering the reconstructed mesh can reduce noise and make it closer to the encoded mesh at the encoding end. Using the filtered mesh as a reference mesh, other meshes can be reconstructed based on this reference mesh. This approach minimizes errors in the reconstructed mesh while preserving as much detail as possible, resulting in a dynamic mesh reconstruction and improved overall reconstruction quality.

[0251] Since the decoder cannot obtain the input mesh to the encoder corresponding to the reconstructed mesh, the decoder can filter the reconstructed mesh according to predetermined rules. For example, rules related to filtering, compensation, and reference mesh are pre-defined.

[0252] Example 1: The decoder pre-determines to filter the reconstructed grid of each grid in the dynamic grid. After receiving the bitstream, the decoder filters the reconstructed grid of each decoded grid.

[0253] Example 2: The decoder pre-determines to filter the reconstructed mesh. For example, the decoder pre-determines to filter each vertex position in the reconstructed mesh. After receiving the bitstream, the decoder filters each vertex position in the reconstructed mesh.

[0254] Example 3: The decoder pre-defines the compensation for the filtered mesh, and the calculation rules for the compensation value are pre-defined. The calculation rules for the compensation value are as described in the previous embodiment. The decoder compensates the filtered mesh according to the calculated compensation value.

[0255] Example 4: The decoder pre-defines a reference grid. For example, the decoder can pre-define the reference grid as the previous grid of the current grid.

[0256] In other embodiments, the bitstream contains auxiliary parameters, from which the decoder can obtain the auxiliary parameters and determine relevant information such as filtering, compensation, and reference grid based on the auxiliary parameters, and then filter the reconstructed grid.

[0257] In Example 1, the decoder may not need to filter the reconstructed grid for each grid in the dynamic grid. Instead, the decoder can first determine whether to filter the reconstructed grid. For example, the bitstream includes auxiliary parameters, such as a grid filtering flag, which indicates whether to filter the reconstructed grid. If the grid filtering flag indicates that the first reconstructed grid should be filtered, the decoder filters the first reconstructed grid to obtain the filtered grid.

[0258] Example 2: The decoder may not need to filter every vertex position in the reconstructed mesh. The decoder can first determine whether to filter the vertex positions in the reconstructed mesh. For example, the bitstream includes auxiliary parameters, including a region filtering flag, which indicates whether to filter regions in the reconstructed mesh. If the region filtering flag indicates filtering of one or more vertices in the first reconstructed mesh, then one or more vertices in the first reconstructed mesh are filtered to obtain a filtered mesh. If the region filtering flag indicates filtering of blocks in the first reconstructed mesh, then blocks in the first reconstructed mesh are filtered to obtain a filtered mesh. If the region filtering flag indicates filtering of a specified region in the first reconstructed mesh, then the specified region in the first reconstructed mesh is filtered to obtain a filtered mesh.

[0259] After filtering the reconstructed mesh, the decoder can compensate for some or all vertices in the filtered mesh to further reduce noise and make it closer to the encoded mesh at the encoding end. Using the filtered mesh as a reference mesh, other meshes are reconstructed based on it, ensuring the quality of the reconstructed mesh while improving the overall reconstruction quality.

[0260] Example 3: The bitstream includes auxiliary parameters, which also include compensation information. The compensation information is used to determine the compensation value for the first reconstructed grid. The decoder filters the first reconstructed grid based on the compensation information to obtain the filtered grid.

[0261] Optionally, the auxiliary parameters may also include at least one of the following: filter coefficients, filter rules, or reference grid identifiers.

[0262] The decoder determines at least one of the following: filtering arithmetic, filtering weight, or filtering order, based on the filtering coefficients, and then filters the reconstructed grid according to the filtering coefficients. The filtering algorithm used at the encoding and decoding ends is the same.

[0263] The reference grid identifier is used to indicate a reference grid. For example, the reference grid identifier indicates that the reference grid is the filtered version of the previous grid. Or, the reference grid identifier indicates that the reference grid is the filtered version of the previous two grids.

[0264] For example, such as Figure 6As shown, auxiliary parameters may include grid filter identifier, area filter identifier, filter coefficients, and compensation values.

[0265] The decoder determines whether to filter the reconstructed grid based on the grid filtering flag (step 610). If the grid filtering flag indicates that the reconstructed grid should be filtered, the decoder filters the reconstructed grid.

[0266] Furthermore, the decoder determines whether to filter regions (e.g., vertices or blocks) in the reconstructed mesh based on the region filtering identifier (step 620). If the mesh filtering identifier indicates that regions in the reconstructed mesh should be filtered, the decoder will filter the regions in the reconstructed mesh.

[0267] Furthermore, the decoder filters the regions in the reconstructed grid based on the filtering coefficients (step 630).

[0268] Furthermore, the decoder filters the regions in the reconstructed mesh according to the filtering coefficients, and then compensates for the filtering results to obtain the filtered mesh (step 640).

[0269] It should be understood that steps 610, 620, and 640 above are optional, and their execution can be determined based on auxiliary parameters in the bitstream. Furthermore, the above steps can be combined or performed independently.

[0270] The reference mesh is obtained by filtering the reconstructed mesh from the decoded mesh. The reference mesh consists of one or more filtered meshes.

[0271] The decoded mesh includes the preceding mesh. The preceding mesh can also be called the prior mesh. The prior mesh refers to the mesh that has been decoded at a time point before the current time. The prior mesh can be the mesh from the time point immediately preceding the current time or from several previous time points. The current mesh refers to the mesh that is being decoded at the current time. The current mesh can indicate the mesh that needs to be decoded currently, as described above.

[0272] After the decoder finishes decoding a grid, it filters the reconstructed grid of the decoded grid and uses the filtered grid as a reference grid to predict subsequent grids.

[0273] At the decoding end, the reference grid includes at least one grid obtained by filtering the reconstructed grid of the decoded grid.

[0274] In other embodiments, the decoded mesh includes a portion of the decoded region within the mesh to be decoded. The portion of the decoded region includes one or more decoded vertex regions, decoded blocks, or a specified decoded region.

[0275] The reference grid includes at least one grid obtained by filtering the reconstructed grid of the decoded grid, and at least one reference information obtained by filtering a portion of the decoded region in the grid to be decoded.

[0276] After the decoder has decoded a portion of a grid, it filters the reconstructed region of the partially decoded region and uses the filtered region to predict other undecoded regions of the current grid.

[0277] For example, such as Figure 7 As shown, after the decoder decodes one or more vertex regions in the grid, it filters the reconstructed vertices of one or more vertices and uses the filtered vertices to predict other vertices in the grid to be decoded.

[0278] For example, after the decoder decodes a block in the grid, it filters the reconstructed block of the block and uses the filtered block to predict other blocks in the current grid. The difference between filtering the reconstructed vertices of one or more vertices and filtering the reconstructed block of a block is that when filtering the reconstructed vertices of one or more vertices, the degree of change in the positions of the vertices is different, while the degree of change in the positions of the one or more vertices contained in the reconstructed block is the same.

[0279] For example, after the decoder decodes a specified region in the grid, it filters the reconstructed region of that specified region and uses the filtered region to predict other regions in the grid to be decoded. The specified region can be a boundary region in the grid, such as the fingertip of a hand.

[0280] Optionally, the source device 110 can divide the grid to be decoded into multiple regions and decode the multiple regions.

[0281] At the decoding end, the reference grid includes at least one reference information obtained by filtering a portion of the decoded region in the grid to be decoded.

[0282] Optionally, the decoded mesh includes the preceding mesh and a portion of the decoded region in the mesh to be decoded. For example, the decoded mesh includes decoded and filtered vertices in the mesh to be decoded and filtered vertices in the decoded mesh.

[0283] The decoder filters the reconstructed mesh to obtain a filtered mesh, or the filtered mesh can be used as the reconstructed mesh output by the decoder.

[0284] The decoder can filter not only the vertex positions of the reconstructed mesh, but also the attribute information of the vertex positions.

[0285] In some embodiments, the decoder adjusts the attribute information of the first reconstructed mesh according to the vertex positions of the first reconstructed mesh to obtain the adjusted attribute information of the first reconstructed mesh.

[0286] In other embodiments, the decoder adjusts the attribute information of the filtered mesh based on the vertex positions of the filtered mesh to obtain the adjusted attributes of the filtered mesh.

[0287] For example, the bitstream contains attribute information. Figure 8 As shown, the decoder decodes the bitstream to obtain an attribute map. During the decoding process of the attribute map, since filtering changes the vertex positions of the mesh, the attribute map corresponding to those vertex positions also needs to be changed accordingly. Therefore, an attribute transfer operation can be performed to generate a new attribute map based on the vertex positions of the reconstructed mesh, the vertex positions of the filtered mesh, and the attribute map. Attribute transfer refers to transferring an attribute from one part of a geometry to the nearest point of another geometry, thus mapping the attribute map from one mesh to another. For example, attributes include color, RGB, and pixel coordinates.

[0288] The decoder modifies the attribute values ​​of the original attribute map based on the correspondence between the filtered mesh and the reconstructed mesh, and obtains a new attribute map. The filtered mesh corresponds to the new attribute map, for example, the texture mapping relationship corresponds.

[0289] For example, such as Figure 9 As shown, since filtering alters the vertex positions of the mesh, the corresponding texture coordinates must also be changed accordingly. Therefore, a re-parameterization operation can be performed to generate new texture coordinates based on the vertex positions of the reconstructed mesh, the vertex positions of the filtered mesh, and the decoded texture coordinates. Re-parameterization can employ algorithms such as UVAtlas or orthoAtlas.

[0290] The decoder modifies the original texture coordinates based on the correspondence between the filtered mesh and the reconstructed mesh to obtain new texture coordinates. The vertex coordinates of the filtered mesh correspond to the new texture coordinates. For example, the texture mapping relationship corresponds, and the texture map and the filtered mesh are associated based on the texture coordinates.

[0291] For example, such as Figure 10 As shown, since filtering changes the vertex positions of the mesh, the corresponding normal coordinates must also be changed accordingly. Therefore, after obtaining the vertices of the filtered mesh, the normal coordinates are recalculated based on the vertices of the filtered mesh, and then combined with the decoded normal coordinates to generate reference normal coordinates. The reference normal coordinates can be generated by specifying them through an identifier in the bitstream or by calculating them a priori at the decoding end; alternatively, they can be generated by averaging or linearly weighting the normal calculated from the vertex position and the decoded normal coordinates of that vertex.

[0292] The decoder calculates the first normal coordinates based on the vertex positions of the filtered mesh, and uses the first normal coordinates as the reference normal coordinates; the reconstructed normal coordinates are obtained based on the first normal coordinates, and the reconstructed normal coordinates are filtered to obtain the filtered normal coordinates.

[0293] This improves the quality of reconstructed attribute information of the grid.

[0294] This application does not limit the apparatus for performing filtering functions on the reconstructed mesh. The filtering function can be performed by the encoder and decoder. For example, the encoder and decoder contain filters, and the filtering function is performed by the filters; or, the filtering function can be performed by external components of the encoder and decoder.

[0295] The dynamic grid encoding and decoding method provided in this application filters the reconstructed grid at both the encoding and decoding ends to reduce the error of the reconstructed grid. The filtering results in a more accurate reconstructed grid, which can provide better prediction for subsequent grids, improve encoding efficiency, and reduce the bit rate.

[0296] Special 3D Scene: Digital Native Mesh Encoding and Decoding. Digital native meshes are generated directly from data collected from the real world and are commonly used in virtual reality, augmented reality, and game development. Digital native meshes are obtained through methods such as CG generation or physical simulation, and their main characteristic is that the topological structure (connectivity) is consistent across all frames. During loop filtering, thanks to the generation method of digital native meshes, the parameters of mesh loop filtering, whether vertices / blocks are filtered, and the compensation information for the filtering results, etc., can all or part of them be calculated at the decoding end or through prior calculation of the mesh vertex positions.

[0297] It is understood that, in order to achieve the functions in the above embodiments, the encoder and decoder include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and method steps of the various examples described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0298] The above text combines Figures 1 to 10 This document describes in detail the dynamic mesh encoding method and dynamic mesh decoding method provided according to this embodiment. The following will combine... Figure 11 and Figure 12 This describes the encoding and decoding apparatus provided according to this embodiment.

[0299] Figure 11This is a schematic diagram of a possible encoding device provided in this embodiment. These encoding devices can be used to implement the function of encoding dynamic meshes in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments. In this embodiment, the encoding device can be as follows: Figure 3 The encoder 300 shown can also be a module (such as a chip) applied to terminal devices or servers.

[0300] like Figure 11 As shown, the encoding device 1100 includes a communication module 1110, an encoding module 1120, and a storage module 1130.

[0301] Encoding device 1100 is used to implement the above. Figure 4 or Figure 6 The method embodiment shown illustrates the function of the encoder.

[0302] Communication module 1110 is used to acquire the dynamic mesh to be encoded. For example, communication module 1110 is used to perform... Figure 4 Step 410.

[0303] The encoding module 1120 is used to predict the dynamic grid to be encoded based on the reference grid to obtain the prediction result of the dynamic grid to be encoded, and to encode the dynamic grid to be encoded based on the prediction result to obtain the bitstream. For example, the encoding module 1120 is used to perform... Figure 4 Steps 420 and 430.

[0304] Communication module 1110 is also used to send code streams.

[0305] Storage module 1130 is used to store a reference grid, so that the dynamic grid to be encoded can be predicted at the encoding end based on the reference grid.

[0306] Figure 12 This is a schematic diagram of a possible decoding device provided in this embodiment. These decoding devices can be used to implement the function of decoding the bitstream in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments. In this embodiment, the decoding device can be as follows: Figure 3 The decoder 400 shown can also be a module (such as a chip) applied to terminal devices or servers.

[0307] like Figure 12 As shown, the decoding device 1200 includes a communication module 1210, a decoding module 1220, and a storage module 1230.

[0308] Decoding device 1200 is used to achieve the above. Figure 4 or Figure 6 The method embodiment shown illustrates the functionality of the decoder.

[0309] The communication module 1210 is also used to acquire the bitstream. For example, the communication module 1210 is used to perform... Figure 4 Step 440.

[0310] The decoding module 1220 is used to decode the bitstream to obtain a first dynamic grid, and to reconstruct the first dynamic grid based on a reference grid to obtain a first reconstructed grid. For example, the decoding module 1220 is used to perform... Figure 4 Steps 450 and 460. Decoding module 1220 is used to perform... Figure 6 Steps 610 to 640.

[0311] Storage module 1230 is used to store a reference mesh so that the first dynamic mesh can be reconstructed at the decoding end based on the reference mesh.

[0312] It should be understood that the encoding device 1100 and decoding device 1200 in the embodiments of this application can be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Alternatively, they can be implemented using software. Figure 4 or Figure 6 In the encoding and decoding method shown, the encoding device 1100 and the decoding device 1200 and their respective modules can also be software modules.

[0313] For a more detailed description of the communication module, encoding module, decoding module, and storage module mentioned above, please refer to [reference needed]. Figure 4 or Figure 6 The relevant descriptions in the method embodiments shown are directly obtained and will not be repeated here.

[0314] Figure 13 This is a schematic diagram of the structure of an encoder 1300 provided in this embodiment. Figure 13 As shown, the encoder 1300 includes a processor 1310, a bus 1320, a memory 1330, and a communication interface 1340.

[0315] It should be understood that in this embodiment, the processor 1310 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0316] The processor may also be a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, or one or more integrated circuits used to control the execution of the program in this application.

[0317] The communication interface 1340 is used to enable communication between the encoder 1300 and external devices or components. In this embodiment, the communication interface 1340 is used to acquire dynamic meshes.

[0318] Bus 1320 may include a pathway for transmitting information between the aforementioned components (such as processor 1310 and memory 1330). In addition to a data bus, bus 1320 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus 1320 in the figure.

[0319] As an example, encoder 1300 may include multiple processors. A processor may be a multi-CPU processor. Here, "processor" can refer to one or more devices, circuits, and / or computing units for processing data (e.g., computer program instructions). Processor 1310 can predict the dynamic grid to be encoded based on a reference grid to obtain a prediction result of the dynamic grid to be encoded, and encode the dynamic grid to be encoded based on the prediction result to obtain a bitstream.

[0320] It is worth noting that, Figure 13 Taking encoder 1300 as an example, which includes one processor 1310 and one memory 1330, the processor 1310 and the memory 1330 are used to indicate a type of device or equipment. In specific embodiments, the number of each type of device or equipment can be determined according to business needs.

[0321] The memory 1330 can correspond to the storage medium used to store information such as the reference grid in the above method embodiments, such as a disk, like a mechanical hard disk or a solid-state hard disk.

[0322] The encoder 1300 described above can be a general-purpose device or a special-purpose device. For example, the encoder 1300 can be an x86 or ARM-based server, or other special-purpose servers, such as a policy control and charging (PCC) server. This application does not limit the type of encoder 1300.

[0323] It should be understood that the encoder 1300 in this embodiment can correspond to the encoding device 1100 in this embodiment, and can correspond to the execution according to Figure 4 or Figure 6 The corresponding subject in any of the methods, and the above and other operations and / or functions of each module in the encoding device 1100 are respectively for implementing Figure 4 or Figure 6 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0324] Figure 14 This is a schematic diagram of the structure of a decoder 1400 provided in this embodiment. Figure 14 As shown, the decoder 1400 includes a processor 1410, a bus 1420, a memory 1430, and a communication interface 1440.

[0325] It should be understood that in this embodiment, the processor 1410 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0326] The processor may also be a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, or one or more integrated circuits used to control the execution of the program in this application.

[0327] The communication interface 1440 is used to enable communication between the decoder 1400 and external devices or components. In this embodiment, the communication interface 1440 is used to acquire dynamic meshes.

[0328] Bus 1420 may include a pathway for transmitting information between the aforementioned components (such as processor 1410 and memory 1430). In addition to a data bus, bus 1420 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus 1420 in the figure.

[0329] As an example, decoder 1400 may include multiple processors. A processor may be a multi-CPU processor. Here, "processor" can refer to one or more devices, circuits, and / or computing units used to process data (e.g., computer program instructions). Processor 1410 may decode the bitstream to obtain a first dynamic mesh, and reconstruct the first dynamic mesh based on a reference mesh to obtain a first reconstructed mesh.

[0330] It is worth noting that, Figure 14 Taking the decoder 1400 as an example, which includes one processor 1410 and one memory 1430, the processor 1410 and the memory 1430 are used to indicate a type of device or equipment. In specific embodiments, the number of each type of device or equipment can be determined according to business requirements.

[0331] The memory 1430 can correspond to the storage medium used to store information such as the reference grid in the above method embodiments, such as a disk, like a mechanical hard disk or a solid-state hard disk.

[0332] The decoder 1400 described above can be a general-purpose device or a dedicated device. For example, the decoder 1400 can be an x86 or ARM-based server, or other dedicated servers, such as a policy control and charging (PCC) server. This application does not limit the type of decoder 1400.

[0333] It should be understood that the decoder 1400 according to this embodiment can correspond to the decoding device 1200 in this embodiment, and can correspond to the execution of the decoding device 1200 according to this embodiment. Figure 4 or Figure 6 The corresponding entities in any of the methods, and the above and other operations and / or functions of each module in the encoding device 1100 and the decoding device 1200 are respectively implemented for the purpose of... Figure 4 or Figure 6 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0334] The encoder 1300 and decoder 1400 may also include filters for filtering the reconstructed mesh. Alternatively, the encoder 1300 and decoder 1400 may not include filters; instead, filters are located outside the encoder 1300 and decoder 1400, and the encoder 1300 and decoder 1400 send the reconstructed mesh to the filter and receive the filtered mesh from the filter.

[0335] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a network device or terminal device. Of course, the processor and storage medium can also exist as discrete components in the network device or terminal device.

[0336] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0337] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0338] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0339] In the description of this application, unless otherwise stated, " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between the related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. In the description of this application, unless otherwise stated, "multiple" means two or more. "At least one of the following or similar expressions" refers to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and / or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0340] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0341] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0342] It is understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, throughout the specification, various embodiments do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0343] Some optional features in the embodiments of this application can be implemented independently in certain scenarios without relying on other features, such as the current underlying solution, to solve the corresponding technical problems and achieve the corresponding effects. Alternatively, they can be combined with other features as needed in other scenarios. Correspondingly, the apparatus given in the embodiments of this application can also implement these features or functions, which will not be elaborated upon here.

[0344] In this application, unless otherwise specified, the same or similar parts between the various embodiments can be referred to each other. In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be mutually referenced. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. The following embodiments of this application do not constitute a limitation on the scope of protection of this application.

Claims

1. A dynamic grid coding method, characterized in that, include: Obtain the dynamic mesh to be encoded; The dynamic grid to be encoded is predicted based on a reference grid to obtain the prediction result of the dynamic grid to be encoded. The reference grid is obtained by filtering the reconstructed grid of the encoded grid. The dynamic grid to be encoded is encoded based on the prediction results to obtain a bitstream.

2. The method according to claim 1, characterized in that, The encoded grid includes the preceding grid.

3. The method according to claim 1 or 2, characterized in that, The encoded mesh includes a portion of the encoded region in the mesh to be encoded, and the portion of the encoded region includes one or more encoded vertex regions, encoded blocks, or encoded designated regions.

4. The method according to any one of claims 1-3, characterized in that, The reference grid is obtained by filtering the reconstructed grid of the encoded grid, including: The reference mesh is obtained by filtering the vertex positions and / or attribute information of the reconstructed mesh from the encoded mesh.

5. The method according to any one of claims 1-4, characterized in that, The filtering of the reconstructed mesh of the encoded mesh also includes compensating for the filtering result obtained by filtering the reconstructed mesh of the encoded mesh.

6. The method according to any one of claims 1-5, characterized in that, The bitstream includes auxiliary parameters, which include at least one of the following: grid filter identifier, region filter identifier, filter coefficients, compensation information, or reference grid identifier; wherein... The grid filter identifier is used to indicate the filtering of the reconstructed grid; The region filtering identifier is used to indicate that regions in the reconstructed grid are filtered. The filtering coefficients include at least one of the following: the identifier of the filtering algorithm, the filtering weight, or the number of filtering iterations; The compensation information includes compensation values ​​for the reconstructed grid and / or compensation values ​​for the region; The reference grid identifier is used to indicate the reference grid.

7. The method according to claim 6, characterized in that, The auxiliary parameters are determined based on the grid to be encoded and its reconstructed grid, and / or based on the grid to be encoded and the reference grid.

8. A dynamic grid decoding method, characterized in that, include: Acquire a bitstream, the bitstream including a first dynamic grid; The bitstream is processed to obtain the first dynamic grid; The first dynamic grid is reconstructed based on a reference grid to obtain a first reconstructed grid, wherein the reference grid is obtained by filtering the reconstructed grid of the decoded grid.

9. The method according to claim 8, characterized in that, The decoded mesh includes the preceding mesh.

10. The method according to claim 8 or 9, characterized in that, The decoded mesh includes a portion of the decoded region in the first dynamic mesh, and the portion of the decoded region includes one or more decoded vertex regions, decoded blocks, or decoded designated regions.

11. The method according to any one of claims 8-10, characterized in that, The reference grid is obtained by filtering the reconstructed grid of the decoded grid, including: The reference mesh is obtained by filtering the vertex positions and / or attribute information of the reconstructed mesh from the decoded mesh.

12. The method according to any one of claims 8-11, characterized in that, The method further includes: The first reconstructed mesh is filtered to obtain a filtered mesh.

13. The method according to claim 12, characterized in that, The bitstream includes auxiliary parameters, including a grid filter identifier, which is used to indicate grid filtering for reconstruction. The step of filtering the first reconstructed mesh to obtain the filtered mesh includes: When the mesh filtering identifier indicates that the first reconstructed mesh should be filtered, the first reconstructed mesh is filtered to obtain a filtered mesh.

14. The method according to claim 12 or 13, characterized in that, The bitstream includes auxiliary parameters, which include a region filtering identifier, which is used to indicate region filtering in the reconstructed grid. The step of filtering the first reconstructed mesh to obtain the filtered mesh includes: When the region filtering identifier indicates that one or more vertices in the first reconstructed mesh are filtered, one or more vertices in the first reconstructed mesh are filtered to obtain a filtered mesh.

15. The method according to any one of claims 12-14, characterized in that, The bitstream includes auxiliary parameters, and the auxiliary parameters further include compensation information, which is used to determine the compensation value for the first reconstructed grid. The step of filtering the first reconstructed mesh to obtain the filtered mesh includes: The first reconstructed mesh is filtered based on the compensation information to obtain a filtered mesh.

16. The method according to any one of claims 13-15, characterized in that, The auxiliary parameters also include at least one of the following: filter coefficients, filter rules, or reference grid identifiers.

17. The method according to any one of claims 8-16, characterized in that, The method further includes: The attribute information of the first reconstructed mesh is adjusted according to the vertex position of the first reconstructed mesh to obtain the adjusted attribute information of the first reconstructed mesh.

18. The method according to any one of claims 12-16, characterized in that, The method further includes: The attribute information of the filtered mesh is adjusted according to the vertex position of the filtered mesh to obtain the adjusted attributes of the filtered mesh.

19. An encoding device, characterized in that, The apparatus is used to implement the method as described in any one of claims 1-7.

20. A decoding device, characterized in that, The apparatus is used to implement the method as described in any one of claims 8-18.

21. An encoder, characterized in that, The encoder includes at least one processor and a memory, wherein the memory is used to store a computer program such that when the computer program is executed by the at least one processor, it implements the method as described in any one of claims 1-7.

22. The encoder according to claim 21, characterized in that, The encoder also includes a filter for filtering the reconstructed mesh of the encoded mesh.

23. A decoder, characterized in that, The decoder includes at least one processor and a memory, wherein the memory is used to store a computer program such that when the computer program is executed by the at least one processor, it implements the method as described in any one of claims 8-18.

24. The decoder according to claim 23, characterized in that, The decoder also includes a filter for filtering the reconstructed grid of the decoded grid.

25. A codec system, characterized in that, The encoding / decoding system includes an encoder as described in claim 21 or 22, and a decoder as described in claim 23 or 24, wherein the encoder is used to perform the operational steps of the method according to any one of claims 1-7, and the decoder is used to perform the method according to any one of claims 8-18.

26. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-18.

27. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions or programs that, when executed on a computer, implement the method as described in any one of claims 1-18.

28. A bitstream, characterized in that, The bitstream is obtained by the method according to any one of claims 1-7.

Images