Coding method, decoding method, system, and related device

Abstract

The present application provides a coding method, a decoding method, a system, and a related device. In the method, when a three-dimensional mesh is coded, actual motion trend information of a three-dimensional mesh to be coded in a three-dimensional mesh sequence may be acquired, the actual motion trend information indicating an actual motion trend of a motion vertex that performs non-uniform motion in the three-dimensional mesh sequence in a slot corresponding to said three-dimensional mesh; and coded data of said three-dimensional mesh is acquired on the basis of the actual motion trend information, so as to code the three-dimensional mesh. The method can achieve a better coding and decoding effect.

Description

Encoding and decoding methods, systems and related equipment

[0001] This application claims priority to Chinese Patent Application No. 202410799179.5, filed on June 19, 2024, entitled "Encoding and Decoding Method, System and Related Equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of three-dimensional mesh technology, and in particular to an encoding and decoding method, system and related equipment. Background Technology

[0003] Three-dimensional meshes (such as dynamic meshes) can be used to represent volumetric videos, digital humans, computer graphics (CG) content, and more. A 3D mesh is a three-dimensional structure composed of a series of vertices and corresponding edges. For example, a 3D structure may include several planar patches, each composed of a set of vertices and corresponding edges. The data of a 3D mesh includes the coordinates of the vertices and the connections between them. Because 3D meshes have a large number of vertices, their data includes the coordinate information of many vertices, resulting in a large data volume. Therefore, 3D meshes generally need to be compressed and encoded before transmission.

[0004] However, current encoding and decoding schemes need improvement for 3D meshes with vertices exhibiting non-uniform motion. Summary of the Invention

[0005] This application provides an encoding and decoding method, system, and related equipment to improve current encoding and decoding schemes for 3D meshes.

[0006] In a first aspect, this application provides an encoding method, comprising: acquiring actual motion trend information of a first three-dimensional grid to be encoded in a three-dimensional grid sequence, wherein the actual motion trend information indicates the actual motion trend of a motion vertex in the three-dimensional grid sequence that is undergoing non-uniform motion in the time slot corresponding to the first three-dimensional grid; acquiring encoded data of the first three-dimensional grid based on the actual motion trend information; and encoding the encoded data into a bitstream corresponding to the three-dimensional grid sequence.

[0007] Among them, motion trend information can be motion acceleration.

[0008] In this embodiment, when encoding the current frame in a 3D mesh sequence, the encoding end can obtain the encoded data of the current frame based on the actual motion trend information of the current frame. The actual motion trend information indicates the actual motion trend of a non-uniformly moving vertex in the 3D mesh sequence within the corresponding time slot of the current frame. In scenarios where a moving vertex in the 3D mesh sequence undergoes non-uniform motion (e.g., variable speed motion or rotational motion), although the vertex position in the 3D mesh sequence may change, causing deformation of the 3D mesh, the actual motion trend of the moving vertex within the corresponding time slot of each frame in the 3D mesh sequence changes relatively little. By encoding the current frame in the 3D mesh sequence based on this actual motion trend information, better encoding results can be obtained.

[0009] In one possible implementation, obtaining the encoded data of the first three-dimensional mesh based on the actual motion trend information includes: obtaining the predicted motion trend information of the first three-dimensional mesh, wherein the predicted motion trend information indicates the predicted motion trend of the moving vertex in the corresponding time slot of the first three-dimensional mesh; and obtaining the encoded data based on the actual motion trend information and the predicted motion trend information.

[0010] In the embodiments of this application, when a moving vertex in the three-dimensional mesh to be encoded moves at a non-uniform speed, although the uniform speed of the same moving vertex differs greatly between the time slots corresponding to different frames, the actual motion trend information (e.g., offset acceleration) of the same moving vertex in the time slots corresponding to different frames is close. This makes the predicted motion information (including predicted motion trend information) predicted by this application for the moving vertex closer to the actual motion information (including actual motion trend information) of the moving vertex, thereby improving the accuracy of the predicted information (here, the predicted motion trend) of the moving vertex in the current frame.

[0011] In one possible implementation, the encoded data includes the residual between the actual motion trend information and the predicted motion trend information.

[0012] In this embodiment, when the current frame includes moving vertices undergoing non-uniform motion, the encoding end of this application can encode the residual between the actual motion trend information and the predicted motion trend information of the current frame to achieve encoding of the current frame. Since the vertex positions predicted for the current frame based on historical frames are inaccurate in scenarios where the 3D mesh undergoes self-deformation or non-uniform motion (e.g., fabric deformation), the actual motion trend information of the same moving vertex in the 3D mesh sequence remains almost unchanged across different frames. Therefore, the predicted motion trend information of the current frame obtained by this application is closer to the actual motion trend information of the current frame, thereby reducing the residual that needs to be encoded, reducing the amount of compressed data, and improving the data compression rate. Because the predicted residuals of each moving vertex in the 3D mesh are small, the interval formed by the quantization results of the residuals between the predicted and actual motion trend information of each moving vertex in the 3D mesh is smaller, resulting in fewer bits occupied by the encoded data of the residuals, thus shortening the length of the encoded data of the 3D mesh, and consequently shortening the length of the bitstream corresponding to the 3D mesh sequence and reducing the bit rate.

[0013] In one possible implementation, the encoded data includes the actual motion trend information.

[0014] In related technologies, encoding 3D meshes assumes that the mesh moves at a constant speed, meaning the velocity of the vertices remains constant between adjacent frames. Based on this assumption, the vertex positions of the current frame are predicted using the vertex positions of historical frames. Then, the residual (also called position residual) between the predicted and actual vertex positions is encoded to achieve the encoding of the current frame. However, when the 3D mesh is a model of a flexible or elastic body or other easily deformable 3D object, the predicted vertex positions based on the assumption of constant speed are inaccurate when vertices in the mesh undergo non-uniform motion (e.g., vertices rotate, undergo variable speed motion, or the 3D mesh deforms within the 3D mesh sequence). This results in a large deviation between the predicted and actual vertex positions. Furthermore, in scenarios where vertices move at non-uniform speeds within the 3D mesh sequence, many vertex predictions assume constant speed, leading to significant differences in the magnitude of the prediction residuals for a large number of vertices and resulting in a low compression ratio for the 3D mesh.

[0015] In data encoding, the data can be quantized, and then the quantized values ​​are converted into binary sequences for encoding. The higher the frequency of the quantized values, the smaller the range formed by the quantized values ​​(e.g., [-1, 1]). Furthermore, the higher the frequency of the quantized values, the fewer bits are needed to convert them into binary data. Therefore, the encoding scheme for 3D meshes in related technologies results in a larger range formed by the quantized residual values ​​of a large number of vertices in the 3D mesh (e.g., [-10, 10]), leading to a larger number of bits occupied by the encoded data in the current frame, and consequently, a longer bitstream and higher bitrate for the 3D mesh sequence.

[0016] In this embodiment, when encoding the current frame in the 3D mesh sequence, the encoding end does not need to calculate the prediction residual of the vertices (e.g., the residual between the predicted position and the actual position of the vertex) or encode the residual. Instead, it directly encodes the actual motion trend information of the current frame to obtain the encoded data of the current frame, and encodes this encoded data into the bitstream corresponding to the 3D mesh sequence. Therefore, in scenarios where moving vertices move at non-uniform speeds in the 3D mesh sequence, a large number of moving vertices in the current frame may have similar actual motion trends (e.g., acceleration) information. When encoding the actual motion trend information of a large number of moving vertices in the current frame, the range formed by the quantized values ​​of the actual motion trend information can be smaller, and the number of bits occupied by the encoded data of the current frame can be reduced, thereby improving the compression ratio and shortening the length of the bitstream corresponding to the 3D mesh sequence, thus reducing the bit rate. Furthermore, compared to the scheme of encoding the residual between the actual motion trend information and the predicted motion trend information of the current frame, this embodiment only needs to calculate and encode the actual motion trend information of the current frame, resulting in lower encoding complexity and reduced encoding latency.

[0017] In one possible implementation, the bitstream includes a first syntax structure corresponding to the first three-dimensional grid, the first syntax structure including an identifier of the first three-dimensional grid.

[0018] In one possible implementation, the first syntax structure further includes information corresponding to the motion vertex, the motion vertex information including: a first residual of the motion vertex on a first component; a second residual of the motion vertex on a second component; and a third residual of the motion vertex on a third component.

[0019] The actual motion information of the moving vertex may include the actual component information of the moving vertex in three components, namely the actual component information in the first component, the actual component information in the second component, and the actual component information in the third component;

[0020] The predicted motion information of the moving vertex may include the predicted component information of the moving vertex in three components, namely the predicted component information in the first component, the predicted component information in the second component, and the predicted component information in the third component.

[0021] The three components mentioned above are three components in a three-dimensional coordinate system, which is either a Cartesian coordinate system or a cylindrical coordinate system.

[0022] When the three-dimensional coordinate system is a Cartesian coordinate system, these three components can be the x component, y component, and z component, respectively.

[0023] When the three-dimensional coordinate system is a cylindrical coordinate system, these three components can be the radius component, the height component, and the angle (e.g., deflection angle) component, respectively.

[0024] The first residual is the residual between the actual component information (e.g., actual motion trend information (e.g., actual motion acceleration)) of the motion vertex in the first component and the predicted component information (e.g., predicted motion trend information (e.g., predicted motion acceleration)) in the first component;

[0025] The second residual is the residual between the actual component information of the motion vertex in the second component and the predicted component information in the second component. For example, the actual component information is the actual motion speed and the predicted component information is the predicted motion speed; or the actual component information is the actual motion acceleration and the predicted component information is the predicted motion acceleration.

[0026] The third residual is the residual between the actual component information of the motion vertex in the third component and the predicted component information in the third component. For example, the actual component information is the actual motion speed and the predicted component information is the predicted motion speed; or the actual component information is the actual motion acceleration and the predicted component information is the predicted motion acceleration.

[0027] At least one of the aforementioned first residual, second residual, and third residual is the residual between the predicted motion acceleration and the actual motion acceleration on the corresponding component.

[0028] In one possible implementation, the first syntax structure further includes: the vertex identifier of the moving vertex.

[0029] When encoding a portion of the vertices in the first 3D mesh, the first syntax structure may include the vertex identifiers of the portion of vertices to facilitate the decoding end in locating the three types of residuals of which moving vertex.

[0030] In one possible implementation, obtaining the predicted motion trend information of the first three-dimensional mesh includes: obtaining N sets of reconstructed data corresponding to N encoded second three-dimensional meshes in the three-dimensional mesh sequence, where N is an integer greater than or equal to a set value; and determining the predicted motion trend information based on the N sets of reconstructed data.

[0031] There is a one-to-one correspondence between the N second-dimensional grids and the N sets of reconstructed data.

[0032] In this embodiment, because the decoding end cannot obtain the original data of each frame in the 3D mesh sequence, but can only obtain the reconstructed data of each frame through decoding, in order to ensure that the predicted motion trend information calculated by the encoding end and the decoding end for the same frame is the same, when the encoding end obtains the predicted motion trend information of the current frame, it can obtain the reconstructed data of at least three previously encoded historical frames in the 3D mesh sequence, and determine the predicted motion trend information of the current frame based on the reconstructed data (rather than the original data of the at least three historical frames). This ensures that the predicted motion trend information determined by the encoding end for the current frame is consistent with the predicted motion trend information obtained by the decoding end for the current frame, thereby ensuring the accuracy of the reconstructed data of the 3D mesh sequence. Furthermore, the predicted motion information calculated by the encoding end and the decoding end for the current frame remains consistent, facilitating accurate decoding of the current frame by the decoding end, improving decoding accuracy, ensuring that the difference in vertex position between the reconstructed 3D mesh and the original 3D mesh is small, and improving the quality of the reconstructed 3D mesh.

[0033] In one possible implementation, obtaining the actual motion trend information of the first three-dimensional mesh to be encoded in the three-dimensional mesh sequence includes: determining the actual motion information of the first three-dimensional mesh based on the original data of the first three-dimensional mesh and the original data of a plurality of encoded third three-dimensional meshes in the three-dimensional mesh sequence.

[0034] In this application, when the encoding end obtains the actual motion trend information of the current frame, in order to ensure the accuracy of the actual motion trend information, it can determine the actual motion trend information of the current frame based on the original data of the current frame and the original data of at least two encoded historical frames, thus ensuring the accuracy of the actual motion information calculated by the encoding end.

[0035] In one possible implementation, obtaining N sets of reconstructed data corresponding to the N encoded second 3D meshes in the 3D mesh sequence includes: obtaining N sets of encoded data corresponding to the N second 3D meshes; obtaining N sets of predicted motion trend information for the N second 3D meshes, wherein the N sets of predicted motion trend information are N predicted motion trends of the motion vertex in the N time slots corresponding to the N second 3D meshes; and decoding the N sets of encoded data based on the N predicted motion trend information to obtain the N sets of reconstructed data. Here, there is a one-to-one correspondence between the N second 3D meshes and the N sets of predicted motion information.

[0036] In this application, the process of the encoding end obtaining the reconstruction data of at least three historical frames that have been encoded before the current frame is the same as the process of the decoding end obtaining the reconstruction data of one frame that has been encoded in the three-dimensional grid sequence, which can ensure that the reconstruction data of the N second three-dimensional grids obtained by both the encoding and decoding ends are consistent.

[0037] In one possible implementation, decoding the N sets of encoded data based on the N predicted motion trend information to obtain N sets of reconstructed data for the N second 3D meshes includes: decoding the N sets of encoded data to obtain N sets of residuals between the N sets of actual motion trend information for the N second 3D meshes and the N sets of predicted motion trend information, wherein the N sets of actual motion trend information include N sets of actual motion trend information of the motion vertex in the N time slots corresponding to the N sets of second 3D meshes; and determining the N based on the N predicted motion trend information and the N residuals. The N sets of actual motion trend information of the second three-dimensional mesh (essentially the reconstructed N sets of actual motion trend information, also expressed as N sets of reconstructed motion information) are the N sets of actual motion trend information of the motion vertex in the N time slots corresponding to the N second three-dimensional meshes; based on the N sets of actual motion trend information and the reconstructed data of the S fourth three-dimensional meshes corresponding to each of the N second three-dimensional meshes, the N sets of reconstructed data are determined, and the S fourth three-dimensional meshes are the decoded three-dimensional meshes in the three-dimensional mesh sequence, where S is an integer greater than or equal to 2.

[0038] Among them, there is a one-to-one correspondence between the N second three-dimensional meshes and the N sets of reconstructed motion information.

[0039] Furthermore, there is a one-to-one correspondence between the N second three-dimensional grids and the N sets of actual motion information.

[0040] In this embodiment, the encoder can use the N residuals of the N historical frames obtained by decoding the encoded data of the N historical frames, and the N sets of predicted motion trend information of the N historical frames to reconstruct the N sets of actual motion trend information of the N historical frames (referred to as reconstructed motion information). Then, the encoder can determine the reconstructed data of the historical frame based on the reconstructed data of at least two frames encoded before one of the N historical frames and the actual motion trend information of the reconstructed historical frame of the aforementioned N historical frames. The reconstructed data may include the reconstructed position of the motion vertex of the current frame, and optionally, may also include the connection relationship between motion vertices (e.g., the topology of the current frame), thereby enabling the reconstruction of each historical frame in the N historical frames. When the three-dimensional mesh sequence can be deformed, the offset acceleration of the same motion vertex (referring to the same vertex identifier or the same vertex sorting position) in different frames is similar. Therefore, in the scenario where the three-dimensional mesh sequence is deformed, the correct decoding of the three-dimensional mesh can be achieved at a reduced bit rate. Due to the smaller residual, the amount of decoded data is reduced, and the transmission efficiency and storage efficiency are improved.

[0041] In one possible implementation, at least one of the first residual, the second residual, and the third residual is a residual of motion acceleration.

[0042] In one possible implementation, the first component is the X component, the second component is the Y component, and the third component is the Z component.

[0043] In one possible implementation, the first component is an angle component, the second component is a height component, and the third component is a radius component.

[0044] In one possible implementation, the first three-dimensional mesh, the N second three-dimensional meshes, and the plurality of third three-dimensional meshes have the same topology.

[0045] The topological structure of each of the three-dimensional meshes represents the connection relationship of vertices in the three-dimensional mesh. The connection relationship of vertices of each three-dimensional mesh in the three-dimensional mesh sequence remains unchanged. Each three-dimensional mesh has the same number of vertices and the same vertex identifier. This makes it easy to track the changes in the motion trend information of the same vertex in the three-dimensional mesh sequence, so as to determine the predicted motion trend information and the actual motion trend information of the moving vertex.

[0046] In one possible implementation, the N second three-dimensional meshes are displayed in the three-dimensional mesh sequence before the first three-dimensional meshes are displayed in the three-dimensional mesh sequence.

[0047] For example, after decoding N second-dimensional meshes, they can be rendered and displayed, and their display order can precede the rendering of the first-dimensional mesh image.

[0048] In one possible implementation, the plurality of third 3D meshes are displayed in the 3D mesh sequence before the first 3D meshes are displayed in the 3D mesh sequence.

[0049] For example, multiple third-dimensional meshes can be decoded and rendered, and their display order can precede the rendering of the first-dimensional mesh image.

[0050] The aforementioned N second-dimensional meshes can be the same as multiple third-dimensional meshes, in which case the number of third-dimensional meshes is also N, and N is greater than or equal to 3. Alternatively, the N second-dimensional meshes and the multiple third-dimensional meshes can be completely different 3D meshes, or not completely identical 3D meshes. The number of multiple third-dimensional meshes is at least two.

[0051] Secondly, this application provides a decoding method, comprising: acquiring a bitstream corresponding to a three-dimensional mesh sequence; acquiring encoded data of a first three-dimensional mesh to be decoded in the three-dimensional mesh sequence from the bitstream; acquiring actual motion trend information of the first three-dimensional mesh based on the encoded data, wherein the actual motion trend information indicates the actual motion trend of a non-uniformly moving vertex in the three-dimensional mesh sequence in the time slot corresponding to the first three-dimensional mesh; and obtaining reconstructed data of the first three-dimensional mesh based on the actual motion trend information.

[0052] In this embodiment, when decoding the current frame in a 3D mesh sequence, the decoding end can obtain the encoded data of the current frame from the bitstream and, based on the encoded data, obtain the actual motion trend information of the current frame. This actual motion trend information indicates the actual motion trend of a non-uniformly moving vertex in the 3D mesh sequence within the corresponding time slot of the current frame. In scenarios where a moving vertex in the 3D mesh sequence undergoes non-uniform motion (e.g., variable speed motion or rotational motion), although the vertex position in the 3D mesh sequence may change, causing deformation of the 3D mesh, the actual motion trend of the moving vertex within the corresponding time slot of each frame in the 3D mesh sequence changes relatively little. By decoding the current frame in the 3D mesh sequence based on this actual motion trend information, better encoding results can be obtained.

[0053] In one possible implementation, obtaining the actual motion trend information of the first three-dimensional mesh based on the encoded data includes: obtaining the predicted motion trend information of the first three-dimensional mesh, wherein the predicted motion trend information indicates the predicted motion trend of the moving vertex in the corresponding time slot of the first three-dimensional mesh; and obtaining the actual motion trend information based on the encoded data and the predicted motion trend information.

[0054] In this embodiment, when decoding the current frame, the decoding end can obtain the predicted motion trend information of the current frame. This predicted motion information is the same as the predicted motion trend information of the current frame obtained by the encoding end when encoding the current frame. Thus, the decoding end can use this predicted motion information to decode the encoded data of the current frame, ensuring the accuracy of the reconstructed data of the current frame, such as the closeness of the reconstructed position of the moving vertex in the current frame to its original position. In addition, the decoding end can also obtain the encoded data of the current frame from the bitstream. This encoded data is data encoded based on the actual motion information and the predicted motion information. When the moving vertex in the three-dimensional mesh to be encoded moves at a non-uniform speed, although the difference in the uniform speed of the same moving vertex in the time slots corresponding to different frames is large, the actual motion trend information (e.g., offset acceleration) of the same moving vertex in the time slots corresponding to different frames is close. This makes the difference in motion trend information between the predicted motion information (including the predicted motion trend information) of the moving vertex predicted by this application and the actual motion information (including the actual motion trend information) of the moving vertex extremely small. Therefore, the shorter bitstream length of the 3D grid sequence results in a lower bitrate, requiring less transmission bandwidth and improving transmission and storage efficiency (more frames of encoded data can be stored in the same storage space). Furthermore, when decoding the current frame in the 3D grid sequence, the encoding end can utilize the aforementioned predicted motion information to decode the encoded data of the current frame, obtaining the reconstructed data. This reduces the decoding workload of the 3D grid sequence bitstream and minimizes screen stuttering.

[0055] In one possible implementation, the encoded data includes the residual between the actual motion trend information and the predicted motion trend information.

[0056] In this embodiment, the encoded data includes the residual between the actual motion information and the predicted motion information of the current frame. Thus, when a moving vertex in the 3D mesh to be decoded moves at a non-uniform speed, although the uniform speed of the same moving vertex differs greatly between the time slots corresponding to different frames, the actual motion trend information (e.g., offset acceleration) of the same moving vertex in the time slots corresponding to different frames is similar. This makes the residual of the current frame decoded by the decoding end extremely small, thereby reducing the amount of decoded data, reducing the occupation of transmission bandwidth, and improving transmission efficiency.

[0057] In one possible implementation, the encoded data includes the actual motion trend information.

[0058] In this embodiment, when decoding the current frame in a 3D mesh sequence, the decoding end can directly decode the encoded data of the current frame in the bitstream to obtain the actual motion trend information of the current frame. In scenarios where moving vertices move at non-uniform speeds in the 3D mesh sequence, a large number of moving vertices in the current frame may have similar actual motion trend (e.g., acceleration) information. Since the encoded data of the current frame also includes the actual motion trend information of the moving vertices, a large number of moving vertices in the encoded data have similar or identical actual motion trend information. This results in fewer bits occupied by the encoded data, a shorter bitstream length, and a lower bitrate. Decoding this encoded data can improve the decompression rate. Furthermore, compared to schemes where the encoded data includes the residual between the actual motion trend information and the predicted motion trend information of the current frame, this embodiment only needs to decode the encoded data of the current frame, without calculating the predicted motion trend information or decoding the residual, resulting in lower decoding complexity and reduced decoding latency.

[0059] In one possible implementation, the encoded data has a first syntactic structure, which includes an identifier of a first three-dimensional mesh.

[0060] In one possible implementation, the first syntax structure further includes information corresponding to the motion vertex, the motion vertex information including: a first residual of the motion vertex on a first component; a second residual of the motion vertex on a second component; and a third residual of the motion vertex on a third component.

[0061] In one possible implementation, the first syntax structure further includes: the vertex identifier of the moving vertex.

[0062] In one possible implementation, obtaining the predicted motion trend information of the first three-dimensional grid includes: based on the bitstream, obtaining N sets of reconstructed data corresponding to N decoded second three-dimensional grids in the three-dimensional grid sequence, where N is an integer greater than or equal to a set value; and determining the predicted motion trend information based on the N sets of reconstructed data.

[0063] There is a one-to-one correspondence between the N second-dimensional grids and the N sets of reconstructed data.

[0064] In this embodiment of the application, in order to ensure that the predicted motion trend information calculated by the encoding end and the decoding end for the same frame is the same, when the decoding end obtains the predicted motion trend information of the current frame, it can obtain the reconstruction data of at least three historical frames that have been encoded in the 3D mesh sequence, and determine the predicted motion trend information of the current frame based on the reconstruction data (rather than the original data of the at least three historical frames). This ensures that the predicted motion trend information determined by the decoding end for the current frame is consistent with the predicted motion trend information obtained by the encoding end for the current frame, thereby ensuring the accurate decoding of the 3D mesh sequence by the decoding end, ensuring that the difference in vertex position between the reconstructed 3D mesh and the original 3D mesh is small, and improving the quality of the reconstructed 3D mesh.

[0065] In one possible implementation, obtaining the reconstruction data of the first three-dimensional mesh based on the actual motion trend information includes: acquiring reconstruction data of multiple decoded third three-dimensional meshes corresponding to the three-dimensional mesh sequence based on the bitstream; and obtaining the reconstruction data of the first three-dimensional mesh based on the actual motion trend information and the reconstruction data of the multiple third three-dimensional meshes.

[0066] In this embodiment, the decoding end can obtain the reconstruction data of at least two frames that have been decoded before the current frame based on the bitstream; then, using the actual motion trend information of the current frame and the reconstruction data, the reconstruction data of the current frame is obtained, thus ensuring the accurate reconstruction of the three-dimensional mesh.

[0067] In one possible implementation, at least one of the first residual, the second residual, and the third residual is a residual of motion acceleration.

[0068] In one possible implementation, the first component is the X component, the second component is the Y component, and the third component is the Z component.

[0069] In one possible implementation, the first component is an angle component, the second component is a height component, and the third component is a radius component.

[0070] In one possible implementation, the first three-dimensional mesh, the N second three-dimensional meshes, and the plurality of third three-dimensional meshes have the same topology.

[0071] The effects of the second aspect and its possible partial implementations are similar to those of the first aspect and its possible implementations, and will not be elaborated here.

[0072] Thirdly, this application provides an encoding device, comprising: an acquisition module, configured to acquire actual motion trend information of a first three-dimensional mesh to be encoded in a three-dimensional mesh sequence, wherein the actual motion trend information indicates the actual motion trend of a non-uniformly moving vertex in the three-dimensional mesh sequence in the time slot corresponding to the first three-dimensional mesh; and an encoding module, configured to acquire encoded data of the first three-dimensional mesh based on the actual motion trend information; and encode the encoded data into a bitstream corresponding to the three-dimensional mesh sequence.

[0073] Fourthly, this application provides a decoding apparatus, comprising: an acquisition module for acquiring a bitstream corresponding to a three-dimensional mesh sequence; a decoding module for acquiring encoded data of a first three-dimensional mesh to be decoded in the three-dimensional mesh sequence from the bitstream; acquiring actual motion trend information of the first three-dimensional mesh based on the encoded data, the actual motion trend information indicating the actual motion trend of a non-uniformly moving vertex in the three-dimensional mesh sequence in the time slot corresponding to the first three-dimensional mesh; and obtaining reconstructed data of the first three-dimensional mesh based on the actual motion trend information.

[0074] Fifthly, this application provides an encoding apparatus, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described in any one of the first aspects above.

[0075] In a sixth aspect, this application provides a decoding apparatus, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described in any one of the second aspects above.

[0076] In a seventh aspect, this application provides a computer-readable storage medium storing program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of the first to second aspects above.

[0077] Eighthly, this application provides a computer program product comprising computer program code that, when executed on a processor, causes the processor to perform the method described in any one of the first to second aspects.

[0078] Ninthly, this application provides a chip including a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the method as described in any one of the first to second aspects above.

[0079] In a tenth aspect, this application provides a bitstream, the bitstream corresponding to a three-dimensional grid sequence, the bitstream including encoded data of a first three-dimensional grid, the encoded data being encoded data obtained based on actual motion trend information of the first three-dimensional grid, the actual motion trend information indicating the actual motion trend of a motion vertex in the three-dimensional grid sequence that is undergoing non-uniform motion in the corresponding time slot of the first three-dimensional grid.

[0080] In one possible implementation, the encoded data includes the residual between the actual motion trend information and the predicted motion trend information of the first three-dimensional mesh, the predicted motion trend information indicating the predicted motion trend of the motion vertex in the corresponding time slot of the first three-dimensional mesh.

[0081] In one possible implementation, the encoded data includes the actual motion trend information.

[0082] In one possible implementation, the bitstream includes a first syntax structure corresponding to the first three-dimensional grid, the first syntax structure including an identifier of the first three-dimensional grid.

[0083] In one possible implementation, the first syntax structure further includes information about the motion vertex, which includes: a first residual of the motion vertex on a first component; a second residual of the motion vertex on a second component; and a third residual of the motion vertex on a third component.

[0084] In one possible implementation, the information of the moving vertex further includes: the vertex identifier of the moving vertex.

[0085] In one possible implementation, at least one of the first residual, the second residual, and the third residual is a residual of motion acceleration.

[0086] In one possible implementation, the first component is the X component, the second component is the Y component, and the third component is the Z component.

[0087] In one possible implementation, the first component is an angle component, the second component is a height component, and the third component is a radius component.

[0088] In one aspect, this application provides a computer-readable storage medium storing a bitstream as described in any one of the tenth aspects above.

[0089] In a twelfth aspect, this application provides a method for transmitting an encoded bitstream of video data, the method comprising: acquiring a bitstream from a storage medium, the bitstream being the bitstream described in the tenth aspect above or in any possible implementation thereof and stored in the storage medium; and transmitting the bitstream.

[0090] In a thirteenth aspect, this application provides a system for transmitting an encoded bitstream of video data, the system comprising: an acquisition unit for acquiring a bitstream from a storage medium, the bitstream being the bitstream described in the tenth aspect above or in any possible implementation thereof and stored in the storage medium; and a transmission unit for transmitting the bitstream.

[0091] In a fourteenth aspect, this application provides a method for storing an encoded bitstream of video data, the method comprising: receiving the bitstream described in the tenth aspect or any possible implementation thereof; and storing the bitstream in a storage medium.

[0092] In a fifteenth aspect, this application provides a system for storing an encoded bitstream of video data, comprising: a receiving unit for receiving the bitstream described in the tenth aspect or any possible implementation thereof; and a storage unit for storing the bitstream. Attached Figure Description

[0093] Figure 1A is an architecture diagram of the encoding and decoding system provided in an embodiment of this application;

[0094] Figure 1B is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0095] Figure 1C is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0096] Figure 2A is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0097] Figure 2B is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0098] Figure 2C is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0099] Figure 2D is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0100] Figure 3A is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0101] Figure 3B is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0102] Figure 3C is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0103] Figure 3D is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0104] Figure 3E is a flowchart of one of the encoding methods provided in the embodiments of this application;

[0105] Figure 3F is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0106] Figure 3G is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0107] Figure 4A is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0108] Figure 4B is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0109] Figure 4C is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0110] Figure 5A is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0111] Figure 5B is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0112] Figure 5C is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0113] Figure 5D is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0114] Figure 5E is a flowchart of one of the decoding methods provided in the embodiments of this application;

[0115] Figure 6 is a block diagram of the encoding device provided in an embodiment of this application;

[0116] Figure 7 is a block diagram of the decoding device provided in an embodiment of this application;

[0117] Figure 8 is a schematic diagram of the structure of the device provided in the embodiment of this application. Detailed Implementation

[0118] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0119] The terms "first," "second," etc., used in the specification, embodiments, claims, and drawings of this application are for distinguishing purposes only and should not be construed as indicating or implying relative importance or order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0120] It should be understood that in the embodiments of this application, "at least one (item)" refers to one or more, and "more than one" refers to two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0121] The following is a brief introduction to the terminology used in the embodiments of this application:

[0122] 1) 3D Mesh (e.g., Dynamic Mesh): Also known as a 3D model, it is a three-dimensional structure composed of a series of vertices and corresponding edges. For example, a 3D structure may include planar patches (also called polygons, typically quadrilaterals or triangles). Each planar patch can be composed of a set of vertices and corresponding edges formed by connecting these vertices. 3D meshes can be used to represent volumetric videos, digital humans, computer graphics (CG) content, etc. The data of a 3D mesh typically includes: vertex coordinates, connectivity, texture coordinates, and texture maps. Vertex coordinates indicate the position of vertices in three-dimensional (3D) space, and connectivity indicates the vertices that make up the patches (e.g., triangular patches) in the 3D mesh.

[0123] 2) Topology of a 3D mesh: Represents the connection relationships between multiple vertices within all faces of a 3D mesh.

[0124] 3) Current frame: At the encoding end, the "current frame" represents the 3D mesh to be encoded; at the decoding end, the "current frame" represents the 3D mesh to be decoded.

[0125] 4) Historical frames: At the encoding end, "historical frames" refer to the 3D meshes that have been encoded before the current frame (the 3D mesh to be encoded); at the decoding end, "historical frames" refer to the 3D meshes that have been decoded before the current frame (the 3D mesh to be decoded).

[0126] 5) Frame sequence: Also known as a 3D mesh sequence, it consists of multiple 3D meshes with the same topological structure arranged in a specific order. This order can be the display order or the encoding order of the 3D meshes; there is no restriction here. Multiple 3D meshes in a 3D mesh sequence share the same vertices; "same vertices" refers to vertices with the same vertex identifier or vertex number.

[0127] For example, a 3D mesh sequence consists of 3D meshes 1 to n, where n is an integer greater than 1, and each 3D mesh in 3D meshes 1 to n has vertices 1 to m, where m is an integer greater than or equal to 3. Furthermore, the vertex connections of vertices 1 to m in 3D mesh 1 are the same as the vertex connections of vertices 1 to m in each of the 3D meshes 2 to n. However, the same vertex (e.g., vertex 1) may have different vertex positions in different 3D meshes within the 3D mesh sequence. This application does not impose any restriction on whether the vertex positions corresponding to the "same vertex" (also expressed as "the same vertex") in different 3D meshes are the same.

[0128] 6) Moving Vertices: Because the vertices in a 3D mesh sequence are identical, and the position of the same vertex can change within the sequence, the motion information of any vertex in the 3D mesh sequence can be tracked. A moving vertex refers to a vertex in the 3D mesh sequence that has motion information (or a vertex that is moving). Moving vertices can move at a uniform or non-uniform speed within the 3D mesh sequence; there are no restrictions here.

[0129] 7) The time slot corresponding to the 3D mesh: It can be the start time of the presentation of a frame of 3D mesh, the end time of the presentation of a frame of 3D mesh, the middle time of the presentation period of a frame of 3D mesh, or the entire presentation period (time period) of a frame of 3D mesh. This application does not limit this. Taking a frame rate of 25 frames / second as an example, the presentation duration of each frame of 3D mesh is 40 milliseconds. Then the time slot corresponding to the first frame of 3D mesh can be the 0th millisecond, the 39th millisecond, the 19th millisecond (middle time), or the first 40-millisecond time period. The time slot corresponding to the second frame of 3D mesh can be the 40th millisecond, the 79th millisecond, the 59th millisecond, or the second 40-millisecond time period, and so on.

[0130] 8) Motion trend: The motion trend of a moving vertex refers to the trend of the moving vertex in motion, including the acceleration of the moving vertex, and optionally, the velocity of the moving vertex; the motion trend can be used to calculate the position or coordinates of the moving vertex; in a given coordinate system, the motion trend includes all projection components of the acceleration / velocity of the moving vertex on the coordinate system; taking the Cartesian coordinate system as an example, the motion trend includes the projection components of the acceleration / velocity of the moving vertex on the x-axis, the projection components on the y-axis, and the projection components on the z-axis; the motion trend of the moving vertex in the time slot corresponding to the three-dimensional grid can be the average acceleration / average velocity of the moving vertex in the time slot corresponding to the three-dimensional grid, or it can be the instantaneous acceleration / instantaneous velocity of the moving vertex in the time slot corresponding to the three-dimensional grid. This application does not limit this aspect.

[0131] 9) Movement trend information: Information used to describe or indicate movement trends.

[0132] 10) Actual motion trend: The actual motion trend of a moving vertex refers to the motion trend calculated based on the position or coordinate information of the vertex in the current frame and several frames before the current frame.

[0133] 11) Actual motion trend information: Information used to describe or indicate actual motion trends;

[0134] 12) Predicting motion trend: The predicted motion trend of a moving vertex refers to the motion trend calculated based on the position or coordinate information of the vertex in the previous frames (excluding the current frame).

[0135] 13) Predictive motion trend information: Information used to describe or indicate predicted motion trends.

[0136] In the following embodiments, the current frame of the encoding end corresponds to the first three-dimensional grid of the encoding side (including the encoding method and encoding device) in the invention. In the following embodiments, the N historical frames of the encoding end correspond to the N second three-dimensional grids of the encoding side (including the encoding method and encoding device) in the invention. In the following embodiments, the M historical frames of the encoding end correspond to multiple third three-dimensional grids of the encoding side (including the encoding method and encoding device) in the invention. In the following embodiments, the S historical frames of the encoding end correspond to multiple fourth three-dimensional grids of the encoding side (including the encoding method and encoding device) in the invention.

[0137] In the following embodiments, the current frame of the decoding end corresponds to the first three-dimensional grid of the decoding side (including decoding method and decoding device) in the invention. In the following embodiments, the N historical frames of the decoding end correspond to the N second three-dimensional grids of the decoding side (including decoding method and decoding device) in the invention. In the following embodiments, the M historical frames of the decoding end correspond to the multiple third three-dimensional grids of the decoding side (including decoding method and decoding device) in the invention.

[0138] Figure 1A is a schematic diagram of a three-dimensional mesh encoding and decoding system architecture provided in an embodiment of this application. As shown in Figure 1A, the system may include an encoding end 100 and a decoding end 200.

[0139] As shown by the dashed arrow in Figure 1A, the encoding end 100 and the decoding end 200 can be connected in communication (e.g., directly connected through a network or indirectly connected through a server, communication equipment, etc., which is not restricted here), or the encoding end 100 and the decoding end 200 can be disconnected.

[0140] The encoding end 100 can encode the acquired three-dimensional mesh data and output the bit stream.

[0141] For example, the encoding end 100 can output the bitstream to a local storage device or storage medium, or to a remote device or server (such as a video website), or output the bitstream to the decoding end 200 through the above communication connection. There are no restrictions here.

[0142] The decoding end 200 can decode the input bitstream to obtain reconstructed 3D mesh data. For example, the decoding end can read the bitstream from a local storage device or storage medium, or receive the bitstream from a remote device or server (such as a video website), or receive the input bitstream through the aforementioned communication connection, and then decode the bitstream.

[0143] Although Figure 1A shows the encoder 100 and decoder 200 as independent devices, the device embodiment may also include both the encoder 100 and decoder 200, or both the corresponding encoding function of the encoder 100 and the corresponding decoding function of the decoder 200 provided in this application. In these embodiments, the encoder 100 or its corresponding encoding function, and the decoder 200 or its corresponding function, can be implemented by hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0144] The encoding end 100 and the decoding end 200 may include any type of device, including any type of handheld device, fixed device, or wearable device, such as a laptop or tablet computer, smartphone, tablet or tablet PC, desktop computer, smart TV, in-vehicle display, smart bracelet, smartwatch, augmented reality (AR) device, virtual reality (VR) device, etc., and may or may not use an operating system. In some cases, the encoding end 100 and the decoding end 200 may be wireless communication devices equipped with wireless communication components.

[0145] The system shown in Figure 1A can be applied to various business scenarios involving the encoding and decoding of three-dimensional meshes. For example, video conferencing, video telephony, online education, remote tutoring, live streaming, cloud gaming, etc., and this application does not limit the application to these scenarios.

[0146] For example, when this system is applied to a live streaming scenario, the encoding end 100 can be a server for a live streaming application (App), and the decoding end 200 can be a terminal device such as a mobile phone running the live streaming application. The 3D mesh can be a 3D mesh of the clothing of a digital human in the live stream. The encoding end 100 encodes the 3D mesh of the clothing of the digital human, the decoding end 200 decodes the 3D mesh, and displays the image of the clothing of the digital human rendered from the decoded 3D mesh.

[0147] For example, this system can be applied to cloud gaming scenarios. The encoding end 100 can be the cloud providing the game application, and the decoding end 200 can be a terminal device such as a mobile phone running the game application. The 3D mesh can be a 3D mesh of the virtual clothing of the virtual character in the game. The encoding end 100 encodes the 3D mesh of the virtual clothing of the virtual character, and the decoding end 200 decodes the 3D mesh and displays the image of the virtual clothing of the virtual character rendered from the decoded 3D mesh.

[0148] The encoding and decoding process of the three-dimensional mesh by the encoder 100 and decoder 200 is described below with reference to Figure 1A.

[0149] The three-dimensional mesh to be encoded / decoded in the three-dimensional mesh sequence includes moving vertices. Taking a vertex in the three-dimensional mesh sequence that is undergoing non-uniform motion as an example, the encoding / decoding process of this application is explained. The encoding / decoding process of the encoding end 100 and the decoding end 200 of this application for moving vertices undergoing uniform motion in the three-dimensional mesh sequence is the same as the encoding / decoding process for moving vertices undergoing non-uniform motion in the three-dimensional mesh sequence.

[0150] The non-uniform motion can be a motion in which at least one of the speed and direction of motion changes. For example, the non-uniform motion can be a rotational motion (either uniform or non-uniform), or it can be a variable-speed motion (e.g., interval running).

[0151] For example, the 3D object corresponding to this 3D mesh is cloth.

[0152] In a cloud gaming scenario, the three-dimensional object can be virtual clothing worn by a virtual character. When the virtual character undergoes a self-rotation (e.g., dancing ballet), it can cause the virtual clothing to rotate as well. This allows the corresponding three-dimensional mesh sequence to include non-uniform vertices undergoing self-rotation, causing the virtual clothing to deform.

[0153] For example, in a cloud gaming scenario, a virtual avatar wearing virtual clothes runs at high speed, causing the virtual clothes to deform. This means that the corresponding 3D mesh sequence of the virtual clothes can include moving vertices undergoing variable speed motion, resulting in deformation of the virtual clothes.

[0154] Of course, the three-dimensional object corresponding to the three-dimensional mesh in this application can also be a flexible object other than cloth, such as clay, ball, jelly, etc. When there are vertices in the three-dimensional mesh of these objects that are undergoing non-uniform motion, the encoding and decoding method and system of this application are also applicable.

[0155] In other embodiments, the moving vertex can also be a vertex in a three-dimensional mesh sequence that moves at a constant speed. In other words, the encoding and decoding method and system of this application can be applied not only to scenarios where vertices in a three-dimensional mesh sequence move at a non-uniform speed, but also to scenarios where vertices in a three-dimensional mesh sequence move at a constant speed.

[0156] In some scenarios, such as when the 3D mesh is a deformable 3D object like a flexible body (or other scenarios, without limitation), the vertices in this 3D mesh do not move at a constant velocity, but rather at a non-constant velocity. Related technologies for encoding 3D meshes assume that all vertices in the mesh move at a constant velocity. Specifically, this technology predicts the predicted positions of vertices in the current frame (i.e., the 3D mesh to be encoded) based on the assumption of constant velocity motion, calculates the residual between the predicted and actual positions, and encodes this residual into the bitstream to encode the current frame. However, because the vertices do not move at a constant velocity, the predicted positions are inaccurate, and the encoding and decoding methods for 3D meshes in these technologies need improvement.

[0157] The following example uses a 3D mesh as a 3D model of the virtual clothing worn by a virtual character in a cloud gaming scene to illustrate the processing procedure of the system in this application.

[0158] As shown in Figure 1A, the process may include the following steps:

[0159] S102: The encoding end 100 can obtain the encoded data of the current frame in the three-dimensional grid sequence and encode the encoded data into the bit stream corresponding to the three-dimensional grid sequence.

[0160] The encoding end 100 can obtain the actual motion trend information of the current frame in the three-dimensional grid sequence, and based on the actual motion trend information, obtain the encoded data of the current frame, and then encode the encoded data into the bit stream corresponding to the three-dimensional grid sequence.

[0161] For example, the encoded data may include residuals or actual motion trend information.

[0162] This encoded data is the data before it is converted into a bitstream; therefore, this encoded data can also be called the data to be encoded.

[0163] For example, the encoding end 100 can perform entropy encoding or compression encoding on the encoded data to convert the encoded data into a bit stream and write it into the bit stream.

[0164] The actual motion trend information indicates the actual motion trend of vertices (also referred to as moving vertices) that are undergoing non-uniform motion in the three-dimensional mesh sequence within the corresponding time slot of the current frame.

[0165] In some embodiments, each frame in the 3D mesh sequence may have original data, which includes the original position information of each vertex in the 3D mesh, and optionally, the connection relationships between each vertex in the 3D mesh (also expressed as the topology of the 3D mesh).

[0166] To encode the original position information of the vertices in the current frame, the actual motion trend information of the current frame can be calculated to obtain the encoded data of the current frame.

[0167] The three-dimensional mesh sequence of this application can be generated by the encoding terminal 100, or by other devices or software connected to the encoding terminal 100, without any limitation.

[0168] The 3D mesh sequence (also referred to as a frame sequence) can be generated using a physics simulator or a physics simulation algorithm. Each frame in the frame sequence has the same topological structure and vertex identifiers. The topological structure of the 3D mesh includes the connectivity relationships between all vertices in the mesh. Therefore, the position of the same vertex in different frames may change, but the connectivity relationships between that vertex and other vertices remain unchanged.

[0169] The topology of different frames in a 3D mesh sequence (also known as a frame sequence) generated by a physics simulator or physics simulation algorithm remains unchanged. However, the vertex position of the same vertex in the 3D mesh can change in different frames (e.g., non-uniform motion).

[0170] For example, the encoder 100 can obtain the raw data of the current frame and the historical frame from the physics simulator. The raw data may include the raw position information of the vertices of the current frame and the historical frame respectively.

[0171] It should be understood that the three-dimensional mesh sequence of this application is not limited to being generated by a physical simulator or a physical simulation algorithm, but can also be generated by other means, as long as the topology of different frames in the three-dimensional mesh sequence is kept consistent and has the same vertices (referring to the same vertex identifier or vertex number).

[0172] For example, the encoder 100 can collect the original position (e.g., 3D coordinates) information of each vertex in the current frame (specifically, the 3D mesh to be encoded), as well as the original position information of each vertex in historical frames (specifically, the 3D meshes that have been encoded before the 3D mesh to be encoded).

[0173] Alternatively, when only a portion of the vertices in the 3D mesh needs to be encoded or decoded, the original position information of the vertices in the current frame and historical frames collected by the encoding end 100 can also be the original position information of a portion of the vertices in the 3D mesh, rather than the original position information of each vertex.

[0174] For example, in cloud gaming scenarios, the cloud can use physical simulation algorithms to generate a 3D mesh of the virtual clothes worn by virtual characters in the game, and collect the original position information of each vertex in the 3D mesh of the clothes.

[0175] In a sequence of three-dimensional meshes generated by a physical simulation algorithm or a physical simulator, the topology of different frames is the same. However, the original position information of the vertices under the topology changes between different frames. Therefore, the method of this application can calculate the actual motion trend information of the vertices in the three-dimensional mesh based on the changed original positions of the vertices while keeping the topology of the three-dimensional mesh unchanged, so as to realize the encoding and decoding of the three-dimensional mesh.

[0176] S103: Encoding end 100 outputs the above bitstream.

[0177] For example, the encoding end 100 can output the bitstream to a local storage device or storage medium, or to a server, etc. There are no restrictions here.

[0178] S201: Decoder input bitstream 200.

[0179] For example, the decoding end 200 can read the bitstream from a local storage device or storage medium, or it can receive the bitstream from a remote device or server (such as a video website) to obtain the input bitstream.

[0180] S202: Decoding end 200 decodes the input bitstream to obtain the reconstructed three-dimensional mesh data.

[0181] For example, the decoder 200 can decode the bitstream to obtain the encoded data of the current frame to be decoded in the three-dimensional mesh sequence from the bitstream, and obtain the actual motion trend information of the current frame based on the encoded data, so as to use the actual motion trend information to reconstruct the vertex position of the current frame, thereby obtaining the reconstruction data of the current frame (such as the reconstructed position information of the vertex and the above-mentioned topology) to reconstruct the current frame.

[0182] The current frame is a 3D mesh to be decoded.

[0183] Optionally, the system may also include a rendering end (not shown), which can be communicatively connected to the decoding end 200. The rendering end can be software, hardware, chips, etc., capable of rendering 3D meshes into images; no limitations are imposed here.

[0184] The rendering end can render the 3D mesh data decoded by the decoding end 200 through S202 to obtain an image or image sequence (e.g., video) for display on the screen.

[0185] Taking cloud gaming as an example, the rendering end can render the 3D mesh of the virtual clothing reconstructed by the decoding end to obtain the rendered image of the virtual clothing for on-screen display.

[0186] The following describes the encoding process of the three-dimensional mesh (e.g., the process executed by the encoding end 100 in FIG1A) and the decoding process of the three-dimensional mesh (e.g., the process executed by the decoding end 200 in FIG1A) of this application with reference to different embodiments.

[0187] Figure 1B is a flowchart of a method for encoding the current frame (a 3D mesh to be encoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, but is not limited to the architecture shown in Figure 1A. The method may include the following steps:

[0188] S330: The encoding end acquires the actual motion trend information of the current frame to be encoded in the three-dimensional mesh sequence.

[0189] The actual motion trend information indicates the actual motion trend of the motion vertex that is moving at a non-uniform speed in the three-dimensional mesh sequence in the corresponding time slot of the current frame.

[0190] The actual motion trend can be motion acceleration. Since the current frame is a three-dimensional structure, the actual motion trend can be motion acceleration in at least one of the three components of the three-dimensional coordinate system.

[0191] In some embodiments, when the actual motion trend is the motion acceleration on one component (e.g., the X component), the encoder can also obtain the actual motion rate information of the current frame, which is the actual motion rate of the motion vertex on the other two components (e.g., the Y component and the X component) in the corresponding time slot of the current frame.

[0192] In some embodiments, when the actual motion trend is the motion acceleration of each of the two components (e.g., the X component and the Y component), the encoder can also obtain the actual motion rate information of the current frame, which is the actual motion rate of the motion vertex in the other component (e.g., the Z component) in the corresponding time slot of the current frame.

[0193] The data of the 3D mesh sequence may include the original data of each 3D mesh to be encoded. The original data may include the topology of the 3D mesh and the original vertex positions of all or some vertices in the 3D mesh. Optionally, it may also include the frame rate of the 3D mesh sequence. Optionally, it may also include the instantaneous frame rate of each 3D mesh in the 3D mesh sequence.

[0194] The three-dimensional mesh sequence has a frame rate, the reciprocal of which is the total motion time of the aforementioned moving vertices in the three-dimensional mesh sequence.

[0195] Each frame in a 3D mesh sequence can have its own instantaneous frame rate. For example, the average instantaneous frame rate of all frames in the 3D mesh sequence is the frame rate of the 3D mesh sequence. The instantaneous frame rates of different frames in the 3D mesh sequence can be the same or different, without restriction. Based on this, the time slot corresponding to a frame in the 3D mesh sequence can be the reciprocal of the instantaneous frame rate of that frame, and the sum of the time slots corresponding to each frame in the 3D mesh sequence can be the overall motion time mentioned above.

[0196] In the method flow shown in Figure 2B below, the data of the three-dimensional mesh sequence can be divided into the raw data of the current frame and the raw data of the historical frames.

[0197] In order to encode the current frame (e.g., the original vertex position of the moving vertex), embodiments of this application can encode the current frame based on the actual motion trend information of the moving vertex of the current frame to obtain the encoded data of the current frame.

[0198] S331: The encoding end obtains the encoded data of the current frame based on the actual motion trend information of the current frame.

[0199] The encoding end can directly encode the actual motion trend information, or it can encode the current frame based on the actual motion trend information and other information of the current frame to obtain the encoded data of the vertex positions of each motion vertex of the current frame.

[0200] The encoded data is obtained by encoding the vertex positions of the moving vertices in the current frame; however, the encoded data is not a sequence of 0 and 1 bits.

[0201] S332: The encoding end encodes the encoded data into the bitstream corresponding to the three-dimensional grid sequence.

[0202] The encoding end can perform entropy encoding or compression encoding on the encoded data to convert the encoded data into a 0 and 1 bit sequence, and write the 0 and 1 bit sequence into the bit stream.

[0203] In this embodiment, when encoding the current frame in a 3D mesh sequence, the encoding end can obtain the encoded data of the current frame based on the actual motion trend information of the current frame. The actual motion trend information indicates the actual motion trend of a non-uniformly moving vertex in the 3D mesh sequence within the corresponding time slot of the current frame. In scenarios where a moving vertex in the 3D mesh sequence undergoes non-uniform motion (e.g., variable speed motion or rotational motion), although the vertex position in the 3D mesh sequence may change, causing deformation of the 3D mesh, the actual motion trend of the moving vertex within the corresponding time slot of each frame in the 3D mesh sequence changes relatively little. By encoding the current frame in the 3D mesh sequence based on this actual motion trend information, better encoding results can be obtained.

[0204] Figure 1C is a flowchart of a method for encoding the current frame (a 3D mesh to be encoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, but is not limited to the architecture shown in Figure 1A. This method can be combined with the process shown in Figure 1B, but is not limited to the process shown in Figure 1B. As shown in Figure 1C, the method may include the following steps:

[0205] S3301: The encoder obtains the actual motion trend information of the current frame to be encoded in the three-dimensional mesh sequence.

[0206] The execution principle of S3301 is the same as that of S330 shown in Figure 1B, and will not be repeated here.

[0207] S3312: The encoding end encodes the actual motion trend information to obtain the encoded data of the current frame.

[0208] For example, the encoded data includes the actual movement trend information.

[0209] For example, the encoding end can use at least one encoding method such as quantization or compression to obtain the encoded data of the current frame. This application does not restrict the encoding method for the actual motion trend information.

[0210] S332: The encoding end encodes the encoded data into the bitstream corresponding to the three-dimensional grid sequence.

[0211] In related technologies, encoding 3D meshes assumes that the mesh moves at a constant speed, meaning the velocity of the vertices remains constant between adjacent frames. Based on this assumption, the vertex positions of the current frame are predicted using the vertex positions of historical frames. Then, the residual (also called position residual) between the predicted and actual vertex positions is encoded to achieve the encoding of the current frame. However, when the 3D mesh is a model of a flexible or elastic body or other easily deformable 3D object, the predicted vertex positions based on the assumption of constant speed are inaccurate when vertices in the mesh undergo non-uniform motion (e.g., vertices rotate, undergo variable speed motion, or the 3D mesh deforms within the 3D mesh sequence). This results in a large deviation between the predicted and actual vertex positions. Furthermore, in scenarios where vertices move at non-uniform speeds within the 3D mesh sequence, many vertex predictions assume constant speed, leading to significant differences in the magnitude of the prediction residuals for a large number of vertices and resulting in a low compression ratio for the 3D mesh.

[0212] In data encoding, the data can be quantized, and then the quantized values ​​are converted into binary sequences for encoding. The higher the frequency of the quantized values, the smaller the range formed by the quantized values ​​(e.g., [-1, 1]). Furthermore, the higher the frequency of the quantized values, the fewer bits are needed to convert them into binary data. Therefore, the encoding scheme for 3D meshes in related technologies results in a larger range formed by the quantized residual values ​​of a large number of vertices in the 3D mesh (e.g., [-10, 10]), leading to a larger number of bits occupied by the encoded data in the current frame, and consequently, a longer bitstream and higher bitrate for the 3D mesh sequence.

[0213] In this embodiment, when encoding the current frame in the 3D mesh sequence, the encoding end does not need to calculate the prediction residual of the vertices (e.g., the residual between the predicted position and the actual position of the vertex) or encode the residual. Instead, it directly encodes the actual motion trend information of the current frame to obtain the encoded data of the current frame, and encodes this encoded data into the bitstream corresponding to the 3D mesh sequence. Therefore, in scenarios where moving vertices move at non-uniform speeds in the 3D mesh sequence, a large number of moving vertices in the current frame may have similar actual motion trends (e.g., acceleration) information. When encoding the actual motion trend information of a large number of moving vertices in the current frame, the range formed by the quantized values ​​of the actual motion trend information can be smaller, and the encoded data of the current frame occupies fewer bits in the bitstream, thereby improving the compression ratio and shortening the length of the bitstream corresponding to the 3D mesh sequence, thus reducing the bit rate. Furthermore, compared to the scheme of encoding the residual between the actual motion trend information and the predicted motion trend information of the current frame, this embodiment only needs to calculate and encode the actual motion trend information of the current frame, resulting in lower encoding complexity and reduced encoding latency.

[0214] For example, consider a static curtain being pushed open. The 3D mesh of this curtain might contain many sets of vertices, each with essentially the same actual motion trend (e.g., acceleration). To help the reader understand, the 3D mesh can be compared to an image to be encoded, and the actual motion trend information of the vertices in the 3D mesh to the pixel values ​​of the image. If a large group of pixels in an image has similar pixel values, the compression rate of that image will be relatively high. Similarly, if many vertices in the 3D mesh to be encoded have similar or identical actual motion trend information (e.g., acceleration), then encoding the 3D mesh directly by using the actual motion trend information of the moving vertices can improve the compression rate of the 3D mesh.

[0215] Figure 2A is a flowchart of a method for encoding the current frame (a 3D mesh to be encoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, but is not limited to the architecture shown in Figure 1A. This method can be combined with Figure 1B or Figure 1C (but is not limited to combining with Figure 1B or Figure 1C). The method may include the following steps:

[0216] S3100a: The encoder obtains the predicted motion trend information of the current frame to be encoded in the three-dimensional mesh sequence.

[0217] The predicted motion trend information indicates the predicted motion trend (e.g., predicted motion acceleration) of the motion vertex in the corresponding time slot of the current frame;

[0218] S3100b: The encoding end obtains the actual motion trend information of the current frame.

[0219] For the specific implementation principle and extended implementation method, please refer to S330 shown in Figure 1B, which will not be repeated here.

[0220] This application does not restrict the execution order between S3100a and S3100b; they can be executed serially or in parallel.

[0221] S3102: The encoding end obtains the encoded data of the current frame based on the predicted motion trend information and the actual motion trend information of the current frame.

[0222] In this case, the vertex position of the motion vertex of the current frame can be encoded based on the predicted motion trend information and the actual motion trend information of the current frame in any way to obtain the encoded data of the current frame. There are no restrictions here.

[0223] S332: The encoding end encodes the encoded data into the bitstream corresponding to the three-dimensional grid sequence.

[0224] In one possible implementation, the encoded data includes a residual between the actual motion trend information and the predicted motion trend information. The encoder can determine the residual between the actual motion trend information and the predicted motion trend information; then, the residual is encoded into the bitstream corresponding to the three-dimensional grid sequence.

[0225] For example, the encoder can quantize and entropy encode the residual to convert it into a bit sequence and write the bit sequence into the bit stream.

[0226] There are no restrictions on the encoding method used to encode the residuals.

[0227] In related technologies, when encoding a 3D mesh, the assumption is that the 3D mesh moves at a constant speed to predict the vertex position of the current frame. This causes the predicted vertex position to deviate too much from the actual vertex position, resulting in an excessively large residual in the vertex encoding of the 3D mesh, a large amount of compressed data, and a low compression ratio.

[0228] In encoding the prediction residuals of each vertex of a 3D mesh, the prediction residuals of each vertex need to be quantized, and the quantized values ​​are converted into binary sequences for encoding. The higher the frequency of the quantized values, the smaller the interval formed by the quantization results of the prediction residuals of each vertex (e.g., [-1, 1]). Furthermore, the higher the frequency of the quantized values, the fewer bits are needed to convert them into binary data. Therefore, the larger the residual corresponding to a vertex of the 3D mesh, the larger the interval formed by the quantized values ​​of the residuals of each vertex (e.g., [-10, 10]), resulting in a larger number of bits occupied by the encoded result of the residual (a bit sequence), thus making the code stream of the 3D mesh sequence longer and the code rate higher.

[0229] However, in this embodiment, when the current frame includes moving vertices undergoing non-uniform motion, the encoding end of this application can encode the residual between the actual motion trend information and the predicted motion trend information of the current frame to achieve encoding of the current frame. Since the vertex positions predicted for the current frame based on historical frames are inaccurate in scenarios where the 3D mesh undergoes self-deformation or non-uniform motion (e.g., fabric deformation), the actual motion trend information of the same moving vertex in the 3D mesh sequence remains almost unchanged between different frames. Therefore, the predicted motion trend information of the current frame obtained by this application is closer to the actual motion trend information of the current frame, thereby reducing the residual that needs to be encoded, reducing the amount of compressed data, and improving the data compression rate. Because the predicted residuals of each moving vertex in the 3D mesh are small, the interval formed by the quantization result of the residual between the predicted motion trend information and the actual motion trend information of each moving vertex in the 3D mesh is small, resulting in fewer bits occupied by the encoded result of the residual (which is a bit sequence), thereby shortening the length of the bitstream corresponding to the 3D mesh sequence and reducing the bit rate.

[0230] In other implementations, the encoder may also determine the difference between the predicted motion trend information and the actual motion trend information of the current frame in a manner other than residuals, and encode the current frame into a three-dimensional mesh sequence based on the difference. No limitation is imposed here.

[0231] When a moving vertex in a 3D mesh undergoes non-uniform motion, the difference in the uniform speed of the same moving vertex in the time slots corresponding to different frames is significant. In related technologies, when encoding a 3D mesh, regardless of whether the vertex undergoes uniform or variable motion, the assumption that the vertex undergoes uniform motion is used. Based on the actual position of the vertex in the previous frame, the predicted position of the vertex in the current frame is predicted, which leads to inaccurate prediction of the vertex's position in the current frame.

[0232] In the method flow corresponding to Figure 2A, when a moving vertex in the 3D mesh to be encoded undergoes non-uniform motion, although the uniform velocity of the same moving vertex differs significantly between time slots corresponding to different frames, the actual motion trend information (e.g., offset acceleration) of the same moving vertex in different time slots is similar. This makes the predicted motion information (including predicted motion trend information) predicted by this application for the moving vertex closer to the actual motion information (including actual motion trend information) of the moving vertex, thereby improving the accuracy of the predicted motion trend information (here, the predicted motion trend) for the moving vertex in the current frame, while the actual motion trend information of the current frame is also accurate. Therefore, when the encoding end encodes the current frame in the 3D mesh sequence, it obtains the encoded data of the current frame based on the accurate actual motion trend information and accurate predicted motion trend information of the current frame, thus facilitating the encoding and decoding of the 3D mesh.

[0233] Figure 2B is a flowchart of a method for encoding the current frame (a 3D mesh to be encoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, but is not limited to the architecture shown in Figure 1A. Furthermore, the method flowchart shown in Figure 2B can be combined with Figures 2A, 1B, and 1C, and any possible implementation thereof. As shown in Figure 2B, the method flowchart may include the following steps:

[0234] S300a: The encoding end acquires N sets of reconstructed data from N historical frames in the 3D mesh sequence.

[0235] Among them, N historical frames correspond one-to-one with N sets of reconstructed data, with one historical frame corresponding to one set of reconstructed data.

[0236] The historical frame is a 3D mesh that has been encoded in the 3D mesh sequence.

[0237] For example, the historical frame is a 3D mesh that was encoded before the current frame.

[0238] In some embodiments, the number of historical frames in S300a may be N, where N is an integer greater than or equal to a preset threshold (e.g., 3).

[0239] Since the historical frame has already been encoded, it can be encoded using the encoding method of this application, or using other known or future encoding methods, without any restrictions.

[0240] The reconstructed data of a historical frame can be called a set of reconstructed data. Here, we can obtain N sets of reconstructed data corresponding to N historical frames respectively.

[0241] The encoding end of this application can obtain the reconstructed data of the historical frame obtained by decoding the encoded data of the encoded historical frame. This decoding process can be performed by the encoding end of this application or by other decoding devices, and there is no limitation here.

[0242] Each historical frame in the aforementioned N historical frames corresponds to a set of reconstructed data, which may include the reconstructed positions of the moving vertices in that historical frame (i.e., the reconstructed vertex positions). Optionally, it may also include the topology corresponding to that historical frame. Since the topology of each frame in the 3D mesh sequence is the same, it is not necessary to reconstruct the topology. Optionally, the set of reconstructed data for the historical frames may also include the frame rate of the 3D mesh sequence, and optionally, it may also include the instantaneous frame rate of each 3D mesh in the 3D mesh sequence.

[0243] In some embodiments, the frame rate of the 3D grid sequence, or the instantaneous frame rate of each 3D grid in the 3D grid sequence, may not be encoded into the bitstream. The encoding and decoding ends may agree on the frame rate and instantaneous frame rate in advance.

[0244] S301a: The encoder determines the predicted motion information of the current frame based on the reconstructed data of N historical frames.

[0245] For example, the encoder can determine the predicted motion information of the motion vertices in the current frame based on the reconstructed positions of the motion vertices in each of the N historical frames.

[0246] The predicted motion information is the predicted motion information of a non-uniformly moving vertex in the three-dimensional mesh sequence in the corresponding time slot of the current frame. The predicted motion information may include the predicted motion trend information mentioned above, and optionally include the predicted motion rate information.

[0247] The predicted motion rate information is the predicted motion rate of the motion vertex in the corresponding time slot of the current frame.

[0248] In this step, the encoder can use historical frames that have been encoded before the current frame to predict the motion information of the current frame (represented as predicted motion information).

[0249] S301b: The encoder determines the actual motion information of the current frame based on the raw data of M historical frames and the raw data of the current frame.

[0250] The actual motion information refers to the actual motion information of the motion vertex in the three-dimensional mesh sequence that is undergoing non-uniform motion in the corresponding time slot of the current frame. This actual motion information may include the aforementioned actual motion trend information and optionally include actual motion rate information.

[0251] The actual motion rate information is the actual motion rate of the moving vertex in the corresponding time slot of the current frame.

[0252] Where M is an integer greater than or equal to 2.

[0253] The data of the three-dimensional mesh sequence in the method flow shown in Figure 2A may include the original data of each three-dimensional mesh to be encoded, which may include the topology of the three-dimensional mesh and the original positions of all or some of the vertices in the three-dimensional mesh.

[0254] In this method, the data of the aforementioned three-dimensional mesh sequence may include the original data of the current frame and the original data of historical frames.

[0255] For example, in this embodiment, the original data of the M historical frames and the original data of the current frame are the data obtained through S101 in FIG1A. The original data of the historical frames may include the original position of the aforementioned moving vertex in the historical frame. The original data of the current frame may include the original position of the aforementioned moving vertex in the current frame.

[0256] For example, the raw data in S301b can be obtained from a physical simulator.

[0257] S301a and S301b can be executed serially or in parallel, and this application does not impose any restrictions on this.

[0258] In this case, the historical frames in S301b can be the same as the historical frames in S301a, so M = N, and both M and N are greater than or equal to 3. Alternatively, the historical frames in S301a and S301b can not be completely the same (i.e., there are some historical frames that are the same between the two steps), or the historical frames in S301a and S301b can be completely different.

[0259] S302: The encoder encodes the current frame into the bitstream corresponding to the three-dimensional grid sequence based on the predicted motion information and the actual motion information of the current frame.

[0260] The implementation principle of S302 is the same as that of S3102 in the method flow shown in Figure 2A, and will not be repeated here.

[0261] In the method flow corresponding to Figure 2B, since the decoding end cannot obtain the original data of each frame in the 3D mesh sequence, but can only obtain the reconstructed data of each frame through decoding, in order to ensure that the predicted motion information calculated by the encoding and decoding ends for the same frame is the same, when the encoding end obtains the predicted motion information of the current frame, it can obtain the reconstructed data of at least three previously encoded historical frames in the 3D mesh sequence, and determine the predicted motion information of the current frame based on this reconstructed data (rather than the original data of the at least three historical frames). This ensures that the predicted motion information determined by the encoding end for the current frame is consistent with the predicted motion information obtained by the decoding end for the current frame, thereby ensuring the accuracy of the reconstructed data of the 3D mesh sequence. When the encoding end obtains the actual motion information of the current frame, in order to ensure the accuracy of the actual motion information, it can determine the actual motion information of the current frame based on the original data of the current frame and the original data of at least two previously encoded historical frames. Finally, based on the actual motion information and the predicted motion information of the current frame, the current frame is encoded into the bitstream. In this way, on the one hand, the residual of the encoding is reduced, and on the other hand, the accuracy of the actual motion information calculated by the encoding end is ensured, and the predicted motion information calculated by the encoding end and the decoding end for the current frame is kept consistent, so that the decoding end can accurately decode the current frame, improve the decoding accuracy, and ensure that the difference between the vertex positions of the motion vertices in the reconstructed 3D mesh and the original 3D mesh is small, thereby improving the quality of the reconstructed 3D mesh.

[0262] Figures 3A-3E are flowcharts of various methods for encoding three-dimensional meshes provided in the embodiments of this application. These methods can be implemented based on the architecture shown in Figure 1A, and the processes shown in Figures 1B, 1C, 2A to 2D, but are not limited to combining the above architecture and processes.

[0263] The method flow shown in Figure 3A can be a specific example of the method flow shown in Figure 2B to encode the current frame. The method mainly includes the following steps:

[0264] S3001a: The encoder determines the actual motion information of the motion vertex in the i-1 frame based on the reconstructed position of the motion vertex in each frame of the i-3, i-2, and i-1 frames, as well as the instantaneous frame rate of each frame in the i-2 and i-1 frames, and uses it as the predicted motion information of the motion vertex in the i-1 frame.

[0265] Optionally, the instantaneous frame rate of the (i-3)th frame can also be incorporated.

[0266] Among them, the (i-3)th frame, the (i-2)th frame, and the (i-1)th frame are examples of N historical frames in the method flow shown in Figure 2B.

[0267] The reconstructed data of the N historical frames acquired in S300a of the method flow shown in Figure 2B may include the vertex positions of the motion vertices in each of the (i-3), (i-2), and (i-1) frames. Optionally, it may also include the instantaneous frame rate of each of the (i-2) and (i-1) frames. For an explanation and description of the instantaneous frame rate, please refer to the relevant explanation and description of the embodiment in Figure 2A, which will not be repeated here. The instantaneous frame rate in S3001a may be the instantaneous frame rate of each frame in the three-dimensional mesh sequence agreed upon with the encoding end. For example, the three-dimensional mesh data acquired by the encoding end through S101 may include the instantaneous frame rate of each frame.

[0268] The frame in this application is a three-dimensional mesh, which may include the position information of each vertex. This position information is three-dimensional coordinates. For example, the coordinates of a vertex c may be c(x, y, z) in a three-dimensional Cartesian coordinate system, or c(h, r, θ) in a cylindrical coordinate system, where h represents the height, r represents the radius, and θ represents the angle (e.g., deflection angle).

[0269] In this application, different frames in the frame sequence have the same topological structure of a three-dimensional mesh (e.g., a three-dimensional model), such as sharing the same three-dimensional coordinate system. However, since the vertex positions of the three-dimensional mesh change between different frames, the vertex of the same vertex may be in a different position in different frames. In other words, when the three-dimensional mesh is deformed, the position of the vertex in the three-dimensional mesh may change in different frames.

[0270] The current frame is frame i, where i is an integer greater than or equal to 4. The three frames that have been encoded before frame i (specifically frames i-3, i-2, and i-1) can be used as the aforementioned N historical frames. Since the motion information (e.g., acceleration) of a moving vertex is relatively close between adjacent frames in a scenario where the moving vertex is moving at a non-uniform speed, in order to improve the accuracy of the predicted motion information of the moving vertex in the current frame, the actual motion information of the moving vertex in the previous frame (frame i-1) (which belongs to the actual motion information reconstructed from frame i-1) can be used as the predicted motion information of the moving vertex in the current frame. This improves the difference between the predicted motion information and the actual motion information in the current frame, thereby reducing the residual between the actual motion information and the predicted motion information in the current frame.

[0271] In this embodiment, when determining the actual motion information of the moving vertex in the (i-1)th frame (which belongs to the actual motion information reconstructed from the (i-1)th frame), the actual motion information of the moving vertex in the (i-1)th frame (i.e., the frame preceding the current frame) can be calculated based on the reconstructed position of the moving vertex in each of the (i-3), (i-2), and (i-1)th frames, and the actual motion information of the vertex in the (i-1)th frame can be used as the motion information predicted for the moving vertex in the current frame.

[0272] In other embodiments, when determining the actual motion information of the moving vertex in the (i-1)th frame (which belongs to the actual motion information reconstructed from the (i-1)th frame), the actual motion information of the moving vertex in the (i-1)th frame (which belongs to the actual motion information reconstructed from the (i-1)th frame) can also be calculated based on the reconstructed positions of the moving vertices in any two frames before the (i-1)th frame (not limited to the (i-3)th and (i-2)th frames mentioned above) and each frame in the (i-1)th frame, and used as the predicted motion information of the moving vertex in the current frame.

[0273] The first three frames in the frame sequence (e.g., frame 1, frame 2, and frame 3) can be encoded into a 3D mesh using currently known single-frame compression methods, without any restrictions here.

[0274] In other embodiments, when determining the predicted motion information of the motion vertices of the current frame, it is not limited to using the actual motion information of the previous frame (e.g., the aforementioned (i-1)th frame) (which belongs to the actual motion information reconstructed from the (i-1)th frame) as the predicted motion information of the current frame. Instead, the actual motion information of any frame that has been encoded before the current frame (e.g., any frame such as the (i-2), (i-3), or (i-4)th frame, etc.) (which also belongs to the actual motion information reconstructed from the previous frame) can be used as the predicted motion information of the current frame. Therefore, when determining the predicted motion information of the motion vertices of the current frame, the N historical frames referenced are not limited to the first three encoded frames in the frame sequence that are located before the i-th frame and whose display order is adjacent to the current frame. In other embodiments, the N historical frames shown in Figure 2B can also be the (i-4)th frame, the (i-2)th frame, the (i-1)th frame, or the (i-5)th frame, the (i-4)th frame, the (i-3)th frame (by calculating the actual motion information reconstructed from the (i-3)th frame as the predicted motion information for the (i)th frame), or the (i-5)th frame, the (i-3)th frame, the (i-1)th frame, etc. In other words, the N historical frames can be any three frames encoded before the (i)th frame, without any limitation.

[0275] The predicted motion information includes the predicted motion trend information of the moving vertex in the time slot corresponding to the i-th frame. In this embodiment, the predicted motion trend information can be specifically the reconstruction result of the actual motion trend information of the moving vertex in the time slot corresponding to the (i-1)-th frame.

[0276] Therefore, when determining the reconstruction result of the actual motion trend information, it is necessary to refer not only to the reconstructed position of the motion vertex of each of the N historical frames, but also to the time slot corresponding to each of the N historical frames. This time slot is the reciprocal of the instantaneous frame rate of the corresponding frame. For example, the time slot corresponding to the (i-1)th frame is the time interval between the (i-2)th frame and the (i-1)th frame, and this time interval is the reciprocal of the instantaneous frame rate of the (i-1)th frame.

[0277] S3001b: The encoder determines the actual motion information of the motion vertex in the i-th frame based on the original position of the above motion vertex in each frame of the (i-2)th frame, the (i-1)th frame, and the i-th frame, as well as the instantaneous frame rate of each frame (e.g., the instantaneous frame rate of the i-1th frame and the i-th frame, and optionally the instantaneous frame rate of the i-2th frame).

[0278] S3001a and S3001b can be executed serially or in parallel, and this application does not impose any restrictions on this.

[0279] The original positions of the motion vertices in each of the (i-2), (i-1), and i-th frames in S3001b can be obtained by acquiring the original data of the M historical frames shown in Figure 2B and the original data of the current frame, where M = 2, and the M historical frames are the (i-2)-th and (i-1)-th frames, respectively. The method for obtaining the instantaneous frame rate of these M historical frames is the same as the method for obtaining the instantaneous frame rate of each frame in S3001a, and will not be repeated here.

[0280] In this embodiment, in order to improve the accuracy of calculating the actual motion information of the current frame (the i-th frame), the original position of the motion vertex in the current frame and the original position of the motion vertex in each of the M historical frames, as well as the instantaneous frame rate of each frame, can be used to calculate the actual motion information of the current frame. In this example, the M historical frames are the two adjacent frames that have been encoded before the current frame (e.g., the i-th frame) (specifically the i-2 frame and the i-1 frame).

[0281] In other embodiments, when calculating the actual motion information of a frame in the frame sequence (e.g., the (i-1)th frame or the ith frame, which is not limited here), the M historical frames referenced are not limited to the two frames encoded before that frame, but can be a larger number of frames (e.g., 3 or 4 frames), i.e., M≥2, where M is an integer. Furthermore, the M historical frames are not limited to frames that are adjacent to the frame in the display order, but can also be non-adjacent frames.

[0282] When calculating the actual motion information of a frame, if the referenced M historical frames are three or more frames that have been encoded before this frame, the implementation principle of the referenced M historical frames being two frames can be combined, and algorithms such as weighted averaging or averaging can be used (no restrictions are placed here) to calculate the multiple actual motion information to obtain the actual motion information of the frame.

[0283] For example, when calculating the actual motion information of the i-th frame, the M historical frames referenced include the (i-3)-th frame, the (i-2)-th frame, and the (i-1)-th frame. Therefore, when determining the actual motion information of the motion vertex in the i-th frame based on the original positions of the motion vertices in the (i-3)-th frame, the (i-2)-th frame, and the (i-1)-th frame, the actual motion information of the motion vertex in the (i-1)-th frame can be calculated based on the original positions of the motion vertices in the (i-3)-th frame, the (i-2)-th frame, and the (i-1)-th frame. Similarly, the actual motion information of the motion vertex in the i-th frame can be calculated based on the original positions of the motion vertices in the (i-3)-th frame, the (i-2)-th frame, and the (i-1)-th frame. Then, the average of the actual motion information of the motion vertex in the (i-1)-th frame and the actual motion information of the same motion vertex in the i-th frame is calculated (or a weighted summation of the actual motion information of each frame is performed) to obtain the final actual motion information of the motion vertex in the i-th frame.

[0284] S3001a details how to use the vertex positions (specifically, the reconstructed positions) of each frame in frames i-3, i-2, and i-1, along with the corresponding instantaneous frame rate, to determine the actual motion information (essentially the reconstructed actual motion information) of the moving vertex in frame i-1. Similarly, in S3001b, a similar method can be used to use the vertex positions (specifically, the original positions) of each frame in frames i-2, i-1, and i, along with the instantaneous frame rate of these frames, to determine the actual motion information (not the reconstructed actual motion information) of the moving vertex in frame i.

[0285] The actual motion information includes the actual motion trend information of the motion vertex in the time slot corresponding to the i-th frame.

[0286] Therefore, when determining the actual motion trend information, it is necessary to refer not only to the original position of the motion vertex of each of the M historical frames, but also to the time slot corresponding to each of the M historical frames, which is the reciprocal of the instantaneous frame rate of the corresponding frame. For example, the time slot corresponding to the i-th frame is the time interval between the (i-1)-th frame and the i-th frame, which is the reciprocal of the instantaneous frame rate of the i-th frame.

[0287] S3002: The encoding end can calculate the residual between the predicted motion information of the motion vertex in the i-th frame and the actual motion information of the motion vertex in the i-th frame, and encode the residual to obtain the bitstream.

[0288] Each moving vertex has predicted motion information and actual motion information. The encoder can calculate the difference between the actual motion information and the predicted motion information of each moving vertex to obtain the residual of each moving vertex. Then, entropy coding is performed on the residual to encode the original position of each moving vertex in the current frame.

[0289] Entropy coding can include, but is not limited to, Shannon coding, Huffman coding, and arithmetic coding, etc., without any restrictions.

[0290] Furthermore, both the actual motion information and the predicted motion information of the current frame can include the offset acceleration of the motion vertex in at least one offset direction.

[0291] For example, when calculating the residual of a moving vertex in the i-th frame, one can calculate the residual between the actual offset acceleration and the predicted offset acceleration of the vertex in the X-axis direction of the three-dimensional Cartesian coordinate system, the residual between the actual offset rate and the predicted offset rate of the same vertex in the Y-axis direction of the three-dimensional Cartesian coordinate system, and the residual between the actual offset acceleration and the predicted offset acceleration (or offset rate) of the same vertex in the Z-axis direction of the three-dimensional Cartesian coordinate system. These three residuals are then encoded to encode the i-th frame into the bitstream.

[0292] In the embodiment corresponding to Figure 3A, the encoder can use the original positions of the motion vertices of at least two adjacent historical frames encoded before the current frame and the instantaneous frame rate of each historical frame to calculate the actual motion information of the motion vertex in the current frame, thereby improving the accuracy of the calculation of the actual motion information. Furthermore, the encoder can use the reconstructed positions of the motion vertices of at least three adjacent historical frames encoded before the current frame and the instantaneous frame rate of each historical frame to calculate the actual motion information (which is the reconstructed actual motion information) of the motion vertex in the previous frame of the current frame, and use the actual motion information of the motion vertex in the previous frame of the current frame as the predicted motion information for the motion vertex in the current frame. Since the actual motion trend information of the moving vertex in adjacent frames is very close when the moving vertex is not moving, using the actual motion information of the moving vertex in the previous frame of the current frame as the predicted motion information of the moving vertex in the current frame can improve the closeness between the predicted motion information of the current frame and the actual motion information of the current frame. This can reduce the coding residual, reduce the coding amount, improve the compression ratio, shorten the bit stream length and reduce the bit rate, and improve the transmission efficiency and storage efficiency.

[0293] Figure 3B is a flowchart of a method for encoding the current frame according to an embodiment of this application. The method flowchart takes encoding a moving vertex (e.g., vertex P) of the current frame as an example to illustrate the principle of the encoding method of this application. Figure 3B can be implemented based on the architecture shown in Figure 1A, and the flowcharts shown in Figures 1B, 1C, 2A to 2D, 3A, and 3B, but is not limited to combining the above architecture and flowcharts.

[0294] As shown in Figure 3B, the process of encoding each moving vertex in the current frame is introduced by taking the encoding of vertex P in the current frame (the i-th frame) as an example.

[0295] It should be understood that even if the vertices in the current frame are moving at a constant speed in the 3D mesh sequence, the encoding and decoding process of the moving vertices in this application is also applicable to the vertices that are moving at a constant speed.

[0296] Figure 3B shows a schematic diagram of the three-dimensional mesh of the i-th frame to be encoded, the (i-3)-th frame, the (i-2)-th frame, and the (i-1)-th frame already encoded.

[0297] Referring to Figure 3B, frames i-3, i-2, and i-1 can share the same three-dimensional coordinate system. However, because the position of vertex P changes in different frames, the original positions of vertex P in frames i-3, i-2, i-1, and i-1, respectively, are Pi and Pi, respectively, within the same three-dimensional coordinate system. i-3 P i-2 P i-1 P i When vertex P moves at a non-uniform velocity in a 3D mesh sequence, the position of vertex P changes from position P... i-3 Sequentially shift to position P i-2 Location P i-1 Location P i For example, if the instantaneous frame rate of the i-th frame in the frame sequence is fi, then the time interval between the (i-1)-th frame and the i-th frame (i.e., the time slot corresponding to the i-th frame) Δti = 1 / fi.

[0298] For ease of explanation, the encoding process of this application is illustrated by taking the example that the original position of vertex P in the (i-3), (i-2), and (i-1)th frames is the same as the reconstructed position of vertex P in the (i-3), (i-2), and (i-1)th frames.

[0299] Specifically, when the encoder calculates the actual motion information (essentially the reconstructed actual motion information, such as offset acceleration in at least one offset direction) of vertex P in the (i-3), (i-2), and (i-1) frames respectively, as the predicted motion information of vertex P in the (i-1) frame, it can be achieved through the following process:

[0300] The encoding end can be based on the position P of vertex P. i-3 and position P i-2 To calculate the position of vertex P from the position P in the (i-3)th frame. i-3 Offset to the position P of vertex P in frame i-2. i-2 Offset displacement Δ in at least one offset direction s2 (e.g., the difference between the x components); and based on this offset displacement Δ s2 Given the time interval Δt2, calculate the offset rate v of vertex P from frame i-3 to frame i-2 in at least one of the aforementioned offset directions. i-2 For example, v i-2 =Δ s2 / Δt2;

[0301] Similarly, the encoding end can be based on the position P of vertex P. i-2 and position P i-1 To calculate the offset displacement Δ of vertex P from frame (i-2) to frame (i-1) in at least one of the aforementioned offset directions. s1 (e.g., the difference between the x components); and based on this offset displacement Δ s1 Given the time interval Δt1, calculate the offset rate v of P from frame i-2 to frame i-1 in at least one of the aforementioned offset directions. i-1 For example, v i-1 =Δ s1 / Δt1;

[0302] Then, the encoder can base its work on the aforementioned offset rate v of vertex P in the (i-2)th frame. i-2 And the aforementioned offset rate v of vertex P in the (i-1)th frame i-1 The position P of vertex P in the (i-2)th frame is calculated using the time interval Δt(i-1) between the (i-2)th and (i-1)th frames. i-2 Offset to position P in frame i-1 i-1 The offset acceleration a in at least one of the aforementioned offset directions i-1 For example, a i-1 =(v i-1 -v i-2 ) / Δt(i-1);

[0303] In this way, the encoder can calculate the reconstruction information of the actual motion information of vertex P in the (i-3), (i-2), and (i-1)th frames, as well as the corresponding instantaneous frame rate, based on the reconstruction position of vertex P in each of the (i-3), (i-2), and (i-1)th frames, and the corresponding instantaneous frame rate (e.g., the offset acceleration a in at least one offset direction mentioned above). i-1 ), to serve as the predicted motion information for vertex P of the i-th frame.

[0304] Similarly, please refer to Figure 3B to calculate the position P of vertex P from the (i-2)th frame. i-2 Offset to position P in frame i-1 i-1 The offset acceleration a in at least one of the aforementioned offset directions i-1 The principle is similar; the encoding end can also base its work on the original positions of vertex P in the (i-2)th, (i-1)th, and ith frames (e.g., position P). i-2 Location P i-1 Location P i The position P of vertex P in the (i-1)th frame is calculated using the corresponding instantaneous frame rate. i-1 Offset to position P in frame i i The offset acceleration a in at least one of the aforementioned offset directions i To obtain the actual motion information of vertex P in the i-th frame (e.g., the offset acceleration a in at least one offset direction mentioned above). i ).

[0305] Finally, please refer to Figure 3B. The encoder can control the offset acceleration a. i With offset acceleration a i-1 The difference is encoded to encode the residual between the actual motion information and the predicted motion information of vertex P in the i-th frame, thereby completing the encoding of vertex P in the i-th frame. The encoding process for other vertices in the i-th frame is similar to that for vertex P, and will not be described in detail here.

[0306] Figure 3C is a flowchart of a method for encoding the current frame (a three-dimensional mesh to be encoded) provided in an embodiment of this application. The method can be implemented based on the architecture shown in Figure 1A, the process shown in Figure 2B, Figure 3A, and Figure 3B, but is not limited to combining the architecture and process shown in Figure 1A, Figure 1B, Figure 1C, Figure 2B, Figure 3A, and Figure 3B.

[0307] In the method flow shown in Figure 3C, the encoding end can convert the vertex positions of each moving vertex in the historical frame and the current frame from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system, so as to calculate the actual motion information and predicted motion information of the vertex in the three components of height, radius and angle, so as to realize the encoding of the vertex of the current frame.

[0308] In the method flow shown in Figure 3C, regardless of whether it is actual motion information or predicted motion information, the motion information may include the offset motion information of the current frame's moving vertex in the three offset directions of the cylindrical coordinate system. The offset motion information includes the offset rate of the moving vertex in the height direction (also expressed as the height component), the offset rate of the moving vertex in the radius direction (also expressed as the radius component), and the offset acceleration of the vertex in the angular direction (also expressed as the angular component).

[0309] Among them, the offset rate of the moving vertex in the height component is also simply referred to as the height change rate of the vertex, the offset rate of the moving vertex in the radius component is also simply referred to as the radius change rate of the vertex, and the offset acceleration of the moving vertex in the angular component is also simply referred to as the angular acceleration of the vertex.

[0310] As shown in Figure 3C, this method mainly includes the following steps:

[0311] S501: The encoding end converts the reconstructed position of the motion vertex in each frame of the (i-3), (i-2), and (i-1) frames from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system to obtain the height, radius, and angle (specifically, reconstructed height, reconstructed radius, and reconstructed angle) of the motion vertex in each frame of the (i-3), (i-2), and (i-1) frames; and the encoding end converts the original position of the motion vertex in each frame of the (i-2), (i-1), and (i) frames from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system to obtain the height, radius, and angle (specifically, original height, original radius, and original angle) of the motion vertex in each frame of the (i-2), (i-1), and (i) frames.

[0312] The reconstructed positions of the motion vertices in each of the above-mentioned frames i-3, i-2, and i-1 can be obtained through S300a shown in Figure 2B.

[0313] The original positions of the motion vertices in each of the above-mentioned (i-2), (i-1), and (i) frames can be obtained through S101 shown in Figure 1A.

[0314] Specifically, the encoding end can convert the coordinates (x, y, z) of the reconstructed position of the moving vertex in the (i-3)th frame from a three-dimensional Cartesian coordinate system to a cylindrical coordinate system with the centroid of the (i-3)th frame as the origin. This yields the component values ​​(h, r, θ) of the height, radius, and angle components of the moving vertex in the (i-3)th frame, where h represents height, r represents radius, and θ represents angle. The (i-3)th frame is a three-dimensional mesh containing multiple vertices. The centroid of the (i-3)th frame can be the centroid of all vertices in the three-dimensional mesh, or any other method of determining the centroid; no restrictions are placed here.

[0315] Similarly, the encoding end can convert the reconstructed positions of the moving vertices in each frame of the (i-2)th and (i-1)th frames from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system with the centroid of the (i-3)th frame as the origin, so that different frames share the same cylindrical coordinate system. This facilitates the accurate calculation of the vertex motion information and obtains the component values ​​of the height, radius, and angle components of the moving vertex in each frame of the (i-2)th and (i-1)th frames.

[0316] Of course, the origin of the same coordinate system shared by different frames is not limited to the centroid of the (i-3)th frame. It can also be the centroid of other frames, such as the centroid of the first frame in the frame sequence. There is no restriction here, as long as the vertices of the frame sequence share the same coordinate system.

[0317] Similarly, the encoding end can convert the original positions of the motion vertices in each of the (i-2), (i-1), and (i) frames from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system to obtain the height, radius, and angle (specifically, the original height, original radius, and original angle) of the motion vertices in each of the (i-2), (i-1), and (i) frames.

[0318] Taking vertex P shown in Figure 3B as an example, the encoding end can obtain the vertex coordinates P of vertex P in the (i-3), (i-2), (i-1), and (i-1)th frames through coordinate system transformation in this step. i-3 P i-2 P i-1 P i In these four frames, the coordinate values ​​of vertex P are all component values ​​of height, radius and angle.

[0319] S502a: The encoder calculates the reconstruction offset rate of the vertex in the height component, the reconstruction offset rate of the vertex in the radius component, and the reconstruction offset acceleration of the vertex in the angle component of the i-1 frame based on the reconstructed height, reconstruction radius, and reconstruction angle of the moving vertex in the i-3, i-2, and i-1 frames, as well as the instantaneous frame rate of each frame (e.g., i-2 and i-1 frames), to obtain the actual motion information of the reconstructed moving vertex in the i-1 frame (which can be used as the predicted motion information of the moving vertex in the i-1 frame).

[0320] As described in the above embodiments, the actual motion information may include the offset acceleration of the motion vertex of the (i-1)th frame in at least one of the three offset directions. In the embodiment of Figure 3C, the offset direction corresponding to the calculated offset acceleration is the angular component. In other embodiments, the offset direction corresponding to the calculated offset acceleration may also be at least one of the height component and the radius component, which is not limited here.

[0321] For ease of explanation, the method of this application is illustrated by taking the example that the instantaneous frame rate of each frame in the three-dimensional mesh sequence is the same. The time slot of each frame is the time interval Δt, where Δt = 1 / f, and f is the frame rate of the three-dimensional mesh sequence.

[0322] In S502a, referring to Figure 3B, taking vertex P as an example, when the encoder calculates the reconstructed offset rate of the moving vertex in the height direction of the (i-1)th frame (the same applies to calculating the actual offset rate), it can first calculate the distance from position P to vertex P. i-2Offset to position P i-1 Offset displacement Δh in the height h direction i-1 , where the offset displacement Δh i-1 For position P i-1 Height component and P i-2 The difference in height components; then, based on this offset displacement Δh i-1 The position P of vertex P from frame i-2 is calculated using the time interval Δt. i-2 Offset to position P in frame i-1 i-1 Offset rate in the height h direction For example The time interval Δt is the time interval between two adjacent frames.

[0323] In S502a, referring to Figure 3B, taking vertex P as an example, when the encoder calculates the reconstruction offset rate of the vertex in the radial direction of the (i-1)th frame (the calculation of the actual offset rate is similar), it can first calculate the reconstructed offset rate of vertex P from position P. i-2 Offset to position P i-1 Offset displacement Δr on the radius r component i-1 Wherein, the offset displacement Δr i-1 For position P i-1 The radius component and P i-2 The difference between the radius components; then, based on this offset displacement Δr i-1 The position P of vertex P from frame i-2 is calculated using the time interval Δt. i-2 Offset to position P in frame i-1 i-1 Offset rate in the direction of radius r For example

[0324] In S502a, referring to Figure 3B, taking vertex P as an example, when the encoder calculates the reconstructed offset acceleration of vertex P in the angular component of the (i-1)th frame (the same applies to calculating the actual reconstructed offset acceleration), it can first calculate the offset rate of vertex P in the angular component of the (i-2)th frame. and the offset rate of vertex P in the angular component of the (i-2)th frame. Then, based on these two offset rates and the time interval Δt, the offset acceleration of vertex P in the angular component of the (i-1)th frame is calculated. For example

[0325] Among them, in calculating the above offset rate and the aforementioned offset rate The implementation process is similar to the principle of calculating the offset rate of vertex P in the radial direction of the (i-1)th frame. Specifically, in calculating the offset rate... At that time, the encoder can calculate the vertex P from position P i-3 Offset to position P i-2 Offset angle Δθ in the angular component i-2 Where, the offset angle Δθ i-2 For position P i-3 Angular components and position P i-2 The difference of the angular components; then, based on this offset angle Δθ i-2 The position P of vertex P from frame i-3 is calculated using the time interval Δt. i-3 Offset to position P in frame i-2 i-2 Offset rate in the angular direction (Also expressed as angular velocity), for example

[0326] Similarly, the encoder can use the position P of vertex P in the (i-2)th frame. i-2 The angle components, and the position P of vertex P in the (i-1)th frame. i-1 Using the angle component and the time interval Δt, we can calculate the offset rate of vertex P in the (i-1)th frame in the angle component. (Also expressed as angular velocity) is used as the predicted offset rate of vertex P in the angular component of the i-th frame.

[0327] S502b: The encoder calculates the actual offset rate of the moving vertex in the height direction, the actual offset rate of the vertex in the radius component, and the actual offset acceleration of the vertex in the angle component in the i-th frame based on the original height, original radius, and original angle of the moving vertex in the i-2th frame, the i-1th frame, and the i-th frame, as well as the instantaneous frame rate of each frame (e.g., the i-1th frame and the i-th frame), in order to obtain the actual motion information of the moving vertex in the i-th frame.

[0328] S502a and S502b can be executed serially or in parallel, and this application does not impose any restrictions on this.

[0329] The execution principle of S502b is similar to that of S502a. The only difference is that when calculating the actual offset rate or actual offset acceleration, the components of the cylindrical coordinate system used are the actual components (such as the original height, original radius, and original angle), which will not be elaborated here.

[0330] Referring to the embodiment in Figure 3B, taking vertex P as an example, the actual offset rate of vertex P in the height component of the i-th frame can be obtained through S502b. The actual offset rate of vertex P in the radius component of the i-th frame The actual offset acceleration of vertex P in the angular component of the i-th frame

[0331] S503: The encoding end can perform residual coding on the actual motion information of the vertex of the i-th frame and the predicted motion information of the vertex of the i-th frame to obtain the bitstream of the i-th frame.

[0332] Specifically, taking vertex P as an example, the encoder can calculate the difference between the actual offset rate of vertex P in the height component and the predicted offset rate in the height component of the i-th frame (e.g., ), to obtain the residual of vertex P in the height component of the i-th frame; and the encoder can calculate the difference between the actual offset rate of vertex P in the radius component and the predicted offset rate in the radius component of the i-th frame (e.g. ), to obtain the residual of vertex P in the radius component of the i-th frame; and the encoder can calculate the difference between the actual offset acceleration of vertex P in the angular component and the predicted offset acceleration in the angular component of the i-th frame (e.g. ), to obtain the residual of vertex P in the angular component of the i-th frame.

[0333] In the method flow shown in Figure 3C, the encoder encodes not the residual between the predicted and accurate coordinates of the vertex in the current frame, but the residual between the predicted and accurate motion information of the vertex. When vertices in a 3D mesh undergo non-uniform motion, the offset accelerations of vertices in adjacent frames are similar. By mining the offset acceleration of the deformable 3D mesh (e.g., cloth) in the offset direction, the residual between the actual and predicted offset accelerations of the vertices in the current frame can be encoded. This reduces the amount of data in the encoded residual, thereby improving the compression rate of deformable 3D mesh sequences such as cloth or elastic bodies, and thus reducing the bit rate.

[0334] In the method flow shown in Figure 3C, the encoding end converts the vertex positions of both the historical frame and the current frame into cylindrical coordinates for encoding the current frame. In this way, when the moving vertex rotates in the 3D mesh sequence (for example, a person wearing virtual clothes is dancing ballet, causing the virtual clothes to rotate), the angular acceleration of the moving vertex in the angle component changes very little between frames. Therefore, by calculating the residual between the actual offset acceleration and the predicted offset acceleration in the angle component and encoding the residual, a smaller residual can be used to encode the 3D mesh. Thus, the motion information components (height, radius, angle) calculated by the encoding end of this application can match the motion pattern of the 3D mesh, thereby making the predicted motion information closer to the actual motion information and improving the data compression rate.

[0335] Figure 3D is a flowchart of another method for encoding the current frame (a three-dimensional mesh to be encoded) provided in an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, and the processes shown in Figures 1B, 1C, 2A to 2D, 3A, and 3B, but is not limited to combining the above architecture and processes.

[0336] The method flow shown in Figure 3D is largely the same as that shown in Figure 3C. The difference is that in the method flow shown in Figure 3D, the motion information (including predicted motion information and actual motion information) calculated by the encoder for the current frame is three offset accelerations on three offset components, not just the offset acceleration of the moving vertex on the angle component as shown in the method flow shown in Figure 3C. It also includes the offset acceleration of the moving vertex on the radius component and the offset acceleration of the moving vertex on the height component.

[0337] Specifically, taking vertex P shown in Figure 3B as an example, the actual motion information may include the actual offset acceleration of vertex P in the height component of the i-th frame, the actual offset acceleration of vertex P in the radius component of the i-th frame, and the actual offset acceleration of vertex P in the angle component of the i-th frame. The predicted motion information may include the reconstructed actual offset acceleration of vertex P in the height component of the (i-1)-th frame, the reconstructed actual offset acceleration of vertex P in the radius component of the (i-1)-th frame, and the reconstructed actual offset acceleration of vertex P in the angle component of the (i-1)-th frame.

[0338] As shown in Figure 3D, this method mainly includes the following steps:

[0339] S601: The encoding end converts the reconstructed position of the motion vertex in each frame of the (i-3), (i-2), and (i-1)th frames from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system to obtain the height, radius, and angle (specifically, the reconstructed height, reconstructed radius, and reconstructed angle) of the motion vertex in each frame of the (i-3), (i-2), and (i-1)th frames; and the encoding end converts the original position of the motion vertex in each frame of the (i-2), (i-1), and (i)th frames from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system to obtain the height, radius, and angle (specifically, the original height, original radius, and original angle) of the motion vertex in each frame of the (i-2), (i-1), and (i)th frames.

[0340] The execution principle of S601 is the same as that of S501 in the method flow shown in Figure 3C, and will not be repeated here.

[0341] S602a: The encoder calculates the reconstruction offset rate of the moving vertex in the height component of the i-2 frame, the reconstruction offset rate of the moving vertex in the height component of the i-2 frame, the reconstruction offset rate of the moving vertex in the radius component of the i-2 frame, the reconstruction offset rate of the moving vertex in the radius component of the i-1 frame, the reconstruction offset rate of the moving vertex in the angle component of the i-2 frame, and the reconstruction offset rate of the moving vertex in the angle component of the i-2 frame, based on the reconstruction height, reconstruction radius, and reconstruction angle of the vertices in the i-3 frame, the i-2 frame, and the i-1 frame.

[0342] After S601, the encoding end can execute S602a.

[0343] Specifically, taking vertex P as an example, the encoder can calculate the offset rate of vertex P in the height component of the (i-1)th frame based on the height component values ​​of vertex P in the (i-2)th and (i-1)th frames, as well as the instantaneous frame rate of the (i-1)th frame. (This refers to the reconstructed offset rate). The specific implementation process can be found in the relevant description of the embodiment in Figure 3C, and will not be repeated here. Similarly, the encoder can calculate the offset rate of vertex P in the height component of the (i-2)th frame based on the height component values ​​of vertex P in the (i-3)th and (i-2)th frames, and the instantaneous frame rate of the (i-2)th frame. (This is the reconstruction offset rate.)

[0344] Similarly, the encoder can calculate the offset rate of vertex P in the radius component of the (i-2)th frame based on the radius component values ​​of vertex P in the (i-1)th frame and the instantaneous frame rate of the (i-1)th frame. (This refers to the reconstructed offset rate). The specific implementation process can be found in the relevant description of the embodiment in Figure 3C, and will not be repeated here. Similarly, the encoder can calculate the offset rate of vertex P in the radius component of the (i-2)th frame based on the radius component values ​​of vertex P in the (i-3)th and (i-2)th frames, and the instantaneous frame rate of the (i-2)th frame. (This is the reconstruction offset rate.)

[0345] Similarly, the encoder can calculate the offset rate of vertex P in the angle component of the (i-2)th frame based on the angle component values ​​of vertex P in the (i-1)th and (i-2)th frames, as well as the instantaneous frame rate of the (i-1)th frame. (This refers to the reconstructed offset rate). The specific implementation process can be found in the relevant description of the embodiment in Figure 3C, and will not be repeated here. Similarly, the encoder can calculate the offset rate of vertex P in the angle component of the (i-2)th frame based on the angle component values ​​of vertex P in the (i-3)th and (i-2)th frames, and the instantaneous frame rate of the (i-2)th frame. (This is the reconstruction offset rate.)

[0346] S603a: The encoder calculates the offset acceleration of the motion vertex in the height component of the (i-2)th frame based on the reconstructed offset rate of the motion vertex in the height component of the (i-1)th frame, the reconstructed offset rate of the motion vertex in the height component of the (i-2)th frame, and the instantaneous frame rate of the (i-1)th frame; and calculates the reconstructed offset acceleration of the motion vertex in the radius component of the (i-2)th frame based on the reconstructed offset rate of the motion vertex in the radius component of the (i-1)th frame, and the instantaneous frame rate of the (i-1)th frame; and calculates the reconstructed offset acceleration of the motion vertex in the angle component of the (i-2)th frame based on the reconstructed offset rate of the motion vertex in the angle component of the (i-1)th frame, and the instantaneous frame rate of the (i-1)th frame, so as to obtain the actual motion information of the reconstructed motion vertex in the (i-1)th frame (which can be used as the predicted motion information of the motion vertex in the (i-1)th frame).

[0347] Following S602a, the encoder can execute S603a. Continuing with vertex P as an example, the encoder can base its work on the offset rate of vertex P in the angular components of the (i-2)th and (i-1)th frames. And the instantaneous frame rate of the (i-1)th frame, to calculate the reconstructed offset acceleration of vertex P in the angular component of the (i-1)th frame. The specific implementation process can be referred to the relevant description of the embodiment in Figure 3C, and will not be repeated here;

[0348] Similarly, the encoding end can reconstruct the offset rate of vertex P in the height component based on the (i-2)th frame and the (i-1)th frame, respectively. And the instantaneous frame rate of the (i-1)th frame, to calculate the reconstructed offset acceleration of vertex P in the height component of the (i-1)th frame.

[0349] Similarly, the encoding end can reconstruct the offset rate of vertex P on the radius component based on the (i-2)th frame and the (i-1)th frame, respectively. And the instantaneous frame rate of the (i-1)th frame, to calculate the reconstructed offset acceleration of vertex P in the radius component of the (i-1)th frame.

[0350] S602b: The encoder calculates the actual offset rate of the motion vertex in the height component of the (i-2)th frame, the actual offset rate of the motion vertex in the height component of the i-th frame, the actual offset rate of the motion vertex in the radius component of the (i-2)th frame, the actual offset rate of the motion vertex in the radius component of the i-th frame, the actual offset rate of the motion vertex in the angle component of the (i-1)th frame, and the actual offset rate of the motion vertex in the angle component of the i-th frame based on the original height, original radius, and original angle of the motion vertex in the (i-2)th frame, the (i-1)th frame, and the i-th frame.

[0351] After S601, the encoding end can also execute S602b.

[0352] This application does not restrict the execution order of S602a and S602b; they can be executed serially or in parallel.

[0353] The implementation principle of S602b is similar to that of S602a, and will not be elaborated here.

[0354] S603b: The encoder calculates the actual offset acceleration of the moving vertex in the height component of the i-th frame based on the actual offset rate of the moving vertex in the height component of the i-th frame, the actual offset rate of the moving vertex in the height component of the i-th frame, and the instantaneous frame rate of the i-th frame; and calculates the actual offset acceleration of the moving vertex in the radius component of the i-th frame based on the actual offset rate of the moving vertex in the radius component of the i-th frame, the actual offset rate of the moving vertex in the radius component of the i-th frame, and the instantaneous frame rate of the i-th frame; and calculates the actual offset acceleration of the moving vertex in the angle component of the i-th frame based on the actual offset rate of the moving vertex in the angle component of the i-th frame, the actual offset rate of the moving vertex in the angle component of the i-th frame, and the instantaneous frame rate of the i-th frame, so as to obtain the actual motion information of the moving vertex in the i-th frame.

[0355] Following S602b, the encoder can execute S603b. Continuing with vertex P as an example, the encoder can base its output on the actual offset rate of vertex P in the angular components in the (i-1)th frame and the i-th frame. And the instantaneous frame rate of the i-th frame, to calculate the actual offset acceleration of vertex P in the angular component of the i-th frame. The specific implementation process can be referred to the relevant description of the embodiment in Figure 3C, and will not be repeated here;

[0356] Similarly, the encoding end can base its calculations on the actual offset rate of vertex P in the height component of each of the (i-1)th and i-th frames. And the instantaneous frame rate of the i-th frame, to calculate the actual offset acceleration of vertex P in the height component of the i-th frame.

[0357] Similarly, the encoding end can base its calculations on the actual offset rate of vertex P in the radius component of each of the (i-1)th and i-th frames. And the instantaneous frame rate of the i-th frame, to calculate the actual offset acceleration of vertex P in the radius component of the i-th frame.

[0358] S604: The encoder can base its work on the actual offset acceleration of the vertex in the three offset directions of the i-th frame (taking vertex P as an example, specifically the actual offset acceleration in the height component). Actual offset acceleration in the radius component Actual offset acceleration in the angular component ), and the predicted offset acceleration of the vertex in the i-th frame in the three offset directions (taking vertex P as an example, specifically the reconstructed offset acceleration in the height component). Reconstructed offset acceleration in the radius component Reconstructed offset acceleration in angular components The residuals of offset acceleration in the three offset directions are calculated and encoded to obtain the bitstream.

[0359] Specifically, taking vertex P as an example, the encoder can measure the actual offset acceleration of vertex P in the height component of the i-th frame. Offset acceleration of vertex P in height component with respect to the i-th frame Calculate residuals

[0360] Furthermore, the encoding end can measure the actual offset acceleration of vertex P in the radius component of the i-th frame. Offset acceleration of vertex P in the radius component with respect to the i-th frame Calculate residuals

[0361] Furthermore, the encoding end can measure the actual offset acceleration of vertex P in the angular component of the i-th frame. Offset acceleration of vertex P in the radius component with respect to the i-th frame Calculate residuals

[0362] Then, the three residuals of the above vertex P are encoded to encode vertex P of the i-th frame. The encoding process for the other vertices of the i-th frame is similar and will not be repeated here. In this way, the i-th frame is encoded and the bitstream is obtained.

[0363] Unlike the method flow shown in Figure 3C, in the method flow shown in Figure 3D, the encoding end can encode the residual of the offset acceleration of the vertices in the current frame in all three offset directions, but does not include the step of encoding the residual of the offset rate in the offset direction. Compared to predicting the offset rate of the vertex in one offset direction of the current frame (which has a large difference from the actual offset rate of the vertex in that offset direction, resulting in a large residual), and then encoding the residual of the offset rate, in this embodiment, the predicted offset acceleration of the current frame in that offset direction can be closer to its actual acceleration, thereby further reducing the size of the encoded residual, thereby improving the compression rate of deformable three-dimensional mesh sequences such as flexible bodies or elastic bodies such as cloth, and thus reducing the bit rate.

[0364] Figure 3E is a flowchart of another method for encoding the current frame (a 3D mesh to be encoded) provided in an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, and the processes shown in Figures 1B, 1C, 2A to 2D, 3A, and 3B, but is not limited to combining the above architecture and processes.

[0365] The implementation principle of the method flow shown in Figure 3E is the same as that of the method flow shown in Figure 3D. The difference is that in the method flow shown in Figure 3D, the motion information (including predicted motion information and actual motion information) calculated by the encoder for the current frame is three offset accelerations on the three offset components (h, r, θ) of the cylindrical coordinate system. However, in the method flow shown in Figure 3E, the encoder does not need to perform coordinate system transformation on the vertex coordinates of the current frame and the historical frames. Instead, it directly calculates the vertex motion information (including predicted motion information and actual motion information) on the Cartesian coordinate system. This motion information is three offset accelerations on the three offset components (x, y, z) of the Cartesian coordinate system.

[0366] Specifically, taking vertex P shown in Figure 3B as an example, its coordinates are in a three-dimensional Cartesian coordinate system (including the x, y, and z directions). The actual motion information of vertex P in the i-th frame may include the actual offset acceleration of vertex P in the x-direction, the actual offset acceleration of vertex P in the y-direction, and the actual offset acceleration of vertex P in the z-direction. The predicted motion information of vertex P in the i-th frame may include the reconstructed actual offset acceleration of vertex P in the x-direction, the reconstructed actual offset acceleration of vertex P in the y-direction, and the reconstructed actual offset acceleration of vertex P in the z-direction of the (i-1)-th frame.

[0367] As shown in Figure 3E, this method mainly includes the following steps:

[0368] S702a: The encoder calculates the reconstruction offset rate of the motion vertex in each of the x, y, and z directions in the i-2 frame and the reconstruction offset rate of the motion vertex in each of the i-3, i-2, and i-1 frames based on the reconstruction position of the motion vertex in each of the i-2 and i-1 frames.

[0369] In this context, the reconstructed position of the motion vertex in each of the (i-3), (i-2), and (i-1) frames is the coordinate in the Cartesian coordinate system, and the reconstructed position includes the x-component value, y-component value, and z-component value.

[0370] Specifically, continuing with vertex P as an example, the encoder can calculate the reconstruction offset rate of vertex P in each of the x, y, and z directions in the i-2th frame based on the vertex position of vertex P in each frame of the i-3rd and i-2th frames and the instantaneous frame rate of the i-2th frame (in terms of...). (represented). And the encoder can calculate the reconstruction offset rate of vertex P in each of the x, y, and z directions in the i-1 frame based on the vertex position of vertex P in each frame of the i-2 and i-1 frames and the instantaneous frame rate of the i-1 frame (represented by...). express).

[0371] The execution principle of S702a is similar to that of S602a in the method flow shown in Figure 3D, and will not be repeated here.

[0372] S703a: Based on the reconstruction offset rate of the moving vertex in each of the x, y, and z directions in the (i-2)th frame, the reconstruction offset rate of the moving vertex in each of the x, y, and z directions in the (i-1)th frame, and the instantaneous frame rate of the (i-1)th frame, the encoder calculates the reconstruction offset acceleration of the moving vertex in each of the x, y, and z directions in the (i-1)th frame to obtain the actual motion information of the reconstructed moving vertex in the (i-1)th frame (which can be used as the predicted motion information of the moving vertex in the i-th frame).

[0373] Following S702a, the encoder can execute S703a. Continuing with vertex P as an example, the encoder can base its work on the reconstruction offset rates of vertex P in the x, y, and z directions of the (i-2)th frame (in terms of...). (represented by) the reconstruction offset rate of vertex P in the x, y, and z directions of the (i-1)th frame (represented by) The reconstructed offset acceleration of vertex P in the (i-1)th frame is calculated in each of the x, y, and z directions, using the instantaneous frame rate of the (i-1)th frame as (represented by) the instantaneous frame rate of the (i-1)th frame. The specific implementation principle is similar to that of S603a in the embodiment shown in Figure 3D, and will not be repeated here.

[0374] S702b: The encoder calculates the actual offset rate of the moving vertex in each of the x, y, and z directions in the i-1 frame and the actual offset rate of the moving vertex in each of the i-2, i-1, and i frames based on the original positions of the moving vertices in each of the i-2, i-1, and i frames and the instantaneous frame rates of the i-1 and i frames.

[0375] Continuing with the example of vertex P, the encoder can calculate the actual offset rate of vertex P in each of the x, y, and z directions in the (i-2)th frame based on the original position of vertex P in each of the (i-1)th and (i-2)th frames and the instantaneous frame rate of the (i-1)th frame (in terms of...). (Represented). And the encoder can calculate the actual offset rate of vertex P in the x, y, and z directions of the i-th frame based on the (i-1)-th frame, the original position of vertex P in each frame of the i-th frame, and the instantaneous frame of the i-th frame (represented by...). express).

[0376] The implementation principle of S702b is similar to that of S702a, and will not be repeated here.

[0377] S703b: The encoder calculates the actual offset acceleration of the moving vertex in each of the x, y, and z directions in the (i-1)th frame, the actual offset rate of the moving vertex in each of the x, y, and z directions in the i-th frame, and the instantaneous frame rate of the i-th frame, so as to obtain the actual motion information of the moving vertex in the i-th frame.

[0378] Following S702b, the encoder can execute S703b. Continuing with vertex P as an example, the encoder can base its work on the actual offset rates of vertex P in the x, y, and z directions of the (i-1)th frame (respectively...). (represented by) the actual offset rate of vertex P in the x, y, z directions of the i-th frame (represented by) Using the instantaneous frame rate of the i-th frame and the instantaneous frame rate of the i-th frame, we can calculate the actual offset acceleration of vertex P in the x, y, and z directions of the i-th frame (represented by...). The specific implementation principle is similar to that of S603a in the embodiment shown in Figure 3D, and will not be repeated here.

[0379] The implementation principle of S703b is similar to that of S703a, and will not be repeated here.

[0380] S704: The encoder can base its work on the actual offset acceleration of the vertex in the x, y, and z offset directions of the i-th frame (taking vertex P as an example, specifically the offset acceleration in the x, y, and z directions). ), and the offset acceleration of the vertex in the i-th frame in the three offset directions of x, y, and z (taking vertex P as an example, specifically the three reconstructed offset accelerations in the x, y, and z directions). The residuals of offset acceleration in the three offset directions are calculated and encoded to obtain the bitstream.

[0381] After S703a and S703b, the encoding end can execute S704.

[0382] Specifically, taking vertex P as an example, the encoder can measure the actual offset acceleration of vertex P in the x-direction of the i-th frame. The predicted offset acceleration of vertex P in the x-direction relative to the i-th frame. Calculate residuals

[0383] Furthermore, the encoding end can measure the actual offset acceleration of vertex P in the y-direction of the i-th frame. The predicted offset acceleration of vertex P in the y-direction relative to the i-th frame. Calculate residuals

[0384] Furthermore, the encoder can measure the actual offset acceleration of vertex P in the z-direction of the i-th frame. The predicted offset acceleration of vertex P in the z-direction of the i-th frame. Calculate residuals

[0385] Then, the three residuals of the above vertex P are encoded to encode vertex P of the i-th frame. The encoding process for the other vertices of the i-th frame is similar and will not be repeated here. This is how the i-th frame is encoded to obtain the bitstream.

[0386] Unlike the method flows shown in Figures 3C and 3D, the method flow shown in Figure 3E eliminates the need for coordinate system transformation at the position of the moving vertices in each frame. Instead, it directly encodes the vertices of the 3D mesh established in a 3D Cartesian coordinate system, encoding the residuals of the offset accelerations of these vertices in the three offset directions within the 3D Cartesian coordinate system. Thus, when vertices in the 3D mesh undergo non-uniform motion with relatively small changes in direction (e.g., linear motion) within the 3D mesh sequence (e.g., a virtual character accelerating and causing non-uniform motion of virtual clothing), the acceleration of the moving vertices in the Cartesian coordinate system can change significantly. The encoder calculates the offset motion information (including offset acceleration) of the moving vertices in each offset direction within the Cartesian coordinate system, ensuring that the offset direction of the offset acceleration matches the direction and pattern of the non-uniform motion. This improves the approximation between the predicted motion information and the actual motion information, reduces residuals, and increases the compression ratio.

[0387] It should be understood that Figures 3C, 3D, and 3E above are merely illustrative examples of three methods for encoding the current frame by the encoding end of this application. In other embodiments, this application may also provide more encoding methods. The object encoded by the encoding method is still the residual between the actual motion information and the predicted motion information of the vertices of the current frame. The motion information (actual motion information and predicted motion information) is the offset motion information in each of the three offset directions of the three-dimensional coordinate system of the current frame. Among the offset motion information in each of the three offset directions, there is an offset acceleration in at least one offset direction. The offset motion information in the other offset directions may be offset rate or offset acceleration.

[0388] Please return to S300a shown in Figure 2B. In order to obtain the reconstruction data of N historical frames, the method flow shown in Figure 2C or Figure 2D can be used.

[0389] Figure 2C is a flowchart of a method for obtaining reconstructed data of N historical frames of the current frame (a 3D mesh to be encoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, and can be implemented using Figures 2A, 2B, and any related possible implementations. As shown in Figure 2C, the method includes the following steps:

[0390] S3201a: The encoding end acquires N sets of encoded data corresponding to the N historical frames (corresponding to the N second three-dimensional grids in the invention content) that have been encoded in the three-dimensional grid sequence.

[0391] Among them, N historical frames correspond one-to-one with N sets of coded data.

[0392] The N historical frames are the N 3D grids that have been encoded before the current frame, where N is an integer greater than or equal to 3.

[0393] The encoding end can then obtain N sets of encoded data for the N 3D grids, with each 3D grid having one set of encoded data.

[0394] In some embodiments, when the N historical frames are three-dimensional grids other than the first three frames (i.e., the first, second, and third frames) in the three-dimensional grid sequence, each of the N historical frames is encoded by the encoding method of this application, and the process shown in FIG2C can be used to decode the encoded data to obtain the reconstructed data of the N historical frames.

[0395] In other embodiments, when the N historical frames include any one of the first, second, and third frames, the decoding of any one of the first, second, and third frames can be performed using any decoding method that is either existing or developed in the future to obtain the reconstructed data of the frame. For the frames other than the first, second, and third frames in the N historical frames, the process shown in Figure 2C can be used to obtain their reconstructed data.

[0396] In some embodiments, the encoded data of each of the N historical frames may include the residual between the predicted motion information of the historical frame and the actual motion information of the historical frame.

[0397] S3201b: The encoder acquires N sets of predicted motion information from N historical frames.

[0398] Among them, the N sets of predicted motion information are the N sets of motion information predicted by the motion vertex of non-uniform motion in the three-dimensional mesh sequence in the corresponding time slots of N historical frames. The N sets of predicted motion information may include N sets of predicted motion trend information, and optionally include N sets of predicted motion rate information.

[0399] The N sets of predicted motion trend information represent the predicted motion trend of the motion vertex in the corresponding time slots of the N historical frames.

[0400] The N sets of predicted motion rate information represent the predicted motion rate of the motion vertex in the corresponding time slots of the N historical frames.

[0401] The principle of obtaining a set of predicted motion information for a historical frame is the same as that of obtaining the predicted motion information (e.g., predicted motion trend information of the current frame) for the current frame mentioned in the various embodiments of the above encoding method, and will not be repeated here.

[0402] S3202: Based on the N sets of predicted motion information, the encoder decodes the N sets of encoded data to obtain N sets of reconstructed data for N historical frames.

[0403] In the method flow shown in Figure 3E, the implementation principle of the encoding end acquiring the reconstructed data of each of the N historical frames is the same as the implementation principle of the decoding end acquiring the reconstructed data of the current frame through Figure 4A. In this embodiment, the process of the encoding end acquiring the reconstructed data of at least three historical frames that have been encoded before the current frame is the same as the process of the decoding end acquiring the reconstructed data of one frame that has been encoded in the three-dimensional mesh sequence. This ensures that the decoding end correctly decodes the current frame and improves the decoding accuracy.

[0404] Figure 2D is a flowchart of a method for obtaining reconstructed data of N historical frames of the current frame (the 3D mesh to be encoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, and can be implemented using any possible implementations shown in Figures 1B, 1C, 2A, 2B, 2C, and related methods. As shown in Figure 2D, the method includes the following steps:

[0405] S3201a: The encoding end acquires N sets of encoded data from N historical frames that have been encoded in the three-dimensional mesh sequence.

[0406] The execution principle of S3201a is the same as that of S3201a shown in Figure 2C, and will not be repeated here.

[0407] S3201c: The encoder decodes N sets of encoded data to obtain N residuals.

[0408] Here, the N residuals are the residuals between the N sets of actual motion information (e.g., actual motion trend information) of the N historical frames and the aforementioned N sets of predicted motion information (e.g., predicted motion trend information).

[0409] The N sets of actual motion information include the actual N motion trend information of the motion vertex in the N time slots corresponding to the N sets of second three-dimensional grids.

[0410] S3201c is executed after S3201a.

[0411] S3201b: The encoder acquires N sets of predicted motion information from N historical frames.

[0412] The N sets of predicted motion information include N predicted motion trend information of the motion vertex in the N time slots corresponding to the N sets of historical frames.

[0413] This application does not impose any restrictions on the execution order between S3201a and S3201b.

[0414] This application does not impose any restrictions on the execution order between S3201c and S3201b.

[0415] S3203: Based on the above N predicted motion information and the N residuals, the encoding end determines N sets of reconstructed motion information for N historical frames.

[0416] The N sets of reconstructed motion information include N reconstructed information of the actual N motion trend information of the motion vertex in the N time slots corresponding to the N historical frames.

[0417] Where N is an integer greater than or equal to 3.

[0418] S3204: The encoder determines N sets of reconstruction data for the N historical frames based on N sets of reconstructed motion information and the reconstruction data of S historical frames corresponding to each of the N historical frames.

[0419] The encoder can use a set of reconstructed motion information from one of the N historical frames, along with S sets of reconstructed data from the corresponding S historical frames, to determine a set of reconstructed data for that historical frame. The determination of the reconstructed data for the remaining N historical frames is similar. These S historical frames are the S decoded historical frames in the 3D mesh sequence; therefore, the encoder can obtain the reconstructed data for these S historical frames, which may include the reconstructed positions of the motion vertices.

[0420] Where S is an integer greater than or equal to 2, and the S historical frames corresponding to one of the N historical frames are at least two 3D meshes that have been encoded before the corresponding historical frame in the 3D mesh sequence.

[0421] For example, if the current frame is the 7th frame and the N historical frames are the 4th to the 6th frames, then when the encoding end determines the reconstructed data of the 4th frame, it can use the reconstructed motion information of the 4th frame and at least two sets of reconstructed data from at least two frames (e.g., the 2nd and the 3rd frames) that have been decoded before the 4th frame to determine the reconstructed data of the 4th frame.

[0422] Similarly, when determining the reconstructed data of the 5th frame, the encoding end can use the reconstructed motion information of the 5th frame and at least two sets of reconstructed data from at least two frames that have been decoded before the 5th frame (e.g., the 2nd and 4th frames, which are not limited here) to determine the reconstructed data of the 5th frame.

[0423] Similarly, when determining the reconstructed data of the 6th frame, the encoding end can use the reconstructed motion information of the 6th frame and at least two sets of reconstructed data from at least two frames that have been decoded before the 6th frame (e.g., the 3rd and 4th frames, which are not limited here) to determine the reconstructed data of the 6th frame.

[0424] It should be understood that the S historical frames corresponding to each of the N historical frames are completely independent, and the S historical frames corresponding to different historical frames in the N historical frames are not related.

[0425] The principle by which the encoding end of this application acquires the reconstructed data of each of the N historical frames is the same as the principle by which the decoding end acquires the reconstructed data of the current frame (the three-dimensional mesh to be decoded, for example, the i-th frame) in the following embodiments of Figure 4C and Figures 5A to 5E. For details, please refer to the decoding process of the three-dimensional mesh to be decoded in the following decoding end, which will not be repeated here.

[0426] In the method flow corresponding to Figure 4C, the encoder can use the N residuals of the N historical frames obtained by decoding the encoded data of the N historical frames, and the N sets of predicted motion information of the N historical frames to reconstruct the N sets of actual motion information of the N historical frames (referred to as reconstructed motion information). Then, the encoder can determine the reconstructed data of the historical frame based on the reconstructed data of at least two frames encoded before one of the N historical frames and the reconstructed actual motion information of the aforementioned historical frame. The reconstructed data may include the reconstructed position of the motion vertices of the current frame, and optionally, may also include the connection relationship between motion vertices (e.g., the topology of the current frame), thereby enabling the reconstruction of each of the N historical frames. When the three-dimensional mesh sequence can be deformed, the offset acceleration of the same motion vertex (referring to the same vertex identifier or the same vertex sorting position) in different frames is similar, and both the predicted motion information and the actual motion information can include motion trend information (e.g., offset acceleration). Thus, in scenarios where the three-dimensional mesh sequence undergoes deformation, it is possible to achieve correct decoding of the three-dimensional mesh at a relatively low bit rate. Due to the small residual, the amount of decoding data can be reduced, thereby improving transmission and storage efficiency.

[0427] The following are examples illustrating the structures of several bitstreams in this application:

[0428] The following bitstream structure is the data structure of the encoded data of frames after the 3rd frame in the 3D mesh sequence. The data structure of the first frame in the bitstream of the 3D mesh sequence may include the topology of any frame in the 3D mesh sequence, wherein the topology of each frame is the same.

[0429] Example 1

[0430] In the data structure of the bitstream, a 3D mesh identifier can be marked to indicate which frame the data structure belongs to. Since all vertices in that frame are processed according to the encoding / decoding method of this application, the data structure for a frame does not need to carry vertex identifiers; the syntax structures of all vertices can be arranged sequentially within the data structure of that frame.

[0431] The actual motion information of the moving vertex in the above embodiments may include the actual component information of the moving vertex in three components, namely the actual component information in the first component, the actual component information in the second component, and the actual component information in the third component.

[0432] The predicted motion information of the moving vertex may include the predicted component information of the moving vertex in three components, namely the predicted component information in the first component, the predicted component information in the second component, and the predicted component information in the third component.

[0433] The three components mentioned above are three components in a three-dimensional coordinate system, which is either a Cartesian coordinate system or a cylindrical coordinate system.

[0434] When the three-dimensional coordinate system is a Cartesian coordinate system, these three components can be the x component, y component, and z component, respectively.

[0435] When the three-dimensional coordinate system is a cylindrical coordinate system, these three components can be the radius component, the height component, and the angle (e.g., deflection angle) component, respectively.

[0436] The first residual is the residual between the actual component information (e.g., actual motion acceleration) of the moving vertex in the first component and the predicted component information (e.g., predicted motion acceleration) in the first component;

[0437] The second residual is the residual between the actual component information of the motion vertex in the second component and the predicted component information in the second component. For example, the actual component information is the actual motion speed and the predicted component information is the predicted motion speed; or the actual component information is the actual motion acceleration and the predicted component information is the predicted motion acceleration.

[0438] The third residual is the residual between the actual component information of the motion vertex in the third component and the predicted component information in the third component. For example, the actual component information is the actual motion speed and the predicted component information is the predicted motion speed; or the actual component information is the actual motion acceleration and the predicted component information is the predicted motion acceleration.

[0439] At least one of the aforementioned first residual, second residual, and third residual is the residual between the predicted motion acceleration and the actual motion acceleration on the corresponding component.

[0440] The data structure of this bitstream consists of the data structure excluding the first three frames of the 3D mesh sequence. The first three frames are transmitted without encoding compression, but they encode the static model and vertex connectivity. The vertex connectivity in each frame of the 3D mesh sequence remains unchanged and can be reused. Combined with the vertex positions obtained from decoding and reconstruction, the 3D mesh can be recovered.

[0441] Example 2

[0442] Example 3

[0443] Example 4

[0444] Example 5

[0445] For example, when encoding the vertex positions of some vertices in a 3D mesh, the first data structure of a frame of 3D mesh may include not only the 3D mesh identifier but also the vertex identifier of the moving vertex and the information of the moving vertex. The information of the moving vertex may include the three residuals of the moving vertex in the first component, the second component, and the third component, which is the same as in Embodiment 1 and will not be repeated here.

[0446] It should be noted that the above examples illustrate the structures of five bitstreams, but this does not constitute a limitation on the bitstream structure, and this application does not impose any specific limitations on it.

[0447] The following describes the decoding process of the decoding end of this application for the bitstream corresponding to the three-dimensional mesh sequence. This bitstream can be the bitstream of the three-dimensional mesh sequence encoded in any of the above embodiments. The data structure of this bitstream can be referred to the above description.

[0448] Figure 3F is a flowchart of a method for decoding the current frame (a 3D mesh to be decoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A. The bitstream can be the bitstream obtained by the encoding end through an encoding method in any of the above embodiments. As shown in Figure 3F, the method includes the following steps:

[0449] S440: The decoding end acquires the bitstream corresponding to the three-dimensional grid sequence.

[0450] It should be understood that vertices moving at a constant speed in a three-dimensional mesh sequence can also be decoded using the decoding methods of the various embodiments of this application.

[0451] In one possible implementation, the various 3D meshes in the 3D mesh sequence have the same topological structure. For example, the number of vertices, the vertex identifiers, and the connection relationships between vertices are the same in each frame of the 3D mesh sequence. This facilitates tracking and determining the motion information of a moving vertex undergoing non-uniform motion within the 3D mesh sequence.

[0452] S441: The decoding end obtains the encoded data of the current frame from the bitstream.

[0453] The encoded data is not the original bit sequence in the bitstream, but rather the data obtained after decoding the original bit sequence in the bitstream.

[0454] This encoded data can also be referred to as decoded data at the decoding end.

[0455] For example, the decoding end can decode the bitstream (e.g., entropy decoding) to read the encoded data of the current frame from the bitstream, or it can read the encoded data of the current frame from other storage media or devices, without any restrictions.

[0456] S442: The decoding end obtains the actual motion trend information of the current frame based on the encoded data.

[0457] For example, the decoding end can obtain the actual fortune information from the encoded data, or the decoding end can use the encoded data in combination with other data to obtain the actual fortune trend information of the current frame. There are no restrictions here.

[0458] The actual motion trend information indicates the actual motion trend of the non-uniform motion vertices in the three-dimensional mesh sequence in the corresponding time slot of the current frame.

[0459] The actual motion trend can be motion acceleration. Since the current frame is a three-dimensional structure, the actual motion trend can be motion acceleration in at least one of the three components of the three-dimensional coordinate system.

[0460] In some embodiments, when the actual motion trend is the motion acceleration on one component (e.g., the X component), the encoder can also obtain the actual motion rate information of the current frame based on the encoded data. The actual motion rate information is the actual motion rate of the motion vertex on the other two components (e.g., the Y component and the X component) in the corresponding time slot of the current frame.

[0461] In some embodiments, when the actual motion trend is the motion acceleration of each of the two components (e.g., the X component and the Y component), the encoder can also obtain the actual motion rate information of the current frame based on the encoded data. The actual motion rate information is the actual motion rate of the motion vertex in the other component (e.g., the Z component) in the corresponding time slot of the current frame.

[0462] The three-dimensional mesh sequence has a frame rate, the reciprocal of which is the total motion time of the aforementioned moving vertices in the three-dimensional mesh sequence.

[0463] Each frame in a 3D mesh sequence can have its own instantaneous frame rate. For example, the average instantaneous frame rate of all frames in the 3D mesh sequence is the frame rate of the 3D mesh sequence. The instantaneous frame rates of different frames in the 3D mesh sequence can be the same or different, without restriction. Based on this, the time slot corresponding to a frame in the 3D mesh sequence can be the reciprocal of the instantaneous frame rate of that frame, and the sum of the time slots corresponding to each frame in the 3D mesh sequence can be the overall motion time mentioned above.

[0464] S443: Based on this actual motion trend information, the decoding end decodes and obtains the reconstructed data of the current frame.

[0465] The reconstructed data for the current frame may include the reconstructed positions of the moving vertices in the current frame, and optionally may also include the connection relationships between the moving vertices. Considering that the topological structure of each frame in the 3D mesh sequence is the same, this connection relationship does not need to be decoded, because the decoding end can obtain the topological structure when decoding the first frame or the frame before the current frame of the 3D mesh sequence.

[0466] In this embodiment, when decoding the current frame in a 3D mesh sequence, the decoding end can obtain the encoded data of the current frame from the bitstream and, based on the encoded data, obtain the actual motion trend information of the current frame. This actual motion trend information indicates the actual motion trend of a non-uniformly moving vertex in the 3D mesh sequence within the corresponding time slot of the current frame. In scenarios where a moving vertex in the 3D mesh sequence undergoes non-uniform motion (e.g., variable speed motion or rotational motion), although the vertex position in the 3D mesh sequence may change, causing deformation of the 3D mesh, the actual motion trend of the moving vertex within the corresponding time slot of each frame in the 3D mesh sequence changes relatively little. By decoding the current frame in the 3D mesh sequence based on this actual motion trend information, a better decoding effect can be obtained.

[0467] Figure 3G is a flowchart of a method for encoding the current frame (a 3D mesh to be encoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, but is not limited to the architecture shown in Figure 1A. This method can be combined with the process shown in Figure 3F, but is not limited to the process shown in Figure 3F. As shown in Figure 3G, the method may include the following steps:

[0468] S440: The decoding end acquires the bitstream corresponding to the three-dimensional grid sequence.

[0469] The implementation principle of this step is the same as that of S440 shown in Figure 3F, and will not be repeated here.

[0470] S441: The decoding end obtains the encoded data of the current frame from the bitstream.

[0471] The implementation principle of this step is the same as that of S441 shown in Figure 3F, and will not be repeated here.

[0472] S442a: The decoding end decodes the encoded data to obtain the actual motion trend information of the current frame.

[0473] For example, the encoded data includes the actual movement trend information.

[0474] The decoding end can use at least one decoding method, such as decompression, to obtain the actual motion trend information of the current frame.

[0475] S443: Based on this actual motion trend information, the decoding end decodes and obtains the reconstructed data of the current frame.

[0476] The implementation principle of this step is the same as that of S443 shown in Figure 3F, and will not be repeated here.

[0477] In this embodiment, when decoding the current frame in a 3D mesh sequence, the decoding end can directly decode the encoded data of the current frame in the bitstream to obtain the actual motion trend information of the current frame. In scenarios where moving vertices undergo non-uniform motion in the 3D mesh sequence, a large number of moving vertices in the current frame may have similar actual motion trend (e.g., acceleration) information. Since the encoded data of the current frame also includes the actual motion trend information of the moving vertices, a large number of moving vertices in the encoded data have similar or identical actual motion trend information. Thus, the encoded data in the bitstream occupies fewer bits, the bitstream length is shorter, and the bit rate is lower. Therefore, decoding this encoded data can improve the decompression rate.

[0478] Figure 4A is a flowchart of a method for decoding the current frame (a 3D mesh to be decoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, but is not limited to the architecture shown in Figure 1A. This method can be combined with the processes shown in Figures 3F and 3G, but is not limited to combining the processes shown in Figures 3F and 3G. The bitstream obtained by the decoding end can be the bitstream obtained by the encoding end in any of the above embodiments.

[0479] As shown in Figure 4A, the method includes the following steps:

[0480] S400: The decoding end acquires the bitstream corresponding to the three-dimensional grid sequence.

[0481] For details, please refer to the description of S440 in Figure 3F, which will not be repeated here.

[0482] S401a: The decoding end obtains the predicted motion trend information of the current frame.

[0483] The predicted motion trend information indicates the predicted motion trend (e.g., predicted motion acceleration) of the motion vertex in the corresponding time slot of the current frame;

[0484] As shown by the dashed arrow in Figure 4A, the decoding end can obtain the predicted motion trend information based on the bitstream. Alternatively, the decoding end can pre-decode the bitstream to obtain the predicted motion trend information of the current frame and cache it locally. Then, when decoding the current frame, the cached predicted motion trend information can be retrieved locally.

[0485] This application does not restrict the execution order of S401a and S401b; they can be executed sequentially or in parallel. For example, S401b can be executed before S401a; there is no restriction on this.

[0486] S401b: The decoding end obtains the encoded data of the current frame from the bitstream.

[0487] The encoded data includes data encoded based on actual motion trend information and predicted motion trend information, wherein the actual motion trend information is the actual motion trend of the motion vertex in the corresponding time slot of the current frame.

[0488] The implementation principle of S401b is the same as that of S441 shown in Figure 3F, and will not be repeated here.

[0489] S403: The decoding end obtains the actual motion trend information of the current frame based on the predicted motion trend information and the encoded data.

[0490] For example, if the encoded data includes data encoded based on the actual motion trend information and the predicted motion trend information, then the decoding end can obtain the actual motion trend information of the current frame based on the predicted motion trend information and the encoded data.

[0491] S443: The decoding end obtains the reconstructed data of the current frame based on the actual motion trend information.

[0492] The implementation principle of S443 is the same as that of S443 shown in Figure 3F, and will not be repeated here.

[0493] In one possible implementation, the encoded data includes the residual between the actual motion trend information and the predicted motion trend information.

[0494] In this embodiment, the encoded data includes the residual between the actual motion information and the predicted motion information of the current frame. Thus, when a moving vertex in the 3D mesh to be decoded moves at a non-uniform speed, although the uniform speed of the same moving vertex differs greatly between the time slots corresponding to different frames, the actual motion trend information (e.g., offset acceleration) of the same moving vertex in the time slots corresponding to different frames is similar. This makes the residual of the current frame decoded by the decoding end extremely small, thereby reducing the amount of decoded data, reducing the occupation of transmission bandwidth, and improving transmission efficiency.

[0495] In the method flow corresponding to Figure 4A, when the decoding end decodes the current frame, it can obtain the predicted motion trend information of the current frame. This predicted motion information is the same as the predicted motion trend information of the current frame obtained by the encoding end when encoding the current frame. In this way, the decoding end can use this predicted motion information to decode the encoded data of the current frame, ensuring the accuracy of the reconstructed data of the current frame, such as the closeness of the reconstructed position of the moving vertex in the current frame to its original position. In addition, the decoding end can also obtain the encoded data of the current frame from the bitstream. This encoded data is data encoded based on the actual motion information and the predicted motion information. So when the moving vertex in the three-dimensional mesh to be encoded is moving at a non-uniform speed, although the difference in the uniform speed of the same moving vertex in the time slots corresponding to different frames is large, the actual motion trend information (e.g., offset acceleration) of the same moving vertex in the time slots corresponding to different frames is close, making the difference in motion trend information between the predicted motion information (including the predicted motion trend information) of the moving vertex predicted by this application and the actual motion information (including the actual motion trend information) of the moving vertex extremely small. Therefore, the shorter bitstream length of the 3D mesh sequence results in a lower bitrate, requiring less transmission bandwidth and improving transmission and storage efficiency (more frames of encoded data can be stored in the same storage space). Furthermore, when decoding the current frame in the 3D mesh sequence, the encoding end can utilize the aforementioned predicted motion information to decode the encoded data of the current frame, obtaining the reconstructed data of the current frame. This reduces the decoding workload of the 3D mesh sequence bitstream and minimizes screen stuttering associated with it.

[0496] Figure 4B is a flowchart of a method for decoding the current frame (a 3D mesh to be decoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, but is not limited to the architecture shown in Figure 1A. Furthermore, the method flowchart shown in Figure 4B can be combined with Figures 3F, 3G, and 4A, and any possible implementation thereof. As shown in Figure 4B, the method flowchart may include the following steps:

[0497] S400: The decoding end acquires the bitstream corresponding to the three-dimensional grid sequence.

[0498] The execution principle of S400 is the same as that of S400 in Figure 4A, and will not be repeated here.

[0499] S4101a: Based on this bitstream, the decoding end obtains the reconstructed data of N decoded historical frames in the three-dimensional grid sequence.

[0500] In this context, each of the N historical frames has a set of reconstructed data. The reconstructed data of the N historical frames in S4101a are the N sets of reconstructed data corresponding to the N historical frames respectively, where N is an integer greater than or equal to a preset threshold (e.g., 3).

[0501] For example, the decoding end can obtain the reconstructed data of N historical frames by decoding N sets of encoded data from N historical frames in the bitstream.

[0502] The reconstruction data of a historical frame may include the reconstructed positions of the moving vertices, optionally the connection relationships between the moving vertices, and optionally the instantaneous frame rate corresponding to the historical frame.

[0503] Of course, the instantaneous frame rate corresponding to each frame in the 3D grid sequence may not be encoded into the bitstream, but rather agreed upon in advance by both the encoding and decoding ends. In this way, the decoding end can determine the instantaneous frame rate of the current frame based on the frame identifier of the current frame.

[0504] In some embodiments, when the decoding end acquires N sets of reconstructed data of N historical frames, it can also read the N sets of reconstructed data of the N historical frames at the decoding end. In this case, before decoding the current frame, the decoding end can pre-decode the N sets of encoded data of the N historical frames in the bitstream and store the N sets of reconstructed data of the N historical frames obtained by decoding.

[0505] Each historical frame in the aforementioned N historical frames corresponds to a set of reconstructed data, which may include the reconstructed positions of the moving vertices in that historical frame (i.e., the reconstructed vertex positions). Optionally, it may also include the topology corresponding to that historical frame. Since the topology of each frame in the 3D mesh sequence is the same, it is not necessary to reconstruct the topology. Optionally, the set of reconstructed data for the historical frames may also include the frame rate of the 3D mesh sequence, and optionally, it may also include the instantaneous frame rate of each 3D mesh in the 3D mesh sequence.

[0506] S4102a: The decoding end determines the predicted motion information of the current frame based on the N sets of reconstructed data.

[0507] The execution principle of S4102a is the same as that of S301a shown in Figure 2B of the encoding end. For details, please refer to the specific implementation process of S301a, which will not be repeated here.

[0508] The predicted motion information is the predicted motion information of a non-uniformly moving vertex in the three-dimensional mesh sequence in the corresponding time slot of the current frame. The predicted motion information may include the predicted motion trend information mentioned above, and optionally include the predicted motion rate information.

[0509] The predicted motion rate information is the predicted motion rate of the motion vertex in the corresponding time slot of the current frame.

[0510] S401b: The decoding end obtains the encoded data of the current frame from the bitstream.

[0511] The execution principle of S401b is the same as that of S401b in the embodiment of Figure 4A, and will not be repeated here.

[0512] S403: The decoding end determines the reconstructed data of the current frame based on the above predicted motion information and the encoded data of the current frame.

[0513] The execution principle of S403 is the same as that of S403 in the embodiment of Figure 4A, and will not be repeated here.

[0514] Furthermore, S4101a is executed before S4102a, and this application does not restrict the execution order between S401b and S4101a. S403 can be executed after S4102a and S401b.

[0515] In the method flow corresponding to Figure 4B, in order to ensure that the predicted motion information calculated by the encoding end and the decoding end for the same frame is the same, when the decoding end obtains the predicted motion information of the current frame, it can obtain the reconstruction data of at least three previously encoded historical frames in the 3D mesh sequence, and determine the predicted motion information of the current frame based on the reconstruction data (rather than the original data of the at least three historical frames). This ensures that the predicted motion information determined by the decoding end for the current frame is consistent with the predicted motion information obtained by the encoding end for the current frame, thereby ensuring the accurate decoding of the 3D mesh sequence by the decoding end, ensuring that the difference in vertex position between the reconstructed 3D mesh and the original 3D mesh is small, and improving the quality of the reconstructed 3D mesh.

[0516] Figure 4C is a flowchart of a method for decoding the current frame (a 3D mesh to be decoded) according to an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, but is not limited to the architecture shown in Figure 1A. Furthermore, the method flowchart shown in Figure 4C can be combined with Figures 3F, 3G, 4A, and 4B, and any possible implementation thereof. As shown in Figure 4C, the method flowchart may include the following steps:

[0517] S4101a: The decoding end acquires N sets of reconstructed data from N decoded historical frames in a 3D grid sequence based on the bitstream.

[0518] The execution principle of S4101a in Figure 4C is the same as that of S4101a shown in Figure 4B, and will not be repeated here.

[0519] S4102a: The decoding end determines the predicted motion information of the current frame based on the N sets of reconstructed data.

[0520] The execution principle of S4102a is the same as that shown in Figure 4B, and will not be repeated here.

[0521] S4101b: The decoding end decodes the encoded data of the current frame to obtain the residual.

[0522] The encoded data of the current frame can be obtained through S401b as shown in Figure 4B, which will not be described in detail here.

[0523] The residual is the difference between the actual motion information and the predicted motion information of the current frame.

[0524] S4103: The decoding end determines the reconstructed motion information of the current frame based on the above predicted motion information and the above residual.

[0525] The reconstructed motion information is the reconstructed information of the actual motion information of the current frame (e.g., actual motion trend information), and the reconstructed motion information is also expressed as the reconstructed actual motion information.

[0526] For example, the decoder can add the predicted motion information to the residual to obtain the reconstructed motion information of the current frame.

[0527] S4104: The decoding end determines the reconstruction data of the current frame based on the reconstructed motion information and the reconstruction data of the M previously decoded historical frames in the three-dimensional mesh sequence.

[0528] The decoding end can obtain the reconstructed data of the M historical frames by decoding the encoded data of the M historical frames in the bitstream. Alternatively, the decoding end can also read the decoded data of the M historical frames from another device, which can decode the encoded data of the M historical frames in the bitstream to obtain the decoded data of the M historical frames.

[0529] Where M is an integer greater than or equal to 2.

[0530] In the method flow corresponding to Figure 4C, the decoding end can reconstruct the actual motion information (referred to as reconstructed motion information) of the current frame by using the residual between the actual motion information and the predicted motion information of the current frame obtained from decoding the bitstream, as well as the determined predicted motion information of the current frame. Then, the decoding end can determine the reconstructed data of the current frame based on the reconstructed data of at least two frames decoded before the current frame and the reconstructed actual motion information of the current frame. This reconstructed data may include the reconstructed positions of the motion vertices of the current frame, and optionally, may also include the connection relationships between motion vertices (e.g., the topology of the current frame), thereby realizing the reconstruction of the current frame. When the 3D mesh sequence can be deformed, the offset acceleration of the same motion vertex (referring to the same vertex identifier or the same vertex sorting position) in different frames is similar, and both the predicted motion information and the actual motion information can include motion trend information (e.g., offset acceleration). Thus, in the scenario where the 3D mesh sequence is deformed, the correct decoding of the 3D mesh can be achieved at a smaller bitrate, and because the residual is small, the amount of data to be decompressed can be reduced.

[0531] Figures 5A-5E are flowcharts of various methods for decoding three-dimensional meshes provided in the embodiments of this application; these methods can be implemented based on the architecture shown in Figure 1A, the processes shown in Figure 3F, Figures 4A to 4C, and various possible implementation methods.

[0532] The method flow shown in Figure 5A can be a specific example of the method flow shown in Figure 4C to implement the decoding of the current frame.

[0533] In the method flow shown in Figure 5A, the decoding end can use the reconstructed positions of each motion vertex in at least three historical frames adjacent to the current frame and the instantaneous frame rate of at least two of the three historical frames to determine the predicted motion information of each motion vertex in the current frame; and based on the residual between the actual motion information and the predicted motion information of the current frame obtained by decoding the bitstream and the predicted motion information, the actual motion information of each motion vertex in the current frame is reconstructed; finally, based on the reconstructed positions of the motion vertices in the two frames preceding the current frame (the two decoded and reconstructed frames before the current frame and adjacent to the current frame) and the actual motion information of the current frame, the reconstructed positions of each motion vertex in the current frame are reconstructed to achieve decoding of the current frame. This method mainly includes the following steps:

[0534] S4001a: The decoding end determines the actual motion information (essentially the reconstructed actual motion information) of the motion vertex in the i-1 frame based on the reconstructed position of the motion vertex in each frame of the i-3, i-2, and i-1 frames, as well as the instantaneous frame rate of each frame in the i-2 and i-1 frames, and uses it as the predicted motion information of the motion vertex in the i-1 frame.

[0535] The reconstructed positions of the motion vertices in the above frames can be derived from the reconstructed data of N historical frames obtained in S4101a of the embodiment in Figure 4B or Figure 4C.

[0536] The instantaneous frame rate of each of the above frames can also come from the reconstructed data of the above N historical frames, or the instantaneous frame rate of each frame in the three-dimensional grid sequence can be pre-agreed by both the encoding and decoding ends. In this way, the decoding end can determine the instantaneous frame rate of the corresponding frame based on the frame identifier of each frame.

[0537] The execution principle of S4001a is the same as that of S3001a shown in Figure 3A. The specific implementation process and other scalable implementation methods can be found in S3001a and its possible extensions.

[0538] The three historical frames used in S4001a can be examples of the N historical frames mentioned in the various implementations of the encoding and decoding ends.

[0539] S4001b: The decoding end decodes the received bitstream to obtain the residual between the actual motion information and the predicted motion information of the motion vertex of the i-th frame.

[0540] For example, the decoder can perform entropy decoding on the bitstream to obtain the residual between the actual motion information and the predicted motion information of the vertices in the i-th frame.

[0541] S4001a and S4001b can be executed serially or in parallel, and this application does not impose any restrictions on this.

[0542] S4002: Based on the residual and the predicted motion information of the motion vertices of the i-th frame obtained from S4001a, the decoding end reconstructs the actual motion information of the motion vertices of the i-th frame, and obtains the reconstructed actual motion information of the motion vertices of the i-th frame (also referred to as reconstructed motion information).

[0543] For example, if the residual is the difference between the actual motion information and the predicted motion information of the motion vertex in the i-th frame, then in this step, the decoding end can add the residual to the predicted motion information of the motion vertex in the i-th frame obtained in S4001a, and can reconstruct the actual motion information of the motion vertex in the i-th frame.

[0544] The actual motion information of the motion vertex in the i-th frame corresponds to the predicted motion information of the motion vertex in the i-th frame.

[0545] For example, the actual motion information of a moving vertex in the i-th frame may include the vertex's offset motion information in each of the three offset directions in the three-dimensional coordinate system. In at least one of these offset directions, the offset motion information is offset acceleration. The offset motion information in the remaining offset directions may be offset velocity or offset acceleration, without limitation. However, this at least one offset direction is the same as the offset direction corresponding to the offset acceleration in the predicted motion information.

[0546] Both the actual motion information and the predicted motion information of the current frame can include the offset acceleration of the motion vertex in at least one offset direction. Then the above residual includes the residual between the actual offset motion information and the predicted offset motion information in the same offset direction.

[0547] S4003: The decoding end determines the reconstructed position of the motion vertex in the i-1 frame based on the reconstructed position of the motion vertex in the i-2 frame, the instantaneous frame rate of each frame (e.g., the instantaneous frame rate of the i-1 frame and the i frame, optionally including the instantaneous frame rate of the i-2 frame), and the reconstructed information (also expressed as reconstructed motion information) of the actual motion information of the vertex in the i frame.

[0548] In this step, the reconstructed positions of the motion vertices in the (i-1)th frame and the (i-2)th frame can be obtained in the same way as those used in S4001a, and will not be described again here.

[0549] In S4003, the decoding end can reconstruct the vertex position of the motion vertex in the current frame based on the reconstructed positions of the motion vertices of at least two historical frames that have been decoded before the current frame (here, the i-th frame), the instantaneous frame rate, and the reconstruction information of the actual motion information of the motion vertex in the current frame obtained by S4002, so as to obtain the reconstructed positions of each motion vertex in the current frame and thus complete the reconstruction of the current frame.

[0550] Among them, the at least two historical frames used in S4003 can be examples of the M historical frames mentioned in the various implementations of the encoding end and the decoding end.

[0551] It should be understood that the N historical frames used by the encoding end and the N historical frames used by the decoding end have the same frame identifier.

[0552] Similarly, it should be understood that the M historical frames used by the encoding end and the M historical frames used by the decoding end have the same frame identifier.

[0553] In other embodiments, when reconstructing the position of the motion vertex of a frame (e.g., the (i-1)th frame or the ith frame, which is not limited here), the M historical frames referenced are not limited to the two frames encoded before the frame, but can be a larger number of frames (e.g., 3 or 4 frames), i.e., M≥2, where M is an integer. Furthermore, the M historical frames are not limited to frames that are adjacent to the frame in display order, but can also be non-adjacent frames.

[0554] When reconstructing the position of the motion vertex in a frame, if the referenced M historical frames are three or more frames that have been decoded before this frame, the implementation principle of the referenced M historical frames being two frames can be combined, and algorithms such as weighted averaging or averaging (not limited here) can be used to calculate the multiple reconstructed positions of the motion vertex to obtain the reconstructed position of the motion vertex in this frame.

[0555] For example, the M historical frames referenced in S4003 (here, the (i-1)th frame and the (i-2)th frame) could also be the (i-3)th frame, the (i-2)th frame, and the (i-1)th frame. When determining the reconstructed position of the motion vertex in frame i based on the reconstructed positions of the motion vertex in each of the (i-3)th, (i-2)th, and (i-1)th frames, and the reconstructed information of the actual motion information of the motion vertex in frame i, candidate reconstructed positions of the motion vertex in frame i can be calculated based on the reconstructed positions of the motion vertex in each of the (i-3)th and (i-2)th frames, and the reconstructed information of the actual motion information of the motion vertex in frame i. Similarly, candidate reconstructed positions of the motion vertex in frame i can be calculated based on the reconstructed positions of the motion vertex in each of the (i-2)th and (i-1)th frames, and the reconstructed information of the actual motion information of the motion vertex in frame i. Then, the average (or weighted summation, etc.) of the two candidate reconstructed positions of the motion vertex in frame i is calculated to obtain the reconstructed position of the motion vertex in frame i in S4003.

[0556] Optionally, after S4003, the method may also include S4004.

[0557] S4004: The decoding end smooths the reconstructed position of the motion vertex of the i-th frame to obtain the reconstructed i-th frame.

[0558] Considering that the vertex positions of each moving vertex in the current frame obtained by the decoder through S4003 may have wrinkled meshes with uneven surfaces after connecting each moving vertex according to the topological structure of the three-dimensional mesh sequence, the decoder can smooth the reconstructed positions of the moving vertices of the i-th frame obtained by S4003 to correct the reconstructed positions of some moving vertices, so that the surface of the three-dimensional mesh (i-th frame) formed by the vertex positions of each vertex of the i-th frame after smoothing is smoother, so as to obtain the reconstructed i-th frame.

[0559] In the embodiment corresponding to Figure 5A, the decoding end can use the reconstructed positions of the motion vertices and the corresponding instantaneous frame rates of the three previously decoded frames that are adjacent in display order before the current frame to calculate the reconstructed information of the actual motion information of the motion vertices of the previous frame, which serves as the predicted motion information for the motion vertices of the current frame. Both the predicted and actual motion information reconstruction information can include the offset acceleration of the motion vertices of the current frame in at least one offset direction. Since the offset acceleration of motion vertices in adjacent frames is close during the non-uniform motion of the 3D mesh, using the reconstructed information of the actual motion information of the previous frame as the predicted motion information for the current frame can improve the accuracy of the predicted motion information for the current frame, making it closer to the actual motion information of the motion vertices of the current frame. Then, using the residual between the actual motion information and the predicted motion information of the current frame obtained from the bitstream decoding, and the reconstructed predicted motion information of the current frame, the actual motion information of the current frame can be reconstructed. Since the predicted motion information is close to the actual motion information, the reconstructed actual motion information is also closer to the actual motion information of the current frame. Finally, based on the reconstructed positions of the motion vertices of two historical frames that were decoded before the current frame and whose display order is adjacent to the current frame, the reconstructed information of the actual motion information of the motion vertex in the current frame, and the corresponding instantaneous frame rate, the reconstructed positions of the motion vertices of the current frame can be reconstructed. This ensures that the decoding end correctly decodes the 3D mesh when the residual of the bitstream is small, the bandwidth occupied is small, and the bit rate is low.

[0560] In another embodiment, this application also provides a decoding method that can be implemented based on the process shown in Figure 3G.

[0561] The process may include reconstructing information of the actual motion information of the motion vertices of the i-th frame, as shown in Figure 5A, by decoding the encoded data of the current frame; and obtaining the reconstructed position of the motion vertices of the i-th frame through S4003 as shown in Figure 5A.

[0562] In this method, it is not necessary to calculate the predicted motion information of the moving vertices of the i-th frame. Instead, the actual motion information of the current frame obtained by decoding the encoded data is used to reconstruct the vertex positions of the current frame, which has lower encoding complexity.

[0563] Figure 5B is a flowchart of a method for decoding the current frame by taking a vertex P of the current frame as an example, according to an embodiment of this application. The method can be implemented based on the architecture shown in Figure 1A, and can be implemented based on the process shown in Figures 4C and 5A, but is not limited to the embodiment combining Figures 1A, 4C and 5A.

[0564] As shown in Figure 5B, the process of decoding each vertex of the current frame is introduced by taking the decoding of vertex P in the current frame (the i-th frame) as an example.

[0565] It should be understood that even if the vertices in the current frame are moving at a constant speed in the 3D mesh sequence, the encoding and decoding process of the moving vertices in this application is also applicable to the vertices that are moving at a constant speed.

[0566] Figure 5B shows a schematic diagram of the three-dimensional mesh of the decoded i-3, i-2, and i-1 frames.

[0567] Referring to Figure 5B, the reconstructed positions of vertex P in the 3D mesh in the same 3D coordinate system are respectively Pi-3, Pi-2, and Pi-1. i-3 P i-2 P i-1 Let P be the actual position (also referred to as the original position), the reconstructed position, and the smoothed target position of vertex P in the i-th frame. i P i ′,P i "In the case where vertex P undergoes non-uniform motion in the 3D mesh sequence, the position of vertex P changes from position P..." i-3 Sequentially shift to position P i-2 Location P i-1 For example, if the instantaneous frame rate of the i-th frame in the frame sequence is fi, then the time interval between the (i-1)-th frame and the i-th frame (i.e., the time slot corresponding to the i-th frame) Δti = 1 / fi.

[0568] For ease of explanation, the decoding process of this application is illustrated by taking the example that the original position of vertex P in the (i-3), (i-2), and (i-1) frames is the same as the reconstructed position of vertex P in the (i-3), (i-2), and (i-1) frames.

[0569] Specifically:

[0570] first:

[0571] When the decoding end calculates the actual motion information (essentially the reconstructed actual motion information, such as offset acceleration in at least one offset direction) of vertex P in the (i-3), (i-2), and (i-1) frames respectively, as the predicted motion information of vertex P in the (i-1) frame, the specific implementation principle is exactly the same as the process of determining the predicted motion information of vertex P in the (i-1) frame mentioned in Figure 3B, and will not be repeated here.

[0572] In this way, the decoding end can calculate the reconstruction information of the actual motion information of vertex P in the (i-1)th frame (e.g., the offset acceleration a in at least one offset direction mentioned above) based on the reconstruction position of vertex P in the (i-3), (i-2), and (i-1)th frames and the corresponding frame rate. i-1 ), to serve as the predicted motion information for vertex P of the i-th frame.

[0573] Secondly:

[0574] The decoding end can decode the bitstream to obtain the residual between the actual motion information and the predicted motion information of vertex P in the i-th frame, such as the offset acceleration residual (a i -a i-1 ).

[0575] then:

[0576] The decoding end can add the predicted motion information of vertex P in the i-th frame to the decoded residual to obtain the reconstructed motion information of vertex P in the i-th frame (e.g., offset acceleration a in at least one offset direction). i =a i-1 +a i -a i-1 , and the offset rate in the remaining offset directions).

[0577] also:

[0578] The decoder can also obtain the reconstructed position of vertex P in the (i-1)th frame (e.g., position P). i-1 ); and based on the reconstructed position of vertex P in frame i-1 and the reconstructed position of vertex P in frame i-2, and the instantaneous frame rate (at least one instantaneous frame rate) of each of the above two frames, determine the actual motion information of vertex P reconstructed in frame i-1 (e.g., the offset rate v of vertex P in at least one of the above offset directions). i-1 (This is not offset acceleration; see the above explanation for the specific calculation process).

[0579] Then:

[0580] The decoding end can utilize the reconstructed position of vertex P in the (i-1)th frame (e.g., position P). i-1), and the actual motion information of the reconstructed vertex P of the (i-1)th frame (e.g., the offset rate v in at least one of the offset directions mentioned above). i-1 And the actual motion information of vertex P in the reconstructed i-th frame (e.g., the offset acceleration a in at least one offset direction mentioned above). i and the offset rate in the remaining offset directions. (e.g., the y-direction) (e.g., the z-direction) to calculate the reconstructed position P of vertex P in the i-th frame. i ′.

[0581] For example, the offset acceleration a in the actual motion information of vertex P in the i-th frame above. i The corresponding offset direction and the motion information reconstructed from vertex P in the (i-1)th frame (e.g., the offset rate v in at least one of the offset directions mentioned above). i-1 The corresponding offset direction is the X direction in the three-dimensional Cartesian coordinate system. For example, the position of vertex P in frame i-1 is P. i-1 The coordinates are (x1, y1, z1), and the reconstructed position P of the vertex P to be solved in the i-th frame is P1. i The coordinates of ′ are (x2, y2, z2).

[0582] For example, when calculating x2, it can be achieved using the following formula 1.1:

[0583] For example, when calculating y2, it can be achieved using the following formula 2.1:

[0584] For example, when calculating z2, it can be achieved using the following formula 3.1:

[0585] In other embodiments, the formula 2.1 above... Can be replaced with Or replace with and Any value between.

[0586] Similarly, in formula 3.1 above... Can be replaced with Or replace with and Any value between.

[0587] in, This represents the offset rate of the motion vertex in the i-1th frame in the Z component within the corresponding time slot of the i-1th frame;

[0588] It represents the offset rate of the motion vertex in the (i-1)th frame in the corresponding time slot of the (i-1)th frame in the Y component.

[0589] In this way, the decoding end can decode and obtain the reconstructed position P of vertex P in the i-th frame. i Similarly, for vertex P'(x2,y2,z2), the reconstructed positions of other vertices in the i-th frame can be obtained. Although the reconstructed position of vertex P is P''(x2,y2,z2), the reconstructed positions of other vertices in the i-th frame can be obtained. i (x2, y2, z2) and the actual position P of vertex P in the i-th frame i There is a certain error between them, but the error is within an acceptable range, making it difficult for the above-mentioned error in the image sequence obtained by rendering after decoding the 3D mesh to be observed by the naked eye.

[0590] Optionally, the decoding end can perform smoothing processing on the reconstructed positions of the moving vertices in the i-th frame. For example, as shown in Figure 5B, the position of vertex P in the i-th frame can be obtained from the reconstructed position P. i Smoothing process for target position P i This results in higher quality, less noise, and greater smoothness in the rendered image of the smoothed 3D mesh.

[0591] Figure 5C is a flowchart of a method for decoding the current frame (a 3D mesh to be decoded) provided in an embodiment of this application. The method can be implemented based on the architecture shown in Figure 1A, the process shown in Figure 4C, Figure 5A, and Figure 5B, but is not limited to combining the architecture and process shown in Figure 1A, Figure 4C, Figure 5A, and Figure 5B.

[0592] The bitstream in the process shown in Figure 5C can be the bitstream obtained by encoding through the process shown in Figure 3C.

[0593] In the method flow shown in Figure 5C, the decoding end can convert the reconstructed position of each motion vertex in the decoded and reconstructed historical frame from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system, so as to calculate the predicted motion information of the motion vertex of the current frame in the three components of height, radius and angle of the motion vertex, and then complete the reconstruction of the actual motion information of the vertex of the current frame, so as to use the actual motion information of the historical frame and the current frame to reconstruct the current frame.

[0594] In the method flow shown in Figure 5C, regardless of whether it is actual motion information or predicted motion information, the motion information may include the offset motion information of the vertex of the current frame in the three offset directions of the cylindrical coordinate system. The offset motion information includes the offset rate of the vertex in the height direction (also expressed as the height component), the offset rate of the vertex in the radius direction (also expressed as the radius component), and the offset acceleration of the vertex in the angle direction (also expressed as the angle component).

[0595] Among them, the rate of offset of a vertex in the height component is also simply called the rate of change of the vertex's height, the rate of offset of a vertex in the radius component is also simply called the rate of change of the vertex's radius, and the acceleration of offset of a vertex in the angle component is also simply called angular acceleration.

[0596] As shown in Figure 5C, this method mainly includes the following steps:

[0597] S801a: The decoding end decodes the bitstream to obtain the residual between the actual motion information and the predicted motion information of the i-th frame.

[0598] The residuals include the residuals of the offset rate of the moving vertex in the height component, the residuals of the offset rate in the radius component, and the residuals of the acceleration in the angle component between the i-th and i-1-th frames.

[0599] Specifically, referring to the data input in S503 of Figure 3C, the residual includes the residual between the actual offset rate of the moving vertex in the height component of the i-th frame and the reconstructed offset rate of the moving vertex in the height component of the (i-1)-th frame, the residual between the actual offset rate of the moving vertex in the radius component of the i-th frame and the reconstructed offset rate of the moving vertex in the radius component of the (i-1)-th frame, and the residual between the actual offset acceleration of the moving vertex in the angle component of the i-th frame and the reconstructed offset acceleration of the moving vertex in the angle component of the (i-1)-th frame.

[0600] S801b: The decoding end converts the reconstructed position of the motion vertex in each frame of the (i-3), (i-2), and (i-1) frames from the three-dimensional Cartesian coordinate system to the cylindrical coordinate system to obtain the reconstructed height, reconstructed radius, and reconstructed angle of the vertex in each frame of the (i-3), (i-2), and (i-1) frames.

[0601] S802b and S802c can be executed after S801b. This application does not restrict the execution order between S802b and S802c, and they can be executed serially or in parallel.

[0602] Furthermore, this application does not impose any restrictions on the execution order between S801a and S801b; they can be executed serially or in parallel.

[0603] The execution principle of S801b is similar to that of S501 in the embodiment of Figure 3C. The specific implementation process is described above and will not be repeated here.

[0604] S802b: The decoder calculates the reconstruction offset rate of the vertex in the height component, the reconstruction offset rate of the vertex in the radius component, and the reconstruction offset acceleration of the vertex in the angle component of the i-1 frame based on the reconstructed height, reconstruction radius, and reconstruction angle of the moving vertex in the i-3, i-2, and i-1 frames, as well as the instantaneous frame rate of each frame (e.g., i-2 and i-1 frames), to obtain the actual motion information of the reconstructed moving vertex in the i-1 frame (which can be used as the predicted motion information of the moving vertex in the i-1 frame).

[0605] The execution principle of S802b is exactly the same as that of S502a in the embodiment of Figure 5C, and will not be repeated here.

[0606] For example, referring to Figure 5B, taking vertex P as an example, the reconstruction offset rate of vertex P in the height component of the (i-1)th frame can be obtained through S802b. The reconstructed offset rate of vertex P in the radius component of the (i-1)th frame Reconstruction offset acceleration of the vertex in the angular component of the (i-1)th frame

[0607] S803a can be executed after S801a and S802b.

[0608] S803a: The decoding end determines the reconstruction information of the actual motion information of the motion vertices in the i-th frame based on the residual obtained from S801a and the predicted motion information of the motion vertices in the i-th frame obtained from S802b.

[0609] Specifically, continuing with vertex P as an example, the decoding end can calculate the residual (e.g., the offset rate of the same vertex in the height component between the i-th frame and the (i-1)-th frame obtained by S801a) ), and the reconstructed offset rate of vertex P in the height component of the (i-1)th frame obtained by S802b. The values ​​are added together to obtain the reconstructed offset rate of vertex P in the height component of the i-th frame. (Specifically, this refers to the reconstruction information of the actual offset rate).

[0610] Similarly, the decoding end can obtain the residual of the offset rate in the radius component between the same vertex of the i-th frame and the (i-1)-th frame obtained by S801a (e.g., ), and the reconstructed offset rate of vertex P in the radius component of the (i-1)th frame obtained by S802b. The values ​​are added together to obtain the reconstructed offset rate of vertex P in the radius component of the i-th frame.

[0611] Similarly, the decoding end can obtain the residual of the offset acceleration in the angular component between the same vertex of the i-th frame and the (i-1)-th frame obtained by S801a (e.g., ), and the reconstructed offset acceleration of vertex P in the angular component of the (i-1)th frame obtained by S802b. The values ​​are added together to obtain the reconstructed offset acceleration of vertex P in the angular component of the i-th frame.

[0612] For example, the reconstruction information of the actual motion information of vertex P in the i-th frame may include the aforementioned reconstruction offset rate. Reconstruction offset rate and reconstruction offset acceleration

[0613] It should be understood that in the method flow shown in Figure 5C, three graphic shapes are used to represent the data that needs to be processed. These graphics are not intended to limit the technical solution of this application, but are intended to help readers understand how the decoding end of this application uses the obtained data to decode and obtain the reconstructed position of the motion vertex.

[0614] S802c: The decoding end can calculate the reconstruction offset rate of the motion vertex in the angular component of the i-1 frame based on the reconstruction angle of the motion vertex in each frame of the i-2 and i-1 frames obtained through S801b and the instantaneous frame rate of the i-1 frame.

[0615] For ease of explanation, the method of Figures 5C to 5E of this application is illustrated by taking the example that the instantaneous frame rate of each frame in the three-dimensional mesh sequence is the same. The time slot of each frame is the time interval Δt, where Δt = 1 / f, and f is the frame rate of the three-dimensional mesh sequence.

[0616] Continuing with vertex P in Figure 5B as an example, the decoding end can utilize the position P of vertex P in the (i-2)th frame. i-2 The angle components, and the position P of vertex P in the (i-1)th frame. i-1 The angle component and the time slot Δt of the (i-1)th frame are used to calculate the reconstruction offset rate of vertex P in the (i-1)th frame on the angle component. (Also expressed as the reconstructed angular velocity), for example Wherein, the offset angle Δθ i-1 For position P i-1 Angular components and position P i-2 The difference between the angular components;

[0617] S804 can be executed after S801a, S803a, and optionally after S802c. In some embodiments, the reconstruction of the height and radius of the vertices in S804 can also be performed before S802c. S802c is mainly used to obtain the data needed to reconstruct the angle of the vertices.

[0618] S804: The decoding end can determine the reconstruction height, reconstruction radius, and reconstruction angle of the motion vertex in the i-th frame based on the actual motion information of the reconstructed motion vertex in the i-th frame obtained in S803a, the reconstruction height, reconstruction radius, and reconstruction angle of the motion vertex after the reconstruction position transformation in the (i-1)-th frame, the reconstruction offset rate of the motion vertex in the angular component in the (i-1)-th frame, and the instantaneous frame rate of the i-th frame.

[0619] Specifically, let's continue with the example of vertex P shown in Figure 5B:

[0620] The actual motion information reconstructed by the decoder for vertex P of the i-th frame includes the reconstruction offset rate in the height component. Reconstruction offset rate on the radius component and the reconstructed offset acceleration in the angular component

[0621] The reconstructed height, reconstructed radius, and reconstructed angle corresponding to the reconstructed position of vertex P in the (i-1)th frame are h, ... i-1 r i-1 ,θ i-1 ;

[0622] The reconstruction offset rate of vertex P in the (i-1)th frame on the angular component is

[0623] The time slot of the i-th frame is Δt.

[0624] The following decoder can reconstruct the reconstructed height h of vertex P in the i-th frame. i Reconstruction radius r i Reconstructing angle θ i :

[0625] The decoder can reconstruct the offset rate (e.g., offset rate) on the height component within the time slot from vertex P to frame i-1. The offset displacement of vertex P from frame (i-1) to frame (i-1) is determined based on the time slot Δt of frame i. Then, based on this offset displacement and the height component h corresponding to the position of vertex P in frame (i-1), the offset displacement is further determined. i-1 To determine the height h of vertex P in the i-th frame. i .

[0626] Wherein, the offset rate of vertex P in the height component within the time slot from frame (i-1) to frame i can be the aforementioned reconstructed offset rate. It could also be the reconstruction offset rate. It can also be and The average value is equal to and Any value within the range. The decoding end can be based on the position P of vertex P in the (i-2)th frame. i-2 The height component, and the position P of vertex P in the (i-1)th frame. i-1 The height component and the time slot Δt of the (i-1)th frame are used to calculate the reconstruction offset rate of vertex P in the (i-1)th frame on the height component. For example Wherein, the offset height Δh i-1 For position P i-1 Height components and position P i-2 The difference in height components.

[0627] For example, the decoding end can obtain the reconstructed height h of vertex P in the i-th frame using formula 1.2, formula 1.3, or formula 1.4. i .

[0628] Similarly, the decoding end can base its work on the offset rate of vertex P in the radial component within the time slot of the i-th frame (e.g., offset rate). The offset displacement of vertex P from frame (i-1) to frame (i-1) is determined based on the time slot Δt of frame i. Then, based on this offset displacement and the radius component r corresponding to the position of vertex P in frame (i-1), the offset displacement of vertex P in the radius component r is determined. i-1 To determine the radius r of vertex P in the i-th frame. i .

[0629] Wherein, the offset rate of vertex P from frame (i-1) to frame i in the radius component can be the offset rate described above. It could also be the offset rate. It can also be and The average value is equal to and Any value between [values]. The decoding end can base its value on the position P of vertex P in the (i-2)th frame. i-2 The radius component, and the position P of vertex P in the (i-1)th frame. i-1 The radius component and the time interval Δt are used to calculate the reconstruction offset rate of vertex P in the (i-1)th frame on the radius component. For example Wherein, the offset radius Δr i-1 For position P i-1 The radius component and position P i-2 The difference between the radius components.

[0630] For example, the decoding end can obtain the reconstruction radius r of vertex P in the i-th frame using formula 2.2, formula 2.3, or formula 2.4.i .

[0631] The decoding end can determine the offset rate (e.g., offset rate) of vertex P on the intrinsic angular component of the time slot of frame i-1 (the time interval from frame i-2 to frame i-1). The reconstructed offset acceleration of vertex P in the angular component of the i-th frame. And the time slot Δt of the i-th frame, to determine the time slot of vertex P in the i-th frame (from position P in the (i-1)-th frame). i-1 Reconstruction position P offset to the i-th frame i The offset angle is calculated based on the intrinsic angular component; then, based on this offset angle and the angular component θ corresponding to the position of vertex P in the (i-1)th frame,... i-1 To determine the reconstruction angle θ corresponding to the reconstruction position of vertex P in the i-th frame. i .

[0632] Wherein, the offset rate of vertex P from frame i-2 to frame i-1 in terms of the angular component can be the offset rate described above. It could also be the offset rate. It can also be and The average value is equal to and Any value between [values]. The decoding end can base its value on the position P of vertex P in the (i-2)th frame. i-2 The angle components, and the position P of vertex P in the (i-1)th frame. i-1 The angle component and the time slot Δt of the (i-1)th frame are used to calculate the reconstruction offset rate of vertex P in the (i-1)th frame on the angle component. For example Wherein, the offset angle Δθ i-1 For position P i-1 Angular components and position P i-2 The difference between the angular components. Offset rate. The calculation process is as follows:

[0633] For example, the decoding end can reconstruct the angle θ of vertex P in the i-th frame using formula 3.2, 3.3, or 3.4. i .

[0634] S805: The decoding end converts the reconstructed height, reconstructed radius, and reconstructed angle of the motion vertex of the i-th frame into a Cartesian coordinate system to obtain the reconstructed position of the motion vertex of the i-th frame.

[0635] In the method flow shown in Figure 5C, the decoding end decodes the residual between the predicted and accurate coordinate values ​​of the moving vertex in the current frame, rather than the residual between the predicted motion information and the actual motion information of the moving vertex. By mining the offset acceleration of vertices in the deformable 3D mesh (e.g., cloth) in any offset direction in the 3D mesh sequence, the residual of the offset acceleration is decoded, which can reduce the amount of data of the decoded residual, thereby reducing the bitstream length of the deformable 3D mesh sequence of flexible bodies such as cloth or elastic bodies, and reducing the bit rate.

[0636] In the method flow shown in Figure 5C, the encoding end converts the reconstructed positions of the motion of N and M historical frames into cylindrical coordinates to decode the current frame. In this way, when a moving vertex in the frame sequence performs a self-rotation motion (for example, a person wearing virtual clothes is dancing ballet, causing the virtual clothes to rotate), the angular acceleration of the moving vertex in the 3D mesh sequence about the cylindrical coordinate system is almost constant. Then, the decoding end calculates the predicted offset motion information (including offset acceleration) of the moving vertex in each offset direction in the cylindrical coordinate system, so that the calculated predicted motion information can match the motion pattern of the 3D mesh, thereby making the predicted motion information closer to the actual motion information, thus improving the accuracy of the reconstructed actual motion information of the current frame and ensuring the accurate decoding of the current frame.

[0637] Unlike the embodiment in Figure 5C, the application also provides a decoding method, which can be implemented based on the process shown in Figure 3G.

[0638] The method flow in this embodiment is largely the same as that in the embodiment shown in Figure 5C. The only difference is that the actual motion information of the current frame reconstructed by S803a, as shown in Figure 5C, is not implemented according to the process shown in Figure 5C. Instead, the encoded data of the current frame (including the actual motion information of the current frame, such as actual motion trend information) is directly decoded to obtain the actual motion information of the current frame reconstructed. Other processes are the same as those in Figure 5C and will not be described again here.

[0639] In this way, when decoding the current frame, it is not necessary to decode the residual, calculate the predicted motion information, or superimpose the predicted motion information and the residual to obtain the reconstruction result of the actual motion information. Instead, the reconstruction result of the actual motion information can be obtained directly by decoding the encoded data, which reduces the decoding complexity and thus reduces the encoding and decoding latency.

[0640] Figure 5D is a flowchart of another method for decoding the current frame (the three-dimensional mesh to be decoded) provided in an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, the process shown in Figure 4C, Figure 5A, and Figure 5B, but is not limited to combining the architecture and process shown in Figure 1A, Figure 4C, Figure 5A, and Figure 5B.

[0641] The method flow shown in Figure 5D is largely the same as that shown in Figure 5C. The difference is that in the method flow shown in Figure 5D, the actual motion information reconstructed by the decoding end for the current frame is the three offset accelerations on the three offset components, not just the offset acceleration of the moving vertex on the angle component as shown in the method flow shown in Figure 5C, but also the offset acceleration of the vertex on the radius component and the offset acceleration of the vertex on the height component.

[0642] Specifically, taking vertex P shown in Figure 5B as an example, the actual motion information reconstructed by vertex P in the i-th frame at the decoding end may include the reconstructed offset acceleration of vertex P in the height component, the reconstructed offset acceleration of vertex P in the radius component, and the reconstructed offset acceleration of vertex P in the angle component. The predicted motion information of vertex P in the i-th frame determined by the decoding end may include the reconstructed offset acceleration of vertex P in the height component, the reconstructed offset acceleration of vertex P in the radius component, and the reconstructed offset acceleration of vertex P in the angle component in the (i-1)-th frame.

[0643] The bitstream received in the method flow shown in Figure 5D can be the bitstream encoded by the method flow shown in Figure 3D.

[0644] As shown in Figure 5D, the method mainly includes the following steps:

[0645] S901a: Based on the reconstructed position of the motion vertex in each frame of the (i-3), (i-2), and (i-1) frames, and the instantaneous frame rate of the (i-2) and (i-1) frames, the decoding end calculates the reconstructed offset acceleration of the motion vertex in the (i-1) frame in the height component, the reconstructed offset acceleration of the motion vertex in the (i-1) frame in the radius component, and the reconstructed offset acceleration of the motion vertex in the (i-1) frame in the angle component, in order to obtain the actual motion information of the reconstructed motion vertex in the (i-1) frame (which can be used as the predicted motion information of the motion vertex in the (i) frame).

[0646] The principle of the implementation process of S901a is the same as that of the implementation processes of S601, S602a and S603 in the embodiment of Figure 3D, and will not be repeated here.

[0647] S901b: The decoding end decodes the received bitstream to obtain the residual between the actual motion information and the predicted motion information of the i-th frame.

[0648] The residuals include the residuals of the offset rate of the moving vertex in the height component, the residuals of the offset rate in the radius component, and the residuals of the acceleration in the angle component between the i-th and i-1-th frames.

[0649] Specifically, referring to the data input in S604 of Figure 3D, the residual includes the residual between the actual acceleration of the moving vertex in the height component of the i-th frame and the reconstructed acceleration of the moving vertex in the height component of the (i-1)-th frame, the residual between the actual acceleration of the moving vertex in the radius component of the i-th frame and the reconstructed acceleration of the moving vertex in the radius component of the (i-1)-th frame, and the residual between the actual acceleration of the moving vertex in the angle component of the i-th frame and the reconstructed acceleration of the moving vertex in the angle component of the (i-1)-th frame.

[0650] S902a can be executed after S901a and S901b.

[0651] S902a: The decoding end reconstructs the actual motion information of the motion vertices of the i-th frame based on the residual obtained from S901b and the predicted motion information of the motion vertices of the i-th frame obtained from S901a.

[0652] The decoding end can process the residual of the offset acceleration of vertex P in the height component between the i-th frame and the (i-1)-th frame obtained by S901b (e.g. ), and the reconstructed offset acceleration of vertex P in the height component of the (i-1)th frame obtained by S901a. The values ​​are added together to obtain the reconstructed offset acceleration of vertex P in the height component of the i-th frame.

[0653] The decoding end can obtain the residual of the offset acceleration of vertex P in the radius component between the i-th frame and the (i-1)-th frame from S901b (e.g. ), and the reconstructed offset acceleration of vertex P in the radius component of the (i-1)th frame obtained by S901a. The values ​​are added together to obtain the reconstructed offset acceleration of vertex P in the radius component of the i-th frame.

[0654] The decoding end can process the residual of the offset acceleration of vertex P in the angular component between the i-th frame and the (i-1)-th frame obtained by S901b (e.g. ), and the reconstructed offset acceleration of vertex P in the angular component of the (i-1)th frame obtained by S802b. The values ​​are added together to obtain the reconstructed offset acceleration of vertex P in the angular component of the i-th frame.

[0655] For example, the actual motion information of vertex P in the i-th frame includes the reconstructed offset acceleration. Reconstructing offset acceleration and reconstruction offset acceleration

[0656] S901c: Based on the reconstructed positions of the moving vertices in each frame of the (i-2)th and (i-1)th frames and the instantaneous frame rate of the (i-1)th frame, the decoder calculates the reconstructed offset rate of the moving vertices in the height component, the reconstructed offset rate of the moving vertices in the radius component, and the reconstructed offset rate of the moving vertices in the angle component of the (i-1)th frame; and the decoder can obtain the reconstructed height, reconstructed radius, and reconstructed angle of the moving vertices in the (i-1)th frame through coordinate system transformation.

[0657] The specific implementation principle of S901c at the decoding end is similar to that in the embodiment of Figure 3C, S501 and S502a are used to calculate the reconstruction offset rate of the motion vertex in the (i-1)th frame in the height component and the reconstruction offset rate of the vertex in the (i-1)th frame in the radius component. In this way, the reconstruction offset rates of the motion vertex in the (i-1)th frame in the height component, radius component and angle component can be realized. It will not be elaborated here.

[0658] After S901c and S902a, the decoding end can execute S903.

[0659] S903: The decoding end can determine the reconstructed height, reconstructed radius, and reconstructed angle of the motion vertex in the i-th frame based on the actual motion information of the reconstructed motion vertex in the i-th frame obtained in S902a, the reconstructed height, reconstructed radius, and reconstructed angle of the motion vertex in the (i-1)-th frame after the reconstructed position transformation obtained in S901c, the reconstructed offset rate of the motion vertex in the height component, the reconstructed offset rate of the motion vertex in the radius component, the reconstructed offset rate of the motion vertex in the angle component, and the instantaneous frame rate of at least one of the (i-1)-th and i-th frames.

[0660] Specifically, let's continue with the example of vertex P shown in Figure 5B:

[0661] The actual motion information reconstructed by the decoder for vertex P of the i-th frame includes the reconstructed offset acceleration in the height component. Reconstructed offset acceleration in the radius component and the reconstructed offset acceleration in the angular component

[0662] The reconstructed height, reconstructed radius, and reconstructed angle of vertex P at the position in frame i-1 are h, ...i-1 r i-1 ,θ i-1 ;

[0663] The reconstruction offset rate of vertex P in the height component of the (i-1)th frame is The reconstruction offset rate of vertex P in the (i-1)th frame on the radius component is The reconstruction offset rate of vertex P in the (i-1)th frame on the angular component is

[0664] The time slot for each frame is Δt.

[0665] The following decoder can reconstruct the reconstructed height h of vertex P in the i-th frame. i Reconstruction radius r i Reconstructing angle θ i :

[0666] The decoding end can reconstruct the offset rate (e.g., offset rate) of vertex P in the height component based on the offset from the (i-2)th frame to the (i-1)th frame. The reconstructed offset acceleration of vertex P in the height component of the i-th frame. And the time slot of the i-th frame, to determine the position P of vertex P from the (i-1)-th frame. i-1 Reconstruction position P in frame i i The reconstructed offset height is calculated based on the height component; then, based on this reconstructed offset height and the height component h corresponding to the position of vertex P in the (i-1)th frame,... i-1 To determine the reconstructed height h corresponding to the reconstructed position of vertex P in the i-th frame. i .

[0667] Wherein, the offset rate of vertex P from frame i-2 to frame i-1 in the height component can be the offset rate described above. It could also be the offset rate. It can also be and The average value is equal to and Any value between [a_i] and [a_i]. The decoding end can base its position on vertex P in the (i-2)th frame. i-2 The height component, and the position P of vertex P in the (i-1)th frame. i-1 The height component and the time slot Δt of the (i-1)th frame are used to calculate the reconstruction offset rate of vertex P in the (i-1)th frame on the height component. For example Wherein, the offset height Δh i-1 For position P i-1 Height components and position P i-2 The difference in height components. Offset rate. The calculation process is as follows

[0668] For example, the decoding end can reconstruct the height h of vertex P in the i-th frame using formula 4.1, 4.2, or 4.3. i .

[0669] The decoding end can reconstruct the offset rate (e.g., reconstruction offset rate) based on the radius component of vertex P offset from frame i-2 to frame i-1. The reconstructed offset acceleration of vertex P in the i-th frame on the radius component. And the time slot of the i-th frame, to determine the position P of vertex P from the (i-1)-th frame. i-1 Reconstruction position P in frame i i The offset radius in the radius component; then based on this offset radius and the radius component r corresponding to the position of vertex P in the (i-1)th frame. i-1 To determine the reconstruction radius r corresponding to the reconstruction position of vertex P in the i-th frame. i .

[0670] Wherein, the reconstruction offset rate of vertex P from frame i-2 to frame i-1 in the radius component can be the above-mentioned reconstruction offset rate. It could also be the reconstruction offset rate. It can also be and The average value is equal to and Any value within the range. The decoding end can be based on the position P of vertex P in the (i-2)th frame. i-2 The radius component, and the position P of vertex P in the (i-1)th frame. i-1 The radius component and the time slot Δt of the (i-1)th frame are used to calculate the reconstruction offset rate of vertex P in the (i-1)th frame on the radius component. For example Wherein, the offset radius Δr i-1 For position P i-1 The radius component and position P i-2 The difference between the radius components. Reconstructed offset rate. The calculation process is as follows:

[0671] For example, the decoding end can reconstruct the reconstruction radius r of vertex P in the i-th frame using formula 5.1, 5.2, or 5.3. i .

[0672] The decoding end can reconstruct the offset rate (e.g., reconstruction offset rate) based on the angular component of vertex P offset from frame i-2 to frame i-1. The offset acceleration in the angular component reconstructed from vertex P of the i-th frame. And the i-th frame time slot Δt, to determine the position P of vertex P from the (i-1)-th frame. i-1 Reconstruction position P in frame i i The offset angle in the angular component; then based on this offset angle and the angular component θ corresponding to the position of vertex P in the (i-1)th frame. i-1 To determine the reconstruction angle θ corresponding to the reconstruction position of vertex P in the i-th frame. i .

[0673] Wherein, the offset rate of vertex P from frame i-2 to frame i-1 in terms of the angular component can be the reconstructed offset rate described above. It could also be the reconstruction offset rate. It can also be and The average value is equal to and Any value between [a_i] and [a_i]. The decoding end can base its position on vertex P in the (i-2)th frame. i-2 The angle components, and the position P of vertex P in the (i-1)th frame. i-1 The angle component and the time slot Δt of the (i-1)th frame are used to calculate the reconstruction offset rate of vertex P in the (i-1)th frame on the angle component. For example Wherein, the offset angle Δθ i-1 For position P i-1 Angular components and position P i-2 The difference between the angular components. Offset rate. The calculation process:

[0674] For example, the decoding end can reconstruct the reconstructed angle θ of vertex P in the i-th frame using formula 6.1, 6.2, or 6.3. i .

[0675] S904: The decoding end converts the reconstructed height, reconstructed radius, and reconstructed angle of the motion vertex of the i-th frame into a Cartesian coordinate system to obtain the reconstructed position of the motion vertex of the i-th frame.

[0676] Unlike the method flow shown in Figure 5C, the method flow shown in Figure 5D allows the decoding end to decode the residuals of the offset acceleration of the vertices in the current frame in all three offset directions. This reduces the decoding residuals and improves the decoding rate. When decoding the current frame, the decoding end can use the reconstructed positions of the vertices in the previous two frames (the two frames that were decoded before the current frame and are adjacent to the current frame) to determine the offset rates of the moving vertices in the previous frame in the three offset directions. Then, using the offset rates of the moving vertices in the previous frame in the three offset directions, the reconstructed positions of the vertices in the previous frame, and the actual motion information reconstructed in the current frame, the current frame is decoded. This ensures correct decoding of the 3D mesh even with small residuals and a low bitrate. This can reduce the bitrate of deformable 3D mesh sequences such as fabric or elastic materials.

[0677] Unlike the embodiment in Figure 5D, the application also provides a decoding method that can be implemented based on the process shown in Figure 3G.

[0678] The method flow in this embodiment is largely the same as that in the embodiment shown in Figure 5D. The only difference is that the actual motion information of the current frame reconstructed by S902a, as shown in Figure 5D, is not implemented according to the process shown in Figure 5D. Instead, the encoded data of the current frame (including the actual motion information of the current frame, such as actual motion trend information) is directly decoded to obtain the actual motion information of the current frame reconstructed. Other processes are the same as those in Figure 5D and will not be described again here.

[0679] In this way, when decoding the current frame, it is not necessary to decode the residual, calculate the predicted motion information, or superimpose the predicted motion information and the residual to obtain the reconstruction result of the actual motion information. Instead, the reconstruction result of the actual motion information can be obtained directly by decoding the encoded data, which reduces the decoding complexity and thus reduces the encoding and decoding latency.

[0680] Figure 5E is a flowchart of another method for decoding the current frame (the 3D mesh to be decoded) provided in an embodiment of this application. This method can be implemented based on the architecture shown in Figure 1A, the process shown in Figure 4C, Figure 5A, and Figure 5B, but is not limited to combining the architecture and process shown in Figure 1A, Figure 4C, Figure 5A, and Figure 5B.

[0681] The implementation principle of the method flow shown in Figure 5E is the same as that shown in Figure 5D. The difference is that in the method flow shown in Figure 5D, the residual of the motion information of the current frame and the actual motion information of the reconstructed current frame are the three offset accelerations on the three offset components (h, r, θ) of the cylindrical coordinate system. However, in the method flow shown in Figure 5E, the decoding end does not need to perform coordinate system transformation on the vertex coordinates of the current frame and the historical frame. Instead, it directly reconstructs the actual motion information of the vertices of the current frame in the Cartesian coordinate system and reconstructs the offset rate of the previous frame in the Cartesian coordinate system.

[0682] Specifically, taking vertex P shown in Figure 5B as an example, its coordinates are in a three-dimensional Cartesian coordinate system (including the x, y, and z directions). The actual motion information reconstructed for vertex P in the i-th frame may include the reconstructed offset acceleration of vertex P in the x-direction, the reconstructed offset acceleration of vertex P in the y-direction, and the reconstructed offset acceleration of vertex P in the z-direction of the i-th frame. The predicted motion information for vertex P in the i-th frame may include the reconstructed offset acceleration of vertex P in the x-direction, the reconstructed offset acceleration of vertex P in the y-direction, and the reconstructed offset acceleration of vertex P in the z-direction of the (i-1)-th frame.

[0683] The bitstream received in the method flow shown in Figure 5E can be the bitstream encoded by the method flow shown in Figure 3E.

[0684] As shown in Figure 5E, the method mainly includes the following steps:

[0685] S1101a: Based on the reconstructed position of the motion vertex in each frame of the (i-3), (i-2), and (i-1) frames, and the instantaneous frame rate of the (i-2) and (i-1) frames, the decoding end calculates the reconstructed offset acceleration of the motion vertex in the (i-1) frame in each of the x, y, and z directions to obtain the actual motion information of the reconstructed motion vertex in the (i-1) frame (which can be used as the predicted motion information of the motion vertex in the (i) frame).

[0686] In this context, the reconstructed position of the motion vertex in each of the (i-3), (i-2), and (i-1) frames is the coordinate in the Cartesian coordinate system, and the reconstructed position includes the x-component value, y-component value, and z-component value.

[0687] In this step, the decoding end can first use the reconstructed position of the moving vertex in each of the (i-3), (i-2), and (i-1) frames, as well as the instantaneous frame rate of the (i-2) and (i-1) frames, to determine the reconstruction offset rate of the vertex in each of the x, y, and z directions in the (i-2) and (i-1) directions, respectively. Then, using the reconstruction offset rates of the vertex in each of the x, y, and z directions in the (i-2) and (i-1) directions, as well as the instantaneous frame rate of the (i-1) frame, the decoding end can calculate the reconstruction offset acceleration of the moving vertex in each of the x, y, and z directions in the (i-1) and (i-1) frames.

[0688] Taking vertex P shown in Figure 5B as an example, the decoding end can reconstruct the offset rate of vertex P in each of the x, y, and z directions in the (i-2)th frame (in terms of...). (represented by) the reconstruction offset rate of vertex P in the x, y, and z directions of the (i-1)th frame (represented by) The reconstructed offset acceleration of vertex P in the (i-1)th frame is calculated in each of the x, y, and z directions, using the instantaneous frame rate of the (i-1)th frame as (represented by) the instantaneous frame rate of the (i-1)th frame. (represented) to obtain the actual motion information of the reconstructed vertex P in the (i-1)th frame, which is used as the predicted motion information of the vertex P in the i-th frame.

[0689] The implementation principle of S1101a is similar to that of S702a and S703a in the embodiment of Figure 3E, and will not be repeated here.

[0690] S1001b: The decoding end decodes the received bitstream to obtain the residual between the actual motion information and the predicted motion information of the motion vertex of the i-th frame.

[0691] The residual includes the residuals of the offset acceleration of the motion vertex in the x, y, and z directions between the i-th frame and the (i-1)-th frame.

[0692] Specifically, referring to the data input in S704 of Figure 3E, the residual includes the residual between the actual offset acceleration of the motion vertex in the x direction of the i-th frame and the reconstructed offset acceleration of the motion vertex in the x direction of the (i-1)-th frame, the residual between the actual offset acceleration of the motion vertex in the y direction of the i-th frame and the reconstructed offset acceleration of the motion vertex in the y direction of the (i-1)-th frame, and the residual between the actual offset acceleration of the motion vertex in the z direction of the i-th frame and the reconstructed offset acceleration of the motion vertex in the z direction of the (i-1)-th frame.

[0693] S1102a can be executed after S1101a and S1101b.

[0694] S1102a: The decoding end reconstructs the actual motion information of the vertices of the i-th frame based on the residual obtained from S1101b and the predicted motion information of the vertices of the i-th frame obtained from S1101a.

[0695] Continuing with the example of vertex P shown in Figure 5B, the actual motion information reconstructed for vertex P in the i-th frame can include the reconstructed offset acceleration of vertex P in each of the x, y, and z directions (respectively expressed as follows). express).

[0696] The calculation principle of the offset acceleration of the reconstructed vertex P of the i-th frame in each component in S1102a is similar to the calculation principle of the offset acceleration of the reconstructed vertex P of the i-th frame in the angular direction in the embodiment of Figure 5D, and will not be repeated here.

[0697] S1101c: The decoding end calculates the reconstruction offset rate of the motion vertex in the x direction, the reconstruction offset rate of the motion vertex in the y direction, and the reconstruction offset rate of the motion vertex in the z direction of the i-1 frame based on the reconstruction position of the motion vertex in each frame of the i-2 and i-1 frames and the instantaneous frame rate of the i-1 and i frames.

[0698] The specific implementation principle of S1101c at the decoding end is similar to that of S901c in the embodiment shown in Figure 5D, and will not be repeated here.

[0699] After S1101c and S1102a, the decoding end can execute S1103.

[0700] S1103: The decoding end can reconstruct the position of the vertex in the i-th frame based on the actual motion information of the reconstructed vertex of the i-th frame obtained in S1102a, the reconstruction offset rate of the moving vertex in the x, y, z directions of the i-1-th frame obtained in S1101c, the instantaneous frame rate of the i-th frame and the reconstruction position of the moving vertex of the i-1-th frame.

[0701] The specific implementation principle of S1103 at the decoding end is similar to that of S903 in the embodiment shown in Figure 5D, and will not be repeated here.

[0702] Unlike the method flows shown in Figures 5C and 5D, in the method flow shown in Figure 5E, the decoding end does not need to perform coordinate system transformation on the positions of vertices in each frame. It can directly decode the vertices of the 3D mesh established in the 3D Cartesian coordinate system to decode the residuals of the offset acceleration of the vertex in the three offset directions in the 3D Cartesian coordinate system, thereby decoding the current frame and improving the decoding rate. In this way, when the vertices in the 3D mesh undergo non-uniform motion with small changes in motion direction in the 3D mesh sequence (e.g., a virtual character running at high speed causes virtual clothing to move at a non-uniform speed), the acceleration of the vertices in the 3D mesh in the Cartesian coordinate system can change little. Then, the decoding end calculates the predicted motion information of the vertex in each offset direction in the Cartesian coordinate system and the reconstructed actual offset motion information (including offset acceleration) so that the offset direction of the offset acceleration can match the motion direction and motion pattern of the non-uniform motion, thereby improving the closeness between the predicted motion information and the actual motion information, reducing residuals, reducing the amount of decoded data, and improving transmission and storage efficiency.

[0703] Unlike the embodiment in Figure 5E, the application also provides a decoding method, which can be implemented based on the process shown in Figure 3G.

[0704] The method flow in this embodiment is largely the same as that in the embodiment shown in Figure 5E. The only difference is that the actual motion information of the current frame reconstructed through S1102a, as shown in Figure 5E, is not implemented according to the process shown in Figure 5E. Instead, the encoded data of the current frame (including the actual motion information of the current frame, such as actual motion trend information) is directly decoded to obtain the actual motion information of the current frame reconstructed. Other processes are the same as those in Figure 5E and will not be described again here.

[0705] In this way, when decoding the current frame, it is not necessary to decode the residual, calculate the predicted motion information, or superimpose the predicted motion information and the residual to obtain the reconstruction result of the actual motion information. Instead, the reconstruction result of the actual motion information can be obtained directly by decoding the encoded data, which reduces the decoding complexity and thus reduces the encoding and decoding latency.

[0706] It should be understood that Figures 5C, 5D, and 5E above are merely illustrative examples of three methods for decoding the current frame by the decoding end of this application. In other embodiments, this application may also provide more decoding methods. The bitstream decoded by the decoding method is still the residual between the actual motion information and the predicted motion information of the vertices of the current frame. The motion information (actual motion information and predicted motion information) is the offset motion information in each of the three offset directions of the three-dimensional coordinate system of the current frame. Among the offset motion information in each of the three offset directions, there is an offset acceleration in at least one offset direction. The offset motion information in the other offset directions can be offset rate or offset acceleration.

[0707] Figure 6 is a schematic diagram of the structure of the encoding device 1000 of this application. As shown in Figure 6, the encoding device 1000 of this embodiment can be applied to the encoding end mentioned above. The encoding device 1000 may include: an acquisition module 1001 and an encoding module 1002. The acquisition module 1001 is used to acquire the actual motion trend information of the first three-dimensional grid to be encoded in the three-dimensional grid sequence. The actual motion trend information indicates the actual motion trend of the non-uniformly moving vertices in the three-dimensional grid sequence in the corresponding time slot of the first three-dimensional grid. The encoding module 1002 is used to acquire the encoded data of the first three-dimensional grid based on the actual motion trend information; and to encode the encoded data into the bitstream corresponding to the three-dimensional grid sequence.

[0708] In one possible implementation, the acquisition module 1001 is further configured to acquire the predicted motion trend information of the first three-dimensional mesh, the predicted motion trend information indicating the predicted motion trend of the motion vertex in the corresponding time slot of the first three-dimensional mesh; the encoding module 1002 is further configured to obtain the encoded data based on the actual motion trend information and the predicted motion trend information.

[0709] In one possible implementation, the encoded data includes the residual between the actual motion trend information and the predicted motion trend information.

[0710] In one possible implementation, the encoded data includes the actual motion trend information.

[0711] In one possible implementation, the bitstream includes a first syntax structure corresponding to the first three-dimensional grid, the first syntax structure including an identifier of the first three-dimensional grid.

[0712] In one possible implementation, the first syntax structure further includes information corresponding to the motion vertex, the motion vertex information including: a first residual of the motion vertex on a first component; a second residual of the motion vertex on a second component; and a third residual of the motion vertex on a third component.

[0713] In one possible implementation, the first syntax structure further includes: the vertex identifier of the moving vertex.

[0714] In one possible implementation, the acquisition module 1001 is specifically used to acquire N sets of reconstructed data corresponding to N encoded second three-dimensional grids in the three-dimensional grid sequence, where N is an integer greater than or equal to a set value; and to determine the predicted motion trend information based on the N sets of reconstructed data.

[0715] In one possible implementation, the acquisition module 1001 is specifically used to determine the actual motion information of the first three-dimensional mesh based on the original data of the first three-dimensional mesh and the original data of a plurality of encoded third three-dimensional meshes in the three-dimensional mesh sequence.

[0716] In one possible implementation, the acquisition module 1001 is specifically used to acquire N sets of encoded data corresponding to the N second three-dimensional grids; acquire N sets of predicted motion trend information for the N second three-dimensional grids, wherein the N sets of predicted motion trend information are the N predicted motion trends of the moving vertex in the N time slots corresponding to the N second three-dimensional grids; and decode the N sets of encoded data based on the N predicted motion trend information to obtain the N sets of reconstructed data. Here, there is a one-to-one correspondence between the N second three-dimensional grids and the N sets of predicted motion information.

[0717] In one possible implementation, the acquisition module 1001 is specifically used to decode the N sets of encoded data to obtain N sets of residuals between the N sets of actual motion trend information and the N sets of predicted motion trend information of the N second three-dimensional grids. The N sets of actual motion trend information include the N sets of actual motion trend information of the moving vertex in the N time slots corresponding to the N sets of second three-dimensional grids. Based on the N sets of predicted motion trend information and the N sets of residuals, the N sets of actual motion trend information of the N second three-dimensional grids are determined (essentially the N sets of reconstructed actual motion trend information, also expressed as N sets of reconstructed motion information). The N sets of actual motion trend information are the N sets of reconstructed information of the N actual motion trends of the moving vertex in the N time slots corresponding to the N second three-dimensional grids. Based on the N sets of actual motion trend information and the reconstructed data of the S sets of fourth three-dimensional grids corresponding to each of the N second three-dimensional grids, the N sets of reconstructed data are determined. The S sets of fourth three-dimensional grids are the decoded three-dimensional grids in the three-dimensional grid sequence, and S is an integer greater than or equal to 2.

[0718] In one possible implementation, at least one of the first residual, the second residual, and the third residual is a residual of motion acceleration.

[0719] In one possible implementation, the first component is the X component, the second component is the Y component, and the third component is the Z component.

[0720] In one possible implementation, the first component is an angle component, the second component is a height component, and the third component is a radius component.

[0721] In one possible implementation, the first three-dimensional mesh, the N second three-dimensional meshes, and the plurality of third three-dimensional meshes have the same topology.

[0722] In one possible implementation, the N second three-dimensional meshes are displayed in the three-dimensional mesh sequence before the first three-dimensional meshes are displayed in the three-dimensional mesh sequence.

[0723] In one possible implementation, the plurality of third 3D meshes are displayed in the 3D mesh sequence before the first 3D meshes are displayed in the 3D mesh sequence.

[0724] The encoding device 1000 in this embodiment can be used to execute the technical solutions of the above-described encoding method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0725] Figure 7 is a schematic diagram of the structure of the decoding device 2000 of this application. As shown in Figure 7, the decoding device 2000 of this embodiment can be applied to the decoding end mentioned above. The decoding device 2000 may include: an acquisition module 2001 and a decoding module 2002. The acquisition module 2001 is used to acquire the bitstream corresponding to the three-dimensional mesh sequence; the decoding module 2002 is used to acquire the encoded data of the first three-dimensional mesh to be decoded in the three-dimensional mesh sequence from the bitstream; based on the encoded data, acquire the actual motion trend information of the first three-dimensional mesh, the actual motion trend information indicating the actual motion trend of the non-uniform motion vertex in the three-dimensional mesh sequence in the time slot corresponding to the first three-dimensional mesh; based on the actual motion trend information, obtain the reconstructed data of the first three-dimensional mesh.

[0726] In one possible implementation, the acquisition module 2001 is specifically used to acquire the predicted motion trend information of the first three-dimensional mesh, wherein the predicted motion trend information indicates the predicted motion trend of the moving vertex in the corresponding time slot of the first three-dimensional mesh; and to obtain the actual motion trend information based on the encoded data and the predicted motion trend information.

[0727] In one possible implementation, the encoded data includes the residual between the actual motion trend information and the predicted motion trend information.

[0728] In one possible implementation, the encoded data includes the actual motion trend information.

[0729] In one possible implementation, the encoded data has a first syntactic structure, which includes an identifier of a first three-dimensional mesh.

[0730] In one possible implementation, the first syntax structure further includes information corresponding to the motion vertex, the motion vertex information including: a first residual of the motion vertex on a first component; a second residual of the motion vertex on a second component; and a third residual of the motion vertex on a third component.

[0731] In one possible implementation, the first syntax structure further includes: the vertex identifier of the moving vertex.

[0732] In one possible implementation, the decoding module 2002 is further configured to acquire, based on the bitstream, N sets of reconstructed data corresponding to the N decoded second three-dimensional grids in the three-dimensional grid sequence, where N is an integer greater than or equal to a set value; and determine the predicted motion trend information based on the N sets of reconstructed data.

[0733] In one possible implementation, the decoding module 2002 is specifically used to acquire, based on the bitstream, reconstruction data of multiple decoded third three-dimensional grids corresponding to the three-dimensional grid sequence; and to obtain reconstruction data of the first three-dimensional grid based on the actual motion trend information and the reconstruction data of the multiple third three-dimensional grids.

[0734] In one possible implementation, at least one of the first residual, the second residual, and the third residual is a residual of motion acceleration.

[0735] In one possible implementation, the first component is the X component, the second component is the Y component, and the third component is the Z component.

[0736] In one possible implementation, the first component is an angle component, the second component is a height component, and the third component is a radius component.

[0737] In one possible implementation, the first three-dimensional mesh, the N second three-dimensional meshes, and the plurality of third three-dimensional meshes have the same topology.

[0738] The decoding device 2000 in this embodiment can be used to execute the technical solutions of the above-described decoding method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0739] This application also provides an encoding / decoding system, which may include the encoding device 1000 and the decoding device 2000 described above. This application does not limit the communication method or connection method between the encoding device 1000 and the decoding device 2000.

[0740] Figure 8 is a schematic structural diagram of the device 1200 provided in this application. The device 1200 may include: a processor 1201, and optionally, a transceiver circuit 1202. Optionally, it may also include a memory 1203.

[0741] The various components of device 1200 are coupled together via bus 1204, which includes not only a data bus but also a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 1204 in the figure.

[0742] Optionally, the memory 1203 can be used to store the instructions in the above method embodiments.

[0743] The processor 1201 can be used to execute instructions in the memory 1203, control the transceiver circuit 1202 to receive signals, and control the transceiver circuit 1202 to send signals.

[0744] The device 1200 may be an electronic device at the encoding / decoding end in the above method embodiments, or a chip or server in an electronic device.

[0745] In implementation, each step of the above method embodiments can be completed by the integrated logic circuit of the hardware in the processor 1201 or by instructions in the form of software. The processor 1201 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware encoding processor, or implemented by a combination of hardware and software modules in the encoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads the information in the memory and combines it with its hardware to complete the steps of the above method.

[0746] The memory mentioned in the above embodiments can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0747] In one possible implementation, this application provides a computer-readable storage medium storing program instructions that, when executed by a processor, cause the processor to perform the encoding or decoding method as described in any of the above embodiments.

[0748] In one possible implementation, this application provides a computer program product comprising computer program code that, when executed on a processor, causes the processor to perform the encoding or decoding method described in any of the above embodiments.

[0749] In one possible implementation, this application provides a chip including a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the encoding or decoding method as described in any of the above embodiments.

[0750] In one possible implementation, this application provides a bitstream corresponding to a three-dimensional mesh sequence. The bitstream includes encoded data of a first three-dimensional mesh. The encoded data is encoded data obtained based on the actual motion trend information of the first three-dimensional mesh. The actual motion trend information indicates the actual motion trend of a motion vertex in the three-dimensional mesh sequence that is undergoing non-uniform motion in the corresponding time slot of the first three-dimensional mesh.

[0751] In one possible implementation, the encoded data includes the residual between the actual motion trend information and the predicted motion trend information of the first three-dimensional mesh, the predicted motion trend information indicating the predicted motion trend of the motion vertex in the corresponding time slot of the first three-dimensional mesh.

[0752] In one possible implementation, the encoded data includes the actual motion trend information.

[0753] In one possible implementation, the bitstream includes a first syntax structure corresponding to the first three-dimensional grid, the first syntax structure including an identifier of the first three-dimensional grid.

[0754] In one possible implementation, the first syntax structure further includes information about the motion vertex, which includes: a first residual of the motion vertex on a first component; a second residual of the motion vertex on a second component; and a third residual of the motion vertex on a third component.

[0755] In one possible implementation, the information of the moving vertex further includes: the vertex identifier of the moving vertex.

[0756] In one possible implementation, at least one of the first residual, the second residual, and the third residual is a residual of motion acceleration.

[0757] In one possible implementation, the first component is the X component, the second component is the Y component, and the third component is the Z component.

[0758] In one possible implementation, the first component is an angle component, the second component is a height component, and the third component is a radius component.

[0759] In one possible implementation, this application provides a computer-readable storage medium storing a bitstream as described in any of the possible embodiments above.

[0760] In one possible implementation, this application provides a method for transmitting an encoded bitstream of video data, the method comprising: acquiring the bitstream from a storage medium, the bitstream being the bitstream described in any of the possible embodiments above and stored in the storage medium; and transmitting the bitstream.

[0761] In one possible implementation, this application provides a system for transmitting an encoded bitstream of video data, the system comprising: an acquisition unit for acquiring a bitstream from a storage medium, the bitstream being the bitstream described in any of the possible embodiments above and the bitstream being stored in the storage medium; and a transmission unit for transmitting the bitstream.

[0762] In one possible implementation, this application provides a method for storing an encoded bitstream of video data, the method comprising: receiving the bitstream as described in any of the possible implementations above; and storing the bitstream in a storage medium.

[0763] In one possible implementation, this application provides a system for storing an encoded bitstream of video data, comprising: a receiving unit for receiving the bitstream as described in any of the possible implementations above; and a storage unit for storing the bitstream.

[0764] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

An encoding method, characterized in that, The method includes: Obtain the actual motion trend information of the first three-dimensional grid to be encoded in the three-dimensional grid sequence. The actual motion trend information indicates the actual motion trend of the motion vertex that is moving at a non-uniform speed in the three-dimensional grid sequence in the corresponding time slot of the first three-dimensional grid. Based on the actual motion trend information, the encoded data of the first three-dimensional mesh is obtained; The encoded data is encoded into the bitstream corresponding to the three-dimensional mesh sequence. The method according to claim 1, characterized in that, The step of obtaining the encoded data of the first three-dimensional mesh based on the actual motion trend information includes: Obtain the predicted motion trend information of the first three-dimensional mesh, wherein the predicted motion trend information indicates the predicted motion trend of the moving vertex in the corresponding time slot of the first three-dimensional mesh; The encoded data is obtained based on the actual motion trend information and the predicted motion trend information. The method according to claim 2, characterized in that, The encoded data includes the residual between the actual motion trend information and the predicted motion trend information. The method according to claim 1, characterized in that, The encoded data includes the actual movement trend information. The method according to any one of claims 1 to 4, characterized in that, The bitstream includes a first syntax structure corresponding to the first three-dimensional grid, and the first syntax structure includes an identifier of the first three-dimensional grid. The method according to claim 5, characterized in that, The first grammatical structure also includes information about the motion vertices, which includes: The first residual of the motion vertex on the first component; The second residual of the motion vertex on the second component; The third residual of the motion vertex on the third component. The method according to claim 6, characterized in that, The information of the motion vertex also includes: The vertex identifier of the moving vertex. The method according to claim 2, characterized in that, The step of obtaining the predicted motion trend information of the first three-dimensional mesh includes: Obtain N sets of reconstructed data corresponding to the N encoded second three-dimensional grids in the three-dimensional grid sequence, where N is an integer greater than or equal to a set value; Based on the N sets of reconstructed data, the predicted motion trend information is determined. The method according to any one of claims 1 to 8, characterized in that, The step of obtaining the actual motion trend information of the first three-dimensional mesh to be encoded in the three-dimensional mesh sequence includes: Based on the original data of the first three-dimensional mesh and the original data of multiple encoded third three-dimensional meshes in the three-dimensional mesh sequence, the actual motion information of the first three-dimensional mesh is determined. The method according to claim 8 or 9, characterized in that, The step of obtaining N sets of reconstructed data corresponding to the N encoded second three-dimensional meshes in the three-dimensional mesh sequence includes: Obtain N sets of encoded data corresponding to the N second three-dimensional grids; Obtain N sets of predicted motion trend information for the N second three-dimensional grids, wherein the N sets of predicted motion trend information are the N predicted motion trends of the motion vertex in the N time slots corresponding to the N second three-dimensional grids; Based on the N predicted motion trend information, the N sets of encoded data are decoded to obtain the N sets of reconstructed data. The method according to claim 6, characterized in that, At least one of the first residual, the second residual, and the third residual is a residual of motion acceleration. A decoding method, characterized in that, The method includes: Obtain the bitstream corresponding to the 3D mesh sequence; Obtain the encoded data of the first three-dimensional grid to be decoded from the three-dimensional grid sequence in the bitstream; Based on the encoded data, the actual motion trend information of the first three-dimensional grid is obtained. The actual motion trend information indicates the actual motion trend of the non-uniform motion vertex in the three-dimensional grid sequence in the corresponding time slot of the first three-dimensional grid. Based on the actual motion trend information, the reconstruction data of the first three-dimensional mesh is obtained. The method according to claim 12, characterized in that, The step of obtaining the actual motion trend information of the first three-dimensional mesh based on the encoded data includes: Obtain the predicted motion trend information of the first three-dimensional mesh, wherein the predicted motion trend information indicates the predicted motion trend of the moving vertex in the corresponding time slot of the first three-dimensional mesh; Based on the encoded data and the predicted motion trend information, the actual motion trend information is obtained. The method according to claim 13, characterized in that, The encoded data includes the residual between the actual motion trend information and the predicted motion trend information. The method according to claim 12, characterized in that, The encoded data includes the actual movement trend information. The method according to any one of claims 12 to 15, characterized in that, The bitstream includes a first syntax structure corresponding to the first three-dimensional grid, and the first syntax structure includes an identifier of the first three-dimensional grid. The method according to claim 16, characterized in that, The first grammatical structure also includes information about the motion vertices, which includes: The first residual of the motion vertex on the first component; The second residual of the motion vertex on the second component; The third residual of the motion vertex on the third component. The method according to claim 17, characterized in that, The information of the motion vertex also includes: The vertex identifier of the moving vertex. The method according to claim 13, characterized in that, The step of obtaining the predicted motion trend information of the first three-dimensional mesh includes: Based on the bitstream, N sets of reconstructed data corresponding to the N decoded second three-dimensional grids in the three-dimensional grid sequence are obtained, where N is an integer greater than or equal to a set value; Based on the N sets of reconstructed data, the predicted motion trend information is determined. The method according to any one of claims 12 to 19, characterized in that, The process of obtaining the reconstructed data of the first three-dimensional mesh based on the actual motion trend information includes: Based on the bitstream, reconstructed data corresponding to multiple decoded third-dimensional grids in the three-dimensional grid sequence are obtained; Based on the actual motion trend information and the reconstruction data of the multiple third-dimensional meshes, the reconstruction data of the first three-dimensional mesh is obtained. The method according to claim 17, characterized in that, At least one of the first residual, the second residual, and the third residual is a residual of motion acceleration. An encoding / decoding system, characterized in that, include: Encoding device and decoding device; The encoding device is used to implement the method as described in any one of claims 1 to 11; The decoding device is used to implement the method as described in any one of claims 12 to 21. An encoding device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the encoding device implements the method as described in any one of claims 1 to 11. A decoding device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the decoding device implements the method as described in any one of claims 12 to 21. A computer-readable storage medium, characterized in that, Includes a computer program, which, when executed on a computer, causes the computer to perform the method of any one of claims 1 to 11, or the method of any one of claims 12 to 21. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method of any one of claims 1 to 11, or the method of any one of claims 12 to 21.

Images