Animation stream coding method, animation stream decoding method, apparatus, storage medium, and chip

Abstract

The present disclosure relates to an animation stream coding method, an animation stream decoding method, an apparatus, a storage medium, and a chip. The decoding method comprises: receiving an animation stream sequence; reading header information; reading body information; and decoding on the basis of the header information and the body information to obtain animation data. The coding method and the decoding method provided in the present disclosure can separate out shared parameters in an animation stream sequence so as to send same in a more reliable manner, and improve the precision of coding and decoding by means of scaling up the parameters. In addition, different prediction modes are also introduced, thus providing free prediction mode selection.

Description

Methods, devices, storage media, and chips for encoding and decoding animation streams

[0001] This disclosure claims priority to Chinese Patent Application No. 202510053355.5, filed with the Chinese Patent Office on January 13, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to an encoding and decoding method, and more specifically, to an encoding and decoding method, apparatus, storage medium, and chip for an animation stream. Background Technology

[0003] Against the backdrop of advancements in communication technology and continuous upgrades in hardware, the demand for immersive multimedia content such as 3D scenes, virtual reality (VR), and augmented reality (AR) is increasing. Digital avatars, as virtual humanoid representations of real users in these scenarios, play a bridging role between real users and the virtual world within the future metaverse system. Currently, the digital avatar models provided by major rendering engine platforms and 3D production software all employ their own proprietary file encapsulation formats and animation-driven mechanisms. At the MPEG conference in November 2024, MPEG-I proposed a draft and basic structure for the animation bitstream format of digital avatars. However, a unified industry standard has not yet been formed regarding the encoding format, bitstream structure, and streaming protocol for animation data. Furthermore, the bitstream structure and animation compression encoding schemes of related technologies still have functional deficiencies and areas for improvement. For example, the encoding scheme does not provide lossless encoding options, nor does it configure parameter sets including parameters shared by different frames in the animation sequence, and it does not utilize the intra-frame correlation between different animation parameters within the same frame. The aforementioned problems severely restrict the interoperability and seamless integration of digital human models across different platforms, thereby hindering the widespread application of digital human characters in diverse virtual environments. Summary of the Invention

[0004] This disclosure provides an encoding and decoding method, apparatus, storage medium, and chip for animation streams, which provides a unified encoding format and bitstream structure for animation streams, enhancing the interoperability and integrability of digital human models across different platforms and strengthening the broad interconnectivity of digital human characters in diverse virtual environments.

[0005] According to a first aspect of this disclosure, a method for decoding an animation stream is provided. The method includes: receiving the animation stream sequence; reading header information; reading volume information; and decoding animation data based on the header information and the volume information.

[0006] According to a second aspect of this disclosure, an encoding method for an animation stream is provided, characterized in that the method includes: receiving an animation stream; setting header information; setting body information; and generating encoded data of the animation stream based on the header information and the body information.

[0007] According to a third aspect of this disclosure, a hardware device is provided. The hardware device includes: a memory for storing program instructions; and a processor for invoking the program instructions to execute any of the above-described decoding or encoding methods.

[0008] According to a fourth aspect of this disclosure, a non-volatile storage medium is provided, comprising computer program instructions, wherein when the computer program instructions are invoked by a processor, the processor performs any of the above-described decoding or encoding methods.

[0009] According to a fifth aspect of this disclosure, a chip is provided, including a processor and a memory; the memory stores program instructions, and the processor is configured to invoke the program instructions to execute any of the above-described decoding or encoding methods.

[0010] In some embodiments, when encoding and decoding an animation stream, the encoding and decoding methods provided in this disclosure can encode and decode animation parameters according to different prediction modes and control parameters. These encoding and decoding methods not only extend the original prediction modes by adding intra-frame prediction modes and enabling free selection of different prediction modes, but also extract the common parameters of the entire animation stream sequence to form a sequence parameter set, avoiding data redundancy caused by the loss of header information in a certain frame and the need to transmit control parameters for each frame. Furthermore, the encoding and decoding methods provided in this disclosure also introduce scale variation parameters, thereby providing a lossless encoding option and avoiding data loss during encoding and decoding due to quantization. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments of this disclosure will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 is a block diagram illustrating exemplary application scenarios according to some embodiments of the present disclosure;

[0013] Figure 2A illustrates the frame structure of the sequence parameter set according to some embodiments of the present disclosure;

[0014] Figure 2B illustrates the frame structure of an I-frame according to some embodiments of the present disclosure;

[0015] Figure 2C illustrates the frame structure of a P-frame according to some embodiments of the present disclosure;

[0016] Figure 3 is a flowchart illustrating a BS facial animation stream encoding method according to some embodiments of the present disclosure;

[0017] Figure 4 is a flowchart illustrating a BS facial animation stream decoding method according to some embodiments of the present disclosure;

[0018] Figure 5 is an exemplary block diagram illustrating an encoding system according to some embodiments of the present disclosure;

[0019] Figure 6 is an exemplary block diagram illustrating a decoding system according to some embodiments of the present disclosure;

[0020] Figure 7 is a schematic diagram illustrating an exemplary structure of a computer device according to some embodiments of the present disclosure. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the implementation methods of this disclosure will be further described in detail below with reference to the accompanying drawings.

[0022] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0023] The following explanations of some terms used in the embodiments of this disclosure are provided to facilitate understanding by those skilled in the art.

[0024] (1) In the embodiments of this disclosure, the terms “apparatus”, “device” and “system” are often used interchangeably, but their meanings can be understood by those skilled in the art.

[0025] (2) In the embodiments of this disclosure, the term “multiple” refers to two or more, and other quantifiers are similar.

[0026] (3) "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A alone, A and B at the same time, and B alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0027] FIG. 1 is a block diagram of an exemplary application scenario 100 according to some embodiments of the present disclosure. As shown in FIG. 1, the application scenario 100 may include a sending terminal 101. Among them, the sending terminal 101 is used to capture an animation action, encode the animation action, and send it through a network. In some embodiments, the sending terminal 101 may be and / or include a VR device, a MR device, an AR device, a mobile phone, a camera, a tablet computer, or other devices that can be used to capture an animation action. In some embodiments, the network includes but is not limited to a wired network, a wireless network (such as a cellular network, a WIFI (Wireless Fidelity) network, etc.).

[0028] In some embodiments, the application scenario 100 may further include a cloud server 102. Among them, the cloud server 102 can receive the animation stream sent from the sending terminal 101 and forward the animation stream to other devices. In some embodiments, the cloud server 102 can also perform preprocessing of graphics processing (such as animation processing), generate data of intermediate results (such as intermediate results of game screens, live broadcast screens, conference screens, or concert screens, etc.); then, encode the data of the intermediate results to obtain a corresponding code stream (such as an animation stream of a digital human, etc.). In some embodiments, the cloud server 102 can also store other information of the digital human, such as information of the static model of the digital human. The cloud server 102 can send the animation stream to one or more receiving terminals.

[0029] In some embodiments, the application scenario 100 may further include a receiving terminal 103. The receiving terminal 103 may be and / or include a VR device, a MR device, an AR device, a mobile phone, a camera, a tablet computer, or other devices that can be used to display the digital human. In some embodiments, the receiving terminal can receive the animation stream from the cloud server 102. The receiving terminal can also obtain the basic format of the digital human from the cloud server 102 or a local storage device. The basic format of the digital human can be used to provide the static model of the digital human. The receiving terminal 103 can perform animation processing on the digital human based on the animation stream and the static model of the digital human, and render and display the processed digital human data.

[0030] In some embodiments, the application scenario can be an AR or VR meeting using AR or VR glasses. The sending terminal 101 can capture the movements of the participants, animate the movements to generate an animation stream, and upload it to the cloud server 102. The cloud server 102 can forward the animation stream to the receiving terminal 103. After receiving the animation stream, the receiving terminal 103 can decode, animate, and render it in conjunction with the basic digital human information received from the local storage device or the cloud server. Then, the receiving terminal can obtain and display the digital human's meeting screen. In some embodiments, rendering processing can refer to the process of displaying a three-dimensional model in a three-dimensional scene onto a two-dimensional plane. The rendering processing algorithm can include various methods, such as Dynamic Diffuse Global Illumination (DDGI) algorithm.

[0031] In some embodiments, the static model of the digital human may include an animated portion. The techniques disclosed herein are primarily used for encoding and decoding the animated portion of a three-dimensional digital human static model. The animated portion may include at least one of facial animation and body animation. In some embodiments, the animation methods used to represent the animated portion include Blendshape-based and joint rotation-based animation methods, controller-based animation methods, and any other suitable methods that can be used to implement the animation.

[0032] In some embodiments, the Blendshape-based and joint rotation-based animation method includes at least one of the Blendshape-based facial animation method and the joint rotation-based body animation method. The Blendshape-based facial animation method is a technique for animatering virtual characters. The Blendshape-based facial animation method is generated based on the Facial Action Coding System (FACS). The Facial Action Coding System defines a series of Action Units (AUs), each corresponding to a basic movement of a single muscle or muscle group and associated with a specific facial movement. The Blendshape-based facial animation method includes multiple Blendshapes (referred to as "BS expressions"). Each BS expression represents a specific facial expression, such as blinking the left eye (EyeBlinkLeft), opening the mouth (JawOpen), pouting to the left (MouthLeft), raising the eyebrows inward (BrowInnerUp), puffing the cheeks outward (CheekPuff), sticking out the tongue (TongueOut), and other facial expressions. Each BS expression can be used to simulate the corresponding facial expression movement.

[0033] In some embodiments, the number of BS emojis can be predefined by the creator; the more BS emojis, the more detailed the expressions. Currently, a typical BS emoji set widely used generally includes more than fifty shapes. For example, Apple's ARKit framework contains 52 BS emojis. The BS emoji set in the ARKit framework is widely used on multiple open-source platforms. Therefore, in order to better promote interoperability between digital human formats, the BS emoji set in this disclosure defaults to including 52 BS emojis. In some embodiments, the number of BS emojis included in the BS emoji set in this disclosure can also be predefined by the creator or user.

[0034] Figure 2A is a frame structure of a sequence parameter set including parameters common to all frames in an animation stream sequence according to some embodiments of the present disclosure. Figure 2B is a frame structure of an I-frame in an animation stream sequence according to some embodiments of the present disclosure. Figure 2C is a frame structure of a P-frame in an animation stream sequence according to some embodiments of the present disclosure.

[0035] In some embodiments, the BS facial animation data stream is used to encode animation parameters for a digital human face. An animation stream sequence often contains multiple data frames, and these multiple data frames share some common parameters, which together form the sequence parameter set.

[0036] As shown in Figure 2A, the sequence parameter set includes a first sequence parameter set ID. The first sequence parameter set ID is a label used to indicate the current sequence parameter set. The first sequence parameter set ID can be used to determine which sequence parameter set to use when processing a specific frame in the animation stream sequence. In some embodiments, the first sequence parameter set ID can be a 4-bit unsigned integer. The first sequence parameter set ID can occupy 4 bits.

[0037] In some embodiments, the sequence parameter set further includes a scaling parameter. This scaling parameter can be used to amplify the original value while preserving precision, thereby achieving lossless encoding. In some embodiments, when encoding animation frames, the original values ​​are decimal values ​​between 0 and 1. During encoding, quantization of decimal values ​​causes precision loss; therefore, lossless encoding cannot use quantizers, while entropy encoders support integer data input. Therefore, the scaling parameter multiplies the original value by an integer power of 10, thereby amplifying the decimal data while preserving precision. In some embodiments, the scaling parameter can be a 4-bit unsigned integer. The scaling parameter can occupy 4 bits.

[0038] In some embodiments, the sequence parameter set further includes an offset identifier. The offset identifier can be used to indicate whether the sequence parameter set includes an offset value information table. The offset value information table includes a mapping table between BS expressions and their offset values. In some embodiments, when the offset identifier is 1, the offset identifier indicates that the sequence parameter set includes an offset value information table; when the offset identifier is 0, the offset identifier indicates that the sequence parameter set does not include an offset value information table. In other embodiments, when the offset identifier is 0, the offset identifier indicates that the sequence parameter set includes an offset value information table; when the offset identifier is 1, the offset identifier indicates that the sequence parameter set does not include an offset value information table. The offset identifier can be a 1-bit unsigned integer. The offset identifier can occupy 1 bit. In some embodiments, when the sequence parameter set includes the offset value information table, any one of info1, info2, or info3 in the sequence parameter set frame structure can be the offset value information table.

[0039] In some embodiments, when entropy encoding uses the exponential Golomb encoding method, the input value to be encoded is required to be an integer value, and the smaller the absolute value, the higher the efficiency. Therefore, the closer the peak value of the probability distribution of the value to be encoded is to 0, the more suitable exponential Golomb encoding is. Thus, when encoding a specific frame of an animation stream sequence, by subtracting the peak probability value of the corresponding BS expression from the value to be encoded after quantization, making the absolute value of the value to be encoded close to 0, encoding efficiency can be improved. The offset value information table includes the mapping relationship between the BS expression and the offset value corresponding to the BS expression. The offset value corresponding to the BS expression can be the peak probability value of the encoding parameter corresponding to the BS expression.

[0040] In some embodiments, the sequence parameter set further includes a reference BS information table identifier. The reference BS information table identifier indicates whether the sequence parameter set includes a reference BS information table. The reference BS information table includes a mapping table of BS expressions and BS reference expressions. In some embodiments, when the reference BS information table identifier is 1, the reference BS information table identifier indicates that the sequence parameter set includes a reference BS information table; when the reference BS information table identifier is 0, the reference BS information table identifier indicates that the sequence parameter set does not include a reference BS information table. In some embodiments, when the reference BS information table identifier is 0, the reference BS information table identifier indicates that the sequence parameter set includes a reference BS information table; when the reference BS information table identifier is 1, the reference BS information table identifier indicates that the sequence parameter set does not include a reference BS information table. The reference BS information table identifier can be a 1-bit unsigned integer. The reference BS information table identifier can occupy 1 bit. In some embodiments, when the sequence parameter set includes the reference BS information table, any one of info1, info2, or info3 in the sequence parameter set frame structure can be the reference BS information table.

[0041] In some embodiments, a single frame in an animation stream sequence includes multiple Baseline Expressions (BS), each representing the animation of a different part of the digital human. When different BS expressions represent the same semantic animation, they exhibit a certain correlation. For example, consider the facial BS expressions "left eye wide open" and "right eye wide open." When the left eye widens, the right eye often makes the same movement, thus showing a strong correlation between the two BS expressions. Therefore, the movement of the right eye can be predicted by the movement of the left eye. Similar correlations exist for other BS expressions such as "left corner of the mouth" and "right corner of the mouth," "left eyebrow" and "right eyebrow." The reference BS information table includes a pre-defined mapping relationship between BS expressions and BS reference expressions. For example, when the BS expression has the weight value of "left eye wide open," the corresponding BS reference expression has the weight value of "right eye wide open."

[0042] In some embodiments, the sequence parameter set further includes a head pose identifier. The head pose identifier indicates whether head pose parameters are enabled in the animation stream sequence. In some embodiments, the head pose identifier can be a 1-bit Boolean value. In some embodiments, when the head pose identifier is 1, the head pose identifier indicates that the volume information of each frame in the animation stream sequence includes head pose parameters; when the head pose identifier is 0, the head pose identifier indicates that the volume information of each frame in the animation stream sequence does not include head pose parameters. In some embodiments, when the head pose identifier is 0, the head pose identifier indicates that the volume information of each frame in the animation stream sequence includes head pose parameters; when the head pose identifier is 1, the head pose identifier indicates that the volume information of each frame in the animation stream sequence does not include head pose parameters.

[0043] In some embodiments, the sequence parameter set further includes BS emoji set quantity information. This BS emoji set quantity information can be used to indicate the number of BS emojis included in the BS emoji set used by the animation stream sequence. In some embodiments, the BS emoji set quantity information can be a user-predefined quantity. In some embodiments, the BS emoji set quantity information can be a 16-bit unsigned integer. The BS emoji set quantity information can occupy 2 bytes or 16 bits. In some embodiments, the BS emoji set quantity information can also be an 8-bit unsigned integer, occupying 1 byte or 8 bits.

[0044] In some embodiments, the animation stream sequence may include multiple data frames. These multiple data frames may include two types: I-frames and P-frames. An I-frame, also called a keyframe, is a data frame that can be encoded independently without referencing information from other frames. A P-frame is a prediction frame, a data frame that is encoded and decoded by referencing preceding I-frames or P-frames, recording the changes in data relative to the reference frame. In some embodiments, each data frame includes header information and body information. In some embodiments, the header information includes control information required for decoding each data frame, such as timestamps and data precision for each frame. The header information may also include a control information identifier. The control information identifier can be used to determine the data structure of the payload (i.e., body information), such as whether at least one of BS expression quantity selection and residuals is used. In some embodiments, the body information is used to determine the payload of each data frame.

[0045] In some embodiments, the facial expressions of the digital human can be controlled via Base-Side (BS) expression weights. Each BS expression weight controls a specific facial expression. The value of the BS expression weight ranges from 0 to 1, where 0 indicates that the corresponding BS expression has no deformation, and 1 indicates that the corresponding BS expression has maximum deformation. In some embodiments, the data frame further includes head pose parameters. The head pose parameters are animation parameters used to encode the rotation of the digital human's head. The rotation of the digital human's head can be controlled by Euler angles of the head joints rotating around the x-axis, y-axis, and z-axis, respectively.

[0046] In some embodiments, the frame structure of the I-frame can be referred to as shown in Figure 2B. In some embodiments, the I-frame includes header information and body information. In some embodiments, the header information includes a timestamp. In some embodiments, the timestamp is used to determine the timestamp of the current frame, thereby synchronizing the current frame with other frames. The timestamp can be a 32-bit unsigned integer. The timestamp can occupy 4 bytes or 32 bits.

[0047] In some embodiments, the header information further includes a prediction mode. The prediction mode may include at least one of the following modes: no prediction, intra-frame prediction, inter-frame prediction, and joint intra- and inter-frame prediction. In some embodiments, when the data frame is an I-frame, the prediction mode may be no prediction or intra-frame prediction. In some embodiments, the prediction mode may be a 2-bit unsigned integer. In some embodiments, the first bit of the prediction mode may be used to indicate whether inter-frame prediction is enabled; the second bit of the prediction mode may be used to indicate whether intra-frame prediction is enabled. For example, when the prediction mode is 00, the prediction mode indicates that the current prediction mode is no prediction; when the prediction mode is 01, the prediction mode indicates that the current prediction mode is intra-frame prediction; when the prediction mode is 10, the prediction mode indicates that the current prediction mode is inter-frame prediction; when the prediction mode is 11, the prediction mode indicates that the current prediction mode is joint intra- and inter-frame prediction.

[0048] In some embodiments, the header information further includes a second sequence parameter set label. The second sequence parameter set label indicates the sequence parameter set used in the current frame. In some embodiments, the second sequence parameter set label is a 4-bit unsigned integer. In some embodiments, the second sequence tag label occupies 4 bits.

[0049] In some embodiments, the header information further includes a BS emoji quantity selection flag. This flag indicates whether BS emoji quantity selection is enabled in the current frame. The BS emoji quantity selection flag can be a 1-bit Boolean value. In some embodiments, when the BS emoji quantity selection is 1, the flag indicates that BS emoji quantity selection is enabled in the current frame. When the BS emoji quantity selection is 0, the flag indicates that BS emoji quantity selection is not enabled in the current frame.

[0050] In some embodiments, the header information further includes quantization precision. The quantization precision indicates the quantization precision information of the BS expression weight. The BS expression weight is a floating-point number between 0 and 1. The quantization precision determines how many decimal places the BS expression weight can be accurate to. The number of precise decimal places of the BS expression weight can be determined based on the value of the quantization precision. For example, the BS expression weight can be accurate to the value of the quantization precision divided by 10 decimal places. In some embodiments, the quantization precision can be a 16-bit unsigned integer. The quantization precision can occupy 2 bytes or 16 bits.

[0051] In some embodiments, when the BS emoji quantity selection identifier indicates that BS emoji quantity selection is enabled in the current frame, the header information further includes BS emoji quantity information. The BS emoji quantity information indicates the number of BS emojis used in the current frame, and the BS emoji quantity information is not greater than the number of BS emoji sets in the sequence parameter set.

[0052] In some embodiments, the header information may further include general frame information reserved bits, which can be used for future expansion and parameter definition. The general frame information reserved bits may be 1 bit.

[0053] In some embodiments, when the prediction mode is no prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter, and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the volume information includes a BS index, first weight information, and head pose parameters. In some embodiments, the number of BS indices and first weight information included in the volume information is the number of BS expressions indicated by the BS expression quantity information. The BS index and the first weight information correspond to each other. The BS index can be used to indicate the BS expression used, and the first weight information can be used to indicate the weight value of the BS expression after offset corresponding to the BS index, that is, the weight value of the BS expression after offset according to the corresponding offset value in the offset value information table.

[0054] The head posture parameters may include a first Euler angle (Head_rotation_x), a second Euler angle (Head_Rotation_y), and a third Euler angle (Head_Rotation_z). The first Euler angle may be the Euler angle (Pitch) of the head joint rotating about the x-axis; the second Euler angle may be the Euler angle (Yam) of the head joint rotating about the y-axis; and the third Euler angle may be the Euler angle (Roll) of the head joint rotating about the z-axis. In some embodiments, the values ​​of the second Euler angle (Yam) and the third Euler angle (Roll) may be in the range of (-π, π), and the value of the first Euler angle (Pitch) may be in the range of (-π / 2, π / 2). The quantization precision of the head posture parameters may be determined based on the quantization precision information in the control information identifier.

[0055] In some embodiments, when the prediction mode is no prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter, and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the volume information includes a BS index and first weight information. In some embodiments, the number of BS indices and first weight information included in the volume information is the number of BS expressions indicated by the BS expression quantity information. The BS index and the first weight information correspond to each other. The BS index and the first weight information can be referred to the above description and will not be repeated here.

[0056] In some embodiments, when the prediction mode is no prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter, and the BS expression quantity selection identifier in the head information indicates that the current frame does not enable BS expression quantity selection, the volume information includes first weight information and head pose parameters. In some embodiments, the quantity of the first weight information may be the quantity of BS expressions indicated by the BS expression set quantity information in the sequence parameter set. The first weight information and the head pose parameters are as described above and will not be repeated here.

[0057] In some embodiments, when the prediction mode is no prediction, and the head pose identifier in the sequence parameter set indicates that the head pose parameter is not enabled in the animation stream sequence, and the BS expression quantity selection identifier in the head information indicates that the BS expression quantity selection is not enabled in the current frame, the volume information includes first weight information. In some embodiments, the quantity of the first weight information may be the quantity of BS expressions indicated by the BS expression set quantity information in the sequence parameter set. The first weight information can be referred to the above description and will not be repeated here.

[0058] In some embodiments, when the prediction mode is intra-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameters, and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the volume information includes a BS index, first weight information, and head pose parameters. In some embodiments, the number of BS indices and first weight information included in the volume information is the number of BS expressions indicated by the BS expression quantity information. The BS index corresponds to the first weight information. The BS index can be used to indicate the BS expressions used. The head pose parameters can be referred to the above description and will not be repeated here.

[0059] The first weight information can be confirmed in the following ways:

[0060] When the BS expression indicated by the BS index has a reference BS expression in the sequence parameter set, the first weight information includes a first weight value and a first residual, wherein the first weight value is the weight value of the reference BS expression corresponding to the BS expression after offset according to the offset value of the BS expression in the offset value information table, and the first residual is the difference between the weight of the BS expression and the weight of the reference BS expression.

[0061] When the BS expression indicated by the BS index does not have a reference BS expression in the sequence parameter set, the first weight information includes a second weight value, wherein the second weight value is the weight value of the BS expression after offset according to the offset value of the BS expression in the offset value information table.

[0062] In some embodiments, when the prediction mode is intra-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter, and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the volume information includes a BS index, first weight information, and head pose parameters. In some embodiments, the number of BS indices and first weight information included in the volume information is the number of BS expressions indicated by the BS expression quantity information. The BS index and the first weight information correspond to each other. The first weight information can be referred to the above description and will not be repeated here.

[0063] In some embodiments, when the prediction mode is intra-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame does not enable the BS expression quantity selection, the volume information includes first weight information and head pose parameters. In some embodiments, the number of first weight information included in the volume information is the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set. The first weight information and head pose parameters can be referred to the above description, and will not be repeated here.

[0064] In some embodiments, when the prediction mode is intra-frame prediction, and the head pose identifier in the sequence parameter set indicates that the head pose parameter is not enabled in the animation stream sequence, and the BS expression quantity selection identifier in the head information indicates that the BS expression quantity selection is not enabled in the current frame, the volume information includes first weight information. In some embodiments, the number of first weight information included in the volume information is the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set. The first weight information can be referred to the above description and will not be repeated here.

[0065] In some embodiments, the frame structure of the P-frame can be referred to as shown in Figure 2C. In some embodiments, the P-frame includes header information and body information. In some embodiments, the header information includes a timestamp. In some embodiments, the timestamp is used to determine the timestamp of the current frame, thereby synchronizing the current frame with other frames. The timestamp can be a 32-bit unsigned integer. The timestamp can occupy 4 bytes or 32 bits.

[0066] In some embodiments, the header information further includes a prediction mode. When the data frame is a P-frame, the prediction mode can be inter-frame prediction or intra-frame joint inter-frame prediction. That is, when the data frame is a P-frame, the prediction mode can be 10 or 11.

[0067] In some embodiments, when the current frame is a P-frame, encoding or decoding of the current frame can be performed using one or more of the header information of the previous I-frame, the sequence parameter set, and / or information from the previous frame.

[0068] In some embodiments, when the prediction mode is inter-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables the BS expression number selection, the volume information includes a BS index, second weight information, and a head pose residual. The number of the BS index and the second weight information can be the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame. The BS index corresponds to the second weight information. In some embodiments, the second weight information includes a second residual, which is the difference between the weight value of the BS expression and the weight value of the corresponding BS expression in the previous frame.

[0069] In some embodiments, the head pose residual can be the difference between the head pose parameters of the current frame and the head pose parameters of the previous frame. For example, the head pose residual may include a first Euler angle residual, a second Euler angle residual, and a third Euler angle residual. The first Euler angle residual can be the difference between the Euler angle of the head joint rotation around the x-axis in the current frame and the Euler angle of the head joint rotation around the x-axis in the previous frame; the second Euler angle can be the difference between the Euler angle of the head joint rotation around the y-axis in the current frame and the Euler angle of the head joint rotation around the y-axis in the previous frame; the third Euler angle can be the difference between the Euler angle of the head joint rotation around the z-axis in the current frame and the Euler angle of the head joint rotation around the z-axis in the previous frame. The quantization precision of the head pose residual can be determined based on the quantization precision information in the control information identifier of the previous I-frame.

[0070] In some embodiments, when the prediction mode is inter-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose residual, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables BS expression number selection, the volume information includes a BS index and second weight information. The number of the BS index and the second weight information can be the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame. The second weight information includes a second residual, which is the difference between the weight value of the BS expression and the weight value of the corresponding BS expression in the previous frame.

[0071] In some embodiments, when the prediction mode is inter-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame does not enable BS expression number selection, the volume information includes second weight information and head pose residual. The quantity of the second weight information can be the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0072] In some embodiments, when the prediction mode is inter-frame prediction, and the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the BS expression number selection is not enabled in the current frame, the volume information includes second weight information. The quantity of the second weight information can be the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0073] In some embodiments, when the prediction mode is joint intra-frame and inter-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables BS expression number selection, the volume information includes a BS index, second weight information, and a head pose residual. The number of the BS index and the second weight information can be the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame. The BS index corresponds to the second weight information.

[0074] The second weight information can be confirmed in the following ways:

[0075] When the BS expression indicated by the BS index has a reference BS expression in the sequence parameter set, the second BS weight information is determined to include a third residual and a fourth residual. The third residual is the difference between the weight value of the reference BS expression and the weight value of the corresponding reference BS expression in the previous frame. The fourth residual is the difference between a first difference and a second difference. The first difference is the difference between the weight value of the BS and the weight value of the reference BS. The second difference is the difference between the weight value of the corresponding BS in the previous frame and the weight value of its corresponding reference BS.

[0076] When the BS expression indicated by the BS index does not have a reference BS expression in the sequence parameter set, it is determined that the second BS weight information includes a fifth residual, wherein the fifth residual is the difference between the BS weight value and the corresponding BS weight value in the previous frame.

[0077] In some embodiments, when the prediction mode is joint intra-frame and inter-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose residual, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables BS expression number selection, the volume information includes a BS index and second weight information. The number of the BS index and the second weight information can be the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame.

[0078] In some embodiments, when the prediction mode is joint intra-frame and inter-frame prediction, and the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame does not enable BS expression number selection, the volume information includes second weight information and head pose residual. In some embodiments, the quantity of the second weight information may be the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0079] In some embodiments, when the prediction mode is joint intra-frame and inter-frame prediction, and the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the BS expression number selection is not enabled in the current frame, the volume information includes second weight information. In some embodiments, the quantity of the second weight information may be the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0080] In some embodiments, the sequence parameter set and the data frames of the I-frames or P-frames can be transmitted through two independent channels. For example, the sequence parameter set can be reliably transmitted and retransmitted using a connection-oriented TCP protocol to ensure correct reception of the parameter set; the data frames can use the UDP protocol to meet real-time requirements. In some embodiments, the sequence parameter set can be transmitted at the very beginning of the session.

[0081] In some embodiments, the frame structure design described above allows animation parameters to be encoded according to different prediction modes and control parameters when encoding the animation bitstream. This frame structure extends the original prediction modes by adding intra-frame prediction modes and enabling free selection of different prediction modes. Furthermore, this frame structure extracts parameters shared by the entire animation stream sequence into a separate sequence parameter set, avoiding data redundancy caused by the loss of header information in a particular frame and the need to transmit control parameters for each frame. This frame structure also introduces scale variation parameters, thereby providing a lossless encoding option and avoiding data loss during encoding and decoding due to quantization.

[0082] Figure 3 is a flowchart of a method for encoding data frames for a BS emoji animation stream according to some embodiments of the present disclosure. This method can be applied to apparatuses for generating BS emoji animation streams, such as the sending terminal described in Figure 1. Referring to Figure 3, the flow of this method is as follows.

[0083] In step 301, an animation stream is received. The animation stream is a BS emoticon animation stream. The animation stream is the animation stream to be encoded.

[0084] In step 302, the header information is set.

[0085] In some embodiments, common parameters of the animation stream can be determined and a sequence parameter set can be generated. The sequence parameter set can be referred to in Figures 2A-2B, and will not be described again here.

[0086] In some embodiments, setting the header information includes setting header information for the current frame, where the current frame is a frame in the animation stream sequence, and the header information includes a timestamp and a prediction mode. In some embodiments, the timestamp is used to determine the timestamp of the current frame, thereby synchronizing the current frame with other frames.

[0087] The prediction mode includes at least one of the following modes: no prediction, intra-frame prediction, inter-frame prediction, and / or joint intra-frame and inter-frame prediction.

[0088] In some embodiments, it can be determined whether the current frame is an I-frame based on the prediction mode. If the current frame is an I-frame, a control information identifier is set in the header information of the current frame. If the current frame is not an I-frame, the header information of the current frame does not include the control information identifier.

[0089] In some embodiments, the current frame is determined to be an I-frame based on the prediction mode being no prediction or intra-frame prediction.

[0090] In some embodiments, based on the prediction mode being inter-frame prediction or intra-inter-frame joint prediction, it is determined that the current frame is not an I-frame, that is, the current frame is a P-frame.

[0091] In some embodiments, based on the prediction mode, a control information identifier is set in the header information, wherein the control information identifier is used to determine the data structure of the body information in the current frame.

[0092] In some embodiments, the control information identifier includes a second sequence parameter set label. The second sequence parameter set label indicates the sequence parameter set used in the current frame.

[0093] In some embodiments, the control information identifier further includes a BS emoji quantity selection identifier. The BS emoji quantity selection identifier is used to indicate whether BS quantity selection is enabled in the current frame.

[0094] In some embodiments, the control information identifier further includes quantization precision. The quantization precision indicates the quantization precision information of the BS expression weight. The BS expression weight is a floating-point number between 0 and 1. The quantization precision determines how many decimal places the BS expression weight can be accurate to. The number of decimal places of the BS expression weight can be determined based on the value of the quantization precision. For example, the BS expression weight can be accurate to the value of the quantization precision divided by 10 decimal places.

[0095] In some embodiments, when the BS emoji quantity selection identifier indicates that BS emoji quantity selection is enabled in the current frame, the control information identifier further includes BS emoji quantity information. The BS emoji quantity information indicates the number of BS emojis used in the current frame, and the BS emoji quantity information is not greater than the number of BS emoji sets in the sequence parameter set.

[0096] In step 303, body information is set. In some embodiments, the body information of the current frame can be set based on the control information identifier and the sequence parameter set.

[0097] In some embodiments, when the prediction mode is no prediction, the volume information of the current frame includes one or more of the following: BS index, first weight information, and head pose information.

[0098] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the volume information includes BS indexes, first weight information, and head pose parameters. In some embodiments, setting the volume information of the current frame includes setting N BS indices, setting N first weight information, and setting head pose parameters, where N is equal to the number of BS expressions indicated by the BS expression quantity information. The BS indexes and the first weight information correspond to each other. The BS indexes can be used to indicate the BS expressions used, and the first weight information can be used to indicate the weight value of the BS expression after offset corresponding to the BS index, that is, the weight value of the BS expression after offset according to the corresponding offset value in the offset value information table.

[0099] The head posture parameters may include a first Euler angle (Head_rotation_x), a second Euler angle (Head_Rotation_y), and a third Euler angle (Head_Rotation_z). The first Euler angle may be the Euler angle (Pitch) of the head joint rotating about the x-axis; the second Euler angle may be the Euler angle (Yam) of the head joint rotating about the y-axis; and the third Euler angle may be the Euler angle (Roll) of the head joint rotating about the z-axis. In some embodiments, the values ​​of the second Euler angle (Yam) and the third Euler angle (Roll) may be in the range of (-π, π), and the value of the first Euler angle (Pitch) may be in the range of (-π / 2, π / 2). The quantization precision of the head posture parameters may be determined based on the quantization precision information in the control information identifier.

[0100] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter, and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the volume information includes BS indexes and first weight information. In some embodiments, setting the volume information of the current frame includes setting N BS indices and N first weight information, where N is equal to the number of BS expressions indicated by the BS expression quantity information.

[0101] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame does not enable the BS expression quantity selection, the volume information includes first weight information and head pose parameters. In some embodiments, setting the volume information of the current frame includes setting M first weight information and head pose parameters, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0102] When the head pose identifier in the sequence parameter set indicates that the head pose parameter is not enabled in the animation stream sequence, and the BS expression quantity selection identifier in the head information indicates that the BS expression quantity selection is not enabled in the current frame, the volume information includes first weight information. In some embodiments, setting the volume information of the current frame includes setting M first weight information, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0103] In some embodiments, when the prediction mode is intra-frame prediction, the volume information of the current frame includes one or more of the following: BS index, first weight information, and head pose information.

[0104] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the volume information includes BS indexes, first weight information, and head pose parameters. In some embodiments, setting the volume information of the current frame includes setting N BS indices, setting N first weight information, and setting head pose parameters, where N is equal to the number of BS expressions indicated by the BS expression quantity information. The BS indexes correspond to the first weight information. The BS indexes can be used to indicate the BS expressions used.

[0105] The first weight information can be confirmed in the following ways: when the BS expression indicated by the BS index has a reference BS expression in the sequence parameter set, the first weight information includes a first weight value and a first residual, wherein the first weight value is the weight value of the reference BS expression corresponding to the BS expression after offset according to the offset value of the BS expression in the offset value information table, and the first residual is the difference between the weight of the BS expression and the weight of the reference BS expression; when the BS expression indicated by the BS index does not have a reference BS expression in the sequence parameter set, the first weight information includes a second weight value, wherein the second weight value is the weight value of the BS expression after offset according to the offset value of the BS expression in the offset value information table.

[0106] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter, and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the volume information includes BS indexes and first weight information. In some embodiments, setting the volume information of the current frame includes setting N BS indices and N first weight information, where N is equal to the number of BS expressions indicated by the BS expression quantity information.

[0107] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame does not enable the BS expression quantity selection, the volume information includes first weight information and head pose parameters. In some embodiments, setting the volume information of the current frame includes setting M first weight information and head pose parameters, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0108] When the head pose identifier in the sequence parameter set indicates that the head pose parameter is not enabled in the animation stream sequence, and the BS expression quantity selection identifier in the head information indicates that the BS expression quantity selection is not enabled in the current frame, the volume information includes first weight information. In some embodiments, setting the volume information of the current frame includes setting M first weight information, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0109] In some embodiments, when the prediction mode is inter-frame prediction, the volume information of the current frame includes one or more of the following: BS index, second weight information, and head pose residual.

[0110] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables the BS expression number selection, the volume information includes a BS index, second weight information, and a head pose residual. In some embodiments, setting the volume information of the current frame includes setting N BS indices, setting N second weight information, and setting a head pose residual, where N is equal to the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame. The BS index and the second weight information correspond to each other. The BS index can be used to indicate the BS expressions used. The second weight information includes a second residual, which is the difference between the weight value of the BS expression and the weight value of the corresponding BS expression in the previous frame.

[0111] In some embodiments, the head pose residual can be the difference between the head pose parameters of the current frame and the head pose parameters of the previous frame. For example, the head pose residual may include a first Euler angle residual, a second Euler angle residual, and a third Euler angle residual. The first Euler angle residual can be the difference between the Euler angle of the head joint rotation around the x-axis in the current frame and the Euler angle of the head joint rotation around the x-axis in the previous frame; the second Euler angle can be the difference between the Euler angle of the head joint rotation around the y-axis in the current frame and the Euler angle of the head joint rotation around the y-axis in the previous frame; the third Euler angle can be the difference between the Euler angle of the head joint rotation around the z-axis in the current frame and the Euler angle of the head joint rotation around the z-axis in the previous frame. The quantization precision of the head pose residual can be determined based on the quantization precision information in the control information identifier of the previous I-frame.

[0112] When the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the BS expression number selection is enabled in the current frame, the volume information includes BS indexes and second weight information. In some embodiments, setting the volume information of the current frame includes setting N BS indices and N second weight information, where N is equal to the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame.

[0113] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual, and the BS expression quantity selection identifier in the head information of the previous I-frame indicates that the current frame does not enable the BS expression quantity selection, the volume information includes second weight information and head pose residual. In some embodiments, setting the volume information of the current frame includes setting M second weight information and head pose residual, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0114] When the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the BS expression number selection is not enabled in the current frame, the volume information includes second weight information. In some embodiments, setting the volume information of the current frame includes setting M second weight information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0115] In some embodiments, when the prediction mode is joint intra-frame and inter-frame prediction, the volume information of the current frame includes one or more of the following: BS index, second weight information, and head pose information.

[0116] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables BS expression number selection, the volume information includes BS indexes, second weight information, and head pose residuals. In some embodiments, setting the volume information of the current frame includes setting N BS indices, setting N second weight information, and setting head pose residuals, where N is equal to the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame. The BS indexes correspond to the second weight information. The BS indexes can be used to indicate the BS expressions used.

[0117] The second weight information can be determined in the following ways: when the BS expression indicated by the BS index has a reference BS expression in the sequence parameter set, the second BS weight information is determined to include a third residual and a fourth residual, wherein the third residual is the difference between the weight value of the reference BS expression and the weight value of the corresponding reference BS expression in the previous frame, and the fourth residual is the difference between a first difference and a second difference, wherein the first difference is the difference between the weight value of the BS and the weight value of the reference BS, and the second difference is the difference between the weight value of the corresponding BS in the previous frame and the weight value of its corresponding reference BS; when the BS expression indicated by the BS index does not have a reference BS expression in the sequence parameter set, the second BS weight information is determined to include a fifth residual, wherein the fifth residual is the difference between the BS weight value and the corresponding BS weight value in the previous frame.

[0118] When the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the BS expression number selection is enabled in the current frame, the volume information includes BS indexes and second weight information. In some embodiments, setting the volume information of the current frame includes setting N BS indices and N second weight information, where N is equal to the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame.

[0119] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual, and the BS expression quantity selection identifier in the head information of the previous I-frame indicates that the current frame does not enable the BS expression quantity selection, the volume information includes second weight information and head pose residual. In some embodiments, setting the volume information of the current frame includes setting M second weight information and head pose residual, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0120] When the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence, and the BS expression number selection identifier in the head information of the previous I-frame indicates that the BS expression number selection is not enabled in the current frame, the volume information includes second weight information. In some embodiments, setting the volume information of the current frame includes setting M second weight information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0121] In step 304, encoded data for the animation stream is generated based on the header information and the body information.

[0122] Based on the descriptions in steps 301-303, the sequence parameter set of the animation stream, as well as the header and body information structures of the data frames of the animation stream, can be generated. The encoded data of the animation stream, i.e., the animation stream sequence, can be generated based on the header and body information of the data frames.

[0123] In some embodiments, the sequence parameter set and the data frame can be transmitted through independent channels. For example, the sequence parameter set can be reliably transmitted and retransmitted using a connection-oriented TCP protocol to ensure correct reception of the parameter set; the data frame can use the UDP protocol to meet real-time requirements. In some embodiments, the sequence parameter set can be transmitted at the very beginning of the session.

[0124] Figure 4 is an exemplary flowchart illustrating a decoding method for an animation stream sequence according to some embodiments of the present disclosure. In some embodiments, the decoding method can be applied to a device for decoding an animation stream, such as the receiving terminal 103 shown in Figure 1. Referring to Figure 4, the flow of the method is as follows.

[0125] In step 401, the animation stream sequence is received. In some embodiments, the animation stream sequence includes multiple data frames. The multiple data frames may be I-frames or P-frames.

[0126] In some embodiments, receiving the animation stream sequence further includes receiving a sequence parameter set. The sequence parameter set can be referred to in Figures 2A-2C, and will not be described further here.

[0127] In step 402, header information is read, including the header information of the current frame, where the current frame is a frame in the animation stream sequence. The header information includes a timestamp and a prediction mode. The timestamp is used to determine the timestamp of the current frame, thereby synchronizing the current frame with other frames. The prediction mode includes at least one of the following: no prediction, intra-frame prediction, inter-frame prediction, and / or joint intra-frame and inter-frame prediction.

[0128] In some embodiments, it is determined whether the current frame is an I-frame based on the prediction pattern.

[0129] In some embodiments, the current frame is determined to be an I-frame based on the prediction mode being no prediction or intra-frame prediction. The current frame is determined to be a P-frame based on the prediction mode being inter-frame prediction or joint intra- and inter-frame prediction.

[0130] In some embodiments, based on the current frame being an I-frame, a control information identifier is read from the header information, wherein the control information identifier is used to determine the data structure of the body information in the current frame.

[0131] In some embodiments, when the current frame is an I-frame, the header information of the current frame includes a control information identifier. The control information identifier can be used to determine the data structure of the volume information in the current frame.

[0132] In some embodiments, when the current frame is a P-frame, the header information of the current frame does not include a control information identifier. The control information identifier in the header information of the previous I-frame can be used to determine the data structure of the volume information in the current frame.

[0133] In some embodiments, the control information identifier includes a second sequence parameter set label. The second sequence parameter set label indicates the sequence parameter set used in the current frame.

[0134] In some embodiments, the control information identifier further includes a BS emoji quantity selection identifier. The BS emoji quantity selection identifier is used to indicate whether BS quantity selection is enabled in the current frame.

[0135] In some embodiments, the control information identifier further includes quantization precision. The quantization precision indicates the quantization precision information of the BS expression weight. The BS expression weight is a floating-point number between 0 and 1. The quantization precision determines how many decimal places the BS expression weight can be accurate to. The number of decimal places of the BS expression weight can be determined based on the value of the quantization precision. For example, the BS expression weight can be accurate to the value of the quantization precision divided by 10 decimal places.

[0136] In some embodiments, when the BS emoji quantity selection identifier indicates that BS emoji quantity selection is enabled in the current frame, the control information identifier further includes BS emoji quantity information. The BS emoji quantity information indicates the number of BS emojis used in the current frame, and the BS emoji quantity information is not greater than the number of BS emoji sets in the sequence parameter set.

[0137] In step 403, volume information is read. In some embodiments, the volume information of the current frame is decoded based on the control information identifier and the sequence parameter set.

[0138] In some embodiments, decoding the body information of the current frame includes decoding the body information of the current frame based on one or more of the control information identifier in the header information of the current frame, the control information identifier in the header information of the previous frame I of the current frame, and / or the sequence parameter set.

[0139] Because different data frames have different prediction modes, their volume information will have different data structures. Even within the same prediction mode, different control information identifiers will result in different volume information structures. Therefore, there will be multiple different volume information structures for data frames under different prediction modes. Decoding these volume information structures under different prediction modes includes the following methods.

[0140] 1. When the prediction mode is no prediction, the volume information of the current frame is determined to include one or more of the following: BS index, first weight information, and head pose information.

[0141] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the body information is determined to include N BS indices, N first weight information, and head pose parameters, where N is equal to the number of BS expressions indicated by the BS expression quantity information. The BS indices and the first weight information correspond to each other. The BS indexes can be used to indicate the BS expressions used, and the first weight information can be used to indicate the weight value of the BS expression after offset corresponding to the BS index, that is, the weight value of the BS expression is the weight value after offset according to the corresponding offset value in the offset value information table.

[0142] The head posture parameters may include a first Euler angle (Head_rotation_x), a second Euler angle (Head_Rotation_y), and a third Euler angle (Head_Rotation_z). The first Euler angle may be the Euler angle (Pitch) of the head joint rotating about the x-axis; the second Euler angle may be the Euler angle (Yam) of the head joint rotating about the y-axis; and the third Euler angle may be the Euler angle (Roll) of the head joint rotating about the z-axis. In some embodiments, the values ​​of the second Euler angle (Yam) and the third Euler angle (Roll) may be in the range of (-π, π), and the value of the first Euler angle (Pitch) may be in the range of (-π / 2, π / 2). The quantization precision of the head posture parameters may be determined based on the quantization precision information in the control information identifier.

[0143] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame enables the BS expression number selection, it is determined that the body information includes N BS indices and N first weight information, where N is equal to the number of BS expressions indicated by the BS expression number information.

[0144] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame does not enable the BS expression quantity selection, it is determined that the body information includes M first weight information and head pose parameters, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0145] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame does not enable the BS expression number selection, it is determined that the body information includes M first weight information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0146] 2. When the prediction mode is intra-frame prediction, the volume information of the current frame includes one or more of the following: BS index, first weight information, and head pose information.

[0147] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame enables BS expression number selection, the body information is determined to include N BS indices, N first weight information and head pose parameters, wherein N is equal to the number of BS expressions indicated by the BS expression number information.

[0148] The first weight information can be confirmed in the following ways: when the BS expression indicated by the BS index has a reference BS expression in the sequence parameter set, the first weight information includes a first weight value and a first residual, wherein the first weight value is the weight value of the reference BS expression corresponding to the BS expression after offset according to the offset value of the BS expression in the offset value information table, and the first residual is the difference between the weight of the BS expression and the weight of the reference BS expression; when the BS expression indicated by the BS index does not have a reference BS expression in the sequence parameter set, the first weight information includes a second weight value, wherein the second weight value is the weight value of the BS expression after offset according to the offset value of the BS expression in the offset value information table.

[0149] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame enables the BS expression number selection, it is determined that the body information includes N BS indices and N first weight information, where N is equal to the number of BS expressions indicated by the BS expression number information.

[0150] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame does not enable the BS expression quantity selection, it is determined that the body information includes M first weight information and head pose parameters, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0151] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame does not enable the BS expression number selection, it is determined that the body information includes M first weight information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0152] 3. When the prediction mode is inter-frame prediction, the volume information of the current frame includes one or more of the following: BS index, second weight information, and head pose residual.

[0153] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables the BS expression number selection, the volume information is determined to include N BS indices, N second weight information, and head pose residuals, where N is equal to the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame. The BS indices and the second weight information correspond to each other. The BS indices can be used to indicate the BS expressions used. The second weight information includes a second residual, which is the difference between the weight value of the BS expression and the weight value of the corresponding BS expression in the previous frame.

[0154] In some embodiments, the head pose residual can be the difference between the head pose parameters of the current frame and the head pose parameters of the previous frame. For example, the head pose residual may include a first Euler angle residual, a second Euler angle residual, and a third Euler angle residual. The first Euler angle residual can be the difference between the Euler angle of the head joint rotation around the x-axis in the current frame and the Euler angle of the head joint rotation around the x-axis in the previous frame; the second Euler angle can be the difference between the Euler angle of the head joint rotation around the y-axis in the current frame and the Euler angle of the head joint rotation around the y-axis in the previous frame; the third Euler angle can be the difference between the Euler angle of the head joint rotation around the z-axis in the current frame and the Euler angle of the head joint rotation around the z-axis in the previous frame. The quantization precision of the head pose residual can be determined based on the quantization precision information in the control information identifier of the previous I-frame.

[0155] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables the BS expression number selection, it is determined that the volume information includes N BS indices and N second weight information, where N is equal to the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame.

[0156] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame does not enable the BS expression number selection, it is determined that the volume information includes M second weight information and head pose residual, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0157] When the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence and the BS expression number selection identifier in the head information of the previous I-frame indicates that the BS expression number selection is not enabled in the current frame, it is determined that the volume information includes M second weight information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0158] 4. When the prediction mode is joint intra-frame and inter-frame prediction, the volume information of the current frame includes one or more of the following: BS index, second weight information, and head pose information.

[0159] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables the BS expression number selection, the volume information is determined to include N BS indices, N second weight information, and head pose residuals, where N is equal to the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame. The BS indices correspond to the second weight information. The BS indices can be used to indicate the BS expressions used.

[0160] The second weight information can be determined in the following ways: when the BS expression indicated by the BS index has a reference BS expression in the sequence parameter set, the second BS weight information is determined to include a third residual and a fourth residual, wherein the third residual is the difference between the weight value of the reference BS expression and the weight value of the corresponding reference BS expression in the previous frame, and the fourth residual is the difference between a first difference and a second difference, wherein the first difference is the difference between the weight value of the BS and the weight value of the reference BS, and the second difference is the difference between the weight value of the corresponding BS in the previous frame and the weight value of its corresponding reference BS; when the BS expression indicated by the BS index does not have a reference BS expression in the sequence parameter set, the second BS weight information is determined to include a fifth residual, wherein the fifth residual is the difference between the BS weight value and the corresponding BS weight value in the previous frame.

[0161] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame enables the BS expression number selection, it is determined that the volume information includes N BS indices and N second weight information, where N is equal to the number of BS expressions indicated by the BS expression number information in the head information of the previous I-frame.

[0162] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame does not enable the BS expression number selection, it is determined that the volume information includes M second weight information and head pose residual, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0163] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose residual and the BS expression number selection identifier in the head information of the previous I-frame indicates that the current frame does not enable the BS expression number selection, it is determined that the volume information includes M second weight information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0164] In step 404, animation data is obtained by decoding based on the head information and body information.

[0165] According to the description in steps 401-403, the data structure in the volume information is determined based on the header information, and decoded to obtain decoded values. The decoded values ​​are then reconstructed to obtain animation data.

[0166] Figure 5 is an exemplary block diagram illustrating an encoding system for an animation stream according to some embodiments of the present disclosure. The encoding system may be and / or include devices applied to generate the animation stream, such as the sending terminal 101 shown in Figure 1. Referring to Figure 5, the encoding system 500 may include a receiving module 501. The receiving module 501 is used to receive the animation stream sequence. The animation stream sequence is a BS emoticon animation stream sequence. The animation stream sequence includes a plurality of data frames. In some embodiments, the data frame structure may be and / or include the frame structure described in conjunction with Figures 2B and 2C.

[0167] In some embodiments, the encoding system 500 may include a sequence parameter set generation module 502. The sequence parameter generation module is used to determine common parameters of the animation stream sequence and generate a sequence parameter set. The definition of the sequence parameter set can be found in Figures 2A-2C and will not be repeated here.

[0168] In some embodiments, the encoding system 500 includes a header information setting module 503. The header information setting module is used to set the header information of the data frame. The header information includes a timestamp and a prediction mode. In some embodiments, the timestamp is used to determine the timestamp of the current frame, thereby synchronizing the current frame with other frames.

[0169] The prediction mode includes at least one of the following modes: no prediction, intra-frame prediction, inter-frame prediction, and / or joint intra-frame and inter-frame prediction.

[0170] In some embodiments, it can be determined whether the current frame is an I-frame based on the prediction mode. If the current frame is an I-frame, a control information identifier is set in the header information of the current frame. If the current frame is not an I-frame, the header information of the current frame does not include the control information identifier.

[0171] In some embodiments, the current frame is determined to be an I-frame based on the prediction mode being no prediction or intra-frame prediction.

[0172] In some embodiments, based on the prediction mode being inter-frame prediction or intra-inter-frame joint prediction, it is determined that the current frame is not an I-frame, that is, the current frame is a P-frame.

[0173] When the current frame is an I-frame, a control information identifier is set in the header information. This control information identifier is used to determine the data structure of the volume information in the current frame. The control information identifier includes one or more of the following: a second sequence parameter set label, a BS expression quantity selection identifier, quantization precision, and / or a BS expression quantity selection identifier.

[0174] In some embodiments, the second sequence parameter set label is used to indicate the sequence parameter set used in the current frame. The BS expression quantity selection flag is used to indicate whether BS quantity selection is enabled in the current frame. The quantization precision is used to indicate the quantization precision information of the BS expression weights.

[0175] In some embodiments, when the BS emoji quantity selection identifier indicates that BS emoji quantity selection is enabled in the current frame, the control information identifier further includes BS emoji quantity information. The BS emoji quantity information can be referred to in Figures 2A-2C, and will not be repeated here.

[0176] In some embodiments, the encoding system 500 includes a volume information setting module 504. The volume information setting module 504 is used to set the volume information of the data frame. The volume information setting module can set the volume information of the data frame according to a prediction mode, the control information identifier of the current frame, and / or the control information identifier of the previous I-frame of the current frame.

[0177] In some embodiments, the body information setting module can set the body information of the data frame in conjunction with the process described in step 303 (setting body information) in Figure 3. To avoid repetition, it will not be described again here.

[0178] It should be noted that the coding system provided in this embodiment can implement all the method steps implemented in the above method embodiment and achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0179] Figure 6 is an exemplary block diagram illustrating a decoding system for decoding an animation stream according to some embodiments of the present disclosure. The decoding system may be and / or include a device for decoding the animation stream, such as the receiving terminal 103 shown in Figure 1. Referring to Figure 6, the decoding system may include a receiving module 601. The receiving module 601 can be used to receive a sequence parameter set and the animation stream sequence. The sequence parameter set can be referred to the definitions in Figures 2A-2C, and will not be repeated here.

[0180] In some embodiments, the animation stream sequence includes multiple data frames. The multiple data frames may be I-frames or P-frames.

[0181] In some embodiments, the decoding system may further include an I-frame processing module 602. In some embodiments, the decoding system can determine whether the current frame is an I-frame by parsing the header information of the current frame. The current frame is one of the plurality of data frames. The header information of the current frame includes a timestamp and a prediction mode. The prediction mode can be used to determine whether the current frame is an I-frame.

[0182] When the prediction mode is no prediction or intra-frame prediction, the I-frame processing module can determine that the current frame is an I-frame. The I-frame processing module can parse the control information identifier in the current frame header information. The control information identifier includes one or more of the following: second sequence parameter set label, BS expression quantity selection identifier, and / or quantization precision.

[0183] In some embodiments, when the BS emoji quantity selection identifier indicates that BS emoji quantity selection is enabled in the current frame, the control information identifier further includes BS emoji quantity information. The BS emoji quantity information can be referred to in Figures 2A-2C, and will not be repeated here.

[0184] Based on the sequence parameter set, the prediction mode, and the control information identifier, the I-frame processing module can process the volume information of the current frame.

[0185] When the prediction mode is no prediction, the I-frame processing module can process the BS index, first weight information and head pose information in the volume information of the current frame to obtain the decoded value.

[0186] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameters and the BS expression quantity selection identifier in the head information indicates that the current frame enables BS expression quantity selection, the I-frame processing module can process N BS indices, N first weight information, and head pose parameters in the volume information, where N is equal to the number of BS expressions indicated by the BS expression quantity information. The first weight information can be used to indicate the weight value of the BS expression after offset corresponding to the BS index, that is, the weight value of the BS expression after offset according to the corresponding offset value in the offset value information table. The head pose parameters may include a first Euler angle (Head_Rotation_x), a second Euler angle (Head_Rotation_y), and a third Euler angle (Head_Rotation_z).

[0187] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame enables the BS expression number selection, the I-frame processing module can process N BS indices and N first weight information in the volume information, where N is equal to the number of BS expressions indicated by the BS expression number information.

[0188] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame does not enable the BS expression quantity selection, the I-frame processing module can process the M first weight information and head pose parameters in the body information, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0189] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame does not enable the BS expression number selection, the I-frame processing module can process M first weight information in the body information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0190] When the prediction mode is intra-frame prediction, the I-frame processing module can process the BS index, first weight information and head pose information in the volume information of the current frame to obtain the decoded value.

[0191] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame enables BS expression number selection, the I-frame processing module can process N BS indices, N first weight information and head pose parameters in the volume information, wherein N is equal to the number of BS expressions indicated by the BS expression number information.

[0192] The first weight information can be confirmed in the following ways: when the BS expression indicated by the BS index has a reference BS expression in the sequence parameter set, the first weight information includes a first weight value and a first residual, wherein the first weight value is the weight value of the reference BS expression corresponding to the BS expression after offset according to the offset value of the BS expression in the offset value information table, and the first residual is the difference between the weight of the BS expression and the weight of the reference BS expression; when the BS expression indicated by the BS index does not have a reference BS expression in the sequence parameter set, the first weight information includes a second weight value, wherein the second weight value is the weight value of the BS expression after offset according to the offset value of the BS expression in the offset value information table.

[0193] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame enables the BS expression number selection, the I-frame processing module can process N BS indices and N first weight information in the volume information, where N is equal to the number of BS expressions indicated by the BS expression number information.

[0194] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose parameter and the BS expression quantity selection identifier in the head information indicates that the current frame does not enable the BS expression quantity selection, the I-frame processing module can process the M first weight information and head pose parameters in the body information, where M is equal to the number of BS expressions indicated by the BS expression set quantity information in the sequence parameter set.

[0195] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose parameter and the BS expression number selection identifier in the head information indicates that the current frame does not enable the BS expression number selection, the I-frame processing module can process M first weight information in the body information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0196] In some embodiments, the decoding system may further include a P-frame processing module 603. When the prediction mode is inter-frame prediction or intra-inter-frame joint prediction, the P-frame processing module can determine that the current frame is a P-frame. The P-frame processing module can obtain the control information identifier from the I-frame header information of the previous frame of the current frame.

[0197] Based on the sequence parameter set, the prediction mode, and the control information identifier in the header information of the previous I-frame of the current frame, the P-frame processing module can process the body information of the current frame.

[0198] When the prediction mode is inter-frame prediction, the P-frame processing module can process the BS index, second weight information and head pose residual in the volume information of the current frame to obtain the decoded value.

[0199] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the header information of the previous I-frame of the current frame indicates that the current frame enables the BS expression number selection, the P-frame processing module can process N BS indices, N second weight information, and the head pose residual in the volume information, where N is equal to the number of BS expressions indicated by the BS expression number information in the header information of the previous I-frame of the current frame. The second weight information includes a second residual, which is the difference between the weight value of the BS expression and the weight value of the corresponding BS expression in the previous frame. In some embodiments, the head pose residual can be the difference between the head pose parameters of the current frame and the head pose parameters of the previous frame.

[0200] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence does not enable the head pose residual and the BS expression number selection identifier in the header information of the previous I-frame of the current frame indicates that the BS expression number selection is enabled in the current frame, the P-frame processing module can process N BS indices and N second weight information in the volume information, where N is equal to the number of BS expressions indicated by the BS expression number information in the header information of the previous I-frame of the current frame.

[0201] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the header information of the previous I-frame of the current frame indicates that the current frame does not enable the BS expression number selection, the P-frame processing module can process M second weight information and head pose residual in the volume information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0202] When the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence and the BS expression number selection identifier in the header information of the previous I-frame of the current frame indicates that the BS expression number selection is not enabled in the current frame, the P-frame processing module can process M second weight information in the volume information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0203] When the prediction mode is joint intra-frame and inter-frame prediction, the P-frame processing module can process the BS index, second weight information and head pose residual in the volume information of the current frame to obtain the decoded value.

[0204] In some embodiments, when the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the header information of the previous I-frame of the current frame indicates that the current frame enables the BS expression number selection, the P-frame processing module can process N BS indices, N second weight information and head pose residual in the volume information, wherein N is equal to the number of BS expressions indicated by the BS expression number information in the header information of the previous I-frame of the current frame.

[0205] The second weight information can be determined in the following ways: when the BS expression indicated by the BS index has a reference BS expression in the sequence parameter set, the second BS weight information is determined to include a third residual and a fourth residual, wherein the third residual is the difference between the weight value of the reference BS expression and the weight value of the corresponding reference BS expression in the previous frame, and the fourth residual is the difference between a first difference and a second difference, wherein the first difference is the difference between the weight value of the BS and the weight value of the reference BS, and the second difference is the difference between the weight value of the corresponding BS in the previous frame and the weight value of its corresponding reference BS; when the BS expression indicated by the BS index does not have a reference BS expression in the sequence parameter set, the second BS weight information is determined to include a fifth residual, wherein the fifth residual is the difference between the BS weight value and the corresponding BS weight value in the previous frame.

[0206] When the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence and the BS expression number selection identifier in the header information of the previous I-frame of the current frame indicates that the BS expression number selection is enabled in the current frame, the P-frame processing module can process N BS indices and N second weight information in the volume information, where N is equal to the number of BS expressions indicated by the BS expression number information in the header information of the previous I-frame of the current frame.

[0207] When the head pose identifier in the sequence parameter set indicates that the animation stream sequence enables the head pose residual and the BS expression number selection identifier in the header information of the previous I-frame of the current frame indicates that the current frame does not enable the BS expression number selection, the P-frame processing module can process M second weight information and head pose residual in the volume information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0208] When the head pose identifier in the sequence parameter set indicates that the head pose residual is not enabled in the animation stream sequence and the BS expression number selection identifier in the header information of the previous I-frame of the current frame indicates that the BS expression number selection is not enabled in the current frame, the P-frame processing module can process M second weight information in the volume information, where M is equal to the number of BS expressions indicated by the BS expression set number information in the sequence parameter set.

[0209] In some embodiments, the decoding system further includes a reconstruction module 604. The reconstruction module 604 is used to determine the reconstruction value of the current frame. When the current frame is an I-frame, the reconstruction module 604 calculates the reconstruction value of the current frame based on the decoding value of the I-frame processing module. When the current frame is a P-frame, the reconstruction module 604 calculates the reconstruction value of the current frame based on the decoding value of the P-frame processing module.

[0210] It should be noted that the coding system provided in this embodiment can implement all the method steps implemented in the above method embodiment and achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0211] Figure 7 is a schematic diagram illustrating an exemplary structure of a computer device according to some embodiments of the present disclosure. Referring to Figure 7, the computer device may include: a processor 701, a memory 702, a transceiver 703, and a bus interface 704.

[0212] Processor 701 is responsible for managing the bus interface and general processing, while memory 702 stores data used by processor 701 during operation. Transceiver 703 is used to receive and send data under the control of processor 701.

[0213] Bus interface 704 may include any number of interconnected buses and bridges, specifically connecting various circuits of one or more processors represented by processor 701 and memory represented by memory 702. The bus interface may also connect various other circuits and / or devices such as peripherals, voltage regulators, power management circuitry, and displays, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. Processor 701 is responsible for managing the bus interface and general processing, and memory 702 may store data used by processor 701 during operation.

[0214] The encoding and / or decoding methods disclosed in this disclosure can be applied to or implemented by the processor 701. During implementation, each step of the encoding and / or decoding method can be completed by integrated logic circuits in the hardware of the processor 701 or by instructions in software form. The processor 701 can be a general-purpose processor, digital signal processor, application-specific integrated circuit, field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in this disclosure can be directly implemented by the hardware processor, or implemented by a combination of hardware and software modules in the processor. The software modules can be located in non-volatile memory, random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 702, and the processor 701 reads the information in memory 702 and, in conjunction with its hardware, completes the steps of the encoding and / or decoding methods.

[0215] Specifically, the processor 701 is used to read computer instructions from the memory 702 and execute the functions implemented by the relevant devices in the encoding and / or decoding methods shown in any of the figures in Figures 3 and 4.

[0216] It should be noted that the device provided in this embodiment can implement all the method steps implemented in the above method embodiment and achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0217] This disclosure also provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the encoding and / or decoding methods performed by the device described in the above embodiments. In some embodiments, the computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0218] This disclosure also provides a computer program product, which, when invoked by a computer, causes the computer to execute the encoding and / or decoding methods performed by the device described in the above embodiments.

[0219] This disclosure also provides a chip. The chip includes a processor and a memory. The memory stores program instructions, and the processor can invoke the program instructions to execute the encoding and / or decoding methods performed by the aforementioned device.

[0220] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0221] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0222] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0223] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0224] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for decoding an animation stream, comprising: Receive the animation stream sequence; Read header information; Read body information; Animation data is obtained by decoding the header information and the body information.

2. The method according to claim 1, further comprising: A sequence parameter set is received, wherein the sequence parameter set is used to determine the common parameters of the animation stream sequence, wherein the sequence parameter set includes at least one of the following: a first sequence parameter set ID, a scale magnification parameter, an offset identifier, a reference information table identifier, a head pose identifier, and BS expression set quantity information.

3. The method according to claim 2, further comprising: The offset identifier indicates that the sequence parameter set includes an offset value information table, wherein the offset value information table includes a mapping table between BS expressions and the offset values ​​of the BS expressions; The reference information table identifier indicates that the sequence parameter set includes a reference information table, and the sequence parameter set further includes the reference information table, wherein the reference information table includes a mapping table between BS expressions and BS reference expressions.

4. The method according to claims 1-3, wherein, The header information includes a timestamp and a prediction pattern; reading the header information includes: Based on the prediction pattern, it is determined whether the current frame is an I-frame, wherein the current frame is a data frame in the animation stream sequence; Based on the fact that the current frame is an I-frame, the control information identifier in the header information is read, wherein the control information identifier is used to determine the data structure of the body information in the current frame; Based on the fact that the current frame is not an I-frame, the control information identifier of the previous I-frame of the current frame is obtained, wherein the control information identifier of the previous I-frame of the current frame is used to determine the data structure of the volume information in the current frame.

5. The method according to claim 4, wherein, The prediction mode includes at least one of no prediction, intra-frame prediction, inter-frame prediction, and joint intra-frame and inter-frame prediction.

6. The method according to claim 5, wherein, Determining whether the current frame is an I-frame based on the prediction mode information includes: When the prediction mode is no prediction or intra-frame prediction, the current frame is determined to be an I-frame; When the prediction mode is inter-frame prediction or intra-inter-frame joint prediction, it is determined that the current frame is not an I-frame.

7. The method according to claim 4, wherein, The control information identifier of the current frame or the previous I-frame of the current frame includes at least one of the following: Second sequence parameter set ID, wherein the second sequence parameter set ID is used to determine the sequence parameter set used in the current frame; BS emoji quantity selection flag, wherein the BS emoji quantity selection flag is used to indicate whether BS emoji quantity selection is enabled; Quantization precision, wherein the quantization precision is used to indicate the quantization scale of BS expressions.

8. The method according to claim 7, wherein, Based on the BS emoji quantity selection identifier indicating that BS emoji quantity selection is enabled, the control information identifier of the current frame or the previous I-frame of the current frame further includes BS emoji quantity information, wherein the BS emoji quantity information is used to indicate the number of BS emojis used in the current frame, and the BS emoji quantity information is not greater than the BS emoji set quantity information.

9. The method according to claim 7, wherein, Based on the fact that the prediction mode is no prediction, the current frame is an I-frame, and the read volume information includes: Based on the BS expression quantity selection flag indicating that BS expression quantity selection is enabled and the head pose flag indicating that head pose is enabled, the body information is determined to include BS expression index, first weight information and head pose information. The first weight information is the weight value of the BS expression corresponding to the BS expression index after offsetting according to the offset value information table in the sequence parameter set. Based on the BS expression quantity selection flag indicating that BS expression quantity selection is enabled and the head pose flag indicating that head pose is not enabled, the body information is determined to include the BS expression index and the first weight information. Based on the BS expression quantity selection flag indicating that BS expression quantity selection is not enabled and the head posture flag indicating that head posture is enabled, it is determined that the body information includes first weight information and head posture information. Based on the fact that the BS expression quantity selection flag indicates that BS expression quantity selection is not enabled and the head posture flag indicates that head posture is not enabled, it is determined that the body information includes first weight information.

10. The method according to claim 7, wherein, Based on the prediction mode being intra-frame prediction, and the current frame being an I-frame, the read volume information includes: Based on the BS expression quantity selection flag indicating that BS expression quantity selection is enabled and the head pose flag indicating that head pose is enabled, the body information is determined to include BS index, first weight information and head pose information. Based on the BS expression quantity selection flag indicating that BS expression quantity selection is enabled and the head pose flag indicating that head pose is not enabled, it is determined that the body information includes the BS index and the first weight information. Based on the BS expression quantity selection flag indicating that BS expression quantity selection is not enabled and the head posture flag indicating that head posture is enabled, it is determined that the body information includes first weight information and head posture information. Based on the fact that the BS expression quantity selection flag indicates that BS expression quantity selection is not enabled and the head posture flag indicates that head posture is not enabled, it is determined that the body information includes first weight information.

11. The method according to claim 10, wherein, The first weight information is determined in the following way: Based on the reference BS that exists in the reference information table, the first weight information is determined to include a first weight value and a first residual, wherein the first weight value is the weight value of the reference BS corresponding to the BS expression after offset, and the first residual is the difference between the weight of the BS expression and the weight of the reference BS. Based on the fact that the BS does not have a mapped reference BS in the reference information table, it is determined that the first weight information includes a second weight value, wherein the second weight value is the weight value after the BS is offset.

12. The method according to claim 7, wherein, Based on the prediction mode being inter-frame prediction, and the current frame being a P-frame, the read volume information includes: Based on the BS expression quantity selection flag indicating that BS expression quantity selection is enabled and the head pose flag indicating that head pose is enabled in the previous I-frame, the volume information is determined to include BS index, second weight information and head pose residual, wherein the second weight information includes second residual, the second residual is the weight difference between the BS weight and the corresponding BS weight in the previous frame, and the head pose residual is the difference between the head pose parameters of the current frame and the head pose parameters of the previous frame. Based on the BS expression quantity selection flag in the previous I-frame indicating that BS expression quantity selection is enabled and the head pose flag indicating that head pose is not enabled, the body information is determined to include BS index and second weight information. Based on the BS expression quantity selection flag indicating that BS expression quantity selection is not enabled and the head pose flag indicating that head pose is enabled in the previous I-frame, the volume information is determined to include second weight information and head pose residual. Based on the BS expression quantity selection flag indicating that BS expression quantity selection is not enabled and the head pose flag indicating that head pose is not enabled in the previous I-frame, it is determined that the body information includes second weight information.

13. The method according to claim 7, wherein, Based on the prediction mode being joint intra-frame and inter-frame prediction, and the current frame being a P-frame, the read volume information includes: Based on the BS expression quantity selection flag indicating that BS expression quantity selection is enabled and the head pose flag indicating that head pose is enabled in the previous I-frame, the volume information is determined to include BS index, second BS weight information and head pose residual, wherein the head pose residual is the difference between the head pose parameters of the current frame and the head pose parameters of the previous frame. Based on the BS expression quantity selection flag in the previous I-frame indicating that BS expression quantity selection is enabled and the head pose flag indicating that head pose is not enabled, the body information is determined to include BS index and second BS weight information. Based on the BS expression quantity selection flag indicating that BS expression quantity selection is not enabled and the head pose flag indicating that head pose is enabled in the previous I-frame, the volume information is determined to include the second BS weight information and the head pose residual. Based on the BS expression quantity selection flag indicating that BS expression quantity selection is not enabled and the head pose flag indicating that head pose is not enabled in the previous I-frame, it is determined that the body information includes the second BS weight information.

14. The method according to claim 13, wherein, The second BS weight information is determined based on the following method: Based on the reference BS that exists in the reference information table, the weight information of the second BS is determined to include a third residual and a fourth residual. The third residual is the difference between the weight value of the reference BS and the weight value of the corresponding reference BS in the previous frame. The fourth residual is the difference between a first difference and a second difference. The first difference is the difference between the weight value of the BS and the weight value of the reference BS. The second difference is the difference between the weight value of the corresponding BS in the previous frame and the weight value of its corresponding reference BS. Based on the fact that the BS does not have a mapped reference BS in the reference information table, the second BS weight information is determined to include a fifth residual, wherein the fifth residual is the difference between the BS weight value and the corresponding BS weight value in the previous frame.

15. The method according to any one of claims 9-14, wherein, The read body information includes: Based on the BS emoji quantity selection identifier indicating that BS emoji quantity selection is enabled, it is determined that the body information includes at least one of N BS indices, N first weight information and / or N second weight information, where N is the number of BS emojis indicated by the BS emoji quantity information in the control information identifier; Based on the BS emoji quantity selection flag indicating that BS emoji quantity selection is not enabled, it is determined that the volume information includes M first weight information, where M is the number of BS emojis indicated by the BS emoji set quantity information in the sequence parameter set.

16. The method according to claims 1-15, wherein, The animation data obtained by decoding based on the header and body information includes: Based on the header information, the body information is decoded to obtain the decoded value; The animation data is obtained by reconstructing based on the decoded values.

17. An encoding method for an animation stream, comprising: Receive animation stream; Set header information; Set body information; The encoded data of the animation stream is generated based on the header information and the body information.

18. A hardware device comprising: Memory, used to store program instructions; A processor for invoking the program instructions to execute the decoding method according to any one of claims 1-16 or the encoding method according to any one of claims 17.

19. A non-volatile storage medium comprising computer program instructions, wherein when the computer program instructions are invoked by a processor, the processor causes the processor to perform a decoding method according to any one of claims 1-16 or an encoding method according to any one of claims 17.

20. A chip comprising a processor and memory; The memory stores program instructions, and the processor is used to call the program instructions to execute the decoding method according to any one of claims 1-16 or the encoding method according to any one of claims 17.

Images