Method for generating motion poses based on similarity disentangling diffusion model

By employing a similarity-based deentanglement motion pose generation method based on a diffusion model, and utilizing similarity deentanglement and four-dimensional skeleton representation, global semantics are fused and pose features are optimized. This solves the problem of unnatural joint movements and achieves natural and smooth motion pose generation.

CN120599688BActive Publication Date: 2026-03-03HEFEI UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510492536.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-04-18
Publication Date
2026-03-03
Estimated Expiration
2045-04-18

AI Technical Summary

Technical Problem

Existing motion posture generation methods ignore the relative positional relationships between joints, resulting in unnatural postures, especially when dealing with multi-joint movements, where the movements are stiff, lacking fluidity and coordination.

Method used

A similarity-based unentanglement motion pose generation method based on a diffusion model is adopted. By adding noise to the real pose sequence, similarity unentanglement and four-dimensional skeleton representation are used to fuse global semantics, separate and optimize pose features, introduce skeleton constraints, and train the model to generate natural motion poses.

Benefits of technology

It improves the accuracy and naturalness of generated poses, and the generated motion posture videos have high smoothness and consistency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120599688B_ABST
    Figure CN120599688B_ABST
Patent Text Reader

Abstract

The application discloses a similarity disentangled motion pose generation method based on a diffusion model and relates to the technical field of motion pose generation.The method is as follows: similarity disentangled motion poses are obtained to obtain four-dimensional skeleton representations; noisy poses and corresponding four-dimensional skeleton representations are fused to form a pose feature sequence; the global semantics of text are fused in the pose feature sequence to obtain a fused pose feature sequence; the fused pose features are separated into coordinate features and skeleton features to form a coordinate feature sequence and a skeleton feature sequence; the skeleton feature sequence is integrated into the coordinate feature sequence to obtain an optimized pose feature sequence; the optimized pose feature sequence is projected into a pose sequence hypothesis; the pose sequence hypothesis is denoised, and the denoised pose sequence is taken as a new round of noisy pose sequence for iterative denoising until the number of denoising reaches a set number, and the pose sequence denoised for the set number of times is output as a predicted pose sequence.The application improves the accuracy and naturalness of generated poses.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of motion pose generation technology, and in particular to a similarity-based unentangled motion pose generation method based on a diffusion model. Background Technology

[0002] Motion pose generation is a key technology for simulating and generating the postures and movements of the human body or other objects in space. Early methods mainly focused on unconditional motion pose generation, that is, generating motion sequences without relying on specific input conditions. However, with the rapid development of deep learning technology and the gradual maturation of various generative models, research on conditional motion pose generation has been greatly promoted. These methods, by introducing specific input conditions (such as text statements), can generate more accurate motion poses that meet real-world needs, greatly expanding their application scenarios.

[0003] Existing methods typically treat posture as discrete three-dimensional coordinates, directly fitting them while ignoring the relative positions of joints. However, human posture is not solely determined by the positions of individual joints, but rather by the interrelationships and cooperation between joints to accomplish complex movements. If these relative positions of joints are not considered during the generation process, the generated postures often appear unnatural, with movements that are too stiff and lack realistic fluidity and coordination, especially when dealing with multi-joint movements. For example, the movements of complex parts such as the arms and legs require coordinated operation between joints; ignoring this may result in postures that fail to accurately reflect the natural movement patterns of the limbs. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, this invention provides a similarity-based unentangled motion pose generation method based on a diffusion model, which improves the accuracy and naturalness of the generated poses.

[0005] To achieve the above objectives, the present invention adopts the following technical solution, including:

[0006] A similarity-based unentanglement motion pose generation method based on a diffusion model trains the diffusion model using a sample set, which includes text and the corresponding real pose sequences. The specific method is as follows:

[0007] S1, add noise to each real pose in the real pose sequence to obtain a noisy pose sequence; where the pose is represented by the three-dimensional coordinates of the joints;

[0008] S2, encode the text to obtain global semantics;

[0009] S3, perform similarity deentanglement on each noisy pose p in the noisy pose sequence to obtain the four-dimensional skeleton representation p';

[0010] S4. Fuse each noisy pose p in the noisy pose sequence with the corresponding four-dimensional skeleton representation p' and encode it to obtain the encoded pose features. Constructing a pose feature sequence

[0011] S5, in the pose feature sequence Integrate global semantics g to obtain fused pose feature sequence d;

[0012] S6, perform feature separation on each fused pose feature sequence d, separating it into coordinate features and skeleton features, forming coordinate feature sequence and skeleton feature sequence;

[0013] S7, integrate the skeletal feature sequence into the coordinate feature sequence to obtain the optimized pose feature sequence d. p The optimized pose feature sequence d p Projection into attitude sequence hypothesis

[0014] S8, Assumptions on attitude sequences Denoising is performed to obtain the denoised pose sequence. Then, step S3 is skipped. The denoised pose sequence is input into step S3 and used as the new noisy pose sequence. The next denoised pose sequence is obtained by following steps S3-S8 until the denoising count reaches the set number. The denoised pose sequence after the set number of times is output as the predicted pose sequence.

[0015] S9. By minimizing the difference between the real pose sequence and the predicted pose sequence, the model is trained to obtain the trained diffusion model.

[0016] The prediction is performed using the trained diffusion model as follows: a noisy pose sequence is generated and processed according to steps S2-S8 to obtain the predicted pose sequence of the text.

[0017] Preferably, in step S3, the specific method for performing similarity deentanglement of noisy poses is as follows:

[0018] Based on the topological structure of the human body, the father's joint q is divided. p and children's joints q c The noisy pose p t The three-dimensional coordinate representation of the joint is converted into a four-dimensional skeletal representation, transforming the absolute position of the joint into a relative position. The calculation expression is:

[0019]

[0020] m = ||q c -qp ||2;

[0021] Where b = 0, 1, 2, ..., B, B = J - 1, B represents the number of bones, and J represents the number of joints; q * Indicates the direction of the skeleton; The child joint q in space c Pointing to the father's joint q p The direction vector, i.e., the three-dimensional representation of the direction of bone movement; m represents the bone length; ||·||2 represents the L2 norm, used to calculate the child's joint q c and father joint q p Euclidean distance in space; q c =(x c ,y c ,z c ) represents the three-dimensional coordinates of the child's joints; q p =(x p ,y p ,z p () represents the three-dimensional coordinates of the father's joint;

[0022] Preferably, in step S4, the noisy pose p is fused with the corresponding four-dimensional skeleton representation p' to obtain a new pose representation. Attitude representation Perform pose self-embedding and position embedding to obtain the encoded pose features. The calculation expression is:

[0023]

[0024] Where SE(·) represents implementation through a linear layer. The attitude encoding; PE(·) indicates that the attitude position is encoded using a predefined sine function, and s represents the noisy attitude p. t Location is time.

[0025] Preferably, in step S5, the calculation expression for the fused pose feature sequence d is:

[0026]

[0027] MHA stands for Multi-Attention Layer.

[0028] Preferably, the specific process of step S7 is as follows:

[0029] First, the coordinate feature sequence d c The updated coordinate feature sequence is obtained through a separate MHA layer.

[0030] Then, the skeletal feature sequence d is processed using an MHA-based attribute control layer. a Integrate into the updated coordinate feature sequence In the process, the optimized pose feature sequence d is obtained. p The calculation expression is:

[0031]

[0032] Finally, the optimized pose feature sequence d is processed using a multilayer perceptron (MLP) and a normalized layer (LayerNorm). p Reprojection into attitude sequence hypothesis The calculation expression is:

[0033]

[0034] Preferably, in step S1, the noise is added in the following way:

[0035] Gaussian noise ε~N(0,1) is injected into the true attitude p0. The noise intensity increases with the number of noise additions, as expressed by:

[0036]

[0037] Where Q(p(t)|p0) represents the probability distribution of the noisy pose p(t) after t noise additions, given the true pose p0. α t =1-β t ,β t ε represents the cosine variance of the noise ε; t represents the number of times noise is added, i.e., the number of times Gaussian noise is added to the true pose p0; the maximum number of noise additions is T; ∏ represents the product operation.

[0038] Preferably, in step S8, the denoising expression is:

[0039]

[0040] Among them, P T-1 The pose sequence is after the first denoising step; ε ~ N(0,1) represents the pose sequence independent assumption. Gaussian noise;

[0041]

[0042] Where, ε T This refers to the noise generated during the first denoising process, which is also the noise generated during the Tth denoising process; σ T is a scaling factor used to control the randomness in the diffusion process; p(T) is the noisy posture after T noise additions.

[0043] Preferably, in step S9, the total model loss L is:

[0044] L = L joint +λL bone ;

[0045] Joint loss L joint for:

[0046]

[0047] Where, p s and These are the actual pose sequence and the predicted pose sequence, respectively.

[0048] Bone loss L bone for:

[0049]

[0050] in, and From p respectively s and The calculated bone orientation.

[0051] The present invention also provides a readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the aforementioned similarity-based unentanglement motion pose generation method based on a diffusion model.

[0052] The present invention also provides a computer program product, comprising a computer program / instruction that, when executed by a processor, implements the aforementioned diffusion model-based similarity-based unentanglement motion pose generation method.

[0053] The advantages of this invention are:

[0054] (1) This invention provides a new perspective by modeling the limb skeleton to constrain joint associations and motion details, thereby improving the accuracy and naturalness of the generated poses. At the same time, this invention proposes a similarity-based unentanglement diffusion framework to further optimize the generation process.

[0055] (2) This invention introduces the concept of similarity unentanglement, which is a new strategy that surpasses traditional joint coordinate regression fitting. Unlike most previous methods that only use three-dimensional joint coordinate representation, this invention uses a decomposed four-dimensional skeleton representation to further constrain the relative positions of joints, thereby ensuring the accuracy of the marked pose details.

[0056] (3) This invention proposes a diffusion model with controllable properties, which can generate accurate and robust 3D poses based on textual vocabulary. At the same time, skeletal constraints are introduced to further improve the quality of the generated poses.

[0057] (4) The motion posture generation method proposed in this invention adopts a diffusion model, realizing the process of generating realistic motion postures from natural sentences. Experiments show that the generated motion posture videos have high fluency and consistency. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the process of the method of the present invention.

[0059] Figure 2 This is a schematic diagram comparing the generation results of the method of the present invention on PHOENIX14T with those of the standard pose, PT-GN method and GEN-OBT method.

[0060] Figure 3 This is a schematic diagram comparing the generation results of the method of the present invention on USTC-CSL with those of the standard attitude, PT-GN method and GEN-OBT method. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Depend on Figure 1 As shown, the similarity-based unentangled motion pose generation method based on the diffusion model mainly includes a training process and a prediction process.

[0063] The training process is as follows:

[0064] Step 1: Noisy pose p t The acquisition of.

[0065] Gaussian noise ε ~ N(0,1) is injected into the motion posture p0, and the noise intensity is increased over time. The formula for this process is as follows:

[0066]

[0067] Where Q(p) t |p0) represents the noisy posture p after t noise additions, given the motion posture p0. t The probability distribution; α t =1-β t ,β tLet represent the cosine variance of the noise ε; t represents the number of times Gaussian noise is added to the motion posture p0, T is the maximum number of times noise is added, t ~ U(0,T), t follows a uniform distribution between 0 and T and t is an integer; ∏ represents the product operation; when T is sufficiently large, Q(p T The distribution of |p0) is close to an isotropic Gaussian distribution.

[0068] p0∈R J×3 , where J represents the number of joints; 3 represents the dimension, corresponding to the three-dimensional coordinates of the joint.

[0069] Step 2: Text Sequence Encoding

[0070] Before performing text sequence encoding, this embodiment first constructs a text word vector library, as follows:

[0071] (1) Obtain the text database, perform word segmentation on the sentences in the text database, and obtain the segmented words. The text database uses the sign language corpus RWTH-PHOENIX-Weather 2014T (PHOENIX14T) and USTC-CSL.

[0072] (2) Select all words with a frequency greater than a set threshold from the segmented words and construct a word index table; perform one-hot encoding on each word in the index table to obtain the encoding vector, i.e., the one-hot vector, and denote the one-hot vector table, i.e., the word index table, as O = [o1, o2…o2]. u ], where o u Let be the encoding vector of the u-th word in the word index table, where the total number of words in the word index table is u.

[0073] (3) Randomly initialize a word embedding matrix W, and use the word embedding matrix W to map the encoding vector of each word in the word index table to the corresponding word vector. The dimension of the word vector is d. w This results in a text word vector library.

[0074] After constructing the text word vector library, the text word vector library is used for text sequence encoding. The specific process is as follows:

[0075] Step 2.1: Use a linear embedding layer to map the word vectors of each word in the sentence text to a high-dimensional feature space. Then, in order to capture the positional order of each word, use a positional encoding layer to supplement the positional information of each word in the sentence text.

[0076]

[0077] Among them, g n The word vector for the nth word is obtained from a text word vector library; We and b e These represent the weights and biases of the linear embedding layer, respectively; PE(·) represents the positional coding layer, which implements positional coding using sine and cosine functions in time sequence; g e n The word features are encoded for the nth word; n represents the nth word, n = 1, 2, ..., N, and there are N words in the sentence text.

[0078] Based on the word sequence in the sentence text, an encoded word feature sequence is constructed, thus obtaining the sentence feature g. e 1:N ={g e 1g e 2,...,g e n ,...,g e N}

[0079] Step 2.2: Process the sentence features g as described above. e 1:N The text is fed into a text encoder (TextEncoder) for encoding to obtain the global semantics g of the text. The text encoder consists of a multi-head attention layer (MHA), a normalization layer (Norm), and a front feedback layer (FL); the encoding process of the text encoder (TextEncoder) can be represented as follows:

[0080]

[0081] In this process, MHA plays a crucial role in handling contextual dependencies within the sequence. As is well known, MHA learns the relationships between words in a text sequence by performing dot product attention using a set of variables: Query-Q, Key-K, and Value-V. The computation process can be represented as follows:

[0082]

[0083] Where D is the scaling factor, and Q, K, and V refer to g e n The characteristic quantity obtained through MHA calculation. g e n This represents the word features encoded from the nth word in the sentence text.

[0084] In this embodiment, an MHA with M heads is constructed, as follows:

[0085]

[0086] in, and W OThese are learnable parameters; [h1,…,h M [] represents the attention vector, h1 represents the first head attention, h m This represents the attention of the m-th head.

[0087] Step 3: Similarity-based unentanglement.

[0088] In the pose sequence P, each noisy pose p t Corresponding to a series of 4D skeletal representations In this embodiment, the cervical node is used as the root node, and the parent joint q is divided according to the topological structure of the human body. p ∈R 3 and children's joints q c ∈R 3 Therefore, discrete 3D coordinate representations can be converted into 4D skeleton representations, and the absolute positions of joints can be converted into relative positions. The orientation and length of the skeleton can be obtained by calculating the orientation vectors and L2 norms of adjacent joints in 3D space, a process known as deentanglement. The deentanglement calculation process for 3D to 4D similarity can be represented as:

[0089]

[0090] m = ||q c -q p ||2

[0091] Where b = 0, 1, 2, ..., B, B = J - 1, B represents the number of bones, and J represents the number of joints; q * Indicates the direction of the skeleton; m represents the length of the skeleton; q c Indicates a child's joint; q p Indicates the father's joint; The child joint q in space c Pointing to the father's joint q p The direction vector, i.e., the three-dimensional representation of the direction of bone movement; ||·||2 represents the L2 norm, used to calculate the child's joint q c and father joint q p The Euclidean distance in space. Since the number of bones B is one less than the number of joints J, to ensure the 3D noisy pose p... t and the corresponding 4D skeletal representation p t The space size is the same, so add At this point, the 3D noisy pose p t ∈R J×3 The corresponding 4D skeletal representation p is obtained after similarity-based unentanglement. t '∈R J×4 .

[0092] Step 4: Motion posture sequence encoding

[0093] We will noisy pose p t and the corresponding 4D skeletal representation p t 'Connection, fusion into a new gesture representation'

[0094]

[0095] Subsequently, in order to refine the input sequence, the pose representation was... Pose self-embedding and position embedding are performed, expressed by the following formulas:

[0096]

[0097] Where SE(·) represents implementation through a linear layer. The pose encoding; PE(·) indicates that the position is encoded using a predefined sine function, and s represents the noisy pose p. t Location (time); This represents the encoded pose features.

[0098] Based on the pose sequence P, the encoded pose feature sequence is obtained. pose feature sequence It consists of the encoded pose features of each pose. The sequence formed.

[0099] Step 5: Semantic Guidance

[0100] The property-controllable diffusion model proposed in this invention uses the encoded pose feature sequence The global semantics g and the noisy time step t are used as inputs. The global semantics g is introduced to guide the encoded pose feature sequence. Feature embedding in the encoded pose feature sequence The process of fusing global semantics g to obtain the fused pose feature sequence d can be represented as follows:

[0101]

[0102] Among them, the fused pose feature sequence D is a sequence composed of the fused pose features of each pose.

[0103] Step Six: Feature Separation

[0104] For a fused pose feature sequence d that already contains global semantics g, this invention aims to separate 4D skeleton attributes and 3D joint attributes as supervisory cues to control motion pose generation. Therefore, an attribute separation layer is designed to reproject the fused pose feature sequence d into 7D space and separate the 3D coordinate feature sequence... and 4D skeletal feature sequences Separation.

[0105] Among them, the 3D coordinate feature sequence is a sequence composed of the 3D coordinate features of each pose; the 4D skeleton feature sequence is a sequence composed of the 4D skeleton features of each pose.

[0106] Step 7: Attribute Control

[0107] First, the coordinate feature sequence d c The updated coordinate feature sequence is obtained through a separate MHA layer. This process can be represented as:

[0108]

[0109] Then, the attribute control layer based on MHA will process the skeletal feature sequence d a Integrated into In this process, the motion details in the pose sequence are refined and optimized to obtain the optimized pose feature sequence d. p This process can be represented as:

[0110]

[0111] Finally, the optimized pose feature sequence d is processed using an MLP (Multilayer Perceptron) and a normalization layer LayerNorm. p Reprojection into attitude sequence hypothesis Represented as:

[0112]

[0113] Step 8: Noise Reduction

[0114] Assumptions on attitude sequence Denoising is performed to obtain the denoised pose sequence. Then, step S3 is skipped. The denoised pose sequence is input into step S3 and used as the new noisy pose sequence. The next denoised pose sequence is obtained by following steps S3-S8 until the denoising count reaches the set number. The denoised pose sequence after the set number of times is output as the predicted pose sequence.

[0115] Assumptions on attitude sequence Denoising is performed to form the input of the diffusion model with controllable properties at subsequent time steps. The denoising is expressed as:

[0116]

[0117] Among them, P T-1 The pose sequence is after the first denoising step; ε ~ N(0,1) represents the pose sequence independent assumption. Gaussian noise; PT-1 This is the pose sequence after the first denoising step;

[0118] ε~N(0,1) is independent of Standard Gaussian noise, and:

[0119]

[0120] Where, σ t Used to control the randomness in the diffusion process; ε T denoised as the noise in the first denoising process, which is also the noise in the Tth denoising process; p(T) is the noisy posture after T denoising processes;

[0121] The pose sequence P after the first denoising is obtained. T-1 Then, proceed to step two to process the attitude sequence P. T-1 The noisy poses in the image are deentangled using similarity, and the second denoised pose sequence is obtained by following steps two to eight. After T denoising operations, the pose sequence after T denoising operations is used as the final generated pose sequence.

[0122] Step Nine: Joint Loss Optimization

[0123] Joint constraint: Joint loss is used to constrain the accuracy of joint position during posture. Joint loss L joint The definition is as follows:

[0124]

[0125] Where, p s and These are the actual pose sequence and the predicted pose sequence, respectively.

[0126] Skeletal Constraints: To better constrain the complex motion details during training, skeletal loss L was introduced. bone To improve the accuracy of bone orientation in generated poses. Bone loss L bone The definition is as follows:

[0127]

[0128] in, and From p respectively s and The calculated bone orientation.

[0129] In summary, the total loss L of the model is:

[0130] L = L joint +λL bone

[0131] The model is trained by minimizing the difference between the true pose sequence and the predicted pose sequence (total model loss L), resulting in a trained diffusion model.

[0132] Reasoning process:

[0133] Unlike training, where noise is added to the motion posture sequence, inference initializes the motion posture sequence directly from Gaussian noise. Subsequent steps during inference are the same as those during training.

[0134] In this embodiment, the DEV set and TEST set were obtained based on the sign language corpus (PHOENIX14T). The performance of the present invention was verified on the DEV set and TEST set, respectively. The present invention outperforms existing technologies in the most commonly used sign language generation metrics BLEU (BLEU-1 and BLEU-4, i.e., B1 and B4), ROUGR, WER, FID, MPJPE, and MPJAE. For example, the existing technologies PT-base method and PT-FP&GN method disclosed in Progressive Transformers for End-to-end Sign Language Production, NAT-AT method disclosed in Towards Fast and High-Quality Sign Language Production, DET method disclosed in Including Facial Expressions in Contextual Embeddings for Sign Language Generation, GEN-OBT method disclosed in Gloss Semantic-Enhanced Network with Online Back-Translation for Sign Language Production, and Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion are all superior. The model discloses the G2P-DDM method, while the existing technology, Gloss-driven Conditional Diffusion Models for Sign Language Production, discloses the GCDM method. The specific comparison results are shown in Tables 1 and 2 below.

[0135] Table 1. Index values ​​of each method in the DEV set.

[0136]

[0137] Table 2. Index values ​​of each method on the TEST set.

[0138]

[0139] Furthermore, this embodiment also demonstrates two examples of sign language generation to illustrate the generation effect of the method of the present invention. The generation result (sign language gesture) of the present invention is compared with the generation results of standard gestures (standard sign language gestures extracted from sign language videos), the PT-GN method, and the GEN-OBT method, respectively. Figure 2 , Figure 3 As shown. Among them, Figure 2 and Figure 3 The images show the visualization results of the method of this invention on PHOENIX14T and USTC-CSL, respectively. Experiments show that the motion posture videos generated by the method of this invention have high smoothness and consistency.

[0140] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating a motion pose based on a similarity disentanglement model, characterized in that, The diffusion model is trained by using a sample set, and the sample includes text and a real pose sequence corresponding to the text, in the following specific manner: S1. Add noise to each real pose in the real pose sequence to obtain a noisy pose sequence; wherein the pose is represented by three-dimensional coordinates of a joint; S2. Encode the text to obtain global semantics; S3. performing similarity disentangling on each noisy pose in the sequence of noisy poses p performing similarity disentangling to obtain a four-dimensional skeletal representation p’ ; S4, fusing each noisy pose in the noisy pose sequence p with the corresponding four-dimensional bone representation p’ are fused and encoded to obtain an encoded pose feature , to form a pose feature sequence ; S5, in the pose feature sequence global semantic g , obtaining the fused pose feature sequence d ; S6, performing feature separation on each fusion pose feature in the fusion pose feature sequence d to form a coordinate feature sequence and a skeleton feature sequence; S7, integrate the bone feature sequence set into the coordinate feature sequence to obtain an optimized pose feature sequence d p ; the optimized pose feature sequence d p projected into the pose sequence hypothesis ; S8, on the attitude sequence hypothesis de-noising, the current de-noised attitude sequence, jump to step S3, the current de-noised attitude sequence as a new round of noisy attitude sequence, according to the way of step S3-S8, get the next de-noised attitude sequence, until the de-noising number reaches the set number, output the set number de-noised attitude sequence as the predicted attitude sequence; S9. Perform model training by minimizing the difference between the real pose sequence and the predicted pose sequence, to obtain the trained diffusion model; The trained diffusion model is used for prediction in the following specific manner: a noisy pose sequence is generated, and processing is performed in the manner of steps S2-S8 to obtain a predicted pose sequence of the text; In step S3, the specific manner of similarity disentanglement of the noisy pose is as follows: Based on the topological structure of the human body, the father's joints are divided. q p and children's joints q c Noisy posture p t The three-dimensional coordinate representation of the joint is converted into a four-dimensional skeletal representation, transforming the absolute position of the joint into a relative position. The calculation expression is: ; ; wherein, b= 0, 1, 2,..., B , B = J - 1, B denotes the number of bones, J denotes the number of joints; denotes the bone orientation; denotes the spatial child joint q c pointing to the father joint q p 's direction vector, i.e. the three-dimensional representation of the bone movement direction; m denotes the bone length; || · ||2denotes the L2-norm, used to calculate the Euclidean distance in space between the child joint q c and the father joint q p ; q c = ( x c , y c , z c ) are the three-dimensional coordinates of the child joint; q p = ( x p , y p , z p ) are the three-dimensional coordinates of the father joint; ;​ The specific process of step S7 is as follows: First, the coordinate feature sequence is obtained by independent MHA layers, to obtain an updated coordinate feature sequence ; Then, the sequence of bone features is integrated into the updated sequence of coordinate features by using the attribute control layer based on MHA , to obtain an optimized sequence of pose features d p The calculation expression is as follows:​​ ; Finally, the optimized pose feature sequence MLP and a normalization layer is projected back into the pose sequence hypothesis d p The expression is calculated as:​ 。 2. The similarity disentangled motion pose generation method based on a diffusion model according to claim 1, characterized in that, In step S4, the noisy pose p with the corresponding four-dimensional bone representation p’ fusion to obtain a new pose representation [ p ; p’ ];pose representation pose self-embedding and position embedding to obtain the encoded pose feature The calculation expression is: ; wherein, represents the pose encoding by a linear layer; represents the pose position is encoded using a predefined sinusoidal function, represents the noisy pose position, i.e. time.​ 3. The diffusion model based similarity disentangling motion pose generation method of claim 1, wherein, In step S5, the fused pose feature sequence d is calculated as follows: ; wherein, MHA is a multi-headed attention layer.

4. The method of claim 1, wherein, In step S1, the manner of adding noise is as follows: Gaussian noise ε~N (0,1) injected real poses p 0, with increasing noise strength as the number of noise injections increases, expressed as: ; in, Indicates the actual posture p Under the condition of 0, after t Noisy posture after secondary noise addition p ( t The probability distribution of ). , α t =1- β t , β t Table noise ε The cosine variance; This indicates the number of times noise was added, i.e., the distance to the true pose. p The number of times Gaussian noise is added to 0; the maximum number of noise additions is... T ; ∏ represents the product operation.

5. The method of claim 4, wherein, In step S8, the denoising expression is as follows: ; wherein, P T-1 is the first denoised pose sequence; is a Gaussian noise independent of the pose sequence hypothesis ​ ; ; in, This refers to the noise in the first denoising process, which is also the noise in the Tth denoising process; This is a proportionality coefficient used to control the randomness in the diffusion process; p ( T ) for the process T Noisy posture after adding noise.

6. The diffusion model based similarity disentangling motion pose generation method of claim 1, wherein, In step S9, the model total loss L is calculated as follows: L L = L1 + L2 + L3 ; Joint loss Is: ; wherein, and are the real and predicted pose sequences, respectively; Bone loss To: ; wherein, and are bone directions calculated from and respectively.

7. A readable storage medium characterized by, It has a computer program stored thereon, and the computer program is executed to implement the similarity disentanglement motion pose generation method based on the diffusion model according to any one of claims 1-6.

8. A computer program product, characterised in that, It includes computer programs / instructions that are executed by a processor to implement the similarity disentanglement motion pose generation method based on the diffusion model according to any one of claims 1-6.

Citation Information

Patent Citations

  • Method for generating 3D human body action based on text label

    CN116805046A

  • Digital human animation generation method and device, digital human animation driving model training method and device, and electronic equipment

    CN118397155A