Long human motion sequence generation method based on diffusion model

Abstract

The invention provides a long human motion sequence generation method based on a diffusion model, and belongs to the technical field of computer vision and computer graphics. Comprising the following steps: constructing a long-time human body action data set; constructing a long text-to-long action generation network, wherein the long text-to-long action generation network comprises a text-to-short action sequence generation model and a motion sequence splicing model; the text-to-short action sequence generation model adopts a variational auto-encoder and diffusion model combined frame to generate a short-time human action sequence based on text conditions; the motion sequence splicing model realizes seamless splicing of motion sequences; training a long text-to-long action generation network by using the long-time human body action data set; and inputting the long text description into the trained long text-to-long action generation network to obtain a long human motion sequence matched with the text semantics. According to the invention, the problem of fragment transition is effectively solved, so that long-time human body action generation is realized.

Description

Technical Field

[0001] The invention relates to a method for generating a long human motion sequence based on a diffusion model, and belongs to the technical field of computer vision and computer graphics. Background Art

[0002] In the fields of computer vision and computer graphics, generating realistic and diverse human motion sequences is a core requirement for applications such as virtual character animation, game development, and human-computer interaction. In the early stages, the movement of virtual characters was primarily driven by manually designed keyframe animation or motion capture technology. These methods performed well in certain scenarios, but they were highly dependent on manual control and time-consuming to produce. Keyframe animation required designers to adjust the character's posture and motion trajectory frame by frame, while motion capture relied on specialized equipment and high-cost filming environments, factors that significantly limited the technology's widespread adoption and application.

[0003] The rapid development of deep generative models, particularly the introduction of technologies such as generative adversarial networks (GANs), variational autoencoders (VAEs), and denoising diffusion probabilistic models (DDPMs), has led to significant breakthroughs in generative digital human driving technology. Diffusion models, with their powerful probabilistic modeling capabilities, have demonstrated unique advantages in image and video generation tasks. Through a progressively denoised data generation mechanism, these models offer a new technical approach for generating human motion sequences. However, existing methods still face numerous challenges in generating long-duration motions, primarily in two key areas: First, high-quality long-duration motion data is expensive and difficult to acquire, and training data is relatively scarce, resulting in a lack of sufficient support for model training and optimization. Second, current models struggle to accurately understand and capture semantic information in long sequences, resulting in a gradual decline in generation quality as sequence length increases. Furthermore, due to computational resource constraints, models are prone to memory bottlenecks and computational inefficiencies when processing very long sequences. Fixed-length architectures also limit the model's ability to model long-range dependencies, leading to a significant decrease in the coherence and naturalness of generated sequences with increasing length. Therefore, how to effectively realize the generation of long-term human motion sequences while maintaining the generation advantages of the diffusion model has become a hot issue in current research. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for generating long human motion sequences based on a diffusion model, which effectively solves the problem of segment transition and thus realizes the generation of long-term human motion.

[0005] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions:

[0006] Construct a long-term human motion dataset, including filtering, spatial alignment, and interpolation of existing short-term motion data to form a continuous long motion sequence;

[0007] A long-text-to-long-action generation network is constructed, comprising a text-to-short-action sequence generation model and a motion sequence splicing model. The text-to-short-action sequence generation model employs a variational autoencoder combined with a diffusion model framework to generate short-duration human motion sequences based on text conditions. The motion sequence splicing model inserts Gaussian noise transition segments, takes two temporally discontinuous action segments as input, and utilizes the denoising process of the diffusion model to generate natural transition motions, thereby achieving seamless splicing of action sequences.

[0008] Use long-term human action datasets to train long text to long action generation networks;

[0009] The long text description is input into the trained long text to long action generation network to obtain a long human motion sequence that matches the text semantics.

[0010] Preferably, the specific construction process of constructing the long-term human motion dataset is as follows:

[0011] Filter the sequences with at least M frames (value is 10) in the original dataset, and remove the action sequences whose head-to-tail orientation difference exceeds the preset threshold;

[0012] Perform root node spatial alignment on the filtered action segments, aligning the root node trajectory with the last frame of the previous segment by translating the latter segment of the action;

[0013] Perform linear interpolation on the last N frames of the first segment and the first N frames of the second segment at the splicing point to generate a transition frame, where N is a preset positive integer;

[0014] The action sequences whose total number of frames after splicing reaches a preset length are saved as training data as a long-term human action dataset.

[0015] Preferably, the consistency of the movement direction is determined by calculating the Euclidean distance between the head and neck direction vectors, and if the difference between the head and tail direction vectors exceeds a preset threshold, the movement sequence is eliminated.

[0016] Preferably, the text-to-short action sequence generation model comprises: encoding the action sequence into a low-dimensional latent space through a variational autoencoder, performing multi-step noise diffusion and reverse denoising in the latent space in combination with a diffusion model, injecting text conditions through a cross-attention mechanism, and reconstructing the short human action sequence through a variational autoencoder;

[0017] The diffusion model includes a forward process and a reverse process. The forward process is a Markov noise process, and the reverse process predicts a human motion representation that is semantically consistent with the input text through a denoiser.

[0018] Preferably, the text-to-short action sequence generation model training process is as follows:

[0019] The human action sequence is input into the action encoder module of the variational autoencoder to map it to a latent representation in a low-dimensional latent space;

[0020] The forward process of the diffusion model is applied in this latent space. By gradually adding Gaussian noise, the originally ordered action representation gradually degenerates into a pure noise distribution.

[0021] The text description corresponding to the human work sequence is segmented into short text descriptions, which are then encoded into text feature vectors through a text encoder;

[0022] In the reverse process of the diffusion model, the model uses the encoded text feature vector as the conditional input, restores the noise through the denoiser, and obtains the human action representation that is consistent with the semantics of the input text;

[0023] The denoised human motion representation is input into the motion decoder of the variational autoencoder, and decoding is performed to obtain a semantically aligned short human motion sequence.

[0024] Preferably, in the cross-attention mechanism, the text feature vector is used as the key and value, and the action feature is used as the query.

[0025] Preferably, the motion sequence splicing model training process is as follows:

[0026] Receive two human motion sequence segments as input, and insert a Gaussian noise sequence between the two sequences as the initial sequence;

[0027] Taking the actions at both ends as conditions, the middle noise segment is gradually optimized through reverse sampling of the diffusion model to generate a smooth transition action sequence; the reverse sampling of the diffusion model is modeled as a Markov chain;

[0028] Output a complete long action sequence consisting of two human action sequence fragments and a transition action sequence.

[0029] Preferably, the diffusion model of the motion sequence splicing model introduces a cross-attention mechanism in the denoising process, using the conditional frame features of the previous and next action segments as keys and values, and the intermediate representation of each frame of the transition action sequence as a query; the conditional frame features are the last N frames of the starting action sequence and the first N frames of the ending action sequence.

[0030] Preferably, the long text description is input into the trained long text to long action generation network to obtain a long human motion sequence that matches the text semantics, specifically in the following manner:

[0031] Input the long text description into the trained long text to long action generation network and process it into a short text description;

[0032] The text is encoded through a text encoder, and the generated text feature vector is input into the reverse process of the diffusion model in the latent space. Based on the noise prediction mechanism, the redundant information in the initial random noise is gradually removed through multi-step iterative back diffusion to generate a short action sequence;

[0033] The short action sequence is passed to the motion sequence concatenation model to output a long human motion sequence that matches the text semantics.

[0034] Preferably, the action sequence includes rotation, position, speed and foot contact information of the three-dimensional joint.

[0035] The advantages of the present invention are that the present invention uses long text to achieve extended action generation, effectively bridging the gap between short-duration and long-duration action synthesis. First, in response to the current lack of training data and framework problems in the field of text-action generation, a long-duration text-action dataset was created to support the training of long action sequence generation models. Then, a two-stage generation framework was designed, which accepts text of arbitrary length as input to generate long human action sequences that match the text description. Users only need to input a text description to generate the required human action sequence. Not only that, the design of the framework also allows users to accurately edit local segments in the generated sequence. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0037] Figure 1 Schematic diagram of the process of the present invention.

[0038] Figure 2 Schematic diagram of human joints in the dataset of the present invention.

[0039] Figure 3 Schematic diagram of the text-to-short action sequence generation model of the present invention.

[0040] Figure 4 Schematic diagram of the motion sequence splicing model of the present invention.

[0041] Figure 5 This is a training diagram for a method for generating long human motion sequences based on a diffusion model according to the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] A method for generating long human motion sequences based on a diffusion model addresses the inability of existing text-driven methods to generate long human motion sequences. This method first naturally and rationally concatenates existing short text-motion datasets to form the largest and longest text-motion dataset available. A two-stage generation framework is then proposed: in the first stage, short-duration human motion sequences are generated using text input; in the second stage, a concatenation module is designed to achieve seamless transitions through an improved diffusion model, effectively resolving segment transition issues and enabling the generation of long-duration human motion sequences.

[0044] like Figure 1 Specifically, including:

[0045] S1: Construct a long-term human motion dataset, including screening, spatial alignment and interpolation of existing short-term motion data to form a continuous long motion sequence.

[0046] S2: Construct a long text to long action generation network, which includes a text to short action sequence generation model and a motion sequence splicing model; the text to short action sequence generation model adopts a variational autoencoder combined with a diffusion model framework to generate short-duration human action sequences based on text conditions; the motion sequence splicing model inserts Gaussian noise transition segments, takes two temporally discontinuous action segments as input conditions, and uses the denoising process of the diffusion model to generate natural transition actions, thereby achieving seamless splicing of action sequences.

[0047] S3: Using long-term human action dataset to train long text to long action generation network.

[0048] S4: Input the long text description into the trained long text to long action generation network to obtain a long human motion sequence that matches the text semantics.

[0049] As a refinement of the above embodiment, step S1 reasonably splices the text and motion data in the existing HumanML3D dataset, and accurately marks the start frame and end frame of each action in the text description.

[0050] First, the original dataset was preprocessed to remove short sequences with fewer than 10 frames to ensure that the action sequences were of sufficient duration for subsequent analysis. Subsequently, the poses of the start and end frames of each sequence were analyzed. Based on the principle of consistent orientation of the two frames, two actions with a small difference in orientation were spliced ​​together to avoid unnatural splicing results caused by large differences in posture. Furthermore, the root node position of the subsequent action was aligned with the root node position of the previous action. Finally, the last five frames of the previous action were interpolated with the first five frames of the subsequent action to achieve a smooth transition between action sequences.

[0051] The specific operation process is as follows:

[0052] Step 101: Initial data screening: For each action sequence M in the original data set i , filter the number of frames. If the sequence length is less than 10 frames (i.e., M i <10), the sequence is considered too short to provide stable transition information, so it is directly removed to avoid interference with the subsequent splicing process. This embodiment uses the standard SMPL model key points, and the positions of the key points of each human body are as follows: Figure 2 , for the remaining sequences, by calculating the direction vector between the head and neck And normalize it to get the unit direction vector It is used to indicate the current direction of the character. Then extract the direction vector of the first frame and the last frame of the sequence. and Calculate the Euclidean distance between them. If the difference exceeds a set threshold of 0.05, it indicates that the direction of the action has changed significantly during execution and is unsuitable for splicing with other sequences. Such sequences will be skipped. This ensures that the direction of the movement before and after splicing is consistent, ensuring a natural and smooth result after splicing.

[0053] Step 102: 3D space alignment: To achieve a natural connection between actions in space, before joining two action segments, their root joint trajectories need to be aligned to avoid incoherent jumps or drifts at the joints. The next segment is Each frame Represents the position of J joints in three-dimensional space. The root node position of the last frame of the current segment is: The root node position of the first frame of the next segment is: Then calculate the spatial offset between the two: Then perform a translation transformation on all frames of the next segment so that their root nodes are aligned with the last frame of the current segment: Processed sequence This is the aligned action clip. Finally, the two clips can be seamlessly spliced ​​in physical space:

[0054]

[0055] This alignment method can effectively avoid connection jumps caused by inconsistent local coordinate systems, ensuring that the action sequence has visual trajectory continuity and posture coherence.

[0056] Step 103, action transition interpolation: After completing the spatial alignment of the spliced ​​segments, in order to further improve the natural splicing of the two action sequences in the temporal dimension and avoid sudden changes at the connection between the two actions, a linear interpolation strategy is used to generate a transition frame between the end of the current sequence and the beginning of the next sequence.

[0057] Specifically, select the current sequence X a The last N frames (denoted as ), and the next sequence The first N frames (denoted as ). In the experiment, N is usually set to 5. Then, for each pair of corresponding frames Perform linear interpolation to generate intermediate transition frames

[0058]

[0059] Where t is the linear interpolation parameter, representing the normalized weight of the time step. In practice, a fixed step size Δt is usually selected for discrete interpolation (for example, Δt = 0.1, corresponding to 10 frames), then t∈{0.1, 0.2, …, 0.9}. Finally, the interpolated intermediate frame sequence is:

[0060]

[0061] These transition frames are inserted in the X a and , construct a complete smooth splicing sequence:

[0062]

[0063] Step 104, save the motion data: After the above processing, make a final judgment on the spliced ​​sequence. If the sequence length reaches or exceeds 600 frames Then add it to the target long action sequence set In, that is As a high-quality data sample after the construction is completed. On the contrary, if the length of the spliced ​​sequence is insufficient, the sample will be discarded or continued to be spliced ​​and extended with other short segments until the frame number requirement is met.

[0064] As a refinement of the above embodiment, the text-to-short action sequence generation model training process in step S2 is as follows:

[0065] Given an input condition c, such as a text describing an action The generative model generates a sequence of human actions based on text descriptions Where L represents the length or number of frames of the action, and the action representation combines the rotation, position, velocity, and foot contact information of three-dimensional joints. This generative model framework combines a variational autoencoder (VAE) with a diffusion model. First, a VAE action encoder based on the Transformer architecture is used to encode a human action sequence of length L into a low-dimensional latent space. Next, a diffusion model is introduced into the latent space to train the action sequence generation: in the forward pass, Gaussian noise is gradually added to the sequence to convert it into pure noise. In the backward pass, a denoiser is trained to gradually restore the noise, ultimately obtaining a high-quality action sequence. A preliminary action sequence is then reconstructed through a VAE decoder with a cross-attention mechanism to achieve conditional generation. The diffusion process introduces textual conditions, which are injected into the diffusion model via a cross-attention mechanism. A guidance strength factor is used to control the effect of the conditional guidance, thereby generating short human action sequences that match the textual semantics.

[0066] Specifically, first, during the generation process, the VAE action encoder E is used to transform the input action sequence Encoded as a latent variable z = E(x 1:L ).

[0067] Then, the diffusion process of the model is carried out in the latent space. The diffusion model predicts the noise from a T-step Markov noise process and realizes the ability to gradually retreat from the Gaussian distribution to the data distribution, which is expressed as Specifically, it is to train a Transformer-based denoiser ∈ θ (x t ,t), to gradually anneal the random noise into an action sequence Since direct diffusion calculation on the original action sequence is inefficient and requires a lot of computing resources, in order to reduce the computational requirements of the diffusion model on the original action sequence and improve the synthesis quality, the diffusion process is transferred to the above-mentioned action potential space. Based on this, a denoiser ∈ θThe diffusion process in the latent space is modeled as a Markov noise process, which is of the form:

[0068]

[0069] Among them, the constant α t ∈(0,1) is a hyperparameter in the sampling process, I represents the identity matrix, Represents a Gaussian distribution with a mean of The covariance is (1-α t )I; q represents the conditional probability distribution in the forward process, q(z t ∣z t―1 ) represents the latent variable z given the previous moment t―1 , find the latent variable z at the current moment t Distribution. Use Represents the noise-added sequence, and each step in the denoising process is z t―1 =∈ θ (z t ,t), in training denoiser∈ θ During the diffusion process, the encoder E is frozen and used to compress the action sequence into a latent variable z0. The forward sampling of the diffusion process samples from the latent distribution p(z0). In the reverse phase of the diffusion, the denoiser ∈ θ Through T iterations, we predict Represents the latent variable at time t=0, which does not contain noise information but only human motion information.

[0070] Finally, the action decoder D Decoded into a new action sequence, i.e.

[0071] The text-to-short action sequence generation model is used as follows:

[0072] like Figure 3 , firstly, the long text input is segmented. That is, C,L→(c1,l a ),(c2,l b )···(c n ,l m ), where C represents the long text description and L represents the length of the entire action sequence. c1,c2···c n is the short text description split from the long text C, l a ,l b ···l mRepresents action clips of different lengths. The text encoder encodes the text description (c) and inputs the encoded text feature vector into the latent space of the variational autoencoder (VAE). Then, a noise sequence is randomly generated in the latent space of the VAE and the conditional denoiser ∈ θ Perform denoising, that is, Z t―1 =∈ θ (Z t ,t) represents the denoising at time t. Through continuous denoising iteration to Z T →Z0. For the conditional diffusion model generation task, in the denoising process, in addition to the input time (t) information, the text condition Z t―1 =∈ θ (Z t ,t,c), where the text encoding used by the text condition is used. These conditions are effectively embedded into ∈ θ In [1], the cross attention mechanism formula is:

[0073]

[0074] Where Q = W Q h i The action feature of the current time step is used as the query (Q), K = W K c,V=W V c is the text feature vector as key (K) and value (V), h i is the intermediate feature in the diffusion network, and c is the encoded text sequence. The output of the cross attention is used as an enhanced context representation, which is fused with the current action feature and then passed to the subsequent layers, and is finally used to calculate:

[0075]

[0076] Among them, DenoiseNet θ Represents a denoising neural network used to predict the current latent variable z t The corresponding noise residual, (z t ,t,c) represents the predicted noise residual result, which is used to guide the reverse denoising process in the diffusion process.

[0077] This mechanism ensures that in each back-diffusion step, the model can refer to the text semantic information and gradually remove the predicted noise residuals, thereby generating an action sequence that is highly consistent with the input text. θ A cross attention module is inserted in some layers, and ∈ θIt is trained by classifier-free diffusion guidance, and the conditional and unconditional distributions are linearly combined as follows:

[0078]

[0079] Among them, c represents the text condition, Unconditional, Represents a denoising network guided by diffusion without a classifier, s is a scaling factor of the guidance strength, and when s>1, the guidance effect can be enhanced. The generated z0 is decoded by the VAE decoder, that is, X is a short sequence of human actions that matches the text.

[0080] As a refinement of the above embodiment, the motion sequence splicing model training in step S2 is specifically as follows:

[0081] To achieve seamless connection between human motion clips of arbitrary length, two temporally discontinuous motion clips are used as dual conditional inputs, and the conditional diffusion denoising process is used to generate natural and smooth transition motions, thereby integrating the originally fragmented motion streams into a coherent overall sequence.

[0082] Specifically, the model accepts two human action sequence segments as input, denoted as and Where a and b represent the number of frames in the first and last segments respectively. In order to construct the transition segment in the middle, a Gaussian noise sequence of length m is inserted between the two sequences. As an initial state, the length of the generated spliced ​​sequence is therefore L = a + m + b. (0) =[X (a) ||Z (m) ||X (b) ] is input into the diffusion model and a step-by-step reverse sampling process is performed under fixed conditions. The process is modeled as a Markov chain that gradually anneals from pure noise to the data distribution:

[0083]

[0084] Among them, x t represents the intermediate state at time t, α t ∈(0,1) is the preset annealing coefficient. The goal of the diffusion process is to learn a conditional denoiser ∈ θ (x t ,t,c), reconstruct the original action signal, where condition c={X (a) ,X (b)} provides contextual information on both sides of the splicing. In the sampling phase, a step-by-step denoising operation is performed from the final time T to t = 0:

[0085]

[0086] The model gradually generates a natural and smooth transition sequence under the dual guidance of the previous and next action segments. The final output is a complete human motion sequence with a continuous action structure. It successfully integrates the originally separate action segments into a logically consistent and smoothly timed whole.

[0087] The splicing model of the motion sequence is used as follows:

[0088] The purpose of this model is to splice together multiple short action segments generated by the text-to-short action sequence generation model, insert randomly initialized noise as transition segments between the segments, and then use the diffusion model to model these transition regions to generate complete long action sequences with temporal continuity and natural movements, such as Figure 4 . In this framework, the core task of the diffusion model is to conditionally generate the intermediate noise segment while retaining the existing short segments. The conditional information here mainly comes from the key frames before and after the short segment (start frame and end frame), that is, the "anchor" action state of each transition segment, which is used to constrain the spatial and temporal trend of the transition segment. The cross-attention mechanism is introduced into the diffusion model so that the action generation process of each transition segment can be conditioned on two conditional frames, thereby realizing reasonable motion transition frame generation. Assume that a certain transition segment is Z i ={z1,…,z n}, take 5 frames as the conditional frames at both ends, and the starting frame is the last 5 frames of the previous action segment (c start ) and the end frame is the first 5 frames of the next action segment (c end ), and encode them into feature vectors C = {c start ,c end At each time step t of the diffusion model, the input is the noised action state x t , denoising it through several layers of residual modules and attention layers. The attention layer uses x t The intermediate representation h of each frame in i As the query (Query), the conditional frame feature C as the key and value (Key, Value), calculate the cross attention:

[0089]

[0090] Where W Q ,W K ,W VIt is a learnable linear mapping matrix, and the output attention result is a context vector that integrates the start and end frame information, which is used to guide the denoising prediction of the current frame.

[0091] As a refinement of the above embodiment, the training process of step S3 is as follows:

[0092] Process such as Figure 5 As shown, the model first inputs the original human action sequence into the action encoder module of a variational autoencoder (VAE), mapping it to a latent representation in a low-dimensional latent space. Subsequently, the forward pass of a diffusion model is applied to this latent space. By gradually adding Gaussian noise, the originally ordered action representation is gradually degraded into a pure noise distribution, thereby learning the distributional properties of the action structure. In the backward diffusion process, the model uses the corresponding text encoding as a conditional input to guide each step of the denoising process, so that the generated latent representation can be gradually restored to a human action representation that is semantically consistent with the input text. Finally, the fully denoised latent representation is input into the action decoder of the VAE, which decodes it into a semantically aligned short human action sequence. To enable the model to generate longer coherent action sequences, a concatenation module is further added: multiple independent short action segments are used as diffusion conditions, random noise transition segments are inserted between the short action segments, and these transition regions are learned and generated using the conditional diffusion model. This makes the resulting long action sequence more natural and continuous in terms of timing and action transitions, and consistent with the input text semantics.

[0093] As a refinement of the above embodiment, the process of step S4 is as follows:

[0094] After training is complete, the model first receives a long text description and processes it into concise short text descriptions. This is then encoded using a text encoder, and the resulting text feature vector is fed into the reverse process of the diffusion model in the latent space. Based on a noise prediction mechanism, the model uses multi-step iterative back-diffusion to gradually remove redundant information from the initial random noise, generating a series of high-quality short action sequences. These short sequences are then passed to the motion sequence concatenation module, where they are concatenated naturally and smoothly, ultimately outputting a long, structurally complete human motion sequence that closely matches the textual semantics.

[0095] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for generating long human motion sequences based on a diffusion model, characterized in that: The following steps are involved: Construct a long-term human motion dataset, including filtering, spatial alignment, and interpolation of existing short-term motion data to form a continuous long motion sequence; A long-text-to-long-action generation network is constructed, comprising a text-to-short-action sequence generation model and a motion sequence splicing model. The text-to-short-action sequence generation model employs a variational autoencoder combined with a diffusion model framework to generate short-duration human motion sequences based on text conditions. The motion sequence splicing model inserts Gaussian noise transition segments, takes two temporally discontinuous action segments as input, and utilizes the denoising process of the diffusion model to generate natural transition motions, thereby achieving seamless splicing of action sequences. Use long-term human action datasets to train long text to long action generation networks; The long text description is input into the trained long text to long action generation network to obtain a long human motion sequence that matches the text semantics.

2. The method for generating long human motion sequences based on a diffusion model according to claim 1, characterized in that: The specific construction process of the long-term human motion dataset is as follows: Filter the sequences with at least M frames in the original dataset and remove the action sequences whose head-to-tail orientation difference exceeds a preset threshold; Perform root node spatial alignment on the filtered action segments, aligning the root node trajectory with the last frame of the previous segment by translating the latter segment of the action; Perform linear interpolation on the last N frames of the first segment and the first N frames of the second segment at the splicing point to generate a transition frame, where N is a preset positive integer; The action sequences whose total number of frames after splicing reaches a preset length are saved as training data as a long-term human action dataset.

3. The method for generating long human motion sequences based on a diffusion model according to claim 2, characterized in that: The consistency of the movement direction is determined by calculating the Euclidean distance between the head and neck direction vectors. If the difference between the head and tail direction vectors exceeds a preset threshold, the movement sequence is eliminated.

4. The method for generating long human motion sequences based on a diffusion model according to claim 1, characterized in that: The text-to-short action sequence generation model includes: encoding the action sequence into a low-dimensional latent space through a variational autoencoder, performing multi-step noise diffusion and reverse denoising in the latent space in combination with a diffusion model, injecting text conditions through a cross-attention mechanism, and reconstructing the short human action sequence through a variational autoencoder; The diffusion model includes a forward process and a reverse process. The forward process is a Markov noise process, and the reverse process predicts a human motion representation that is semantically consistent with the input text through a denoiser.

5. The method for generating long human motion sequences based on a diffusion model according to claim 4, characterized in that: The training process of the text-to-short action sequence generation model is as follows: The human action sequence is input into the action encoder module of the variational autoencoder to map it to a latent representation in a low-dimensional latent space; The forward process of the diffusion model is applied in this latent space. By gradually adding Gaussian noise, the originally ordered action representation gradually degenerates into a pure noise distribution. The text description corresponding to the human work sequence is segmented into short text descriptions, which are then encoded into text feature vectors through a text encoder; In the reverse process of the diffusion model, the model uses the encoded text feature vector as the conditional input, restores the noise through the denoiser, and obtains the human action representation that is consistent with the semantics of the input text; The denoised human motion representation is input into the motion decoder of the variational autoencoder, and decoding is performed to obtain a semantically aligned short human motion sequence.

6. The method for generating long human motion sequences based on a diffusion model according to claim 4, characterized in that: In the cross-attention mechanism, the text feature vector is used as the key and value, and the action feature is used as the query.

7. The method for generating long human motion sequences based on a diffusion model according to claim 1, characterized in that: The motion sequence splicing model training process is as follows: Receive two human motion sequence segments as input, and insert a Gaussian noise sequence between the two sequences as the initial sequence; Taking the actions at both ends as conditions, the middle noise segment is gradually optimized through reverse sampling of the diffusion model to generate a smooth transition action sequence; the reverse sampling of the diffusion model is modeled as a Markov chain; Output a complete long action sequence consisting of two human action sequence fragments and a transition action sequence.

8. The method for generating long human motion sequences based on a diffusion model according to claim 6, characterized in that: The diffusion model of the motion sequence splicing model introduces a cross-attention mechanism in the denoising process, using the conditional frame features of the preceding and following action segments as keys and values, and the intermediate representation of each frame of the transition action sequence as a query; the conditional frame features are the last N frames of the starting action sequence and the first N frames of the ending action sequence.

9. The method for generating long human motion sequences based on a diffusion model according to claim 6, characterized in that: The long text description is input into the trained long text to long action generation network to obtain a long human motion sequence that matches the text semantics. The specific method is as follows: Input the long text description into the trained long text to long action generation network and process it into a short text description; The text is encoded through a text encoder, and the generated text feature vector is input into the reverse process of the diffusion model in the latent space. Based on the noise prediction mechanism, the redundant information in the initial random noise is gradually removed through multi-step iterative back diffusion to generate a short action sequence; The short action sequence is passed to the motion sequence concatenation model to output a long human motion sequence that matches the text semantics.

10. The method for generating long human motion sequences based on a diffusion model according to claim 1, characterized in that: The action sequence includes rotation, position, velocity and foot contact information of three-dimensional joints.

Images