Text-driven three-dimensional human body action generation method

Abstract

The invention discloses a text-driven three-dimensional human body action generation method. Fine-grained human body action generation is realized through a diffusion model and a large language model. The method comprises the following steps: firstly, extracting multilayer semantic features of a text by using a text encoder, and identifying core and non-core body parts by using a large language model; then dividing a human body into five areas including a left arm, a right arm, a left leg, a right leg and a trunk, and fusing and refining word-level text features and action features of all parts through a multi-head cross attention mechanism; using a self-adaptive loss function to optimize the action of the core part, and keeping the non-core part natural and smooth; and finally, performing denoising through a diffusion model to generate whole-body motion. In addition, the method supports parameter editing and motion complementing functions, and can flexibly control the motion amplitude and complement the motion of sub-input. The method can be applied to the fields of virtual reality, animation production and man-machine interaction, and generated actions are natural and accurate and accord with text description.

Description

Technical Field

[0001] The present invention relates to a text-driven three-dimensional human motion generation method based on a diffusion model and a large language model, which is a fine-grained human motion generation method and belongs to the technical field of artificial intelligence content generation. Background Art

[0002] With the continuous development of fields such as virtual reality, animation production, and human-computer interaction, generating fine-grained human motion from text descriptions has become an important requirement for these applications. Text-driven motion generation can significantly improve the efficiency of animation production, enhance the immersion in virtual reality, and make human-computer interaction more natural and intuitive. Existing text-to-motion generation methods are mainly based on generative adversarial networks (GANs), variational autoencoders (VAEs), and diffusion models. However, these methods have significant difficulties in generating complex, fine-grained multi-action motions.

[0003] The main problem with existing methods is that, on the one hand, the human body is usually considered as a unified whole, while the independent movements between various body parts are ignored. Human body movements are often composed of multiple body parts performing different atomic movements. This simplified global perspective makes it difficult for the generative model to capture details, and the resulting movement lacks naturalness and realism. On the other hand, existing models often ignore the details in the text description, only focusing on the global semantics, and are unable to flexibly adjust to small changes in the text. For example, for subtle differences such as "waving the left hand" and "waving the right hand", existing models often generate similar movements, while ignoring the key differences described in the text, resulting in a mismatch between the generated results and the input.

[0004] Therefore, existing technologies have limitations in generating fine-grained motion that conforms to specific text descriptions, and it is difficult to simultaneously take into account both the overall consistency of the whole body and the accuracy of details. Summary of the invention

[0005] Purpose of the invention: In view of the shortcomings of the prior art in fine-grained human motion generation, the present invention proposes a fine-grained text-driven human motion generation method by partitioning the human body and introducing a large language model for text semantic analysis, so as to solve the shortcomings of the prior art in capturing multi-action details and text alignment, and generate human motion that is more natural, accurate and consistent with the text description.

[0006] Technical solution: A text-driven 3D human motion generation method based on diffusion model and large language model. This method achieves complex and diverse precise motion generation through partition processing and multi-level text-motion alignment strategy. This method includes the following key steps:

[0007] Step 1: Text feature extraction module construction

[0008] In this step, a text feature extraction module (TEM) is constructed, which processes the input text using a pre-trained text model (CLIP-ViT-B / 32) to obtain word-level semantic representations and sentence-level features. In addition, a large language model (LLAMA2 or GPT-3.5Turbo) is used to analyze the actions described in the text. Through structured prompts and example analysis, the large language model can deeply understand the text and accurately identify the actions involved in the core and non-core participating parts, ensuring that key actions receive high attention during the generation process.

[0009] Step 2: Human motion zoning processing

[0010] In this step, the human body motion is divided into five independent regions: left arm, right arm, left leg, right leg and torso. Specifically, the motion characteristics of each part, namely the joint point information, are separated from the overall motion characteristics, and the human body motion is divided into five regions so that the motion characteristics of each part can be analyzed separately. This partitioning process helps to capture more detailed motion details, thereby improving the accuracy of motion generation.

[0011] Based on the "core body parts" extracted in step 1, the five regions are further divided into core participating parts and non-core participating parts, making the partition of each part more targeted. This allows for more targeted analysis and generation of motion features of different parts in subsequent processing.

[0012] Step 3: Construct a multi-head cross-attention fusion refinement layer

[0013] The refinement layer combines word-level text features with local action features through a multi-head cross-attention mechanism to achieve fine-grained adjustment of each body part.

[0014] The specific operations are as follows:

[0015] First, the motion features of each segment are mapped to the same latent space as the word-level features of the text through a linear mapping layer, ensuring consistent dimensionality and representation space between different feature types. Then, a multi-head cross-attention mechanism is applied to combine the word-level text features with the local motion features of each body part. The multi-head structure enables the model to learn from multiple attention points, thereby capturing information at different levels and achieving fine optimization of the movements of each part. Finally, the fused features are further processed using an encoder, which consists of multiple self-attention modules and a feedforward neural network, ensuring that the generated motion is smooth and coherent in the temporal dimension while maintaining the naturalness and accuracy of the motion.

[0016] Step 4: Adaptively optimize core and non-core areas

[0017] Adaptive loss function is used to distinguish and optimize core and non-core parts. Step 1: The core parts (such as head, arms and legs) are identified through a large language model and assigned higher optimization weights to ensure accurate capture of key parts. First, the distribution of core and non-core parts is determined by analyzing the action description in the input text, and the corresponding loss weights are adjusted dynamically. This method focuses more on processing core sub-actions, ensuring precise control of core parts in the generated motion, while non-core parts retain naturalness and flexibility.

[0018] Step 5: Integration and synthesis of whole body movements

[0019] In this step, the refined local motion features are synthesized into complete human motion through an integration layer. First, the refined motion features of each body part are merged to generate a complete full-body motion representation. The merging process includes aligning the spatial and temporal dimension features of each body part to ensure seamless motion. Next, the merged full-body motion is fused with sentence-level text features to ensure the semantic consistency of the overall motion. Sentence-level text features provide the semantic context of the overall motion, ensuring that the generated full-body motion meets the contextual requirements. Finally, the full-body motion is further optimized through a deep encoder that captures the detailed relationship between the full-body motion dynamics and the text description, thereby generating a natural motion sequence that conforms to the text description.

[0020] Step 6: Human motion generation based on diffusion model

[0021] Human motion is generated based on a diffusion model, and a conditional diffusion model is used to ensure that the generation process conforms to the text description. During the training process, noise is gradually added to the input motion data through a Markov noise chain to generate multi-level noise samples, which are used to train the model's denoising ability so that the model can gradually restore clear motion that conforms to the text description. In the generation phase, starting from full Gaussian noise, inverse denoising generates human motion that conforms to the text input. The entire denoising process consists of multiple time steps, and the motion sequence generated at each step is based on the input text and optional control parameters to gradually approach the ideal motion output at each stage.

[0022] Step 7: Implement parameter editing and motion completion functions

[0023] In addition to generating corresponding three-dimensional human body movements based on text descriptions, the present invention can also realize parameter editing functions and movement completion functions.

[0024] Parameter editing function: By adjusting the scaling factor τ of the local action, fine-grained control of the generated motion is achieved. The scaling factor is used to adjust the amplitude of the movement of each body part, and users can flexibly adjust the details of the generated motion through the interactive interface. For example, increasing the scaling factor of the arm can generate a larger arm swing, while reducing the scaling factor of the leg produces a softer leg movement.

[0025] Motion completion function: supports generating complete human body motion based on partial input. Users can provide partial motion input (e.g. upper body motion only), and generate unspecified body part motion by combining existing motion input and text description. First, feature extraction is performed on the existing motion input, and the remaining motion is generated by combining the text description, so that the generated full-body motion is seamlessly integrated with the input part, maintaining the naturalness and overall coordination of the motion.

[0026] Step 8: Loss function design and optimization target

[0027] The quality of the generated motion is optimized through adaptive loss functions in refinement and integration layers:

[0028] Refinement layer loss function: used to optimize the local action generation effect, distinguish the core and non-core parts, and ensure the fine capture of the core action. This loss function contains position loss and posture loss, which is used to ensure that the joint position of the generated action is consistent with the target while maintaining the rationality of the motion posture. The optimization goal of this process is: where λ 核心 and λ 非核心 Represent the parameter weights of the core and non-core parts respectively, and is the loss calculated using the same loss function: Represents the action features generated from the 1st to the Tth time step The mean square error loss with the true action feature X0.

[0029] Integration layer loss function: used to optimize the overall consistency of whole-body motion and alignment with text descriptions. The loss function of this process is similar to that of the refinement layer.

[0030] Total loss function: By combining the loss of the refinement layer and the integration layer and assigning appropriate weights, the generated human motion sequence is ensured to achieve the optimal effect. The total loss function introduces regularization loss to prevent overfitting, improve the generalization ability of the model, and make the generated motion adapt to different text descriptions and application scenarios. The optimization goals of this process are: Among them, λ 细化层 With λ 集成层 Used to balance the roles of refinement layers and integration layers in the entire model during training.

[0031] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the text-driven three-dimensional human motion generation method based on a diffusion model and a large language model are implemented.

[0032] A computer-readable storage medium stores a computer program for executing the above-mentioned text-driven three-dimensional human motion generation method based on a diffusion model and a large language model.

[0033] The text-driven human fine-grained motion generation method based on a diffusion model and a large language model provided by the present invention realizes the accurate generation of complex multi-actions by segmenting the human body and using a multi-level text-motion alignment strategy. Compared with the prior art, it has the following advantages:

[0034] 1. Improved generation accuracy: By dividing the human body into five parts (left arm, right arm, left leg, right leg and torso) and combining the multi-head cross-attention mechanism to independently optimize each part, the present invention significantly improves the accuracy of motion generation, effectively captures fine-grained multi-action details, and makes the generated motion more natural and consistent with the text description.

[0035] 2. Enhanced text alignment capability: The present invention utilizes a large language model to extract core and non-core body parts from the input text, and adopts a multi-level text-motion alignment strategy to ensure accurate mapping of key actions described in the text, achieving sensitivity and accuracy to subtle text differences, thereby improving the text alignment capability of the generated motion.

[0036] 3. Operational flexibility and control: By introducing the parameter editing function, the present invention realizes parameterized adjustment of the movements of various body parts, so that users can flexibly control the amplitude and intensity of the movements without retraining the model, which significantly improves the operational flexibility of the model in animation production and virtual reality applications.

[0037] 4. Natural and consistent whole-body motion generation: Through the combined processing of the refinement layer and the integration layer, the present invention effectively solves the shortcomings of the prior art in terms of whole-body motion consistency, so that the fine-grained movements of various body parts can be naturally integrated into an overall whole-body motion, ensuring the coordination and naturalness of the movements.

[0038] 5. Reduce the need for additional training: In motion completion applications, the method generates a complete motion sequence through existing partial input and text description, and can achieve natural completion of partial motion without additional training, significantly saving training time and computing resources.

[0039] The present invention can be widely used in virtual reality, animation production, human-computer interaction and other fields, and provides a more precise, natural and easy-to-operate solution for generating human motion in these application scenarios. It has broad application prospects and significant technical advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a method principle diagram of an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of an action perfection and parameter editing application of an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of body divisions according to an embodiment of the present invention;

[0043] Figure 4 is a schematic diagram of action sampling according to an embodiment of the present invention;

[0044] Figure 5 It is a schematic diagram of the action generation function of an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0046] A text-driven 3D human motion generation method based on a diffusion model and a large language model generates a complex multi-action motion sequence that conforms to the text description through human segmentation processing and a multi-level text-motion alignment strategy. The following is a specific embodiment and operation flow of the present invention.

[0047] like Figure 1 As shown, the framework of the present invention includes core components such as text feature extraction module, refinement layer, integration layer and diffusion model, which are used to achieve accurate generation of complex actions.

[0048] Step 1: Text feature extraction module construction

[0049] In this step, we first build a text feature extraction module (TEM). This module uses the pre-trained text model CLIP-ViT-B / 32 to process the input text to obtain word-level semantic representation and sentence-level features. The specific operations are as follows:

[0050] (111) The input text is processed by word segmentation, and each word is mapped to the word embedding space of the pre-trained model.

[0051] (112) Using the CLIP-ViT-B / 32 model, the word embedding of each word is encoded to obtain word-level text features.

[0052] (113) The CLIP-ViT-B / 32 model is used to encode the complete sentence and extract sentence-level features to obtain the overall semantic information.

[0053] Next, use a large language model (such as LLAMA2 or GPT-3.5Turbo) to understand and analyze the actions described in the text, and obtain the core and non-core body parts involved in the corresponding generated actions in the text. During the understanding and analysis process, the input and output introduction of the action generation work of <Problem Description> is adopted; the body partition introduction is introduced, and the joint chain contained in each part is noted.

[0054] <Example> Given two examples, for example, “a person greets others with his right hand while walking”, the core parts involved are “left leg, right leg, right arm”.

[0055] <Problem details and output settings> Given the text description "<to be input>", give the corresponding sequence of core participating parts, and require the output format to be several items in "[left arm, right arm, left leg, right leg, torso]".

[0056] In this way, the large language model can analyze and obtain the core participating parts of the text description, and the remaining parts are non-core participating parts.

[0057] By formula During the training process, the core body parts are given higher weights, thereby increasing the attention paid to the action features of the core parts during the generation process. The specific operation is as follows:

[0058] (121) Input text conditions into a large language model.

[0059] (122) Large language models provide in-depth understanding and analysis of text based on preset structured prompts and example analysis.

[0060] (123) Output the list of core participating parts corresponding to the text conditions to provide input for subsequent processing.

[0061] Step 2: Human motion zoning processing

[0062] In this step, human body motion is divided into five independent regions: left arm, right arm, left leg, right leg and torso. The specific operation is as follows:

[0063] (211) Figure 3 As shown, define the joints and range of motion for each area.

[0064] (212) Separate the joint information of each part from the overall motion features.

[0065] (213) Human body movements are divided into five areas so that the movement characteristics of each part can be analyzed separately.

[0066] Based on the "core body parts" extracted in step 1, these five areas are further divided into core participating parts and non-core participating parts, making the partition of each part more targeted. The specific operations are as follows:

[0067] (221) Based on the classification results of the large language model, determine whether each area is a core area.

[0068] (222) The same feature extraction and processing operations are performed on five independent body parts, and in the subsequent step 3, the parameter λ 核心 and λ 非核心 The weights are adjusted so that the weights of the core participating parts are greater than those of the non-core participating parts, thereby enhancing the model's attention to the core parts.

[0069] Step 3: Multi-head cross-attention fusion refinement layer

[0070] In this step, the word-level text features extracted in step 1 are combined with the motion features of the five independent body parts in step 2 through a multi-head cross-attention mechanism to achieve fine-grained adjustment of each body part. The specific operations are as follows:

[0071] (311) The motion features of each body part are mapped to the same latent space as the word-level text features through a linear mapping layer to obtain the motion features in the latent space.

[0072] (312) A multi-head cross-attention mechanism is applied to combine the word-level text features with the motion features of each body part in the latent space obtained in the above step (311) to obtain the fused features of each body part.

[0073] (313) The fused features of each body part are further processed using an encoder based on the Transformer architecture to obtain temporally continuous and fine-grained motion features. The encoder can effectively learn temporal features, thereby ensuring that the generated motion is smooth and coherent in the temporal dimension.

[0074] In the refinement layer (refinement process), the present invention adopts the loss function During the training process, the parameter λ is adjusted 核心 and λ 非核心 The weights are used to distinguish and optimize the core and non-core parts. The specific operations are as follows:

[0075] (321) By analyzing the action descriptions in the input text, the distribution of core and non-core parts is determined.

[0076] (322) Dynamically adjust the loss weights λ of the core and non-core parts 核心 and 非核心 , so that the model prioritizes the atomic actions corresponding to the core parts, ensuring the precise control of the core parts in the generated motion.

[0077] Step 4: Integration and synthesis of whole body movements

[0078] In this step, the refined local motion features are synthesized into complete human motion using the integration layer. The specific operations are as follows:

[0079] (411) The fusion features of each body part obtained by the refinement layer in step 3 are passed through their own independent mapping layers to restore the motion features of the latent space into the position information of the joint points corresponding to different body parts, that is, the motion representation of each body part. Subsequently, according to the composition method corresponding to the body partition, the motion representations of the five independent parts are merged to generate a complete whole body motion representation.

[0080] (412) The complete whole-body motion representation is converted into an embedded representation consistent with the sentence-level feature dimension through the mapping layer, and then fused with the sentence-level feature to obtain the concatenated whole-body motion feature, thus ensuring the semantic consistency of the overall motion.

[0081] (413) By using an encoder with a deeper encoder layer than the Transformer architecture in the refinement layer, the spliced ​​whole-body motion features are further optimized and learned to obtain a diffusion model. The integration layer uses the whole-body motion features as the training unit, so that it can learn the smoothness and naturalness of the overall motion. Unlike the aforementioned refinement layer that independently learns the motion features of five parts, the refinement layer cannot achieve overall optimization due to the lack of global information. The design of the integration layer makes up for this shortcoming. The reason for using a deeper encoder is that the integration layer processes the overall motion features of the whole body, and the amount of feature information to be learned is larger, while the refinement layer only processes the motion features of independent body parts.

[0082] Step 5: Motion generation process based on diffusion model

[0083] like Figure 4 As shown, this method generates human motion based on the diffusion model, and adopts the conditional diffusion model to ensure that the generation process conforms to the text description.

[0084] The specific operations are as follows:

[0085] (511) During the training phase, noise is gradually added to the input motion data through a Markov noise chain to generate multi-level noise samples, and the model learns the characteristics of the original input data through a denoising process.

[0086] (512) In the generation phase, starting from pure Gaussian noise, human motion that matches the text input is generated through an inverse denoising process.

[0087] Step 6: Parameter Editing and Motion Completion Function

[0088] The present invention also provides parameter editing function and motion completion function. The specific operations are as follows:

[0089] (611) Fine-grained control of the generated motion is achieved by adjusting the scaling factor τ of the local action.

[0090] (612) Supports generating complete human body motion based on partial input, and generates unspecified body part motion by combining existing motion input and text description. Specifically, during the generation process, the existing part motion input will replace the motion output of the corresponding part in the refinement layer, directly participate in the splicing process of independent part motion features, and be passed as input to the integration layer. At the same time, the text description is combined to guide the generation of new motion.

[0091] Step 7: Loss function and optimization objective

[0092] This method optimizes the generated motion quality through adaptive loss functions of refinement and integration layers. The specific operation is as follows:

[0093] (711) Design of refinement layer loss function in and The calculation principle is the same: first, according to Calculate the loss function for each body part, where x0 represents the original motion features of the body part in the training set, represents the motion characteristics of the generated body parts. Then, the loss functions of all core parts are summed up to get Sum the loss functions of all non-core parts to get This design is used to optimize the local action generation effect by distinguishing the core and non-core parts and specifically adjusting the model's attention direction.

[0094] (712) Design the integrated layer loss function: Where X0 represents the original motion features of the complete body in the training dataset, The motion features of the whole body generated by the integration layer are used to optimize the overall consistency of the whole body motion and the alignment with the text description.

[0095] (713) By combining the losses of the refinement layer and the integration layer and assigning the optimal weights obtained by the grid analysis method, it is ensured that the human motion sequence finally generated achieves the optimal effect.

[0096] Application Examples

[0097] like Figure 5 As shown, the present invention has wide applications in virtual reality scenarios. For example, when a user inputs the text "a person walks forward and waves with his right hand", the present invention generates human body movements that meet the description, presenting accurate walking and right hand waving movements while maintaining overall natural consistency.

[0098] In summary, the present invention provides a text-driven method for generating fine-grained human motion based on a diffusion model and a large language model. Through human segmentation processing and a multi-level text-motion alignment strategy, the fine generation of complex human motions and accurate alignment of text-driven motions are achieved. This method is applicable to multiple fields such as animation production, virtual reality, and human-computer interaction, and significantly improves the accuracy and control of motion generation, with broad application prospects and practical value.

[0099] Obviously, those skilled in the art should understand that the above-mentioned steps of the text-driven three-dimensional human body motion generation method based on the diffusion model and the large language model of the embodiment of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules respectively, or multiple modules or steps therein can be made into a single integrated circuit module to implement. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

Claims

1. A text-driven 3D human motion generation method based on diffusion model and large language model, characterized in that: The steps include: Step 1: Text feature extraction module construction Build a text feature extraction module, which uses the pre-trained text model to process the input text to obtain word-level text features and sentence-level features; The actions described in the text are analyzed by a large language model to distinguish between actions involving core and non-core body parts. The specific process is to enable the large language model to understand the text and extract the actions involving core and non-core participating parts through structured prompts and example analysis; Step 2: Human motion zoning processing Divide human body movement into multiple independent areas; separate the movement characteristics of each part from the overall movement characteristics; divide the five independent areas into core participating parts and non-core participating parts; Step 3: Construct a multi-head cross-attention fusion refinement layer The refinement layer combines word-level text features with local action features through a multi-head cross-attention mechanism to achieve fine-grained adjustments to each body part; Step 4: Adaptively optimize core and non-core areas Adopt adaptive loss function to distinguish and optimize core and non-core parts; determine the distribution of core and non-core parts by analyzing the action description in the input text, and dynamically adjust the corresponding loss weights; Step 5: Integration and synthesis of whole body movements The refined local motion features are synthesized into complete human motion through the integration layer. First, the refined motion features of each body part are merged to generate a complete full-body motion representation. The merging process includes aligning the spatial and temporal dimension features of each body part. Then, the merged full-body motion is fused with the sentence-level text features. Finally, the encoder optimizes the whole body motion to generate natural motion sequences that match the text description; Step 6: Human motion generation based on diffusion model Generate human motion based on the conditional diffusion model. During the training process, noise is gradually added to the input motion data through the Markov noise chain to generate multi-level noise samples, which are used to train the denoising ability of the model so that the model can gradually restore clear motion that conforms to the text description. In the generation stage, starting from complete Gaussian noise, reverse denoising is used to generate human motion that conforms to the text input. The entire denoising process includes multiple time steps, and the motion sequence generated in each step is based on the input text and optional control parameters, so as to be output at each stage. Step 7: Loss function design and optimization target; optimize the generated motion quality through adaptive loss functions of refinement layer and integration layer: Refinement layer loss function: used to optimize the local action generation effect, distinguish the core and non-core parts, and the optimization goal is: where λ 核心 and λ 非核心 Represent the parameter weights of the core and non-core parts respectively, and is the loss calculated using the same loss function: Represents the action features generated from the 1st to the Tth time step Mean square error loss with the true action feature X0; Integration layer loss function: used to optimize the overall consistency of the whole body movement and the alignment with the text description; Total loss function: By combining the losses of the refinement layer and the integration layer and assigning corresponding weights, the optimization goal is: Among them, λ 细化层 With λ 集成层 Used to balance the role of refinement layers and integration layers in the entire model during training.

2. The method for generating 3D human motion driven by text based on diffusion model and large language model according to claim 1, characterized in that: The method also includes realizing the functions of parameter editing and motion completion; Parameter editing function: fine-grained control of the generated motion is achieved by adjusting the scaling factor τ of the local motion; the scaling factor is used to adjust the amplitude of the motion of each body part, and the user adjusts the generated motion information according to the parameter value set; Motion completion function: supports generating complete human motion based on partial input; users provide partial motion input, and generate unspecified body part motions by combining existing motion input and text description; firstly, feature extraction is performed on the existing motion input, and the remaining motion is generated by combining the text description, so that the generated whole-body motion is integrated with the input part.

3. The text-driven 3D human motion generation method based on diffusion model and large language model according to claim 1, characterized in that: In step 3, the refinement layer integrates text features with local action features through a multi-head cross attention mechanism to achieve fine-grained adjustment of each body part; the specific operations are as follows: Firstly, the motion features of each body part are mapped to the same latent space as the word-level text features through a linear mapping layer to obtain the motion features in the latent space. Then, a multi-head cross-attention mechanism is applied to combine the word-level text features with the motion features in the latent space of each body part to obtain the fused features of each body part.

4. The method for generating 3D human motion driven by text based on diffusion model and large language model according to claim 1, characterized in that: In step 1, a text feature extraction module is first constructed. The text feature extraction module uses the pre-trained text model CLIP-ViT-B / 32 to process the input text to obtain word-level semantic representation and sentence-level features. The specific operations are as follows: (111) The input text is processed by word segmentation, and each word is mapped to the word embedding space of the pre-trained model; (112) Using the CLIP-ViT-B / 32 model, the word embedding of each word is encoded to obtain word-level text features; (113) Use the CLIP-ViT-B / 32 model to encode the complete sentence and extract sentence-level features; Use a large language model to understand and analyze the actions described in the text, and obtain the core and non-core body parts involved in the corresponding generated actions in the text; in the process of understanding and analyzing, use the input and output introduction of the action generation work of <Problem Description>; introduce the body partitions, and indicate the joint chains contained in each part; The large language model analysis obtains the core participating parts of the text description, and the rest are non-core participating parts; By formula Give more weight to core body parts during training, as follows: (121) Input text conditions into a large language model; (122) Large language models understand and analyze text based on preset structured prompts and example analysis; (123) Output the list of core participating parts corresponding to the text conditions.

5. The method for generating 3D human motion driven by text based on diffusion model and large language model according to claim 1, characterized in that: In step 2, the human body movement is divided into five independent areas: left arm, right arm, left leg, right leg and trunk. Step 2 includes the following steps: (211) Define the joints and range of motion for each region; (212) Separate the joint information of each part from the overall motion features; (213) divided human body movements into five areas; Based on the core body parts extracted in step 1, the five regions are divided into core participating parts and non-core participating parts; the specific operation is as follows: (221) According to the classification results of the large language model, determine whether each area is a core part; (222) The same feature extraction and processing operations are performed on five independent body parts, and in the subsequent step 3, the parameter λ 核心 and λ 非核心 The weights are adjusted so that the weights of the core participating parts are greater than those of the non-core participating parts, thereby enhancing the model's attention to the core parts.

6. The method for generating 3D human motion driven by text based on diffusion model and large language model according to claim 1, characterized in that: In step 3, the word-level text features extracted in step 1 are combined with the motion features of the five independent body parts in step 2 through a multi-head cross-attention mechanism to achieve fine-grained adjustment of each body part. The specific operations are as follows: (311) Mapping the motion features of each body part to the same latent space as the word-level text features through a linear mapping layer to obtain the motion features in the latent space; (312) applying a multi-head cross attention mechanism, combining the word-level text features with the motion features of each body part in the latent space obtained in the above step (311), and obtaining the fusion features of each body part; (313) The encoder based on the Transformer architecture is used to process the fusion features of each body part to obtain temporally continuous and fine-grained motion features; In the refinement layer, the loss function is used During the training process, the parameter λ is adjusted 核心 and λ 非核心 The weights are used to distinguish and optimize the core and non-core parts. The specific operations are as follows: (321) Determine the distribution of core and non-core parts by analyzing the action descriptions in the input text; (322) Dynamically adjust the loss weights λ of the core and non-core parts 核心 and 非核心 , so that the model prioritizes the atomic actions corresponding to the core parts.

7. The method for generating 3D human motion driven by text based on diffusion model and large language model according to claim 1, characterized in that: In step 4, the refined local motion features are synthesized into complete human motion using the integration layer. The specific operations are as follows: (411) The fusion features of each body part obtained by the refinement layer in step 3 are passed through respective independent mapping layers to restore the motion features of the latent space into the position information of the joint points corresponding to different body parts, i.e., the motion representation of each body part; then, according to the composition method corresponding to the body partitions, the motion representations of the five independent parts are merged to generate a complete whole body motion representation; (412) The complete whole-body motion representation is converted into an embedding representation consistent with the sentence-level feature dimension through a mapping layer, and then fused with the sentence-level feature to obtain the concatenated whole-body motion feature; (413) By using an encoder with a deeper encoder layer than the Transformer architecture in the refinement layer, the concatenated full-body motion features are further optimized and learned to obtain a diffusion model.

8. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the text-driven three-dimensional human motion generation method based on a diffusion model and a large language model as described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the text-driven three-dimensional human motion generation method based on a diffusion model and a large language model as described in any one of claims 1-7.

Images