A method and system for generating human motion based on a multi-token large language model

Abstract

This invention relates to the field of computer software technology and discloses a method and system for generating human motion based on a multi-token large language model. The construction and training process of the human motion generation model includes mapping user-input text descriptions to text features, and sharing a backbone network, multiple neck networks, and a prediction head with task prompts and text descriptions as input; encoding the human motion corresponding to the text description as motion features, and narrowing the distance between text features and motion features through contrastive learning; generating human motion by using multiple motion tokens through motion dediscretization and decoding techniques; and performing preference alignment based on dynamic boundaries according to human preferences. This invention enhances the modeling ability for sparse motion tokens through a multi-token prediction strategy and introduces cross-modal features to guide global semantic alignment. Simultaneously, this invention proposes a human preference optimization method with similarity offset to improve the alignment between generated actions and human preferences.

Description

Technical Field

[0001] This invention relates to the field of computer software technology, and specifically to a method and system for generating human motion based on a multi-token large language model. Background Technology

[0002] In recent years, motion generation models have achieved remarkable success in generating realistic and imaginative human movements based on textual descriptions. Therefore, this type of human motion generation technology has been widely applied in various fields, including filmmaking and virtual reality.

[0003] Text-driven human motion generation technology aims to transform text descriptions into 3D human motion sequences. A key challenge lies in bridging the semantic gap between natural language and 3D human motion, two heterogeneous modalities. 1) Early models employed a recurrent neural network-based architecture combined with a curriculum learning strategy to achieve the conversion from text to motion sequences, but the generated results showed significant deficiencies in motion quality and global displacement. Later, researchers effectively improved the diversity of motion sequences by introducing variational autoencoders and enhanced text encoders. Recently, researchers constructed the large-scale dataset HumanML3D and proposed a joint framework of motion length prediction-based generation methods and generative pre-trained Transformers, while also designing a temporal model-based evaluator to measure generation quality. 2) With the development of diffusion models, researchers have implemented motion generation with multiple conditional inputs based on a diffusion generation architecture. Motion generation technology based on diffusion models mainly consists of two processes: forward and backward. In the forward process, Gaussian-distributed noise is gradually added to the motion features at each time step. In the backward process, a deep neural network is typically used as a noise estimator, using text features as conditions to predict the noise added at each time step. A noise estimator is trained by minimizing the estimated noise compared to the actual noise. During inference, the noise estimator predicts noise from a Gaussian-distributed noise system based on text features, performs denoising, and finally obtains action features. The final generated action is then obtained through action decoding technology. 3) Although the above two methods have achieved significant results in the field of human action generation, their specialized model architecture limits the possibility of unified cross-modal modeling and is not conducive to building a general generative model for multiple tasks. Today, large language models, represented by the ChatGPT series, have demonstrated strong generalization capabilities, reaching new heights in task adaptability and handling complex problems. Inspired by this, motion generation methods based on large language models have gradually become a research hotspot. Researchers have proposed treating human actions as special action tokens, discretizing three-dimensional motion into action tokens through vector quantization, achieving unified modeling of motion and text, significantly improving the ability to model cross-modal relationships, and supporting a wider range of motion-related tasks.

[0004] However, the rapid development of action generation technology based on large language models has also brought new challenges, namely how to make the generated actions highly correlated with the text descriptions input by humans. Related technologies (such as multi-token alignment, reinforcement learning, etc.) have shown good human preference alignment capabilities in large language models (such as ChatGPT, DeepSeek, Doubao, etc.), but have not been fully explored in the field of human motion generation. This technological gap seriously restricts the practical effect of existing methods. Summary of the Invention

[0005] While existing text-driven human action generation technologies have made significant progress in action quality and diversity, they still face serious challenges in semantic alignment and human preference. The main drawbacks of existing methods are: 1) Insufficient modeling of short-term relationships between action tokens: Traditional large language model methods employ single-token prediction strategies, which struggle to effectively capture short-term dependencies between sparse action tokens, leading to disjointed generated actions; 2) Poor global semantic consistency: A semantic gap exists between text descriptions and action sequences, and existing methods lack effective cross-modal guidance mechanisms, making it difficult to guarantee global consistency between generated actions and textual intent; 3) Insufficient alignment with human preferences: Existing training paradigms do not fully consider human subjective preferences, resulting in quality issues such as mirroring errors and incomplete actions in generated actions.

[0006] To address the aforementioned technical problems, this invention provides a method and system for generating human motion based on a multi-token large language model. It enhances the modeling capability for sparse action tokens through a multi-token prediction strategy and introduces cross-modal features to guide global semantic alignment. Furthermore, this invention proposes a human preference optimization method with similarity offset to improve the alignment between generated actions and human preferences.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] Firstly, this invention provides a method for generating human motion based on a multi-token large language model. The construction and training process of the human motion generation model includes:

[0009] The text encoder maps the user-inputted text description into text features, and together with the task prompt words and the text description, it is input into a shared backbone network based on a large language model, multiple parallel neck networks, and a prediction head, predicting and outputting multiple action tokens.

[0010] An action encoder is introduced during the training phase to encode the human actions corresponding to the text description into action features. Comparative learning is used to narrow the gap between text features and action features.

[0011] Human actions are generated based on the multiple action tokens using action dediscretion and decoding technology.

[0012] Based on human preferences, the generated human actions are divided into preferred generated actions and averse generated actions. The human action generation model is optimized by performing preference alignment based on dynamic boundaries through the semantic similarity between the text description and the preferred and averse generated actions.

[0013] In one embodiment, the text encoder maps the user-inputted text description to text features, and inputs these features, along with task prompts and the text description, into a shared backbone network based on a large language model, multiple parallel neck networks, and a prediction head. The predicted output includes multiple action tokens, specifically:

[0014] The shared backbone network includes a large language model and fine-tuning branches built on a low-rank matrix;

[0015] The basic features extracted by the shared backbone network are used to predict multiple future action features through multiple parallel neck networks, and the prediction head outputs the prediction results of multiple future action tokens.

[0016] In one embodiment, the fine-tuning branch based on the low-rank matrix specifically includes:

[0017] Trainable low-rank matrices are inserted as bypasses into the intermediate layers of the large language model; when training the shared backbone network, the original weights W of the backbone network of the intermediate layers of the large language model are frozen, and only the parameters of the low-rank matrices are updated.

[0018] In one embodiment, the optimization of the human action generation model by performing preference alignment based on dynamic boundaries using the semantic similarity between text descriptions and preferred / disliked actions specifically includes:

[0019] Calculate text description Text features and preference generation actions semantic similarity of action features Calculate the text features of the text description and the aversion generation action. semantic similarity of action features ;

[0020] The human motion generation model is optimized using the following formula:

[0021] ;

[0022] Represents the optimization function. Represents the mathematical expectation. This represents the Sigmoid function. It is an adjustable parameter. This represents the mathematical distribution of the output of the current human motion generation model. This represents the mathematical distribution of the reference model output. Indicate preferences to generate actions The corresponding action token, Indicates aversion to generating actions The corresponding action token; This refers to all inputs to the human motion generation model, including text descriptions, text features, and task prompts. This indicates that for the current human motion generation model, given input... At that time, generate The mathematical probability; This means that for the reference model, given the input At that time, generate The mathematical probability; Represents an adjustable logarithmic function used to... Mapped to the corresponding dynamic boundary .

[0023] In one embodiment, the computational text description Text features and preference generation actions semantic similarity of action features Calculate the text features of the text description and the aversion generation action. semantic similarity of action features Specifically, it includes:

[0024] ;

[0025] ;

[0026] Represents text description Textual features, Indicate preferences to generate actions The characteristics of movement, Indicates aversion to generating actions The characteristics of the movement; For text encoders, For motion encoders.

[0027] Secondly, this invention provides a human motion generation system based on a multi-token large language model, wherein the construction and training process of the human motion generation model includes:

[0028] The action token generation module maps the user-input text description into text features through a text encoder, and inputs them together with the task prompt words and the text description into a shared backbone network, multiple parallel neck networks and a prediction head based on a large language model, to predict and output multiple action tokens.

[0029] The basic training module introduces an action encoder during the training phase to encode the human actions corresponding to the text descriptions into action features. By comparing and learning, the distance between text features and action features is narrowed.

[0030] The human motion generation module generates human motion based on the multiple motion tokens using motion dediscretion and decoding technology;

[0031] The preference alignment training module distinguishes generated human actions into preferred generated actions and averse generated actions based on human preferences. It optimizes the human action generation model by performing preference alignment based on dynamic boundaries through the semantic similarity between text descriptions and preferred and averse generated actions.

[0032] In one embodiment, the text encoder maps the user-inputted text description to text features, and inputs these features, along with task prompts and the text description, into a shared backbone network based on a large language model, multiple parallel neck networks, and a prediction head. The predicted output includes multiple action tokens, specifically:

[0033] The shared backbone network includes a large language model and fine-tuning branches built on a low-rank matrix;

[0034] The basic features extracted by the shared backbone network are used to predict multiple future action features through multiple parallel neck networks, and the prediction head outputs the prediction results of multiple future action tokens.

[0035] In one embodiment, the fine-tuning branch based on the low-rank matrix specifically includes:

[0036] Trainable low-rank matrices are inserted as bypasses into the intermediate layers of the large language model; when training the shared backbone network, the original weights W of the backbone network of the intermediate layers of the large language model are frozen, and only the parameters of the low-rank matrices are updated.

[0037] In one embodiment, the optimization of the human action generation model by performing preference alignment based on dynamic boundaries using the semantic similarity between text descriptions and preferred / disliked actions specifically includes:

[0038] Calculate text description Text features and preference generation actions semantic similarity of action features Calculate the text features of the text description and the aversion generation action. semantic similarity of action features ;

[0039] The human motion generation model is optimized using the following formula:

[0040] ;

[0041] Represents the optimization function. Represents the mathematical expectation. This represents the Sigmoid function. It is an adjustable parameter. This represents the mathematical distribution of the output of the current human motion generation model. This represents the mathematical distribution of the reference model output. Indicate preferences to generate actions The corresponding action token, Indicates aversion to generating actions The corresponding action token; This refers to all inputs to the human motion generation model, including text descriptions, text features, and task prompts. This indicates that for the current human motion generation model, given input... At that time, generate The mathematical probability; This means that for the reference model, given the input At that time, generate The mathematical probability; Represents an adjustable logarithmic function used to... Mapped to the corresponding dynamic boundary .

[0042] In one embodiment, the computational text description Text features and preference generation actions semantic similarity of action features Calculate the text features of the text description and the aversion generation action. semantic similarity of action features Specifically, it includes:

[0043] ;

[0044] ;

[0045] Represents text description Textual features, Indicate preferences to generate actions The characteristics of movement, Indicates aversion to generating actions The characteristics of the movement; For text encoders, For motion encoders.

[0046] The system and method in this invention correspond to each other; the specific technical solutions applicable to the method are also applicable to the system.

[0047] Compared with the prior art, the beneficial technical effects of the present invention are:

[0048] 1. This invention addresses the problem of insufficient short-term relationship modeling between action tokens in current action generation methods, providing an end-to-end solution. By proposing a multi-token prediction strategy, this invention significantly improves the short-term modeling capability of generated actions, providing strong support for the application of action generation technology.

[0049] 2. This invention addresses the problems in current methods, such as the semantic gap between text descriptions and action sequences, the lack of effective cross-modal guidance mechanisms, and the difficulty in ensuring global consistency between generated actions and user input text. It proposes a cross-modal guidance mechanism. By guiding through textual features at the sequence level, this invention can significantly improve the global consistency of generated actions, thereby significantly enhancing the consistency between generated actions and user prompts without compromising the quality of generated actions.

[0050] 3. This invention addresses the quality issues of current methods, which often fail to adequately consider human subjective preferences, leading to mirroring errors and incomplete actions in generated actions. It proposes a general preference alignment framework. This framework improves the preference alignment of generated actions by introducing dynamic semantic similarity boundaries, significantly increasing user satisfaction with the generated actions. Attached Figure Description

[0051] Figure 1 This is a flowchart of the method in an embodiment of the present invention.

[0052] Figure 2 This is a schematic diagram of the multi-token prediction process and cross-modal guidance process in an embodiment of the present invention.

[0053] Figure 3 This is a schematic diagram of the preference alignment stage in an embodiment of the present invention. Detailed Implementation

[0054] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0055] like Figure 1 As shown, this invention provides a method for generating human motion based on a multi-token large language model. The construction and training process of the human motion generation model includes the following steps:

[0056] S1, the text description input by the user is mapped to text features by the text encoder, and together with the task prompt words and the text description, it is input into a shared backbone network based on a large language model, multiple parallel neck networks and a prediction head, and the prediction outputs multiple action tokens.

[0057] S2 introduces an action encoder during the training phase to encode the human actions corresponding to the text description into action features, and narrows the gap between text features and action features through comparative learning.

[0058] S3, Based on the multiple action tokens, human actions are generated using action dediscretion and decoding technology;

[0059] S4 categorizes generated human actions into preferred and disliked actions based on human preferences. The human action generation model is then optimized by performing preference alignment based on dynamic boundaries using the semantic similarity between text descriptions and preferred / disliked actions.

[0060] This invention consists of three parts: multi-token prediction, cross-modal guidance, and preference alignment. Each part will be described in detail below.

[0061] Multi-token prediction. To address the issue of insufficient modeling of short-term relationships between action tokens, multi-token prediction employs a parallel prediction architecture. For example... Figure 2 As shown on the left, multi-token prediction first employs a shared backbone network. Extract basic features; then use multiple parallel neck networks Predict multiple future action features separately; finally, predict the head... Output the final prediction results of multiple future action tokens.

[0062] The shared backbone network includes a large language model and fine-tuning branches. In a preferred embodiment, the large language model can be a commonly used large language model, such as Llama, Qwen, etc.

[0063] The fine-tuning branch is a bypass structure implemented using low-rank matrices A and B. Specifically, trainable low-rank matrices A and B are inserted into the intermediate layers of the large language model as bypasses. The original weights W of the backbone network in the intermediate layers of the large language model are frozen (not participating in gradient updates), and only A and B are trained. After fine-tuning, the actual output of the intermediate layers of the large language model is the superposition of the output of the backbone network and the bypass output. The fine-tuning branch requires training only a very small number of parameters (low-rank matrices), significantly reducing computational costs. The behavior of the backbone network is adjusted through the fine-tuning branch to adapt it to a specific human task without modifying the original model. Freezing the backbone network preserves pre-trained knowledge, while the bypass branch only learns task-related incremental changes.

[0064] In a preferred embodiment, each neck network consists of two linear layers, each followed by a SiLU (Sigmoid Linear Unit) activation function; the prediction head uses a linear layer plus a Softmax activation function.

[0065] Cross-modal guidance. To address the semantic consistency issue, this invention employs feature-consistent alignment to strengthen the semantic relationship between text and actions. For example... Figure 2 As shown on the right, a text encoder is first used. Text description Mapping to text features Then through the motion encoder The action corresponding to the text description Encoding as action features By using comparative learning, text features are brought closer together. With action characteristics The distance between them enables cross-modal semantic consistency and strengthens the alignment between text and actions. During inference, text features... As part of the input to the large language model, action tokens are used to guide the output of the large language model to better satisfy human semantics. Note that the action tokens can be directly used to obtain the final 3D human action through subsequent action dediscretion and decoding techniques.

[0066] In a preferred embodiment, the text encoder can employ a network model consisting of 12 Transformer layers; the motion encoder is also based on a network model consisting of 12 Transformer layers.

[0067] Preference alignment. For example... Figure 3 As shown, to further improve the alignment between generated human actions and human preferences, this invention introduces a dynamic semantic similarity boundary to enhance the preference alignment of model-generated actions. Firstly, based on text description... And preferences generate actions Dislike generating actions The semantic similarity between them is calculated as follows:

[0068] ;

[0069] ;

[0070] For text description Text features and preference generation actions The semantic similarity of action features, Text features and aversion generation actions for text description The semantic similarity of action features Represents text description Textual features, Indicate preferences to generate actions The characteristics of movement, Indicates aversion to generating actions The characteristics of its movements.

[0071] Next, based on the above similarity, the human motion generation model is optimized using the following formula:

[0072] ;

[0073] Represents the optimization function. Represents the mathematical expectation. This represents the Sigmoid function. It is an adjustable parameter. This represents the mathematical distribution of the output of the current human motion generation model. This represents the mathematical distribution of the reference model output. Indicate preferences to generate actions The corresponding action token, Indicates aversion to generating actions The corresponding action token; This refers to all inputs to the human motion generation model, including text descriptions, text features, and task prompts. This indicates that for the current human motion generation model, given input... At that time, generate The mathematical probability; This means that for the reference model, given the input At that time, generate The mathematical probability; Represents an adjustable logarithmic function used to... Mapped to the corresponding dynamic boundary .

[0074] Through the above steps, the present invention can significantly improve the quality of action generation, ensuring the consistency of generated actions in the semantic dimension and the overall temporal smoothness.

[0075] The multi-token prediction strategy proposed in this invention effectively enhances the ability of large language models to model short-term relationships between sparse action tokens, while also improving generation efficiency.

[0076] This invention designs a cross-modal guidance mechanism that significantly improves the global consistency of generated actions by guiding text features at the sequence level.

[0077] This invention designs a preference alignment stage, which innovatively introduces semantic similarity as a dynamic boundary into the preference alignment process to achieve more refined human preference modeling.

[0078] To better illustrate the human motion generation method based on a multi-token large language model proposed in this invention, the following example uses the user-inputted text description "a man kicks with his left leg" to further explain the invention.

[0079] Figure 2 This invention demonstrates the internal structure and training method of the human motion generation model, including two parts: multi-token prediction and cross-modal guidance.

[0080] Figure 3 This invention demonstrates the preference alignment phase process proposed in this invention, and the internal structure of the human motion generation model is shown in [reference needed]. Figure 2 , Figure 3 This document provides a detailed introduction to how training data is used during the training process and the preference alignment training workflow.

[0081] The training process of the human motion generation model is described in detail below:

[0082] Step 1. As Figure 2 As shown, by encoding elements such as the input task prompt "generate an action token sequence that matches the following description" and the text description "a man kicks with his left leg," a shared backbone network is used. Extract basic features, then pass them through multiple parallel neck networks The neck network in this embodiment predicts multiple future action features, and consists of four networks, denoted as follows: Finally, by predicting the head Output the prediction results of multiple future action tokens, denoted as follows: Due to the multi-token prediction mechanism, this invention can output multiple action tokens each time, for example... At any given moment, the human motion generation model simultaneously outputs motion tokens 2, 1, 4, and 9. The number of motion tokens in the figure is for illustrative purposes only; the actual number of parallel operations can be adjusted. Motion tokens 2, 1, 4, and 9 are not simply numbers, but rather indexes of discretized motion primitives. For example, motion token 2 can represent the second motion primitive (such as raising a leg).

[0083] Step 2. Cross-modal guidance during the training phase. If only task cues and text descriptions are used as input, the consistency between the resulting actions and text is insufficient. Therefore, to address the semantic consistency problem, this invention employs feature-consistent alignment to strengthen the semantic relationship between text and actions. For example... Figure 2 As shown, cross-modal guidance first employs a text encoder during the training phase. Text description Mapping to text features Then through the motion encoder Actions Encoding as action features By using comparative learning, text features are brought closer together. With action characteristics This distance between them enables cross-modal semantic consistency and strengthens the alignment between text and actions. It also brings text features closer together. With action characteristics When calculating the distance between them, text features can be calculated first. With action characteristics The cosine similarity between them is calculated, and the cosine similarity is optimized to be close to 1.

[0084] During the inference phase, the action encoder for the cross-modal guided process is not used; only the text encoder and corresponding text features are used. As part of the input tokens for the large language model, action tokens guide the output of the large language model to better satisfy human semantics. Note that action tokens can directly yield the final 3D human action during the inference phase through action dediscretion and decoding techniques, but they do not need to be considered during the training phase.

[0085] Step 3. Preference alignment training in the second training phase. After initial training of the human motion generation model based on Steps 1 and 2, as follows... Figure 3 As shown, this invention introduces a human preference alignment stage to further train the human action generation model. This training stage introduces similarity preference for training, based on the direct preference learning method commonly used in reinforcement learning.

[0086] After training, the human motion generation model is used for inference, specifically including:

[0087] Use a text encoder User-input text description Mapping to text features ;

[0088] Text description Encoded as a text token, with text features Together with the task prompts, they serve as input to the human motion generation model, and motion tokens are obtained through parallel output.

[0089] The obtained action tokens are then used to obtain the final three-dimensional human motion through action dediscretion and decoding technology.

[0090] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0091] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0092] Based on the description of the above method embodiments, the present invention also provides a system. The system may be a system that uses software (applications), modules, components, servers, clients, etc., using the methods described in the embodiments of this specification, combined with necessary implementation hardware. Based on the same innovative concept, the systems in one or more embodiments provided in this disclosure are as described in the following embodiments. Since the implementation schemes and methods for solving the problem are similar, the specific system implementations in the embodiments of this specification can refer to the implementations of the foregoing methods, and repeated details will not be repeated. As used below, the terms "module" or "module group" refer to a combination of software and / or hardware capable of implementing a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0093] A human motion generation system based on a multi-token large language model, the construction and training process of the human motion generation model includes:

[0094] The action token generation module maps the user-input text description into text features through a text encoder, and inputs them together with the task prompt words and the text description into a shared backbone network, multiple parallel neck networks and a prediction head based on a large language model, to predict and output multiple action tokens.

[0095] The basic training module introduces an action encoder during the training phase to encode the human actions corresponding to the text descriptions into action features. By comparing and learning, the distance between text features and action features is narrowed.

[0096] The human motion generation module generates human motion based on the multiple motion tokens using motion dediscretion and decoding technology;

[0097] The preference alignment training module distinguishes generated human actions into preferred generated actions and averse generated actions based on human preferences. It optimizes the human action generation model by performing preference alignment based on dynamic boundaries through the semantic similarity between text descriptions and preferred and averse generated actions.

[0098] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0099] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0100] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for generating human motion based on a multi-token large language model, characterized in that, The construction and training process of the human motion generation model used includes: The text encoder maps the user-inputted text description to text features, and together with the task prompt words and the text description, it is input into a shared backbone network based on a large language model, multiple parallel neck networks, and a prediction head, predicting and outputting multiple action tokens. An action encoder is introduced during the training phase to encode the human actions corresponding to the text description into action features. Comparative learning is used to narrow the gap between text features and action features. Human actions are generated based on the multiple action tokens using action dediscretion and decoding technology. Based on human preferences, generated human actions are categorized into preferred generated actions and averse generated actions. Preference alignment based on dynamic boundaries is performed using the semantic similarity between text descriptions and preferred / averse generated actions to optimize the human action generation model. Specifically, this includes: Calculate text description Text features and preference generation actions semantic similarity of action features Calculate the text features of the text description and the aversion generation action. semantic similarity of action features The human motion generation model is optimized using the following formula: ； Represents the optimization function. Represents the mathematical expectation. This represents the Sigmoid function. It is an adjustable parameter. This represents the mathematical distribution of the output of the current human motion generation model. This represents the mathematical distribution of the reference model output. Indicate preferences to generate actions The corresponding action token, Indicates aversion to generating actions The corresponding action token; This refers to all inputs to the human motion generation model, including text descriptions, text features, and task prompts. This indicates that for the current human motion generation model, given input... At that time, generate The mathematical probability; This means that for the reference model, given the input At that time, generate The mathematical probability; Represents an adjustable logarithmic function used to... Mapped to the corresponding dynamic boundary .

2. The method for generating human motion based on a multi-token large language model according to claim 1, characterized in that, The process involves mapping user-inputted text descriptions to text features via a text encoder, and then inputting these features, along with task prompts and the text descriptions, into a shared backbone network based on a large language model, multiple parallel neck networks, and a prediction head. The predicted output includes multiple action tokens, specifically: The shared backbone network includes a large language model and fine-tuning branches built on a low-rank matrix; The basic features extracted by the shared backbone network are used to predict multiple future action features through multiple parallel neck networks, and the prediction head outputs the prediction results of multiple future action tokens.

3. The method for generating human motion based on a multi-token large language model according to claim 2, characterized in that, The fine-tuning branch based on the low-rank matrix specifically includes: Trainable low-rank matrices are inserted as bypasses into the intermediate layers of the large language model; when training the shared backbone network, the original weights W of the backbone network of the intermediate layers of the large language model are frozen, and only the parameters of the low-rank matrices are updated.

4. The method for generating human motion based on a multi-token large language model according to claim 1, characterized in that, The computational text description Text features and preference generation actions semantic similarity of action features Calculate the text features of the text description and the aversion generation action. semantic similarity of action features Specifically, it includes: ； ； Represents text description Textual features, Indicate preferences to generate actions The characteristics of movement, Indicates aversion to generating actions The characteristics of the movement; For text encoders, For motion encoders.

5. A human motion generation system based on a multi-token large language model, characterized in that, The construction and training process of the human motion generation model used includes: The action token generation module maps the user-input text description into text features through a text encoder, and inputs them together with the task prompt words and the text description into a shared backbone network, multiple parallel neck networks and a prediction head based on a large language model, to predict and output multiple action tokens. The basic training module introduces an action encoder during the training phase to encode the human actions corresponding to the text descriptions into action features. By comparing and learning, the distance between text features and action features is narrowed. The human motion generation module generates human motion based on the multiple motion tokens using motion dediscretion and decoding technology; The preference alignment training module distinguishes generated human actions into preferred and averse actions based on human preferences. It optimizes the human action generation model by performing preference alignment based on dynamic boundaries using the semantic similarity between text descriptions and preferred / averse actions. Specifically, this includes calculating the text descriptions... Text features and preference generation actions semantic similarity of action features Calculate the text features of the text description and the aversion generation action. semantic similarity of action features The human motion generation model is optimized using the following formula: ； Represents the optimization function. Represents the mathematical expectation. This represents the Sigmoid function. It is an adjustable parameter. This represents the mathematical distribution of the output of the current human motion generation model. This represents the mathematical distribution of the reference model output. Indicate preferences to generate actions The corresponding action token, Indicates aversion to generating actions The corresponding action token; This refers to all inputs to the human motion generation model, including text descriptions, text features, and task prompts. This indicates that for the current human motion generation model, given input... At that time, generate The mathematical probability; This means that for the reference model, given the input At that time, generate The mathematical probability; Represents an adjustable logarithmic function used to... Mapped to the corresponding dynamic boundary .

6. A human motion generation system based on a multi-token large language model according to claim 5, characterized in that, The process involves mapping user-inputted text descriptions to text features via a text encoder, and then inputting these features, along with task prompts and the text descriptions, into a shared backbone network based on a large language model, multiple parallel neck networks, and a prediction head. The predicted output includes multiple action tokens, specifically: The shared backbone network includes a large language model and fine-tuning branches built on a low-rank matrix; The basic features extracted by the shared backbone network are used to predict multiple future action features through multiple parallel neck networks, and the prediction head outputs the prediction results of multiple future action tokens.

7. A human motion generation system based on a multi-token large language model according to claim 6, characterized in that, The fine-tuning branch based on the low-rank matrix specifically includes: Trainable low-rank matrices are inserted as bypasses into the intermediate layers of the large language model; when training the shared backbone network, the original weights W of the backbone network of the intermediate layers of the large language model are frozen, and only the parameters of the low-rank matrices are updated.

8. A human motion generation system based on a multi-token large language model according to claim 5, characterized in that, The computational text description Text features and preference generation actions semantic similarity of action features Calculate the text features of the text description and the aversion generation action. semantic similarity of action features Specifically, it includes: ； ； Represents text description Textual features, Indicate preferences to generate actions The characteristics of movement, Indicates aversion to generating actions The characteristics of the movement; For text encoders, For motion encoders.

Images