A 3D human motion sequence generation method and system, and a model training method

Abstract

This invention discloses a method and system for generating 3D human motion sequences, as well as a model training method. The method includes the following steps: obtaining a first text description input by the user and inputting it into a text refinement model to obtain a second text description; the text refinement model is used to transform coarse-grained short text into fine-grained long text. The second text description is input into a pre-trained generation model to output a first motion sequence. The contrastive reinforcement learning module divides the reward function into several dimensions and calculates the reward value of the second text description under each dimension based on a baseline contrast mechanism; the parameters of the text refinement model are fine-tuned based on the reward value, and the first motion sequence is reconstructed to obtain a 3D human motion sequence. Compared with traditional technologies, the second text description output by the text refinement model is more accurate and rigorous, and the illusion and bias information in its output distribution can be suppressed by fine-tuning the text refinement model.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and more specifically, to a method and system for generating 3D human motion sequences and a model training method. Background Technology

[0002] Text-driven 3D human motion sequence generation technology has wide applications in animation, virtual reality, and embodied intelligence. With the continuous development of deep learning algorithms such as multimodal understanding and generative models, current text-driven 3D human motion sequence generation models can generate high-quality 3D human motion sequences that conform to the text descriptions, guided by short (or coarse-grained) descriptions. However, the motion information contained in such texts is limited and cannot describe actions in the real world that contain complex temporal relationships and fine-grained step descriptions, such as the detailed operation steps of a robot performing a task. Such actions require long texts containing more fine-grained information for description.

[0003] Current research extends short text descriptions of existing action sequences to create fine-grained long text descriptions. Models that generate 3D human action sequences based on these extended datasets can be trained using these fine-grained long text descriptions. However, these models require the same fine-grained long text descriptions as the training data, which is time-consuming and laborious for users. Furthermore, due to the difficulty in obtaining all textual features (grammar, writing structure, and vocabulary) of the training data, the results generated by the model under the guidance of manually written text descriptions may be of low quality and deviate from the user's intent.

[0004] Existing technology provides a training method, generation method, and system for a human motion sequence generation model. By introducing a stick figure as an intermediate control structure, and combining a diffusion model with a multi-layer, multi-condition fusion module, text, stick figures, and noisy motion sequences are input and processed hierarchically. An attention mechanism is used to dynamically adjust the influence of various conditions on the motion vector, reducing reliance on complex text descriptions and improving generation quality and efficiency. However, this existing technology has the following problems: the training text has limited information capacity and cannot describe actions in the real world that contain complex temporal relationships and fine-grained step descriptions.

[0005] Therefore, in light of the above requirements and the shortcomings of existing technologies, this application proposes a method and system for generating 3D human motion sequences, as well as a model training method. Summary of the Invention

[0006] This invention provides a method and system for generating 3D human motion sequences, as well as a model training method. It introduces a baseline comparison mechanism into the reward function of reinforcement learning. The reward is calculated by comparing the sampling results of the text refinement model with the baseline samples in terms of generation effect. Finally, the results are fed back to the text refinement model. The comparison with the baseline samples ensures that the model can more accurately judge the correctness and rigor of the sampling results and suppress illusions and bias information in the output distribution.

[0007] The primary objective of this invention is to solve the aforementioned technical problems. The technical solution of this invention is as follows: The first aspect of this invention provides a method for generating 3D human motion sequences, the method comprising the following steps: S1. Obtain the first text description input by the user and input it into the text refinement model to obtain the second text description; the text refinement model is used to transform coarse-grained short text into fine-grained long text.

[0008] S2. Input the second text description into the pre-trained generative model and output the first action sequence.

[0009] S3. Input the second text description and the first action sequence into the contrastive reinforcement learning module. The contrastive reinforcement learning module divides the reward function into several dimensions and calculates the reward value of the second text description under each dimension based on the baseline contrast mechanism. Based on the reward value, the parameters of the text refinement model are fine-tuned. Based on the fine-tuned text refinement model, the second text description is updated and the first action sequence is reconstructed to obtain the 3D human action sequence.

[0010] Furthermore, the first text description is a coarse-grained short text description, and the second text description is a fine-grained long text description; the text refinement model is a large language model that includes only a first decoder, used to transform coarse-grained short text into fine-grained long text; the pre-trained generative model includes a first encoder and a second decoder, used to transform fine-grained long text into semantically aligned discrete action sequences, i.e., the first action sequence; the contrastive reinforcement learning module divides the reward function into text alignment reward, action sequence alignment reward, and format reward.

[0011] Further, step S3 specifically involves: based on the second text description, standardizing the reward values ​​of all samples corresponding to the same coarse-grained short text, and using the standardized reward values ​​as the advantage values ​​of each sample; updating the parameters of the text refinement model, increasing the output probability of samples with positive advantage values ​​in the text refinement model, and suppressing the output probability of samples with negative advantage values ​​in the text refinement model.

[0012] The sample is the fine-grained long text converted from the coarse-grained short text and the discrete action sequence generated by it, namely the first action sequence.

[0013] A second aspect of the present invention provides a model training method, the method comprising: S1. Discretely encode and reconstruct the 3D human motion sequence to obtain a discrete motion sequence.

[0014] S2. Obtain the coarse-grained short text description and fine-grained long text description corresponding to the 3D human motion sequence. Based on the training data pair of fine-grained long text description and discrete motion sequence, train the generation model. The generation model is used to generate 3D human motion sequence according to fine-grained long text description.

[0015] S3. Based on the labeled data pairs of coarse-grained short text descriptions and fine-grained long text descriptions, the text refinement model is fine-tuned. The text refinement model is used to convert coarse-grained short text descriptions into fine-grained long text descriptions.

[0016] The fine-tuning of the text refinement model includes supervised fine-tuning and reinforcement learning fine-tuning. Specifically, the reinforcement learning fine-tuning involves: for the input coarse-grained short text description, sampling multiple fine-grained long text samples using the current text refinement model; inputting each fine-grained long text sample into the generation model to obtain corresponding 3D human motion samples; calculating the reward value for each fine-grained long text sample based on a baseline comparison mechanism, where the labeled fine-grained long text description in the dataset serves as the baseline sample; comparing the score corresponding to the output of the generation model guided by each sample with the score corresponding to the output of the generation model guided by the baseline sample to obtain the reward value; calculating the intra-group advantage value based on the reward value of each sample; and updating the parameters of the text refinement model based on the advantage value.

[0017] Furthermore, the process of discretely encoding and reconstructing the 3D human motion sequence to obtain the discrete motion sequence is as follows: inputting the real 3D human motion sequence into the motion encoder to obtain the encoded vector sequence, quantizing the encoded vector sequence through the codebook to obtain the discrete motion sequence; inputting the discrete motion sequence into the motion decoder to obtain the reconstructed 3D human motion sequence, updating the parameters of the encoder and decoder based on the reconstruction loss, and updating the parameters of the codebook using exponential moving average and code reset methods.

[0018] Furthermore, training the generative model specifically involves using conditional language modeling as the optimization objective, which is used to minimize the autoregressive prediction error of discrete action sequences given a fine-grained long text description.

[0019] Furthermore, the supervised fine-tuning of the text refinement model specifically involves using autoregressive language modeling as the optimization objective. This objective is used to minimize the autoregressive prediction error of the fine-grained long text description under the input conditions of preset task prompts and coarse-grained short text descriptions.

[0020] Furthermore, the reward value for each fine-grained long text sample is calculated based on the baseline comparison mechanism, including: based on text alignment reward, comparing the cosine similarity between the action vector generated by the sample and the text vector corresponding to the coarse-grained short text with the cosine similarity corresponding to the baseline sample; if the similarity of the sample is not lower than the baseline, a positive reward is given; based on action alignment reward, comparing the cosine similarity between the action vector generated by the sample and the action vector corresponding to the real 3D human action with the cosine similarity corresponding to the baseline sample; if the similarity of the sample is not lower than the baseline, a positive reward is given; based on format reward, checking whether the output of the sample conforms to the predefined structured format; if it does, a positive reward is given; using a reference model with the same initial parameters as the text refinement model, the output distribution of the text refinement model is regularized using KL divergence to constrain the exploration degree of the model.

[0021] The action vector and text vector are obtained by encoding using a pre-trained action encoder and a pre-trained text encoder, respectively.

[0022] Further, this includes: based on text alignment reward, comparing the cosine similarity between the action vector generated by the sample and the text vector corresponding to the coarse-grained short text with the cosine similarity corresponding to the baseline sample; if the similarity of the sample is not lower than the baseline, a positive reward is assigned; based on action alignment reward, comparing the cosine similarity between the action vector generated by the sample and the action vector corresponding to the real 3D human action with the cosine similarity corresponding to the baseline sample; if the similarity of the sample is not lower than the baseline, a positive reward is assigned; based on format reward, checking whether the output of the sample conforms to a predefined structured format; if it does, a positive reward is assigned; using a reference model consistent with the initial parameters of the text refinement model, and calculating the intra-group advantage value based on the reward value of each sample using KL, including: standardizing the reward values ​​of all samples corresponding to the same coarse-grained short text, and using the standardized reward value as the advantage value of each sample; updating the parameters of the text refinement model with the optimization purpose of increasing the output probability corresponding to samples with positive advantage values ​​and suppressing the output probability corresponding to samples with negative advantage values.

[0023] A third aspect of the present invention provides a 3D human motion sequence generation system, the system comprising: a text refinement module, a discrete motion generation module, a human motion sequence generation module, and a model training module.

[0024] The text refinement module obtains the first text description input by the user and inputs it into the text refinement model to obtain the second text description. The text refinement model is used to transform coarse-grained short text into fine-grained long text. The discrete action generation module inputs the second text description into a pre-trained generation model and outputs a first action sequence, which is a discrete action sequence. The model training module fine-tunes the parameters of the text refinement model based on the second text description and the first action sequence. The human body action sequence generation module updates the second text description and reconstructs the first action sequence based on the fine-tuned text refinement model to obtain a 3D human body action sequence.

[0025] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention provides a method and system for generating 3D human motion sequences, as well as a model training method. The provided 3D human motion sequence generation method uses a text refinement model as the first layer of input structure. Users only need to input simple, coarse-grained short text to complete the task, without having to manually write long text descriptions containing complex temporal sequences and fine-grained steps, directly saving users' input costs. The use of reinforcement learning fine-tuning can alleviate the illusion phenomenon or bias information that occurs when supervised fine-tuning models or untrained models provide fine-grained long text descriptions.

[0026] The provided model training method directly compares the model's output generation effect with the benchmark through a baseline comparison reward mechanism, rather than relying on absolute similarity to judge quality. This can identify low-quality outputs with illusions or biases, avoiding and reducing the problem of erroneous text guiding the generation of incorrect actions, and ensuring that the lower limit of the model's output text quality is not lower than the labeled level of existing data. At the same time, within-group reward standardization is performed on sampled samples with the same coarse-grained input, eliminating the difference in reward scale between different input samples. This allows the model to accurately identify the relative merits of different outputs under the same input, avoiding the bias towards samples with high absolute rewards during the optimization process, significantly improving the accuracy of policy updates and accelerating the model's convergence speed. Attached Figure Description

[0027] Figure 1 This is a flowchart of a 3D human motion sequence generation method according to the present invention.

[0028] Figure 2 This is a flowchart of a model training method according to the present invention.

[0029] Figure 3 This is a schematic diagram illustrating the training of a model based on a contrastive reward function in one embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram of a 3D human motion sequence generation system according to the present invention. Detailed Implementation

[0031] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0032] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0033] Currently, to address the issue of low-quality human action sequences generated by models guided by handwritten text descriptions, which may deviate from user intent, some research utilizes existing large language models to refine and expand user descriptions through cue learning, or employs supervised fine-tuning based on short-text description-long-text description data to create an expert model that specifically expands short text descriptions into long text descriptions. While these methods simplify user input, their output may still contain illusions or biased information, leading the generative model to produce results that deviate from user intent or are of low quality. Therefore, it is necessary to investigate a model training method to post-train these large language models used for refining and optimizing user text. By evaluating the results generated by the generative model guided by the post-training method's output, and then feeding this evaluation back into the large language model, the model can adaptively optimize its output distribution, reducing illusions and biased information in the output content.

[0034] Before describing the invention, let me first give a general description of the model involved in each step of the invention: The Discrete Motion Sequence Autoencoder (Motion VQ-VAE) is used to discretely encode and reconstruct 3D human motion sequence data, consisting of a motion encoder (Motion VQ-VAE Encoder). VQ-VAE Codebook and Motion VQ-VAE Decoder Composition: A text refinement model expands coarse-grained short text descriptions into fine-grained long text descriptions containing temporal information and detailed action steps. It consists of a decoder-only large language model, which also serves as the text refinement model in the fine-tuning step. A generation model generates 3D human action sequences based on the fine-grained long text descriptions. It consists of an encoder-decoder language model and a pre-trained action encoder. ) and a pre-trained text encoder ( This is used to encode samples generated by the generative model, real action samples, and their corresponding real short text descriptions, facilitating the evaluation of the generation quality and text fidelity of the generated samples, as well as scoring the sampled samples of the text refinement model during reinforcement learning fine-tuning.

[0035] Example 1 like Figure 1 As shown, this invention provides a method for generating 3D human motion sequences, which includes the following steps: S1. Obtain the first text description input by the user and input it into the text refinement model to obtain the second text description; the text refinement model is used to transform coarse-grained short text into fine-grained long text.

[0036] S2. Input the second text description into the pre-trained generative model and output the first action sequence.

[0037] S3. Input the second text description and the first action sequence into the reinforcement learning fine-tuning module. The reinforcement learning fine-tuning module divides the reward function into several dimensions and calculates the reward value of the second text description under each dimension based on the baseline comparison mechanism. Based on the reward value, the parameters of the text refinement model are fine-tuned. Based on the fine-tuned text refinement model, the second text description is updated and the first action sequence is reconstructed to obtain the 3D human action sequence.

[0038] It should be noted that, in response to the above-mentioned difficulties, this invention proposes a reinforcement learning model fine-tuning method for fine-grained text-driven 3D human action sequence generation tasks. This method introduces a baseline comparison mechanism into the reward function of reinforcement learning. The reward is calculated by comparing the sampling results of the text refinement model with the baseline samples in terms of generation effect, and then feeding it back to the text refinement model. The comparison with the baseline samples ensures that the model can more accurately judge the correctness and rigor of the sampling results and suppress illusions and bias information in the output distribution.

[0039] The first text description is a coarse-grained short text description, and the second text description is a fine-grained long text description; the text refinement model is a large language model that includes only a first decoder, used to transform coarse-grained short text into fine-grained long text; the pre-trained generative model includes a first encoder and a second decoder, used to transform fine-grained long text into semantically aligned discrete action sequences, i.e., the first action sequence; the contrastive reinforcement learning module divides the reward function into text alignment reward, action sequence alignment reward, and format reward.

[0040] Step S3 specifically involves: based on the second text description, standardizing the reward values ​​of all samples corresponding to the same coarse-grained short text, and using the standardized reward values ​​as the advantage values ​​of each sample; updating the parameters of the text refinement model, increasing the output probability of samples with positive advantage values ​​in the text refinement model, and suppressing the output probability of samples with negative advantage values ​​in the text refinement model.

[0041] The sample is the fine-grained long text converted from the coarse-grained short text and the discrete action sequence generated by it, namely the first action sequence.

[0042] like Figure 2 As shown, the present invention also provides a model training method, the method comprising: S1. Discretely encode and reconstruct the 3D human motion sequence to obtain a discrete motion sequence.

[0043] S2. Obtain the coarse-grained short text description and fine-grained long text description corresponding to the 3D human motion sequence. Based on the training data pair of fine-grained long text description and discrete motion sequence, train the generation model. The generation model is used to generate 3D human motion sequence according to fine-grained long text description.

[0044] S3. Based on the labeled data pairs of coarse-grained short text descriptions and fine-grained long text descriptions, the text refinement model is fine-tuned. The text refinement model is used to convert coarse-grained short text descriptions into fine-grained long text descriptions.

[0045] The fine-tuning of the text refinement model includes supervised fine-tuning and reinforcement learning fine-tuning: For the input coarse-grained short text description, multiple fine-grained long text samples are obtained by sampling using the current text refinement model; each fine-grained long text sample is input into the generation model to obtain corresponding 3D human motion samples; a reward value for each fine-grained long text sample is calculated based on a baseline comparison mechanism, where the labeled fine-grained long text description in the dataset is used as the baseline sample; the score corresponding to the output obtained by the generation model guided by each sample is compared with the score corresponding to the output obtained by the generation model guided by the baseline sample to obtain the final reward value; the intra-group advantage value is calculated based on the reward value of each sample, and the parameters of the text refinement model are updated based on the advantage value.

[0046] The process of discretely encoding and reconstructing a 3D human motion sequence to obtain a discrete motion sequence is as follows: the real 3D human motion sequence is input into the motion encoder to obtain the encoded vector sequence; the encoded vector sequence is quantized using a codebook to obtain a discrete motion sequence; the discrete motion sequence is input into the motion decoder to obtain the reconstructed 3D human motion sequence; the parameters of the encoder and decoder are updated based on the reconstruction loss; and the parameters of the codebook are updated using an exponential moving average and code reset method.

[0047] The training of the generative model specifically involves using conditional language modeling as the optimization objective, which is used to minimize the autoregressive prediction error of discrete action sequences given a fine-grained long text description.

[0048] Supervised fine-tuning of the text refinement model specifically involves using autoregressive language modeling as the optimization objective. This objective aims to minimize the autoregressive prediction error of fine-grained long text descriptions under the input conditions of preset task prompts and coarse-grained short text descriptions.

[0049] The reward value for each fine-grained long text sample is calculated based on a baseline comparison mechanism, including: a text alignment reward, comparing the cosine similarity between the action vector generated by the sample and the text vector corresponding to the coarse-grained short text, and the cosine similarity between the sample and the baseline sample; if the similarity is not lower than the baseline, a positive reward is given; an action alignment reward, comparing the cosine similarity between the action vector generated by the sample and the action vector corresponding to the real 3D human action, and the cosine similarity between the sample and the baseline sample; if the similarity is not lower than the baseline, a positive reward is given; a format reward, checking whether the output of the sample conforms to a predefined structured format, and if so, a positive reward is given; a reference model with the same initial parameters as the text refinement model is used, and the output distribution of the text refinement model is regularized using KL divergence to constrain the exploration extent of the model.

[0050] The action vector and text vector are obtained by encoding using a pre-trained action encoder and a pre-trained text encoder, respectively.

[0051] Further, this includes: Based on text alignment reward, comparing the cosine similarity between the action vector generated by the sample and the text vector corresponding to the coarse-grained short text with the cosine similarity corresponding to the baseline sample; if the similarity of the sample is not lower than the baseline, a positive reward is assigned; Based on action alignment reward, comparing the cosine similarity between the action vector generated by the sample and the action vector corresponding to the real 3D human action with the cosine similarity corresponding to the baseline sample; if the similarity of the sample is not lower than the baseline, a positive reward is assigned; Based on format reward, checking whether the output of the sample conforms to a predefined structured format; if it does, a positive reward is assigned; Calculating the intra-group advantage value based on the reward value of each sample includes: standardizing the reward values ​​of all samples corresponding to the same coarse-grained short text, and using the standardized reward value as the advantage value of each sample; updating the text refinement model parameters with the optimization purpose of increasing the output probability corresponding to samples with positive advantage values ​​and suppressing the output probability corresponding to samples with negative advantage values.

[0052] Based on the above technical solution, this invention introduces a comparison mechanism between the generated results and those of labeled data into the reward function. This allows for a significant determination of which samples in a set produce guided human action sequences that outperform the labeled data, further increasing the probability of such samples being sampled. This minimizes erroneous information (illusions and biased content) in the model's output. Ultimately, when the model receives coarse-grained short text descriptions from users, it can generate more accurate fine-grained long text descriptions, leading to 3D human action sequences generated by the guided model that are more faithful to the user's intent.

[0053] Furthermore, the reinforcement learning fine-tuning method proposed in this invention, as a framework, can generalize to other text-driven 3D human motion sequence datasets by fine-tuning the generative model, text refinement model, and evaluation model. Moreover, the transfer of different modalities can be achieved by replacing the generative model, text refinement model, and evaluation model in the framework.

[0054] Meanwhile, the text refinement model proposed in this invention can directly refine a user's short text description into a long text description, eliminating the need for the user to write the long text description directly, thus saving the user's time. At the same time, because different outputs of the text refinement model are encouraged or suppressed during the fine-tuning process, its output is more accurate and contains fewer errors compared to directly using supervised fine-tuning models or untrained models.

[0055] Example 2 Based on the above embodiment 1, as Figure 3 As shown in the figure, this embodiment elaborates in detail the specific process of the training method for fine-grained text-driven 3D human motion sequence generation model based on contrastive reward function enhancement fine-tuning proposed in this invention.

[0056] The first step involves training a discrete motion sequence autoencoder based on the centralized motion sequence data in the dataset. Real 3D human motion sequence data is input into the motion encoder (Motion VQ-VAE Encoder) for encoding. The encoder outputs a vector sequence, which is then used to find the nearest vector sequence in Euclidean space within the codebook. This vector sequence replaces the encoder's output. The sequence formed by the indexes of this vector sequence in the codebook is the discrete motion sequence; the operation of obtaining this vector is called quantization. The motion decoder (Motion VQ-VAE Decoder) accepts the quantized vector sequence as input and outputs a reconstructed 3D human motion sequence. It calculates the reconstruction loss by comparing this reconstructed sequence with the input of the motion encoder (the real 3D human motion sequence) and updates the model parameters. For the codebook parameters, this embodiment uses exponential moving average (EMA) updates and codebook resets for updates.

[0057] The second step is to train the generative model. The task of the generative model is to convert fine-grained long text descriptions into semantically aligned discrete action sequences. To ensure that the generative model can generate high-quality results that conform to the input long text descriptions, it needs to be fine-tuned to adapt to this downstream task. Training data pairs for the generative model are constructed based on the fine-grained long text descriptions and the encoded discrete action sequence data: ( (discrete action sequence).

[0058]

[0059] Based on training data, an optimization objective is modeled using conditional language. , Represents fine-grained long text descriptions Condition information, Represents a discrete action sequence. Indicates a time step. for Length, Indicates a given Center front When the word and conditional information C are given, the first... The word is The probability of.

[0060] The third step is supervised fine-tuning of the text refinement model. The text refinement model optimizes coarse-grained short texts into longer texts containing finer-grained semantic information. To ensure the text refinement model receives more rewards in subsequent reinforcement fine-tuning processes, it needs supervised fine-tuning (i.e., cold-start training) to adapt it to the context of... generate This is a downstream task.

[0061] First, cold-start training data is constructed based on existing coarse-grained short text and fine-grained long text annotation data, consisting of ( , Training data pairs. Cold start training uses an autoregressive language modeling optimization objective. , Indicates a time step. The complete input to the model is represented as task prompts in this embodiment. and The orderly splicing, for Length, Indicates a given Center front When the word is 1, the 1st The word is The probability of.

[0062]

[0063] Cold start training through optimization This allows the text refinement model to better adapt to the aforementioned downstream tasks, ensuring that the model's output is related to action descriptions during subsequent reinforcement learning. At the same time, the formatted output is easy to parse and provide to the generative model to generate action sequences, thereby increasing the probability of obtaining rewards during training.

[0064] The fourth step involves fine-tuning the reinforcement learning process, sampling the text refinement model, and generating the model's output. Based on the current parameters, the text refinement model samples and generates a set of samples for each coarse-grained short text input. Each sample contains a fine-grained long text description. Due to the diversity of the large language model's output, the sample diversity within the same group is high, facilitating exploration by the text refinement model during reinforcement learning. To prevent forgetting of original knowledge during exploration, a reference model with the same initial parameters as the text refinement model but not involved in updates is used. KL divergence is used for regularization in the outputs of both models to constrain the exploration extent of the text refinement model. To measure the guiding effect of the sampling results within each group of the text refinement model on the generated model's output, the sampling results are input into the generated model. The reward value for different samples is calculated by combining the 3D human motion sequences generated by the generated model and reconstructed by the Motion VQ-VAE Decoder.

[0065] The fifth step is reinforcement learning fine-tuning and reward calculation. This embodiment constructs a reward function from three dimensions to evaluate the sampling results of the text refinement model: Text alignment reward based on baseline comparison mechanism :

[0066] Fine-grained text descriptions generated by text refinement models Input generation model The generation of discrete action sequences is guided by a decoder. Discrete action sequences are decoded into 3D human action sequences, and a pre-trained action encoder is used. Encoding 3D human motion sequences yields motion vectors, which are then compared with a pre-trained text encoder. coding The cosine similarity is calculated from the obtained text vectors. At the same time Replace with existing fine-grained long text descriptions in the dataset The same calculation process is used to obtain the cosine similarity. ,Will and To make a comparison, if If the result is 1, it indicates that the current sampling performance of the text refinement model is better than or equal to that of the labeled fine-grained text, and the reward value is 1; otherwise, it indicates that the current sampling performance of the text refinement model is worse than that of the labeled fine-grained text. This indicates the calculation of the cosine similarity between two vectors.

[0067] Baseline-based action sequence alignment reward:

[0068] Fine-grained text descriptions generated by text refinement models Put into generative model The generation of discrete action sequences is guided by a decoder. Discrete action sequences are decoded into 3D human action sequences, and a pre-trained action encoder is used. Encoding the action sequence yields an action vector, which is then compared with... Encoding real 3D human motion sequence data Calculate the cosine similarity of the obtained action vectors At the same time Replace with The same calculation process is used to obtain the cosine similarity. ,Will and To make a comparison, if If the result is 1, it indicates that the current sampling performance of the text refinement model is better than or equal to that of the labeled fine-grained text, and the reward value is 1; otherwise, it indicates that the current sampling performance of the text refinement model is worse than that of the labeled fine-grained text. This indicates the calculation of the cosine similarity between two vectors.

[0069] Format reward: Check if the model output conforms to the preset structured format. <think> {Fine-grained long text description}< / think> In one specific embodiment, the output content does not contain <think>< / think> If the tag exists in the tag or output content but the fine-grained long text description is not inside the tag, it indicates that the format is not conforming, and the reward value is 0; if the format is conforming, the reward value is 1.

[0070] Step 6: Within-group advantage calculation and gradient update. After calculating each reward, the sum is used to obtain the final reward for each sample. The mean and variance of the reward values ​​within each group are then calculated and standardized. The standardized reward value is the advantage value of each sample within its group. Finally, the advantage value and the probability distribution of the model's current sampling output are used to generate the gradient and backpropagate to update the model's parameters. Using this training method, the model tends to increase the output probability distribution corresponding to samples with positive advantage values ​​and suppress the output probability distribution corresponding to samples with negative advantage values. Consequently, the fine-grained text descriptions output by the model, guiding the generated human action sequences, will be more faithful to the initially set coarse-grained short text descriptions, reducing illusions and bias information in the content.

[0071] like Figure 4 As shown, the present invention also provides a 3D human motion sequence generation system, which is used in the aforementioned 3D human motion sequence generation method and includes: a text refinement module, a discrete motion generation module, a human motion sequence generation module, and a model training module.

[0072] The text refinement module obtains the first text description input by the user and inputs it into the text refinement model to obtain the second text description. The text refinement model is used to transform coarse-grained short text into fine-grained long text. The discrete action generation module inputs the second text description into a pre-trained generation model and outputs a first action sequence, which is a discrete action sequence. The model training module fine-tunes the parameters of the text refinement model based on the second text description and the first action sequence. The human body action sequence generation module reconstructs the first action sequence based on the fine-tuned text refinement model to obtain a 3D human body action sequence.

[0073] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] Alternatively, if the above embodiments of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0075] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. The icons depicting structural positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for generating 3D human motion sequences, characterized in that, Includes the following steps: S1. Obtain the first text description input by the user and input it into the text refinement model to obtain the second text description. The text refinement model is used to transform coarse-grained short text into fine-grained long text. S2. Input the second text description into the pre-trained generative model and output the first action sequence; S3. Input the second text description and the first action sequence into the contrastive reinforcement learning module. The contrastive reinforcement learning module divides the reward function into several dimensions and calculates the reward value of the second text description under each dimension based on the baseline contrast mechanism. Based on the reward value, the parameters of the text refinement model are fine-tuned. Based on the fine-tuned text refinement model, the second text description is updated and the first action sequence is reconstructed to obtain the 3D human action sequence.

2. The method for generating a 3D human motion sequence according to claim 1, characterized in that, The first text description is a coarse-grained short text description, and the second text description is a fine-grained long text description; the text refinement model is a large language model that includes only a first decoder, used to transform coarse-grained short text into fine-grained long text; the pre-trained generative model includes a first encoder and a second decoder, used to transform fine-grained long text into semantically aligned discrete action sequences, i.e., the first action sequence; the contrastive reinforcement learning module divides the reward function into text alignment reward, action sequence alignment reward, and format reward.

3. The method for generating a 3D human motion sequence according to claim 2, characterized in that, Step S3 specifically involves: based on the second text description, standardizing the reward values ​​of all samples corresponding to the same coarse-grained short text description, and using the standardized reward values ​​as the advantage values ​​of each sample; updating the parameters of the text refinement model, increasing the output probability of samples with positive advantage values ​​in the text refinement model, and suppressing the output probability of samples with negative advantage values ​​in the text refinement model.

4. A model training method, characterized in that, The method includes: S1. Discretely encode and reconstruct the 3D human motion sequence to obtain a discrete motion sequence; S2. Obtain the coarse-grained short text description and fine-grained long text description corresponding to the 3D human motion sequence. Based on the training data pair of fine-grained long text description and discrete motion sequence, train the generation model. The generation model is used to generate 3D human motion sequence according to fine-grained long text description. S3. Based on the labeled data pairs of coarse-grained short text descriptions and fine-grained long text descriptions, supervised fine-tuning and reinforcement learning fine-tuning are performed on the text refinement model, which is used to convert coarse-grained short text descriptions into fine-grained long text descriptions.

5. The model training method according to claim 4, characterized in that, The reinforcement learning fine-tuning specifically involves: for the input coarse-grained short text description, sampling multiple fine-grained long text samples using the current text refinement model; inputting each of the fine-grained long text samples into the generation model to obtain corresponding 3D human motion samples; calculating the reward value of each fine-grained long text sample based on a baseline comparison mechanism, wherein the fine-grained long text description labeled in the dataset is used as the baseline sample, and comparing the score corresponding to the output obtained by the generation model guided by each sample with the score corresponding to the output obtained by the generation model guided by the baseline sample to obtain the final reward value; calculating the intra-group advantage value based on the reward value of each sample, and updating the parameters of the text refinement model based on the advantage value.

6. The model training method according to claim 5, characterized in that, The process of discretely encoding and reconstructing a 3D human motion sequence to obtain a discrete motion sequence is as follows: the real 3D human motion sequence is input into the motion encoder to obtain the encoded vector sequence; the encoded vector sequence is quantized using a codebook to obtain a discrete motion sequence; the discrete motion sequence is input into the motion decoder to obtain the reconstructed 3D human motion sequence; the parameters of the encoder and decoder are updated based on the reconstruction loss; and the parameters of the codebook are updated using an exponential moving average and code reset method.

7. The model training method according to claim 5, characterized in that, Training the generative model specifically involves using conditional language modeling as the optimization objective, which is used to minimize the autoregressive prediction error of discrete action sequences under a preset fine-grained long text description condition. Supervised fine-tuning of the text refinement model specifically involves using autoregressive language modeling as the optimization objective. This objective aims to minimize the autoregressive prediction error of fine-grained long text descriptions under the input conditions of preset task prompts and coarse-grained short text descriptions.

8. The model training method according to claim 7, characterized in that, The reward value for each fine-grained long text sample is calculated based on a baseline comparison mechanism, including: a text alignment reward, comparing the cosine similarity between the action vector generated by the sample and the text vector corresponding to the coarse-grained short text, and the cosine similarity between the sample and the baseline sample; if the similarity is not lower than the baseline, a positive reward is given; an action alignment reward, comparing the cosine similarity between the action vector generated by the sample and the action vector corresponding to the real 3D human action, and the cosine similarity between the sample and the baseline sample; if the similarity is not lower than the baseline, a positive reward is given; a format reward, checking whether the output of the sample conforms to a predefined structured format, and if so, a positive reward is given; a reference model with the same initial parameters as the text refinement model is used, and the output distribution of the text refinement model is regularized using KL divergence to constrain the exploration extent of the model.

9. A model training method according to claim 8, characterized in that, The calculation of the intra-group advantage value based on the reward value of each sample includes: standardizing the reward values ​​of all samples corresponding to the same coarse-grained short text, and using the standardized reward values ​​as the advantage value of each sample; updating the text refinement model parameters with the optimization purpose of increasing the output probability of samples with positive advantage values ​​and suppressing the output probability of samples with negative advantage values.

10. A 3D human motion sequence generation system, characterized in that, It includes: a text refinement module, a discrete action generation module, a human action sequence generation module, and a model training module; The text refinement module obtains the first text description input by the user and inputs it into the text refinement model to obtain the second text description. The text refinement model is used to transform coarse-grained short text into fine-grained long text. The discrete action generation module inputs the second text description into a pre-trained generation model and outputs a first action sequence, which is a discrete action sequence. The model training module fine-tunes the parameters of the text refinement model based on the second text description and the first action sequence. The human motion sequence generation module reconstructs the first motion sequence based on the second text description output by the finely tuned text refinement model, thereby obtaining a 3D human motion sequence.

Images