Action generative model training method and apparatus, device, medium, and program product

Abstract

An action generative model training method, comprising: acquiring a priori action and a priori action representation of the priori action, the priori action having a diversified action feature learned in pre-training of an action generative model, and acquiring an example action and an example action representation of the example action, the example action having a personalized action feature (210); by means of the pre-trained action generative model and on the basis of the priori action representation, performing action generation to obtain a first predicted action, and by means of the pre-trained action generative model and on the basis of the example action representation, performing action generation to obtain a second predicted action (220); and on the basis of the priori loss between the first predicted action and the priori action and the example loss between the second predicted action and the example action, re-training the pre-trained action generative model to obtain a trained action generative model (230).

Description

Method, device, equipment, medium and program product for training action generation model

[0001] Related applications

[0002] The present application claims priority to the Chinese patent application No. 2024104341831, filed on April 10, 2024, and entitled "Method, device, equipment, medium and program product for training action generation model", the contents of which are hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the field of action generation, and in particular, to a method, device, equipment, medium and program product for training an action generation model. BACKGROUND

[0004] An action generation model is a model capable of automatically generating natural, diverse and richly detailed action sequences. The action generation model can be widely used in the fields of games, film and television production, virtual reality and augmented reality, etc.

[0005] If an action generation model meeting the above requirements is to be generated, a large number of training samples are often needed for training. In related technologies, a pre-trained model is usually trained using a small number of training samples to obtain an action generation model meeting the individual customization requirements. The pre-trained model refers to a neural network model that has been trained on a large-scale training sample set.

[0006] However, this method often destroys the prior knowledge learned by the pre-trained model during pre-training due to the use of a small number of training samples, resulting in overfitting, i.e., the action generation model obtained by training is difficult to generate action sequences other than the training samples. Therefore, how to design a training method for the action generation model so that the action generation model can meet the individual customization requirements based on a small number of samples and retain the prior knowledge learned during pre-training is a problem to be solved at present. SUMMARY

[0007] According to various embodiments provided by the present application, a method, device, equipment, medium and program product for training an action generation model are provided.

[0008] The present application provides a method for training an action generation model, which is executed by a computer device, and the method comprises:

[0009] obtaining a prior action and a prior action representation of the prior action, the prior action having diversified action features learned by the action generation model during pre-training;

[0010] obtaining a sample action and a sample action representation of the sample action, the sample action having individualized action features;

[0011] perform action generation based on the prior action representation through the pre-trained action generation model to obtain a first predicted action;

[0012] perform action generation based on the sample action representation through the pre-trained action generation model to obtain a second predicted action; and

[0013] retrain the pre-trained action generation model based on a prior loss between the first predicted action and the prior action and a sample loss between the second predicted action and the sample action to obtain a trained action generation model.

[0014] In another aspect, the present application provides a device for training an action generation model, the device comprising:

[0015] an obtaining module configured to obtain a prior action and a prior action representation of the prior action, the prior action having diversified action features learned by the action generation model in pre-training, and obtain a sample action and a sample action representation of the sample action, the sample action having personalized action features;

[0016] a generating module configured to perform action generation based on the prior action representation through a pre-trained action generation model to obtain a first predicted action, and perform action generation based on the sample action representation through the pre-trained action generation model to obtain a second predicted action; and

[0017] a training module configured to retrain the pre-trained action generation model based on a prior loss between the first predicted action and the prior action and a sample loss between the second predicted action and the sample action to obtain a trained action generation model.

[0018] In another aspect, the present application further provides a computer device, comprising a processor and a memory, the memory storing a computer program, and the processor executing the computer program to perform the steps of the method embodiments of the present application.

[0019] In another aspect, a computer readable storage medium is provided, the computer readable storage medium storing executable instructions, the executable instructions being loaded and executed by a processor to perform the steps of the method embodiments of the present application. According to an aspect of the present application, a computer program product is provided, the computer program product comprising computer instructions stored in a computer readable storage medium, the computer program being executed by a processor to perform the steps of the method embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the technical solutions in the prior art, the accompanying drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art based on the disclosed drawings without any creative effort should be covered by the present application.

[0021] FIG. 1 is a generation principle diagram of a generative diffusion model in the related art;

[0022] FIG. 2 is a generation principle diagram of an adversarial network in the related art;

[0023] FIG. 3 is a generation principle diagram of a variational autoencoder in the related art;

[0024] FIG. 4 is an architecture diagram of a computer system provided by an example embodiment of the present application;

[0025] FIG. 5 is a flowchart of a training method of an action generation model provided by an example embodiment of the present application;

[0026] FIG. 6 is a schematic diagram of a three-dimensional human skeleton model provided by an example embodiment of the present application;

[0027] FIG. 7 is a schematic diagram of a spatial constraint provided by an example embodiment of the present application;

[0028] FIG. 8 is a structural block diagram of a training device of an action generation model provided by an example embodiment of the present application;

[0029] FIG. 9 is a structural schematic diagram of a computer device provided by an example embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort should be covered by the scope of the present application.

[0031] The terms used in the present disclosure are merely for the purpose of describing some specific embodiments, and are not intended to limit the present disclosure. The singular forms "a", "an" and "the" used in the present disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.

[0032] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions. For example, the setting operation and other information involved in the present application are obtained under sufficient authorization.

[0033] It should be understood that although the terms first, second, etc. can be employed in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to differentiate one piece of information from another piece of information of the same type. For example, a first parameter can also be referred to as a second parameter without departing from the scope of the present disclosure, and similarly, a second parameter can also be referred to as a first parameter. Depending on the context, the word "if' as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0034] Firstly, the related terms involved in the present application are introduced:

[0035] Artificial Intelligence (AI): is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0036] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-training model technology, operation / interaction system, mechatronics, etc. Among them, the pre-training model is also called large model, basic model, which can be widely applied to downstream tasks in various directions of artificial intelligence after fine-tuning. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0037] With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous driving, drones, digital twins, virtual humans, robots, artificial intelligence generative content (AIGC), conversational interaction, smart medical care, smart customer service, game AI, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0038] Pre-training model (PTM): also known as cornerstone model, large model, refers to a deep neural network (DNN) with large parameters, which is trained on a large amount of unlabeled data. PTM extracts common features from data using the function approximation capability of large parameter DNN. Through fine tuning, parameter efficient fine tuning (PEFT), prompt tuning and other technologies, it is suitable for downstream tasks. Therefore, the pre-training model can achieve ideal results in few-shot or zero-shot scenarios. PTM can be divided into language models (ELMO, BERT, GPT), visual models (swin-transformer, ViT, V-MOE), speech models (VALL-E), and multi-modal models (ViBERT, CLIP, Flamingo, Gato) according to the data modalities it processes. The multi-modal model refers to a model that establishes a feature representation of two or more data modalities. The pre-training model is an important tool for outputting artificial intelligence generative content, and can also be used as a general interface connecting multiple specific task models.

[0039] Action sequence: a sequence of actions or poses with a time sequence relationship. The unit of the action sequence can be a frame or a second.

[0040] Representation space: a set of representations learned by a model during training. It can also be said that the representation space is a set of all possible conditions or factors that the model learns and adapts during the training process. These conditions can include features of input data, labels, constraints, or any other factors that affect the model's output or decision. Optionally, the representation space is a discrete set, i.e., the representation space stores a finite number of conditions; or, the representation space is a continuous set, i.e., the representation space stores a range of conditions or factors. The information stored in the representation space can be referred to as an action vector or action representation. The model can generate conditionally according to the representation space, for example, during model training, input the input value (original data or noise) and its corresponding action vector into the model, and the model will learn how to generate the expected output value according to the input value and the action vector. It should be noted that the above conditions can also be referred to as factors, and the representation set can also be referred to as a factor set.

[0041] Action representation: an action representation is usually a manual or identified feature representation of an action, and an action sequence representation is usually a manual or identified feature representation of an action sequence. Both action representation and action sequence representation are information stored in the representation space, and action representation can also be referred to as action vector, and action sequence representation can also be referred to as action sequence vector. Action representation or action sequence representation can indicate that the model will generate an input value into an expected output value given the action representation or action sequence representation. For example, in an action generation model, the training sample includes an action sequence representation and an action sequence, and the action sequence is the expected output value. The action sequence a is an action sequence of an old man walking forward, and the conditions of the action sequence can be the description words "old man" and "walk forward" for the action sequence a, wherein the word "old man" describes the characteristics of the performer of the action sequence, and the word "walk forward" describes the action content of the action sequence. Generally, the above "old man" and "walk forward" are encoded into a feature vector by a text encoder, and the feature vector can be referred to as an action sequence representation corresponding to the action sequence.

[0042] Conditional generation model / conditional generation network: a generation model based on input action representation for generation. The generation model can be a generative diffusion model, a generative adversarial network (GAN), a variational autoencoder (VAE), or any other model with generation capability, and the embodiments of the present application do not limit the above. Next, the training process of the above several generation models is briefly introduced.

[0043] The generative diffusion model includes a forward process and a reverse process, or a diffusion process and an inverse diffusion process, or a forward process and a reverse process. The forward process is a Markov chain, which is a process of gradually adding noise to the input value (input sample) until a pure noise is obtained, and each step can be referred to as a time step. The reverse process is a process of gradually denoising a noise data until an input value (input sample) is obtained. That is, the forward process is a noise adding process, and the reverse process is a denoising inference process. The generative diffusion model mainly learns the reverse process, that is, how to infer the input value (input sample) from the noise data. As shown in FIG. 1, the solid arrow is the forward process, x0 is the input value (input sample), and in the embodiments of the present application, the input value is an action sequence, x T is the noise data obtained after T time steps of noise adding. The dashed arrow is the reverse process. Since the forward process and the reverse process are actually steps on a Markov chain, they both satisfy the Markov property, that is, the probability distribution of x t is only related to x t-1 . In the training process, step-by-step learning or overall learning can be performed.

[0044] 1) Step-by-step learning: at the t-th time step, the data corresponding to the t-th time step and the data of the t-1-th time step are obtained, and the predicted data of the t-1-th time step is obtained according to the data corresponding to the t-th time step. The model is trained according to the difference between the data of the t-1-th time step and the predicted data of the t-1-th time step.

[0045] 2) Overall learning: there are T time steps, and the predicted data is obtained by denoising the noise data for T steps. The model is trained according to the difference between the predicted data and the input value.

[0046] GAN: including a generator (generative network) and a discriminator (discriminative network). As shown in FIG. 2, the generator G(z) is used to generate similar data x' similar to the input value (input sample) x according to the noise data z; the discriminator is used to judge the difference between the similar data x' generated by the generator and the input value x.

[0047] In training, usually first train the discriminator, label the input value as "true", and label the similar data generated by the generator as "false", and input the above two data into the discriminator, and when calculating the loss, make the discriminator tend to judge the input value as true (1) and the similar data as false (0); when training the discriminator, fix the parameters of the generator unchanged, and only change the parameters of the discriminator. Then train the generator, the generator generates similar data from noise data, and labels the similar data as "true" and inputs it into the discriminator, and changes the parameters of the generator according to the discriminator's judgment loss of the similar data, at this time the parameters of the discriminator will not be changed. In this way, the discriminator and the generator are alternately trained until the training termination condition is met.

[0048] VAE: includes an encoder and a decoder. As shown in FIG. 3, the encoder q Φ (z|x) is used to encode the input value (input sample) x to obtain the feature z corresponding to the input value, and the dimension of the feature z is usually smaller than that of the input value (input sample) x, and the decoder P θ (x|z) is used to restore similar data x' similar to the input value according to the feature z.

[0049] In the training process, the parameters of the encoder and the decoder are updated according to the error between the similar data and the input value.

[0050] Markov Chain (MC): a stochastic process in probability theory and mathematical statistics with Markov property and existing in discrete index set and state space.

[0051] Markov Property: refers to the probability distribution of the current state in a random process, which only relates to the previous state, and is irrelevant to the previous state sequence.

[0052] Overfitting: refers to the problem that the machine learning model tends to generate data similar to the training sample after training, and is difficult to generate new data. That is, the machine learning model trained loses the generalization ability, or in other words, loses the creativity. The intuitive manifestation of overfitting is that the prediction ability on the training sample set is very strong, but the prediction ability on the new sample set drops significantly compared to the training sample set. When the training sample is too small or the sample distribution of the training sample set is too single or there is noise in the training sample, overfitting phenomenon is easy to occur.

[0053] Action generation model: a model capable of generating action sequences. A well-trained action generation model is capable of generating natural, diverse, and detailed action sequences quickly and automatically. In order to better control the action generation model to generate action sequences that meet the requirements, a conditional variable is input to the action generation model, so that the action generation model generates personalized actions based on the conditional variable. According to different use scenarios of the action generation model, the conditional variable can be a conditional variable of each modality. Next, various modal conditional variable controlled action generation models will be briefly illustrated.

[0054] (1) Text: text is the most extensive type of conditional variable. Controlling the action generation model through text usually uses text to describe the action features of the action sequence expected to be generated, such as the execution subject (or performer) features (or role features) of the action sequence, the features of the action content. For example, “an old man walks like a zombie”, which describes both the role feature “old man” and the action content feature “walks like a zombie”.

[0055] (2) Image / video: describe the action features of the action sequence expected to be generated through video or image. A set of images is usually used, which are video frames extracted from the video. This type of conditional variable is usually used to specify the initial state and / or terminal state of the action sequence through a set of images or videos, and the action generation sequence is used to complete the intermediate process to generate a complete action sequence.

[0056] (3) Audio: used to instruct the action generation model to generate action sequences corresponding to the rhythm of the audio. It is usually used to generate dance actions corresponding to the audio according to the audio.

[0057] (4) Trajectory / space constraint: trajectory can be a two-dimensional trajectory or a three-dimensional trajectory, used to indicate that the generated action sequence needs to move along the trajectory.

[0058] FIG. 4 shows an architecture diagram of a computer system according to an example embodiment of the present application. The computer system includes a computer device 110.

[0059] Optionally, the computer device 110 is a server or a terminal.

[0060] Optionally, the server can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. Optionally, the terminal can be an electronic device such as a mobile phone, a tablet computer, a vehicle terminal (car machine), a wearable device, a PC (Personal Computer), an unmanned terminal, etc.

[0061] The computer device 110 is configured to obtain a prior action and a prior action representation of the prior action, the prior action being an action sequence with diversified action features learned by the action generation model in pre-training. The computer device 110 obtains a sample action representation and a sample action with personalized action features. The computer device 110 is further configured to input the prior action representation into the action generation model to generate the prior action. The computer device 110 is further configured to input the prior action representation into the action generation model to obtain a first predicted action, and input the sample action representation into the action generation model to obtain a second predicted action. The computer device 110 is further configured to retrain the pre-trained action generation model based on a prior loss between the first predicted action and the prior action, and a sample loss between the second predicted action and the sample action, to obtain a trained action generation model.

[0062] Exemplarily, the training method includes a pre-training stage 50 and a fine-tuning stage 60. In the pre-training stage 50, the action generation model needs to be pre-trained according to the input conditions (which can be directly input by the user or extracted by the action generation model based on the action sequence), i.e., the text prompt 51 and the spatial constraint 52. The text prompt 51 includes the role feature and the action content. The role feature is used to indicate the characteristics of the performer, i.e., the executor of the action. The action content is the text content for the action sequence. For example, for the action sequence "an old man is walking", the corresponding role feature can be summarized as "old man", and the action content can be summarized as "walking". The spatial constraint is used to indicate the movement trajectory (or action trajectory) of the performer in the action sequence. The spatial constraint can be a set of two-dimensional trajectory points or a set of three-dimensional trajectory points. For the text prompt 51, a text encoder such as the text encoder in CLIP (Contrastive Language-Image Pre-training, a pre-training method or model based on contrastive text-image pairs) can be used for encoding to obtain a text prompt feature representation. For the spatial constraint 52, the action generation model can extract a spatial constraint feature representation by itself. In the pre-training, two parallel representation spaces 53 are constructed based on the text prompt feature representation and the spatial constraint feature representation, respectively. The two parallel representation spaces 53 are used to guide the conditional training of the action generation model. Optionally, an MLP (Multi-Layer Perceptron) can be used to compress the data of the representation space 53. In the pre-training process, if a generative diffusion model is used, a noise action 54 (also referred to as noise data) needs to be randomly sampled and input into a neural network for generation 55 to obtain a prediction value 56, which can also be referred to as a predicted action. It should be noted that the neural network for generation 55 used here can be any architecture of neural network, such as an MLP model, a Transformer model, a Convolutional Network model, etc. In the pre-training process based on the generative diffusion model, the noise action 54 needs to be denoised to obtain the prediction value 56 repeatedly, and the prediction value 56 is added with noise t times to obtain the noise action 54 until the training termination condition is reached. The training termination condition includes the convergence of the loss function of the action generation model. After the pre-training of the action generation model is completed, different actions 57 can be generated by inputting different conditions.

[0063] Exemplarily, in the fine-tuning stage 60, mainly includes reverse optimization 70 and fine-tuning 80. The reverse optimization 70 is to obtain the sample action 75 to the prior style identifier 71, and the style identifier 71 is the role feature or the style feature extracted from the sample action 75 by the action generation model. The reverse optimization 70 needs to fix the sample action representation 72 of the sample action, and inputs the style identifier 71 and the sample action representation 72 into the action generation model 73 with fixed model parameters, so as to obtain the predicted action 74, and updates the style identifier 71 according to the loss between the predicted action 74 and the sample action 75, and finally calculates the style identifier that makes the loss minimum and meets the iteration termination condition. After obtaining the style identifier 71, fine-tuning 80 can be entered. The fine-tuning 80 needs to fine-tune the action generation model 83 based on the sample action representation 81 and the prior action representation 82 randomly sampled from the representation space. The prior action representation 82 is input into the action generation model 83 to generate the first predicted action 85, and the sample action representation 81 is input into the action generation model 83 to generate the second predicted action 84. And fine-tune the model parameters of the action generation model 83 based on the prior loss and the sample loss. The sample loss is the loss between the second predicted action 84 and the sample action 86, and the prior loss is the loss 87 between the first predicted action 85 and the prior action.

[0064] After the fine-tuning 80 ends, inference 90 can be performed, which refers to a way of verifying the prediction ability of the model by using a new action representation to make a prediction. In the inference 90 stage, the style identifier used in the fine-tuning 80, that is, the style identifier finally generated for the sample action in the reverse optimization 70, is used to indicate the style that the action generation model is expected to generate. In the inference 90 stage, the style identifier and the action representation input by the user constitute the user expected action 91, and are input into the fine-tuned action generation model 92. The action generation model 92 will perform feature extraction on the user expected action to obtain a user expected action representation, and then generate a new action 93 that retains the role feature (which can also be said to be the style) of the sample action indicated by the style identifier. It should be noted that this is an example of using the style identifier extracted by the action generation model to indicate the personalized style of the sample action, but in fact, the action content identifier, the emotion identifier and the like of the sample action can also be used. These identifiers can be collectively referred to as identifiers, and the embodiments of the present application do not limit them.

[0065] In this embodiment, the actions can be replaced by action sequences, the action representations can be replaced by action sequence representations, and the predicted actions can be replaced by predicted action sequences. For example, the example actions can be replaced by example action sequences, the example action representations can be replaced by example action sequence representations, the prior actions can be replaced by prior action sequences, the prior action representations can be replaced by prior action sequence representations, the first predicted actions can be replaced by first predicted action sequences, the second predicted actions can be replaced by second predicted action sequences, the user desired actions can be replaced by user desired action sequences, and the user desired action representations can be replaced by user desired action sequence representations.

[0066] FIG. 5 shows a flowchart of a method for training an action generation model according to an example embodiment of the present application. The method is performed by a computer device, which can be the computer device in FIG. 4 described above. The method comprises:

[0067] In step 210, the prior actions and prior action representations of the prior actions are obtained, the prior actions having diversified action features learned by the action generation model in pre-training; and the example actions and example action representations of the example actions are obtained, the example actions having personalized action features.

[0068] Optionally, the action generation model is a conditional generation model. The action representation is a feature representation or a feature vector extracted by the action generation model from at least one of the input features, labels, and constraint conditions. The prior action representations and the example action representations can be collectively referred to as action representations.

[0069] Optionally, the prior action representations and the example action representations can be single-modal or multi-modal. For example, the prior action representations are any one of text features, image features, music features, video features, and trajectory features; or the prior action representations include at least two of the text features, the image features, the music features, the video features, and the trajectory features.

[0070] Optionally, the prior actions are actions having diversified action features learned by the action generation model in pre-training. That is, the prior actions can be actions in a pre-training sample set of the action generation model, or can be actions generated by the pre-trained action generation model according to prior action representations.

[0071] Optionally, the prior action sequences are action sequences having diversified action features learned by the action generation model in pre-training. That is, the prior action sequences can be action sequences in a pre-training sample set of the action generation model, or can be action sequences generated by the pre-trained action generation model according to prior action sequence representations.

[0072] Optionally, the diversified action features refer to at least two personalized action features learned by the action generation model from the training samples in the pre-training; or, the diversified action features refer to different styles learned by the action generation model in the pre-training; or, the diversified action features refer to different action contents learned by the action generation model in the pre-training; or, the diversified action features refer to different styles and action contents learned by the action generation model in the pre-training.

[0073] Optionally, the sample action is an action with personalized action features to be learned by the action generation model, that is, the sample action is an action including personalized action features expected to be learned by the action generation model, or the sample action is an action including personalized action features.

[0074] In an embodiment, the prior action can be replaced by a prior action sequence, the prior action representation can be replaced by a prior action sequence representation, the sample action can be replaced by a sample action sequence, and the sample action representation can be replaced by a sample action sequence representation. The sample action sequence is a sequence of actions with personalized action features to be learned by the action generation model, that is, the sample action sequence is a sequence of actions including personalized action features expected to be learned by the action generation model, or the sample action sequence is a sequence of actions including personalized action features. For example, a user expects the action generation model to be able to generate a series of action instructions such as "an old man stumbles forward", "an old man shakily retreats", "an old man walks a few steps and falls down", but when the text is directly input into the action generation model as a condition, the generated action sequence cannot achieve the expected effect. At this time, the user can first make a sample action sequence "an old man walks forward" to make the action generation model learn the personalized action features in the sample action sequence, such as some gait features and habitual actions.

[0075] Step 220: generating an action based on the prior action representation by the pre-trained action generation model to obtain a first predicted action; and generating an action based on the sample action representation by the pre-trained action generation model to obtain a second predicted action.

[0076] Optionally, the pre-trained action generation model takes the prior action representation as a constraint condition to generate the first predicted action, and takes the sample action representation as a constraint condition to generate the second predicted action.

[0077] In an embodiment, the pre-trained action generation model takes the prior action sequence representation as a constraint condition to generate the first predicted action sequence, and takes the sample action sequence representation as a constraint condition to generate the second predicted action sequence.

[0078] In one embodiment, the prior action representation is input into a pre-trained action generation model. The pre-trained action generation model obtains noise data, which is sampled from at least one of a Gaussian distribution, a uniform distribution, a multinomial distribution, or an exponential distribution. The pre-trained action generation model performs multiple denoising processes on the noise data to obtain a first predicted action, with the prior action representation as a constraint.

[0079] The example action representation is input into the pre-trained action generation model. The pre-trained action generation model performs multiple denoising processes on the noise data to obtain a second predicted action, with the example action representation as a constraint.

[0080] In one embodiment, the pre-trained action generation model obtains a latent action representation, which is a latent feature representation learned in pre-training. The pre-trained action generation model decodes the latent action representation using a decoder to obtain a first predicted action, with the prior action representation as a constraint.

[0081] The pre-trained action generation model decodes the latent action representation using a decoder to obtain a second predicted action, with the example action representation as a constraint.

[0082] Step 230: retrain the pre-trained action generation model based on a prior loss between the first predicted action and the prior action, and an example loss between the second predicted action and the example action, to obtain a trained action generation model.

[0083] The retraining target of the action generation model is to make the prior loss and the example loss as small as possible, that is, to make the generated first predicted action as similar as possible to the prior action, and to make the generated second predicted action as similar as possible to the example action. Alternatively, it can also be understood that the features in the example action are learned as much as possible while the prior knowledge is preserved as much as possible, so that the action generation model can generate the example action, the prior action, and an action or action sequence with both the prior action features and the example action features.

[0084] In summary, the method provided in the embodiments of the present application protects the prior knowledge of the action generation model by the prior action. The prior loss can be understood as the degree of preservation of the prior knowledge in the adjustment process of the model parameters, and the example action is learned at the same time. The example loss can be understood as the learning degree of the example action by the action generation model in the adjustment process of the model parameters. The example action includes personalized action features learned by the user expected action generation model, so that the action generation model obtained by training can learn the personalized action features in the example action and can preserve the prior knowledge as much as possible without being destroyed by the personalized action features.

[0085] For example, a user expects the action generation model to generate a series of action instructions such as "an old man stumbles forward", "an old man trembles and retreats", "an old man falls after walking a few steps", but when the text is directly input as a condition into the action generation model, the generated action sequence may not achieve the expected effect. At this time, the user can first make a sample action sequence "an old man walks forward", so that the action generation model learns the action features in the sample action sequence, such as some gait features, habitual actions, etc. The action generation model can extract the personalized action features in the sample action sequence, such as some gait features, habitual actions, etc., during training (also known as fine-tuning). After training, inputting the above several action instructions can achieve an action sequence that is more in line with the user's expectations. That is, by learning the features included in the sample action sequence, the action generation model can achieve personalized customization (generating an action sequence with the same or similar personalized action features as the sample action sequence).

[0086] In addition, if only the sample action sequence is used to adjust the model parameters of the action generation model, the fine-tuned action generation model may tend to generate action sequences with similar features to the sample action sequence in subsequent use. That is, during fine-tuning based on the sample action sequence, the action generation model has the overfitting problem in small sample fine-tuning, which means that the fine-tuned action generation model tends to generate action sequences similar to the sample action sequence used for fine-tuning and is difficult to generate new action sequences (i.e., loses the generalization ability or the creativity). At this time, if the fine-tuned action generation model is continuously input with the sample action sequence for fine-tuning, and if the new sample action sequence is insufficient to improve the overfitting problem, the action generation model generated by the continuously fine-tuned action generation model will tend to generate the input sample action sequence, and will lose the diversified features it has learned originally, i.e., it will be difficult to continue personalized customization. Training the action generation model based on the sample loss and the prior loss can preserve the prior knowledge as much as possible while learning the features in the sample action as much as possible, so that the action generation model can generate both the sample action sequence and the prior action sequence, and can also generate an action sequence with both the prior action features and the sample action features. For example, if the prior action sequence is "an old man stumbles forward" and the sample action sequence is "a teenager makes a three-step shot", then for the trained action generation model, it is expected to be able to generate both the prior action sequence "an old man stumbles forward" and the prior action sequence "a teenager makes a three-step shot", and also to generate the action sequences "an old man makes a three-step shot" and "a teenager stumbles forward".

[0087] In an embodiment, the pre-trained action generation model is retrained based on the prior loss between the first predicted action and the prior action and the sample loss between the second predicted action and the sample action to obtain a trained action generation model, comprising:

[0088] A model loss of the action generation model is calculated, the model loss being calculated based on a first intermediate value and a second intermediate value, the first intermediate value being a product of the prior loss and a prior weight, and the second intermediate value being a product of the sample loss and a sample weight; the pre-trained action generation model is retrained based on the model loss to obtain the trained action generation model.

[0089] Optionally, the prior weight is determined according to expert experience; or, the prior weight is set by a user; or, the prior weight is dynamically adjusted according to an expected training effect in a training process, the training process referring to a training process for the same batch of sample actions. For example, for a first batch of sample actions, it is desired to retain more prior knowledge, and therefore the prior weight is set to 1; for a second batch of sample actions, it is desired to learn more features of the batch of sample actions, and therefore the prior weight is set to 0.8. Batch refers to grouping of training samples in a model training process. Each batch of training samples (sample actions) is input into the action generation model at the same time or in sequence, and the action generation model calculates the total model loss corresponding to the batch of training samples, usually taking the mean or median of the model losses as the total model loss corresponding to the batch of training samples, and the action generation model updates the model parameters according to the total model loss until the action generation model converges for the batch of training samples. The next batch of training is then performed. It should be noted that the training batches of the action generation model are not limited by the embodiments of the present application, and can be adjusted according to the number and characteristics of the training samples during training.

[0090] Optionally, the sample weight is determined according to expert experience; or, the sample weight is set by a user; or, the sample weight is dynamically adjusted according to a training effect in a training process, the training process referring to a training process for the same batch of sample actions. The way in which the sample weight is dynamically adjusted according to the expected training effect in the training process is similar to the way in which the prior weight is dynamically adjusted according to the expected training effect in the training process, and will not be described here.

[0091] The computer device calculates a product of the prior loss and the prior weight to obtain the first intermediate value, and calculates a product of the sample loss and the sample weight to obtain the second intermediate value. The model loss is calculated based on the first intermediate value and the second intermediate value, and the model loss can be a sum of the first intermediate value and the second intermediate value, or a mean of the first intermediate value and the second intermediate value, or a value obtained by weighted sum of the first intermediate value and the second intermediate value.

[0092] In an embodiment, the prior loss at least comprises a prior generation loss, and the example loss at least comprises an example generation loss. The prior generation loss is a generation loss between the first predicted action and the prior action, and the example generation loss is a generation loss between the second predicted action and the example action. The generation loss is used to indicate a similarity or a difference degree between the predicted actions (i.e., the first predicted action and the second predicted action) and the real actions (i.e., the prior action and the example action). That is, in the training process of the action generation model, the prior action and the example action are regarded as the real actions or the actions that the action generation model can generate. At the beginning of the training of the action generation model, since the model parameters of the action generation model have not been adjusted, the first predicted action generated is the same as or similar to the prior action generated by using the action generation model before the training starts; however, since the example action is usually an action that the action generation model has not learned, there may be a large difference between the second predicted action and the example action. The training target of the action generation model is to continuously adjust the model parameters to balance the prior loss and the example loss, so that the action generation model can learn the example action as much as possible while preserving the prior knowledge as much as possible.

[0093] Optionally, the prior loss further comprises a prior geometric loss, and the example loss further comprises an example geometric loss. The prior geometric loss is a geometric loss between the first predicted action and the prior action, and the example geometric loss is a geometric loss between the second predicted action and the example action. The geometric loss is used to indicate a similarity or a difference degree or a preservation degree of the geometric characteristics of the joints in the predicted actions relative to the geometric characteristics of the joints in the real actions. The geometric characteristics of the joints include positions, rotational poses, velocities, friction forces, etc. of the joints.

[0094] Optionally, the prior loss comprises the prior generation loss and the prior geometric loss, and the prior loss is a sum of the prior generation loss and a first intermediate loss. The first intermediate loss is a product of the prior geometric loss and a first geometric weight. The example loss comprises the example generation loss and the example geometric loss, and the example loss is a sum of the example generation loss and a second intermediate loss. The second intermediate loss is a product of the example geometric loss and a second geometric weight. The values of the first geometric loss and the second geometric loss are the same or different.

[0095] Optionally, according to the model loss, the model parameters of the action generation model are updated by using a back propagation algorithm. That is, according to the model loss, a gradient vector of each parameter in the model parameters is calculated by using the back propagation algorithm, and the model parameters are updated in the opposite direction of the gradient vector to train the action generation model.

[0096] Optionally, the prior action is one or more, and the sample action is one or more. When training the action generation model, the training is based on a sample loss corresponding to a sample action and a prior loss corresponding to a prior action; or, the training is based on a sample loss corresponding to a sample action and prior losses corresponding to multiple prior actions; or, the training is based on sample losses corresponding to multiple sample actions and prior losses corresponding to multiple prior actions, and the multiple sample actions and the multiple prior actions can be combined in any manner, for example, there are multiple sample actions {a1, a2} and multiple prior actions {b1, b2, b3}, and in the training process of the action generation model, the model loss of the action generation model can be calculated based on a sample loss corresponding to the sample action a1 and a prior loss corresponding to the prior action b1, the model loss of the action generation model can be calculated based on a sample loss corresponding to the sample action a1 and a prior loss corresponding to the prior action b2, the model loss of the action generation model can be calculated based on a sample loss corresponding to the sample action a1 and a prior loss corresponding to the prior action b3, the model loss of the action generation model can be calculated based on a sample loss corresponding to the sample action a2 and a prior loss corresponding to the prior action b1, the model loss of the action generation model can be calculated based on a sample loss corresponding to the sample action a2 and a prior loss corresponding to the prior action b2, and the model loss of the action generation model can be calculated based on a sample loss corresponding to the sample action a2 and a prior loss corresponding to the prior action b3; and the action generation model is trained based on at least one of the above model losses. The embodiments of the present application are exemplified by one sample action and one prior action, but this is not limited thereto.

[0097] In the embodiments, the model loss is calculated based on the sample loss and the prior loss, so as to determine the final model loss according to the desired retention degree of prior knowledge and the learning degree of new knowledge (sample action), so that the action generation model obtained by training has the desired prior knowledge and new knowledge, thereby realizing the personalized fine-tuning of the action generation model. In this way, the action generation model obtained by training can generate the prior action learned in the pre-training stage, generate the sample action learned in the fine-tuning stage, and generate a new action by combining the features of the prior action and the sample action. Moreover, since the prior knowledge is retained, after the end of the current fine-tuning, the action generation model does not have the overfitting phenomenon, and can continue to be fine-tuned according to the requirements in the actual application process, thereby realizing the personalized requirements of action generation.

[0098] Further, during the training of the action generation model, only one sample action and one prior action are used. Due to the small amount of data of one sample action and one prior action, the action generation model can quickly learn the individualized features in the sample action while retaining the prior knowledge. Moreover, the performance requirement of the computer device used in this way is relatively low, and this way can be widely applied to small computer devices such as mobile phones and vehicle terminals. Using one sample action and multiple prior actions can reduce the overfitting risk caused by the similarity between the sample action and the prior action as much as possible (if the sample action is similar to the prior action, the action generation model has a high probability of generating an action sequence with similar features), thereby improving the protection degree of the action generation model for prior knowledge. Using multiple sample actions and multiple prior actions can reduce the probability of overfitting due to the large number of samples (sample actions and prior actions) used for training, and enable the action generation model to learn multiple individualized features, thereby achieving batch learning and greatly improving the training efficiency of the action generation model in use scenarios requiring learning of multiple individualized features.

[0099] In order to achieve better training effect, the action generation model usually needs to be trained for multiple rounds. The specific training method is as follows.

[0100] In some embodiments, the 0th round action generation model is an initial action generation model, and the initial action generation model is a pre-trained action generation model capable of generating diversified action sequences. The initial action generation model can also be referred to as the 0th round action generation model, or the action generation model trained for 0 rounds, or the action generation model trained for the 0th round.

[0101] The model loss of the action generation model is calculated, including: in the i-th round of training process, calculating the i-th round of model loss, the i-th round of model loss being the sum of the i-th round of first intermediate value and the i-th round of second intermediate value, the i-th round of first intermediate value being the product of the i-th round of prior loss and the prior weight, the i-th round of second intermediate value being the product of the i-th round of sample loss and the sample weight, i being a positive integer;

[0102] Based on the model loss, the pre-trained action generation model is retrained to obtain a trained action generation model, including: determining the i-th round of model parameters that reduce the i-th round of model loss; updating the model parameters of the action generation model to the i-th round of model parameters to obtain the action generation model trained for the i-th round; wherein the i-th round of prior loss is the loss between the i-th round of first predicted action and the prior action, the i-th round of first predicted action being generated by inputting the representation of the prior action into the action generation model trained for the (i-1)th round; the i-th round of sample loss is the loss between the i-th round of second predicted action and the sample action, the i-th round of second predicted action being generated by inputting the representation of the sample action into the action generation model trained for the (i-1)th round.

[0103] wherein the ith round prior loss is a loss between the ith round first predicted action and the prior action, the ith round first predicted action is generated by inputting the prior action representation into the action generation model trained in the (i-1)th round, the ith round example loss is a loss between the ith round second predicted action and the example action, the ith round second predicted action is generated by inputting the example action representation into the action generation model trained in the (i-1)th round.

[0104] That is, in the ith round of training, the prior action representation is input into the action generation model trained in the (i-1)th round to generate the ith round first predicted action, and the example action representation is input into the action generation model trained in the (i-1)th round to generate the ith round second predicted action. The ith round prior loss is calculated, which is a loss between the ith round first predicted action and the prior action. The ith round example loss is calculated, which is a loss between the ith round second predicted action and the example action. The ith round model loss is calculated, which is a sum of the ith round first intermediate value and the ith round second intermediate value, the ith round first intermediate value is a product of the ith round prior loss and the prior weight, and the ith round second intermediate value is a product of the ith round example loss and the example weight.

[0105] Optionally, inputting the prior action representation into the action generation model trained in the (i-1)th round to generate the ith round first predicted action can be understood as that the action generation model trained in the (i-1)th round generates the ith round first predicted action based on the prior action representation. Inputting the example action representation into the action generation model trained in the (i-1)th round to generate the ith round second predicted action can be understood as that the action generation model trained in the (i-1)th round generates the ith round second predicted action based on the example action representation.

[0106] Optionally, the action generation model trained in the (i-1)th round generates the ith round first predicted action based on the prior action representation, which includes that the action generation model trained in the (i-1)th round generates the ith round first predicted action based on the prior action representation and noise data, the noise data is noise with the same dimension or dimension randomness as the example action collected by the action generation model, and the noise data is sampled from at least one of Gaussian distribution, uniform distribution, multinomial distribution or exponential distribution by the action generation model; or, the action generation model trained in the (i-1)th round generates the ith round first predicted action based on the prior action representation and latent representation, the latent representation is a corresponding latent feature representation of a training sample learned by the action generation model during pre-training.

[0107] Optionally, the action generation model trained in the i-1th round generates the first predicted action in the ith round based on the prior action representation and the noise data, including: the action generation model trained in the i-1th round performs T times of denoising on the noise data according to the prior action representation to obtain the first predicted action in the ith round, where T is a positive integer; or, the action generation model trained in the i-1th round uses the generator to generate the noise data according to the prior action representation to obtain the first predicted action in the ith round.

[0108] Optionally, the action generation model trained in the i-1th round generates the first predicted action in the ith round based on the prior action representation and the latent representation, including: the action generation model trained in the i-1th round uses the decoder to decode the latent representation based on the prior action representation to obtain the first predicted action in the ith round.

[0109] Optionally, the action generation model trained in the i-1th round generates the second predicted action in the ith round based on the sample action representation, including: the action generation model trained in the i-1th round generates the second predicted action in the ith round based on the sample action representation and the noise data, the noise data being noise with the same dimension as the sample action collected by the action generation model, the noise data being sampled from at least one of a Gaussian distribution, a uniform distribution, a multinomial distribution or an exponential distribution; or, the action generation model trained in the i-1th round generates the second predicted action in the ith round based on the sample action representation and the latent representation, the latent representation being a corresponding latent feature representation of a training sample learned by the action generation model during pre-training.

[0110] Optionally, the action generation model trained in the i-1th round generates the second predicted action in the ith round based on the sample action representation and the noise data, including: the action generation model trained in the i-1th round performs T times of denoising on the noise data according to the sample action representation to obtain the second predicted action in the ith round, where T is a positive integer; or, the action generation model trained in the i-1th round uses the generator to generate the noise data according to the sample action representation to obtain the second predicted action in the ith round.

[0111] Optionally, the action generation model trained in the i-1th round generates the second predicted action in the ith round based on the sample action representation and the latent representation, including: the action generation model trained in the i-1th round uses the decoder to decode the latent representation based on the sample action representation to obtain the second predicted action in the ith round.

[0112] Optionally, according to the model loss, gradient information corresponding to the model loss is calculated by using a back propagation algorithm, and the ith round of model parameters is determined according to the gradient information, the ith round of model parameters being model parameters that reduce the model loss of the action generation model.

[0113] Optionally, the i-th round of model parameters that reduce the model loss of the action generation model are determined by using an optimization algorithm. The optimization algorithm includes gradient descent, stochastic gradient descent, Adam optimization, etc. Gradient descent is an iterative optimization algorithm that updates the model parameters by calculating the gradient information of the model loss with respect to the model parameters. In each step of iteration, it updates the model parameters in the negative direction of the gradient information (i.e. the direction in which the loss function decreases the fastest) in order to reach the minimum value of the model loss. Gradient descent can be batch gradient descent, i.e. using all samples to calculate the gradient information to update the parameters each time. Stochastic gradient descent is a variant of gradient descent, which uses only one sample's gradient to update the parameters in each iteration. It is fast in calculation and can update the model parameters online, suitable for large-scale data sets and real-time applications. Adam optimization is an adaptive learning rate optimization algorithm that combines the ideas of momentum method and adaptive gradient algorithm. It dynamically adjusts the learning rate of each parameter by calculating the first moment estimate (similar to momentum) and second moment estimate (similar to adaptive gradient algorithm) of the gradient. Adam optimization can adaptively balance the learning speed of different parameters during training, and has good performance for sparse gradients or non-stationary objective functions. In addition, Adam optimization usually converges faster and performs well in many deep learning applications.

[0114] In this embodiment, in each round of training process, the prior action representation is input into the action generation model trained in the previous round to obtain the prior prediction sample, and the sample action representation is input into the action generation model trained in the previous round to obtain the sample prediction sample. The prior loss is the loss between the prior prediction sample and the prior sample, and the sample loss is the loss between the sample prediction sample and the sample sample. In the training process of the action generation model, in addition to being trained based on the sample loss, it is also trained based on the prior loss, thereby preserving the prior knowledge of the action generation model during pre-training, preventing overfitting problems in small sample fine-tuning. If only the model parameters of the action generation model are adjusted by the sample action, it may cause the fine-tuned action generation model to generate action sequences with similar characteristics to the sample action in the subsequent use process, resulting in the action sequences generated by the fine-tuned action generation model becoming more and more similar to the input sample action, losing the diversified features it originally learned, i.e. it is difficult to continue to achieve personalized customization. Retaining prior knowledge during training can avoid overfitting problems, thereby ensuring that the fine-tuned action generation model can still continue to achieve personalized customization requirements.

[0115] Generally, the fine-tuning stage is to enable the action generation model to learn features in the example action and generate action sequences similar to the example action based on part of the features. For example, the example action is "an old man is walking forward", and the animator or user may expect the action generation model to learn some features about "old man" in the example action, and then generate new action sequences such as "an old man is staggering forward" and "an old man makes a three-step layup" based on the feature "old man". Therefore, in order to guide the action generation model to generate based on part of the features in the example action later, the features need to be identified, and then the action representation is added to the action generation model according to the identification for the generation of new action sequences. In related technologies, identifiers are usually used for identification. However, the identifiers in related technologies are usually manually set, for example, the identifier of the example action "an old man is walking forward" is "old man". At this time, if the action generation model has learned some features a about "old man" before, but the feature of the old man shown in the example action is feature b, then when generating according to the identifier "old man", the action generation model may generate based on feature a, or may generate based on feature b, or may generate based on feature a and feature b. At this time, the language drift problem occurs, that is, the feature described by the user using the identifier "old man" is different from the feature related to the identifier "old man" saved in the action generation model, and language drift occurs for the identifier "old man". Language drift refers to the difference or change between the language or data distribution of the action generation model and the language or data distribution of the actual application environment (it can be understood that for the same action sequence, the action representation of the action sequence in the fine-tuned action generation model is different from the manually identified action representation). However, the user actually only expects to generate based on feature b, that is, the prior knowledge in the identifier interferes with the user's personalized customization demand for the action generation model. Next, the identifier design method in the present application is shown.

[0116] In some embodiments, the example action representation includes a first action representation and a second action representation, the second action representation being an action representation corresponding to the identifier, and the identifier being used to identify a personalized feature of the example action.

[0117] Optionally, the example action representation is used to instruct the action generation model to generate the example action based on the example action representation, or in other words, the example action representation is used to instruct the action generation model to generate a second predicted action based on the example action representation, and the second predicted action and the example action are used to train the action generation model.

[0118] Optionally, the identifier is used to identify at least one of the following: a sample action, a partial feature representation in the sample action, a partial feature representation in the sample action that needs to be identified, a feature representation in the sample action to be retained, an action style in the sample action, or a personalized feature of the sample action. The action style refers to a characteristic of a performer in the sample action.

[0119] Optionally, the first action representation includes at least one of the following: a feature in the sample action representation to be learned other than the feature corresponding to the identifier, an action representation in the sample action representation other than the identifier, an action representation in the sample action representation used to identify the sample action, another part of the action representation in the sample action representation other than the action representation identifying the sample action, or a feature in the sample action to be fused.

[0120] In one embodiment, the sample action representation of the sample action is obtained, including: obtaining a first action representation, the first action representation being a constraint condition to be replaced in the sample action; and obtaining a second action representation, the second action representation being obtained by performing feature extraction on the identifier by the action generation model.

[0121] The action generation is performed based on the sample action representation by the pre-trained action generation model to obtain a second predicted action, including: performing the action generation based on the first action representation and the second action representation by the pre-trained action generation model to obtain the second predicted action.

[0122] The method further includes updating the identifier based on a loss between the second predicted action and the sample action.

[0123] The second action representation is an action representation corresponding to the identifier. The second action representation is obtained by performing feature extraction on the identifier by the action generation model.

[0124] Optionally, the first action representation is extracted by the action generation model from a condition or a label input by a user or a developer; or, the first action representation is extracted by the action generation model based on the sample action; or, the first action representation is extracted by the computer device based on the sample action and input into the action generation model. For example, the first action representation includes a feature representation corresponding to an audio, the audio being input by the user; or, the first action representation includes a feature representation corresponding to a spatial constraint, the spatial constraint being used to indicate a movement trajectory of a performer in the sample action, the feature representation corresponding to the spatial constraint being obtained by performing feature extraction on the input sample action by the action generation model; or, the first action representation includes a feature representation corresponding to action content, the feature representation corresponding to the action content being obtained by the computer device through a pre-set action recognition method and input into the action generation model.

[0125] Optionally, the action generation model is a pre-trained action generation model; or, the action generation model is an action generation model that has not started the training method of the action generation model; or, the action generation model is an action generation model trained according to other sample actions; or, the action generation model is an action generation model that has not been trained according to the sample action. That is, the action generation model is not in the training process.

[0126] Optionally, based on the loss between the second predicted action and the sample action, the model parameters of the action generation model are fixed, and the identifier is updated. Specifically, the identifier is updated in a direction in which the loss between the second predicted action and the sample action decreases.

[0127] In this embodiment, an identifier is set for the sample action, so that the identifier can identify part of the features in the sample action. In the use process of the action generation model, the user or the animation producer can instruct the action generation model to generate an action sequence with similar or identical features to the features retained by the identifier. For example, for the sample action "an old man walks forward", the identifier retaining the character feature can be determined according to the character feature "old man", or the action generation model extracts the feature representation of the character feature according to the first action representation, i.e., the action feature "walks forward", and the sample action, as the identifier. When the action generation model is used, the action about "old man" can be generated based on the identifier retaining the character feature. Since the identifier is not directly specified as "old man" or other information with priori such as "old man 111" or "old 1231", etc., but is generated by the action generation model itself, that is, the identifier is the information that best represents the feature representation to be retained in the sample action and does not have priori information. Therefore, when personalized customization is performed based on the identifier, the to-be-retained features in the sample action can be completely retained and will not be disturbed by the priori knowledge in the action generation model, that is, a series of new actions about "old man" can be generated, such as "an old man is walking" and "an old man is pacing", etc. If it is desired to retain the action feature "walks forward" in the sample action, an identifier retaining the action feature can be determined according to the action feature "walks forward" (or the action generation model extracts the feature representation of the action feature according to the first action representation, i.e., the character feature "old man", and the sample action, as the identifier). The user can define the features desired to be retained (set as the identifier) and the features desired to be diversified or replaced (set as the first action representation) in the input sample action to perform personalized customization.

[0128] Similarly, in order to achieve better de-priori effect, the generation of the identifier also needs multiple rounds of iteration.

[0129] Optionally, the identifier is a number; or, the identifier is a sequence composed of numbers and letters.

[0130] Optionally, the initial value of the identifier is 0. That is, in the first iteration process, the 0th round of the second action representation is obtained by the action generation model performing feature extraction based on the identifier "0".

[0131] In an embodiment, the action generation based on the first action representation and the second action representation by the pre-trained action generation model to obtain the second predicted action comprises:

[0132] In the jth iteration process, the first action representation and the (j-1)th round of the second action representation are input into the action generation model to obtain the jth round of the second predicted action, j being a positive integer;

[0133] The identifier is updated based on the loss between the second predicted action and the sample action, comprising:

[0134] The (j-1)th round of the identifier is updated based on the loss between the jth round of the second predicted action and the sample action to obtain the jth round of the identifier; the jth round of the second action representation is obtained, which is obtained by the action generation model performing feature extraction on the jth round of the identifier; j is updated to j+1, and the jth iteration process is continued to be executed until the jth round of the identifier meets the iteration termination condition.

[0135] Optionally, the (j-1)th round of the second action representation and the first action representation are input into the action generation model to obtain the jth second predicted action, which can be understood as that the action generation model generates the jth round of the second predicted action based on the (j-1)th round of the second action representation and the first action representation as constraint conditions.

[0136] Optionally, the action generation model generates the jth round of the second predicted action based on the (j-1)th round of the second action representation and the first action representation as constraint conditions, comprising: the action generation model generates the jth round of the second predicted action based on the (j-1)th round of the second action representation, the first action representation as constraint conditions, and noise data as initial value, the noise data being noise with the same dimension as the sample action collected by the action generation model, the noise data being sampled from at least one of Gaussian distribution, uniform distribution, multinomial distribution or exponential distribution by the action generation model; or, the action generation model generates the jth round of the second predicted action based on the (j-1)th round of the second action representation, the first action representation as constraint conditions, and latent representation as initial value, the latent representation being the corresponding latent feature representation of the training sample learned by the action generation model during pre-training.

[0137] Optionally, the action generation model generates the jth round of second predicted action based on the (j-1)th round of second action representation and the first action representation as constraint conditions, and noise data as initial value, including: the action generation model performs T times of denoising on the noise data to obtain the jth round of second predicted action, with the (j-1)th round of second action representation and the first action representation as constraint conditions, where T is a positive integer; or, the action generation model uses a generator to generate the jth round of second predicted action from the noise data, with the (j-1)th round of second action representation and the first action representation as constraint conditions.

[0138] Optionally, the action generation model generates the jth round of second predicted action based on the (j-1)th round of second action representation and the first action representation as constraint conditions, and latent representation as initial value, including: the action generation model decodes the latent representation to obtain the jth round of second predicted action based on the (j-1)th round of second action representation and the first action representation as constraint conditions.

[0139] Optionally, in the jth round of iteration, the jth round of identification loss is obtained based on the loss between the jth round of second predicted action and the sample action, and the jth round of identifier is obtained according to the identification loss.

[0140] The identifier is updated by using a back propagation algorithm. That is, the gradient vector corresponding to the identifier is calculated by using the back propagation algorithm according to the identification loss, and the (j-1)th round of identifier is updated in the opposite direction of the gradient vector to obtain the jth round of identifier.

[0141] Optionally, the action generation model comprises a feature extraction module configured to extract the condition input by the user into an action representation. The feature extraction module is configured to perform feature extraction on the conditions of each modality, so as to extract the corresponding action representation.

[0142] Optionally, the iteration termination condition is that the identification loss corresponding to the jth round of identifier reaches a minimum value; or, the iteration termination condition is that the identifier corresponding to the jth round of identifier reaches a minimum value; or, the iteration termination condition is that the identification loss corresponding to the jth round of identifier converges.

[0143] In summary, the method provided in the embodiments of the present application shows a method for an action generation model to extract an identifier, which obtains an identifier that minimizes or minimizes the loss between a second predicted action generated based on a second action representation and a first action representation and a sample action through continuous iteration optimization. It can be understood that the iteration is continuously performed to obtain the action representation related to the identifier in the sample action, and the action representation is taken as the second action representation in the next iteration process until the identifier that minimizes the loss between the second predicted action and the sample action is obtained, and the identifier can identify the individualized feature in the sample action. The identifier extracted in this way is not directly input by the user, but is extracted by the action generation model for the sample action, which can avoid the user introducing prior information that will affect the judgment of the action generation model. For example, the user sets the identifier for the sample action of "an old man walking forward" as "old man", and the action generation model has learned some features a about "old man" before, but the feature of the old man displayed in the sample action is feature b, so when the action generation model generates according to the identifier "old man", it can generate based on feature a, or can generate based on feature b, or can generate based on feature a and feature b. The identifier obtained by the action generation model based on the sample action is equivalent to letting the action generation model judge how to describe the individualized feature in the sample action, which is the most representative of the individualized feature in the sample action, so as to completely and accurately extract the individualized feature in the sample action, thereby ensuring the implementation of individualized customization; according to the different types of individualized features corresponding to the identifier (such as role features or action features), the individualized customization can also be made more diverse, that is, the type of individualized feature in the sample action can be customized to further meet the individualized customization needs of the user.

[0144] In one embodiment, the prior action is generated by an action generation model that has not started training. That is, obtaining the prior action and the prior action representation of the prior action includes: obtaining any prior action representation from a representation space of a pre-trained action generation model, the representation space being a set of action representations learned by the action generation model in pre-training; obtaining the prior action by performing action generation according to the prior action representation through the pre-trained action generation model.

[0145] It should be noted that the representation space is a set of action representations learned by the action generation model during pre-training, but the prior action representation is not necessarily an action representation of a training sample used during pre-training, and the prior action representation can also be a random combination of action representations of multiple training samples used during pre-training; the prior action generated based on the prior action representation is not necessarily an action of a training sample used during pre-training, and the prior action can also be a random combination of actions of multiple training samples used during pre-training. For example, the action representation is a matrix, the action representation during pre-training of the action generation model includes c1, c2, and c3, the prior action representation can be c1 or c2 or c3, or a combination of part of the parameters in c1 and c2, or a combination of part of the parameters in c2 and c3, or a combination of part of the parameters in c1 and c3, or a combination of part of the parameters in c1, c2, and c3, and the embodiments of the present application do not limit this. During the training of the action generation model, the prior action is generated as a real action or an action that the action generation model can generate. At the beginning of the training of the action generation model, since the model parameters of the action generation model have not been adjusted, the first predicted action generated is the same as or similar to the prior action generated before the training of the action generation model starts; however, since the sample action is usually an action that the action generation model has not learned, there can be a large difference between the second predicted action and the sample action. The training target of the action generation model is to continuously adjust the model parameters to balance the prior loss and the sample loss, so that the action generation model can learn the sample action while retaining as much prior knowledge as possible. The prior action is generated before the training starts, and the retention of the prior knowledge of the action generation model is measured by the loss between the first predicted action and the prior action during the training, so that the sample action can be learned on the basis of retaining the prior knowledge.

[0146] In addition to the training method of the action generation model described above, the pre-training method of the action generation model is also illustrated in the embodiments of the present application. Next, the pre-training method provided by the embodiments of the present application will be further introduced. It should be noted that the action generation model in the training method of the action generation model described above is a pre-trained action generation model, and the pre-training method used can be the pre-training method illustrated in the present application or any pre-training method in the related art, and the embodiments of the present application do not limit this.

[0147] In one embodiment, the method further comprises:

[0148] obtaining at least one training sample, each training sample including an action sequence and a set of action representations corresponding to the action sequence, the set of action representations including at least two action representations; pre-training an initial model based on the at least one training sample to obtain a pre-trained action generation model, the initial model being a neural network model for generating actions based on a set of action representations.

[0149] Alternatively, the action in the embodiment can be replaced by an action sequence, and the action representation can be replaced by an action sequence representation. The action sequence is a continuous action or posture with a time sequence relationship. The action sequence can be divided into multiple frames, and the posture information of the performer in each frame of the action sequence, that is, the action sequence can be abstracted as the posture information of the performer in each frame. The performer is the originator or performer of the action in the action sequence. The performer can be a person, an animal, a plant, or other living beings, and can also be a robot, a table, or other artificial objects, and the embodiments of the present application do not limit this.

[0150] Alternatively, the posture information of the performer is the posture information of the model. For example, the performer is a character, and the character model corresponding to the performer can be any one of the following models: a skeleton model including at least a part of the skeleton of the human body, a model including skin and skeleton, and a model wearing costumes. Next, the representation form of the action sequence is further introduced when the performer is a character.

[0151] For example, in the action sequence, only the skeleton is used to represent a person, and when the skeleton model needs to be abstracted into a feature representation, the skeleton model can be represented as a tree structure, and a plurality of joint nodes in the skeleton model are extracted, each joint node corresponding to a respective parent node, and there is a root node in the plurality of joint nodes of the skeleton, and the parent node of the root node can be a certain point on the world coordinate system of the virtual space. As shown in FIG. 6, a skeleton model 100 is determined by using a three-dimensional human skeleton model, which divides the human skeleton into 24 joint nodes, and the root node 0 is the joint node 0 corresponding to the center of the human pelvis. Among them, the parent nodes of the joint node 1, the joint node 2 and the joint node 3 are all the root node 0, the parent node of the joint node 4 is the joint node 1, the parent node of the joint node 5 is the joint node 2, the parent node of the joint node 6 is the joint node 3, and so on. The parent node of the joint node 23 is the joint node 21, and the like. Alternatively, according to the different selected skeleton model, the root node is the joint node corresponding to the center of the human pelvis, or the root node is a node with a height value approximately equal to the average of the height values of the plurality of joint nodes.

[0152] If there are N frames in the action sequence and J joint nodes in the skeleton model, since the human action can actually be understood as the hinge movement of each joint node, when describing the posture of the skeleton model in a certain frame in the action sequence, it can be described as the rotation posture of each joint node relative to its parent node. The rotation posture of the joint node relative to its parent node can be described by using a rotation feature. If the number of rotation features is Q, the skeleton model in a certain frame in the action sequence can be described as wherein G represents the matrix corresponding to the rotation posture of the skeleton model in a certain frame in the action sequence, R represents a real number set, that is, the parameters in the matrix are all real numbers. In the case of knowing the rotation pose of the skeleton model, the world coordinates of the skeleton model, that is, the absolute position of the skeleton model, or the physical coordinates, or the world coordinates, and the like, of the skeleton model also need to be known. The absolute position of the skeleton model can be represented by the absolute position of the root node, which can be represented as P represents a matrix corresponding to the absolute position of the root node, R represents a real number set, that is, the parameters in the matrix are all real numbers. Therefore, an action sequence of N frames can be represented as x represents a matrix corresponding to the action sequence of N frames, and the matrix has N×(J×Q+3) real number parameters, and each frame has (J×Q+3) real number parameters.

[0153] wherein the number of rotation features depends on the selected rotation representation, such as a rotation matrix, Euler angles, a Quaternion, and the like. The rotation matrix is a matrix used to describe the rotation between two coordinate systems. The Euler angles are three independent angle parameters used to uniquely determine the position of a rotating rigid body, which are composed of a nutation angle θ, a precession angle (i.e., a precession angle) ψ, and a rotation angle φ. The Quaternion is a linear equation including a real number and three imaginary numbers (i, j, and k). The Quaternion can be used as a rotation representation in three dimensions. For example, the skeleton model shown in FIG. 6 is taken as an initial state, a three-dimensional coordinate system is constructed for each joint, and the initial state of the three-dimensional coordinate system is recorded. The rotation pose of the skeleton model in each frame of the action sequence is the rotation pose of the joint in the frame relative to the corresponding joint in the initial state. The rotation feature is a feature used to describe the rotation pose.

[0154] Optionally, the action generation model includes a feature extraction module configured to extract the user input condition into an action representation. The feature extraction module is configured to perform feature extraction on the conditions of each modality, so as to extract the corresponding action representation. Alternatively, the computer device includes a feature extraction module configured to extract the user input condition into an action representation.

[0155] Optionally, the at least two action representations are extracted by the action generation model from conditions or labels input by a user or a developer; or, the at least two action representations are extracted by the action generation model based on the action sequence; or, the at least two action representations are extracted by the computer device based on the action sequence and input into the action generation model; or, a part of the at least two action representations are extracted by the action generation model from conditions or labels input by a user or a developer, and another part of the at least two action representations are extracted by the action generation model based on the action sequence; or, a part of the at least two action representations are extracted by the action generation model from conditions or labels input by a user or a developer, and another part of the at least two action representations are extracted by the computer device based on the action sequence and input into the action generation model; or, a part of the at least two action representations are extracted by the action generation model based on the action sequence, and another part of the at least two action representations are extracted by the computer device based on the action sequence and input into the action generation model. It should be noted that, for the extraction method of the at least two action representations, the embodiments of the present application only list some extraction methods, and the method is also applicable to extracting the sample action representations for the sample action, which is not limited in the embodiments of the present application.

[0156] Optionally, the initial model is any conditional generation model or conditional generation network. The conditional generation model is a generation model based on input action representations; or, the conditional generation model is a neural network model based on input action representations.

[0157] In the embodiment, by obtaining the training sample containing the action sequence and the corresponding multiple groups of action representations, the initial neural network model is pre-trained, which can enable the model to learn the complex mapping relationship between the action sequence and the multiple action representations, thereby improving the generalization ability and performance of the model in generating diversified and accurate actions, and finally obtaining a pre-trained action generation model capable of generating high-quality action sequences based on different action representations.

[0158] In an optional embodiment, the at least two action representations correspond to respective representation types, and the pre-training of the initial model based on the at least one training sample to obtain the pre-trained action generation model comprises: constructing at least two parallel representation spaces based on the set of action representations corresponding to each training sample, each representation space including action representations belonging to the same representation type, and the representation types corresponding to respective ones of the at least two representation spaces being different, the at least two parallel representation spaces being used to indicate that the initial model learns different representation types respectively; for each training sample in the at least one training sample, using the action representations of the training sample in the at least two representation spaces as constraint conditions in the generation process of the initial model to generate a first sample predicted action; training the initial model based on a loss between the first sample predicted action and the action; and obtaining the pre-trained action generation model based on the initial model.

[0159] Optionally, in the pre-training process, the at least two action representations included in each training sample are added to the corresponding representation space as the training sample is obtained. Or, in the pre-training process, the action representations included in a batch of training samples are added to the corresponding representation space as the initial model obtains the batch of training samples. Batch refers to grouping of training samples in the model training process. Each batch of training samples (example actions) is input into the initial model at the same time or in sequence, the initial model calculates the total loss corresponding to the batch of training samples, usually using the mean or median of the losses as the total loss corresponding to the batch of training samples, and the initial model updates the model parameters based on the total loss until the initial model converges for the batch of training samples. The next batch of training is performed. Or, in the pre-training process, the initial model obtains at least one training sample, and constructs at least two representation spaces based on the at least two action representations included in each training sample.

[0160] Optionally, one representation type corresponds to one representation space; or, multiple representation types correspond to one representation space. For example, the representation types include {character feature, action content, space constraint, music feature}, and at least two representation spaces can be constructed, such as [character feature], [action content], [space constraint], [music feature], or [character feature, action content], [space constraint], [music feature], or [character feature, action content], [space constraint, music feature], or [character feature, action content, space constraint], [music feature], etc. Wherein, the square brackets [] represent a representation space, and the representation space includes action representations corresponding to one or more representation types.

[0161] Optionally, the at least two parallel representation spaces are used to indicate that the pre-trained action generation model learns different representation types respectively, and the at least two parallel representation spaces refer to at least two independent condition sets or condition dimensions. That is, the parallel representation spaces support the pre-trained action generation model to generate a new action based on the combination of each action representation in the at least two parallel representation spaces. For example, the at least two parallel representation spaces include [role feature] and [action content], the action representations included in the representation space [role feature] are {“old man”, “child”, “patient”}, and the action representations included in the representation space [action content] are {“staggering”, “crawling”, “walking with a drip”}. Then, the pre-trained action generation model can at least generate the following action representations corresponding to the actions {“old man staggering”, “old man crawling”, “old man walking with a drip”, “child staggering”, “child crawling”, “child walking with a drip”, “patient staggering”, “patient crawling”, “patient walking with a drip”}. Mapping the action representations of different representation types into the parallel representation spaces can realize the decoupling of the pre-trained action generation model to the action representations of different representation types, or in other words, realize the decoupling of the representation spaces.

[0162] Optionally, for each training sample in the at least one training sample, the action representation of the training sample in the at least two representation spaces is used as a constraint condition in the initial model generation process to generate the first sample predicted action, including: for each training sample in the at least one training sample, the initial model obtains the first sample predicted action based on the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process.

[0163] Optionally, for each of the at least one training sample, the initial model generates the first sample predicted action based on the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process, including: the initial model generates the first sample predicted action based on the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process, and noise data as an initial value, the noise data being noise of the same dimension or dimension randomly collected by the initial model and the action in the training sample, the noise data being sampled by the initial model from at least one of a Gaussian distribution, a uniform distribution, a polynomial distribution or an exponential distribution; or, the initial model generates the first sample predicted action based on the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process, and a latent representation as an initial value, the latent representation being a latent feature representation of the action in the training sample encoded by the initial model using an encoder, the latent feature representation being understood as an important feature in the action, that is, the encoding of the action is to extract the latent feature representation of the action, and the latent feature representation is used to identify all features in the action.

[0164] Optionally, the initial model generates the first sample predicted action based on the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process, and noise data as an initial value, including: the initial model takes the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process, and performs T times of denoising on the noise data to obtain the first sample predicted action, T being a positive integer; or, the initial model takes the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process, and uses a generator to generate the noise data to obtain the first sample predicted action.

[0165] Optionally, the initial model generates the first sample predicted action based on the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process, and a latent representation as an initial value, including: the initial model takes the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process, and uses a decoder to decode the latent representation to obtain the first sample predicted action.

[0166] Optionally, based on the loss between the first sample predicted action and the action, the model parameters in the initial model are updated to obtain a pre-trained initial model. The method of updating the model parameters based on the loss is similar to the processing of updating the model parameters in the above-mentioned embodiments.

[0167] In this embodiment, by constructing at least two parallel representation spaces, each corresponding to a different action representation type, the initial model can learn the characteristics of different representation types during the training process and generate predicted actions as constraint conditions, thereby optimizing the model's understanding and generation ability of diversified action representations. Through calculating the loss between the predicted action and the actual action, the model is trained, and finally a pre-trained action generation model capable of generating high-quality actions based on multiple representation types is obtained, significantly improving the model's generalization ability and the accuracy of action generation.

[0168] In an optional embodiment, the pre-training of the action generation model includes conditional training and unconditional training. The conditional training is to train the action generation model according to the predicted action generated by the action representation, and the unconditional training is to train the action generation model directly to generate the predicted action.

[0169] Based on the loss between the first sample predicted action and the action, the initial model is trained, including:

[0170] For each action in the at least one training sample, a second sample predicted action is generated without using the constraint condition in the generation process of the initial model; and based on the first loss and the second loss, the initial model is trained, the first loss being the loss between the first sample predicted action and the action, and the second loss being the loss between the second sample predicted action and the action.

[0171] Optionally, for each action in the at least one training sample, a second sample predicted action is generated without using the constraint condition in the generation process of the initial model, including: the initial model generates the second sample predicted action based on noise data as an initial value without using the constraint condition in the generation process, the noise data being noise with the same dimension or random dimension as the action in the training sample collected by the initial model, the noise data being sampled by the initial model from at least one of a Gaussian distribution, a uniform distribution, a polynomial distribution or an exponential distribution; or, the initial model generates the second sample predicted action based on a latent representation as an initial value without using the constraint condition in the generation process, the latent representation being a latent feature representation of the action in the training sample encoded by the initial model using an encoder, the latent feature representation being understood as important features in the action, that is, the encoding of the action is to extract the latent feature representation of the action, and the latent feature representation is used to identify all features in the action.

[0172] Optionally, the initial model does not employ the constraint condition in the generation process, and generates the first sample predicted action based on the noise data as an initial value, including: the initial model does not employ the constraint condition in the generation process, and obtains a second sample predicted action by denoising the noise data T times, where T is a positive integer; or the initial model does not employ the constraint condition in the generation process, and obtains the second sample predicted action by generating the noise data using the generator.

[0173] Optionally, the initial model does not employ the constraint condition in the generation process, and generates the first sample predicted action based on the latent representation as an initial value, including: the initial model does not employ the constraint condition in the generation process, and obtains a second sample predicted action by decoding the latent representation using the decoder.

[0174] Optionally, in the pre-training process, the initial model is conditionally trained based on the first loss calculated in the above manner; and the initial model can also be unconditionally trained based on the second loss calculated in the above manner.

[0175] In the conditional and unconditional fusion training of the initial model, a null probability is set for the initial model, and the null probability is used to indicate a probability that the action representation is nullified when the initial model generates a predicted action according to the action representation in the training process; the null probability can also be understood as a probability that the initial model is unconditionally trained. For example, the null probability is 10%.

[0176] In the training process of the initial model, the initial model is trained based on the first loss when the action representation is not nullified, and the initial model is trained based on the second loss when the action representation is nullified.

[0177] Optionally, the initial model is the pre-trained action generation model; or in the inference stage, the action generation model is obtained by adjusting the hyperparameters in the initial model.

[0178] In summary, the method provided by the embodiments of the present application shows a pre-training method of the action generation model, which is trained based on the training sample including the action and at least two action representations, so as to obtain the pre-trained action generation model. Through the conditional training, the action generation model obtained by the training can generate based on different input action representations, and the generated new action can conform to the action representation, which lays a foundation for personalized customization using the action generation model.

[0179] Further, it is also shown that at least two action representations are mapped into at least two parallel representation spaces, and the pre-training is based on the at least two parallel representation spaces, i.e., one or more independent conditional sets or conditional dimensions. That is, the parallel representation spaces support the pre-trained action generation model to generate new actions based on the combination of the action representations in the at least two parallel representation spaces. For example, the at least two parallel representation spaces include [character features] and [action content], the action representations included in the representation space [character features] are {“old man”, “child”, “patient”}, and the action representations included in the representation space [action content] are {“staggering”, “crawling”, “walking with a drip bottle”}. Then, the pre-trained action generation model can at least generate the following actions {“old man staggering”, “old man crawling”, “old man walking with a drip bottle”, “child staggering”, “child crawling”, “child walking with a drip bottle”, “patient staggering”, “patient crawling”, “patient walking with a drip bottle”}. Mapping the action representations of different representation types into parallel representation spaces can realize the decoupling of the pre-trained action generation model to the action representations of different representation types, or in other words, realize the decoupling of the representation spaces. Thus, the pre-trained action generation model can at least generate various new actions according to the combination of conditions in the representation spaces, and further improve the personalized customization capability of the pre-trained action generation model.

[0180] Further, in addition to conditional training, the initial model is also unconditionally trained, i.e., the initial model is expected to be able to generate the actions in the training samples without the guidance or constraint of the action representations. The implementation of conditional and unconditional fusion training can prevent the model from overfitting, i.e., the model can only generate according to the learned action representations, and also improve the generalization capability of the model, so that the model can learn the method of unconditional generation.

[0181] Next, the use process of the action generation model after the retraining is completed is shown. In this embodiment, the method further includes:

[0182] The trained action generation model is obtained, and the trained action generation model is used to generate n styles of actions. The identifier of the expected style and the user expected action are obtained, and the expected style is one of the n styles. The user expected action and the identifier are generated by the trained action generation model to generate an action with the expected style.

[0183] Optionally, the retraining of the action generation model is performed using at least n example actions, and the trained action generation model is obtained. The at least n example actions have n personalized styles.

[0184] Optionally, the user desired action can be null, i.e., the generation of the action is only based on the identifier of the desired style.

[0185] Optionally, the action generation model performs feature extraction on the user desired action and the identifier to obtain a user desired action representation, and generates the action with the desired style as a constraint condition of the user desired action representation. The generated action not only has the desired style, but also meets the user desired action.

[0186] In this embodiment, the trained action generation model is used to generate n kinds of action sequences with the desired style, and the user desired action can be replaced by a user desired action sequence. By using the trained action generation model, the action generation is performed according to the user desired action sequence and the identifier, and an action sequence with the desired style is obtained.

[0187] In one embodiment, by using the trained action generation model, the action generation is performed according to the user desired action and the identifier, and an action with the desired style is obtained, including:

[0188] By using the trained action generation model, the first output action and the second output action are generated, the first output action is generated by the trained action generation model based on the user desired action and the identifier as a constraint condition, and the second output action is generated by the trained action generation model without using the constraint condition.

[0189] In the generation process of the action generation model, the action generation model generates the second output action based on the noise data sampled without using the constraint condition, and generates the first output action based on the user desired action representation as a constraint condition; or, the action generation model generates the second output action based on the latent representation sampled without using the constraint condition, and generates the first output action based on the input action representation as a constraint condition.

[0190] Optionally, by using the trained action generation model, the first output action and the second output action are fused to obtain the action with the desired style.

[0191] Optionally, the action with the personalized feature is the sum of the first output action and the second output action.

[0192] Optionally, the first output action and the second output action are interpolated to obtain the action with the desired style. For example, the finally obtained action with the desired style is the sum of the conditional action and the unconditional action, the conditional action is the product of the first output sequence and the conditional weight, and the unconditional action is the product of the second output sequence and the unconditional weight.

[0193] In this embodiment, the first output action can be replaced by a first output action sequence, and the second output action can be replaced by a second output action sequence. The first output action sequence and the second output action sequence are generated by the trained action generation model, the first output action sequence is generated by the trained action generation model based on the user expected action sequence and the identifier as the constraint condition, and the second output action sequence is generated by the trained action generation model without the constraint condition.

[0194] In summary, the method provided in the embodiments of the present application shows the specific use process of the trained action generation model, or the prediction process of the action generation model. The user can customize the personalized action with the expected feature by inputting the identifier corresponding to the expected feature. The user can quickly obtain the personalized action with the expected feature, improving the interaction efficiency. And the expected feature is represented by an identifier, which can be determined according to the sample action input by the user in the training process, that is, the user only needs to provide a sample action, so that the action generation model can effectively learn the expected feature and retain the generation mode of the expected feature. Thus, the personalized customization demand of the user for the action sequence with the expected feature is met.

[0195] Further, the above method makes the action sequence finally output by the action generation model fuse the features corresponding to the conditional generation and the unconditional generation, so that the action sequence finally output by the action generation model can have more diversity on the basis of conforming to the action representation. When fusing, the weight values corresponding to the first output sequence and the second output sequence can be set, and then the output action sequence is adjusted to be more in line with the condition or more diversified, thereby improving the generation ability of the action generation model and better meeting the personalized customization demand.

[0196] The training of the action generation model mainly includes two stages of model pre-training and personalized fine-tuning. Next, taking the structure of the action generation model as a generative diffusion model (also referred to as a diffusion generative model), the conditions that can be input into the action generation model include role features, action content, and motion trajectories (also referred to as action trajectories and root trajectories). The two stages will be described in detail.

[0197] 1. Model pre-training

[0198] Step 1, at least one training sample is obtained, and the training sample includes an action sequence, a text prompt, and a spatial constraint.

[0199] The action sequence is a preprocessed sequence. Each frame of the action sequence can be represented as the absolute coordinates of the root node and the rotation information of each joint with respect to the parent joint. The action sequence is used to describe the pose of the skeleton in the virtual space, i.e., the position and posture of the skeleton in the virtual space. When performing feature extraction or encoding on the skeleton, the skeleton can be represented as a tree structure, and a plurality of joint nodes in the skeleton are extracted, each joint node corresponding to a respective parent node, and there is a root node in the plurality of joint nodes of the skeleton, and the parent node of the root node can be a certain point on the world coordinate system of the virtual space. Depending on the adopted skeletal model, the extracted joint nodes in the skeleton and the determined root node are different.

[0200] The text prompt includes character features and action content. The character features are used to indicate the features of the performer, i.e., the performer of the action. The action content is the text content for the action sequence. The character features and the action content can be included in the same text prompt or can be demonstrated separately. For example, the text prompt is "an obese person is walking forward", "obese person" is the character feature, and "walking forward" is the action content; or, the text prompt is "[obese person], [walks forward with hands crossed]”.

[0201] Optionally, the text prompt is manually input; or, the text prompt is determined by the action generation model using an action recognition method; or, the text prompt is determined by the server using an action recognition method and added to the training sample.

[0202] The space constraint is used to indicate the motion trajectory of the performer in the action sequence corresponding to the training sample. The space constraint is a sparse space sample (also referred to as a world coordinate / ground plane coordinate) uniformly sampled on the root trajectory based on the root node of the action sequence. The root trajectory is a two-dimensional trajectory line formed by the projection points after projecting the root node of the skeleton in the action sequence onto the ground plane. The ground plane is a certain coordinate plane in the spatial coordinate system. For example, for an action sequence a including 20 frames, the root trajectory obtained by projecting the root node of the skeleton in the action sequence a is shown in part (1) of FIG. 7, which includes 20 projection points, i.e., projection point 1 to projection point 20; the projection points of the space samples in the space constraint corresponding to the action sequence a are obtained by uniformly sampling the root trajectory, as shown in part (2) of FIG. 7, the sampled points include trajectory point 2, trajectory point 8, trajectory point 11, trajectory point 16 and trajectory point 20, and the purpose of uniform sampling is to obtain sparse points that can represent the trend of the motion trajectory. Optionally, the space constraint is constructed according to the coordinates of the sampled trajectory points on the ground plane, or the space constraint is constructed according to the coordinates of the sampled trajectory points in the world coordinate system.

[0203] Step 2, based on at least one training sample, at least two representation spaces are constructed.

[0204] The representation space comprises at least one action sequence representation corresponding to a training sample, and the action sequence representation comprises at least one of a feature representation corresponding to a text prompt in the training sample and a feature representation corresponding to a space constraint. The feature representation corresponding thereto can be a feature vector, a feature matrix, a feature tensor, etc., and the present application does not limit this.

[0205] Optionally, feature extraction is performed on the text prompt included in each of the at least one training sample to obtain a text feature c v t , wherein c v is a feature representation corresponding to a role feature in the feature representation corresponding to the text prompt; and c t is a feature representation corresponding to an action content in the feature representation corresponding to the text prompt.

[0206] For example, a role feature in the text prompt included in the training sample is obtained. The role feature is preprocessed by denoising, word segmentation, standardization, etc. Denoising (which can also be referred to as text cleaning) is used to remove noise such as special words, words without semantic information, etc. in the role feature, word segmentation is used to divide the text into characters or subwords or words, a character refers to a basic component in the text, for example, each character and punctuation mark in Chinese text or each letter and punctuation mark in English text, a subword refers to a result obtained by dividing a long word or an unknown word or an unknown phrase, for example, the word “natural” can be divided into subword units “natura” and “l” in subword division, and a word refers to a meaningful word unit in the text, for example, “one” in the sentence “I have a cat” is usually divided into a word. The preprocessed role feature is text encoded to obtain a raw feature representation corresponding to the role feature, and the text encoding can be implemented by using any text encoder. Optionally, the raw feature representation corresponding to the role feature is directly output as the feature representation c v corresponding to the role feature; or, features with a contribution degree higher than a contribution threshold are selected from the raw feature representation corresponding to the role feature as the feature representation c v corresponding to the role feature, the contribution degree is used to indicate the influence degree of the feature in the training process; or, the raw feature representation corresponding to the role feature is dimensionally reduced to obtain the feature representation c v corresponding to the role feature.

[0207] For example, an action content in the text prompt included in the training sample is obtained. The action content is preprocessed by denoising, word segmentation, standardization, etc. The preprocessed action content is text encoded to obtain a raw feature representation corresponding to the action content. Optionally, the raw feature representation corresponding to the action content is directly output as the feature representation c t ​Or, selecting a feature with a contribution degree higher than a contribution degree threshold from the original feature representation corresponding to the action content as the feature representation c corresponding to the action content t The contribution degree is used to indicate the influence degree of the feature in the training process; or, performing dimension reduction on the original feature representation corresponding to the action content to obtain the feature representation c corresponding to the action content t .

[0208] Optionally, feature extraction is performed on the spatial constraint included in each of the at least one training sample to obtain a feature representation c corresponding to the spatial constraint r The spatial constraint includes the ground plane coordinates or absolute coordinates (also referred to as world coordinates) of each projection point, and the feature extraction performed on the spatial constraint obtains the feature representation c corresponding to the spatial constraint r .

[0209] The at least two representation spaces are parallel representation spaces, and the at least two representation spaces can guide the action generation model to generate actions in parallel.

[0210] The at least two representation spaces include a text prompt representation space and a spatial constraint representation space, and the target of the action generated by the action generation model is to meet both the text prompt and the spatial constraint.

[0211] According to the text prompt in the training sample, a feature representation corresponding to a role feature and a feature representation corresponding to an action content are obtained, and the feature representation corresponding to the role feature and the feature representation corresponding to the action content of each training sample are spliced to obtain a text prompt representation space.

[0212] Splicing the text prompt representation space and the spatial constraint representation space (splicing the at least two representation spaces) obtains a representation space corresponding to the action generation model.

[0213] Step 3, according to the at least two representation spaces and the training sample, pre-training the action generation model to obtain a pre-trained action generation model.

[0214] According to the at least two representation spaces and the training sample, the action generation model is trained in a conditional and unconditional manner to obtain a pre-trained action generation model.

[0215] The random noise is obtained, and the random noise and the feature representation corresponding to the text prompt in the training sample and the feature representation corresponding to the space constraint are input into the action generation model to obtain a predicted action sequence; the action generation model is trained according to a loss between the predicted action sequence and the action sequence, the loss between the predicted action sequence and the action sequence includes a generation loss and a geometric loss, the generation loss (which can also be referred to as a reconstruction loss) is used to measure the difference between the predicted value and the true value, and the geometric loss is used to indicate the error or inconsistency of the predicted action sequence in the geometric space. The geometric loss of the action generation model includes a rotation position loss, and the rotation position loss is used to indicate the difference between the rotation pose and position of each joint in the predicted action sequence and the rotation pose and position of each joint in the action sequence. The loss function of the action generation model may be represented as follows.

[0216] In the formula, represents the generation loss, represents the rotation position loss, and λ pos is a hyperparameter, that is, a parameter that can be set manually according to the expected effect.

[0217] wherein the generation loss is calculated according to the following formula.

[0218] In the formula, t represents a current time step, T represents a total number of time steps, t~[1:T] represents that t takes a value from 1 to T, x0 represents an action sequence, and x0~q(x0|c) represents that the action sequence is subject to a conditional distribution q(x0|c). represents a predicted action sequence, that is, the predicted action sequence refers to the sequence x t predicted by the action generation model under the constraint of the condition c. represents an expectation (Expectation); represents an expectation calculation on the time step t and the action sequence x0 obtained by sampling.

[0219] The calculation formula of the rotation position loss is as follows.

[0220] In the formula, represents a predicted action sequence, that is, the predicted action sequence refers to the sequence x tis predicted by the action generation model under the constraint of condition c; x0represents the action sequence; s represents the skeleton; FKis the forward kinematics function that converts joint rotation to joint position.

[0221] Optionally, in the pre-training stage of the action generation model, the space c has a 10% probability of being emptied, so that the action generation model is trained for unconditional generation. That is, the training of the action generation model includes conditional training {obtain random noise, and input the random noise and the feature representation corresponding to the text prompt in the training sample and the feature representation corresponding to the space constraint into the action generation model to obtain a predicted action sequence; train the action generation model according to the loss between the predicted action sequence and the action sequence} and unconditional training {obtain random noise, and input the random noise into the action generation model to obtain a predicted action sequence; train the action generation model according to the loss between the predicted action sequence and the action sequence}. The conditional training and the unconditional training correspond to a first probability and a second probability, respectively, the first probability is 0.9, and the second probability is 0.1.

[0222] Optionally, after the pre-training stage is completed, the action generation model can perform conditional and unconditional prediction. In order to make the action generation model obtain more conditional or more diverse actions, the output of the action generation model can be set to fuse the predicted action sequence generated by conditional generation and the predicted action sequence generated by unconditional generation in the inference stage, and the action generation model can be further optimized. The inference stage can be set after the personalized fine-tuning is completed, or the inference stage can be set after the model pre-training and the personalized fine-tuning are completed. The final output of the action generation model is calculated according to the following formula.

[0223] In the formula, t represents the t-th time step, x t represents the action sequence at the t-th time step, c represents the spliced at least two representation spaces. γ is a hyperparameter, that is, a parameter that can be manually adjusted by a worker. The larger γ is, the more the final output of the action generation model conforms to the input condition; the smaller γ is, the more diverse the final output of the action generation model is.

[0224] 2. Personalized fine-tuning, that is, retraining of the model

[0225] Step 1, obtaining at least one sample action sequence.

[0226] Based on the at least one sample action sequence, the feature representation corresponding to the action content and the space constraint representation space corresponding to each sample action sequence are obtained. The obtaining method is as shown in the above pre-training stage, which is not repeated here.

[0227] Step 2, based on the feature representation and the space constraint representation space corresponding to the action content corresponding to the at least one sample action sequence, calculate the unique identifier corresponding to the sample action sequence.

[0228] The unique identifier is used to identify the personalized action features of the sample action sequence, that is, the unique identifier is used to uniquely identify the personalized action features corresponding to the sample action sequence in the action generation model. The unique identifier can also be called a style identifier or an identifier. The unique identifier is a feature representation corresponding to the role feature. The unique identifier The calculation method is as follows.

[0229] In the formula, t is the time step; represents the sample action sequence; represents the predicted action sequence predicted by the action generation model based on the sequence under the condition predicted by the action generation model; represents the parameter corresponding to the minimum value in the parentheses (i.e., the unique identifier to be obtained).

[0230] In the calculation, the initial value of in the representation space is assigned, and the initial value can be 0 or other numbers or letters or a combination of numbers and letters. Then, the minimum is obtained by continuously iterating the calculation as the unique identifier of the sample action sequence.

[0231] Step 3, based on the at least one sample action sequence and the unique identifier of each sample action sequence, determine at least one sample sample; and determine at least one prior sample according to the training sample set of the action generation model.

[0232] Based on each sample action sequence in the at least one sample action sequence and the unique identifier corresponding thereto, determine the sample sample corresponding to the sample action sequence, for example, the sample sample is represented as According to the training sample set of the action generation model, obtain at least one training sample in the training sample set, determine a prior sample based on each training sample, and the prior sample can be represented as

[0233] According to the representation space saved by the action generation model (formed in the pre-training stage), obtain a group of random action sequence representations; input the random action sequence representations into the action generation model to obtain a prior action sequence; and obtain a prior sample according to the random action sequence representations and the prior action sequence.

[0234] Step 4, fine-tuning the action generation model based on at least one sample sample and at least one prior sample.

[0235] Obtain an i-th sample sample in the at least one sample sample and a prior sample randomly obtained from the at least one prior sample; take the i-th sample sample and the randomly obtained prior sample as inputs of the action generation model respectively, and predict to obtain an i-th second predicted action sequence and a first predicted action sequence; adjust model parameters of the action generation model based on a loss between the i-th second predicted action sequence and a sample action sequence in the i-th sample sample and a loss between the first predicted action sequence and an action sequence in the prior sample.

[0236] The model parameters of the action generation model are calculated as follows.

[0237] In the formula, θ * is a target model parameter; θ is an original model parameter; represents a representation space in a sample sample; represents a sample action sequence in a sample sample; represents at least one sample sample; represents a representation space in a prior sample; represents an action sequence in a prior sample; represents at least one prior sample; t is a time step; represents a second predicted action sequence; represents a first predicted action sequence; λ gen is a prior weight.

[0238] After the retraining of the action generation model is completed, the trained action generation model can be used to customize the action. In the customization, there are two cases:

[0239] 1. Personalized features have been learned

[0240] Personalized features have been learned means that the expected personalized features to be generated have been trained as sample action sequences during training. At this time, new action generation can be performed based on the identifier corresponding to the sample action sequence, or the identifier corresponding to the personalized feature, combined with any condition.

[0241] 2. Personalized features have not been learned

[0242] The personalized feature that has not been learned refers to a personalized feature expected to be generated and not appearing in the trained sample action sequence. At this time, an action sequence with the personalized feature can be made as a sample action sequence, the training method of the action generation model is executed, thereby obtaining an identifier corresponding to the personalized feature, and then the generation of a new action is performed based on the identifier combined with any condition.

[0243] In an embodiment, an application scenario of the training method of the action generation model is provided, which can be specifically applied to the field of games. The generated action sequence can be applied in game characters. For example, in order to increase the immersion of game players, game developers usually set different styles of action sequences for different personality NPCs (Non-Player Character) in the game. In order to improve the development efficiency of the game, the action generation model that can meet the personalized customization requirements can be trained by using the training method of the action generation model shown in the present application. In the pre-training stage, any training sample set including action sequences can be used; in the training stage / fine-tuning stage, a role identifier of a to-be-generated role is generated by using a segment of action sequence of the to-be-generated role made by the game developer; in the inference stage, a new action sequence of the to-be-generated role that meets the action description information is generated by using the to-be-generated identifier and the action description information. The specific implementation process is as follows.

[0244] Step 1, at least one training sample is obtained, each training sample including an action sequence and a set of action sequence representations corresponding to the action sequence, the set of action sequence representations including at least two action sequence representations.

[0245] Optionally, the training sample is obtained from a public training sample set; or, the training sample is obtained based on a training sample set designed for the game.

[0246] Step 2, based on the at least one training sample, an initial model is pre-trained to obtain a pre-trained action generation model, the initial model being a neural network model for generating an action sequence based on a set of action sequence representations.

[0247] In some embodiments, based on the set of action sequence representations corresponding to each training sample, at least two parallel representation spaces are constructed, each representation space including action sequence representations belonging to the same representation type, the representation types corresponding to each of the at least two representation spaces being different, and the at least two parallel representation spaces being used to indicate that the initial model learns different representation types respectively.

[0248] In some embodiments, for each of the at least one training sample, the action sequence representation of the training sample in the at least two representation spaces is taken as a constraint condition in the initial model generation process, thereby generating a first predicted action sequence.

[0249] In some embodiments, the initial model is trained based on a loss between the first sample predicted action sequence and the action sequence.

[0250] In some embodiments, the pre-trained action generation model is obtained based on the initial model.

[0251] In some embodiments, the initial model is trained based on a loss between the first sample predicted action sequence and the action sequence, including: for each action sequence in the at least one training sample, generating a second sample predicted action sequence in a generation process of the initial model without using the constraint condition; and training the initial model based on a first loss and a second loss, the first loss being a loss between the first sample predicted action sequence and the action sequence, and the second loss being a loss between the second sample predicted action sequence and the action sequence.

[0252] Step 3, obtaining a prior action sequence representation and a prior action sequence, the prior action sequence having diversified action features learned by the action generation model in pre-training; and obtaining a sample action sequence representation and a sample action sequence, the sample action sequence having personalized role action features.

[0253] The sample action sequence is an action sequence designed by a game developer for a virtual role, and has personalized role features corresponding to the virtual role. That is, the sample action sequence is an action sequence corresponding to the virtual role. Optionally, the sample action sequence is an action sequence designed by the game developer for a new role.

[0254] In some embodiments, the prior action sequence representation and the prior action sequence are obtained by: obtaining any prior action sequence representation from a representation space of the action generation model, the representation space being a set of action sequence representations learned by the action generation model in pre-training; and inputting the prior action sequence representation into the action generation model to generate the prior action sequence.

[0255] In some embodiments, the sample action sequence representation includes a first action sequence representation and a second action sequence representation, the second action sequence representation being an action sequence representation corresponding to a role identifier, the role identifier being used to identify personalized role features of the sample action sequence; the sample action sequence representation is obtained by: obtaining the first action sequence representation, the first action sequence representation being a constraint condition to be replaced in the sample action sequence; and obtaining the second action sequence representation, the second action sequence representation being obtained by performing feature extraction on the role identifier by the action generation model; inputting the first action sequence representation and the second action sequence representation into the pre-trained action generation model to obtain a predicted action sequence; and updating the role identifier based on a loss between the predicted action sequence and the sample action sequence.

[0256] In some embodiments, the first action sequence representation and the second action sequence representation are input into the pre-trained action generation model to obtain a predicted action sequence, including: in a jth iteration process, the first action sequence representation and a (j-1)th round of the second action sequence representation are input into the action generation model to obtain a jth round of the predicted action sequence, j is a positive integer; based on a loss between the predicted action sequence and the sample action sequence, the role identifier is updated, including: based on a loss between the jth round of the predicted action sequence and the sample action sequence, a (j-1)th round of the role identifier is updated to obtain a jth round of the role identifier; a jth round of the second action sequence representation is obtained, which is obtained by performing feature extraction on the jth round of the role identifier by the action generation model; j is updated to j+1, and the jth iteration process is continued to be executed until the jth round of the role identifier meets an iteration termination condition.

[0257] Step 4, inputting the prior action sequence representation into the action generation model to obtain a first predicted action sequence; and inputting the sample action sequence representation into the action generation model to obtain a second predicted action sequence.

[0258] Step 5, training the action generation model based on the prior loss and the sample loss, the prior loss being obtained based on the first predicted action and the prior action sequence, and the sample loss being obtained based on the second predicted action and the sample action sequence.

[0259] In some embodiments, a model loss of the action generation model is calculated, the model loss being a sum of a first intermediate value and a second intermediate value, the first intermediate value being a product of the prior loss and a prior weight, and the second intermediate value being a product of the sample loss and a sample weight; and the action generation model is trained based on the model loss.

[0260] In some embodiments, a model loss of the action generation model is calculated, the model loss being a sum of a first intermediate value and a second intermediate value, the first intermediate value being a product of the prior loss and a prior weight, and the second intermediate value being a product of the sample loss and a sample weight, including: in an ith round of training process, an ith round of the model loss is calculated, the ith round of the model loss being a sum of an ith round of the first intermediate value and an ith round of the second intermediate value, the ith round of the first intermediate value being a product of an ith round of the prior loss and the prior weight, and the ith round of the second intermediate value being a product of an ith round of the sample loss and the sample weight, i being a positive integer; and the action generation model is trained based on the model loss, including: determining an ith round of model parameters that make the ith round of the model loss decrease; and updating the model parameters of the action generation model to the ith round of the model parameters to obtain the action generation model trained for the ith round.

[0261] Step 6, obtaining the trained action generation model, the trained action generation model being used to generate action sequences of n roles.

[0262] Step 7, obtaining role identification and action description information of a to-be-generated role, the to-be-generated role being one of the n roles, and the action description information being used to describe action content of the to-be-generated role expected by the user.

[0263] Step 8, inputting the action description information and the role identification into the trained action generation model to generate an action sequence corresponding to the to-be-generated role by the trained action generation model.

[0264] In some embodiments, the first output action sequence and the second output action sequence are generated by the trained action generation model, the first output action sequence being generated by the action generation model based on the action description information and the role identification as constraint conditions, and the second output action sequence being generated by the action generation model without using the constraint conditions.

[0265] In some embodiments, the action sequence corresponding to the to-be-generated role is obtained based on the first output action sequence and the second output action sequence. The action sequence is an action sequence of the to-be-generated role satisfying the action description information.

[0266] Please refer to FIG. 8, which shows a structural block diagram of an action generation model training device provided in an example embodiment of the present application. The device has the function of implementing the example of the above-mentioned action generation model training method, which can be realized by hardware or corresponding software executed by hardware. The device can be the computer device introduced above or be arranged in the computer device. As shown in FIG. 8, the device can include an obtaining module 510, a generating module 520 and a training module 530.

[0267] The obtaining module 510 is configured to obtain a prior action and a prior action representation of the prior action, the prior action having diversified action features learned by the action generation model during pre-training; and obtain a sample action and a sample action representation of the sample action, the sample action having individualized action features.

[0268] The generating module 520 is configured to perform action generation based on the prior action representation by the pre-trained action generation model to obtain a first predicted action; and perform action generation based on the sample action representation by the pre-trained action generation model to obtain a second predicted action.

[0269] The training module 530 is configured to retrain the pre-trained action generation model based on a prior loss between the first predicted action and the prior action and a sample loss between the second predicted action and the sample action to obtain a trained action generation model.

[0270] In some embodiments, the training module 530 includes a first calculation sub-module and a first training sub-module.

[0271] The first calculation sub-module is configured to calculate a model loss of the action generation model, the model loss being calculated based on a first intermediate value and a second intermediate value, the first intermediate value being a product of a prior loss and a prior weight, and the second intermediate value being a product of a sample loss and a sample weight.

[0272] The first training sub-module is configured to retrain the pre-trained action generation model based on the model loss to obtain a trained action generation model.

[0273] In some embodiments, the first calculation sub-module is further configured to calculate an i-th round model loss in an i-th round of training, the i-th round model loss being calculated based on an i-th round first intermediate value and an i-th round second intermediate value, the i-th round first intermediate value being a product of an i-th round prior loss and the prior weight, and the i-th round second intermediate value being a product of an i-th round sample loss and the sample weight, i being a positive integer; the first training sub-module is further configured to determine i-th round model parameters that reduce the i-th round model loss; and update the model parameters of the action generation model to the i-th round model parameters to obtain an action generation model trained in the i-th round; wherein the i-th round prior loss is a loss between an i-th round first predicted action and the prior action, the i-th round first predicted action being generated by inputting the prior action representation into the action generation model trained in an (i-1)-th round; and the i-th round sample loss is a loss between an i-th round second predicted action and the sample action, the i-th round second predicted action being generated by inputting the sample action representation into the action generation model trained in the (i-1)-th round.

[0274] In some embodiments, the apparatus further includes a sample obtaining module and a pre-training module.

[0275] The sample obtaining module is configured to obtain at least one training sample, each training sample including an action and a set of action representations corresponding to the action, the set of action representations including at least two action representations.

[0276] The pre-training module is configured to pre-train an initial model based on the at least one training sample to obtain a pre-trained action generation model, the initial model being a neural network model for generating an action based on a set of action representations.

[0277] In some embodiments, the at least two action representations correspond to respective representation types; and the pre-training module includes a construction sub-module, a first generation sub-module, a second training sub-module, and a determination sub-module.

[0278] The construction sub-module is configured to construct at least two parallel representation spaces based on the set of action representations corresponding to each training sample, each representation space including action representations belonging to a same representation type, the representation types corresponding to respective representation spaces in the at least two representation spaces being different, and the at least two parallel representation spaces being used to indicate that the initial model learns different representation types respectively.

[0279] a first generating sub-module configured to, for each of the at least one training sample, generate a first sample predicted action by taking, as a constraint condition in an initial model generation process, action representations of the training sample in the at least two representation spaces;

[0280] a second training sub-module configured to train the initial model based on a loss between the first sample predicted action and the action;

[0281] a determining sub-module configured to obtain a pre-trained action generation model based on the initial model.

[0282] In some embodiments, the apparatus further comprises a second generating sub-module. The second generating sub-module is configured to, for each action of the at least one training sample, generate a second sample predicted action without taking a constraint condition in the initial model generation process; and the second training sub-module is configured to train the initial model based on a first loss and a second loss, the first loss being a loss between the first sample predicted action and the action, and the second loss being a loss between the second sample predicted action and the action.

[0283] In some embodiments, the sample action representation comprises a first action representation and a second action representation, the second action representation being an action representation corresponding to an identifier, the identifier being used to identify a personalized feature of the sample action; and the obtaining module 510 comprises an obtaining sub-module, a third generating sub-module, and an updating sub-module.

[0284] The obtaining sub-module is configured to obtain the first action representation and the second action representation, the first action representation being a constraint condition to be replaced in the sample action, and the second action representation being obtained by performing feature extraction on the identifier by the action generation model;

[0285] The third generating sub-module is configured to perform action generation based on the first action representation and the second action representation by the pre-trained action generation model to obtain a second predicted action;

[0286] The updating sub-module is configured to update the identifier based on a loss between the second predicted action and the sample action.

[0287] In some embodiments, the third generating sub-module is further configured to, in the jth iteration process, input the first action representation and the (j-1)th round second action representation into the action generation model to obtain a jth round second predicted action, j being a positive integer; and the updating sub-module is further configured to update the (j-1)th round identifier based on a loss between the jth round second predicted action and the sample action to obtain a jth round identifier; obtain a jth round second action representation, the jth round second action representation being obtained by performing feature extraction on the jth round identifier by the action generation model; and update j to j+1, and continue to perform the jth iteration process until the jth round identifier satisfies an iteration termination condition.

[0288] In some embodiments, the obtaining module 510 is further configured to obtain any prior action representation from a representation space of the pre-trained action generation model, the representation space being a set of action representations learned by the pre-trained action generation model when pre-training; and obtain a prior action by performing action generation on the prior action representation through the pre-trained action generation model.

[0289] In some embodiments, the apparatus further includes a model obtaining module, a condition obtaining module, and a generation module.

[0290] The model obtaining module is configured to obtain a trained action generation model, the trained action generation model being configured to generate actions of n styles.

[0291] The condition obtaining module is configured to obtain an identifier of a desired style and a user desired action, the desired style being one of the n styles, and the user desired action being configured to describe an action desired by a user.

[0292] The generation module is configured to obtain an action with the desired style by performing action generation on the user desired action and the identifier through the trained action generation model.

[0293] In some embodiments, the generation module is further configured to generate a first output action and a second output action through the trained action generation model, the first output action being generated by the trained action generation model based on the user desired action and the identifier as constraint conditions, and the second output being generated by the action generation model without the constraint conditions; and obtain the action with the desired style based on the first output action and the second output action through the trained action generation model.

[0294] It should be noted that the apparatus provided in the above embodiments is only used as an example to illustrate the division of the above functional modules, and in actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be described here.

[0295] FIG. 9 shows a structural block diagram of a computer device according to an example embodiment of the present application.

[0296] The computer device 600 includes a central processing unit (CPU) 601, a system memory 604, including a random access memory (RAM) 602 and a read-only memory (ROM) 603, and a system bus 605 that couples the system memory 604 to the central processing unit 601. The computer device 600 also includes an input / output (I / O) system 606 that helps transfer information between the various devices within the computer device, and a mass storage device 607 for storing an operating system 613, application programs 614, and other program modules 615.

[0297] The input / output system 606 includes a display 608 for displaying information and an input device 609, such as a mouse, keyboard, or the like, for inputting information into the computer device. Both the display 608 and the input device 609 are connected to the central processing unit 601 through an input / output controller 610 that is connected to the system bus 605. The input / output system 606 can also include the input / output controller 610 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 610 provides output to a display screen, printer, or other type of output device.

[0298] The mass storage device 607 is connected to the central processing unit 601 through a mass storage controller (not shown) that is connected to the system bus 605. The mass storage device 607 and its associated computer-readable storage media provide non-volatile storage for the computer device 600. That is, the mass storage device 607 can include a computer-readable storage medium (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0299] Without loss of generality, the computer readable storage medium can include computer storage medium and communication medium. The computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable storage instructions, data structures, program modules or other data. The computer storage medium includes RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid state storage technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. Of course, those skilled in the art can know that the computer storage medium is not limited to the above several. The system memory 604 and the mass storage device 607 described above can be collectively referred to as memory.

[0300] The memory stores one or more programs configured to be executed by the one or more central processing units 601, and the one or more programs contain instructions for implementing the above method embodiments, and the central processing unit 601 executes the one or more programs to implement the method provided by each method embodiment.

[0301] According to various embodiments of the present application, the computer device 600 can also be connected to a remote computer device on a network through a network such as the Internet. That is, the computer device 600 can be connected to a network 612 through a network interface unit 611 connected to the system bus 605, or can be connected to other types of network or remote computer device systems (not shown) using the network interface unit 611.

[0302] The memory also includes one or more programs stored in the memory, and the one or more programs contain steps performed by the computer device in the method provided by the embodiments of the present application.

[0303] In an exemplary embodiment, a non-transitory computer readable storage medium having stored thereon a computer program is also provided, and the computer program is executed by a processor to implement the training method of the action generation model.

[0304] In an exemplary embodiment, a computer program product is also provided, and when the computer program product is executed by a processor, it is used to implement the training method of the action generation model.

[0305] It should be understood that "multiple" mentioned herein refers to two or more than two. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. In addition, the step numbers described herein only exemplarily show a possible execution order between steps, and in some other embodiments, the above steps can also be executed in a non-numbered order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in an order opposite to that shown in the figure, and the embodiments of the present application do not limit this.

[0306] The technical features of the above embodiments can be combined in any way. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0307] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for training an action generation model, performed by a computer device, the method comprising: Obtaining a priori action and a priori action representation of the prior action, wherein the prior action has diverse action features learned by the action generation model in pre-training; Acquire a sample action and a sample action representation of the sample action, wherein the sample action has personalized action features; Performing action generation based on the prior action representation using a pre-trained action generation model to obtain a first predicted action; Performing action generation based on the sample action representation using the pre-trained action generation model to obtain a second predicted action; and Based on the prior loss between the first predicted action and the prior action, and the sample loss between the second predicted action and the sample action, the pre-trained action generation model is retrained to obtain a trained action generation model.

2. The method according to claim 1, wherein retraining the pre-trained action generation model based on the a priori loss between the first predicted action and the a priori action, and the example loss between the second predicted action and the example action, to obtain a trained action generation model, comprises: Calculating a model loss of the action generation model, where the model loss is calculated based on a first intermediate value and a second intermediate value, where the first intermediate value is a product of the prior loss and the prior weight, and the second intermediate value is a product of the example loss and the example weight; and Based on the model loss, the pre-trained action generation model is retrained to obtain a trained action generation model.

3. The method according to claim 2, wherein calculating the model loss of the action generation model comprises: During the i-th round of training, calculate the i-th round model loss, where the i-th round model loss is calculated based on the i-th round first intermediate value and the i-th round second intermediate value, where the i-th round first intermediate value is the product of the i-th round prior loss and the prior weight, and the i-th round second intermediate value is the product of the i-th round sample loss and the sample weight, where i is a positive integer; The retraining of the pre-trained action generation model based on the model loss to obtain a trained action generation model includes: Determining the i-th round model parameters that reduce the i-th round model loss; and Updating the model parameters of the action generation model to the i-th round model parameters to obtain the action generation model trained after the i-th round; Among them, the i-th round prior loss is the loss between the first predicted action of the i-th round and the prior action, and the i-th round first predicted action is generated by inputting the prior action representation into the action generation model trained in the i-1th round. The i-th round sample loss is the loss between the second predicted action of the i-th round and the sample action, and the i-th round second predicted action is generated by inputting the sample action representation into the action generation model trained in the i-1th round.

4. The method according to any one of claims 1 to 3, further comprising: Acquire at least one training sample, each of the training samples comprising an action and a set of action representations corresponding to the action, wherein the set of action representations comprises at least two action representations; and Based on the at least one training sample, an initial model is pre-trained to obtain a pre-trained action generation model, where the initial model is a neural network model that generates the action based on the set of action representations.

5. The method according to claim 4, wherein the at least two action representations correspond to respective representation types; The pre-training of the initial model based on the at least one training sample to obtain a pre-trained action generation model includes: Based on a set of action representations corresponding to each training sample, construct at least two parallel representation spaces, each of the representation spaces including action representations belonging to the same representation type, each of the at least two representation spaces corresponding to a different representation type, and the at least two parallel representation spaces being used to instruct the initial model to learn different representation types respectively; For each of the at least one training sample, using the action representation of the training sample in the at least two representation spaces as a constraint condition in the initial model generation process to generate a first sample predicted action; Training the initial model based on the loss between the first sample predicted action and the action; and A pre-trained action generation model is obtained based on the initial model.

6. The method according to claim 5, wherein the training of the initial model based on the loss between the first sample predicted action and the action comprises: For each of the actions in the at least one training sample, generating a second sample predicted action without using constraints in the process of generating the initial model; and The initial model is trained based on a first loss and a second loss, wherein the first loss is the loss between the first sample predicted action and the action, and the second loss is the loss between the second sample predicted action and the action.

7. The method according to any one of claims 1 to 6, wherein the sample action representation comprises a first action representation and a second action representation, the second action representation being an action representation corresponding to an identifier, the identifier being used to identify a personalized action feature of the sample action; Obtaining a sample action representation of the sample action includes: Obtaining the first action representation and the second action representation, where the first action representation is a constraint condition to be replaced in the sample action, and the second action representation is obtained by performing feature extraction on the identifier by the action generation model; The step of generating an action based on the sample action representation using the pre-trained action generation model to obtain a second predicted action includes: generating an action based on the first action representation and the second action representation using the pre-trained action generation model to obtain a second predicted action; The method further comprises: The identifier is updated based on the loss between the second predicted action and the sample action.

8. The method according to claim 7, wherein generating an action based on the first action representation and the second action representation using the pre-trained action generation model to obtain a second predicted action comprises: In the j-th iteration process, the first action representation and the j-1-th round second action representation are input into the pre-trained action generation model to obtain the j-th round second predicted action, where j is a positive integer; The updating of the identifier based on the loss between the second predicted action and the sample action includes: Based on the loss between the second predicted action in the jth round and the sample action, updating the j-1th round identifier to obtain the jth round identifier; Obtaining a second action representation of the jth round, where the second action representation of the jth round is obtained by performing feature extraction on the jth round identifier by the action generation model; and Update j to j+1, and continue the j-th iteration process until the j-th round identifier meets the iteration termination condition.

9. The method according to any one of claims 1 to 8, wherein obtaining the priori action and the priori action representation of the priori action comprises: Obtaining any prior action representation from a representation space of the pre-trained action generation model, where the representation space is a set of action representations learned by the action generation model during pre-training; and The pre-trained action generation model is used to generate actions according to the prior action representation to obtain the prior actions.

10. The method according to claim 7 or 8, further comprising: Obtaining a trained action generation model, wherein the trained action generation model is used to generate n styles of actions; Obtaining an identifier of a desired style and a user desired action, wherein the desired style is one of the n styles; and The trained action generation model is used to generate actions according to the user's desired action and the identifier, thereby obtaining an action with a desired style.

11. The method according to claim 10, wherein the action generation model completed through the training generates an action according to the user's desired action and the identifier to obtain an action with a desired style, comprising: Generate a first output action and a second output action by the trained action generation model, wherein the first output action is generated by the trained action generation model based on the user desired action and the identifier as constraints, and the second output action is generated by the trained action generation model without using constraints; and The trained action generation model generates an action with a desired style based on the first output action and the second output action.

12. A training device for an action generation model, comprising: an acquisition module, configured to acquire a priori action and a priori action representation of the priori action, wherein the priori action has diverse action features learned by the action generation model in pre-training; Acquire a sample action and a sample action representation of the sample action, wherein the sample action has personalized action features; A generation module is configured to generate an action based on the prior action representation using a pre-trained action generation model to obtain a first predicted action; and to generate an action based on the sample action representation using the pre-trained action generation model to obtain a second predicted action; and A training module is used to retrain the pre-trained action generation model based on the prior loss between the first predicted action and the prior action, and the sample loss between the second predicted action and the sample action, to obtain a trained action generation model.

13. A computer device, comprising: A processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium, wherein executable instructions are stored in the computer-readable storage medium, wherein the executable instructions are loaded and executed by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product, comprising computer instructions, wherein the computer instructions are stored in a computer-readable storage medium, and a processor reads and executes the computer instructions from the computer-readable storage medium to implement the method according to any one of claims 1 to 11.

Images