Training method of animation generation model, animation generation method and device

Abstract

The present disclosure relates to a training method, an animation generation method, and an apparatus for an animation generation model. The method comprises: generating a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set comprises sample input information and sample output information; for each training sample, inputting posture information and trajectory information corresponding to a target object in the training sample into a target model to obtain predicted sample information corresponding to the training sample output by the target model, wherein the predicted sample information comprises predicted posture information, predicted trajectory information, and predicted position information corresponding to the target object in the next frame; determining a target loss of the target model based on the sample input information, the predicted sample information, and the sample output information; and training the target model according to the target loss to obtain a trained animation generation model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a training method for an animation generation model, an animation generation method, and a device. Background Art

[0002] Character animation technology is a core component of many games. In related technologies, character animation is usually generated based on state machines or motion matching.

[0003] When generating character animation based on a state machine, specific animation clips are typically played. These clips can be edited by animators or captured using motion capture equipment. However, character animation systems using state machines are often very complex in structure. In the aforementioned solution, fixed animation clips are played, making them difficult to apply to a wide range of scenarios and achieving natural and realistic effects. When generating character animation based on Motion Matching, a large number of pre-processed animations are required, which consumes a large amount of memory and takes a long time to retrieve. Summary of the Invention

[0004] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a method for training an animation generation model of a virtual object, the method comprising:

[0006] Generate a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set includes sample input information and sample output information, the sample input information includes posture information and trajectory information corresponding to the target object in the current frame, and the sample output information includes target posture information, target trajectory information, and target position information corresponding to the target object in the next frame;

[0007] For each training sample, inputting the posture information and trajectory information corresponding to the target object in the training sample into the target model to obtain predicted sample information corresponding to the training sample output by the target model, the predicted sample information including predicted posture information, predicted trajectory information, and predicted position information corresponding to the target object in the next frame;

[0008] Determining a target loss of the target model based on the sample input information, the predicted sample information, and the sample output information;

[0009] The target model is trained according to the target loss to obtain a trained animation generation model.

[0010] In a second aspect, the present disclosure provides a method for generating an animation of a virtual object, the method comprising:

[0011] Obtain target posture information and target trajectory information corresponding to the virtual object;

[0012] Obtaining control posture information and control position information corresponding to the virtual object in the next frame based on the target posture information, the target trajectory information, and the animation generation model, wherein the animation generation model is a model trained based on the training method for the virtual object animation generation model described in the first aspect;

[0013] The virtual object is controlled to move according to the control posture information and the control position information to generate an animation corresponding to the virtual object.

[0014] In a third aspect, the present disclosure provides a training device for an animation generation model of a virtual object, the device comprising:

[0015] A first generation module is configured to generate a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set includes sample input information and sample output information, the sample input information includes posture information and trajectory information corresponding to the target object in the current frame, and the sample output information includes target posture information, target trajectory information, and target position information corresponding to the target object in the next frame;

[0016] A first processing module is configured to input, for each training sample, the posture information and trajectory information corresponding to the target object in the training sample into a target model to obtain predicted sample information corresponding to the training sample output by the target model, the predicted sample information including predicted posture information, predicted trajectory information, and predicted position information corresponding to the target object in the next frame;

[0017] a determination module, configured to determine a target loss of the target model based on the sample input information, the predicted sample information, and the sample output information;

[0018] A training module is used to train the target model according to the target loss to obtain a trained animation generation model.

[0019] In a fourth aspect, the present disclosure provides a device for generating an animation of a virtual object, the device comprising:

[0020] The second acquisition module is used to obtain target posture information and target trajectory information corresponding to the virtual object;

[0021] a second processing module, configured to obtain control posture information and control position information corresponding to the virtual object in the next frame based on the target posture information, the target trajectory information, and an animation generation model, wherein the animation generation model is a model trained based on the training method for the virtual object animation generation model described in the first aspect;

[0022] A control module is used to control the movement of the virtual object according to the control posture information and the control position information to generate an animation corresponding to the virtual object.

[0023] In a fifth aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect or the second aspect.

[0024] In a sixth aspect, the present disclosure provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect or the second aspect.

[0025] In the above technical solution, a training sample set can be automatically generated based on the pre-acquired motion capture data, and then the target model can be trained based on each sample in the training sample set to obtain an animation generation model. In addition, the animation generation model can predict the posture information and trajectory information corresponding to the next frame based on the posture information and trajectory information of the character object to generate the character animation. Therefore, the above technical solution does not require the playback of specific animation clips, which can greatly improve the diversity and smoothness of the animation generated based on the animation generation model. At the same time, the motion capture data is used as training data to predict the posture information and trajectory information of the next frame for animation generation. This not only eliminates the need to process the animation data and reduces manual workload, but also avoids the time and memory space occupied by searching for animations, thereby providing technical support for improving the efficiency of animation generation and broadening the application scenarios of animation generation.

[0026] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the drawings:

[0028] Figure 1 This is a flowchart of a training method for an animation generation model of a virtual object provided in one embodiment of the present disclosure;

[0029] Figure 2 is a schematic diagram of joint points of an object provided according to an embodiment of the present disclosure;

[0030] Figure 3 is a schematic diagram of trajectory information provided according to an embodiment of the present disclosure;

[0031] Figure 4 is a schematic diagram of a target model provided according to an embodiment of the present disclosure;

[0032] Figure 5 This is a schematic diagram of the encoding sub-model and the decoding sub-model during the training process provided in one embodiment of the present disclosure;

[0033] Figure 6 is a flowchart of a method for generating animation of a virtual object provided in one embodiment of the present disclosure;

[0034] Figure 7 and Figure 8 This is a schematic diagram of a scene interface provided according to an embodiment of the present disclosure;

[0035] Figure 9 is a block diagram of a training device for an animation generation model of a virtual object provided in accordance with an embodiment of the present disclosure;

[0036] Figure 10 is a block diagram of a device for generating animation of a virtual object provided in accordance with an embodiment of the present disclosure;

[0037] Figure 11 A schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0038] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0039] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0040] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0041] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0042] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0043] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0044] All actions of acquiring signals, information or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0045] Figure 1 As shown in FIG. 1 , a flow chart of a method for training an animation generation model of a virtual object provided in an embodiment of the present disclosure is shown in FIG. Figure 1 As shown, the method may include:

[0046] In step 11, a training sample set is generated based on the pre-acquired motion capture data, wherein each training sample in the training sample set includes sample input information and sample output information, the sample input information includes posture information and trajectory information corresponding to the target object in the current frame, and the sample output information includes target posture information, target trajectory information and target position information corresponding to the target object in the next frame.

[0047] The test subject can exercise while wearing a motion capture device, thereby obtaining motion capture data of the human body at various positions during the motion capture process, and generating a training sample set based on the motion capture data. The target object is the object for character animation generation. The posture information corresponding to the target object is used to represent the posture of the target object at its position. The trajectory information corresponding to the target object is used to represent the line formed by the position information of the target object during its motion process. The target position information is used to represent the position of the target object in the next frame.

[0048] In step 12, for each training sample, the posture information and trajectory information corresponding to the target object in the training sample are input into the target model to obtain the predicted sample information corresponding to the training sample output by the target model, and the predicted sample information includes the predicted posture information, predicted trajectory information and predicted position information corresponding to the target object in the next frame.

[0049] For example, the target model can extract and perform predictive calculations based on sample input information, thereby obtaining the predicted posture information, predicted trajectory information and predicted position information corresponding to the target object in the next frame based on the target model, so that the animation of the target object can be generated based on the predicted posture information of the next frame.

[0050] In step 13, the target loss of the target model is determined based on the sample input information, the predicted sample information and the sample output information.

[0051] In step 14, the target model is trained according to the target loss to obtain a trained animation generation model.

[0052] As an example, when the target loss is greater than a loss threshold, the parameters of the target model can be updated according to the target loss, until the determined target loss is less than or equal to the loss threshold, the training is terminated, and the trained target model is used as the animation generation model. As another example, when the number of training times is less than a number threshold, the parameters of the target model can be updated according to the target loss, until the number of training times reaches the number threshold, the training is terminated, and the trained target model is used as the animation generation model. Among them, updating the parameters of the target model according to the target loss can be performed based on the target loss by gradient descent method to perform parameter update so that the model converges. This method can use any update method in the art to update the parameters, and the present disclosure is not limited to this.

[0053] Therefore, in the above technical solution, a training sample set can be automatically generated based on the pre-acquired motion capture data, and then the target model can be trained based on each sample in the training sample set to obtain an animation generation model. And the animation generation model can predict the posture information and trajectory information corresponding to the next frame based on the posture information and trajectory information of the character object to generate the character animation. Therefore, through the above technical solution, there is no need to play a specific animation clip, which can greatly improve the diversity and smoothness of the animation generated based on the animation generation model. At the same time, the motion capture data is used as training data to predict the posture information and trajectory information of the next frame for animation generation. There is no need to process the animation data, reducing manual workload, and avoiding the time and memory space occupied by searching for animations, thereby improving technical support for improving animation generation efficiency and broadening the application scenarios of animation generation.

[0054] In order to enable those skilled in the art to better understand the method provided by the present disclosure, the above steps are described in detail with examples below.

[0055] In a possible embodiment, the target model includes an encoding sub-model, a prediction sub-model and a decoding sub-model;

[0056] The step of inputting the posture information and trajectory information corresponding to the target object in the training sample into the target model to obtain predicted sample information corresponding to the training sample output by the target model includes:

[0057] The posture information and trajectory information corresponding to the target object are encoded based on the encoding sub-model; prediction is performed based on the prediction sub-model and the encoding vector to generate a prediction vector; and the prediction vector is decoded based on the decoding sub-model to obtain the prediction sample information.

[0058] In this embodiment, the target model is composed of an encoding sub-model, a prediction sub-model and a decoding sub-model, so that corresponding operations can be performed based on each sub-model to obtain the intermediate features output by each sub-model and the final features output by the model, so that when the target model is trained, the operation of the sub-model can be evaluated based on the features output by each sub-model, which facilitates parameter adjustment for each sub-model, and improves the accuracy of the target model by improving the accuracy of each sub-model.

[0059] In a possible embodiment, the posture information corresponding to the target object includes the posture information corresponding to each joint point of the target object; the posture information corresponding to each joint point includes the position coordinates, rotation and speed of the joint point relative to the root node of the target object, and the trajectory information is formed by the movement position information of each root node in multiple consecutive frames including the current frame, and the movement position information corresponding to each frame includes the ground position coordinates, direction, speed and movement type of the root node in the frame relative to the root node in the current frame.

[0060] For example, the pose of the target object can be determined by multiple joint points, which are the connection points between bones, such as Figure 2As shown, the connecting line is used to represent the skeleton, and the points at both ends of the connecting line represent the joint points, as shown in A1 and A2. The number of joint points can be set according to the actual application scenario. For example, it can be set to 22 joint points, that is, the posture of the target object is determined by the state of the 22 joint points. The trajectory information of the target object can be composed of the position information of the root node of the target object. The root node of the target object is a point in the predetermined target object. In the world space coordinate system, the projection of the root node of the target object on its walking plane can be determined as the position information of the root node of the target object. Then, the various position information during the movement of the target object forms the trajectory information.

[0061] For example, the space where the target object is located can be used as the world space coordinate system, and the character body coordinate system corresponding to the target object can be constructed with the root node of the target object as the origin. As an example, when determining the posture information of the joint points, the corresponding coordinates of each joint point in the character body coordinate system can be determined, that is, the coordinates relative to the root node. For example, the posture information can include position coordinates, rotation and speed, expressed as follows:

[0062]

[0063] Among them, P i That is, it represents the posture information corresponding to the i-th frame. Used to represent the position coordinates of joint point j relative to the character's body coordinate system, Used to represent the rotation of joint point j relative to the character's body coordinate system, Used to represent the velocity of joint point j relative to the character's body coordinate system,

[0064] For example, Figure 3 As shown, the moving position information corresponding to each trajectory point constitutes the trajectory information corresponding to the target object. Figure 3 In the image, the target object walks from right to left, i.e. in the order of T1, T2, T3, T4, and T5. T4 is the projection of the root node of the target object in the current frame. The trajectory information T corresponding to the i-th frame is i It can be expressed as follows:

[0065]

[0066] in, It is used to indicate the moving position information corresponding to the mth frame before the i-th frame, where: It is used to represent the ground position coordinates of the root node in the mth frame before the i-th frame relative to the root node of the i-th frame. That is, the target object moves on the walking plane. Therefore, it is only necessary to determine the X and Z coordinates of the root node in the world space coordinate system, that is, the ground position coordinates. Used to indicate the direction of the root node in the mth frame before the i-th frame relative to the root node of the i-th frame; Used to represent the velocity of the root node in the mth frame before the i-th frame relative to the root node of the i-th frame; Used to indicate the movement type of the root node in the mth frame before the i-th frame relative to the root node of the i-th frame, where the movement type can include stillness, walking, running, etc. The specific classification can be set based on the actual application scenario, and the movement type corresponding to each movement position information can be labeled by the user. Used to indicate the moving position information corresponding to the nth frame after the i-th frame. It is used to represent the mobile position information corresponding to the i-th frame. The specific meaning can be determined based on the analogy described above and will not be repeated here. direction speed Movement Type k is the number of movement types.

[0067] Therefore, by analyzing and extracting the motion capture data, posture information and trajectory information can be obtained, thereby making a comprehensive and comprehensive representation of the various states of the target object during the movement process, providing accurate data support for the subsequent prediction of the posture information of the next frame based on the posture information and trajectory information, and improving the effectiveness and comprehensiveness of the training data of the animation generation model, so as to improve the prediction accuracy of the animation generation model, and thus ensure the smoothness of the generated animation to a certain extent.

[0068] In a possible embodiment, the posture information corresponding to the target object includes the posture information corresponding to each joint point of the target object, and the target position information corresponding to the target object is the position information of the root node of the target object; the posture information corresponding to each joint point includes current posture information and historical posture information, and the trajectory information corresponding to the target object includes historical trajectory information and future trajectory information. The encoding vector corresponding to the sample input information includes implicit features, key posture features and historical key posture features corresponding to the key joint points of the target object, and future trajectory features corresponding to the target object. The implicit features are used to store features corresponding to the posture and trajectory extracted based on the sample input information.

[0069] For example, the current posture information corresponding to the joint point is the posture information of the joint point determined based on the current frame, which is recorded as P iThe historical posture information corresponding to the joint point can be the posture information of the joint point determined based on the image frame collected before the current frame, or can be the posture information determined by the continuous u frames before the current frame, which is expressed as follows:

[0070]

[0071] Among them, P i H Indicates the historical posture information corresponding to the i-th frame. The number u can be set based on the actual application scenario. For example, u=1, that is, the posture information of the previous frame of the current frame is used as the historical posture information.

[0072] As above and Figure 3 As shown, the moving position information corresponding to the m frames before the current frame can be used as the historical trajectory information, and the moving position information corresponding to the current frame and the n frames after the current frame can be used as the future trajectory information, which are expressed as follows:

[0073] Historical trajectory information corresponding to the i-th frame

[0074] The future trajectory information corresponding to the i-th frame

[0075] The values ​​of m and n can be set according to the actual application scenario and are not limited in this disclosure. For example, m = 6 and n = 5 means that the movement position information in the 6 frames before the current frame is used as the historical trajectory information, and the movement position information in the current frame and the 5 frames after the current frame is used as the future trajectory information.

[0076] For example, the encoding sub-model can be a multi-layer perceptron MLP (Multilayer Perceptron), which can be set to 6 layers, wherein the dimension of the hidden layer can be set to 256, and the activation function can be selected as ReLU. After the sample input information is input into the encoding sub-model, the sample input information can be extracted and encoded based on the encoding sub-model to obtain the implicit feature Z obtained by encoding the entire sample input information. i, the implicit feature can be represented by a 32-dimensional vector. At the same time, the key information in the sample input information can be extracted. For example, the key information can include the current posture features and historical posture features corresponding to the key joints and the future trajectory features of the target object. The key joints are at least one of the multiple joints of the target object, which can be pre-set. For example, the joints corresponding to the head, left hand, right hand, left knee, right knee, left foot, and right foot can be pre-set as key joints. Then, the key posture features can be obtained by performing physical sign extraction from the posture information of the current frame. The historical key posture features corresponding to the key joints represent the key posture features corresponding to multiple historical frames respectively. The determination method is the same as the determination method of the key posture features of the current frame, which will not be repeated here. For example, the future trajectory features can be a vector feature representation from the future trajectory information.

[0077] Then, the encoding sub-model satisfies: E θ (P i ,P i H ,T i H ,T i F )=P i K ,P i HK ,T i F ,Z i

[0078] Among them, P i K It is used to represent the key posture features corresponding to the key joint points in the i-th frame, P i HK Represents the historical key posture features corresponding to the key joint points in the i-th frame.

[0079] Therefore, through the above technical solution, when predicting the posture of the target object in the next frame, the prediction can be made based on the posture information of the current frame and the posture information of the historical frames, and the historical trajectory information and future trajectory information of the target object can be combined for prediction, thereby further improving the comprehensiveness of the features of the input data of the animation generation model. At the same time, in the process of encoding the encoding sub-model, while encoding the overall features of the sample input information, the features of the key joint points can be extracted, so that the obtained encoding vector can not only contain the overall features corresponding to the sample input information, but also display the key information in the sample input information, thereby improving the effectiveness of the encoding vector obtained by the encoding sub-model, and providing accurate and comprehensive data support for the subsequent prediction of the next frame posture based on the encoding vector.

[0080] For the prediction sub-model, the prediction sub-model S φA multi-layer MLP can be used, for example, with 4 layers, a hidden layer dimension of 256, and a ReLU activation function. For example, the input of the prediction sub-model is the output of the encoding sub-model, i.e., the encoding vector, and the prediction feature is output based on the encoding vector. For example, the prediction feature can be expressed as follows:

[0081]

[0082] That is, based on the encoding vector, the key posture features of the key joint points corresponding to the next frame can be predicted Historical key pose features of the key joint points corresponding to the next frame Future trajectory features corresponding to the next frame The hidden feature Z corresponding to the next frame i+1 And the position information R of the root node of the target object in the next frame i+1 , where the position information is the position information of the root node of the target object in the next frame i+1 relative to the root node of the current frame i, in, and They represent the X and Z coordinates of the target object in the next frame relative to the root node of the current frame. The X and Z coordinates are the coordinates in the world space coordinate system. Indicates the direction of the target object in the next frame relative to the root node of the current frame.

[0083] In a possible embodiment, the method may further include:

[0084] Acquire environmental information corresponding to the target object, wherein the environmental information may be external information related to the animation, such as terrain, postures of other characters, obstacles, etc., which may be extracted through feature extraction of the captured image frames.

[0085] Accordingly, inputting the posture information and trajectory information corresponding to the target object in the training sample into the target model to obtain the predicted sample information corresponding to the training sample output by the target model includes:

[0086] The posture information, trajectory information and environmental information corresponding to the target object in the training sample are input into the target model to obtain the predicted sample information output by the target model. For example, the prediction vector can be generated according to the environmental information and the encoding vector based on the prediction sub-model in the target model.

[0087] In this embodiment, when predicting the pose information of the target object in the next frame, the environment information corresponding to the target object can be simultaneously combined. For example, the environment information and the encoding vector determined based on the pose information and trajectory information corresponding to the target object in the training sample can be used as inputs to the prediction sub-model, and the prediction sub-model satisfies:

[0088]

[0089] Among them, I i Used to indicate the environment information corresponding to the i-th frame.

[0090] Therefore, through the above technical solution, the environmental information of the target object can be further obtained, so that when predicting the posture information of the target object in the next frame, the predicted posture information can be consistent with the movement process and environmental state of the target object, thereby improving the accuracy and precision of the animation generation model. At the same time, it can be applied to application scenarios where other objects and background environments exist, broadening the application scenarios of the animation generation model.

[0091] For the decoding sub-model, the decoding sub-model D ω It can be a multi-layer MLP, for example, it can be set to 4 layers, with a hidden layer dimension of 256 and an activation function of ReLU. The data structure of the input of the decoding sub-model is consistent with the data structure of the output of the encoding sub-model. The decoding sub-model can restore the posture based on the predicted features to obtain the posture information corresponding to the target object in the next frame, which is expressed as follows:

[0092]

[0093] Among them, P i+1 It is used to represent the posture information of the target object in the next frame, which may include the posture information of each joint point of the target object in the next frame.

[0094] Therefore, through the above technical solution, the corresponding prediction sample information can be obtained based on the posture information, trajectory information corresponding to the target object in the training sample and the encoding sub-model, prediction sub-model and decoding sub-model in the target model. The prediction sample information Y i That is,

[0095] The following is a detailed description of the target model training process.

[0096] In a possible embodiment, an exemplary implementation of determining the target loss of the target model based on the sample input information, the predicted sample information, and the sample output information in step 13 is as follows. This step may include:

[0097] Based on the sample input information, the encoding vector and the encoding sub-model, a first loss of the encoding sub-model is determined.

[0098] The encoding sub-model is used to encode sample input information for feature extraction and encoding. Therefore, the encoding vector output by the encoding sub-model after encoding must ensure consistency with the original sample input information. Based on this, in this embodiment, the first loss of the encoding sub-model can be determined based on the sample input information and the encoding vector.

[0099] As an example, the encoding vector may include key posture features and historical key posture features corresponding to key joints of the target object, as well as future trajectory features of the target object. At the same time, the current posture information and historical posture information of the key joints can be extracted from the posture information and trajectory information in the sample input information, directly obtaining the key posture features and historical key posture features corresponding to the key joints, and similarly determining the future trajectory information of the target object. Thus, the distance between the above-mentioned features in the encoding vector and the features determined from the sample input information can be calculated to determine the first loss.

[0100] As another example, an exemplary implementation of determining the first loss of the encoding sub-model based on the sample input information, the encoding vector, and the encoding sub-model is as follows. This step may include:

[0101] Noise is added to the sample input information to obtain noise sample information. The noise can be Gaussian noise N(0,α), where α is a hyperparameter ranging from (0,1). The noise sample information can be expressed as X i +N(0,α), where X i ={P i ,P i H ,T i H ,T i F} is used to represent sample input information.

[0102] The noise sample information is input into the coding sub-model to obtain a noise sample vector encoded by the coding sub-model. Similar to the above, the noise sample information can be input into the coding sub-model so that the coding sub-model outputs a coding vector:

[0103] E θ (X i +N(0,α))=P i K ',P i HK ',T iF ',Z i ', where the feature extraction and encoding process of the encoding sub-model has been described in detail above and will not be repeated here.

[0104] The first loss is determined according to the noise sample vector and the sample input information.

[0105] Similarly, the posture information and trajectory information corresponding to the features of the encoding vector can be determined from the sample input information to determine the first loss L by the following formula: E :

[0106]

[0107] Among them, P i K ' is used to represent the key posture features determined based on the noise sample information corresponding to the i-th frame, It is used to represent the key posture information determined from the sample input information corresponding to the i-th frame, P i HK ' is used to represent the historical key posture features determined based on the noise sample information corresponding to the i-th frame, It is used to represent the historical key posture information determined from the sample input information corresponding to the i-th frame, T i F ' is used to represent the future trajectory features determined based on the noise sample information corresponding to the i-th frame, It is used to represent the future trajectory information determined from the sample input information corresponding to the i-th frame.

[0108] Therefore, the loss of the coding sub-model can be determined through the above scheme, and by requiring the coding vector output by the coding sub-model to be mapped to the data of the original sample input information after adding noise to the sample input information, the coding accuracy and robustness of the coding sub-model can be further improved.

[0109] Based on the sample input information, the encoding vector, the encoding sub-model and the decoding sub-model, a second loss of the decoding sub-model is determined.

[0110] like Figure 4 As shown in FIG, a schematic diagram of a target model provided based on an embodiment of the present disclosure is provided. In an embodiment, the input of the decoding sub-model corresponds to the output of the encoding sub-model. Therefore, during the training of the target model, the sample input information is input into the encoding sub-model for encoding to obtain the encoding vector, and the output of the encoding sub-model is directly used as the input of the decoding sub-model, as shown in FIG. Figure 5As shown, the decoding sub-model performs posture restoration based on the encoding vector to obtain the current posture information, so that the current posture information obtained by decoding the decoding sub-model can be calculated with the current posture information in the sample input information to obtain the second loss.

[0111] As another example, an exemplary implementation of determining the second loss of the decoding sub-model based on the sample input information, the encoding vector, the encoding sub-model, and the decoding sub-model is as follows. This step may include:

[0112] Noise is added to the sample input information to obtain noise sample information; the noise sample information is input into the encoding sub-model to obtain a noise sample vector encoded by the encoding sub-model. The implementation of this step has been described in detail above and will not be repeated here.

[0113] The noise sample vector is then input into the decoding sub-model to obtain decoded sample information obtained by the decoding sub-model. The decoded sample information is pose information obtained by performing pose restoration based on the noise sample vector containing noise. The second loss is then determined based on the decoded sample information and the sample input information.

[0114] For example, the second loss L is determined by the following formula: S :

[0115] L S =(D ω (E θ (X i +N(0,α)))-P i ) 2 ;

[0116] Among them, D ω ω represents the decoding process of the decoding sub-model, and ω represents the parameters of the decoding sub-model. Thus, the above scheme can determine the loss of the decoding sub-model, and by requiring the pose information output by the decoding sub-model to be consistent with the pose information in the sample input information after adding noise to the sample input information, the accuracy and robustness of the decoding sub-model's decoding pose restoration can be further improved.

[0117] Based on the predicted sample information and the sample output information, the third loss of the target model is determined. For example, the distance between the two can be calculated to determine the third loss, and the sample input information X i ={P i ,P i H ,T i H ,T i F} Input the encoding sub-model to encode and obtain the encoding vector, which is then predicted by the prediction sub-model and decoded by the decoding sub-model to obtain the predicted sample information, the predicted sample information Y i With sample output information Y i gt One-to-one correspondence is expressed as follows:

[0118]

[0119]

[0120] in, Used to indicate the target posture information corresponding to the joint point in the next frame. Used to indicate the target trajectory information corresponding to the next frame, Used to indicate the location information of the root node corresponding to the next frame target.

[0121] The third loss L can be calculated by the following formula P :

[0122]

[0123] The target loss is determined based on the first loss, the second loss, and the third loss.

[0124] For example, the target loss can be determined by taking the weighted sum of the first loss, the second loss, and the third loss, and then the parameters of the target model can be updated based on the target loss. Thus, through the above technical solution, losses can be calculated for each sub-model of the target model based on its characteristics, thereby improving the accuracy of the sub-models and the overall accuracy of the target model, improving the training efficiency and prediction accuracy of the animation generation model, and enhancing the user experience.

[0125] The present disclosure also provides a method for generating animation of a virtual object, such as Figure 6 As shown, the method includes:

[0126] In step 61, target pose information and target trajectory information corresponding to the virtual object are obtained. For example, the target pose information and target trajectory information can be obtained by collecting the states of the root node and each joint point of the virtual object. The data dimensions of the target pose information and target trajectory information are the same as the dimensions of the input data of the encoding sub-model described above, and are not further described here.

[0127] In step 62, the control posture information and control position information corresponding to the virtual object in the next frame are obtained based on the target posture information, the target trajectory information, and the animation generation model, wherein the animation generation model is a model trained based on any of the training methods for the virtual object animation generation model described above. The control posture information corresponding to the virtual object in the next frame can be the posture information corresponding to each joint of the virtual object in the next frame, and the control position information corresponding to the virtual object in the next frame can be the position information corresponding to the root node of the virtual object in the next frame. The target posture information and the target trajectory information can be input into the animation generation model to obtain the output of the model. As described above, the future trajectory information is initially null. During the movement of the virtual object, the future trajectory features corresponding to the next frame determined by the current prediction animation generation model can be used as the input future trajectory features in the animation generation model during the next prediction, thereby achieving continuous loop prediction of the animation of the virtual object.

[0128] In step 63 , the virtual object is controlled to move according to the control posture information and the control position information to generate an animation corresponding to the virtual object.

[0129] Among them, the control position information corresponding to the virtual object in the next frame can be the position information corresponding to the root node of the virtual object in the next frame, that is, the ground position coordinates and direction of the root node of the next frame relative to the current frame. The control posture information can be the position coordinates, rotation and speed of each joint point of the virtual object relative to the root node. Correspondingly, controlling the movement of the virtual object can be controlling the movement of the joint points and root node of the virtual object. For example, the root node of the virtual object can be assigned based on the control position information, and each joint point in the virtual object can be assigned based on the control posture information to form a new animation.

[0130] Therefore, through the above technical solution, the posture information and trajectory information of the virtual object can be collected, and the next frame animation corresponding to the virtual object can be determined based on the animation generation model. Compared with the state machine or animation query based on the existing technology, the prediction through the animation generation model can improve the smoothness of the generated animation while improving the animation generation efficiency and enhancing the user experience.

[0131] In a possible embodiment, the method may further include:

[0132] Receive a control instruction for the virtual object, wherein the control instruction can be triggered by the user through a control button in the interface, such as Figure 7 As shown, the user can operate the control button B to trigger the control instruction.

[0133] New target trajectory information is generated based on the control instruction and the target trajectory information corresponding to the virtual object. The target trajectory information may include historical trajectory information and future trajectory information, where the future trajectory information is the predicted trajectory output by the animation generation model. Accordingly, upon receiving a control instruction, the operation indicated by the control instruction will modify the future trajectory information. Therefore, in this embodiment, the future trajectory information can be modified based on the operation indicated by the control instruction to generate new future trajectory information. The new target trajectory information then includes both the historical trajectory information and the newly generated future trajectory information.

[0134] For example, Figure 7 The trajectory shown is the future trajectory information T predicted by the previous frame. When the control instruction is to shift to the right, the future trajectory information can be shifted to the right based on the direction corresponding to the control instruction, such as Figure 8 The future trajectory information shown by T' in FIG. 1 , wherein the offset distance can be determined according to the specific speed in the future trajectory information.

[0135] Accordingly, an exemplary implementation of obtaining the control posture information and control position information corresponding to the next frame of the virtual object based on the target posture information, the target trajectory information, and the animation generation model is as follows. This step may include:

[0136] The control posture information and the control position information are obtained according to the target posture information, the new target trajectory information and the animation generation model.

[0137] Among them, the target posture information and new target trajectory information can be input into the animation generation model to generate animations that conform to the motion state of the virtual object and the control instructions triggered by the user, further meeting the user's usage needs, broadening the application scenarios of the animation generation method, and improving the user experience.

[0138] The present disclosure also provides a training device for generating an animation model of a virtual object, such as Figure 9 As shown, the device 10 includes:

[0139] A first generating module 101 is configured to generate a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set includes sample input information and sample output information, wherein the sample input information includes posture information and trajectory information corresponding to the target object in the current frame, and the sample output information includes target posture information, target trajectory information, and target position information corresponding to the target object in the next frame;

[0140] A first processing module 102 is configured to input, for each training sample, the posture information and trajectory information corresponding to the target object in the training sample into a target model to obtain predicted sample information corresponding to the training sample output by the target model, the predicted sample information including predicted posture information, predicted trajectory information, and predicted position information corresponding to the target object in the next frame;

[0141] A determination module 103 is configured to determine a target loss of the target model based on the sample input information, the predicted sample information, and the sample output information;

[0142] The training module 104 is used to train the target model according to the target loss to obtain a trained animation generation model.

[0143] Optionally, the target model includes an encoding sub-model, a prediction sub-model and a decoding sub-model;

[0144] The first processing module includes:

[0145] A first processing submodule is configured to encode the posture information and trajectory information corresponding to the target object based on the encoding submodel to obtain an encoding vector corresponding to the sample input information;

[0146] A second processing submodule, configured to perform prediction based on the prediction submodel and the encoding vector to generate a prediction vector;

[0147] The third processing submodule is configured to decode the prediction vector based on the decoding submodel to obtain the prediction sample information.

[0148] Optionally, the determining module includes:

[0149] A first determining submodule, configured to determine a first loss of the encoding submodel based on the sample input information, the encoding vector, and the encoding submodel;

[0150] A second determination submodule, configured to determine a second loss of the decoding submodel based on the sample input information, the encoding vector, the encoding submodel, and the decoding submodel;

[0151] A third determination submodule, configured to determine a third loss of the target model based on the predicted sample information and the sample output information;

[0152] A fourth determining submodule is configured to determine the target loss based on the first loss, the second loss, and the third loss.

[0153] Optionally, the first determining submodule includes:

[0154] a fourth processing submodule, configured to add noise to the sample input information to obtain noise sample information;

[0155] A first encoding submodule, configured to input the noise sample information into the encoding submodel to obtain a noise sample vector encoded by the encoding submodel;

[0156] A fifth determining submodule is configured to determine the first loss according to the noise sample vector and the sample input information.

[0157] Optionally, the second determining submodule includes:

[0158] a fifth processing submodule, configured to add noise to the sample input information to obtain noise sample information;

[0159] A second encoding submodule, configured to input the noise sample information into the encoding submodel to obtain a noise sample vector encoded by the encoding submodel;

[0160] A decoding submodule, configured to input the noise sample vector into the decoding submodel to obtain decoding sample information obtained by decoding the decoding submodel;

[0161] A sixth determining submodule is configured to determine the second loss according to the decoded sample information and the sample input information.

[0162] Optionally, the posture information corresponding to the target object includes the posture information corresponding to each joint point of the target object, and the target position information corresponding to the target object is the position information of the root node of the target object; the posture information corresponding to each joint point includes current posture information and historical posture information, and the trajectory information corresponding to the target object includes historical trajectory information and future trajectory information. The encoding vector corresponding to the sample input information includes implicit features, key posture features and historical key posture features corresponding to the key joint points of the target object, and future trajectory features corresponding to the target object. The implicit features are used to store features corresponding to the posture and trajectory extracted based on the sample input information.

[0163] Optionally, the device further comprises:

[0164] The first acquisition module is used to obtain the environment information corresponding to the target object.

[0165] The first processing module is used for:

[0166] The posture information, trajectory information and the environmental information corresponding to the target object in the training sample are input into the target model to obtain the predicted sample information output by the target model.

[0167] Optionally, the posture information corresponding to the target object includes the posture information corresponding to each joint point of the target object; the posture information corresponding to each joint point includes the position coordinates, rotation and speed of the joint point relative to the root node of the target object, and the trajectory information is formed by the movement position information of each root node in multiple consecutive frames including the current frame, and the movement position information corresponding to each frame includes the ground position coordinates, direction, speed and movement type of the root node in the frame relative to the root node in the current frame.

[0168] The present disclosure also provides a device for generating animation of a virtual object, such as Figure 10 As shown, the device 20 may include:

[0169] The second acquisition module 201 is used to obtain target posture information and target trajectory information corresponding to the virtual object;

[0170] A second processing module 202 is configured to obtain control posture information and control position information corresponding to the virtual object in the next frame based on the target posture information, the target trajectory information, and an animation generation model, wherein the animation generation model is a model trained based on the training method for the virtual object animation generation model described above;

[0171] The control module 203 is configured to control the movement of the virtual object according to the control posture information and the control position information, so as to generate an animation corresponding to the virtual object.

[0172] Optionally, the device further comprises:

[0173] A receiving module, configured to receive a control instruction for the virtual object;

[0174] A second generating module, configured to generate new target trajectory information according to the control instruction and the target trajectory information corresponding to the virtual object;

[0175] The second processing module is used for:

[0176] The control posture information and the control position information are obtained according to the target posture information, the new target trajectory information and the animation generation model.

[0177] Reference below Figure 11 , which shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 11 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0178] like Figure 11 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0179] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 11 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0180] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0181] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0182] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0183] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0184] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: generates a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set includes sample input information and sample output information, the sample input information includes posture information and trajectory information corresponding to the target object in the current frame, and the sample output information includes target posture information, target trajectory information and target position information corresponding to the target object in the next frame; for each of the training samples, the posture information and trajectory information corresponding to the target object in the training sample are input into the target model to obtain predicted sample information corresponding to the training sample output by the target model, the predicted sample information includes predicted posture information, predicted trajectory information and predicted position information corresponding to the target object in the next frame; determines the target loss of the target model based on the sample input information, the predicted sample information and the sample output information; trains the target model according to the target loss to obtain a trained animation generation model.

[0185] Alternatively, the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: obtains target posture information and target trajectory information corresponding to the virtual object; obtains control posture information and control position information corresponding to the virtual object in the next frame based on the target posture information, the target trajectory information and the animation generation model, wherein the animation generation model is a model trained by the training method of the animation generation model of the virtual object described in the first aspect; controls the movement of the virtual object according to the control posture information and the control position information to generate the animation corresponding to the virtual object.

[0186] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0187] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0188] The modules described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the name of a module does not necessarily limit the module itself. For example, the first generation module may also be described as a "module for generating a training sample set based on pre-acquired motion capture data."

[0189] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0190] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0191] According to one or more embodiments of the present disclosure, Example 1 provides a training method for an animation generation model of a virtual object, wherein the method includes: generating a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set includes sample input information and sample output information, the sample input information includes posture information and trajectory information corresponding to the target object in the current frame, and the sample output information includes target posture information, target trajectory information and target position information corresponding to the target object in the next frame; for each of the training samples, the posture information and trajectory information corresponding to the target object in the training sample are input into a target model to obtain predicted sample information corresponding to the training sample output by the target model, the predicted sample information includes predicted posture information, predicted trajectory information and predicted position information corresponding to the target object in the next frame; determining the target loss of the target model based on the sample input information, the predicted sample information and the sample output information; training the target model according to the target loss to obtain a trained animation generation model.

[0192] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the target model includes a coding sub-model, a prediction sub-model, and a decoding sub-model; inputting the posture information and trajectory information corresponding to the target object in the training sample into the target model to obtain the predicted sample information corresponding to the training sample output by the target model includes: encoding the posture information and trajectory information corresponding to the target object based on the coding sub-model to obtain the coding vector corresponding to the sample input information; performing prediction based on the prediction sub-model and the coding vector to generate a prediction vector; decoding the prediction vector based on the decoding sub-model to obtain the predicted sample information.

[0193] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein the determining the target loss of the target model based on the sample input information, the predicted sample information, and the sample output information includes: determining a first loss of the encoding sub-model based on the sample input information, the encoding vector, and the encoding sub-model; determining a second loss of the decoding sub-model based on the sample input information, the encoding vector, the encoding sub-model, and the decoding sub-model;

[0194] Determining a third loss of the target model based on the predicted sample information and the sample output information;

[0195] The target loss is determined based on the first loss, the second loss, and the third loss.

[0196] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 3, wherein determining the first loss of the encoding sub-model based on the sample input information, the encoding vector and the encoding sub-model includes: adding noise to the sample input information to obtain noise sample information; inputting the noise sample information into the encoding sub-model to obtain a noise sample vector encoded by the encoding sub-model; and determining the first loss based on the noise sample vector and the sample input information.

[0197] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 3, wherein the determining the second loss of the decoding sub-model based on the sample input information, the encoding vector, the encoding sub-model and the decoding sub-model includes: adding noise to the sample input information to obtain noise sample information; inputting the noise sample information into the encoding sub-model to obtain a noise sample vector encoded by the encoding sub-model; inputting the noise sample vector into the decoding sub-model to obtain decoded sample information decoded by the decoding sub-model; and determining the second loss based on the decoded sample information and the sample input information.

[0198] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 2, wherein the posture information corresponding to the target object includes the posture information corresponding to each joint point of the target object, and the target position information corresponding to the target object is the position information of the root node of the target object; the posture information corresponding to each joint point includes current posture information and historical posture information, the trajectory information corresponding to the target object includes historical trajectory information and future trajectory information, the encoding vector corresponding to the sample input information includes implicit features, key posture features and historical key posture features corresponding to the key joint points of the target object, and future trajectory features corresponding to the target object, and the implicit features are used to store features corresponding to the posture and trajectory extracted based on the sample input information.

[0199] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 1, wherein the method further includes: obtaining environmental information corresponding to the target object, inputting the posture information and trajectory information corresponding to the target object in the training sample into the target model to obtain the predicted sample information corresponding to the training sample output by the target model, including: inputting the posture information, trajectory information and the environmental information corresponding to the target object in the training sample into the target model to obtain the predicted sample information output by the target model.

[0200] According to one or more embodiments of the present disclosure, Example 8 provides the method of Example 1, wherein the posture information corresponding to the target object includes the posture information corresponding to each joint point of the target object; the posture information corresponding to each joint point includes the position coordinates, rotation and speed of the joint point relative to the root node of the target object, and the trajectory information is formed by the movement position information of each root node in multiple consecutive frames including the current frame, and the movement position information corresponding to each frame includes the ground position coordinates, direction, speed and movement type of the root node in the frame relative to the root node in the current frame.

[0201] According to one or more embodiments of the present disclosure, Example 9 provides a method for generating an animation of a virtual object, wherein the method includes: obtaining target posture information and target trajectory information corresponding to the virtual object; obtaining control posture information and control position information corresponding to the virtual object in the next frame based on the target posture information, the target trajectory information and the animation generation model, wherein the animation generation model is a model trained by a training method for a virtual object animation generation model based on any one of Examples 1-8; controlling the movement of the virtual object according to the control posture information and the control position information to generate an animation corresponding to the virtual object.

[0202] According to one or more embodiments of the present disclosure, Example 10 provides the method of Example 9, wherein the method further includes: receiving a control instruction for the virtual object; generating new target trajectory information according to the control instruction and the target trajectory information corresponding to the virtual object; obtaining the control posture information and control position information corresponding to the virtual object in the next frame according to the target posture information, the target trajectory information and the animation generation model, including: obtaining the control posture information and the control position information according to the target posture information, the new target trajectory information and the animation generation model.

[0203] According to one or more embodiments of the present disclosure, Example 11 provides a training device for an animation generation model of a virtual object, the device comprising: a first generation module for generating a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set comprises sample input information and sample output information, the sample input information comprises posture information and trajectory information corresponding to the target object in the current frame, and the sample output information comprises target posture information, target trajectory information and target position information corresponding to the target object in the next frame; a first processing module for inputting the posture information and trajectory information corresponding to the target object in the training sample into a target model for each of the training samples, so as to obtain predicted sample information corresponding to the training sample output by the target model, the predicted sample information comprising predicted posture information, predicted trajectory information and predicted position information corresponding to the target object in the next frame; a determination module for determining the target loss of the target model based on the sample input information, the predicted sample information and the sample output information; a training module for training the target model according to the target loss to obtain a trained animation generation model.

[0204] According to one or more embodiments of the present disclosure, Example 12 provides an animation generation device for a virtual object, the device comprising: a second acquisition module for acquiring target posture information and target trajectory information corresponding to the virtual object; a second processing module for obtaining control posture information and control position information corresponding to the virtual object in the next frame based on the target posture information, the target trajectory information and the animation generation model, wherein the animation generation model is a model trained by a training method for the animation generation model of a virtual object based on any one of Examples 1-8; a control module for controlling the movement of the virtual object according to the control posture information and the control position information to generate an animation corresponding to the virtual object.

[0205] According to one or more embodiments of the present disclosure, Example 13 provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in any one of Examples 1-10 when executed by a processing device.

[0206] According to one or more embodiments of the present disclosure, Example 14 provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of any one of the methods described in Examples 1-10.

[0207] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0208] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0209] Although the subject matter has been described using language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method and will not be elaborated upon here.

Claims

1. A training method for a virtual object animation generation model, characterized in that: The method comprises: Generate a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set includes sample input information and sample output information, the sample input information and the sample output information are extracted from the motion capture data, the sample input information includes posture information and trajectory information corresponding to the target object in the current frame, and the sample output information includes target posture information, target trajectory information, and target position information corresponding to the target object in the next frame; For each training sample, inputting the posture information and trajectory information corresponding to the target object in the training sample into the target model to obtain predicted sample information corresponding to the training sample output by the target model, the predicted sample information including predicted posture information, predicted trajectory information, and predicted position information corresponding to the target object in the next frame; Determining a target loss of the target model based on the sample input information, the predicted sample information, and the sample output information; The target model is trained according to the target loss to obtain a trained animation generation model.

2. The method according to claim 1, characterized in that The target model includes an encoding sub-model, a prediction sub-model and a decoding sub-model; The step of inputting the posture information and trajectory information corresponding to the target object in the training sample into the target model to obtain predicted sample information corresponding to the training sample output by the target model includes: Encoding the posture information and trajectory information corresponding to the target object based on the encoding sub-model to obtain an encoding vector corresponding to the sample input information; Perform prediction based on the prediction sub-model and the encoding vector to generate a prediction vector; The prediction vector is decoded based on the decoding sub-model to obtain the prediction sample information.

3. The method according to claim 2, characterized in that The determining the target loss of the target model based on the sample input information, the predicted sample information, and the sample output information includes: Determining a first loss of the encoding sub-model based on the sample input information, the encoding vector, and the encoding sub-model; Determining a second loss of the decoding sub-model based on the sample input information, the encoding vector, the encoding sub-model, and the decoding sub-model; Determining a third loss of the target model based on the predicted sample information and the sample output information; The target loss is determined based on the first loss, the second loss, and the third loss.

4. The method according to claim 3, characterized in that The determining, based on the sample input information, the encoding vector, and the encoding sub-model, a first loss of the encoding sub-model includes: adding noise to the sample input information to obtain noise sample information; Inputting the noise sample information into the coding sub-model to obtain a noise sample vector encoded by the coding sub-model; The first loss is determined according to the noise sample vector and the sample input information.

5. The method according to claim 3, characterized in that The determining, based on the sample input information, the encoding vector, the encoding sub-model, and the decoding sub-model, a second loss of the decoding sub-model includes: adding noise to the sample input information to obtain noise sample information; Inputting the noise sample information into the coding sub-model to obtain a noise sample vector encoded by the coding sub-model; Inputting the noise sample vector into the decoding sub-model to obtain decoding sample information obtained by decoding the decoding sub-model; The second loss is determined according to the decoded sample information and the sample input information.

6. The method according to claim 2, characterized in that The posture information corresponding to the target object includes the posture information corresponding to each joint point of the target object, and the target position information corresponding to the target object is the position information of the root node of the target object; the posture information corresponding to each joint point includes current posture information and historical posture information, and the trajectory information corresponding to the target object includes historical trajectory information and future trajectory information. The encoding vector corresponding to the sample input information includes implicit features, key posture features and historical key posture features corresponding to the key joint points of the target object, and future trajectory features corresponding to the target object. The implicit features are used to store features corresponding to the posture and trajectory extracted based on the sample input information.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining the environment information corresponding to the target object, The step of inputting the posture information and trajectory information corresponding to the target object in the training sample into the target model to obtain predicted sample information corresponding to the training sample output by the target model includes: The posture information, trajectory information and the environmental information corresponding to the target object in the training sample are input into the target model to obtain the predicted sample information output by the target model.

8. The method according to claim 1, characterized in that The posture information corresponding to the target object includes the posture information corresponding to each joint point of the target object; the posture information corresponding to each joint point includes the position coordinates, rotation and speed of the joint point relative to the root node of the target object, and the trajectory information is formed by the movement position information of each root node in multiple consecutive frames including the current frame, and the movement position information corresponding to each frame includes the ground position coordinates, direction, speed and movement type of the root node in the frame relative to the root node in the current frame.

9. A method for generating animation of a virtual object, characterized in that: The method comprises: Obtain target posture information and target trajectory information corresponding to the virtual object; Obtaining control posture information and control position information corresponding to the virtual object in the next frame based on the target posture information, the target trajectory information, and the animation generation model, wherein the animation generation model is a model trained based on the training method for the virtual object animation generation model according to any one of claims 1 to 8; The virtual object is controlled to move according to the control posture information and the control position information to generate an animation corresponding to the virtual object.

10. The method according to claim 9, characterized in that The method further comprises: receiving a control instruction for the virtual object; generating new target trajectory information according to the control instruction and the target trajectory information corresponding to the virtual object; The step of obtaining control posture information and control position information corresponding to the virtual object in the next frame according to the target posture information, the target trajectory information, and the animation generation model includes: The control posture information and the control position information are obtained according to the target posture information, the new target trajectory information and the animation generation model.

11. A training device for an animation generation model of a virtual object, characterized in that: The device comprises: A first generation module is configured to generate a training sample set based on pre-acquired motion capture data, wherein each training sample in the training sample set includes sample input information and sample output information, the sample input information and the sample output information being extracted from the motion capture data, the sample input information including posture information and trajectory information corresponding to a target object in a current frame, and the sample output information including target posture information, target trajectory information, and target position information corresponding to a next frame of the target object; A first processing module is configured to input, for each training sample, the posture information and trajectory information corresponding to the target object in the training sample into a target model to obtain predicted sample information corresponding to the training sample output by the target model, the predicted sample information including predicted posture information, predicted trajectory information, and predicted position information corresponding to the target object in the next frame; a determination module, configured to determine a target loss of the target model based on the sample input information, the predicted sample information, and the sample output information; A training module is used to train the target model according to the target loss to obtain a trained animation generation model.

12. A device for generating animation of a virtual object, characterized in that: The device comprises: The second acquisition module is used to obtain target posture information and target trajectory information corresponding to the virtual object; a second processing module, configured to obtain control posture information and control position information corresponding to the virtual object in the next frame based on the target posture information, the target trajectory information, and an animation generation model, wherein the animation generation model is a model trained based on the training method for a virtual object animation generation model according to any one of claims 1 to 8; A control module is used to control the movement of the virtual object according to the control posture information and the control position information to generate an animation corresponding to the virtual object.

13. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method according to any one of claims 1 to 10 are implemented.

14. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 10.

Images