Data processing method and apparatus, device, and medium

Abstract

Embodiments of the present application provide a data processing method and apparatus, a device, and a medium. The method can be applied to an animation generation scene and used for improving a repair success rate of an action. The method comprises: acquiring action capture data, and constructing a motion state sequence on the basis of the action capture data, wherein the motion state sequence is used for representing a motion state change of a reference object in the action capture data; on the basis of the motion state sequence, performing imitation learning on an action of the reference object in the action capture data to obtain an action prediction sequence corresponding to the action capture data; on the basis of action prediction information contained in the action prediction sequence, determining first offset information corresponding to an intelligent virtual object, and on the basis of the first offset information, correcting the action prediction sequence to obtain an action correction sequence; and by means of the action correction sequence, driving the intelligent virtual object to repair an action in the action capture data, and determining, as animation repair data, the action repaired by the intelligent virtual object.

Description

Data processing method, device, equipment and medium

[0001] The present application claims priority to the Chinese patent application No. 202410804157.3, filed on June 20, 2024, and entitled "Data processing method, device, equipment and medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence, and in particular to a data processing method, device, equipment and medium. BACKGROUND

[0003] In the current video motion capture scene, a method based on iterative optimization and a method based on neural network direct prediction are usually used to generate video motion capture results. In the method based on iterative optimization, a random human pose parameter can be initialized, and the initialized human pose parameter is updated through optimization of an objective function for several iterations to obtain a final human result, thereby generating a video motion capture result. In the method based on neural network direct prediction, a pre-trained neural network can be used to directly predict a three-dimensional human pose parameter to obtain a video motion capture result. However, whether it is the method based on iterative optimization or the method based on deep neural network direct prediction, due to the lack of depth direction information, the video motion capture result inevitably has various problems such as slipping and tilting, which reduces the success rate of action repair. SUMMARY

[0004] The embodiments of the present application provide a data processing method, device, equipment and medium, which can improve the success rate of action repair.

[0005] In an aspect, the embodiments of the present application provide a data processing method, comprising:

[0006] Obtaining motion capture data, constructing a motion state sequence according to the motion capture data, the motion state sequence being used to represent the motion state change of a reference object in the motion capture data;

[0007] According to the motion state sequence, imitating learning the action of the reference object in the motion capture data to obtain an action prediction sequence corresponding to the motion capture data;

[0008] According to the action prediction information contained in the action prediction sequence, determining first offset information corresponding to an intelligent virtual object, correcting the action prediction sequence according to the first offset information to obtain an action correction sequence;

[0009] Driving the intelligent virtual object to restore the action in the motion capture data through the action correction sequence, and determining the action restored by the intelligent virtual object as animation repair data.

[0010] One embodiment of this application provides a data processing method, including:

[0011] Acquire sample motion data, construct a sample state sequence based on the sample motion data, and use the sample state sequence to characterize the motion state changes of sample objects in the sample motion data;

[0012] The sample state sequence is input into the initial network model, which then learns and imitates the actions of the sample objects in the sample action data, and outputs the sample action prediction sequence corresponding to the sample action data.

[0013] Based on the action prediction information contained in the sample action prediction sequence, the second offset information corresponding to the intelligent virtual object is determined, and the sample action prediction sequence is corrected based on the second offset information to obtain the sample action correction sequence.

[0014] By using the sample action correction sequence, the intelligent virtual object is driven to restore the actions in the sample action data, and an action repair sequence is generated based on the actions restored by the intelligent virtual object.

[0015] Based on the action repair sequence and the sample state sequence, determine the imitation loss corresponding to the initial network model;

[0016] The network parameters of the initial network model are corrected based on the imitation loss. The initial network model containing the corrected network parameters is then determined as the policy network model. The policy network model is used to learn the actions in the motion capture data by imitation to obtain the action prediction sequence.

[0017] One embodiment of this application provides a data processing apparatus, including:

[0018] The first state creation module is used to acquire motion capture data and construct a motion state sequence based on the motion capture data. The motion state sequence is used to characterize the motion state changes of the reference object in the motion capture data.

[0019] The first imitation learning module is used to imitate and learn the actions of the reference object in the motion capture data based on the motion state sequence, so as to obtain the motion prediction sequence corresponding to the motion capture data.

[0020] The first action correction module is used to determine the first offset information corresponding to the intelligent virtual object based on the action prediction information contained in the action prediction sequence, and to correct the action prediction sequence based on the first offset information to obtain the action correction sequence.

[0021] The first motion recovery module is used to drive the intelligent virtual object to recover the motion in the motion capture data through the motion correction sequence, and to determine the motion recovered by the intelligent virtual object as the animation repair data.

[0022] The embodiment of the present application provides a data processing device, comprising:

[0023] The second state creation module is configured to acquire sample action data, and construct a sample state sequence according to the sample action data, wherein the sample state sequence is used for representing a motion state change of a sample object in the sample action data.

[0024] The second imitation learning module is configured to input the sample state sequence into an initial network model, and perform imitation learning on an action of the sample object in the sample action data through the initial network model, to output a sample action prediction sequence corresponding to the sample action data.

[0025] The second action correction module is configured to determine second offset information corresponding to the intelligent virtual object according to action prediction information contained in the sample action prediction sequence, and correct the sample action prediction sequence according to the second offset information to obtain a sample action correction sequence.

[0026] The second action recovery module is configured to drive the intelligent virtual object to recover the action in the sample action data through the sample action correction sequence, and generate an action repair sequence according to the action recovered by the intelligent virtual object.

[0027] The loss determination module is configured to determine an imitation loss corresponding to the initial network model according to the action repair sequence and the sample state sequence.

[0028] The parameter correction module is configured to correct network parameters of the initial network model according to the imitation loss, and determine the initial network model containing the corrected network parameters as a policy network model, wherein the policy network model is used for performing imitation learning on the action in the action capture data to obtain an action prediction sequence.

[0029] The embodiment of the present application provides a computer device, comprising a memory and a processor, the memory is connected with the processor, the memory is used for storing a computer program, and the processor is used for calling the computer program, so that the computer device executes the method provided in the above-mentioned aspect of the embodiment of the present application.

[0030] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, the computer program is suitable for being loaded and executed by a processor, so that a computer device with the processor executes the method provided in the above-mentioned aspect of the embodiment of the present application.

[0031] According to an aspect of the present application, a computer program product is provided, which can include a computer program stored in a computer readable storage medium. A processor of a computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program to enable the computer device to perform the method provided in the aspect.

[0032] In the embodiments of the present application, a motion state sequence is constructed based on the acquired motion capture data, and the motion state sequence can be used to represent the motion state changes of the reference object in the motion capture data. An action prediction sequence is obtained by imitative learning on the motion state sequence. First offset information corresponding to the intelligent virtual object is determined according to the action prediction information contained in the action prediction sequence, and the action prediction sequence is corrected according to the first offset information to obtain an action correction sequence. The intelligent virtual object is driven to restore the action in the motion capture data through the action correction sequence, and the action restored by the intelligent virtual object is extracted as animation repair data. As can be seen, after the action prediction sequence is obtained by imitative learning, the action prediction sequence is not directly used to drive the intelligent virtual object to restore the action, but the first offset information of the intelligent virtual object needs to be calculated according to the action prediction information, and the action prediction sequence is corrected based on the first offset information. Then, the intelligent virtual object restores the action of the reference object by using the corrected action prediction sequence (action correction sequence), which can reduce the risk of falling, jumping and tilting of the intelligent virtual object, and thus can improve the success rate of action repair. BRIEF DESCRIPTION OF DRAWINGS

[0033] FIG. 1 is a structural schematic diagram of a network architecture provided by an embodiment of the present application;

[0034] FIG. 2 is a flowchart of a data processing method provided by an embodiment of the present application;

[0035] FIG. 3 is a schematic diagram of a form of an intelligent virtual object provided by an embodiment of the present application;

[0036] FIG. 4 is a pre-processing schematic diagram of an intelligent virtual object provided by an embodiment of the present application;

[0037] FIG. 5 is a structural schematic diagram of a physical engine provided by an embodiment of the present application;

[0038] FIG. 6 is a schematic diagram of an algorithm framework constructed based on a physical engine provided by an embodiment of the present application;

[0039] FIG. 7 is a flowchart of a data processing method provided by an embodiment of the present application;

[0040] FIG. 8 is a flowchart of a reinforcement imitative learning algorithm provided by an embodiment of the present application;

[0041] FIG. 9 is a visual diagram of a result after action repair according to an embodiment of the present application;

[0042] FIG. 10 is a structural diagram of a data processing apparatus according to an embodiment of the present application;

[0043] FIG. 11 is a structural diagram of a data processing apparatus according to an embodiment of the present application;

[0044] FIG. 12 is a structural diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0046] The embodiments of the present application relate to machine learning and reinforcement learning under artificial intelligence, and specifically relate to imitation learning in machine learning methods. The imitation learning can learn a policy network model by observing actions of an existing motion capture dataset, to imitate the actions in the motion capture dataset. The reinforcement learning can learn an optimal behavior policy by interacting with an environment and learning from feedback of the environment; in the reinforcement learning, the state of the environment is observed, and a motion is taken to maximize the accumulated feedback.

[0047] Please refer to FIG. 1, which is a structural diagram of a network architecture according to an embodiment of the present application. The network architecture can include a server 10d and a terminal cluster, which can include one or more terminal devices, and the number of terminal devices included in the terminal cluster is not limited herein. As shown in FIG. 1, the terminal cluster can specifically include a terminal device 10a, a terminal device 10b, and a terminal device 10c, etc. All terminal devices in the terminal cluster (for example, which can include the terminal device 10a, the terminal device 10b, and the terminal device 10c, etc.) can be network-connected with the server 10d, so that each terminal device can perform data interaction with the server 10d through the network connection.

[0048] The terminal devices of the terminal cluster can include smart phones, tablet computers, notebook computers, palm computers, mobile internet devices (MID), wearable devices (such as smart watches, smart bracelets, etc.), smart voice interaction devices, smart home appliances (such as smart televisions, etc.), vehicle-mounted devices, aircraft, and other electronic devices, and the type of terminal device is not limited herein.

[0049] The server 10d can be a standalone physical server, a server cluster or a distributed system formed by multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, and the like basic cloud computing services. The type of the server is not limited in the present application.

[0050] Referring to FIG. 2, FIG. 2 is a flowchart of a data processing method according to an embodiment of the present application. It can be understood that the data processing method can be executed by a computer device, which can be a server (such as the server 10d shown in FIG. 1) or a terminal device (such as any one of the terminal devices in the terminal cluster shown in FIG. 1), and the present application does not limit the same. As shown in FIG. 2, the data processing method can include the following steps S101 to S104:

[0051] In step S101, motion capture data is obtained, and a motion state sequence is constructed according to the motion capture data.

[0052] In a video motion capture scenario, motion capture data (which can be referred to as motion capture data for short) can refer to information about the motion of a human body or an object obtained by motion capture technology. Motion capture data can be recorded in digital form, such as position, velocity, acceleration, attitude, and the like. The motion capture data collected by the motion capture technology described above can be used as the action data to be repaired in the present embodiment.

[0053] The motion state sequence can be used to represent the change in the motion state of the reference object in the motion capture data. The reference object can refer to a human body or an object for which motion capture data is obtained by motion capture technology. For example, the reference object can refer to a virtual image in the form of a non-rigid body mainly based on skin.

[0054] The non-rigid body can refer to an object that can change shape during motion. The state and size of the non-rigid body can change over time, and different parts of the non-rigid body can move in different ways when subjected to external forces. The skin can refer to the process of connecting the surface of a three-dimensional model to the skeleton. Through skinning, the surface of the three-dimensional model can deform accordingly when the skeleton moves, thereby achieving realistic animation effects. The motion state sequence can refer to a sequence obtained by sorting a series of motion states in the motion capture data in chronological order. The motion state can be understood as the action of the reference object in the motion capture data.

[0055] In one or more embodiments, the acquisition process of the motion capture data can include but is not limited to: acquiring motion state information of the key parts of the reference object in a target time range through a capture device associated with the reference object; the motion state information can include but is not limited to: joint rotation angle, joint position information, joint speed and joint angular velocity of the key parts; determining the motion state information of the key parts in the target time range as the motion capture data corresponding to the reference object.

[0056] Wherein, the key parts can include but are not limited to one or more of the head, neck, arm joint, leg joint, shoulder joint, etc., which are not limited by the present application. The target time range can be used to represent the motion capture duration of the reference object, or can be used to represent the acquisition duration corresponding to the motion capture data; for example, the motion capture duration of a reference object can be pre-set to 5 minutes, and the starting time of the motion capture is 9:00, then the target time range can be 9:00 to 9:05. Alternatively, in the acquisition process of the motion capture data, the motion capture duration of the reference object does not need to be pre-set, but a series of actions are pre-set; the reference object can move according to the pre-set series of actions, record and capture the motion data of the reference object to obtain the motion capture data.

[0057] It can be understood that the motion capture data usually needs to be acquired using a dedicated motion capture system, which can include a sensing device and data acquisition software. For example, the motion capture system can be an optical motion capture system, which can include one or more cameras (such as ordinary cameras, infrared cameras) and the like; the motion capture system can also be an inertial motion capture system, which can include inertial sensors, accelerometers, gyroscopes and the like; the motion capture system can also be a magnetic motion capture system, which can include magnetic sensors and the like. The above-mentioned capture device can be one or more of inertial sensors, acceleration sensors, position sensors, speed sensors, angular velocity sensors (such as vibration gyroscopes, etc.), magnetic sensors and the like, which are not limited by the present application.

[0058] The motion capture marker can be attached to or worn on the key part of the human body or object (the above-mentioned reference object) that needs to capture the action; the worn motion capture marker can be a special marker point detected by the motion capture system or the capture device itself. Start the data acquisition software and set up the data acquisition configuration according to the device instructions; then the reference object can perform specific actions or movements, and the capture device can record and capture the action data of the reference object, and the captured action data of the reference object can be called motion capture data. The motion capture data can be saved in a standard data format to a computer or other storage medium, and the standard data format can be CSV (Comma-Separated Values, comma-separated value format) or a proprietary format, etc., which is not limited in the present application. The motion capture data can be applied in the fields of imitation learning, human-computer interaction and animation production, etc.

[0059] In one or more embodiments, for the collected motion capture data, the motion capture data can be smoothed to obtain smoothed motion capture data; then the smoothed motion capture data can be feature extracted to obtain key features in the motion capture data; the key features in the motion capture data are converted into state representations, and a motion state sequence containing the state representations is constructed according to the time sequence of the motion capture data. In other words, the motion state sequence can be constructed according to the obtained motion capture data, such as converting the motion state information in the motion capture data into the state representation required for imitation learning, which can be a representation form of the motion state information recognizable by the imitation learning. The motion capture data can include joint position information, joint rotation angle, joint speed, joint angular velocity and other information, which can be used to describe the motion state of the human body or object in space. The process of constructing the motion state sequence can simplify the data representation of the motion capture data and improve the processing efficiency of subsequent tasks (for example, reinforcement imitation learning).

[0060] Optionally, after the motion capture data is obtained, the motion capture data can be preprocessed, such as data cleaning, data smoothing, etc. Data cleaning can refer to checking and processing missing values, abnormal values, and error data that may exist in the motion capture data. Data smoothing refers to smoothing the motion capture data, which can reduce noise and fluctuations in the motion capture data. For the preprocessed motion capture data, the key features required for imitation learning can be extracted from the motion capture data according to the requirements of the imitation learning task. The key features can refer to the features of the key parts of the reference object. Further, the extracted key features can be converted into state representations according to the requirements of the imitation learning task. For example, the joint rotation angle, joint position information, joint angular velocity, and joint velocity of the key parts can be used as motion state information. The motion state information can be encoded into a format that can be recognized by a machine learning model, such as a vector or a tensor. The vector or tensor can be the data structure of the input data accepted by the machine learning model, and the vector or tensor can be referred to as a state representation.

[0061] Converting the joint rotation angle (e.g., Euler angle, four elements, etc.) in the motion capture data into a state representation can be used to describe the pose and action of the joint of the reference object in the motion capture data. Converting the joint position information (e.g., three-dimensional coordinates) in the motion capture data into a state representation can be used to describe the position change of the joint of the reference object in the motion capture data. Converting the joint velocity and joint angular velocity in the motion capture data into a state representation can be beneficial for capturing the dynamic characteristics of the action in the motion capture data and enriching the motion state information.

[0062] Step S102, according to the motion state sequence, the action of the reference object in the motion capture data is imitated to obtain an action prediction sequence corresponding to the motion capture data.

[0063] Specifically, the above motion state sequence can be used as input data of a policy network model. The policy network model can be used to imitate the motion state sequence to obtain an action prediction sequence. The policy network model can be a trained imitation learning policy network, and the training process of the policy network model will be described later. The policy learning network model can be a machine learning model. Imitation learning is a method of directly learning a policy by observing the action in the motion capture data. For example, the motion state sequence corresponding to the motion capture data can be used as input data, and the policy network model can be used to learn and output an action prediction sequence that matches the action in the motion capture data, thereby realizing the imitation of the action policy in the motion capture data.

[0064] The network structure of the policy network model can be any one of an artificial neural network (ANN), a recurrent neural network (RNN), a long short-term memory (LSTM), a convolutional neural network (CNN), a deep reinforcement learning model, an autoencoder, a Transformer network model (a deep network model using a self-attention mechanism), or a combination of two or more network structures, or a deformation of any network structure, and the application does not limit the network structure of the policy network model.

[0065] In one or more embodiments, the motion state sequence can be input to the policy network model, the motion state sequence is encoded by an encoding layer in the policy network model to obtain a state encoding vector, and the state encoding vector is identified by a prediction layer in the policy network model to obtain an action prediction sequence corresponding to the state encoding vector. Each motion state in the motion state sequence can be input to the policy network model in time sequence, and the policy network model predicts action prediction information matching the motion state according to the input motion state; each motion state in the motion state sequence is input to the policy network model in the form of state representation.

[0066] For example, the motion state received by the policy network model at the t-th moment can be denoted as The policy network model can output action prediction information matching the t-th action in the motion capture data by imitating the first t motion states (or the first t actions in the motion capture data) in the motion state sequence, and the action prediction information can be denoted as a t . Assuming that the motion capture data contains N actions of the reference object, the motion state sequence can contain N motion states, and the motion state sequence can be denoted as The action prediction sequence output by the policy network model can be denoted as {a1,…,a t ,…,a N}. Wherein, N represents the number of actions in the motion capture data, and t is a positive integer less than or equal to N. The action prediction sequence can be a sequence composed of action prediction information corresponding to the N actions in the motion capture data; that is, the action prediction sequence can include N action prediction information.

[0067] In step S103, the first offset information corresponding to the intelligent virtual object is determined according to the action prediction information contained in the action prediction sequence, the action prediction sequence is corrected according to the first offset information, and an action correction sequence is obtained.

[0068] The intelligent virtual object can be a virtual device based on artificial intelligence technology, which can be used to simulate intelligent behavior and language of human beings, and realize functions such as human-computer dialogue and intelligent control. In the embodiments of the present application, the intelligent virtual object can be a virtual image in the form of a rigid body, and the intelligent virtual object can be a rigid body robot. The rigid body refers to an object that does not change shape during movement, has a fixed shape and size, and moves in the same way when subjected to external force.

[0069] It should be noted that the motion capture data can exist in the form of a non-rigid body mainly based on skinning, and such data generally does not have concepts such as mass and density; and it is necessary to automatically create a rigid body mapping with mechanical characteristics such as density and mass for such data in the physical engine, such as creating an intelligent virtual object in the form of a rigid body in the physical engine. The intelligent virtual object in the form of a rigid body can include two types of pure skeleton version and skinning version. In actual application scenarios, different tasks can construct different versions of intelligent virtual objects.

[0070] Please refer to FIG. 3, which is a schematic diagram of a form of an intelligent virtual object according to an embodiment of the present application. As shown in FIG. 3, the reference object in the motion capture data can be in the form of a non-rigid body mainly based on skinning, as shown in the non-rigid form 20a in FIG. 3. The action of the reference object in the motion capture data can be used to create a corresponding intelligent virtual object in the physical engine, and when the intelligent virtual object is a pure skeleton version, the intelligent virtual object can be in the pure skeleton form 20b as shown in FIG. 3, and the pure skeleton form can also be called a capsule form; when the intelligent virtual object is a skinning version, the intelligent virtual object can be in the skinning form 20c as shown in FIG. 3. The creation idea of the intelligent virtual object in the pure skeleton version can be: assigning a capsule joint with a corresponding length to each bone of the intelligent virtual object, and connecting to build the intelligent virtual object in the pure skeleton version. The creation idea of the intelligent virtual object in the skinning version can be: assigning each vertex to the joint with the largest skinning weight, and unifying all vertices of the joint to form a convex hull, and building intelligent virtual objects in different body states.

[0071] Specifically, after outputting the action prediction sequence by the policy network model, the intelligent virtual object in the physics engine can be preprocessed; for example, according to the action prediction information contained in the action prediction sequence, the minimum height corresponding to the intelligent virtual object can be determined, and the height of the plane in the physics engine can be obtained; if the minimum height is less than the height of the plane, the height difference between the height of the plane and the minimum height can be obtained; according to the action prediction information contained in the action prediction sequence, the body offset angle corresponding to the intelligent virtual object can be determined; the height difference and the body offset angle are determined as the first offset information corresponding to the intelligent virtual object. Subtract the height difference in the first offset information from the action prediction sequence, such as subtracting each action prediction information in the action prediction sequence from the height difference in the first offset information, to obtain a candidate action sequence; according to the body offset angle in the first offset information, a rotation angle is added to the target part associated with the candidate action sequence to obtain an action correction sequence.

[0072] The horizontal height in the physics engine can refer to the ground height in the physics engine, and the body offset angle can refer to the inclination angle between the intelligent virtual object and the ground in the physics engine. The minimum height of the intelligent virtual object in the physics engine can be inferred and calculated according to the action prediction sequence, and if the minimum height of the intelligent virtual object is less than the ground height (plane height) in the physics engine, it indicates that the intelligent virtual object may have a problem of penetrating the ground in the physics engine, and therefore the height difference between the plane height in the physics engine and the height of the intelligent virtual object can be calculated, which can be referred to as a compensation value. Then, the action prediction sequence can be subtracted by the uniform height difference to obtain a candidate action sequence, which can constrain the first frame of the intelligent virtual object to stand on the ground in the physics engine, and can solve the problems of penetrating the ground and jumping steps of the intelligent virtual object in the physics engine. In addition, the body offset angle of the intelligent virtual object can be calculated, and if the body offset angle is greater than zero, an additional rotation angle can be added to the pelvic point (the position point of the target part) of the intelligent virtual object, and the posture of the intelligent virtual object can be corrected, which can solve the problems of tilting and falling of the intelligent virtual object.

[0073] Please refer to FIG. 4, which is a pre-processing diagram of an intelligent virtual object provided by an embodiment of the present application. Assuming that the body offset angle of the intelligent virtual object is calculated as the body offset angle w shown in FIG. 4, an additional rotation angle can be added to the pelvic point of the intelligent virtual object, and the additional rotation angle is the body offset angle w described above. The rotation direction can be the direction indicated by the arrow 30a in FIG. 4, so as to correct the posture of the intelligent virtual object.

[0074] In step S104, the intelligent virtual object is driven to recover the action in the motion capture data by the action correction sequence, and the action recovered by the intelligent virtual object is determined as the animation repair data.

[0075] Specifically, the action correction sequence can be imported into the physical engine, and the intelligent virtual object is driven to restore the t th action in the motion capture data in the physical engine through the t th action correction information in the action correction sequence; t is a positive integer less than or equal to N, and N is used to represent the number of actions in the motion capture data; the t th action restored by the intelligent virtual object in the physical engine is obtained, and the N actions restored by the intelligent virtual object are determined as the animation repair data.

[0076] It should be noted that the physical engine used in the embodiments of the present application can be a pure GPU (Graphics Processing Unit, graphics processor) physical engine, for example, the pure GPU physical engine can be Isaac Gym (a physical environment for reinforcement learning research). Compared with the CPU (Central Processing Unit, central processing unit) + GPU physical engine, the action repair system framework is constructed through the pure GPU physical engine, and the training of the physical engine and the policy network model can be run on the GPU, which can improve the model training and inference speed, and the algorithm framework constructed in this way is more advanced and efficient, which is conducive to the subsequent function maintenance and expansion of the algorithm framework.

[0077] The following will be described taking the pure physical engine Isaac Gym as an example. Please refer to FIG. 5, which is a structural schematic diagram of a physical engine provided by an embodiment of the present application. As shown in FIG. 5, Isaac Gym not only runs the physical engine on the GPU, but also uses CUDA (Compute Unified Device Architecture, parallel computing platform and programming model) interoperability to directly copy the physical data to the deep neural network framework (which can refer to the policy network model here) without using the CPU in the process, which can greatly improve the performance of the RL (Reinforcement Learning, reinforcement learning) training process, thereby significantly shortening the training time. As shown in FIG. 5, the embodiments of the present application can use the GPU to perform apply actions+sim (apply actions+sim), observations+reward calculation (observations+reward calculation), deep neural network forward propagation, etc. Among them, apply actions+sim can refer to the process of imitative learning of the reference object in the motion capture data by the policy network model; observations can refer to the process of restoring the action in the motion capture data by the intelligent virtual object in the physical engine; reward can refer to the feedback or reward obtained according to the action prediction sequence output by the policy network model.

[0078] Please refer to FIG. 6, which is a schematic diagram of an algorithm framework constructed based on a physical engine according to an embodiment of the present application. The learning framework shown in FIG. 6 can refer to a software tool or library used to implement a policy network model. The learning framework can provide various functions and tools to make it easier for developers to build, train and deploy a policy network model. In reinforcement learning, the learning framework can initiate an operation command (Step Command) and a set of action tensors to the environment logic. The operation command can refer to a single step of taking an action in the reinforcement learning environment and observing the feedback of the environment. The operation command can include operations such as performing an action, receiving feedback from the environment, updating the state of motion, etc. The set of action tensors can refer to a set of actions that can be selected in reinforcement learning. In each step, an action can be selected from the set of action tensors to be executed. The set of action tensors can include the tensors corresponding to the action prediction sequence output by the policy network model.

[0079] The environment logic can refer to the rules and dynamics of the external environment in which the intelligent virtual object 40a is located. The environment logic can be used to determine the state of the intelligent virtual object at each time and the actions that can be taken by the intelligent virtual object, and can also affect the feedback obtained by the intelligent virtual object from the environment.

[0080] As shown in FIG. 6, the action determined by the environment logic represents an action that can be performed in reinforcement learning, which can be defined by a discrete action space or a continuous action space. The intelligent virtual object can select different actions to interact with the environment. The set of configuration tensors determined by the environment logic can refer to tensors used to configure the parameters of the environment and the intelligent virtual object in reinforcement learning. The set of configuration tensors can include information such as learning rate, feedback function (or loss function) setting, network structure, etc.

[0081] The physical engine (e.g., a pure GPU physical engine Isaac Gym) can obtain a configuration tensor set and an action tensor through a physical engine tensor interface (Isaac Gym Tensor Application Program Interface) to drive the intelligent virtual object to restore the actions in the motion capture data in the physical engine. The physical engine can also transmit an environment state set through the physical engine tensor interface; the environment state set can represent the state of the reinforcement learning environment, and the environment state contains information describing the current environment. After sending the selected action to the environment and obtaining feedback, the environment state will change accordingly. The environment tensor set transmitted by the physical engine can obtain an observation tensor set (Observation Tensors), and the observation tensor set is transmitted to the learning framework. The above operations are repeatedly performed to drive the intelligent virtual object to restore all actions in the motion capture data in the physical engine to obtain animation repair data. The observation tensor set represents a set of observed environment states in the reinforcement learning environment.

[0082] Embodiments of the present application can combine reinforcement imitation learning (a combination of imitation learning and reinforcement learning) and a physical engine to perform action repair. For example, the motion state of the intelligent virtual object in the physical engine (which can be the restored action of the intelligent virtual object) can be used as repaired animation data, i.e., animation repair data. In embodiments of the present application, the implementation process of action repair can include: loading motion capture data to be repaired from an fbx (a three-dimensional model file format) file, performing imitation learning on the obtained motion capture data through a policy network model, and outputting action prediction data; preprocessing the intelligent virtual object to solve problems such as the first frame of animation in the motion capture data penetrating the ground, falling to the ground, jumping, and tilting; importing the actions in the corrected action prediction sequence (action correction sequence) into the physical engine, using a pure GPU physical engine to obtain repaired animation data (animation repair data), and writing the animation repair data into the fbx file, which can improve the success rate of action data repair.

[0083] In embodiments of the present application, after the policy network model performs imitation learning on the obtained motion capture data to obtain an action prediction sequence, the action prediction sequence is not directly used to drive the intelligent virtual object to restore the action, but the first offset information of the intelligent virtual object is calculated according to the action prediction information, and the action prediction sequence is corrected based on the first offset information. Then, the corrected action prediction sequence (action correction sequence) is used to drive the intelligent virtual object to restore the action of the reference object, which can reduce the risk of the intelligent virtual object falling, jumping, tilting, and the like, and thus can improve the success rate of action repair.

[0084] The training process of the policy network model is described below. For ease of understanding, the policy network model in the training stage can be referred to as an initial network model, and the initial network model after the training is completed can be referred to as a policy network model. In the training stage, a discriminator network can be introduced in the form of an adversarial loss, which is trained by sampling sample action data and data of an imitation learning process. The policy network model after training can be used to repair actions in motion capture data.

[0085] Referring to FIG. 7, FIG. 7 is a flow diagram of a data processing method according to an embodiment of the present application. It can be understood that the data processing method can be executed by a computer device, which can be a server or a terminal device, and the present application does not limit this. As shown in FIG. 7, the data processing method can include the following steps S201 to S206:

[0086] Step S201, obtaining sample action data, and constructing a sample state sequence according to the sample action data.

[0087] The specific implementation process of step S201 can refer to step S101 in the embodiment corresponding to FIG. 2, which will not be repeated here.

[0088] It can be understood that the sample action data can be optical capture (light capture) data or motion capture (motion capture) data, and the present application does not limit this. The optical capture data and the motion capture data are motion state information of a human body or an object captured by two different data acquisition technologies. The optical capture data can be action data obtained by shooting the motion trajectory of the measured object based on a camera and an infrared light source, etc., and extracting key point information from images or videos. The optical capture data includes a series of motion states of a sample object, which can be considered as a reference object in the imitation learning process, which is equivalent to the reference object in the motion capture data described above.

[0089] The embodiment of the application takes optical capture data as an example for training; the action of the sample object in the motion capture data carries noise, such as a serious sliding problem. The strategy in the motion capture data is imitated by the initial network model to restore the three-dimensional coordinates of the original action and ensure that the intelligent virtual object does not fall down and can only stabilize the posture by tripping. The embodiment of the application uses the optical capture dataset to train the adversarial motion of the discriminator network, which can constrain the rationality and smoothness of the inter-frame transition in the animation repair data, that is, the rationality between the actions of the intelligent virtual object can be ensured. In the imitation learning process, the three-dimensional position imitation in the motion capture data can be updated to two-dimensional position imitation, which can improve the action imitation quality. Optionally, after the initial network model is pre-trained using the optical capture dataset (the optical capture data in the optical capture dataset can be considered as clean data without noise or with negligible noise), the pre-trained initial network model can be fine-tuned using the motion capture data carrying noise to obtain the trained strategy network model, which can improve the generalization ability of the strategy network and improve the action repair success rate.

[0090] In step S202, the sample state sequence is input to the initial network model, the action of the sample object in the sample action data is imitated by the initial network model through imitation learning, and a sample action prediction sequence corresponding to the sample action data is output.

[0091] In step S203, the second offset information corresponding to the intelligent virtual object is determined according to the action prediction information contained in the sample action prediction sequence, and the sample action prediction sequence is corrected according to the second offset information to obtain a sample action correction sequence.

[0092] In step S204, the sample action correction sequence is used to drive the intelligent virtual object to restore the action in the sample action data, and an action repair sequence is generated according to the action restored by the intelligent virtual object.

[0093] The specific implementation process of steps S202 to S204 can be referred to the related description in steps S102 to S104 in the embodiment corresponding to FIG. 2, which will not be repeated here.

[0094] In step S205, the imitation loss corresponding to the initial network model is determined according to the action repair sequence and the sample state sequence.

[0095] In the training stage, the reinforcement imitation learning can impose two rewards (also referred to as feedbacks, or referred to as losses) on the initial network model: an imitation similarity loss and a discrimination loss. The imitation similarity loss can be used to measure the similarity between the data of the imitation learning process and the actions of the sample object in the sample action data; for example, the imitation similarity loss can be used to represent the distance between the imitation action in the imitation learning process and the joint position information, joint rotation angle, joint speed, angular velocity of the reference object, for constraining the imitation learning to restore the reference action as much as possible; wherein the imitation action can refer to the action obtained by repairing the action through the physics engine, i.e. the action data contained in the action repair sequence; the reference action can refer to the action of the sample object in the sample action data. The discrimination loss can be used to constrain the over-rationalization of the action in the imitation learning process.

[0096] Specifically, the imitation similarity loss between the action repair sequence and the sample state sequence can be obtained; the sample repair sequence can refer to the action data of the imitation learning process, such as the motion state of the intelligent virtual object in the physics engine; the sample state sequence can refer to the motion state of the sample object in the photogrammetry data; the imitation similarity loss can be determined according to the similarity between the motion state of the intelligent virtual object in the physics engine and the motion state of the sample object in the photogrammetry data.

[0097] The action repair sequence and the sample state sequence are input into the discriminator network, and the discrimination loss is obtained through the discriminator network; if the jitter action power of the intelligent virtual object in the physics engine is greater than the action power threshold, the joint power corresponding to the joint part of the intelligent virtual object is obtained, and the energy loss regularization term is determined according to the joint power corresponding to the joint part; the imitation similarity loss, the discrimination loss and the energy loss regularization term are combined to obtain the imitation loss corresponding to the initial network model. The action power threshold can be the power of the normal action of the intelligent virtual object. Optionally, the above imitation loss can also include the adversarial loss or other losses corresponding to the discriminator network, which is not limited in the present application.

[0098] It can be understood that, due to the difference between the structure of the intelligent virtual object and the structure of the real human body or object, such as the inconsistency in the number of bones, the absence of soft tissue, etc., the subtle changes in the action of the sample object in the light capture data cannot be completely restored, and additional jitter is introduced in the repair process to maintain the stability of the intelligent virtual object. There are differences between the mechanical model of the footstep of the intelligent virtual object and the mechanical model of the environment interaction, such as the absence of additional friction, etc., and the intelligent virtual object brings in jitter in the imitation learning process to ensure the stability of the body. In order to solve the jitter problem of the intelligent virtual object, the embodiment of the present application can obtain the jitter action power, and if the jitter action power is greater than the normal action power (action power threshold), the power of each joint in the intelligent virtual object can be added as an energy loss regularization term in the imitation learning; the energy loss regularization term can be used to constrain the energy consumption of the intelligent virtual object in the imitation learning process, and the high-frequency jitter in the imitation learning process is suppressed.

[0099] In the calculation process of the joint power corresponding to the joint position, the joint torque corresponding to the joint position of the intelligent virtual object and the joint angular velocity can be obtained, and the joint torque and the joint angular velocity can be multiplied to obtain the joint power corresponding to the joint position of the intelligent virtual object. The joint torque refers to the rotational force or torque generated at the joint. In the movement of the intelligent virtual object, a certain torque needs to be applied to each joint to realize the rotational movement around the joint axis. The size and direction of the joint torque can be used to determine the motion state of the joint of the intelligent virtual object, such as joint speed, acceleration, position information, etc.

[0100] Optionally, if the jitter action power of the intelligent virtual object in the physical engine is less than or equal to the action power threshold, the imitation similarity loss and the discrimination loss are combined to obtain the imitation loss corresponding to the initial network model. If the jitter action power of the intelligent virtual object in the physical engine is less than or equal to the action power threshold, it indicates that the intelligent virtual object does not have a jitter problem, so it is not necessary to add an energy loss regularization term in the imitation learning process, that is, it is not necessary to add an energy loss regularization term in the model loss.

[0101] In one or more embodiments, if it is detected that there is a failed action in the action repair sequence, the sample state sequence is determined as a failed imitation sequence, and a sequence training weight is added to the failed imitation sequence; the sequence training weight is used to represent the use probability of the failed imitation sequence in the training stage of the initial network model.

[0102] It can be understood that the complexity of different sample state sequences is different. For example, the motion state in the sample state sequence 1 is a walking state, and the motion state in the sample state sequence 2 is a back somersault. The complexity of the sample state sequence 2 is higher than that of the sample state sequence 1. Compared with the sample state sequence with low complexity, the action imitation of the sample state sequence with high complexity is more difficult, and the attention required by the sample state sequences with different complexities is also different. In the embodiment of the present application, when the intelligent virtual object recovers the action in the sample action data in the physical engine, the intelligent virtual object fails to recover, such as the action recovered by the intelligent virtual object is different from the action in the sample action data, or the intelligent virtual object falls down and the like. At this time, the sample state sequence corresponding to the sample action data can be used as a failure sequence, and these failure sequences can be used as difficult examples for mining.

[0103] In the training phase, the sequence training weight of the sample state sequence can be dynamically updated. For example, when the sample state sequence is a failure sequence, the sequence training weight of the sample state sequence can be increased. When the sample state sequence is not a failure sequence, the sequence training weight of the sample state sequence does not need to be increased, that is, the sequence training weight of the sample state sequence can be maintained unchanged. The higher the sequence training weight is, the more likely the sample state sequence is used to train the initial network model. By mining difficult examples (failure sequences) in the training phase and increasing the sequence training weight corresponding to the failure sequence, the action with greater training difficulty of the initial network model can be gradually constrained, and thus the strategy network model trained can improve the success rate of action repair.

[0104] In step S206, the network parameters of the initial network model are corrected according to the imitation loss, and the initial network model containing the corrected network parameters is determined as the strategy network model.

[0105] Specifically, the network parameters of the initial network model can be iterated according to the imitation loss. When the model loss reaches the training end condition, the training of the initial network model can be terminated, and the network parameters at the time of termination can be used as the trained strategy network model. The training end condition can include but is not limited to that the training times of the initial network model reach the pre-set total training times, or the model loss of the initial network model reaches convergence, and the like, which are not limited in the present application.

[0106] Please refer to FIG. 8, which is a flowchart of a reinforcement imitation learning algorithm provided in an embodiment of the present application. As shown in FIG. 8, after the sample action data 50a is obtained, a sample state sequence can be constructed according to the sample action data 50a. The construction process of the sample state sequence can be understood as an instance mining process of the policy network model in the training stage. Each motion state in the sample state sequence can be input into the initial network model in turn. Through the initial network model, the imitation learning of each motion state in the input sample state sequence can be performed, and a sample action prediction sequence corresponding to the sample action data can be output, such as the action prediction information a corresponding to the t th motion state in the sample state sequence t . Wherein, each motion state in the sample state sequence can also be referred to as a reference motion state of the sample object.

[0107] The action prediction information a t is imported into the physics engine, and according to the action prediction information a t , the intelligent virtual object can be driven to restore the action in the sample action data in the physics engine, and further the motion state of the intelligent virtual object in the physics engine can be extracted to obtain an action repair sequence; for example, the action repair sequence can include the repaired motion state s corresponding to the t th motion state in the sample state sequence t ; for example, the repaired motion state s t can be used to represent the difference between the current repaired action and the target action in terms of joint rotation angle, joint position information, joint speed, and joint angular velocity. Further, the motion state (for example, the motion state s t ) in the action repair sequence and the motion state (for example, the reference motion state ) in the sample state sequence can be input into the discriminator network to obtain a discrimination loss r t amp . The motion state (for example, the motion state s t ) in the action repair sequence and the motion state (for example, the reference motion state ) in the sample state sequence can be used to calculate an imitation similarity loss r t g ; further, the discrimination loss r t amp and the imitation similarity loss r t g are added to obtain an imitation loss, and based on the imitation loss, deep reinforcement learning is performed to train the network parameters of the initial network model.

[0108] Optionally, if the driving of the intelligent virtual object to resume the action in the physical engine fails, i.e., the intelligent virtual object fails in the physical engine, the sequence training weight of the sample state sequence can be increased, and the sample state sequence with the increased sequence training weight (imitating the failure sequence) is taken as the mined difficult example. It can be understood that an initial sequence training weight can be set for each sample state sequence, and the initial sequence training weights of each sample state sequence can be the same or different, which is not limited in the present application. The sequence training weight of each sample state sequence in the training stage can be dynamically updated, such as increasing the sequence training weight of the mined difficult example, and reducing or maintaining the sequence training weight of the repaired successful sample state sequence, which is not limited in the present application.

[0109] In one or more embodiments, in order to verify the motion capture effect of the animation repair scheme based on reinforcement learning provided by the embodiments of the present application (which can be specifically a reinforcement imitation learning algorithm based on a policy network model and a physical engine), the motion capture data is repaired in batches, and the experimental effect can be seen from FIG. 9, which is a visual diagram of the result after motion repair provided by the embodiments of the present application. As shown in FIG. 9, if the action prediction sequence output by the policy network model is directly used to drive the intelligent virtual object to resume the action of the reference object in the physical engine, the intelligent virtual object has a ground penetration problem, as shown in region 50a in FIG. 9. The animation repair scheme based on reinforcement learning provided by the embodiments of the present application can correct the action prediction sequence output by the policy network model to obtain an action correction sequence; and the intelligent virtual object is driven to resume the action of the reference object in the physical engine based on the action correction sequence, i.e., the action of the intelligent virtual object is repaired, and the repaired action solves the ground penetration problem of the intelligent virtual object, as shown in region 50b in FIG. 9.

[0110] As shown in FIG. 9, if the action prediction sequence output by the policy network model is directly used to drive the intelligent virtual object to resume the action of the reference object in the physical engine, the intelligent virtual object has a tilt problem, as shown in region 50c in FIG. 9. The animation repair scheme based on reinforcement learning provided by the embodiments of the present application can correct the action prediction sequence output by the policy network model to obtain an action correction sequence; and the intelligent virtual object is driven to resume the action of the reference object in the physical engine based on the action correction sequence, i.e., the action of the intelligent virtual object is repaired, and the repaired action solves the tilt problem of the intelligent virtual object, as shown in region 50d in FIG. 9.

[0111] It can be seen from the experimental results that the repaired motion does not have physical unreasonable problems such as ground penetration, sliding and floating; and the tilt problem of the repaired animation is significantly improved. Specifically, it is found through quantitative result statistics that the physical unreasonable problems are greatly solved; the body tilt rate of the intelligent virtual object is improved, thereby improving the motion capture effect; through the animation repair scheme based on reinforcement learning provided in the embodiments of the present application, the motion restoration degree and smoothness in the animation repair data are improved, thereby improving the motion repair success rate.

[0112] It should be noted that the animation repair scheme based on reinforcement learning provided in the embodiments of the present application can be applied in a game production scene, used to assist animators in efficiently obtaining high-precision three-dimensional motion data, and can improve game development efficiency. Alternatively, the animation repair scheme based on reinforcement learning can also be applied in automatic driving, virtual anchor, VR (Virtual Reality) / AR (Augmented Reality) and other application scenarios. For example, when the animation repair scheme based on reinforcement learning is applied in an automatic driving scene, three-dimensional motion data about a vehicle can be efficiently obtained, thereby improving the automatic driving accuracy of the vehicle. When the animation repair scheme based on reinforcement learning is applied in a virtual anchor scene, three-dimensional motion data of a virtual anchor can be efficiently obtained, thereby improving the motion coherence and rationality of the virtual anchor.

[0113] It can be understood that in the specific embodiments of the present application, the photographed images or videos of the motion state of the human body in space can be involved, and when the above embodiments of the present application are applied to specific products or technologies, the permission or consent of relevant institutions or departments or the user himself / herself needs to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards in the relevant region.

[0114] Please refer to FIG. 10, which is a structural schematic diagram one of a data processing apparatus provided in the embodiments of the present application. As shown in FIG. 10, the data processing apparatus 1 comprises: a first state creating module 101, a first imitation learning module 102, a first motion correction module 103 and a first motion recovery module 104.

[0115] The first state creating module 101 is configured to obtain motion capture data, and construct a motion state sequence according to the motion capture data, wherein the motion state sequence is used to represent the motion state change of a reference object in the motion capture data.

[0116] The first imitation learning module 102 is configured to perform imitation learning on the motion of the reference object in the motion capture data according to the motion state sequence, and obtain a motion prediction sequence corresponding to the motion capture data.

[0117] The first action correction module 103 is configured to determine first offset information corresponding to the intelligent virtual object according to the action prediction information contained in the action prediction sequence, correct the action prediction sequence according to the first offset information, and obtain an action correction sequence.

[0118] The first action recovery module 104 is configured to drive the intelligent virtual object to recover the action in the motion capture data through the action correction sequence, and determine the action recovered by the intelligent virtual object as the animation repair data.

[0119] In one or more embodiments, the first state creation module 101 acquires the motion capture data for performing the following steps:

[0120] The motion state information of the key part of the reference object in the target time range is collected by referring to the capture device associated with the reference object, and the motion state information includes the joint rotation angle, joint position information, joint speed and joint angular velocity of the key part;

[0121] The motion state information of the key part in the target time range is determined as the motion capture data corresponding to the reference object.

[0122] In one or more embodiments, the first state creation module 101 constructs a motion state sequence according to the motion capture data for performing the following steps:

[0123] The motion capture data is smoothed to obtain smoothed motion capture data, and the key features in the smoothed motion capture data are extracted to obtain the key features in the motion capture data;

[0124] The key features in the motion capture data are converted into state representations, and a motion state sequence containing the state representations is constructed according to the time sequence of the motion capture data.

[0125] In one or more embodiments, the first imitation learning module 102 performs imitation learning on the action of the reference object in the motion capture data according to the motion state sequence to obtain an action prediction sequence corresponding to the motion capture data for performing the following steps:

[0126] The motion state sequence is input into a policy network model, and the motion state sequence is encoded by an encoding layer in the policy network model to obtain a state encoding vector;

[0127] The state encoding vector is identified by a prediction layer in the policy network model to obtain an action prediction sequence corresponding to the state encoding vector.

[0128] In one or more embodiments, the first action correction module 103 determines first offset information corresponding to the intelligent virtual object according to the action prediction information contained in the action prediction sequence for performing the following steps:

[0129] determine a lowest height corresponding to the intelligent virtual object according to the action prediction information contained in the action prediction sequence, and obtain a plane height in the physics engine;

[0130] if the lowest height is less than the plane height, obtain a height difference between the plane height and the lowest height;

[0131] determine a body offset angle corresponding to the intelligent virtual object according to the action prediction information contained in the action prediction sequence;

[0132] determine the height difference and the body offset angle as first offset information corresponding to the intelligent virtual object.

[0133] In one or more embodiments, the first action correction module 103 corrects the action prediction sequence according to the first offset information to obtain an action correction sequence, which is used to perform the following steps:

[0134] subtract the action prediction sequence from the height difference in the first offset information to obtain a candidate action sequence;

[0135] add a rotation angle to a target part associated with the candidate action sequence according to the body offset angle in the first offset information to obtain the action correction sequence.

[0136] In one or more embodiments, the first action recovery module 104 drives the intelligent virtual object to recover the action in the motion capture data through the action correction sequence, and determines the action recovered by the intelligent virtual object as animation repair data, which is used to perform the following steps:

[0137] import the action correction sequence into the physics engine, and drive the intelligent virtual object to recover the t th action in the motion capture data in the physics engine through the t th action correction information in the action correction sequence, t being a positive integer less than or equal to N, N being used to represent the number of actions in the motion capture data;

[0138] obtain the t th action recovered by the intelligent virtual object in the physics engine, and determine the N actions recovered by the intelligent virtual object as the animation repair data.

[0139] According to an embodiment of the present application, the steps involved in the data processing method shown in FIG. 2 can be performed by various modules in the data processing apparatus 1 shown in FIG. 10. For example, step S101 shown in FIG. 2 can be performed by the first state creation module 101 shown in FIG. 10, step S102 shown in FIG. 2 can be performed by the first imitation learning module 102 shown in FIG. 10, step S103 shown in FIG. 2 can be performed by the first action correction module 103 shown in FIG. 10, step S104 shown in FIG. 2 can be performed by the first action recovery module 104 shown in FIG. 10, and so on.

[0140] According to an embodiment of the present application, each module in the data processing apparatus 1 shown in FIG. 10 can be combined into one or several modules respectively or all, or some of the modules can be further split into at least two units that are functionally smaller, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above-mentioned modules are divided based on logical functions. In actual application, the function of one module can also be implemented by at least two units, or the functions of at least two modules can be implemented by one module. In other embodiments of the present application, the data processing apparatus 1 can also include other modules or units. In actual application, these functions can also be implemented by other modules, and can be implemented by cooperation of at least two modules.

[0141] In the embodiments of the present application, after the action prediction sequence is obtained by the policy network model for imitative learning of the acquired motion capture data, the action prediction sequence is not directly used to drive the intelligent virtual object to restore the action, but the first offset information of the intelligent virtual object is calculated according to the action prediction information, and the action prediction sequence is corrected based on the first offset information. Then the intelligent virtual object is driven to restore the action of the reference object by using the corrected action prediction sequence (action correction sequence), which can reduce the risk of falling, jumping and tilting of the intelligent virtual object, and thus can improve the success rate of action repair.

[0142] Please refer to FIG. 11, which is a structural schematic diagram of a data processing apparatus according to an embodiment of the present application. As shown in FIG. 11, the data processing apparatus 2 includes a second state creating module 201, a second imitative learning module 202, a second action correction module 203, a second action restoring module 204, a loss determining module 205, and a parameter correction module 206.

[0143] The second state creating module 201 is configured to acquire sample action data, and construct a sample state sequence according to the sample action data, wherein the sample state sequence is used to represent the change of the motion state of a sample object in the sample action data.

[0144] The second imitative learning module 202 is configured to input the sample state sequence into an initial network model, and perform imitative learning on the action of the sample object in the sample action data by using the initial network model, and output a sample action prediction sequence corresponding to the sample action data.

[0145] The second action correction module 203 is configured to determine second offset information corresponding to the intelligent virtual object according to the action prediction information contained in the sample action prediction sequence, and correct the sample action prediction sequence according to the second offset information to obtain a sample action correction sequence.

[0146] The second action recovery module 204 is configured to drive the intelligent virtual object to recover the action in the sample action data through the sample action correction sequence, and generate an action correction sequence according to the action recovered by the intelligent virtual object.

[0147] The loss determination module 205 is configured to determine an imitation loss corresponding to the initial network model according to the action correction sequence and the sample state sequence.

[0148] The parameter correction module 206 is configured to correct the network parameters of the initial network model according to the imitation loss, and determine the initial network model containing the corrected network parameters as a policy network model, the policy network model being used for imitation learning of the action in the motion capture data to obtain an action prediction sequence.

[0149] In one or more embodiments, the loss determination module 205 determines the imitation loss corresponding to the initial network model according to the action correction sequence and the sample state sequence, and is configured to perform the following steps:

[0150] An imitation similarity loss between the action correction sequence and the sample state sequence is obtained, the action correction sequence and the sample state sequence are input into a discriminator network, and a discrimination loss is obtained through the discriminator network;

[0151] If the jitter action power of the intelligent virtual object in the physical engine is greater than the action power threshold, the joint power corresponding to the joint part of the intelligent virtual object is obtained, and an energy loss regularization term is determined according to the joint power corresponding to the joint part.

[0152] The imitation similarity loss, the discrimination loss, and the energy loss regularization term are combined to obtain the imitation loss corresponding to the initial network model.

[0153] In one or more embodiments, the loss determination module 205 obtains the joint power corresponding to the joint part of the intelligent virtual object, and is configured to perform the following steps:

[0154] The joint torque and the joint angular velocity corresponding to the joint part of the intelligent virtual object are obtained.

[0155] The joint torque and the joint angular velocity are multiplied to obtain the joint power corresponding to the joint part of the intelligent virtual object.

[0156] In one or more embodiments, the loss determination module 205 is further configured to perform the following steps:

[0157] If the jitter action power of the intelligent virtual object in the physical engine is less than or equal to the action power threshold, the imitation similarity loss and the discrimination loss are combined to obtain the imitation loss corresponding to the initial network model.

[0158] In one or more embodiments, the data processing apparatus 2 further comprises a failure sequence determination module 207.

[0159] The failure sequence determination module 207 is configured to, if it is detected that there is a recovery failure action in the action repair sequence, determine the sample state sequence as an imitation failure sequence, and add a sequence training weight to the imitation failure sequence, wherein the sequence training weight is used to represent a probability of using the imitation failure sequence in the training stage of the initial network model.

[0160] According to an embodiment of the present application, the steps involved in the data processing method shown in FIG. 7 can be performed by the respective modules in the data processing apparatus 2 shown in FIG. 11. For example, the step S201 shown in FIG. 7 can be performed by the second state creation module 201 shown in FIG. 11, the step S202 shown in FIG. 7 can be performed by the second imitation learning module 202 shown in FIG. 11, the step S203 shown in FIG. 7 can be performed by the second action correction module 203 shown in FIG. 11, the step S204 shown in FIG. 7 can be performed by the second action recovery module 204 shown in FIG. 11, the step S205 shown in FIG. 7 can be performed by the loss determination module 205 shown in FIG. 11, the step S206 shown in FIG. 7 can be performed by the parameter correction module 206 shown in FIG. 11, and so on.

[0161] According to an embodiment of the present application, the respective modules in the data processing apparatus 2 shown in FIG. 11 can be combined into one or several modules respectively or all together, or some of the modules can be further split into at least two units with smaller functions, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above-mentioned modules are divided based on logical functions, and in actual applications, the functions of one module can also be realized by at least two units, or the functions of at least two modules can be realized by one module. In other embodiments of the present application, the data processing apparatus 2 can also include other modules or units, and in actual applications, these functions can also be realized by other modules or units, and can be realized by at least two modules in cooperation.

[0162] In the embodiments of the present application, after the action prediction sequence is obtained by the policy network model through imitative learning on the acquired motion capture data, the action prediction sequence is not directly used to drive the intelligent virtual object to restore the action, but the first offset information of the intelligent virtual object needs to be calculated according to the action prediction information, and the action prediction sequence is corrected based on the first offset information. Then, the corrected action prediction sequence (action correction sequence) is used to drive the intelligent virtual object to restore the action of the reference object, and the repaired action does not have physical unreasonable problems such as stepping through the ground, sliding, and floating; the tilt problem of the repaired animation is significantly improved. Specifically, through quantitative result statistics, it is found that the physical unreasonable problem is greatly solved; the body tilt rate of the intelligent virtual object is improved, thereby improving the motion capture effect; through the animation repair scheme based on reinforcement learning provided in the embodiments of the present application, the action restoration degree and smoothness in the animation repair data are improved, thereby improving the repair success rate of the action.

[0163] Please refer to FIG. 12, which is a structural schematic diagram of a computer device provided in the embodiments of the present application. As shown in FIG. 12, the computer device 1000 can be a terminal device, for example, the terminal device 10a in the embodiments corresponding to FIG. 1, and can also be a server, for example, the server 10d in the embodiments corresponding to FIG. 1, which will not be limited here. For ease of understanding, the computer device in the present application is taken as an example of a terminal device, which can include a processor 1001, a network interface 1004, and a memory 1005, in addition to which the computer device 1000 can also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory, or a non-volatile memory, for example, at least one disk memory. The memory 1005 can optionally be at least one storage device located away from the aforementioned processor 1001. As shown in FIG. 12, the memory 1005 as a computer readable storage medium can include an operating system, a network communication module, a user interface module, and a device control application program.

[0164] The network interface 1004 in the computer device 1000 can also provide network communication functions, and the optional user interface 1003 can also include a display screen (Display) and a keyboard (Keyboard). In the computer device 1000 shown in FIG. 12, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an interface for the user to input; and the processor 1001 can be used to call the device control application program stored in the memory 1005 to realize:

[0165] obtaining motion capture data, constructing a motion state sequence according to the motion capture data, the motion state sequence being used to represent a change in a motion state of a reference object in the motion capture data;

[0166] imitating learning of an action of the reference object in the motion capture data according to the motion state sequence, to obtain an action prediction sequence corresponding to the motion capture data;

[0167] determining first offset information corresponding to the intelligent virtual object according to action prediction information contained in the action prediction sequence, and correcting the action prediction sequence according to the first offset information to obtain an action correction sequence;

[0168] driving the intelligent virtual object to restore the action in the motion capture data through the action correction sequence, and determining the action restored by the intelligent virtual object as animation repair data.

[0169] Optionally, the processor 1001 can be used to call the device control application program stored in the memory 1005 to realize:

[0170] obtaining sample action data, constructing a sample state sequence according to the sample action data, the sample state sequence being used to represent a change in a motion state of a sample object in the sample action data;

[0171] inputting the sample state sequence into an initial network model, imitating learning of an action of the sample object in the sample action data through the initial network model, and outputting a sample action prediction sequence corresponding to the sample action data;

[0172] determining second offset information corresponding to the intelligent virtual object according to action prediction information contained in the sample action prediction sequence, and correcting the sample action prediction sequence according to the second offset information to obtain a sample action correction sequence;

[0173] driving the intelligent virtual object to restore the action in the sample action data through the sample action correction sequence, and generating an action repair sequence according to the action restored by the intelligent virtual object;

[0174] determining an imitation loss corresponding to the initial network model according to the action repair sequence and the sample state sequence;

[0175] The network parameters of the initial network model are corrected according to the imitation loss, and the initial network model containing the corrected network parameters is determined as the policy network model, which is used for imitation learning on the actions in the action capture data to obtain the action prediction sequence.

[0176] It should be understood that the computer device 1000 described in the embodiments of the present application can perform the description of the data processing method in any one of the preceding embodiments of FIG. 2 and FIG. 7, and can also perform the description of the data processing apparatus 1 in the corresponding embodiment of the preceding FIG. 10 or the description of the data processing apparatus 2 in the corresponding embodiment of the preceding FIG. 11, which will not be described here. In addition, the description of the beneficial effects of using the same method will not be described here.

[0177] In addition, it should be pointed out here that the embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium stores the computer program executed by the data processing apparatus 1 or the data processing apparatus 2 mentioned above, and the computer program includes program instructions, and when the processor executes the program instructions, the description of the data processing method in any one of the preceding embodiments of FIG. 2 and FIG. 7 can be executed, therefore, which will not be described here. In addition, the description of the beneficial effects of using the same method will not be described here. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. For technical details of the computer readable storage medium embodiments involved in the present application, please refer to the description of the method embodiments of the present application. As an example, the program instructions can be deployed on one computer device for execution, or on multiple computer devices located in one place for execution, or on multiple computer devices distributed in multiple places and interconnected through a communication network for execution, and the multiple computer devices distributed in multiple places and interconnected through a communication network can constitute a blockchain system.

[0178] In addition, it should be pointed out here that the embodiments of the present application also provide a computer program product, which can include a computer program that can be stored in a computer readable storage medium. The processor of the computer device reads the computer program from the computer readable storage medium, and the processor can execute the computer program to make the computer device execute the description of the data processing method in any one of the preceding embodiments of FIG. 2 and FIG. 7, therefore, which will not be described here. In addition, the description of the beneficial effects of using the same method will not be described here. For technical details of the computer program product or the computer program embodiments involved in the present application, please refer to the description of the method embodiments of the present application.

[0179] The terms "first", "second", etc. in the specification and claims of the present application and the drawings are used to distinguish different media contents, rather than to describe a specific order. In addition, the term "comprising" and any variation thereof is intended to cover the non-exclusive inclusion. For example, a process, method, device, product or equipment including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, device, product or equipment.

[0180] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0181] The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate an apparatus for realizing the functions specified in one or more flows in the flowchart and / or one or more blocks in the structural schematic diagram. These computer program instructions can also be stored in a computer readable memory that can cause the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable memory generate a product including instruction apparatus, which realizes the functions specified in one or more flows in the flowchart and / or one or more blocks in the structural schematic diagram. These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to generate a computer implemented process, so that the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in one or more flows in the flowchart and / or one or more blocks in the structural schematic diagram.

[0182] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0183] The above disclosure is only the preferred embodiments of the present application, and of course cannot limit the scope of the right of the present application, so the equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A data processing method, characterized by, The method is executed by a computer device, and the method comprises: action capture data is acquired, and a motion state sequence is constructed according to the action capture data, the motion state sequence is used to represent the motion state change of a reference object in the action capture data; imitation learning is performed on the action of the reference object in the action capture data according to the motion state sequence, and an action prediction sequence corresponding to the action capture data is obtained; first offset information corresponding to an intelligent virtual object is determined according to action prediction information contained in the action prediction sequence, and the action prediction sequence is corrected according to the first offset information, and an action correction sequence is obtained; the intelligent virtual object is driven to restore the action in the action capture data through the action correction sequence, and the action restored by the intelligent virtual object is determined as animation repair data.

2. The method of claim 1, wherein, The action capture data is acquired, comprising: motion state information of key parts of the reference object in a target time range is collected through a capture device associated with the reference object, and the motion state information comprises joint rotation angles, joint position information, joint velocities and joint angular velocities of the key parts; the motion state information of the key parts in the target time range is determined as the action capture data corresponding to the reference object.

3. The method according to any one of claims 1 to 2, characterized in that, The motion state sequence is constructed according to the action capture data, comprising: the action capture data is smoothed to obtain smoothed action capture data, and key features in the action capture data are extracted to obtain the key features in the action capture data; the key features in the action capture data are converted into state representations, and a motion state sequence containing the state representations is constructed in the time sequence of the action capture data.

4. The method according to any one of claims 1 to 3, characterized in that, The imitation learning is performed on the action of the reference object in the action capture data according to the motion state sequence, and the action prediction sequence corresponding to the action capture data is obtained, comprising: the motion state sequence is input into a policy network model, and the motion state sequence is encoded through an encoding layer in the policy network model to obtain a state encoding vector; the state encoding vector is identified through a prediction layer in the policy network model to obtain an action prediction sequence corresponding to the state encoding vector.

5. The method according to any one of claims 1 to 4, characterized in that, The first offset information corresponding to the intelligent virtual object is determined according to the action prediction information contained in the action prediction sequence, comprising: a lowest height corresponding to the intelligent virtual object is determined according to the action prediction information contained in the action prediction sequence, and a plane height in a physics engine is acquired; if the lowest height is less than the plane height, a height difference between the plane height and the lowest height is acquired; a body offset angle corresponding to the intelligent virtual object is determined according to the action prediction information contained in the action prediction sequence; the height difference and the body offset angle are determined as the first offset information corresponding to the intelligent virtual object.

6. The method of claim 5, wherein, The action prediction sequence is corrected according to the first offset information to obtain the action correction sequence, comprising: Subtract the action prediction sequence from the height difference in the first offset information to obtain a candidate action sequence; According to the body offset angle in the first offset information, add a rotation angle to a target part associated with the candidate action sequence to obtain an action correction sequence.

7. The method according to any one of claims 1 to 6, characterized in that, The driving of the intelligent virtual object to recover the action in the motion capture data through the action correction sequence comprises: Importing the action correction sequence into a physical engine, driving the intelligent virtual object to recover the tth action in the motion capture data in the physical engine through the tth action correction information in the action correction sequence; t is a positive integer less than or equal to N, and N is used to represent the number of actions in the motion capture data; Obtaining the tth action recovered by the intelligent virtual object in the physical engine, and determining the N actions recovered by the intelligent virtual object as animation repair data.

8. A data processing method, characterized by, The method is executed by a computer device, and the method comprises: Obtaining sample action data, and constructing a sample state sequence according to the sample action data, the sample state sequence being used to represent the motion state change of a sample object in the sample action data; Inputting the sample state sequence into an initial network model, and performing imitation learning on the action of the sample object in the sample action data through the initial network model to output a sample action prediction sequence corresponding to the sample action data; According to the action prediction information contained in the sample action prediction sequence, determining second offset information corresponding to an intelligent virtual object, and correcting the sample action prediction sequence according to the second offset information to obtain a sample action correction sequence; Driving the intelligent virtual object to recover the action in the sample action data through the sample action correction sequence, and generating an action repair sequence according to the action recovered by the intelligent virtual object; According to the action repair sequence and the sample state sequence, determining an imitation loss corresponding to the initial network model; According to the imitation loss, correcting the network parameters of the initial network model, and determining the initial network model containing the corrected network parameters as a policy network model, the policy network model being used to perform imitation learning on the action in the motion capture data to obtain an action prediction sequence.

9. The method of claim 8, wherein, The determination of the imitation loss corresponding to the initial network model according to the action repair sequence and the sample state sequence comprises: Obtaining an imitation similarity loss between the action repair sequence and the sample state sequence, inputting the action repair sequence and the sample state sequence into a discriminator network, and obtaining a discrimination loss through the discriminator network; If the jitter action power of the intelligent virtual object in the physical engine is greater than an action power threshold, obtaining the joint power corresponding to the joint part of the intelligent virtual object, and determining an energy loss regularization term according to the joint power corresponding to the joint part; Combining the imitation similarity loss, the discrimination loss and the energy loss regularization term to obtain the imitation loss corresponding to the initial network model.

10. The method of claim 9, wherein, If the jitter motion power of the intelligent virtual object in the physical engine is greater than the motion power threshold, a joint torque and a joint angular velocity corresponding to a joint part of the intelligent virtual object are obtained. If the jitter motion power of the intelligent virtual object in the physical engine is greater than the motion power threshold, a joint torque and a joint angular velocity corresponding to a joint part of the intelligent virtual object are obtained. The joint torque and the joint angular velocity are multiplied to obtain the joint power corresponding to the joint part of the intelligent virtual object.

11. The method of claim 9, wherein, The method further comprises: If the jitter motion power of the intelligent virtual object in the physical engine is less than or equal to the motion power threshold, the imitation similarity loss and the discrimination loss are combined to obtain an imitation loss corresponding to the initial network model.

12. The method of claim 8, wherein, The method further comprises: If it is detected that there is a failed recovery motion in the motion repair sequence, the sample state sequence is determined as a failed imitation sequence, and a sequence training weight is added to the failed imitation sequence, the sequence training weight being used to represent a use probability of the failed imitation sequence in a training stage of the initial network model.

13. A data processing apparatus, characterized by: Comprise: A first state creation module is configured to obtain motion capture data, and construct a motion state sequence according to the motion capture data, the motion state sequence being used to represent a motion state change of a reference object in the motion capture data; A first imitation learning module is configured to perform imitation learning on a motion of the reference object in the motion capture data according to the motion state sequence, and obtain a motion prediction sequence corresponding to the motion capture data; A first motion correction module is configured to determine first offset information corresponding to an intelligent virtual object according to motion prediction information contained in the motion prediction sequence, and correct the motion prediction sequence according to the first offset information to obtain a motion correction sequence; A first motion recovery module is configured to drive the intelligent virtual object to recover the motion in the motion capture data through the motion correction sequence, and determine the recovered motion of the intelligent virtual object as animation repair data.

14. A data processing apparatus, characterized by Comprise: A second state creation module is configured to obtain sample motion data, and construct a sample state sequence according to the sample motion data, the sample state sequence being used to represent a motion state change of a sample object in the sample motion data; A second imitation learning module is configured to input the sample state sequence into an initial network model, and perform imitation learning on a motion of the sample object in the sample motion data through the initial network model to output a sample motion prediction sequence corresponding to the sample motion data; A second motion correction module is configured to determine second offset information corresponding to an intelligent virtual object according to motion prediction information contained in the sample motion prediction sequence, and correct the sample motion prediction sequence according to the second offset information to obtain a sample motion correction sequence; A second motion recovery module is configured to drive the intelligent virtual object to recover the motion in the sample motion data through the sample motion correction sequence, and generate a motion repair sequence according to the recovered motion of the intelligent virtual object; a loss determination module, configured to determine an imitation loss corresponding to the initial network model according to the action repair sequence and the sample state sequence; a parameter correction module, configured to correct network parameters of the initial network model according to the imitation loss, and determine an initial network model containing the corrected network parameters as a policy network model, the policy network model being used to perform imitation learning on actions in the action capture data to obtain an action prediction sequence.

15. A computer device, comprising: comprising a memory and a processor; the memory is connected with the processor, and the memory is used to store a computer program, and the processor is used to call the computer program, so that the computer equipment executes the method in any one of claims 1 to 7, or executes the method in any one of claims 8 to 12.

16. A computer readable storage medium characterized by: the computer readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by the processor, so that the computer equipment with the processor executes the method in any one of claims 1 to 7, or executes the method in any one of claims 8 to 12.

17. A computer program product, characterised in that, comprising a computer program, the computer program is executed by the processor to implement the method in any one of claims 1 to 7, or execute the method in any one of claims 8 to 12.

Images