Action generation method and device of virtual object, cluster, medium and program product

Abstract

The embodiment of the invention discloses an action generation method and device of a virtual object, a cluster, a medium and a program product, relates to the technical field of action capture, and improves the flexibility of generating actions executed by the virtual object. The method comprises the steps of obtaining first action data; the first action data is used for indicating the motion condition of the reference point in the process of executing the target action by the target object; acquiring second action data; the second action data is used for indicating the movement condition of the reference point in the process that the virtual object executes the target action according to the action template; the virtual object is used for representing the target object in the virtual space; generating third action data based on the first action data and the second action data; the third action data is used for indicating the motion condition of the reference point after the target action is adjusted in combination with the action template; generating an animation video according to the third action data; the animation video comprises a fragment of the adjusted target action executed by the virtual object.

Description

Technical Field

[0001] The present application relates to the field of motion capture technology, and in particular to a method, device, cluster, medium and program product for generating motions of virtual objects. Background Art

[0002] With the rapid development of virtual digital space products such as animation, film, television, games, and the metaverse, more and more scenarios require the display of the process of virtual objects performing physical movements in virtual space. Therefore, in order to meet the process of virtual objects performing physical movements displayed in virtual space and approach or even surpass the process of real objects performing physical movements corresponding to virtual objects in reality, it is necessary to continuously improve the technology of capturing and generating virtual objects performing physical movements.

[0003] Currently, motion capture technology can be used to obtain the process of real objects performing limb movements. The motion capture technology can use different sensors to obtain the movement information of real objects in real space, and convert the captured motion information into animation data of virtual skeletons through algorithms. Based on the obtained animation data, the process of virtual objects performing limb movements can be generated, thereby restoring the actions performed by the real objects.

[0004] In the above-mentioned related technologies, the process of objects in reality performing physical movements can be restored more accurately through motion capture technology. However, the need to display virtual objects performing physical movements in virtual space can be to beautify the movements on the basis of restoring the physical movements performed by objects in reality. This leads to the problem of poor flexibility in the process of virtual characters performing physical movements directly displayed in virtual space through motion capture technology. Summary of the invention

[0005] The embodiments of the present application provide a method, device, cluster, medium and program product for generating actions of a virtual object, which ensure that the style of the actions performed by the object is preserved, beautify the actions performed by the virtual object, and improve the flexibility of generating actions performed by the virtual object.

[0006] In a first aspect, the present application provides a method for generating an action of a virtual object, the method comprising: obtaining first action data; the first action data is used to indicate the joint movement of the target object during the execution of a target action; obtaining second action data; the second action data is used to indicate the joint movement of the virtual object during the execution of the target action according to an action template; the virtual object is used to represent the target object in a virtual space; based on the first action data and the second action data, third action data is generated; the third action data is used to indicate the joint movement after the target action is adjusted in combination with the action template; according to the third action data, an animation video is generated; the animation video includes a clip of the virtual object executing the adjusted target action.

[0007] It can be understood that by obtaining the first action data of the target object performing the target action in reality and the second action data of the virtual object performing the target action according to the action template, the first action data is combined with the second action data to modify and obtain the third action data adjusted to the action template, so that the beautified animation video can be generated by rendering according to the third action data. By automatically adjusting the target action according to the action template, the action performed by the virtual object is beautified while ensuring that the style of the action performed by the object is preserved, and the flexibility of the animation generated after the target action performed by the target object is virtualized is improved.

[0008] In one possible implementation, third action data is generated based on the first action data and the second action data, including: extracting first action features based on the first action data; extracting second action features based on the second action data; and mixing the first action features and the second action features according to specified weights to generate third action data.

[0009] It can be understood that by extracting features from the first action data and the second action data respectively, and then mixing the features according to specified weights, third action data after feature mixing can be obtained, thereby achieving the purpose of adjusting the first action data in combination with the second action data.

[0010] In one possible implementation, third action data is generated based on first action data and second action data, including: based on the first action data, obtaining a first animation curve of a target reference point; the target reference point is a joint used to perform a target action; the first animation curve is used to indicate the movement of the reference point during a time period when the target object performs the target action; based on the second action data, obtaining a second animation curve of the target reference point; the second animation curve is used to indicate the movement of the reference point during a time period when the virtual object performs the target action according to an action template; according to a spherical blending algorithm, the first animation curve and the second animation curve are blended to obtain a third animation curve; the third animation curve is used to indicate the movement of the reference point during a time period when the virtual object performs the target action adjusted in combination with the action template.

[0011] It can be understood that by converting the first motion data and the second motion data into animation curves respectively, generating the first animation curve and the second animation curve, and mixing the first animation curve and the second animation curve according to the spherical mixing algorithm, the third animation curve can be obtained, thereby achieving the purpose of adjusting the first motion data in combination with the second motion data.

[0012] In a possible implementation, generating third action data based on the first action data and the second action data includes: if the target action is an action to be adjusted, generating the third action data based on the first action data and the second action data. The action to be adjusted is a target action whose similarity is less than a specified threshold after the frequency domain similarity calculation is performed between the first action data of the target action and the second action data of the target action.

[0013] It can be understood that the frequency domain similarity calculation is performed on the first action data of the target action and the second action data corresponding to the action template of the target action. If the similarity obtained by the frequency domain similarity calculation is less than the specified threshold, it can be determined that the first action data of the target action needs to be adjusted, so the target action can be determined as the action to be adjusted. After determining that the target action is the action to be adjusted, the first action data can be adjusted in combination with the second action data to generate adjusted third action data, thereby achieving the purpose of adjusting the first action data in combination with the second action data.

[0014] In a possible implementation, generating an animation video according to the third motion data includes: generating a mixed animation video according to the third motion data; and obtaining the animation video by performing low-pass filtering on the mixed animation video.

[0015] It can be understood that in order to perform motion smoothing on the mixed animation video generated by adjusting the first motion data and the second motion data to obtain the third motion data, the mixed animation data can be smoothed by a low-pass filtering algorithm to obtain a smooth animation video.

[0016] In a possible implementation, obtaining the first motion data includes: obtaining the first motion data by performing motion capture on a target object.

[0017] It is understandable that the motion capture technology can directly collect the first motion data of the target object when performing the target motion, so as to achieve the purpose of subsequently adjusting the first motion data in combination with the second motion data to obtain beautified third motion data.

[0018] In a possible implementation, obtaining the first action data includes: obtaining an action video; the action video contains a segment of a target object performing a target action; and obtaining the first action data by performing motion capture on the target object in the action video.

[0019] It can be understood that an action video of the target object performing the target action can be obtained, and the first action data can be obtained by motion capturing the process of the target object performing the target action in the action video, thereby achieving the purpose of subsequently adjusting the first action data in combination with the second action data to obtain beautified third action data.

[0020] In a possible implementation, obtaining the second action data includes: in response to a received selection operation, determining an action template corresponding to a target action from at least one action template; and obtaining the second action data of the action template corresponding to the target action.

[0021] It can be understood that after obtaining the first action data, the action template corresponding to the target action can be determined directly from at least one action template according to the received selection operation, and the second action data of the action template can be obtained, thereby enabling the user to directly select the action template corresponding to the target action from each action template. By allowing the user to freely select the action template corresponding to the second action data, the user can easily select the direction to adjust the target action.

[0022] In a possible implementation, obtaining the second action data includes: determining the action type of the target action based on the first action data; determining the action template corresponding to the target action according to the action type of the target action; and obtaining the second action data of the action template corresponding to the target action.

[0023] It can be understood that after obtaining the first action data, the action type of the target action can be obtained, and the action template corresponding to the action type can be determined, and the second action data of the action template can be obtained. This can realize automatic identification of the action type of the target action, and determine the action template corresponding to the action type according to the action type, thereby obtaining the second action data corresponding to the action template.

[0024] In one possible implementation, if each action type corresponds to at least two action templates, determining the action template of the target action according to the action type of the target action includes: determining at least two action templates corresponding to the action type of the target action according to the action type of the target action; and determining the action template of the target action from the at least two action templates in response to a received selection operation.

[0025] It is understandable that after determining the action type of the target action, the action template corresponding to the action type can be automatically obtained. If the action type corresponds to at least two action templates, the user can determine one of the action templates through a selection operation, so that the user can choose the direction to adjust the target action.

[0026] In a second aspect, an embodiment of the present application provides a virtual object action generation device, which is used to execute any one of the virtual object action generation methods provided in the first aspect.

[0027] In a possible implementation, the embodiment of the present application can divide the action generation device of the virtual object into functional modules according to the method provided in the first aspect above. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. Exemplarily, the embodiment of the present application can divide the action generation device of the virtual object into an acquisition module, a processing module, and a generation module, etc. according to the function. The description of the possible technical solutions and beneficial effects executed by the above-mentioned divided functional modules can refer to the technical solutions provided by the above-mentioned first aspect or its corresponding possible implementation method, which will not be repeated here.

[0028] In a third aspect, an embodiment of the present application provides a computing device, the computing device comprising a processor and a memory, the processor being coupled to the memory; the memory being used to store computer instructions, the computer instructions being loaded and executed by the processor so that the computing device implements the method for generating actions of a virtual object as described in the above aspects.

[0029] In a fourth aspect, an embodiment of the present application provides a computing device cluster, which includes at least one computing device, each computing device including: a processor and a memory, the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the virtual object action generation method provided in the various optional implementation methods of the above-mentioned first aspect.

[0030] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which at least one computer program instruction is stored, and the computer program instruction is loaded and executed by a processor to implement the method for generating actions of a virtual object as described in the above aspects.

[0031] In a sixth aspect, an embodiment of the present application provides a computer program product, the computer program product including computer instructions, the computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device cluster executes the method for generating an action of a virtual object provided in various optional implementations of the first aspect.

[0032] For the specific description of the second to sixth aspects and their various implementations in the present application, reference may be made to the detailed description in the first aspect and its various implementations; and for the beneficial effects of the second to sixth aspects and their various implementations, reference may be made to the beneficial effects analysis in the first aspect and its various implementations, which will not be repeated here.

[0033] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a schematic diagram of a scene for generating an action of a virtual object according to an exemplary embodiment;

[0035] Figure 2 yes Figure 1 A schematic diagram of a motion capture device involved in the illustrated embodiment collecting motion data of a target object;

[0036] Figure 3 is a schematic diagram of a hardware structure of a computing device according to an exemplary embodiment;

[0037] Figure 4 is a flowchart of a method for generating an action of a virtual object according to an exemplary embodiment;

[0038] Figure 5 yes Figure 4 A flowchart of abnormality detection of action sequence data involved in the embodiment shown;

[0039] Figure 6 yes Figure 4 A schematic diagram of action beautification involved in the illustrated embodiment;

[0040] Figure 7 is an architecture diagram of a system for generating an action of a virtual object according to an exemplary embodiment;

[0041] Figure 8 is a structural schematic diagram of a device for generating a motion of a virtual object according to an exemplary embodiment;

[0042] Fig. 9 is a schematic diagram of a computing device according to an exemplary embodiment;

[0043] Fig.10 is a schematic diagram of a computing device cluster according to an exemplary embodiment;

[0044] Fig.11 The diagram is a schematic diagram showing a connection mode between computing device clusters according to an exemplary embodiment. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0046] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0047] Furthermore, in the description of this application, unless otherwise specified, "plurality" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0048] In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit the differences. At the same time, in the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0049] First, the application scenarios of the embodiments of the present application are exemplarily introduced.

[0050] Through virtual digital space products such as animation, film, games, and the metaverse, the process of objects performing actions in the real space can be mapped in the virtual space for animation video display. By constructing virtual objects in the virtual space as digital avatars of objects in the real space, the actions performed by virtual objects can be close to or even exceed the actions performed by objects in the real space. The body movements of objects in the real space can be obtained through motion capture technology.

[0051] Among them, motion capture technology can be used to collect the motion data of an object in real space through optical sensors, inertial sensors, cameras and other devices. The motion of the object in real space can be restored and displayed in virtual space through motion capture technology, and the collected object motion data can be used to generate skeleton animation data. Skeleton can be a data structure composed of virtual joint rotation and position information that is not displayed inside the model of the virtual object.

[0052] At present, after obtaining the process of an object performing an action in the display space through motion capture technology, the corresponding virtual object is directly displayed in the virtual space to perform the action. Although the action of the object in the real space can be restored more accurately, in actual application, especially in the scene of virtual people, when constructing virtual objects of real people, most users have a higher perception of the aesthetics of their own actions than the actual situation. This leads to that when the virtual human action is restored and displayed according to the real action, the user is likely to be dissatisfied with the aesthetics of the displayed virtual human action when looking back at his own virtual human action. In other words, the user expects that the displayed virtual human action can have a certain aesthetic feeling under the premise of maintaining his own action style, while the virtual human action restored by the motion capture device is more realistic but lacks aesthetic feeling. Therefore, after the action data of real people is collected through motion capture technology, the virtual human action can be beautified according to artificial experience through action editing.

[0053] Among them, motion editing can be done by technical artists using professional software tools to edit the position information of bone rotation to form animation. Motion editing can be divided into two types: one is to edit directly from scratch, editing video frames one by one; the other is to modify based on the motion data collected by motion capture technology. Professional software tools can be digital content creation (DCC) tools.

[0054] Editing motion data through professional software tools can achieve the purpose of editing and beautifying the motion data collected by motion capture technology. However, manual editing is inefficient, and the degree of beautification depends on the experience and aesthetic level of technical artists, and the quality of the edited motion cannot be guaranteed.

[0055] In addition, in addition to collecting real-person motion data through motion capture technology and manually editing the motion data through professional software tools to achieve editing and beautification of the motion, motion generation technology can also be used to automatically generate virtual object movements based on instructions, text or video input.

[0056] Among them, the action generation technology can be based on an existing action library and machine learning technology to quickly generate the actions of virtual objects according to multimodal instructions.

[0057] That is to say, a database containing various actions is pre-built as an action library, and then action feature information is extracted from the action library or an action prediction model is trained based on the action library. When the action prediction model is running, it can search for matching future actions in the action library based on the user's command output and the current state of the object, or call the trained action prediction model to predict the future actions of the virtual object.

[0058] In this way, the actions of virtual objects can be quickly generated, which improves the efficiency of virtual object action generation. The generated virtual object actions rely on the action data pre-stored in the action library and cannot retain the style characteristics of the actions performed by the objects in reality.

[0059] In view of this, the embodiment of the present application aims to achieve the purpose of editing and beautifying the actions performed by virtual objects and displaying them while retaining the style of the actions performed by real objects, thereby improving the aesthetics of the actions performed by virtual objects. After obtaining the action data of the object, the action data of the object can be automatically edited and beautified according to the pre-set standard action data, thereby automatically obtaining the beautified action data, and generating an animation video based on the beautified action data for display, thereby beautifying the actions performed by virtual objects while ensuring that the style of the actions performed by the object is retained, and improving the flexibility of generating the actions performed by virtual objects.

[0060] in, Figure 1 A schematic diagram of a scene for generating an action of a virtual object provided by an embodiment of the present application is shown. Figure 1 As shown, the target object wears a motion capture device 11, and the motion capture device 11 collects motion data of the target object. The motion capture device 11 sends the collected motion data to the computing device 10, and the computing device 10 automatically edits and processes the collected motion data. According to the automatically edited motion data, an animation video of the virtual object performing beautified actions can be generated, and the animation video with beautified actions is displayed through the animation video display interface 12.

[0061] The computing device 10 may be a server, a computer device or a terminal, and the computing device 10 may be a device with video processing and video display functions.

[0062] The computing device 10 is communicatively connected to the motion capture device 11 , and the motion capture device 11 can transmit data to the computing device 10 .

[0063] The target object can be any moving object in the real space, such as a person, an animal, etc. The virtual object is used to represent the target object in the virtual space. The virtual object is a three-dimensional model created based on the animation skeleton technology. Each virtual object has its own shape, orientation and direction in the three-dimensional virtual space, and occupies a part of the space in the three-dimensional virtual space.

[0064] For example, Figure 2 Schematic diagram of a motion capture device collecting motion data of a target object according to an embodiment of the present application. Figure 2As shown, when the target object 21 moves, the motion capture device collects the motion data of the target object 21. The target object 21 can divide the body skeleton according to various joints according to the animation skeleton technology.

[0065] The motion data may include the rotation angle of the joint, the motion acceleration, the force of the joint, etc. Figure 2 As shown, the rotation angle α of the knee joint, the rotation angle β of the elbow joint, the movement acceleration of the target object 21 is 6m / s^2, the force F of the knee joint, and the force F of the sole joint.

[0066] In a possible implementation, the computing device 10 may obtain a video containing a target object, and obtain motion data of the target object by performing motion capture analysis on the video.

[0067] That is to say, in one case, the computing device 10 can obtain the motion data of the target object captured by the motion capture device 11 through the motion capture device 11; in another case, the computing device 10 can obtain the motion data of the target object by obtaining a video containing the target object and analyzing the motion of the target object in the video.

[0068] Figure 3 It is a hardware structure diagram of a computing device 10 provided in an embodiment of the present application. The computing device 10 may be a server. The server may be an X86 architecture server, specifically a blade server, a high-density server, a rack server or a high-performance server, etc. The server includes a processor 201, a memory 202 and a bus 203. The server may also include an external display device 204.

[0069] The processor 201 may include one or more processing units, for example, the processor 201 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0070] The memory 202 is used to store data, wherein the memory 202 includes but is not limited to a random access memory (RAM), a read-only memory (ROM), a flash memory, an optical memory, a database, etc.

[0071] The memory 202 is used to interact with the processor 201. The memory 202 may store action templates corresponding to various types of actions. Each type of action may correspond to one or more action templates, which are used to indicate the execution standards of various types of actions. That is to say, the actions executed according to the action template do not need to be beautified. The greater the difference from the action template, the higher the degree of beautification required for the corresponding action.

[0072] The processor 201 , the memory 202 and the external display device 204 are usually connected to each other via the bus 203 , or are connected to each other in other ways.

[0073] The external display device 204 can be used to display an animation video display interface to show the user the animation video after motion beautification processing.

[0074] The processor 201 can be used to compare the motion data of the target object with the action data corresponding to the action template to determine whether each action performed by the target object needs to be beautified, that is, the processor 201 can be used to identify the action that needs to be beautified. The processor 201 can also be used to beautify the action performed by the target object, that is, the processor 201 can be used to query the action template closest to the action data from the database based on the collected action data of the target object. The processor 201 can also be used to perform data mixing on the collected action data of the target object in combination with the action template to generate mixed motion data. The processor 201 can also be used to smooth the mixed motion data.

[0075] It should be noted that the application scenarios and system architectures described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0076] For ease of understanding, the following is an exemplary introduction to the virtual object action generation method provided by the present application in conjunction with the accompanying drawings. The virtual object action generation method is applicable to Figure 1 , Figure 3 Computing device 10 is shown.

[0077] Figure 4A flow chart of a method for generating an action of a virtual object provided by an exemplary embodiment of the present application is shown. The method for generating an action of a virtual object can be executed by a computing device, and the method for generating an action of a virtual object includes the following steps:

[0078] S101, obtaining first action data.

[0079] In an embodiment of the present application, a computing device may acquire first motion data, which may be used to indicate a movement of a reference point when a target object performs a target motion.

[0080] Among them, the process of the target object performing the target action can be realized in the real space, and the first action data can include the position information of the reference point in each video frame when the target object performs the target action in the real space, the rotation angle of the reference point, the motion acceleration, and the force condition of the reference point.

[0081] Exemplarily, the reference point may be any point on the skeleton when the target action is performed, and the reference point may include a joint point of the skeleton.

[0082] In one possible implementation, the first action data is obtained by performing motion capture on a target object; or an action video is obtained; the action video contains a segment of the target object performing a target action, and the first action data is obtained by performing motion capture on the target object in the action video.

[0083] That is, the computing device can obtain the first motion data from the motion capture device, and the first motion data is the motion data of the target object when the motion capture device collects the motion data of the target object. The motion capture device can directly send the collected motion data of the target object to the computing device, and the computing device performs subsequent processing.

[0084] In a possible implementation, the movement process of the target object can be recorded in the form of a video, and the computing device obtains the video recording the movement process of the target object, and obtains the motion data of the target object in the video when performing the target action through motion capture technology.

[0085] That is to say, the user uses a camera, optical sensor or inertial sensor to obtain the motion data of the target object. The user can also record the motion video of the target object and send the motion video of the target object to a computing device. The computing device parses the motion video to obtain the motion data corresponding to when the target object performs various types of actions.

[0086] The target action may be an action belonging to a certain action category performed by a certain body part of the target object during the movement.

[0087] For example, if the target object is a person, the body parts of the target object may include limbs, torso, and shoulders, neck, and head, and each body part may include multiple joint points.

[0088] Among them, different types of actions that different body parts can perform are different.

[0089] For example, the types of actions that can be performed by the upper limbs include waving, swinging, punching, elbowing, etc.; the types of actions that can be performed by the lower limbs include walking, running, jumping, squatting, sitting, dancing, etc.; the types of actions that can be performed by the shoulders, neck and head include shaking head and nodding, etc.; the types of actions that can be performed by the torso include bowing and twisting the waist, etc.

[0090] That is to say, the motion capture device can capture the motion data of each joint point, and through the changes in the time series of each motion data, the type of action that each joint point participates in performing within a certain time period can be determined.

[0091] In one possible implementation, if the motion data of each joint point of the target object during the movement process is obtained, since the movement process is carried out in a time sequence, each joint point at each time point has corresponding motion data at that time point, and each motion data within the time period for collecting the motion data can be obtained for each joint point. In other words, based on the motion data at each time point, the type of action that the joint point participates in performing within each time period can be determined.

[0092] For example, the key time points can be determined based on the speed changes, phase changes or position conditions in the motion data of the joints, and the actions that the joints participate in can be divided according to the key time points.

[0093] That is to say, according to the position of the sole joint within the specified time period, and the contact situation of the sole joint with the ground, combined with the rotation of the knee joint within the specified time period and the speed change of the target object within the specified time period, it can be determined that the type of action performed by the lower limbs of the target object within the specified time period is walking.

[0094] In one possible implementation, if a computing device obtains a motion video containing the motion process of a target object, it can determine the key frames in the motion video used to segment the action based on the motion conditions of each joint of the target object. Each video segment segmented according to the key frames can include the action execution process of at least one action type.

[0095] That is to say, according to the position of the target object's foot joints in the motion video, and the contact situation of the foot joints, combined with the rotation of the knee joints and the speed change of the target object, the key frames for the target object to start and end walking can be determined, and the action type of the target object's lower limbs can be determined based on the key frames. The video clip is walking.

[0096] In the embodiment of the present application, the reference point is the joint point of the skeleton, and the process of beautifying the movement of the joint point of the skeleton is only for illustration and does not constitute a limitation on the method of the embodiment of the present application.

[0097] S102, obtaining second action data.

[0098] In an embodiment of the present application, the computing device may obtain second motion data, which may be used to indicate the movement of a reference point when the virtual object performs a target motion according to the motion template.

[0099] In one possible implementation, the computing device divides each action performed during the movement of the target object according to the acquired first action data according to time points, and after determining the action type of each action performed by the target object, the computing device can obtain the action template corresponding to the determined action type.

[0100] In a possible implementation, based on the first action data, an action type of the target action is determined; according to the action type of the target action, an action template of the target action is determined; and action data of the action template of the target action is acquired as second action data.

[0101] In one possible case, if each action type corresponds to at least two action templates, at least two action templates corresponding to the action type of the target action are determined according to the action type of the target action, and in response to a received selection operation, the action template of the target action can be determined from the at least two action templates.

[0102] The action template may be pre-stored in the database and used to indicate standard actions of various action types.

[0103] In a possible implementation manner, the second motion data may be motion data corresponding to an action template under the action category to which the target action belongs.

[0104] That is, if each action type corresponds to at least two action templates, after determining the action type of the target action, the user can select one of the at least two action templates as the action template of the target action, and obtain the second action data corresponding to the action template.

[0105] For example, if the action type of the target action is waving, the action template with the action type of waving may include action template 1 and action template 2, wherein the waving amplitude of action template 1 is larger than that of action template 2, and the user may select one action template as the target action from action template 1 and action template 2 according to the needs. In other words, if the user needs to beautify the waving action performed by the target object in the direction of a larger amplitude, the user may determine action template 1 as the action template of the target action through a selection operation, and then obtain the second action data corresponding to action template 1; if the user needs to beautify the waving action performed by the target object in the direction of a smaller amplitude, the user may determine action template 2 as the action template of the target action through a selection operation, and then obtain the second action data corresponding to action template 2. This makes it convenient for the user to beautify the target action to the effect required by the user more accurately through simple operations.

[0106] In a possible implementation, in response to a received selection operation, an action template corresponding to the target action may be determined from at least one action template, and second action data of the action template corresponding to the target action may be acquired.

[0107] That is, after the computing device obtains the first action data, various action templates can be displayed to the user, and the user directly selects the action template of the target action from the various action templates through a selection operation, and obtains the second action data corresponding to the action template of the target action.

[0108] For example, if the target action is a squat action, after the computing device obtains the first action data generated when the target object performs the squat action, the computing device can display the action templates of different action types stored in the template library to the user, and each action type can correspond to an action template with different action effects. In other words, action templates corresponding to waving with different effects, action templates corresponding to running with different effects, action templates corresponding to squatting with different effects, etc. can be displayed. The computing device determines the action template corresponding to the target action by receiving a selection operation of the action template corresponding to the squatting effect required by the user, thereby obtaining the second action data of the action template corresponding to the target action.

[0109] S103, generating third motion data based on the first motion data and the second motion data.

[0110] In an embodiment of the present application, the computing device may modify the first action data based on the first action data and in combination with the second action data to generate third action data.

[0111] The third motion data may be used to indicate the joint movement after adjusting the target motion in combination with the motion template.

[0112] In a possible implementation, the first action data and the second action data are mixed and calculated according to a specified weight to obtain mixed action data, and then the mixed action data is smoothed to obtain the third action data.

[0113] That is to say, the third action data can be obtained by mixing the features of the target action with the features of the corresponding action template and then performing low-pass filtering on the mixed features to obtain smoothed features.

[0114] In a possible implementation, a first action feature is extracted based on the first action data, a second action feature is extracted based on the second action data, and the first action feature and the second action feature are mixed according to a specified weight to generate a third action data.

[0115] That is to say, by performing time domain-frequency domain conversion processing on the first motion data, the periodic motion features therein are extracted, and by performing mixing processing on the motion features, the third motion features are obtained, thereby obtaining the third motion data.

[0116] For example, Figure 5 This is a flowchart of anomaly detection of action sequence data involved in an embodiment of the present application. Figure 5 As shown, the computing device obtains first motion data, which may be posture sequence data (S11), performs periodic curve extraction on the first motion data through an FFT autoencoder (S12), and then performs frequency domain similarity calculation on the features extracted from the first motion data and the features obtained after periodic curve extraction of the motion template (S13), performs anomaly detection based on the calculated similarity result (S14), and determines whether the target motion is too different from the motion template based on the size of the obtained similarity result. If it is determined that the similarity between the target motion and the motion template is less than a specified threshold, the target motion is determined to be abnormal (S15). If it is determined that the target motion is abnormal, the target motion and the motion template can be mixed (S16), and then the mixed motion data can be filtered to obtain a smooth curve feature, thereby matching the third motion data (S17).

[0117] In one possible implementation, based on first action data, a first animation curve of a target reference point is obtained, the target reference point may be a reference point for executing a target action, the first animation curve may be used to indicate the movement of the reference point during a time period when the target object executes the target action, based on second action data, a second animation curve of the target reference point is obtained, the second animation curve may be used to indicate the movement of the reference point during a time period when the virtual object executes the target action according to an action template, the first animation curve and the second animation curve are mixed according to a spherical blending algorithm to obtain a third animation curve; the third animation curve may be used to indicate the movement of the reference point during a time period when the virtual object executes the target action adjusted in combination with the action template.

[0118] In a possible implementation, if the target action is an action to be adjusted, third action data is generated based on the first action data and the second action data.

[0119] The action to be adjusted may be a target action whose similarity is less than a specified threshold after frequency domain similarity calculation is performed between the first action data of the target action and the second action data of the target action.

[0120] For example, Figure 6 This is a schematic diagram of an action beautification involved in the embodiment of the present application. Figure 6 As shown, if the target action is walking, the target object obtained by the computing device has an outward-facing posture when performing the target action 41. After the target action 41 is beautified by the above-mentioned action mixing and action smoothing methods, the target action 42 after the knee joint point position is adjusted inward is obtained. Among them, the beautified action data can adjust the action in dimensions such as posture, walking posture, gesture, and artistic action.

[0121] S104: Generate an animation video according to the third action data.

[0122] In an embodiment of the present application, a video clip of a virtual object performing a beautified target action can be generated through a third action video. Since the target object can perform multiple actions during the movement process, the target actions that need to be beautified can be multiple actions, so the video clips generated by each target action need to be animated and spliced ​​to generate a complete animation video to show the movement process of the virtualized target object, and the virtualized movement process includes the virtual object performing each beautified target action.

[0123] In a possible implementation, a mixed animation video is generated according to the third motion data; and the animation video is obtained by performing low-pass filtering on the mixed animation video.

[0124] In a possible implementation, skeleton skinning rendering may be performed according to the third action data to generate an animation video.

[0125] In summary, the embodiment of the present application obtains the first action data of the target object performing the target action in reality, and the second action data of the virtual object performing the target action according to the action template, and modifies the first action data in combination with the second action data to obtain the third action data adjusted to the action template, so that the beautified animation video can be generated by rendering according to the third action data. By automatically adjusting the target action according to the action template, the action performed by the virtual object is beautified while ensuring that the style of the action performed by the object is retained, and the flexibility of the animation generated after the target action performed by the target object is virtualized is improved.

[0126] Figure 7 FIG. 1 shows an architecture diagram of a virtual object action generation system provided by an exemplary embodiment of the present application. Figure 7 As shown, the virtual object motion generation system includes a motion data processing module 310 , a motion beautification algorithm module 320 , a motion asset library module 330 and a visualization module 340 .

[0127] The motion data processing module 310 may include a motion segmentation submodule 311 and a motion classification submodule 312 .

[0128] The action segmentation submodule 311 is used to divide the target object's motion process into action segments in time sequence according to the speed changes and phase changes of each joint point in the first action data of the target object performing the target action, or key frames such as the position of the sole joint touching the ground, or the palm joint hanging down.

[0129] The action classification submodule 312 is used to determine the action type in the action segment according to the velocity information or angular velocity information of the key skeletal joints.

[0130] The motion beautification algorithm module 320 may include a motion search algorithm submodule 321 , a motion mixing submodule 322 , a motion smoothing submodule 323 and a problem segment identification submodule 324 .

[0131] The action search algorithm submodule 321 can be used to search for a set of clips with the closest features in the action asset library according to the action clips of the target action of the acquired target object, and sort the closest action clips. The action clips in the action asset library can be action templates.

[0132] The action mixing submodule 322 may be configured to generate a mixed action segment by using a spherical mixing algorithm according to a specified weight based on the action segment of the target action and the action segment corresponding to the action template closest to the action segment of the target action.

[0133] The motion smoothing submodule 323 may be used to smooth the mixed motion clip based on a low-pass filtering algorithm to generate a smoothed motion clip having the same duration as the mixed motion clip.

[0134] The problem segment identification submodule 324 can be used to determine the difference between the rotation information, angular velocity information and position information of each joint point in the action segment corresponding to the action template based on the acquired action segment of the target action of the target object, and determine whether the difference is too large.

[0135] The action asset library module 330 may include a human skeleton asset library submodule 331 , a standard animation asset library submodule 332 , a stylized animation asset submodule 333 and an open interface.

[0136] The human skeleton asset library submodule 331 includes various human skeleton models such as male, female, old, young, tall, short, fat, and thin.

[0137] The standard animation asset library submodule 332 is a standard animation corresponding to the model in the human skeleton asset library submodule 331 , including animation clips and feature clips.

[0138] The stylized animation asset submodule 333 is the animation curves and frequency domain features of the joints such as the root, foot, wrist, elbow, knee, etc. extracted from the standard animation asset library submodule 332 .

[0139] The visualization module 340 may include a skeleton rendering submodule 341 .

[0140] The skeleton rendering submodule 341 and the skin rendering submodule 342 can be used to perform animation rendering of virtual objects.

[0141] In summary, the embodiment of the present application obtains the first action data of the target object performing the target action in reality, and the second action data of the virtual object performing the target action according to the action template, and modifies the first action data in combination with the second action data to obtain the third action data adjusted to the action template, so that the beautified animation video can be generated by rendering according to the third action data. By automatically adjusting the target action according to the action template, the action performed by the virtual object is beautified while ensuring that the style of the action performed by the object is retained, and the flexibility of the animation generated after the target action performed by the target object is virtualized is improved.

[0142] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the method. It is understandable that in order to realize the above functions, the action generation device of the virtual object includes at least one of the hardware structure and software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0143] The embodiment of the present application can divide the action generation device of the virtual object into functional units according to the above method example. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0144] For example, Figure 8 A schematic diagram of the structure of a virtual object motion generation device 500 provided by an exemplary embodiment of the present application is shown. The virtual object motion generation device 500 is applied to a computing device, or the virtual object motion generation device 500 can be a computing device. The virtual object motion generation device 500 includes:

[0145] The acquisition module 510 is used to acquire first action data; the first action data is used to indicate the movement of the reference point during the target object performs the target action; acquire second action data; the second action data is used to indicate the movement of the reference point during the virtual object performs the target action according to the action template; the virtual object is used to represent the target object in the virtual space;

[0146] The processing module 520 is used to generate third motion data based on the first motion data and the second motion data; the third motion data is used to indicate the movement of the reference point after the target motion is adjusted in combination with the motion template;

[0147] The generation module 530 is used to generate an animation video according to the third action data; the animation video includes a segment of the virtual object performing the adjusted target action.

[0148] For example, combined with Figure 4, the acquisition module 510 can be used to perform the following steps: Figure 4 As shown in S101 and S102, the processing module 520 can be used to perform the following steps: Figure 4 As shown in S103, the generating module 530 can be used to perform the following steps: Figure 4 S104 shown.

[0149] In a possible implementation, the acquisition module 510 is further used to, in response to a received selection operation, determine an action template corresponding to the target action from at least one action template; and acquire the second action data of the action template corresponding to the target action.

[0150] In one possible implementation, the processing module 520 is also used to extract the first action feature based on the first action data; extract the second action feature based on the second action data; and mix the first action feature and the second action feature according to specified weights to generate the third action data.

[0151] In one possible implementation, the processing module 520 is also used to obtain a first animation curve of a target reference point based on the first action data; the target reference point is a joint used to perform the target action; the first animation curve is used to indicate the movement of the reference point during the time period when the target object performs the target action; based on the second action data, obtain a second animation curve of the target reference point; the second animation curve is used to indicate the movement of the reference point during the time period when the virtual object performs the target action according to the action template; according to a spherical blending algorithm, the first animation curve and the second animation curve are blended to obtain a third animation curve; the third animation curve is used to indicate the movement of the reference point during the time period when the virtual object performs the target action adjusted in combination with the action template.

[0152] In a possible implementation, the processing module 520 is further configured to, if the target action is an action to be adjusted, generate the third action data based on the first action data and the second action data.

[0153] In a possible implementation, the action to be adjusted is the target action whose similarity is less than a specified threshold after frequency domain similarity calculation is performed on the first action data of the target action and the second action data of the target action.

[0154] In a possible implementation, the generating module 530 is further configured to generate a mixed animation video according to the third motion data;

[0155] The animation video is obtained by performing low-pass filtering on the mixed animation video.

[0156] In a possible implementation, the acquisition module 510 is further configured to acquire the first motion data by performing motion capture on the target object.

[0157] In a possible implementation, the acquisition module 510 is further used to acquire an action video; the action video contains a segment of the target object performing the target action; and the first action data is acquired by performing motion capture on the target object in the action video.

[0158] In a possible implementation, the acquisition module 510 is also used to determine the action type of the target action based on the first action data; determine the action template of the target action according to the action type of the target action; and acquire the action data of the action template of the target action as the second action data.

[0159] In one possible implementation, if each action type corresponds to at least two action templates, the acquisition module 510 is also used to determine at least two action templates corresponding to the action type of the target action according to the action type of the target action; and in response to a received selection operation, determine the action template of the target action from the at least two action templates.

[0160] For the detailed description of the above optional methods, please refer to the above method embodiments, which will not be repeated here. In addition, the explanation of any virtual object motion generation device provided above and the description of the beneficial effects can refer to the above corresponding method embodiments, which will not be repeated here.

[0161] Among them, the acquisition module 510, the processing module 520 and the generation module 530 can all be implemented by software, or can be implemented by hardware. Exemplarily, the following takes the acquisition module 510 as an example to introduce the implementation of the acquisition module 510. Similarly, the implementation of the processing module 520 and the generation module 530 can refer to the implementation of the acquisition module 510.

[0162] As an example of a software functional unit, the acquisition module 510 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the acquisition module 510 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0163] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0164] As an example of a hardware functional unit, the acquisition module 510 may include at least one computing device, such as a server, etc. Alternatively, the acquisition module 510 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0165] The multiple computing devices included in the acquisition module 510 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 510 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the acquisition module 510 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0166] It should be noted that, in other embodiments, the acquisition module 510 can be used to execute any step in the method for generating an action of a virtual object, the processing module 520 can be used to execute any step in the method for generating an action of a virtual object, and the generation module 530 can be used to execute any step in the method for generating an action of a virtual object. The steps that the acquisition module 510, the processing module 520, and the generation module 530 are responsible for implementing can be specified as needed. The acquisition module 510, the processing module 520, and the generation module 530 respectively implement different steps in the method for generating an action of a virtual object to realize all functions of the device for generating an action of a virtual object. The present application also provides a computing device 100. Fig. 9 As shown, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other through the bus 102. The computing device 100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

[0167] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 The bus 104 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 104 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 106, the processor 104, and the communication interface 108).

[0168] The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0169] The memory 106 may include a volatile memory, such as a random access memory (RAM). The processor 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0170] The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned acquisition module 510, processing module 520, and generation module 530, thereby implementing the action generation method of the virtual object. That is, the memory 106 stores instructions for executing the action generation method of the virtual object.

[0171] Alternatively, the memory 106 stores executable codes, and the processor 104 executes the executable codes to respectively implement the functions of the aforementioned virtual object motion generation device, thereby implementing the virtual object motion generation method. That is, the memory 106 stores instructions for executing the virtual object motion generation method.

[0172] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0173] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0174] like Fig.10As shown, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the action generation method of the virtual object.

[0175] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating actions of a virtual object. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the method for generating actions of a virtual object.

[0176] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the virtual object action generation device. That is, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of one or more modules among the acquisition module 510, the processing module 520 and the generation module 530.

[0177] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.11 A possible implementation is shown. Fig.11 As shown, two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the acquisition module 510. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the processing module 520 and the generation module 530.

[0178] Fig.11 The connection method between the computing device clusters shown may be based on the consideration that the action generation method of the virtual object provided in the present application requires a large amount of storage and calculation data, and therefore the functions implemented by the processing module 520 and the generation module 530 may be handed over to the computing device 100B for execution.

[0179] It should be understood that Fig.11 The functions of the computing device 100A shown in FIG. 1 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0180] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.10 and Fig.11The connection mode of the computing device cluster is different in that the memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the method for generating actions of a virtual object.

[0181] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating actions of a virtual object. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the method for generating actions of a virtual object.

[0182] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions for executing part of the functions of the virtual object motion generation system. That is, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of one or more devices in the virtual object motion generation device.

[0183] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the method for generating an action of a virtual object.

[0184] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the action generation method of the virtual object, or instruct the computing device to execute the action generation method of the virtual object.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a motion of a virtual object, characterized in that: The method comprises: Acquire first motion data; the first motion data is used to indicate the movement of the reference point during the target object performing the target motion; Acquire second action data; the second action data is used to indicate the movement of the reference point in the process of the virtual object performing the target action according to the action template; the virtual object is used to represent the target object in the virtual space; generating third motion data based on the first motion data and the second motion data; the third motion data is used to indicate the motion of the reference point after adjusting the target motion in combination with the motion template; An animation video is generated based on the third motion data; the animation video includes a segment of the virtual object performing the adjusted target motion.

2. The method according to claim 1, characterized in that The obtaining of the second action data comprises: In response to the received selection operation, determining an action template corresponding to the target action from at least one action template; The second action data of the action template corresponding to the target action is obtained.

3. The method according to claim 1, characterized in that The obtaining of the second action data comprises: Determining the action type of the target action based on the first action data; Determining an action template corresponding to the target action according to the action type of the target action; The second action data of the action template corresponding to the target action is obtained.

4. The method according to claim 3, characterized in that: If each action type corresponds to at least two action templates, determining the action template of the target action according to the action type of the target action includes: According to the action type of the target action, determining at least two action templates corresponding to the action type of the target action; In response to the received selection operation, an action template of the target action is determined from the at least two action templates.

5. The method according to any one of claims 1 to 4, characterized in that: The generating third action data based on the first action data and the second action data includes: extracting the first action feature based on the first action data; extracting the second action feature based on the second action data; The first action feature and the second action feature are mixed according to a specified weight to generate the third action data.

6. The method according to any one of claims 1 to 5, characterized in that: The generating third action data based on the first action data and the second action data includes: Based on the first action data, a first animation curve of a target reference point is obtained; the target reference point is a reference point used to perform the target action; the first animation curve is used to indicate the movement of the reference point during the time period when the target object performs the target action; Based on the second action data, a second animation curve of the target reference point is obtained; the second animation curve is used to indicate the movement of the reference point during the time period when the virtual object performs the target action according to the action template; The first animation curve and the second animation curve are mixed according to a spherical mixing algorithm to obtain a third animation curve; the third animation curve is used to indicate the movement of a reference point during a time period when the virtual object performs the target action adjusted in combination with the action template.

7. The method according to any one of claims 1 to 6, characterized in that: The generating third action data based on the first action data and the second action data includes: If the target action is an action to be adjusted, the third action data is generated based on the first action data and the second action data; the action to be adjusted is the target action whose similarity is less than a specified threshold after frequency domain similarity calculation is performed between the first action data of the target action and the second action data of the target action.

8. The method according to any one of claims 1 to 7, characterized in that: Generating an animation video according to the third action data includes: generating a mixed animation video according to the third motion data; The animation video is obtained by performing low-pass filtering on the mixed animation video.

9. The method according to any one of claims 1 to 8, characterized in that: The obtaining of the first action data comprises: The first motion data is acquired by performing motion capture on the target object.

10. The method according to any one of claims 1 to 9, characterized in that: The obtaining of the first action data comprises: Acquire an action video; the action video includes a segment of the target object performing the target action; The first action data is obtained by performing action capture on the target object in the action video.

11. A virtual object motion generation device, characterized in that: The device comprises: an acquisition module, configured to acquire first action data, wherein the first action data is used to indicate the movement of a reference point during the target object's execution of a target action; and to acquire second action data, wherein the second action data is used to indicate the movement of the reference point during the execution of the target action by the virtual object according to the action template; and the virtual object is used to represent the target object in a virtual space. A processing module, configured to generate third motion data based on the first motion data and the second motion data; the third motion data is used to indicate the motion of the reference point after adjusting the target motion in combination with the motion template; A generation module is used to generate an animation video based on the third action data; the animation video includes a segment of the virtual object performing the adjusted target action.

12. The device according to claim 11, characterized in that The acquisition module is further used to determine, in response to a received selection operation, an action template corresponding to the target action from at least one action template; and acquire the second action data of the action template corresponding to the target action.

13. The device according to claim 11, characterized in that The acquisition module is further used to determine the action type of the target action based on the first action data; determine the action template corresponding to the target action according to the action type of the target action; and acquire the second action data of the action template corresponding to the target action.

14. The device according to claim 13, characterized in that If each action type corresponds to at least two action templates, the acquisition module is further used to determine at least two action templates corresponding to the action type of the target action according to the action type of the target action; In response to the received selection operation, an action template of the target action is determined from the at least two action templates.

15. The device according to any one of claims 11 to 14, characterized in that The processing module is further used to extract the first action feature based on the first action data; extract the second action feature based on the second action data; and mix the first action feature and the second action feature according to a specified weight to generate the third action data.

16. The device according to any one of claims 11 to 15, characterized in that The processing module is also used to obtain a first animation curve of a target reference point based on the first action data; the target reference point is a reference point used to execute the target action; the first animation curve is used to indicate the movement of the reference point during the time period when the target object executes the target action; based on the second action data, obtain a second animation curve of the target reference point; the second animation curve is used to indicate the movement of the reference point during the time period when the virtual object executes the target action according to the action template; according to a spherical blending algorithm, the first animation curve and the second animation curve are blended to obtain a third animation curve; the third animation curve is used to indicate the movement of the reference point during the time period when the virtual object executes the target action adjusted in combination with the action template.

17. The device according to any one of claims 11 to 16, characterized in that The processing module is also used to generate the third action data based on the first action data and the second action data if the target action is an action to be adjusted; the action to be adjusted is the target action whose similarity is less than a specified threshold after frequency domain similarity calculation is performed between the first action data of the target action and the second action data of the target action.

18. The device according to any one of claims 11 to 17, characterized in that The generating module is further used to generate a mixed animation video according to the third action data; and obtain the animation video by performing low-pass filtering on the mixed animation video.

19. The device according to any one of claims 11 to 18, characterized in that The acquisition module is further used to acquire the first motion data by performing motion capture on the target object.

20. The device according to any one of claims 11 to 19, characterized in that The acquisition module is further used to acquire an action video; the action video contains a segment of the target object performing the target action; and the first action data is acquired by performing action capture on the target object in the action video.

21. A computing device cluster, characterized in that: It includes at least one computing device, each computing device includes: a processor and a memory, the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the action generation method of the virtual object as described in any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a motion of a virtual object according to any one of claims 1 to 10.

23. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed by a computing device cluster, the computing device cluster executes the method for generating an action of a virtual object according to any one of claims 1 to 10.

Images