Method and apparatus for generating action of virtual object, and cluster, medium and program product

Abstract

The embodiments of the present application relate to the technical field of action capture. Disclosed are a method and apparatus for generating an action of a virtual object, and a cluster, a medium and a program product, which improve the flexibility of generating an action executed by a virtual object. The method comprises: acquiring first action data, which is used for indicating the movement situation of a reference point when a target object executes a target action; acquiring second action data, which is used for indicating the movement situation of the reference point when a virtual object executes the target action according to an action template, wherein the virtual object is used for representing the target object in a virtual space; on the basis of the first action data and the second action data, generating third action data, which is used for indicating the movement situation of the reference point after the target action is adjusted in view of the action template; and according to the third action data, generating an animation video, which includes clips of the virtual object executing the adjusted target action.

Description

Method, device, cluster, medium and program product for generating motion of virtual object

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on July 17, 2023, with application number 202310876415.4 and application name “A method, device and computing device cluster for motion beautification”, the entire contents of which are incorporated by reference into this application.

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 16, 2023, with application number 202311544502.6 and application name “Method, device, cluster, medium and program product for generating motion of virtual objects”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of motion capture technology, and in particular to a method, device, cluster, medium, and program product for generating motions of virtual objects. Background Art

[0004] With the rapid development of virtual digital space products such as animation, film and television, games, and the metaverse, more and more scenarios require the display of virtual objects performing physical movements in virtual space. Therefore, in order to meet the requirements of the virtual objects performing physical movements displayed in the virtual space and approach or even surpass the physical movements of the real objects corresponding to the virtual objects in reality, it is necessary to continuously improve the technology of capturing and generating virtual objects performing physical movements.

[0005] Currently, motion capture technology can be used to capture the process of real-life objects performing limb movements. This motion capture technology can use different sensors to obtain the movement information of real-life objects in real space, and convert the captured movement information into animation data of virtual skeletons through algorithms. Based on the obtained animation data, the process of virtual objects performing limb movements can be generated, thereby restoring the movements performed by the real-life objects.

[0006] Among the above-mentioned related technologies, motion capture technology can more accurately restore the process of objects in reality performing physical movements. However, the need to display virtual objects performing physical movements in virtual space may require beautifying the movements on the basis of restoring the physical movements performed by objects in reality. This leads to the problem of poor flexibility in the process of virtual characters performing physical movements directly displayed in virtual space through motion capture technology.

[0007] Summary of the Invention

[0008] The embodiments of the present application provide a method, device, cluster, medium and program product for generating actions of virtual objects, which beautify the actions performed by virtual objects while ensuring that the style of the actions performed by the objects is preserved, and improve the flexibility of generating actions performed by virtual objects.

[0009] In a first aspect, the present application provides a method for generating an action of a virtual object, the method comprising: obtaining first action data; the first action data is used to indicate the joint movement of the target object during the execution of the target action; obtaining second action data; the second action data is used to indicate the joint movement of the virtual object during the execution of the target action according to the action template; the virtual object is used to represent the target object in a virtual space; based on the first action data and the second action data, third action data is generated; the third action data is used to indicate the joint movement after the target action is adjusted in combination with the action template; an animation video is generated according to the third action data; the animation video includes a clip of the virtual object executing the adjusted target action.

[0010] It can be understood that by obtaining first action data of a target object performing a target action in reality, and second action data of a virtual object performing the target action according to an action template, the first action data is combined with the second action data to modify the third action data adjusted to the action template, thereby generating a beautified animation video by rendering according to the third action data. By automatically adjusting the target action according to the action template, the action performed by the virtual object is beautified while ensuring that the style of the action performed by the object is preserved, thereby improving the flexibility of the animation generated after virtualizing the target action performed by the target object.

[0011] In one possible implementation, third action data is generated based on the first action data and the second action data, including: extracting first action features based on the first action data; extracting second action features based on the second action data; and mixing the first action features and the second action features according to specified weights to generate third action data.

[0012] It can be understood that by extracting features from the first action data and the second action data respectively, and then mixing the features according to specified weights, the third action data after feature mixing can be obtained, thereby achieving the purpose of adjusting the first action data in combination with the second action data.

[0013] In one possible implementation, third action data is generated based on the first action data and the second action data, including: obtaining a first animation curve of a target reference point based on the first action data; the target reference point is a joint used to perform a target action; the first animation curve is used to indicate the movement of the reference point during a time period when the target object performs the target action; obtaining a second animation curve of the target reference point based on the second action data; the second animation curve is used to indicate the movement of the reference point during a time period when the virtual object performs the target action according to the action template; mixing the first animation curve and the second animation curve according to a spherical blending algorithm to obtain a third animation curve; the third animation curve is used to indicate the movement of the reference point during a time period when the virtual object performs the target action adjusted in combination with the action template.

[0014] It can be understood that by converting the first motion data and the second motion data into animation curves respectively, generating a first animation curve and a second animation curve, and mixing the first animation curve and the second animation curve according to the spherical mixing algorithm, a third animation curve can be obtained, thereby achieving the purpose of adjusting the first motion data in combination with the second motion data.

[0015] In one possible implementation, generating third motion data based on first motion data and second motion data includes: if the target motion is a to-be-adjusted motion, generating the third motion data based on the first motion data and the second motion data. The to-be-adjusted motion is a target motion for which a frequency-domain similarity calculation between the first motion data and the second motion data of the target motion is less than a specified threshold.

[0016] It can be understood that the frequency domain similarity calculation is performed on the first action data of the target action and the second action data corresponding to the action template of the target action. If the similarity obtained by the frequency domain similarity calculation is less than the specified threshold, it can be determined that the first action data of the target action needs to be adjusted, so the target action can be determined as the action to be adjusted. After determining that the target action is the action to be adjusted, the first action data can be combined with the second action data to adjust it to generate the adjusted third action data, thereby achieving the purpose of adjusting the first action data in combination with the second action data.

[0017] In a possible implementation, generating an animation video according to the third motion data includes: generating a mixed animation video according to the third motion data; and obtaining the animation video by performing low-pass filtering on the mixed animation video.

[0018] It can be understood that in order to perform motion smoothing on the mixed animation video generated by adjusting the first motion data and the second motion data to obtain the third motion data, the mixed animation data can be smoothed by a low-pass filtering algorithm to obtain a smooth animation video.

[0019] In a possible implementation, obtaining the first motion data includes: obtaining the first motion data by performing motion capture on a target object.

[0020] It is understandable that the motion capture technology can directly capture the first motion data of the target object when performing the target motion, thereby achieving the purpose of subsequently adjusting the first motion data in combination with the second motion data to obtain beautified third motion data.

[0021] In a possible implementation, obtaining the first action data includes: obtaining an action video; the action video includes a segment of a target object performing a target action; and obtaining the first action data by performing motion capture on the target object in the action video.

[0022] It can be understood that an action video of the target object performing the target action can be obtained, and the first action data can be obtained by motion capturing the process of the target object performing the target action in the action video, thereby achieving the purpose of subsequently adjusting the first action data in combination with the second action data to obtain beautified third action data.

[0023] In a possible implementation, obtaining the second action data includes: in response to a received selection operation, determining an action template corresponding to a target action from at least one action template; and obtaining the second action data of the action template corresponding to the target action.

[0024] It can be understood that after obtaining the first action data, the action template corresponding to the target action can be determined directly from at least one action template according to the received selection operation, and the second action data of the action template can be obtained, so that the user can directly select the action template corresponding to the target action from each action template. By allowing the user to freely select the action template corresponding to the second action data, the user can easily choose the direction to adjust the target action.

[0025] In one possible implementation, obtaining the second action data includes: determining the action type of the target action based on the first action data; determining the action template corresponding to the target action according to the action type of the target action; and obtaining second action data of the action template corresponding to the target action.

[0026] It can be understood that after obtaining the first action data, the action type of the target action can be obtained, and the action template corresponding to the action type can be determined, and the second action data of the action template can be obtained. This can automatically identify the action type of the target action, and determine the action template corresponding to the action type according to the action type, thereby obtaining the second action data corresponding to the action template.

[0027] In one possible implementation, if each action type corresponds to at least two action templates, determining the action template of the target action according to the action type of the target action includes: determining at least two action templates corresponding to the action type of the target action according to the action type of the target action; and determining the action template of the target action from at least two action templates in response to a received selection operation.

[0028] It is understandable that after determining the action type of the target action, the action template corresponding to the action type can be automatically obtained. If the action type corresponds to at least two action templates, the user can determine one of the action templates through a selection operation, so that the user can choose the direction to adjust the target action.

[0029] In a second aspect, an embodiment of the present application provides a device for generating an action of a virtual object, which is used to execute any one of the methods for generating an action of a virtual object provided in the first aspect.

[0030] In one possible implementation, the embodiment of the present application can divide the action generation device of the virtual object into functional modules according to the method provided in the first aspect above. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. Exemplarily, the embodiment of the present application can divide the action generation device of the virtual object into an acquisition module, a processing module, and a generation module, etc. according to the function. The description of the possible technical solutions and beneficial effects executed by the above-mentioned divided functional modules can refer to the technical solutions provided by the above-mentioned first aspect or its corresponding possible implementation method, and will not be repeated here.

[0031] In a third aspect, an embodiment of the present application provides a computing device, which includes a processor and a memory, wherein the processor is coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computing device to implement the method for generating virtual object actions as described in the above aspects.

[0032] In a fourth aspect, an embodiment of the present application provides a computing device cluster, which includes at least one computing device, each computing device including: a processor and a memory, the processor of at least one computing device being used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the virtual object action generation method provided in the various optional implementations of the above-mentioned first aspect.

[0033] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores at least one computer program instruction, and the computer program instruction is loaded and executed by a processor to implement the method for generating actions of a virtual object as described in the above aspects.

[0034] In a sixth aspect, embodiments of the present application provide a computer program product comprising computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computing device cluster to perform the method for generating a virtual object action provided in various optional implementations of the first aspect.

[0035] For the specific descriptions of the second to sixth aspects and their various implementations in this application, reference can be made to the detailed descriptions in the first aspect and its various implementations; and for the beneficial effects of the second to sixth aspects and their various implementations, reference can be made to the analysis of the beneficial effects in the first aspect and its various implementations, which will not be repeated here.

[0036] These and other aspects of the present application will become more readily apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a schematic diagram of a scene showing action generation of a virtual object according to an exemplary embodiment;

[0038] FIG2 is a schematic diagram of a motion capture device collecting motion data of a target object involved in the embodiment shown in FIG1 ;

[0039] FIG3 is a schematic diagram showing a hardware structure of a computing device according to an exemplary embodiment;

[0040] FIG4 is a schematic flow chart showing a method for generating a motion of a virtual object according to an exemplary embodiment;

[0041] FIG5 is a flow chart of anomaly detection of action sequence data involved in the embodiment shown in FIG4 ;

[0042] FIG6 is a schematic diagram of an action beautification involved in the embodiment shown in FIG4 ;

[0043] FIG7 is an architecture diagram of a system for generating motions of a virtual object according to an exemplary embodiment;

[0044] FIG8 is a schematic structural diagram of a device for generating a motion of a virtual object according to an exemplary embodiment;

[0045] FIG9 is a schematic diagram of a computing device according to an exemplary embodiment;

[0046] FIG10 is a schematic diagram showing a computing device cluster according to an exemplary embodiment;

[0047] FIG11 is a schematic diagram showing a connection method between computing device clusters according to an exemplary embodiment. DETAILED DESCRIPTION

[0048] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0049] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0050] Furthermore, in the description of this application, unless otherwise specified, "plurality" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0051] In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit differences. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0052] First, the application scenarios of the embodiments of the present application are exemplarily introduced.

[0053] Through virtual digital space products such as animation, film, television, games, and the metaverse, the process of objects performing actions in real space can be mapped in virtual space for animation and video display. By constructing virtual objects in virtual space as digital avatars of real-world objects, the actions performed by virtual objects can approach or even surpass those of real-world objects. Motion capture technology can be used to capture the physical movements of objects in real space.

[0054] Motion capture technology can be used to collect motion data of an object in real space through devices such as optical sensors, inertial sensors, and cameras. This technology can then be used to reproduce the object's motion in virtual space and generate skeletal animation data from the collected object's motion data. Skeletons are data structures composed of virtual joint rotation and position information, which are not displayed within the virtual object model.

[0055] Currently, motion capture technology is used to capture the process of an object performing an action in a display space, and then the corresponding virtual object is directly displayed in the virtual space to perform the action. Although this can accurately reproduce the movement of the object in the real space, in actual applications, especially in scenarios with virtual humans, when constructing virtual objects of real people, most users have a higher perception of the aesthetics of their own movements than the actual situation. This results in users being dissatisfied with the aesthetics of the displayed virtual human movements when they review their own virtual human movements after reproducing the movements of the real person. In other words, users expect the displayed virtual human movements to have a certain aesthetic quality while maintaining their own movement style. However, directly displaying the virtual human movements restored by motion capture equipment is more realistic but lacks aesthetic quality. Therefore, after collecting real-life movement data through motion capture technology, it is possible to beautify the virtual human movements through motion editing based on human experience.

[0056] Motion editing involves technical artists using specialized software tools to edit the rotational position of bones to create animations. Motion editing can be divided into two types: one is to edit directly from scratch, frame by frame; the other is to modify motion data captured by motion capture technology. Professional software tools can include digital content creation (DCC) tools.

[0057] Editing motion data through professional software tools can achieve the purpose of editing and beautifying the motion data collected by motion capture technology. However, manual editing is inefficient, and the degree of beautification depends on the experience and aesthetic level of the technical artists, and the quality of the edited motion cannot be guaranteed.

[0058] In addition, in addition to collecting real-person motion data through motion capture technology and then manually editing the motion data through professional software tools to achieve motion editing and beautification, motion generation technology can also be used to automatically generate virtual object movements based on instructions, text or video input.

[0059] Among them, the action generation technology can be based on the existing action library and machine learning technology to quickly generate the actions of virtual objects according to multimodal instructions.

[0060] In other words, a database of various actions is pre-built as an action library. Action feature information is then extracted from the library, or an action prediction model is trained based on the library. When the action prediction model is running, it can search for matching future actions in the library based on the user's command output and the object's current state, or it can use the trained action prediction model to predict the virtual object's future actions.

[0061] In this way, the movements of virtual objects can be generated quickly, which improves the efficiency of virtual object movement generation. The generated virtual object movements rely on the movement data pre-stored in the movement library and cannot retain the style characteristics of the movements performed by objects in reality.

[0062] In view of this, the embodiments of the present application aim to achieve the goal of editing and beautifying the actions performed by virtual objects and displaying them while preserving the style of the actions performed by real objects, thereby improving the aesthetics of the actions performed by virtual objects. After obtaining the action data of the object, the obtained action data of the object can be automatically edited and beautified according to pre-set standard action data, thereby automatically obtaining the beautified action data, and generating an animated video based on the beautified action data for display. In this way, while ensuring that the style of the actions performed by the object is preserved, the actions performed by the virtual object are beautified, thereby improving the flexibility of generating the actions performed by the virtual object.

[0063] Figure 1 shows a schematic diagram of a scenario for generating motions of a virtual object provided by an embodiment of the present application. As shown in Figure 1, a target object wears a motion capture device 11, which collects motion data of the target object. The motion capture device 11 sends the collected motion data to a computing device 10, which automatically edits and processes the collected motion data. Based on the automatically edited motion data, the computing device 10 can generate an animated video of the virtual object performing a beautified motion, and the beautified motion animated video is displayed through an animation video display interface 12.

[0064] The computing device 10 may be a server, a computer device or a terminal, and the computing device 10 may be a device with video processing and video display functions.

[0065] The computing device 10 is communicatively connected to the motion capture device 11 , and the motion capture device 11 can transmit data to the computing device 10 .

[0066] The target object can be any moving object in real space, such as a person or animal. A virtual object is used to represent the target object in virtual space. The virtual object is a three-dimensional model created based on animation skeletal technology. Each virtual object has its own shape, length, and orientation in the three-dimensional virtual space and occupies a portion of the space in the three-dimensional virtual space.

[0067] For example, Figure 2 is a schematic diagram of a motion capture device according to an embodiment of the present application collecting motion data of a target object. As shown in Figure 2, the motion capture device collects motion data of target object 21 as it moves. Target object 21 can be divided into its body skeleton according to various joints using animation skeleton technology.

[0068] The motion data may include the rotation angle of the joint, the motion acceleration, the force applied to the joint, etc. For example, as shown in FIG2 , the rotation angle α of the knee joint, the rotation angle β of the elbow joint, the motion acceleration of the target object 21 being 6 m / s², the force F applied to the knee joint, and the force F applied to the sole joint.

[0069] In one possible implementation, the computing device 10 may obtain a video containing a target object, and obtain motion data of the target object by performing motion capture analysis on the video.

[0070] That is to say, in one case, the computing device 10 can obtain the motion data of the target object captured by the motion capture device 11 through the motion capture device 11; in another case, the computing device 10 can obtain the motion data of the target object by obtaining a video containing the target object and analyzing the motion of the target object in the video.

[0071] Figure 3 is a schematic diagram of the hardware structure of a computing device 10 provided in an embodiment of the present application. The computing device 10 can be a server, and the server can be an X86 architecture server, specifically a blade server, a high-density server, a rack server or a high-performance server, etc., wherein the server includes a processor 201, a memory 202 and a bus 203, and the server can also include an external display device 204.

[0072] The processor 201 may include one or more processing units, for example, the processor 201 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0073] The memory 202 is used to store data, wherein the memory 202 includes but is not limited to random access memory (RAM), read-only memory (ROM), flash memory, optical memory, database, etc.

[0074] The memory 202 is used to interact with the above-mentioned processor 201. The memory 202 can store action templates corresponding to various types of actions. Each type of action can correspond to one or more action templates, which are used to indicate the execution standards of various types of actions. That is to say, the actions executed according to the action template do not need to be beautified. The greater the difference from the action template, the higher the degree of beautification required for the corresponding action.

[0075] The bus 203 , the processor 201 , the memory 202 and the display device 204 are usually connected to each other via the bus 203 , or are connected to each other in other ways.

[0076] The external display device 204 can be used to display an animation video display interface to show the user the animation video after motion beautification processing.

[0077] The processor 201 can be used to compare the motion data of the target object with the action data corresponding to the action template to determine whether each action performed by the target object needs to be beautified. In other words, the processor 201 can be used to identify actions that need to be beautified. The processor 201 can also be used to beautify the actions performed by the target object. In other words, the processor 201 can be used to query the database for the action template that is closest to the action data based on the collected action data of the target object. The processor 201 can also be used to perform data blending on the collected action data of the target object in combination with the action template to generate blended motion data. The processor 201 can also be used to smooth the blended motion data.

[0078] It should be noted that the application scenarios and system architectures described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0079] For ease of understanding, the following is an exemplary introduction to the virtual object action generation method provided by the present application in conjunction with the accompanying drawings. The virtual object action generation method is applicable to the computing device 10 shown in Figures 1 and 3.

[0080] FIG4 shows a flow chart of a method for generating an action of a virtual object provided by an exemplary embodiment of the present application. The method for generating an action of a virtual object can be executed by a computing device and includes the following steps:

[0081] S101, obtaining first action data.

[0082] In an embodiment of the present application, a computing device may obtain first motion data, which may be used to indicate a movement of a reference point when a target object performs a target motion.

[0083] Among them, the process of the target object performing the target action can be realized in the real space, and the first action data can include the position information of the reference point in each video frame when the target object performs the target action in the real space, the rotation angle of the reference point, the motion acceleration, and the force condition of the reference point.

[0084] Exemplarily, the reference point may be any point on the skeleton when performing the target action, and the reference point may include a joint point of the skeleton.

[0085] In one possible implementation, the first motion data is obtained by performing motion capture on the target object; or an action video is obtained; the action video contains a segment of the target object performing the target action, and the first motion data is obtained by performing motion capture on the target object in the action video.

[0086] In other words, the computing device can obtain first motion data from the motion capture device. The first motion data is motion data captured by the motion capture device when the target object performs the target motion. The motion capture device can directly send the captured motion data of the target object to the computing device for subsequent processing.

[0087] In one possible implementation, the movement process of the target object can be recorded in the form of a video, and the computing device obtains the video recording the movement process of the target object, and obtains the motion data of the target object in the video when performing the target action through motion capture technology.

[0088] That is to say, the user uses a camera, optical sensor or inertial sensor to obtain the motion data of the target object. The user can also record the motion video of the target object and send the motion video of the target object to the computing device. The computing device analyzes the motion video and obtains the motion data corresponding to when the target object performs various types of actions.

[0089] The target action may be an action belonging to a certain action category performed by a certain body part of the target object during the movement process.

[0090] For example, if the target object is a person, the body parts of the target object may include limbs, torso, shoulders, neck and head, and each body part includes multiple joints.

[0091] Among them, different body parts can perform different types of actions.

[0092] For example, the types of movements that can be performed by the upper limbs include waving, swinging, punching, elbowing, etc.; the types of movements that can be performed by the lower limbs include walking, running, jumping, squatting, sitting, dancing, etc.; the types of movements that can be performed by the shoulders, neck and head include shaking the head, nodding, etc.; the types of movements that can be performed by the torso include bowing, twisting the waist, etc.

[0093] That is to say, the motion capture device can capture the motion data of each joint point, and through the time-series changes of each motion data, the type of action performed by each joint point within a certain time period can be determined.

[0094] In one possible implementation, if the motion data of each joint point of the target object during the movement process is obtained, since the movement process is carried out in a time sequence, each joint point at each time point has its own corresponding motion data at that time point. The various motion data within the time period for collecting the motion data can be obtained for each joint point. In other words, based on the motion data at each time point, the type of action that the joint point participates in performing within each time period can be determined.

[0095] For example, the key time points can be determined based on the speed changes, phase changes or position conditions in the motion data of the joints, and the actions that the joints participate in can be divided according to the key time points.

[0096] That is to say, according to the position of the sole joint of the foot within the specified time period, and the contact situation of the sole joint of the foot with the ground, combined with the rotation of the knee joint within the specified time period and the speed change of the target object within the specified time period, it can be determined that the type of action performed by the lower limb of the target object within the specified time period is walking.

[0097] In one possible implementation, if a computing device obtains a motion video containing the motion process of a target object, it can determine the key frames in the motion video used to segment the action based on the motion of each joint point of the target object. Each video segment segmented according to the key frames can include the action execution process of at least one action type.

[0098] That is to say, according to the position of the target object's foot joints in the motion video, the contact situation of the foot joints, the rotation of the knee joints and the speed change of the target object, the key frames for the target object to start and end walking can be determined, and the action type of the target object's lower limbs is determined to be a walking video clip based on the key frames.

[0099] In the embodiment of the present application, the reference point is the joint point of the skeleton, and the process of beautifying the movement of the joint point of the skeleton is only for illustration and does not constitute a limitation of the method of the embodiment of the present application.

[0100] S102: Acquire second action data.

[0101] In an embodiment of the present application, the computing device may obtain second motion data, which may be used to indicate the movement of a reference point when the virtual object performs a target motion according to the motion template.

[0102] In one possible implementation, the computing device divides the various actions performed during the movement of the target object according to the acquired first action data according to the time points, and after determining the action type of each action performed by the target object, it can obtain the action template corresponding to the determined action type according to the determined action type.

[0103] In a possible implementation, based on the first action data, the action type of the target action is determined; according to the action type of the target action, an action template of the target action is determined; and the action data of the action template of the target action is obtained as the second action data.

[0104] In one possible case, if each action type corresponds to at least two action templates, at least two action templates corresponding to the action type of the target action are determined according to the action type of the target action. In response to the received selection operation, the action template of the target action can be determined from the at least two action templates.

[0105] The action templates may be pre-stored in a database and used to indicate standard actions of various action types.

[0106] In a possible implementation, the second motion data may be motion data corresponding to an action template under the action category to which the target action belongs.

[0107] That is, if each action type corresponds to at least two action templates, after determining the action type of the target action, the user can select one of the at least two action templates as the action template of the target action and obtain the second action data corresponding to the action template.

[0108] For example, if the target action is a waving action, the action template with the waving action type may include action template 1 and action template 2, wherein the waving amplitude of action template 1 is larger than that of action template 2. The user can select an action template as the target action from action template 1 and action template 2 according to the user's needs. In other words, if the user's need is to beautify the waving action performed by the target object in the direction of a larger amplitude, the user can determine action template 1 as the action template for the target action through a selection operation, and then obtain the second action data corresponding to action template 1; if the user's need is to beautify the waving action performed by the target object in the direction of a smaller amplitude, the user can determine action template 2 as the action template for the target action through a selection operation, and then obtain the second action data corresponding to action template 2. This makes it convenient for the user to beautify the target action to the effect required by the user more accurately through simple operations.

[0109] In a possible implementation, in response to a received selection operation, an action template corresponding to the target action may be determined from at least one action template, and second action data of the action template corresponding to the target action may be acquired.

[0110] That is, after the computing device obtains the first action data, it can display various action templates to the user. The user directly selects the action template of the target action from the various action templates through a selection operation and obtains the second action data corresponding to the action template of the target action.

[0111] For example, if the target action is a squat, after the computing device obtains the first action data generated when the target object performs the squat, the computing device can display the action templates of different action types stored in the template library to the user, and each action type can correspond to an action template with a different action effect. In other words, action templates corresponding to waving with different effects, action templates corresponding to running with different effects, action templates corresponding to squats with different effects, etc. can be displayed. By receiving the user's selection operation for the action template corresponding to the squat effect desired, the computing device determines the action template corresponding to the target action, thereby obtaining the second action data of the action template corresponding to the target action.

[0112] S103: Generate third motion data based on the first motion data and the second motion data.

[0113] In an embodiment of the present application, the computing device may modify the first action data based on the first action data and in combination with the second action data to generate third action data.

[0114] The third motion data may be used to indicate the joint movement after adjusting the target motion in combination with the motion template.

[0115] In a possible implementation, the first motion data and the second motion data are mixed according to a specified weight to obtain mixed motion data, and then the mixed motion data is smoothed to obtain the third motion data.

[0116] That is, by mixing the features of the target action with the features of the corresponding action template, and then performing low-pass filtering on the mixed features to obtain smoothed features, the third action data can be obtained.

[0117] In one possible implementation, a first action feature is extracted based on the first action data, a second action feature is extracted based on the second action data, and the first action feature and the second action feature are mixed according to specified weights to generate third action data.

[0118] That is, by performing time domain-frequency domain conversion processing on the first motion data, the periodic motion features therein are extracted, and by performing mixing processing on the motion features, the third motion features are obtained, thereby obtaining the third motion data.

[0119] For example, FIG5 is a flowchart of an abnormality detection of an action sequence data involved in an embodiment of the present application. As shown in FIG5, a computing device obtains first action data, which may be posture sequence data (S11), performs periodic curve extraction on the first action data through an FFT autoencoder (S12), and then performs frequency domain similarity calculation on the features extracted from the first action data and the features obtained after periodic curve extraction of the action template (S13), performs abnormality detection based on the calculated similarity result (S14), determines whether the target action is too different from the action template based on the size of the obtained similarity result, and determines whether the target action is too different from the action template based on the size of the obtained similarity result. If it is determined that the similarity between the target action and the action template is less than a specified threshold, the target action is determined to be abnormal (S15). If the target action is determined to be abnormal, the target action and the action template can be mixed (S16), and then the mixed action data is filtered to obtain a smooth curve feature, thereby matching the third action data (S17).

[0120] In one possible implementation, based on the first action data, a first animation curve of the target reference point is obtained, where the target reference point may be a reference point for performing the target action, and the first animation curve may be used to indicate the movement of the reference point during the time period when the target object performs the target action. Based on the second action data, a second animation curve of the target reference point is obtained, where the second animation curve may be used to indicate the movement of the reference point during the time period when the virtual object performs the target action according to the action template. According to the spherical blending algorithm, the first animation curve and the second animation curve are blended to obtain a third animation curve; the third animation curve may be used to indicate the movement of the reference point during the time period when the virtual object performs the target action adjusted in combination with the action template.

[0121] In a possible implementation, if the target action is an action to be adjusted, third action data is generated based on the first action data and the second action data.

[0122] The action to be adjusted may be a target action whose similarity is less than a specified threshold after frequency domain similarity calculation is performed between the first action data of the target action and the second action data of the target action.

[0123] For example, Figure 6 is a schematic diagram of an action beautification method used in an embodiment of the present application. As shown in Figure 6, if the target action is walking, and the target object, as captured by the computing device, performs target action 41, with the knee joint in an outward-facing position, target action 42 is obtained by beautifying target action 41 using the aforementioned action blending and action smoothing methods, resulting in an adjusted target action 42 with the knee joint position adjusted inward. The beautified action data can adjust the action in various dimensions, including posture, walking posture, gestures, and artistic movements.

[0124] S104: Generate an animation video according to the third motion data.

[0125] In an embodiment of the present application, a video clip of a virtual object performing a beautified target action can be generated through a third action video. Since the target object can perform multiple actions during the movement process, the target actions that need to be beautified can be multiple actions, so the video clips generated by each target action need to be animated and spliced ​​to generate a complete animation video to show the movement process of the virtualized target object, and the virtualized movement process includes the virtual object performing each beautified target action.

[0126] In a possible implementation, a mixed animation video is generated according to the third motion data; and the animation video is obtained by performing low-pass filtering on the mixed animation video.

[0127] In a possible implementation, skeleton skinning rendering may be performed according to the third action data to generate an animation video.

[0128] In summary, the embodiments of the present application obtain first action data of a target object performing a target action in reality, and second action data of a virtual object performing a target action according to an action template, and modify the first action data in combination with the second action data to obtain third action data adjusted to the action template, so that rendering according to the third action data can generate a beautified animation video. By automatically adjusting the target action according to the action template, the action performed by the virtual object is beautified while ensuring that the style of the action performed by the object is preserved, thereby improving the flexibility of the animation generated after virtualizing the target action performed by the target object.

[0129] Figure 7 shows an architecture diagram of a virtual object motion generation system provided by an exemplary embodiment of the present application. As shown in Figure 7, the virtual object motion generation system includes a motion data processing module 310, a motion beautification algorithm module 320, a motion asset library module 330, and a visualization module 340.

[0130] The motion data processing module 310 may include a motion segmentation submodule 311 and a motion classification submodule 312 .

[0131] The action segmentation submodule 311 is used to divide the target object's motion process into action segments in time sequence based on the speed changes and phase changes of each joint point in the first action data of the target object performing the target action, or the key frames such as the position of the foot joint point touching the ground or the palm joint point hanging down.

[0132] The action classification submodule 312 is used to determine the action type in the action segment based on the velocity information or angular velocity information of the key skeletal joints.

[0133] The motion beautification algorithm module 320 may include a motion search algorithm submodule 321 , a motion mixing submodule 322 , a motion smoothing submodule 323 and a problem segment identification submodule 324 .

[0134] The action search algorithm submodule 321 can be used to search for a set of segments with the closest features in the action asset library according to the acquired action segments of the target action of the target object, and sort the closest action segments. The action segments in the action asset library can be action templates.

[0135] The action mixing submodule 322 may be configured to generate a mixed action segment using a spherical mixing algorithm according to a specified weight based on the action segment of the target action and the action segment corresponding to the action template closest to the action segment of the target action.

[0136] The motion smoothing submodule 323 may be configured to smooth the mixed motion clip based on a low-pass filtering algorithm to generate a smoothed motion clip having the same duration as the mixed motion clip.

[0137] The problem segment identification submodule 324 can be used to determine the difference between the rotation information, angular velocity information and position information of each joint point in the action segment corresponding to the action template based on the acquired action segment of the target action of the target object, and determine whether the difference is too large.

[0138] The action asset library module 330 may include a human skeleton asset library submodule 331 , a standard animation asset library submodule 332 , a stylized animation asset submodule 333 and an open interface.

[0139] The human skeleton asset library submodule 331 includes various human skeleton models such as male, female, old, young, tall, short, fat, and thin.

[0140] The standard animation asset library submodule 332 is a standard animation corresponding to the model in the human skeleton asset library submodule 331 , including animation clips and feature clips.

[0141] The stylized animation asset submodule 333 extracts animation curves and frequency domain features of joints such as roots, feet, wrists, elbows, and knees from the standard animation asset library submodule 332 .

[0142] The visualization module 340 may include a skeleton rendering submodule 341 .

[0143] The skeleton rendering submodule 341 and the skin rendering submodule 342 can be used to perform animation rendering of virtual objects.

[0144] In summary, the embodiments of the present application obtain first action data of a target object performing a target action in reality, and second action data of a virtual object performing a target action according to an action template, and modify the first action data in combination with the second action data to obtain third action data adjusted to the action template, so that rendering according to the third action data can generate a beautified animation video. By automatically adjusting the target action according to the action template, the action performed by the virtual object is beautified while ensuring that the style of the action performed by the object is preserved, thereby improving the flexibility of the animation generated after virtualizing the target action performed by the target object.

[0145] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the method. It can be understood that in order to realize the above functions, the action generation device of the virtual object includes at least one of the hardware structure and software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0146] In the embodiment of the present application, the action generation device of the virtual object can be divided into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0147] For example, FIG8 shows a schematic diagram of the structure of a virtual object motion generation device 500 provided by an exemplary embodiment of the present application. The virtual object motion generation device 500 is applied to a computing device, or the virtual object motion generation device 500 can be a computing device. The virtual object motion generation device 500 includes:

[0148] An acquisition module 510 is configured to acquire first motion data, wherein the first motion data indicates the motion of a reference point when a target object performs a target motion; and acquire second motion data, wherein the second motion data indicates the motion of the reference point when a virtual object performs the target motion according to a motion template; the virtual object is configured to represent the target object in a virtual space.

[0149] The processing module 520 is configured to generate third motion data based on the first motion data and the second motion data; the third motion data is configured to indicate a motion condition of the reference point after adjusting the target motion in combination with the motion template;

[0150] The generating module 530 is configured to generate an animation video according to the third action data; the animation video includes a segment of the virtual object performing the adjusted target action.

[0151] For example, in conjunction with FIG4 , the acquisition module 510 may be used to execute S101 and S102 as shown in FIG4 , the processing module 520 may be used to execute S103 as shown in FIG4 , and the generation module 530 may be used to execute S104 as shown in FIG4 .

[0152] In a possible implementation, the acquisition module 510 is further configured to, in response to a received selection operation, determine an action template corresponding to the target action from at least one action template; and acquire the second action data of the action template corresponding to the target action.

[0153] In one possible implementation, the processing module 520 is further used to extract the first action feature based on the first action data; extract the second action feature based on the second action data; and mix the first action feature and the second action feature according to specified weights to generate the third action data.

[0154] In one possible implementation, the processing module 520 is also used to obtain a first animation curve of a target reference point based on the first action data; the target reference point is a joint used to perform the target action; the first animation curve is used to indicate the movement of the reference point during the time period when the target object performs the target action; based on the second action data, obtain a second animation curve of the target reference point; the second animation curve is used to indicate the movement of the reference point during the time period when the virtual object performs the target action according to the action template; according to the spherical blending algorithm, the first animation curve and the second animation curve are mixed to obtain a third animation curve; the third animation curve is used to indicate the movement of the reference point during the time period when the virtual object performs the target action adjusted in combination with the action template.

[0155] In a possible implementation, the processing module 520 is further configured to, if the target action is an action to be adjusted, generate the third action data based on the first action data and the second action data.

[0156] In a possible implementation, the action to be adjusted is the target action whose similarity is less than a specified threshold after frequency domain similarity calculation is performed between the first action data of the target action and the second action data of the target action.

[0157] In a possible implementation, the generating module 530 is further configured to generate a mixed animation video according to the third motion data;

[0158] The animation video is obtained by performing low-pass filtering on the mixed animation video.

[0159] In a possible implementation, the acquisition module 510 is further configured to acquire the first motion data by performing motion capture on the target object.

[0160] In a possible implementation, the acquisition module 510 is further used to acquire an action video; the action video contains a segment of the target object performing the target action; and the first action data is acquired by performing motion capture on the target object in the action video.

[0161] In one possible implementation, the acquisition module 510 is further used to determine the action type of the target action based on the first action data; determine the action template of the target action according to the action type of the target action; and acquire the action data of the action template of the target action as the second action data.

[0162] In one possible implementation, if each action type corresponds to at least two action templates, the acquisition module 510 is also used to determine at least two action templates corresponding to the action type of the target action according to the action type of the target action; and in response to the received selection operation, determine the action template of the target action from the at least two action templates.

[0163] For the detailed description of the above optional methods, please refer to the above method embodiments, which will not be repeated here. In addition, the explanation of any virtual object motion generation device provided above and the description of the beneficial effects can refer to the above corresponding method embodiments, which will not be repeated here.

[0164] The acquisition module 510, processing module 520, and generation module 530 can all be implemented in software or hardware. For example, the implementation of acquisition module 510 will be described below using acquisition module 510 as an example. Similarly, the implementation of processing module 520 and generation module 530 can refer to the implementation of acquisition module 510.

[0165] As an example of a software functional unit, the acquisition module 510 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the acquisition module 510 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0166] Similarly, the multiple hosts / virtual machines / containers used to run the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0167] As an example of a hardware functional unit, the acquisition module 510 may include at least one computing device, such as a server. Alternatively, the acquisition module 510 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0168] The multiple computing devices included in acquisition module 510 can be distributed in the same region or in different regions. The multiple computing devices included in acquisition module 510 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in acquisition module 510 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0169] It should be noted that, in other embodiments, the acquisition module 510 can be used to execute any step in the method for generating a virtual object's motion, the processing module 520 can be used to execute any step in the method for generating a virtual object's motion, and the generation module 530 can be used to execute any step in the method for generating a virtual object's motion. The steps that the acquisition module 510, the processing module 520, and the generation module 530 are responsible for implementing can be specified as needed. By having the acquisition module 510, the processing module 520, and the generation module 530 respectively implement different steps in the method for generating a virtual object's motion, the full functionality of the device for generating a virtual object's motion can be achieved. This application also provides a computing device 100. As shown in FIG9 , the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other via the bus 102. The computing device 100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 100.

[0170] Bus 102 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG9 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 104 may include a path for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, and communication interface 108).

[0171] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0172] The memory 106 may include volatile memory, such as random access memory (RAM). The processor 104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0173] The memory 106 stores executable program code, which the processor 104 executes to implement the functions of the aforementioned acquisition module 510, processing module 520, and generation module 530, thereby implementing the method for generating motions of a virtual object. In other words, the memory 106 stores instructions for executing the method for generating motions of a virtual object.

[0174] Alternatively, the memory 106 stores executable code, and the processor 104 executes the executable code to respectively implement the functions of the aforementioned virtual object motion generation device, thereby implementing the virtual object motion generation method. In other words, the memory 106 stores instructions for executing the virtual object motion generation method.

[0175] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0176] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0177] As shown in Figure 10, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for executing the method for generating motions of a virtual object.

[0178] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating motions of a virtual object. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the method for generating motions of a virtual object.

[0179] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions, each for executing a portion of the functions of the virtual object motion generation device. In other words, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more of the acquisition module 510, processing module 520, and generation module 530.

[0180] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG11 shows a possible implementation. As shown in FIG11 , two computing devices 100A and 100B are connected via a network. Specifically, the connection to the network is made via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the acquisition module 510. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the processing module 520 and the generation module 530.

[0181] The connection method between the computing device clusters shown in Figure 11 can be that considering that the action generation method of the virtual object provided in this application requires a large amount of storage data and calculation data, it is considered to entrust the functions implemented by the processing module 520 and the generation module 530 to the computing device 100B for execution.

[0182] It should be understood that the functions of the computing device 100A shown in FIG11 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0183] The present application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similar to the connection method of the computing device cluster described in Figures 10 and 11. However, the memory 106 of one or more computing devices 100 in this computing device cluster can store the same instructions for executing the virtual object action generation method.

[0184] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating motions of a virtual object. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the method for generating motions of a virtual object.

[0185] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions for executing partial functions of the virtual object motion generation system. In other words, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more devices in the virtual object motion generation device.

[0186] The present application also provides a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the method for generating an action for a virtual object.

[0187] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the action generation method of the virtual object, or instruct the computing device to execute the action generation method of the virtual object.

[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for generating a motion of a virtual object, characterized in that: The method comprises: Acquire first motion data; the first motion data is used to indicate the movement of the reference point during the target object performing the target motion; Acquire second action data; the second action data is used to indicate the movement of the reference point in the process of the virtual object performing the target action according to the action template; the virtual object is used to represent the target object in the virtual space; generating third motion data based on the first motion data and the second motion data; the third motion data is used to indicate the motion of the reference point after adjusting the target motion in combination with the motion template; An animation video is generated based on the third motion data; the animation video includes a segment of the virtual object performing the adjusted target motion.

2. The method according to claim 1, characterized in that The obtaining of the second action data comprises: In response to the received selection operation, determining an action template corresponding to the target action from at least one action template; The second action data of the action template corresponding to the target action is obtained.

3. The method according to claim 1, characterized in that The obtaining of the second action data comprises: Determining the action type of the target action based on the first action data; Determining an action template corresponding to the target action according to the action type of the target action; The second action data of the action template corresponding to the target action is obtained.

4. The method according to claim 3, characterized in that If each action type corresponds to at least two action templates, determining the action template of the target action according to the action type of the target action includes: According to the action type of the target action, determining at least two action templates corresponding to the action type of the target action; In response to the received selection operation, an action template of the target action is determined from the at least two action templates.

5. The method according to any one of claims 1 to 4, characterized in that: The generating third action data based on the first action data and the second action data includes: extracting the first action feature based on the first action data; extracting the second action feature based on the second action data; The first action feature and the second action feature are mixed according to a specified weight to generate the third action data.

6. The method according to any one of claims 1 to 5, characterized in that: The generating third action data based on the first action data and the second action data includes: Based on the first action data, a first animation curve of a target reference point is obtained; the target reference point is a reference point used to perform the target action; the first animation curve is used to indicate the movement of the reference point during the time period when the target object performs the target action; Based on the second action data, a second animation curve of the target reference point is obtained; the second animation curve is used to indicate the movement of the reference point during the time period when the virtual object performs the target action according to the action template; The first animation curve and the second animation curve are mixed according to a spherical mixing algorithm to obtain a third animation curve; the third animation curve is used to indicate the movement of a reference point during a time period when the virtual object performs the target action adjusted in combination with the action template.

7. The method according to any one of claims 1 to 6, characterized in that: The generating third action data based on the first action data and the second action data includes: If the target action is an action to be adjusted, the third action data is generated based on the first action data and the second action data; the action to be adjusted is the target action whose similarity is less than a specified threshold after frequency domain similarity calculation is performed between the first action data of the target action and the second action data of the target action.

8. The method according to any one of claims 1 to 7, characterized in that: Generating an animation video according to the third action data includes: generating a mixed animation video according to the third motion data; The animation video is obtained by performing low-pass filtering on the mixed animation video.

9. The method according to any one of claims 1 to 8, characterized in that: The obtaining of the first action data comprises: The first motion data is acquired by performing motion capture on the target object.

10. The method according to any one of claims 1 to 9, characterized in that: The obtaining of the first action data comprises: Acquire an action video; the action video includes a segment of the target object performing the target action; The first action data is obtained by performing action capture on the target object in the action video.

11. A virtual object motion generation device, characterized in that: The device comprises: an acquisition module, configured to acquire first action data, wherein the first action data is used to indicate the movement of a reference point during the target object's execution of a target action; and to acquire second action data, wherein the second action data is used to indicate the movement of the reference point during the execution of the target action by the virtual object according to the action template; and the virtual object is used to represent the target object in a virtual space. A processing module, configured to generate third motion data based on the first motion data and the second motion data; the third motion data is used to indicate the motion of the reference point after adjusting the target motion in combination with the motion template; A generation module is used to generate an animation video based on the third action data; the animation video includes a segment of the virtual object performing the adjusted target action.

12. The device according to claim 11, characterized in that The acquisition module is further used to determine, in response to a received selection operation, an action template corresponding to the target action from at least one action template; and acquire the second action data of the action template corresponding to the target action.

13. The device according to claim 11, characterized in that The acquisition module is further used to determine the action type of the target action based on the first action data; determine the action template corresponding to the target action according to the action type of the target action; and acquire the second action data of the action template corresponding to the target action.

14. The device according to claim 13, characterized in that If each action type corresponds to at least two action templates, the acquisition module is further used to determine at least two action templates corresponding to the action type of the target action according to the action type of the target action; In response to the received selection operation, an action template of the target action is determined from the at least two action templates.

15. The device according to any one of claims 11 to 14, characterized in that The processing module is further used to extract the first action feature based on the first action data; extract the second action feature based on the second action data; and mix the first action feature and the second action feature according to a specified weight to generate the third action data.

16. The device according to any one of claims 11 to 15, characterized in that The processing module is also used to obtain a first animation curve of a target reference point based on the first action data; the target reference point is a reference point used to execute the target action; the first animation curve is used to indicate the movement of the reference point during the time period when the target object executes the target action; based on the second action data, obtain a second animation curve of the target reference point; the second animation curve is used to indicate the movement of the reference point during the time period when the virtual object executes the target action according to the action template; according to a spherical blending algorithm, the first animation curve and the second animation curve are blended to obtain a third animation curve; the third animation curve is used to indicate the movement of the reference point during the time period when the virtual object executes the target action adjusted in combination with the action template.

17. The device according to any one of claims 11 to 16, characterized in that The processing module is also used to generate the third action data based on the first action data and the second action data if the target action is an action to be adjusted; the action to be adjusted is the target action whose similarity is less than a specified threshold after frequency domain similarity calculation is performed between the first action data of the target action and the second action data of the target action.

18. The device according to any one of claims 11 to 17, characterized in that The generating module is further used to generate a mixed animation video according to the third action data; and obtain the animation video by performing low-pass filtering on the mixed animation video.

19. The device according to any one of claims 11 to 18, characterized in that The acquisition module is further used to acquire the first motion data by performing motion capture on the target object.

20. The device according to any one of claims 11 to 19, characterized in that The acquisition module is further used to acquire an action video; the action video contains a segment of the target object performing the target action; and the first action data is acquired by performing action capture on the target object in the action video.

21. A computing device cluster, characterized in that: It includes at least one computing device, each computing device includes: a processor and a memory, the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the action generation method of the virtual object as described in any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a motion of a virtual object according to any one of claims 1 to 10.

23. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed by a computing device cluster, the computing device cluster executes the method for generating an action of a virtual object according to any one of claims 1 to 10.

Images