Method, device, equipment, medium and program product for generating action of virtual object

Abstract

The application discloses a virtual object action generation method, device, equipment, medium and program product, and relates to the field of artificial intelligence. The method comprises the following steps: in response to receiving a style conversion instruction for a virtual object, acquiring historical action data of the virtual object; based on the historical action data and a target style, acquiring target weight data between at least two base actions; based on the target weight data, mixing action prediction networks corresponding to the at least two base actions to obtain a target mixed network; and through the target mixed network, analyzing and predicting the historical action data to obtain target action data under the target style. The prediction of the action data is realized through the mixed network, without switching between multiple network models, so that the consumption of resources in the action generation process of the virtual object is reduced, and the generation efficiency of the action data is improved. The embodiment of the application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and vehicle-mounted scenes.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, device, medium, and program product for generating actions of virtual objects. Background Technology

[0002] In applications involving virtual scenes, players can perform different actions by controlling virtual objects in the virtual environment, such as walking, jumping, running, and other repetitive actions. At the same time, the same action can have different action styles in different scenarios, that is, there are also some more stylized action methods, such as actions under different age styles, actions under different virtual props, etc.

[0003] In related technologies, data-driven character style transfer technology is commonly used. The process involves collecting motion data from actors' bodies using sensors to obtain motion data of different action types and different style types. Then, by combining the motion data of different action types and different style types, models corresponding to different style motion data are trained. That is, one model is trained for each style. After the model is trained, it is deployed. Based on the style switching command input by the player, the model of the corresponding style is called to generate new style actions.

[0004] In the process of generating different styles of actions in the above manner, a single model is used to correspond to a single style. This means that when faced with multiple style conversions, after receiving the style switching instruction, it is necessary to switch between multiple models to generate action data. However, the above steps increase the server's operating burden. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and program product for generating actions of virtual objects, which can reduce resource consumption during the generation of virtual object actions. The technical solution is as follows:

[0006] On the one hand, a method for generating actions for virtual objects is provided, the method comprising:

[0007] In response to receiving a style conversion instruction for a virtual object, historical action data of the virtual object is obtained. The style conversion instruction is used to indicate that the action style of the virtual object is switched to the target style, and the historical action data is used to indicate the action sequence of the virtual object in a historical time period.

[0008] Based on the historical action data and the target style, obtain target weight data between at least two base actions, where the base actions are the baseline actions when the virtual object performs actions.

[0009] Based on the target weight data, the action prediction networks corresponding to the at least two base actions are mixed to obtain a target hybrid network, which is used to predict action data.

[0010] The target hybrid network is used to analyze and predict the historical action data to obtain target action data under the target style.

[0011] On the other hand, a device for generating actions of virtual objects is provided, the device comprising:

[0012] The data acquisition module is used to acquire historical action data of the virtual object in response to receiving a style conversion instruction for the virtual object. The style conversion instruction is used to indicate that the action style of the virtual object is switched to the target style. The historical action data is used to indicate the action sequence of the virtual object in a historical period.

[0013] The weight determination module is used to obtain target weight data between at least two base actions based on the historical action data and the target style, wherein the base actions are the baseline actions when the virtual object performs actions.

[0014] A network determination module is used to mix the action prediction networks corresponding to the at least two base actions based on the target weight data to obtain a target hybrid network, wherein the action prediction network is used to predict action data;

[0015] The result prediction module is used to analyze and predict the historical action data through the target hybrid network to obtain target action data under the target style.

[0016] On the other hand, a computer device is provided, wherein the terminal includes a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, wherein the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the virtual object action generation method described in any of the embodiments of this application.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the program code being loaded and executed by a processor to implement the virtual object action generation method described in any of the embodiments of this application.

[0018] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the virtual object action generation method described in any of the above embodiments.

[0019] The technical solution provided in this application includes at least the following beneficial effects:

[0020] When switching the motion style of a virtual object, the weight data corresponding to the target style is determined based on the virtual object's historical motion data. This weight data is then used to obtain a hybrid network, which is a mixture of motion prediction networks corresponding to base actions. This hybrid network predicts the motion data under the target style, thereby controlling the virtual object to complete the motion style switching process. In this implementation, motion data prediction is achieved through a hybrid network, eliminating the need to switch between multiple network models. This reduces resource consumption during the motion generation process and improves the efficiency of motion data generation. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of the data-driven role style conversion technology solution in related technologies;

[0023] Figure 2 This is a structural block diagram of a computer system provided in an exemplary embodiment of this application;

[0024] Figure 3 This is a flowchart of a virtual object action generation method provided in an exemplary embodiment of this application;

[0025] Figure 4 This is a schematic diagram of action transition provided in an exemplary embodiment of this application;

[0026] Figure 5 This is a flowchart of a method for generating actions of virtual objects provided in another exemplary embodiment of this application;

[0027] Figure 6 This is a schematic diagram of footstep sequence annotation provided in an exemplary embodiment of this application;

[0028] Figure 7 This is a schematic diagram of an action phase sequence provided in an exemplary embodiment of this application;

[0029] Figure 8 This is a schematic diagram illustrating the generation process of a target hybrid network provided in an exemplary embodiment of this application;

[0030] Figure 9 This is a schematic diagram of an action execution animation provided in an exemplary embodiment of this application;

[0031] Figure 10 This is a flowchart of a method for generating actions of virtual objects provided in another exemplary embodiment of this application;

[0032] Figure 11 This is a schematic diagram illustrating the generation process of a target hybrid network provided in an exemplary embodiment of this application;

[0033] Figure 12 This is a structural block diagram of a virtual object action generation device provided in an exemplary embodiment of this application;

[0034] Figure 13 This is a structural block diagram of a virtual object action generation device provided in another exemplary embodiment of this application;

[0035] Figure 14 This is a schematic diagram of the structure of a server provided in an exemplary embodiment of this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0037] First, a brief introduction to the terms used in the embodiments of this application:

[0038] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0039] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0040] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0041] Virtual environment: This is a virtual environment displayed (or provided) by an application when it runs on a terminal. This virtual environment can be a simulation of the real world, a semi-simulated / semi-fictional three-dimensional environment, or a purely fictional three-dimensional environment. There is no limitation on whether a virtual environment can be a two-dimensional, 2.5-dimensional, or three-dimensional virtual environment.

[0042] Virtual objects refer to movable objects in a virtual environment. These movable objects can be virtual characters, virtual animals, anime characters, etc., such as people, animals, plants, oil drums, walls, and stones displayed in a 3D virtual environment. Optionally, virtual objects are 3D models created based on animation skeletal technology. Each virtual object has its own shape and volume in the 3D virtual environment, occupying a portion of the space within the 3D virtual environment. Optionally, virtual objects can also be 2D or 2.5D virtual objects; the above example is only illustrative and is not intended to limit their application.

[0043] In this embodiment, players can control virtual objects in a virtual environment to perform actions such as walking, running, jumping, shooting, fighting, driving, picking up items, and using skills. The actions that virtual objects can perform are divided into two categories: basic cyclical actions, such as walking, running, and jumping; and more stylized actions, such as actions in different postures, specifically, postures like holding virtual items, bending over, and crawling. Different types of virtual objects correspond to different actions, specifically, virtual objects of different ages, genders, and factions. Different styles can be combined with basic actions to achieve different action effects; for example, the effect of running empty-handed is different from the effect of running while holding virtual items.

[0044] As an illustration, the method provided in this application embodiment can be applied to game application scenarios, as well as various other scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle scenarios. For example, the method provided in this application embodiment can provide virtual scenes in a smart transportation system to simulate the action execution process of virtual pedestrians and virtual vehicles, without specific limitations.

[0045] In related technologies, since online control of game characters to switch between different styles of actions is highly demanded in game design, neural network learning models are introduced to drive the switching of character action styles, including data-driven character style switching technology.

[0046] like Figure 1 As shown, it illustrates a flowchart of a data-driven character style conversion technology solution in related technologies. By acquiring first action data 110 of different action types and second action data 120 of different style types, a model 130 corresponding to each style type is trained using the first action data 110 and the second action data 120. The model 130 is then applied to the switching process 140 of different style actions of virtual objects in a game application.

[0047] However, the above technical solution has the following problems in the implementation process: training a single model for a single style means that when facing multiple style conversions, it is necessary to first train neural network models for all styles in advance, and then switch between multiple models after receiving the player's style switching instruction, which increases the content and running burden.

[0048] In this embodiment, the motion generation process for virtual objects is achieved using machine learning in artificial intelligence. Upon receiving a style conversion instruction for a virtual object, historical motion data of the virtual object is acquired. Based on this historical motion data and the target style to be converted, a weight relationship between base motions is obtained. This weight relationship is then mixed with the motion prediction network corresponding to the base motions to obtain a hybrid network. The hybrid network then predicts the motion data corresponding to the target style. The motion generation method for virtual objects provided in this embodiment improves the efficiency of motion generation while reducing the waste of device resources.

[0049] Based on the above definitions, the implementation environment of the embodiments of this application will be described. Figure 2 A structural block diagram of a computer system provided in an exemplary embodiment of this application is shown. The computer system 200 includes: a terminal device 210, a server 220, and a communication network 230.

[0050] Terminal device 210 has an application installed and running that supports a virtual environment. This application can be any of the following: virtual reality application, first-person shooter (FPS) game, third-person shooter (TPS) game, multiplayer online battle arena (MOBA) game, or massively multiplayer online role-playing game (MMORPG). The user controls a main virtual object located in the virtual environment through terminal device 210. Terminal device 210 can take many forms, including mobile phones, tablets, desktop computers, laptops, smart home appliances, vehicle terminals, and aircraft.

[0051] Server 220 provides backend services for applications supporting virtual environments. Illustratively, server 220 provides backend logic for the application based on application requests sent by terminal device 210, while terminal device 210 provides frontend logic for the application. Optionally, server 220 performs the primary computational work, and terminal device 210 performs secondary computational work; or, server 220 performs secondary computational work, and terminal device 210 performs primary computational work; or, server 220 and terminal device 210 collaborate using a distributed computing architecture. In some embodiments, when the application is a single-player game, the game logic is entirely handled by terminal device 210.

[0052] It is worth noting that the aforementioned server 220 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. In some embodiments, the aforementioned server 220 can also be implemented as a node in a blockchain system. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0053] In this embodiment, after receiving a style conversion instruction for a virtual object, the terminal device 210 transmits the style conversion instruction to the server 220. The server 220 processes the style conversion instruction by calling the corresponding action data generation module to obtain target action data, and transmits the target action data to the terminal device 210. The terminal device 210 controls the virtual object in the virtual scene to perform the corresponding action according to the target action data.

[0054] In some embodiments, when the computing resources of the terminal device 210 meet the operational requirements of the above-mentioned action data generation module, the process of generating the above-mentioned action data can also be implemented in the terminal device 210 on a standalone basis. No specific limitations are made on the device used to implement the solution.

[0055] Indicatively, terminal device 210 and server 220 are connected via communication network 230.

[0056] Please refer to Figure 3 This illustrates a method for generating actions of virtual objects according to an embodiment of this application. In this embodiment, the method is applied to, for example... Figure 2 The method is illustrated using the server shown as an example. It can be applied to terminal devices, and is not specifically limited here. The method includes:

[0057] Step 301: In response to receiving a style conversion instruction for a virtual object, obtain the historical action data of the virtual object.

[0058] Schematic illustration: The style conversion instruction described above is used to instruct the switching of the virtual object's action style to the target style. In some embodiments, the style conversion instruction is determined by a style conversion operation received by the target application, which is capable of providing virtual environment operation functions. Schematic illustration: The terminal device displays a virtual environment interface through the target application, which displays virtual objects. The operation received by the terminal device can realize the control of the virtual objects in the virtual environment.

[0059] The aforementioned control of virtual objects includes controlling virtual objects to perform specified actions. In this embodiment, the actions that virtual objects can perform include actions of different style types. The style type can be determined based on the type of virtual object, the state of the virtual object, or the style setting operation input by the player.

[0060] In this embodiment, the style of the actions performed by a virtual object can be changed by triggering a style conversion command. Optionally, the virtual object can be a master virtual object controlled by a player, or an intelligent virtual object controlled by AI in the virtual environment.

[0061] Optionally, the aforementioned style conversion instruction can be implemented through a virtual object type switching operation. That is, when the terminal device receives a type switching operation to switch a virtual object from a first type to a second type, the terminal device instructs the server to perform a style conversion, where the action style of the virtual object of the second type is the target style. For example, to switch a virtual object from a child to a teenager, the terminal device sends an instruction to the server to switch the virtual object's action style to the teenager style; similarly, to switch a virtual object from a mortal to a demon, the terminal device sends an instruction to the server to switch the virtual object's action style to the demon style. It is worth noting that virtual object type switching can also take other forms; the above examples are used here for illustration only and do not limit the specific implementation.

[0062] Optionally, the aforementioned style conversion instruction can also be implemented through a virtual object state switching operation. That is, when the terminal device receives a state switching operation to switch a virtual object from a first state to a second state, the terminal device instructs the server to perform a style conversion, where the action style of the virtual object in the second state is the target style. For example, to control a virtual object to switch from a standing state to a crouching state, the terminal device sends an instruction to the server to switch the virtual object's action style to the crouching style; or, to control a virtual object to switch from an empty-handed state to a holding-object state, the terminal device sends an instruction to the server to switch the virtual object's action style to the holding-object style. It is worth noting that the virtual object's state switching can also take other forms; the above examples are used here for illustration only and are not intended to limit its specific implementation.

[0063] Optionally, the aforementioned trigger style conversion instruction can also be implemented through a virtual object's action style switching operation. That is, when the terminal device receives a request to switch the virtual object's action style from the first style to the target style, the terminal device instructs the server to perform a style conversion. For example, the virtual environment interface provides a style switching control. When this style switching control receives a trigger operation, the terminal device instructs the server to perform the corresponding style conversion instruction based on the target style indicated by the trigger operation. It is worth noting that the trigger operation corresponding to the target style can also be determined by shortcut key input through the terminal device's input device. This is only an illustrative example using a control and does not limit the specific triggering method.

[0064] Optionally, the aforementioned style conversion command can also be implemented by judging the style conversion conditions. That is, the server automatically controls the switching of the virtual object's action style based on the display status of the virtual object in the terminal device. For example, in response to determining that the display status of the virtual object meets the style conversion conditions, the terminal device automatically sends a style conversion command to the server. Optionally, the aforementioned style conversion condition can be that the duration of the virtual object's action display in the first style reaches a target duration; the aforementioned style conversion condition can also be that when the terminal device displays the intelligent virtual object in the virtual environment, and it is determined that the main virtual object controlled by the terminal device interacts with the intelligent virtual object, the action style of the intelligent virtual object is converted.

[0065] Optionally, the style conversion instruction can be triggered by a touch gesture. That is, the terminal device determines the corresponding touch gesture based on the touch signal received by the sensor, and in response to determining that the target touch gesture has been received, sends a style conversion instruction to the server. The aforementioned sensor can be a pressure sensor, photoelectric sensor, ultrasonic sensor, speed sensor, or other sensor capable of detecting gestures. The target touch gesture can be at least one or more combinations of gestures such as click, double-click, long press, pressure, and swipe, without specific limitations.

[0066] Schematic illustration: The aforementioned historical action data is used to indicate the action sequence of a virtual object within a historical time period. In some embodiments, the aforementioned historical time period is the time period prior to the moment the style switching command is received, or the aforementioned historical time period may also include the moment the style switching command is received and the time period prior to that moment. Optionally, the aforementioned historical time period may be a preset duration or a time period determined according to specified conditions. For example, when the aforementioned virtual object is in a virtual game, the aforementioned historical time period may be the time period between the start time of the virtual game and the moment the style switching command is received, or the aforementioned historical time period may be the time period between the moment the previous style switching command was received and the moment the current style switching command is received. Schematic illustration: The specific time period corresponding to the aforementioned historical time period can be truncated according to actual needs, and no specific limitations are imposed here.

[0067] In some embodiments, the aforementioned historical action data can be obtained by recording network data packets during the process of the target application controlling the action of virtual objects. That is, when the server provides background support for the target application, the target application in the terminal device will generate network data packets from the action data of virtual objects in the virtual environment and transmit the network data packets to the server. The server will record the network data packets in the form of logs and extract the action data within the corresponding historical period as the aforementioned historical action data when needed.

[0068] In other embodiments, the aforementioned historical action data can also be obtained through data logic frames in the runtime engine corresponding to the virtual environment. The runtime engine is a module in the target application used to generate the interface display content corresponding to the virtual environment. This runtime engine provides functions such as modeling, layout, animation generation, lighting, visual effects, rendering, and compositing. The runtime engine renders the virtual environment to obtain the virtual environment image; similarly, the action images corresponding to the virtual objects are also rendered by the runtime engine. Optionally, the runtime engine includes the Unity3D engine, Unreal Engine, etc. Illustratively, during the execution of actions by the virtual object, the terminal device obtains and stores the corresponding data logic frames from the runtime engine. When a style conversion instruction is received, the data logic frames within the corresponding historical time period are uploaded to the server.

[0069] Step 302: Obtain target weight data between at least two base actions based on historical action data and target style.

[0070] In this embodiment, the actions performed by the virtual object are divided into base actions and style actions. Base actions are the baseline actions for the virtual object when performing actions. In some embodiments, base actions indicate the basic actions that the virtual object can perform, such as walking, running, jumping, standing, and attacking. Base actions are used to fuse with action styles to obtain action data. Based on the type, state, or received style settings of the virtual object, the actions exhibited by the virtual object can be displayed as different style actions. For example, based on the type of the virtual object, it can be controlled to display actions such as elderly style, child style, human style, and demon style; or based on the state of the virtual object, it can be controlled to display actions such as standing, crouching, or holding an object.

[0071] Indicatively, different basic movements can exhibit different display effects under different movement styles. For example, for the same basic movement "walking", the style movement shown in the child style is in the form of hopping and jumping, while the style movement shown in the elderly style is in the form of staggering.

[0072] In some embodiments, upon receiving a style conversion instruction from a terminal device, the target style corresponding to the action data to be generated is determined through the style conversion instruction. At the same time, historical action data corresponding to the virtual object is obtained. The historical action data and the aforementioned style conversion instruction are input to the action generation module. After processing by the action generation module, the target action data corresponding to the target style is obtained, and the target style action is returned to the terminal device.

[0073] Indicative, such as Figure 4As shown, it illustrates an action conversion diagram provided by an exemplary embodiment of this application. In this diagram, before receiving a style conversion instruction, the virtual object in the terminal device is displayed with action data 410 in style A. After receiving a style conversion instruction to convert the action style from style A to style B, the action data 410 in style A and the style conversion instruction 420 are input to the action generation module 401. The action generation module 401 predicts and obtains action data 430 in style B based on the action data 410 in style A and the style conversion instruction 420.

[0074] In some embodiments, the action generation module includes a Mix-of-Expert (MoE) model, which is used to predict actions. The Mix-of-Expert model is a model composed of multiple expert networks and a gating network. Each expert network is a network corresponding to a base action, and the gating network is a network used to generate weight data between expert networks.

[0075] In a schematic way, the aforementioned historical action data and the target style indicated by the style transfer instruction are input into the gate network, and the gate network predicts the target weight data corresponding to each base action.

[0076] Step 303: Based on the target weight data, the action prediction networks corresponding to at least two base actions are mixed to obtain the target hybrid network.

[0077] Schematic, the aforementioned action prediction network is used to predict action data. In this embodiment, the process of mixing the action prediction network based on target weight data indicates merging at least two action prediction networks according to the target weight data to obtain a hybrid network for action data prediction. The mixing process is used to merge action prediction networks corresponding to different base actions, thereby obtaining a hybrid network capable of generating action data for a target style. Schematic, the aforementioned merging process of multiple action prediction networks can be achieved by assigning weights to at least two action prediction networks according to target weight data, and then merging the weighted action prediction networks using a preset addition method to obtain a hybrid network. Optionally, the preset addition method can indicate that the output results of the action prediction networks are weighted and added, and the final weighted calculation result is used as the output of the hybrid network; or it can indicate that the network parameters of the action prediction networks are added, and the network parameters after weighted addition are used as the network parameters of the hybrid network. The aforementioned action prediction network can be an expert network in a hybrid expert network model, that is, after determining the target weight data corresponding to each base action, the expert networks corresponding to the base actions are mixed using the target weight data to obtain a target hybrid network.

[0078] In some embodiments, the action prediction network is weighted by target weight data to obtain the target hybrid network composed of action prediction networks with different weights.

[0079] In some embodiments, the network structure of the action prediction network corresponding to different base actions can be the same or different. Optionally, the action prediction network can be a network with a structure such as a feedforward neural network, a sparse autoencoder (SAE), or a deep residual network (DRN).

[0080] Optionally, the target weight data can be data that assigns weights to the output of the action prediction network, or data that assigns weights to the network parameters in the action prediction network.

[0081] In a schematic representation, when the target weight data is used to weight the output of the action prediction network, the output layers of each action prediction network in the target hybrid network are connected to the output layer of the hybrid network. The output layer of the hybrid network uses the target weight data to perform a weighted summation of the outputs of the action prediction network's output layer, thereby obtaining the overall network output.

[0082] To illustrate, when the target weight data is used to assign weights to the network parameters of the action prediction network, in the process of mixing the various action prediction networks using the target weight data, the action prediction network as a whole can be weighted to obtain the target mixed network, or the individual network layers in the action prediction network can be weighted to obtain the target mixed network.

[0083] Step 304: Analyze and predict historical action data using a target hybrid network to obtain target action data under the target style.

[0084] To illustrate, the aforementioned historical motion data is input into the target mixing network, and the output is the target motion data corresponding to the target style.

[0085] In some embodiments, the data input to the target hybrid network may be all action data within the aforementioned historical period, or action data at a specified time node within the historical period, i.e., the action data corresponding to the first moment in the historical period is input to the target hybrid network.

[0086] In some embodiments, the first moment mentioned above is the final moment in a historical period, that is, the historical period is the time interval from the start of the history to the first moment. In other embodiments, the first moment mentioned above may also be the moment when the style transfer instruction is received. Optionally, the first moment may be a single point in time or a combination of multiple points in time, and this is not limited here.

[0087] To illustrate, when the target weight data is used to assign weights to the output of the action prediction network, the action data input to the target hybrid network is input to different action prediction networks, and the corresponding result data is output. The result data is then input to the output layer of the hybrid network, where it is weighted and summed using the target weight data, and finally the aforementioned target action data is output.

[0088] To illustrate, when the target weight data is used to assign weights to the network parameters of the action prediction network, the action data is input into the target hybrid network, where the neurons process the data and finally output the aforementioned target action data.

[0089] In some embodiments, the target action data may be action data corresponding to a second moment or action data corresponding to a target time period, wherein the second moment is the next moment after the first moment and the target time period is the next time period after the historical time period.

[0090] After outputting the target action data corresponding to the target style, the target action data is sent to the terminal device. The terminal device controls the virtual objects in the virtual environment to perform the corresponding action animation based on the target action data.

[0091] In summary, the virtual object action generation method provided in this application, when it is necessary to switch the action style of a virtual object, determines the weight data corresponding to the target style based on the historical action data of the virtual object. Using this weight data, a hybrid network is obtained by mixing action prediction networks corresponding to base actions. This hybrid network then predicts the action data under the target style, thereby controlling the virtual object to complete the action style switching process. In this implementation, the prediction of action data is achieved through a hybrid network, eliminating the need for switching between multiple network models, reducing resource consumption during the virtual object action generation process, and thus improving the efficiency of action data generation.

[0092] Please refer to Figure 5 This document illustrates a method for generating actions of virtual objects according to an embodiment of this application. In this embodiment, the generation of action data is achieved through a hybrid expert network model. The method includes:

[0093] Step 501: In response to receiving a style conversion instruction for a virtual object, obtain the motion phase sequence and gait trajectory sequence of the virtual object.

[0094] In this embodiment of the application, the historical action data includes an action phase sequence and a gait trajectory sequence. The action phase sequence is used to indicate the contact between the virtual object and the virtual plane when the virtual object performs an action during the historical period, and the gait trajectory sequence is used to indicate the action speed of the virtual object during the action performance during the historical period.

[0095] In some embodiments, the aforementioned action phase sequence and gait trajectory sequence are obtained by data annotation based on action sequences within a historical time period. Illustratively, the action sequence of a virtual object within a historical time period is obtained; the action sequence is annotated based on the contact between the virtual object and the virtual plane when performing actions, resulting in contact action frames; based on the temporal sequence between contact action frames, an action phase sequence is generated; based on the action type of the virtual object when performing actions, the action sequence is annotated to obtain action type data; based on the action type data and the corresponding action speed information of the action sequence, a gait trajectory sequence is generated. The historical action data is composed of the action phase sequence and the gait trajectory sequence.

[0096] To illustrate, taking a virtual character as an example, this virtual character includes a left foot and a right foot. During the generation of the motion phase sequence, the corresponding motion phase can be generated by analyzing the contact between the left and right feet and the virtual plane. Specifically, first, the footstep sequences corresponding to the left and right feet are obtained, and then the motion frames in the footstep sequences where the left or right foot is in contact with the virtual plane are labeled.

[0097] In one example, such as Figure 6 As shown, it illustrates a footstep sequence annotation diagram provided by an exemplary embodiment of this application. In the right foot footstep sequence 610, the contact action frame 611 in which the right foot contacts the virtual plane is annotated, and in the left foot footstep sequence 620, the contact action frame 621 in which the left foot contacts the virtual plane is annotated. The horizontal axis in the right foot footstep sequence 610 and the left foot footstep sequence 620 indicates the number of action frames, and the vertical axis indicates the movement speed of the left or right foot.

[0098] Since basic actions like standing and walking are actually periodic—that is, a cycle of left foot-right foot-left foot landing—the phase variable can represent the current stage of the action cycle. After annotating the contact action frames in the footstep sequence, the corresponding action phase sequence can be generated. Specifically, a cycle is defined as left foot landing-right foot landing-left foot landing, with the phase ranging from 0 to 1. In one example, such as... Figure 7As shown, it illustrates a schematic diagram of an action phase sequence 700 provided in an exemplary embodiment of this application. In the action phase sequence 700, the horizontal axis indicates the number of action frames, and the vertical axis indicates the phase of the action. A phase of 0 represents a standing state, and during walking, the phase changes periodically from 0 to 1.

[0099] In some embodiments, the gait trajectory sequence is determined by the action type and speed information of the virtual object during the execution of the action. Illustratively, the action type indicates the basic action during the execution of the action, such as running, jumping, walking, standing, etc. The speed information indicates the speed changes of the virtual object during the execution of the action.

[0100] In some embodiments, the gait trajectory sequence includes at least one of skeletal joint information, motion trajectory information, and target point information. The skeletal joint information indicates the motion information corresponding to the skeletal joints of the virtual object during the execution of an action, such as the position, velocity, and direction information of the skeletal joints. The motion trajectory information indicates the trajectory information of the virtual object in the virtual environment during the execution of an action, such as the position, direction, action type, and terrain information of the virtual object in the virtual environment. The target point information indicates the information of a reference point corresponding to the virtual object before and after the execution of the action, such as the position, direction, and action type of the reference point. The reference point can be the center of gravity of the virtual object or other points in the virtual environment that can form a reference, and is not limited thereto.

[0101] In some embodiments, since historical action data is generated based on the time series of virtual object positions during acquisition, and the positions are absolute, it needs to be converted into an input format that the model can understand, for example, converting absolute positions into relative positions.

[0102] This example illustrates how historical motion data is preprocessed to obtain preprocessed motion data that can be input into the model. Specifically, the absolute position changes of virtual objects in the historical motion data are converted into relative position changes, resulting in preprocessed motion data. This preprocessed motion data is used in the process of determining target weight data. In one example, the preprocessed motion data includes a preprocessed gait trajectory sequence and a motion phase sequence. The preprocessed gait trajectory sequence includes at least one of preprocessed skeletal joint information, motion trajectory information, and target point information. Specifically, the preprocessed skeletal joint information includes relative position information, relative velocity information, and relative direction information of the skeletal joints; the preprocessed motion trajectory information includes the relative position information, relative direction information, motion type, and terrain information of the virtual object in the virtual environment; and the preprocessed target point information includes the relative position information, relative direction information, and motion type of the reference point.

[0103] Step 502: Obtain hybrid weight data based on the product of the phase elements in the action phase sequence and the trajectory elements in the gait trajectory sequence.

[0104] In this embodiment, hybrid weighted data is generated based on the action phase sequence and the gait trajectory sequence. Optionally, the data format of the action phase sequence and the gait trajectory sequence can be at least one of matrices, arrays, vectors, etc., and is not limited herein. In one example, when the data format of the sequence is a matrix, the phase elements in the action phase sequence are matrix elements, and the trajectory elements in the gait trajectory sequence are matrix elements.

[0105] In this embodiment of the application, the process of generating action data is completed by a hybrid expert network model, wherein the hybrid expert model includes a gate network, which is used to generate the weight data corresponding to the expert network.

[0106] In some embodiments, the gate network obtains the aforementioned mixed weight data by calculating the Kronecker Product between phase elements and trajectory elements, as shown in Equation 1, where p t The phase element at time t, γ t The trajectory element representing the virtual object at time t. For a gate network, α represents the generated mixed weight vector.

[0107] Formula 1:

[0108] Step 503: Modulate the mixed weight data based on the target style to obtain the target weight data.

[0109] Since the above-mentioned mixed weight data is generated only based on historical action data, it needs to be modulated to correspond to the target style to be converted, so as to obtain target weight data corresponding to the target style.

[0110] In some embodiments, the mixed weight data is modulated by calling the target modulation function corresponding to the target style to obtain the target weight data. Illustratively, the target modulation function corresponding to the target style is obtained, and the mixed weight data is subjected to an affine transformation based on the target modulation function to obtain the target weight data.

[0111] In some embodiments, the target modulation function can be generated by a preset neural network. That is, after obtaining the mixed weight data, the target style is input into the preset neural network, and the target modulation function is output. In other embodiments, the modulation functions corresponding to different action styles can also be pre-generated and stored in a database. That is, the modulation functions and the mapping relationship between action styles and modulation functions are stored in the database. When it is determined that action data corresponding to the target style needs to be generated, the target modulation function is determined according to the above mapping relationship and read from the database.

[0112] In one example, to ensure the stability of the overall model and reduce the introduction of redundant parameters, the target modulation function is implemented using one-hot encoding of the target style, as shown in Equation 2. Here, σ and μ are the neural networks used to implement the one-hot encoding process, σ(s) and μ(s) are the one-hot encodings of style s, and α... N This represents the Nth weight value in the weighted data, where N is a positive integer.

[0113] Formula 2: r(α) N ,s)=σ(s)*α N +μ(s)

[0114] When it is necessary to generate target weight data corresponding to the target style, the mixed weight data α and the target style s′ are input into the target modulation function r to obtain the target weight data α′, as shown in Formula 3.

[0115] Formula 3: α′=r(α,s′)

[0116] Schematic, the target style mentioned above is input into the target modulation function in the form of a preset status code, that is, different action styles correspond to different preset status codes. In some embodiments, the preset status code of the action style may be carried as a style identifier in the style conversion instruction, or it may be read from a preset mapping table according to the style identifier indicated in the style conversion instruction, which is not limited here.

[0117] Step 504: Based on the target weight data, perform a weighted summation of the first network parameters corresponding to the N action prediction networks, and set the resulting second network parameters as the network parameters of the target hybrid network.

[0118] In the embodiments of this application, the target hybrid network corresponds to N action prediction networks, where N≥2 and N is an integer, and the N action prediction networks correspond one-to-one with N base actions.

[0119] Schematic, the network structures of the N action prediction networks are identical. Therefore, during the mixing process, the network parameters of the target hybrid network can be obtained by weighted summation of the first network parameters of the action prediction networks. In some embodiments, each network layer in the action prediction network can be mixed according to the weight values ​​indicated by the target weight data to generate the target hybrid network.

[0120] like Figure 8 As shown, this illustrates the generation process of a target hybrid network provided in an exemplary embodiment of this application. Taking a hybrid expert network model as an example, the action phase sequence 801 and gait trajectory sequence 802 are input into a gate network 810, and the output is hybrid weight data (α1, α2, ..., α...). N )803, the mixed weight data 803 and the target style 804 are input into the target modulation function 820, and the target weight data (ω1,ω2,…,ω) is output. N )805, the target weight data 805 is mixed with the corresponding expert network 830 to obtain the target hybrid network 840.

[0121] Step 505: Input the action data at the first moment into the target hybrid network to predict the target action data at the second moment.

[0122] Indicatively, the historical motion data includes motion data at the first moment, where the first moment is the final moment in the historical time period and the second moment is the moment following the first moment. That is, the motion data at the next moment is predicted by using the motion data at the final moment in the historical time period, thereby converting the motion style of the virtual object from the first style to the target style.

[0123] In a schematic representation, the motion data at the first moment is input into the target hybrid network, and the output is the motion data at the second moment. In one example, the function for generating the motion data is shown in Equation 4, where s represents the motion style of the virtual object at the first moment t, i.e., the first style, and s′ represents the motion style of the virtual object at the second moment, i.e., the target style. r represents the skeletal joint information under the first style at time t. tThe first type of information in the trajectory information and target point information at the first time t (including the relative position and direction information of the virtual object indicated by the trajectory information in the virtual environment, the terrain information of the virtual environment, and the relative position and direction information of the reference point indicated by the target point information), γ t The second type of information (action type) represents the trajectory information and target point information at the first time t, and α′ represents the target weight data. Represents a target hybrid network. r t+1 , γ t+1 These are the output data corresponding to the input data predicted at the second time t+1.

[0124] Formula 4:

[0125] In some embodiments, the target hybrid network can also predict the phase change based on the input action phase sequence. The phase value at the second time step can be determined based on the phase value at the first moment in the action phase sequence and the predicted phase change, as shown in Formula 5. The phase value p at the second moment... t+1 The phase value p corresponding to the first moment t And the predicted phase change Δp is calculated.

[0126] Formula 5: p t+1 =p t +Δp

[0127] In some embodiments, the output target motion data is predicted from the preprocessed relative position data. Therefore, the target motion data is also relative position data. Thus, the output target motion data can also be restored, that is, the relative position data indicated by the target motion data is converted into absolute position data, so as to facilitate the display of the motion execution animation corresponding to the virtual object in the virtual environment.

[0128] Indicatively, the method is implemented entirely by a hybrid network model, which includes gate network units for generating weight data, modulation function units for modulating weight data, expert network units for mixing weight data with expert networks to obtain a hybrid network, and hybrid network units for prediction based on action data at the first moment.

[0129] In some embodiments, training the hybrid network model includes: inputting sample data into the model to be trained, obtaining the loss value of the overall model based on the difference between the output prediction result and the standard result corresponding to the sample data, and iteratively training the model parameters of the overall model based on the loss value until the model converges.

[0130] In other embodiments, before the above-mentioned hybrid network model is trained as a whole, the expert network in the expert network unit can be pre-trained. That is, the network parameters of the expert network are trained by using sample data corresponding to different base actions, the trained expert network is added to the overall model for training, and the network parameters of the expert network are retrained until the overall model converges.

[0131] The server sends the generated target action data to the terminal device, which then displays the animation of virtual objects in the virtual environment performing actions in the target style. In one example, such as... Figure 9 As shown, it illustrates an action execution animation diagram provided by an exemplary embodiment of this application. In time period one 910, the virtual object 901 performs an action of style A. When a style conversion instruction is received, the virtual object 901 switches to performing an action of style B in time period two 920. When a style conversion instruction is received, the virtual object 901 switches to performing an action of style C in time period three 930.

[0132] In summary, the virtual object action generation method provided in this application, when it is necessary to switch the action style of a virtual object, determines the weight data corresponding to the target style based on the historical action data of the virtual object. Using this weight data, a hybrid network is obtained by mixing action prediction networks corresponding to base actions. This hybrid network then predicts the action data under the target style, thereby controlling the virtual object to complete the action style switching process. In this implementation, the prediction of action data is achieved through a hybrid network, eliminating the need for switching between multiple network models, reducing resource consumption during the virtual object action generation process, and thus improving the efficiency of action data generation.

[0133] In this embodiment, mixed weight data is generated by historical action data, and the modulation function corresponding to the target style is determined by hot-unique encoding of the target style. The mixed weight data is then modulated according to the modulation function to obtain target weight data that can adapt to the target style, avoiding the introduction of redundant parameters and thus improving the stability of the overall network.

[0134] Please refer to Figure 10 This document illustrates an embodiment of a method for generating actions for a virtual object, where a feature extraction network is introduced to enable the overall model to be implemented in an end-to-end manner. The method includes:

[0135] Step 1001: In response to receiving a style conversion instruction for a virtual object, obtain the gait trajectory sequence of the virtual object.

[0136] In illustrative terms, the above gait trajectory sequence is used to indicate characteristic information of the actions performed by the virtual object during a historical period.

[0137] In some embodiments, the gait trajectory sequence includes at least one of skeletal joint information, motion trajectory information, and target point information. The skeletal joint information indicates the motion information corresponding to the skeletal joints of the virtual object during the execution of an action, such as the position, velocity, and direction information of the skeletal joints. The motion trajectory information indicates the trajectory information of the virtual object in the virtual environment during the execution of an action, such as the position, direction, action type, and terrain information of the virtual object in the virtual environment. The target point information indicates the information of the reference point corresponding to the virtual object before and after the execution of the action, such as the position, direction, and action type of the reference point.

[0138] In other embodiments, the gait trajectory sequence can be motion image frames of a virtual object performing an action. That is, key frames of the virtual object performing an action in a historical time period are extracted, and the gait trajectory sequence is composed of motion image frames corresponding to key frames in each timestamp.

[0139] Step 1002: Input the gait trajectory sequence into the action feature extraction network for encoding to obtain a historical action feature representation.

[0140] In this embodiment of the application, in order to complete the online action generation task more effectively and reduce the overall system complexity, an end-to-end feature extraction is achieved through an action feature extraction network. That is, the gait trajectory sequence is input into the action feature extraction network for encoding to obtain a historical action feature representation.

[0141] In some embodiments, the above-mentioned action feature extraction network can be implemented as a MotionTemporal Convolutional Network (MTCN) model. Traditional temporal convolutional neural networks directly apply symmetric kernels along the time domain, as shown in Equation 6. However, in this embodiment, it is necessary to restrict the kernel to extracting only past input information, thereby avoiding the model using future information as input. This leads to Equation 7, where v represents the value of a channel at a certain time t, w is the kernel value, and τ is the maximum value of the kernel.

[0142] Formula Six:

[0143] Formula 7:

[0144] Schematic diagram: By inputting the gait trajectory sequence into the MTCN mentioned above, the historical action feature representation can be output, as shown in Formula 8, where θ t To obtain from gait trajectory sequences The historical action features extracted are represented by τ, where τ is the preset window size.

[0145] Formula 8:

[0146] Step 1003: Input the historical action feature representation into the gate network to predict the weights between at least two base actions to obtain the target weight data.

[0147] Schematic, the obtained historical action feature representation is input into the gate network of the hybrid expert network model to obtain the target weight data corresponding to the action prediction network of at least two base actions. Its function is shown in Equation 9, where α is the target weight vector. Represents a gate network, θ t This represents the historical action features extracted through MTCN.

[0148] Formula Nine:

[0149] In some embodiments, after the gate network completes the prediction of the weight data, the predicted weight data is input into the target modulation function corresponding to the target style to obtain the target weight data corresponding to the target style. The modulation process is the same as step 503 and will not be described in detail here.

[0150] Step 1004: The first network sub-parameter of the j-th layer and the j-th weight data corresponding to the j-th layer connection layer in the N action prediction network are weighted and summed to obtain the second network sub-parameter of the j-th layer in the second network parameters.

[0151] In this embodiment, the overall model includes N action prediction networks, and each action prediction network includes L connection layers. Here, L is a positive integer, N is a positive integer, j is a positive integer, and j ≤ L.

[0152] In some embodiments, to enhance the model's modulation capability, when generating target weight data through a gate network, the network structure of the gate network is adjusted so that the gate network generates weight data corresponding to each connection layer in each action prediction network. That is, the target weight data includes weight data corresponding to each connection layer in each action prediction network.

[0153] Since each connection layer of the action prediction network corresponds to a weight value, when mixing the action prediction network using target weight data, the first network sub-parameter corresponding to the j-th connection layer between the action prediction networks can be weighted and summed with the corresponding j-th weight data to obtain the second network sub-parameter of the j-th layer of the target hybrid network. In one example, as shown in Formula 10, ω j This represents the j-th connection layer of the target hybrid network. This represents the j-th connection layer in the action prediction network i. Indicates and The corresponding weight value.

[0154] Formula 10:

[0155] Step 1005: The second network parameters are composed of the second network sub-parameters of layer L to obtain the target hybrid network.

[0156] Once the second network parameters corresponding to the L-layer connection layer in the target hybrid network are determined, the target hybrid network can be obtained.

[0157] In one example, such as Figure 11 As shown, it illustrates the generation process of a target hybrid network provided in an exemplary embodiment of this application. The gait trajectory sequence 1101 is input into the gate network 1110, and the output is the hybrid weight data (α1, α2, ..., α...). N )1102, the mixed weight data 1102 and the target style 1103 are input into the target modulation function 1120, and the target weight data (ω1,ω2,…,ω) is output. N )1104, mix the target weight data 1104 with the corresponding expert network 1130 to obtain the target hybrid network 1140.

[0158] Step 1006: Input the action data at the first moment into the target hybrid network to predict the target action data at the second moment.

[0159] Indicatively, the historical motion data includes motion data at the first moment, where the first moment is the final moment in the historical time period and the second moment is the moment following the first moment. That is, the motion data at the next moment is predicted by using the motion data at the final moment in the historical time period, thereby converting the motion style of the virtual object from the first style to the target style.

[0160] The server sends the generated target action data to the terminal device, and the terminal device displays the action animation of the virtual object in the virtual environment under the target style based on the target action data.

[0161] In summary, the virtual object action generation method provided in this application, when it is necessary to switch the action style of a virtual object, determines the weight data corresponding to the target style based on the historical action data of the virtual object. Using this weight data, a hybrid network is obtained by mixing action prediction networks corresponding to base actions. This hybrid network then predicts the action data under the target style, thereby controlling the virtual object to complete the action style switching process. In this implementation, the prediction of action data is achieved through a hybrid network, eliminating the need for switching between multiple network models, reducing resource consumption during the virtual object action generation process, and thus improving the efficiency of action data generation.

[0162] In this embodiment, an end-to-end action generation process is achieved by introducing an action feature extraction network. A single model can complete the conversion of multiple action styles. At the same time, the features of historical action data guide the generation of subsequent action data, ensuring the continuity and flexibility of the action. The end-to-end single model reduces the number of models in the actual application scenario, reduces the memory requirements of the application when the action style is converted online in real time, and reduces the waste of resources while expanding the diversity of action styles of virtual objects in the virtual environment.

[0163] Please refer to Figure 12 The diagram illustrates a block diagram of a virtual object action generation apparatus provided in an exemplary embodiment of this application. The apparatus includes the following modules:

[0164] Data acquisition module 1210 is used to acquire historical action data of virtual object in response to receiving style conversion instruction for virtual object, wherein the style conversion instruction is used to instruct the action style of virtual object to be switched to target style, and the historical action data is used to indicate the action sequence of virtual object in historical time period;

[0165] The weight determination module 1220 is used to obtain target weight data between at least two base actions based on the historical action data and the target style, wherein the base actions are the reference actions when the virtual object performs actions.

[0166] The network determination module 1230 is used to mix the action prediction networks corresponding to the at least two base actions based on the target weight data to obtain a target mixed network, wherein the action prediction network is used to predict the action data;

[0167] The result prediction module 1240 is used to analyze and predict the historical action data through the target hybrid network to obtain target action data under the target style.

[0168] In some optional embodiments, the target hybrid network corresponds to N action prediction networks, where N ≥ 2 and N is an integer;

[0169] The network determination module 1230 is further configured to perform a weighted summation of the first network parameters corresponding to the N action prediction networks based on the target weight data, and set the obtained second network parameters as the network parameters of the target hybrid network.

[0170] In some optional embodiments, the action prediction network includes L connection layers, where L is a positive integer. When the target weight data includes the weight data corresponding to each connection layer in each action prediction network, the network determination module 1230 is further configured to perform a weighted summation of the first network sub-parameter of the j-th layer and the j-th weight data corresponding to the j-th connection layer in the N action prediction networks to obtain the second network sub-parameter of the j-th layer in the second network parameters, where j is a positive integer and j≤L; the second network parameters are composed of the L-th second network sub-parameters to obtain the target hybrid network.

[0171] In some optional embodiments, the historical motion data includes a motion phase sequence and a gait trajectory sequence. The motion phase sequence is used to indicate the contact between the virtual object and the virtual plane when the virtual object performs an action during the historical period, and the gait trajectory sequence is used to indicate the motion speed of the virtual object during the execution of the action during the historical period.

[0172] like Figure 13 As shown, the weight determination module 1220 further includes:

[0173] The weight generation unit 1221 is used to obtain mixed weight data based on the product of the phase element in the action phase sequence and the trajectory element in the gait trajectory sequence;

[0174] The weight modulation unit 1222 is used to modulate the mixed weight data based on the target style to obtain the target weight data.

[0175] In some optional embodiments, the weight determination module 1220 further includes:

[0176] The function determination unit 1223 is used to obtain the target modulation function corresponding to the target style;

[0177] The weight modulation unit 1222 is further configured to perform an affine transformation on the mixed weight data based on the target modulation function to obtain the target weight data.

[0178] In some optional embodiments, the data acquisition module 1210 further includes:

[0179] Data acquisition unit 1211 is used to acquire the action sequence of the virtual object within the historical time period;

[0180] The data annotation unit 1212 is used to annotate the action sequence based on the contact between the virtual object and the virtual plane when the virtual object performs the action, so as to obtain a contact action frame;

[0181] The data generation unit 1213 is used to generate the action phase sequence based on the timing between the contact action frames;

[0182] The data annotation unit 1212 is also used to annotate the action sequence based on the action type of the virtual object when performing the action, so as to obtain action type data;

[0183] The data generation unit 1213 is further configured to generate the gait trajectory sequence based on the action type data and the action speed information corresponding to the action sequence, wherein the action phase sequence and the gait trajectory sequence constitute the historical action data.

[0184] In some optional embodiments, the historical action data includes a gait trajectory sequence, which is used to indicate characteristic information of the actions performed by the virtual object during the historical period;

[0185] The weight determination module 1220 further includes:

[0186] Feature extraction module 1224 is used to input the gait trajectory sequence into the action feature extraction network for encoding to obtain a historical action feature representation;

[0187] The weight generation unit 1221 is further configured to input the historical action feature representation into a gate network to predict the weights between the at least two base actions, thereby obtaining the target weight data.

[0188] In some optional embodiments, the historical action data includes action data at a first moment, where the first moment is the final moment in the historical time period;

[0189] The result prediction module 1240 is further configured to input the action data at the first moment into the target hybrid network to predict the target action data at the second moment, wherein the second moment is the next moment after the first moment.

[0190] In some optional embodiments, when the action sequence corresponding to the historical action data is used to indicate the absolute position of the virtual object during the execution of an action, the device further includes:

[0191] The data preprocessing module 1250 is used to convert the absolute position change of the virtual object in the historical action data into a relative position change to obtain preprocessed action data, which is used in the process of determining the target weight data.

[0192] In some alternative embodiments, the apparatus further includes:

[0193] The data sending module 1260 is used to send the target action data to the terminal corresponding to the virtual object, and the terminal is used to control the virtual object to display the action execution animation under the target style according to the target action data.

[0194] In summary, the virtual object motion generation device provided in this application, when it is necessary to switch the motion style of a virtual object, determines the weight data corresponding to the target style based on the historical motion data of the virtual object. Using this weight data, a hybrid network is obtained by mixing motion prediction networks corresponding to base actions. This hybrid network then predicts the motion data under the target style, thereby controlling the virtual object to complete the motion style switching process. In this implementation, motion data prediction is achieved through a hybrid network, eliminating the need for switching between multiple network models, reducing resource consumption during virtual object motion generation, and thus improving the efficiency of motion data generation.

[0195] It should be noted that the virtual object action generation device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the virtual object action generation device and the virtual object action generation method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0196] Figure 14 This illustration shows a schematic diagram of the structure of a server provided in an exemplary embodiment of this application. Specifically, it includes the following structure.

[0197] Server 1400 includes a Central Processing Unit (CPU) 1401, a system memory 1404 including Random Access Memory (RAM) 1402 and Read Only Memory (ROM) 1403, and a system bus 1405 connecting the system memory 1404 and the CPU 1401. Server 1400 also includes a mass storage device 1406 for storing an operating system 1413, application programs 1414, and other program modules 1415.

[0198] Mass storage device 1406 is connected to central processing unit 1401 via a mass storage controller (not shown) connected to system bus 1405. Mass storage device 1406 and its associated computer-readable media provide non-volatile storage for server 1400. That is, mass storage device 1406 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drives.

[0199] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1404 and mass storage device 1406 described above can be collectively referred to as memory.

[0200] According to various embodiments of this application, server 1400 can also be connected to a remote computer on a network, such as the Internet. That is, server 1400 can be connected to network 1412 via network interface unit 1411 connected to system bus 1405, or it can also use network interface unit 1411 to connect to other types of networks or remote computer systems (not shown).

[0201] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU.

[0202] Embodiments of this application also provide a computer device including a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads and executes the at least one instruction, at least one program, code set, or instruction set to implement the virtual object action generation method provided in the above-described method embodiments. Optionally, the computer device may be a terminal or a server.

[0203] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the virtual object action generation method provided in the above-described method embodiments.

[0204] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the virtual object action generation method described in any of the above embodiments.

[0205] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0206] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0207] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for generating actions for a virtual object, characterized in that, The method includes: In response to receiving a style conversion instruction for a virtual object, historical motion data of the virtual object is obtained. The style conversion instruction is used to instruct the motion style of the virtual object to be switched to the target style. The historical motion data includes a motion phase sequence and a gait trajectory sequence. The motion phase sequence is used to indicate the contact between the virtual object and the virtual plane when performing the action during the historical period. The gait trajectory sequence is used to indicate the motion speed of the virtual object during the execution of the action during the historical period. Hybrid weight data is obtained based on the product of the phase elements in the action phase sequence and the trajectory elements in the gait trajectory sequence; Based on the target style, the hybrid weight data is modulated to obtain target weight data between at least two base actions, wherein the base actions are the baseline actions when the virtual object performs actions. Based on the target weight data, the action prediction networks corresponding to the at least two base actions are mixed to obtain a target hybrid network, which is used to predict action data. The target hybrid network is used to analyze and predict the historical action data to obtain target action data under the target style.

2. The method according to claim 1, characterized in that, The target hybrid network corresponds to N action prediction networks, where N≥2 and N is an integer; The step of mixing the action prediction networks corresponding to the at least two base actions based on the target weight data to obtain the target hybrid network includes: The first network parameters corresponding to the N action prediction networks are weighted and summed according to the target weight data, and the resulting second network parameters are set as the network parameters of the target hybrid network.

3. The method according to claim 2, characterized in that, The action prediction network includes L connection layers, where L is a positive integer. When the target weight data includes the weight data corresponding to each connection layer in each action prediction network, the step of weighted summing of the first network parameters corresponding to the N action prediction networks based on the target weight data, and setting the resulting second network parameters as the network parameters of the target hybrid network, includes: The first network sub-parameter of the j-th layer and the j-th weight data corresponding to the j-th layer connection layer in the N action prediction network are weighted and summed to obtain the second network sub-parameter of the j-th layer in the second network parameter, where j is a positive integer and j≤L; The second network parameters are composed of the second network sub-parameters of layer L, thus obtaining the target hybrid network.

4. The method according to any one of claims 1 to 3, characterized in that, The modulation of the mixed weight data based on the target style to obtain target weight data between at least two base actions includes: Obtain the target modulation function corresponding to the target style; The target weight data is obtained by performing an affine transformation on the mixed weight data based on the target modulation function.

5. The method according to any one of claims 1 to 3, characterized in that, The acquisition of historical action data of virtual objects includes: Obtain the action sequence of the virtual object within the historical time period; The action sequence is labeled based on the contact between the virtual object and the virtual plane when the action is performed, and contact action frames are obtained. The action phase sequence is generated based on the timing of the contact action frames; The action sequence is labeled based on the action type of the virtual object when performing the action to obtain action type data; Based on the action type data and the action speed information corresponding to the action sequence, the gait trajectory sequence is generated, and the action phase sequence and the gait trajectory sequence constitute the historical action data.

6. The method according to any one of claims 1 to 3, characterized in that, The historical action data includes a gait trajectory sequence, which is used to indicate the characteristic information of the actions performed by the virtual object during the historical period. The method further includes: The gait trajectory sequence is input into the action feature extraction network for encoding to obtain a historical action feature representation; The historical action feature representation is input into a gate network to predict the weights between the at least two base actions, thereby obtaining the target weight data.

7. The method according to any one of claims 1 to 3, characterized in that, The historical action data includes action data at a first moment, where the first moment is the final moment in the historical time period. The step of analyzing and predicting the historical action data through the target hybrid network to obtain target action data under the target style includes: The action data at the first moment is input into the target hybrid network to predict the target action data at the second moment, which is the moment after the first moment.

8. The method according to any one of claims 1 to 3, characterized in that, When the action sequence corresponding to the historical action data is used to indicate the absolute position of the virtual object during the execution of an action, the process of obtaining the historical action data of the virtual object further includes: The absolute position changes of the virtual object in the historical action data are converted into relative position changes to obtain preprocessed action data, which is used in the process of determining the target weight data.

9. The method according to any one of claims 1 to 3, characterized in that, After analyzing and predicting the historical action data through the target hybrid network to obtain the target action data under the target style, the method further includes: The target action data is sent to the terminal corresponding to the virtual object, and the terminal is used to control the virtual object to display the action execution animation under the target style according to the target action data.

10. A device for generating actions of a virtual object, characterized in that, The device includes: The data acquisition module is used to acquire historical motion data of the virtual object in response to receiving a style conversion instruction for the virtual object. The style conversion instruction is used to instruct the motion style of the virtual object to be switched to the target style. The historical motion data includes motion phase sequence and gait trajectory sequence. The motion phase sequence is used to indicate the contact between the virtual object and the virtual plane when the virtual object performs the action during the historical period. The gait trajectory sequence is used to indicate the motion speed of the virtual object during the execution of the action during the historical period. The weight generation unit is used to obtain mixed weight data based on the product of the phase elements in the action phase sequence and the trajectory elements in the gait trajectory sequence; A weight modulation unit is used to modulate the mixed weight data based on the target style, and modulate the mixed weight data based on the target style to obtain target weight data between at least two base actions, wherein the base actions are the reference actions when the virtual object performs actions; A network determination module is used to mix the action prediction networks corresponding to the at least two base actions based on the target weight data to obtain a target hybrid network, wherein the action prediction network is used to predict action data; The result prediction module is used to analyze and predict the historical action data through the target hybrid network to obtain target action data under the target style.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the virtual object action generation method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the virtual object action generation method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the action generation method for virtual objects as described in any one of claims 1 to 9.

Images