Method, device and medium for animation processing of virtual scene

Abstract

The application provides an animation processing method and device of a virtual scene, an electronic device, a computer readable storage medium and a computer program product. The method comprises: obtaining an interaction mode corresponding to a virtual object when the virtual object interacts in a virtual scene; determining a target animation mode matched with the interaction mode in an animation mode based on machine learning and an animation mode based on playing a skeletal animation; performing animation simulation processing on the virtual object based on the target animation mode matched with the interaction mode, to obtain an animation of the virtual object; wherein the animation is used to represent the interaction mode of the virtual object in the virtual scene; and playing the animation of the virtual object based on the virtual scene. Through the application, the related computing resources of the graphics processing hardware can be saved.

Description

Technical Field

[0001] This application relates to computer data processing technology, and more particularly to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for animation processing of virtual scenes. Background Technology

[0002] Display technologies based on graphics processing hardware have expanded the channels for perceiving the environment and acquiring information. In particular, virtual scene display technologies can realize diverse interactions between virtual objects controlled by users or artificial intelligence according to actual application needs. They have various typical application scenarios. For example, in virtual scenes such as games, they can simulate real interactions and battles between virtual objects.

[0003] With the development of computer technology, computer animation has been widely used in various fields such as game production and animation production. For example, current animation production mainly relies on various basic skeletal animations. In order to achieve better interactive performance of virtual objects, skeletal animation needs to cover various interactive scenes, such as different speeds, different angles of turning, and terrains with different slopes, in order to generate massive amounts of data.

[0004] However, skeletal animation in related technologies is produced by hand or motion capture technology, which consumes a lot of resources (including hardware and computing resources). Summary of the Invention

[0005] This application provides an animation processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for virtual scenes, which can save the related computing resources of graphics processing hardware.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides an animation processing method for a virtual scene, including:

[0008] To obtain the interaction method corresponding to when a virtual object interacts with another virtual object in a virtual scene;

[0009] Among machine learning-based animation modes and animation modes based on playing skeletal animation, a target animation mode that matches the interaction method is determined;

[0010] The virtual object is animated based on a target animation mode that matches the interaction method to obtain the animation of the virtual object.

[0011] The animation is used to represent the interaction mode of the virtual object in the virtual scene;

[0012] The animation of the virtual object is played based on the virtual scene.

[0013] This application provides an animation processing device for a virtual scene, comprising:

[0014] The acquisition module is used to acquire the interaction methods corresponding to when virtual objects will interact in the virtual scene.

[0015] The determination module is used to determine a target animation mode that matches the interaction method among machine learning-based animation modes and animation modes based on playing skeletal animation.

[0016] The simulation module is used to perform animation simulation processing on the virtual object based on a target animation mode that matches the interaction mode, so as to obtain the animation of the virtual object; wherein, the animation is used to characterize the interaction mode of the virtual object in the virtual scene;

[0017] The playback module is used to play the animation of the virtual object based on the virtual scene.

[0018] In the above technical solution, the determining module is further configured to, when determining that the interaction method is a non-cyclic movement method, use the machine learning-based animation mode as the target animation mode that matches the interaction method;

[0019] When the interaction method is determined to be a non-movement method or a loop movement method, the animation mode based on playing skeletal animation is determined as the target animation mode that matches the interaction method.

[0020] The types of non-mobile interaction include: skill interaction and performance interaction.

[0021] In the above technical solution, before determining the target animation mode that matches the interaction mode, the determining module is further configured to determine the interaction mode as the non-loop movement mode when at least one of the interaction parameters in the interaction mode is greater than the first interaction parameter threshold.

[0022] When all the interaction parameters in the interaction mode are less than or equal to the first interaction parameter threshold, the interaction mode is determined to be the cyclic movement mode;

[0023] The types of interactive parameters include: the angle difference between the virtual object's torso orientation and the direction of its movement speed is less than a first angle threshold; the speed difference between the virtual object's movement speed and the target's movement speed; and the slope of the terrain on which the virtual object moves.

[0024] In the above technical solution, when the previous interaction mode of the interaction mode is the animation mode based on playing skeletal animation, the determining module is further used to adjust the interaction parameters in the interaction mode when the interaction parameters in the interaction mode are greater than the first interaction parameter threshold and less than the second interaction parameter threshold, and determine the interaction mode as the cyclic movement mode;

[0025] Wherein, the second interaction parameter threshold is greater than the first interaction parameter threshold.

[0026] In the above technical solution, the simulation module is further used to perform animation simulation processing on the virtual object based on the interaction method and the target animation mode when the interaction method is the first interaction, so as to obtain the animation of the virtual object;

[0027] When the interaction method is not the first interaction, the virtual object is subjected to animation simulation processing based on the previous interaction method and the target animation mode to obtain the animation of the virtual object.

[0028] In the above technical solution, the simulation module is further used to perform machine learning-based animation generation processing on the virtual object based on the interaction method when the target animation mode is the machine learning-based animation mode, so as to obtain the animation of the virtual object;

[0029] When the target animation mode is the animation mode based on playing skeletal animation, the virtual object is subjected to skeletal animation matching processing based on the interaction method to obtain the animation of the virtual object.

[0030] In the above technical solution, the simulation module is further used to perform machine learning-based animation generation processing on the virtual object based on the previous interaction mode of the interaction mode when the target animation mode is the machine learning-based animation mode, so as to obtain the animation of the virtual object;

[0031] When the target animation mode is the animation mode based on playing skeletal animation, the virtual object is subjected to skeletal animation matching processing based on the previous interaction mode to obtain the animation of the virtual object.

[0032] In the above technical solution, the simulation module is also used to obtain the last frame of the animation image of the previous interaction method;

[0033] Based on the state parameters of the last frame of the animation image, a machine learning model is invoked to perform animation prediction processing to obtain the animation of the virtual object;

[0034] The first frame of the animation of the virtual object is matched with the last frame of the animation of the previous interaction.

[0035] In the above technical solution, the simulation module is also used to obtain the last frame of the animation image of the previous interaction method;

[0036] In the skeletal animation across multiple state machines, determine the target skeletal animation that matches the interaction method;

[0037] Among the multiple animated images of the target skeletal animation, determine the target animated image that matches the last frame of the animated image;

[0038] Using the target animation image as the starting point, the target skeletal animation is cropped to obtain the animation of the virtual object.

[0039] In the above technical solution, the simulation module is further configured to determine the animation image that matches the phase parameter of the last frame animation image as the target animation image when the plurality of animation images include phase parameters;

[0040] When the plurality of animation images do not include phase parameters, motion matching is performed on the last frame animation image and the plurality of animation images to obtain a target animation image that matches the last frame animation image.

[0041] In the above technical solution, the simulation module is further used to obtain the playback frequency of the animation of the virtual object when the target animation mode is the machine learning-based animation mode;

[0042] When the playback frequency is greater than the playback frequency threshold, the animation of the virtual object is determined as the skeletal animation of the animation mode based on the playback skeletal animation.

[0043] When a subsequent interaction matches the first interaction, the virtual object is animated based on the skeletal animation to obtain the animation corresponding to the subsequent interaction.

[0044] This application provides an electronic device for animation processing, the electronic device comprising:

[0045] Memory, used to store executable instructions;

[0046] The processor, when executing executable instructions stored in the memory, implements the animation processing method for virtual scenes provided in the embodiments of this application.

[0047] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the animation processing method for a virtual scene provided in this application.

[0048] This application provides a computer program product, including a computer program or instructions, characterized in that the computer program or instructions, when executed by a processor, implement the animation processing method for a virtual scene provided in this application.

[0049] The embodiments of this application have the following beneficial effects:

[0050] The embodiments of this application have the following beneficial effects:

[0051] By identifying a target animation mode that matches the interaction method from both machine learning-based and skeletal animation-based animation modes, accurate and efficient animation simulation can be achieved using machine learning or skeletal animation techniques. This improves the accuracy of human-computer interaction in virtual scenes and saves computational resources on graphics processing hardware. Attached Figure Description

[0052] Figure 1A-Figure 1B This is a schematic diagram illustrating the application mode of the animation processing method for virtual scenes provided in the embodiments of this application;

[0053] Figure 2 This is a schematic diagram of the structure of an electronic device for animation processing provided in an embodiment of this application;

[0054] Figures 3-5 This is a flowchart illustrating the animation processing method for a virtual scene provided in an embodiment of this application;

[0055] Figure 6 This is a schematic diagram of an animation of walking and running on flat ground provided in the embodiments of this application;

[0056] Figure 7 This is a schematic diagram of a flat-ground turning and running animation provided in an embodiment of this application;

[0057] Figure 8 This is a schematic diagram of walking and running animation in a complex terrain scene provided in an embodiment of this application;

[0058] Figure 9 This is a flowchart illustrating the animation processing method for a virtual scene provided in an embodiment of this application;

[0059] Figure 10 This is a schematic diagram of matching suitable animation frames provided in the embodiments of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0063] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0064] 1) Responding to: used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0065] 2) Client: An application running on a terminal that provides various services, such as a video playback client, a game client, etc.

[0066] 3) Virtual Scene: A virtual game scene displayed (or provided) when the game program runs on the terminal. This virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual environment, or a purely fictional virtual environment. The virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application does not limit the dimension of the virtual scene. For example, a virtual scene may include the sky, land, ocean, etc., and the land may include environmental elements such as deserts and cities. Users can control virtual objects to move within this virtual scene.

[0067] 4) Virtual Objects: These are interactive images of people and objects within a virtual scene, or movable objects within the virtual scene. These movable objects can be virtual characters, virtual animals, anime characters, etc., such as people or animals displayed in a virtual scene. A virtual object can be a virtual character representing the user within the virtual scene. A virtual scene can include multiple virtual objects, each with its own shape and volume, occupying a portion of the space within the virtual scene.

[0068] 5) Scene data: This represents the characteristic data of the virtual scene, such as the area of ​​the construction area in the virtual scene, the current architectural style of the virtual scene, etc.; it can also include the location of the virtual building in the virtual scene, and the area occupied by the virtual building, etc.

[0069] In related technologies, for game character animations, different animations are played to meet different scenarios (such as walking, running, sprinting, turning left, turning right, going uphill, going downhill), and the logic of switching different animations in different scenarios is realized through animation state machines or hybrid trees.

[0070] Because of the large number of combinations of different scenes, such as walking and running at different speeds, turning at different angles, and different slopes, a large amount of basic skeletal animation, complex animation state machine logic, or hybrid tree logic is required. Especially when pursuing the ultimate animation effect, the animation production workload is large and the cycle is long. Moreover, a large number of animations will also occupy a lot of storage resources, especially for mobile games.

[0071] To address the aforementioned problems, embodiments of this application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for animation processing of virtual scenes, which can save computational resources related to graphics processing hardware. To facilitate a clearer understanding of the animation processing method for virtual scenes provided in this application, exemplary implementation scenarios of the method are first described. The virtual objects in the animation processing method provided in this application can be entirely based on terminal output, or based on collaborative output from both the terminal and server.

[0072] In some embodiments, the virtual scene can be an environment for game characters to interact, such as a virtual scene for game characters to fight each other. By controlling the actions of the game characters, the two sides can interact in the virtual scene, thereby allowing users to relieve life stress during the game.

[0073] In one implementation scenario, see Figure 1A , Figure 1AThis is a schematic diagram of the application mode of the animation processing method for virtual scenes provided in the embodiments of this application. It is applicable to some application modes that can complete the relevant data calculation of virtual scene 100 by relying entirely on the graphics processing hardware computing power of terminal 400, such as stand-alone / offline games, and complete the output of virtual scenes through various types of terminals 400 such as smartphones, tablets and virtual reality / augmented reality devices.

[0074] As an example, types of graphics processing hardware include central processing units (CPUs) and graphics processing units (GPUs).

[0075] When visual perception of virtual scene 100 is formed, terminal 400 calculates the data required for display through graphics computing hardware, and completes the loading, parsing and rendering of display data. The graphics output hardware outputs video frames that can form visual perception of virtual scene. For example, two-dimensional video frames are presented on the display screen of a smartphone, or video frames that achieve three-dimensional display effect are projected on the lenses of augmented reality / virtual reality glasses. In addition, in order to enrich the perception effect, terminal 400 can also use different hardware to form one or more of auditory perception, tactile perception, motion perception and taste perception.

[0076] As an example, a client 410 (e.g., a standalone game application) runs on terminal 400. During the operation of client 410, the output includes a virtual scene for role-playing. The virtual scene can be an environment for game characters to interact with, such as a plain, street, valley, etc., for game characters to fight. Taking the virtual scene 100 displayed in a first-person perspective as an example, the animation of a virtual object 110 is displayed in the virtual scene 100. The virtual object 110 can be an artificial intelligence (AI) trained and set in the virtual scene. The virtual object 110 can walk in the virtual scene, and can also stay still, jump, and use various functions (such as skills and items); the virtual object 110 can be a game character controlled by the user (or player), and will operate in the virtual scene in response to the real user's operation on buttons (including joystick buttons, attack buttons, defense buttons, etc.). For example, when the real user moves the joystick button to the left, the virtual object will move to the left in the virtual scene, and can also stay still, jump, and use various functions (such as skills and items); the virtual object 110 can also be a non-user character (NPC, Non-Player Character) set in the interaction of the virtual scene.

[0077] For example, taking the first-person perspective of displaying a virtual scene 100 as an example, the animation of a virtual object 110 is displayed in the virtual scene 100. The animation processing method of the virtual scene provided in this application is used to obtain the interaction mode corresponding to the interaction of the virtual object 110 in the virtual scene. Among the animation mode based on machine learning and the animation mode based on playing skeletal animation, a target animation mode that matches the interaction mode is determined. The virtual object is simulated based on the target animation mode that matches the interaction mode to obtain the animation of the virtual object 110. The animation of the virtual object 110 is played based on the virtual scene 100, such as staying still, jumping, sprinting, turning left, turning right, going uphill, going downhill, and using various skills and props.

[0078] In another implementation scenario, see Figure 1B , Figure 1B This is a schematic diagram of the application mode of the animation processing method for virtual scenes provided in the embodiments of this application. It is applied to the terminal 400 and the server 200, and is suitable for the application mode that relies on the computing power of the server 200 to complete the calculation of virtual scenes and output the virtual scenes on the terminal 400.

[0079] Taking the visual perception of virtual scene 100 as an example, server 200 calculates display data related to the virtual scene (such as scene data) and sends it to terminal 400 via network 300. Terminal 400 relies on graphics computing hardware to load, parse, and render the calculated display data, and relies on graphics output hardware to output the virtual scene to form visual perception. For example, two-dimensional video frames can be displayed on the screen of a smartphone, or video frames with a three-dimensional display effect can be projected onto the lenses of augmented reality / virtual reality glasses. As for the perception of the form of the virtual scene, it can be understood that it can be achieved with the help of the corresponding hardware output of terminal 400, such as using a microphone to form auditory perception, using a vibrator to form tactile perception, and so on.

[0080] As an example, a client 410 (e.g., a network-based game application) runs on terminal 400. It interacts with other users through a connection to server 200 (e.g., a game server). Terminal 400 outputs a virtual scene 100 from client 410. For example, the virtual scene 100 is displayed in a first-person perspective. Animations of virtual objects 110 are displayed within the virtual scene 100. Virtual objects 110 can be artificial intelligence (AI) trained and set up within the virtual scene. Virtual objects 110 can walk within the virtual scene, remain stationary, jump, and use various functions (such as skills and items). Virtual objects 110 can also be game characters controlled by the user (or player), responding to real user actions on buttons (including joystick buttons, attack buttons, defense buttons, etc.) within the virtual scene. For example, when the real user moves the joystick button to the left, the virtual object will move to the left within the virtual scene. It can also remain stationary, jump, and use various functions (such as skills and items). Virtual objects 110 can also be non-user characters (NPCs) set up for interaction within the virtual scene. Character).

[0081] For example, taking the first-person perspective of displaying a virtual scene 100 as an example, the animation of a virtual object 110 is displayed in the virtual scene 100. The animation processing method of the virtual scene provided in this application is used to obtain the interaction mode corresponding to the interaction of the virtual object 110 in the virtual scene. Among the animation mode based on machine learning and the animation mode based on playing skeletal animation, a target animation mode that matches the interaction mode is determined. The virtual object is simulated based on the target animation mode that matches the interaction mode to obtain the animation of the virtual object 110. The animation of the virtual object 110 is played based on the virtual scene 100, such as staying still, jumping, sprinting, turning left, turning right, going uphill, going downhill, and using various skills and props.

[0082] In some embodiments, the terminal 400 can implement the animation processing method for the virtual scene provided in this application embodiment by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a level game APP (i.e., the client 410 mentioned above); it can also be a mini-program, that is, a program that only needs to be downloaded to the browser environment to run; or it can be a game mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plugin.

[0083] Taking a computer program as an example, in actual implementation, terminal 400 has an application that supports virtual scenes installed and running. This application can be any of the following: a first-person shooter (FPS) game, a third-person shooter game, a virtual reality application, a 3D map application, or a multiplayer shooting survival game. Users interact with virtual objects located in the virtual scene using terminal 400. These interactions include, but are not limited to: adjusting body posture, crawling, walking, running, riding, jumping, driving, picking up items, shooting, attacking, throwing, and constructing virtual buildings—at least one of these. For illustrative purposes, the virtual object can be a virtual character, such as a realistic or anime character.

[0084] In some embodiments, the present application embodiments can also be implemented with the aid of cloud technology, which refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize the computation, storage, processing, and sharing of data.

[0085] Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form resource pools, allowing for on-demand use with flexibility and convenience. Cloud computing technology will become a crucial support. The backend services of cloud computing systems require substantial computing and storage resources.

[0086] Example, Figure 1B The server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, etc., but is not limited to these. The terminal 400 and server 200 can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0087] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device for animation processing provided in an embodiment of this application. The description uses terminal 400 as an example. Figure 2The illustrated electronic device 400 includes at least one processor 420, a memory 460, at least one network interface 430, and a user interface 440. The various components in the terminal 400 are coupled together via a bus system 450. It is understood that the bus system 450 is used to implement communication between these components. In addition to a data bus, the bus system 450 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 450.

[0088] Processor 420 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0089] User interface 440 includes one or more output devices 441 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 440 also includes one or more input devices 442, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0090] The memory 460 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 460 may optionally include one or more storage devices physically located away from the processor 420.

[0091] The memory 460 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 460 described in this application embodiment is intended to include any suitable type of memory.

[0092] In some embodiments, memory 460 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0093] Operating system 461 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0094] The network communication module 462 is used to reach other computing devices via one or more (wired or wireless) network interfaces 430, exemplary network interfaces 430 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0095] Presentation module 463 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 441 (e.g., a display screen, a speaker, etc.) associated with user interface 440;

[0096] The input processing module 464 is used to detect and translate one or more user inputs or interactions from one or more input devices 442.

[0097] In some embodiments, the animation processing apparatus for virtual scenes provided in this application can be implemented in software. Figure 2 An animation processing device 465 for a virtual scene stored in memory 460 is shown. It can be software in the form of programs and plug-ins, including the following software modules: acquisition module 4651, determination module 4652, simulation module 4653, and playback module 4654. These modules are logically related and can therefore be arbitrarily combined or further split according to the functions they implement.

[0098] In other embodiments, the animation processing device for virtual scenes provided in this application can be implemented in hardware. As an example, the animation processing device for virtual scenes provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the animation processing method for virtual scenes provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0099] The animation processing method for virtual scenes provided in this application embodiment will be described in detail below with reference to the accompanying drawings. The animation processing method for virtual scenes provided in this application embodiment can be... Figure 1A Terminal 400 can be executed independently, or it can be... Figure 1B Terminal 400 and server 200 work together to execute.

[0100] Below, by Figure 1A The following description uses the example of terminal 400 executing the virtual scene animation processing method provided in this application embodiment. See also... Figure 3 , Figure 3 This is a flowchart illustrating the animation processing method for virtual scenes provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained.

[0101] It should be noted that, Figure 3 The method shown can be executed by various forms of computer programs running on terminal 400, and is not limited to the client 410 described above. It can also be the operating system 461, software modules and scripts mentioned above. Therefore, the client should not be regarded as a limitation on the embodiments of this application.

[0102] In step 101, the interaction method corresponding to the interaction of the virtual object in the virtual scene is obtained.

[0103] It should be noted that the interaction methods of virtual objects refer to the interactive behaviors of virtual objects in various animation scenes (such as walking and running in a straight line on flat ground, skill performances), such as adjusting body posture, crawling, walking, running, turning left, turning right, going uphill, going downhill, riding, jumping, driving, picking up, shooting, attacking, throwing, skill output, standing performances, constructing virtual buildings, etc. Among them, different animation scenes include walking and running at different speeds, turning at different angles, and different slopes.

[0104] In step 102, a target animation mode that matches the interaction method is determined among the machine learning-based animation mode and the animation mode based on playing skeletal animation.

[0105] For example, by selecting a target animation mode that matches the interaction method from machine learning-based animation modes and skeletal animation-based animation modes, subsequent accurate and efficient animation simulation can be performed based on machine learning technology or skeletal animation technology, thereby saving the relevant computing resources of graphics processing hardware.

[0106] It should be noted that the machine learning-based animation mode (i.e., machine learning-generated animation scheme) refers to a mode that uses a machine learning model to generate animations, that is, it uses machine learning technology to generate animations with corresponding interactive methods. The embodiments in this application are not limited to the model structure of the machine learning model; the machine learning model can be a convolutional neural network, a deep neural network, etc. The skeletal animation-based animation mode (i.e., skeletal animation playback scheme) refers to a mode that uses state machine logic or hybrid tree logic to play skeletal animations. That is, state machine logic or hybrid tree logic determines when to play which skeletal animation. For example, a hybrid tree can be used to select the skeletal animation to be played based on the input of the control signal. For example, a control signal indicating slow speed can select to play a walking animation, a control signal indicating a turn can select to play a turning animation at the corresponding angle, and a control signal indicating an uphill slope can select to play a walking / running animation at the corresponding slope, etc.

[0107] A state machine includes a state register and combinational logic circuits. It can transition between pre-defined states based on control signals. For example, a state can be assigned an animation; if the current animation is uphill, the state can transition to a downhill animation, thus achieving both uphill and downhill animations. Unlike a state machine, where a state can only assign one animation to a single state, a blend tree can be configured as a mixture of multiple animations (i.e., states). For example, a left-turn / walk / run animation in a blend tree can be a combination of a left-turn state and a straight-line / walk / run animation.

[0108] See Figure 4 , Figure 4 This is an optional flowchart illustrating an animation processing method for a virtual scene provided in an embodiment of this application. Figure 4 Show Figure 3 Step 102 can be implemented through steps 1021-1022: In step 1021, when the interaction mode is determined to be a non-loop movement mode, the machine learning-based animation mode is used as the target animation mode that matches the interaction mode; In step 1022, when the interaction mode is determined to be a non-movement mode or a loop movement mode, the animation mode based on playing skeletal animation is determined as the target animation mode that matches the interaction mode; Wherein, the types of non-movement modes include: skill interaction and performance interaction.

[0109] It should be noted that cyclic movement refers to the fixed, cyclical movement behavior of virtual objects in a virtual scene, such as walking or running in a straight line on flat ground; non-cyclic movement refers to the fixed, non-cyclical movement behavior of virtual objects in a virtual scene, such as walking or running with a left turn on flat ground, or walking or running uphill; non-movement refers to the non-movement interactive behavior of virtual objects in a virtual scene, such as skill interaction (throwing virtual bombs) or performance interaction (such as dancing or riding horses).

[0110] For example, for non-movement actions, such as skill actions (e.g., jump attacks) and standing performances, although machine learning-based animation modes can also achieve this, on the one hand, it increases the size of the machine learning model, thus increasing performance overhead, and on the other hand, machine learning technology is more prone to distortion when generating such animations, thus affecting the animation performance. In contrast, using an animation mode based on playing skeletal animation can avoid the above problems and is simple and effective.

[0111] For looping movements in locomotion scenarios, while machine learning-based animation can achieve this, it suffers from loss of detail and high performance overhead. Therefore, this application proposes using skeletal animation to improve both performance and visual appeal. For example, in interactions involving walking and running on flat ground, navigation logic drives virtual objects to move, mostly in straight lines. Since most hotspot maps in games are flat, the virtual objects primarily perform straight-line walking / running actions. While such actions can be generated through machine learning, they can also be achieved by directly playing skeletal animation due to their fixed nature. Therefore, replacing machine learning with directly playing straight-line walking / running animation offers several advantages: firstly, the overhead of playing skeletal animation is significantly lower than that of machine learning-generated animation, thus improving performance; secondly, playing skeletal animation avoids any distortion and provides fully controllable visual effects. For instance, playing a longer straight-line animation can prevent monotony and enhance the overall visual experience.

[0112] For non-cyclic movement in Locomotion scenarios, a machine learning-based animation mode is adopted, which can better cover different movement scenarios and avoid storage consumption issues.

[0113] In some embodiments, before determining the target animation mode that matches the interaction mode, when at least one of the interaction parameters in the interaction mode is greater than a first interaction parameter threshold, the interaction mode is determined to be a non-cyclic movement mode; when all the interaction parameters in the interaction mode are less than or equal to the first interaction parameter threshold, the interaction mode is determined to be a cyclic movement mode; wherein, the types of interaction parameters include: the angle difference between the virtual object's torso orientation and the movement speed direction is less than a first angle threshold, the speed difference between the virtual object's movement speed and the target movement speed, and the slope of the virtual object's movement terrain.

[0114] For example, in the case of machine learning-based animation mode, the transition to skeletal animation-based mode will occur when all of the following conditions are met (i.e., all interaction parameters in the interaction mode are less than or equal to the first interaction parameter threshold):

[0115] (1) The current virtual object's orientation is consistent with the current driving speed direction, that is, the angle difference between the current character's orientation and the driving speed direction is less than or equal to the angle threshold 1;

[0116] (2) The current speed of the virtual object is close to the target speed, that is, the speed difference between the current speed and the target speed is less than or equal to the speed threshold 1;

[0117] (3) The current terrain is close to flat ground (used to determine whether the virtual object is walking up or down a slope), that is, the slope is less than or equal to the slope threshold 1.

[0118] When transitioning to the playback-based skeletal animation mode, you can select an animation that matches the target speed or the target gait (e.g., walking, running, sprinting).

[0119] In the case of skeletal animation mode, the system will transition to machine learning-based animation mode when any of the following conditions are met (i.e., at least one interaction parameter in the interaction method is greater than the first interaction parameter threshold):

[0120] (1) The current driving speed direction is inconsistent with the current virtual object orientation direction, that is, the angle difference between the current character orientation and the driving speed direction is greater than the angle threshold 1;

[0121] (2) The target speed is inconsistent with the current speed, that is, the speed difference between the current speed and the target speed is greater than the speed threshold 1;

[0122] (3) The current terrain is not flat, that is, the slope is greater than the slope threshold 1.

[0123] Among them, when the animation in the skeletal animation mode is about to finish playing, it can directly transition to the next animation without having to transition back to the machine learning model to generate the animation, thereby reducing the overhead of the transition.

[0124] In some embodiments, when the previous interaction mode was an animation mode based on playing skeletal animation, when the interaction parameter in the interaction mode is greater than the first interaction parameter threshold and less than the second interaction parameter threshold, the interaction parameter in the interaction mode is adjusted, and the interaction mode is determined to be a loop movement mode; wherein, the second interaction parameter threshold is greater than the first interaction parameter threshold.

[0125] Following the example above, in the case of playing skeletal animation mode, when the interaction parameter in the interaction method is greater than the first interaction parameter threshold and less than the second interaction parameter threshold, the interaction parameter in the interaction method is adjusted, and the playing skeletal animation mode is still executed. It will not transition to the machine learning-based animation mode to avoid frequent switching.

[0126] For example, when the angle difference between the current virtual object's orientation and the driving speed direction is greater than angle threshold 1 and less than angle threshold 2, the orientation of the virtual object is slowly adjusted so that the angle difference between the current virtual object's orientation and the driving speed direction is less than or equal to angle threshold 1; when the speed difference between the current speed of the virtual object and the target speed is greater than speed threshold 1 and less than speed threshold 2, the current speed of the virtual object is slowly adjusted so that the speed difference between the current speed of the virtual object and the target speed is less than or equal to speed threshold 1; when the slope of the current terrain is greater than slope threshold 1 and less than slope threshold 2, the slope is slowly adjusted so that the slope of the current terrain is less than or equal to slope threshold 1.

[0127] In step 103, the virtual object is animated based on the target animation mode that matches the interaction mode to obtain the animation of the virtual object, wherein the animation is used to represent the interaction mode of the virtual object in the virtual scene.

[0128] For example, by selecting a target animation mode that matches the interaction method, accurate and efficient animation simulation can be performed based on machine learning technology or skeletal animation technology, thereby saving the relevant computing resources of graphics processing hardware.

[0129] See Figure 5 , Figure 5 This is an optional flowchart illustrating an animation processing method for a virtual scene provided in an embodiment of this application. Figure 5 Show Figure 3 Step 103 can be implemented through steps 1031-1032: In step 1031, when the interaction method is the first interaction, the virtual object is subjected to animation simulation processing based on the interaction method and the target animation mode to obtain the animation of the virtual object; In step 1032, when the interaction method is not the first interaction, the virtual object is subjected to animation simulation processing based on the previous interaction method and the target animation mode to obtain the animation of the virtual object.

[0130] For example, when the interaction is the first time, the animation of the virtual object generated by either machine learning-based animation or skeletal animation will not have a mismatch in performance. Therefore, when the interaction is the first time, the virtual object can be simulated based solely on the interaction method. However, when the interaction is not the first time, a mismatch may occur between the current and previous interaction methods (the last frame of the previous animation for the previous interaction method may not match the first frame of the current animation for the current interaction method). For example, if the virtual object's left foot is forward in the last frame of the previous animation, but its right foot is forward in the first frame of the current animation, this will result in a sudden change in movement and an unnatural appearance. Therefore, when the interaction is not the first time, it is necessary to simulate the virtual object based on the previous interaction method and the target animation mode to obtain the virtual object's animation.

[0131] In some embodiments, when the interaction method is the first interaction, the animation simulation process is as follows: when the target animation mode is a machine learning-based animation mode, the virtual object is processed by machine learning-based animation generation based on the interaction method to obtain the animation of the virtual object; when the target animation mode is an animation mode based on playing skeletal animation, the virtual object is processed by skeletal animation matching based on the interaction method to obtain the animation of the virtual object.

[0132] For example, when the target animation mode is a machine learning-based animation mode, machine learning techniques are used in conjunction with object parameters in the interaction method to generate animations for virtual objects. When the target animation mode is a skeletal animation-based animation mode, state machine logic or hybrid tree logic is used in conjunction with object parameters in the interaction method to perform animation matching processing on virtual objects, resulting in animations for virtual objects. The object parameters characterize the parameters that the virtual object will interact with, such as the virtual object's phase information, skeletal state (position, velocity, rotation, etc. of each bone), and control trajectory.

[0133] In some embodiments, when the interaction method is not the first interaction, the animation simulation process is as follows: when the target animation mode is a machine learning-based animation mode, the virtual object is processed by machine learning-based animation generation based on the previous interaction method to obtain the animation of the virtual object; when the target animation mode is an animation mode based on playing skeletal animation, the virtual object is processed by skeletal animation matching based on the previous interaction method to obtain the animation of the virtual object.

[0134] For example, when the target animation mode is a machine learning-based animation mode, machine learning techniques are used to combine object parameters from the previous interaction mode (e.g., object parameters from the last frame of the previous animation corresponding to the previous interaction mode, also known as state parameters) to generate animation for the virtual object. When the target animation mode is a skeletal animation-based animation mode, state machine logic or hybrid tree logic is used to combine object parameters from the previous interaction mode to perform animation matching processing on the virtual object to obtain its animation. The object parameters characterize the parameters of the virtual object when it will interact, such as the virtual object's phase information, skeletal state (position, velocity, rotation, etc. of each bone), and control trajectory.

[0135] In some embodiments, the virtual object is subjected to machine learning-based animation generation processing based on the previous interaction method to obtain the animation of the virtual object, including: obtaining the last frame animation image of the previous interaction method; calling a machine learning model to perform animation prediction processing based on the state parameters of the last frame animation image to obtain the animation of the virtual object; wherein the first frame animation image of the virtual object's animation is matched with the last frame animation image of the previous interaction method.

[0136] For example, when transitioning from an animation mode based on skeletal animation (the animation mode corresponding to the previous interaction) to a machine learning-based animation mode (the animation mode corresponding to the current interaction), state parameters (i.e., the state parameters of the last frame of the animation from the previous interaction) need to be input into the machine learning model. This ensures that the output action (i.e., the action of the virtual object in the first frame of the current interaction's animation) closely approximates the cut-out action of the source animation (the animation generated by the skeletal animation mode), resulting in a natural transition. State parameters include phase information, skeletal states (position, velocity, rotation, etc. of each bone), and control trajectories. Phase information can be obtained from the animation information of the last frame (this requires pre-annotation or pre-calculation by the program). Skeletal states can be read from the current skeletal states, and control trajectories can be calculated based on the motion state at the time of the cut-out from the source animation.

[0137] In some embodiments, the virtual object is subjected to skeletal animation matching processing based on the previous interaction mode to obtain the animation of the virtual object, including: obtaining the last frame animation image of the previous interaction mode; determining the target skeletal animation that matches the interaction mode among the skeletal animations in multiple state machines; determining the target animation image that matches the last frame animation image among multiple animation images of the target skeletal animation; and using the target animation image as the starting point of the animation to perform truncation processing on the target skeletal animation to obtain the animation of the virtual object.

[0138] For example, when transitioning from a machine learning-based animation mode to a skeletal animation mode, a target skeletal animation that matches the interaction method is determined from the skeletal animations in multiple state machines. It should be noted that the phase information of the left and right feet in the skeletal animation is also easily obtained (through manual annotation or program pre-calculation), allowing the transition to begin playback from the phase-matching point in the target skeletal animation, achieving a smooth transition.

[0139] like Figure 10 As shown, the last frame of the previous animation (source animation) corresponding to the last interaction is a running motion generated by machine learning, with the left foot forward, while the first frame of the skill animation (i.e., the target skeletal animation) has the right foot forward. If the animation starts playing directly from the first frame of the skill animation, the transition will be very unnatural. However, if the best matching target animation image is found according to the above scheme, and the playback start time of the target animation image is used as the optimal entry point (…), then… Figure 10 The optimal entry point 1001 shown in the diagram is also the state where the left foot is in front. Then, the skeletal animation is played directly from this optimal entry point. For example, the optimal entry point is used as the starting point of the animation and the last frame of the target skeletal animation is used as the ending point of the animation to achieve the interception and processing of the target skeletal animation, thereby achieving a natural transition.

[0140] In some embodiments, determining a target animation image that matches the last frame animation image includes: when multiple animation images include phase parameters, determining the animation image that matches the phase parameters of the last frame animation image as the target animation image; when multiple animation images do not include phase parameters, performing motion matching on the last frame animation image and the multiple animation images to obtain a target animation image that matches the last frame animation image.

[0141] For example, the animated images generated by machine learning models inherently possess phase information, and the phase information of the left and right feet in skeletal animation images is also easily obtained (through manual annotation or pre-calculation by the program). If the animated image includes phase information, matching is performed directly based on this information. For instance, if the phase parameter of the last frame of the animated image is left foot forward, then the phase parameter of the target animated image will also be left foot forward. If the animated image does not include phase information, then motion matching technology is used to select the target animated image for matching. For instance, if motion matching technology determines that the motion of the last frame of the animated image is left foot forward, then the motion of the target animated image will also be left foot forward.

[0142] In step 104, the animation of the virtual object is played based on the virtual scene.

[0143] For example, after determining the animation of a virtual object through animation simulation, a virtual scene can be incorporated into the animation of the virtual object. For example, if the animation of the virtual object is a walking animation, then the background of the virtual scene (such as virtual mountains or virtual roads) can be incorporated into the walking animation, and the animation of the virtual object can be played to form an animation of the virtual object walking on a virtual road.

[0144] In some embodiments, when the target animation mode is a machine learning-based animation mode, the playback frequency of the virtual object's animation is obtained; when the playback frequency is greater than the playback frequency threshold, the animation of the virtual object is determined to be a skeletal animation based on the skeletal animation mode; when the subsequent interaction mode matches the interaction mode, the virtual object is subjected to animation simulation processing based on the skeletal animation to obtain the animation corresponding to the subsequent interaction mode.

[0145] For example, when an animation generated by a machine learning-based animation pattern is played frequently (i.e., the playback frequency exceeds a threshold), the animation generated by the machine learning-based animation pattern is identified as a skeletal animation based on the skeletal animation playback pattern. The animation pattern matching the current interaction method is updated to the skeletal animation playback pattern, and the animation generated by the machine learning-based animation pattern is stored in the skeletal animation library (including multiple skeletal animations). Thus, when subsequent interaction methods match the current interaction method, the virtual object is animated using the skeletal animation playback pattern to obtain the animation corresponding to the subsequent interaction method, avoiding increasing the size of the machine learning model and thus increasing performance overhead.

[0146] In some embodiments, when the target animation mode is an animation mode based on playing skeletal animation, the playback frequency of the virtual object's animation is obtained; when the playback frequency is less than the playback frequency threshold, the animation mode matching the interaction mode is updated to an animation mode based on machine learning; when the subsequent interaction mode matches the interaction mode, the virtual object is processed by the animation mode based on machine learning to obtain the animation corresponding to the subsequent interaction mode.

[0147] For example, if the animation generated by the skeletal animation-based animation mode is rarely played (i.e., the playback frequency is less than the playback frequency threshold), the animation mode matching the current interaction method is updated to a machine learning-based animation mode, and the skeletal animation corresponding to the current interaction method is deleted from the skeletal animation library (including multiple skeletal animations) to reduce the storage space occupied by the skeletal animation-based animation mode. Therefore, when a subsequent interaction method matches the current interaction method, the machine learning-based animation mode is used to simulate the animation of the virtual object, obtaining the animation corresponding to the subsequent interaction method, further reducing the storage space occupied by the skeletal animation-based animation mode.

[0148] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0149] The embodiments of this application can be applied to animation scenes in various games, such as competitive games, racing games, and dress-up games.

[0150] The following explanation uses a virtual game scenario as an example:

[0151] This application proposes an animation implementation scheme that combines machine learning-generated animation with skeletal animation playback (i.e., an animation processing method for virtual scenes). This avoids the complex animation state machine logic of skeletal animation playback, thus shortening the animation development cycle. It only requires storing the trained machine learning model and a small amount of animation resources, thus avoiding excessive storage resource consumption. For example, for actions that are difficult for machine learning models to learn (or other scenarios requiring high-quality animation), skeletal animation playback is used to avoid uncontrollable animation effects. For walking or running in a straight line on flat ground (or other high-frequency repetitive actions), skeletal animation playback is used, greatly reducing performance overhead and preventing distortion of these high-exposure actions, thereby improving the animation effect.

[0152] It should be noted that the embodiments of this application can be used in any product that requires skeletal animation, such as games and animations. It can be used on virtual characters controlled by players, as well as on non-user characters (NPCs, Non-Player Characters) controlled by the program. Figure 6 The animation shown depicts a character walking and running in a straight line on flat ground (also known as movement). The virtual character 601 walks in a straight line from position 602 to position 603. Figure 7 The animation shown depicts a virtual character 701 turning and walking on flat ground, following a turning trajectory 702. Figure 8 The animation shows walking and running in a complex terrain scene. The virtual character 801 walks downhill from position 802 to position 803.

[0153] For character movements in games, skeletal animation is generally used. This skeletal animation is typically provided by motion artists, who usually create it manually or using motion capture technology. To achieve better visual effects, skeletal animation needs to cover various combinations of scenes, such as different speeds, different turning angles, and different terrain slopes. These combinations result in massive amounts of data. On one hand, motion artists need to spend a significant amount of time creating animations; on the other hand, technical staff also need to spend considerable time developing animation state machine logic or hybrid tree logic. This logic determines when to play which animation (for example, using a hybrid tree to select the animation to play based on control signal input: playing a walking animation for slow speeds, a turning animation for turning angles, and a walking / running animation for uphill slopes).

[0154] While motion matching technology avoids the development and configuration of complex animation state machines or hybrid trees, it introduces performance and storage overhead issues. To address these issues, a machine learning-based motion generation technique is proposed. By learning from a large amount of raw animation data (mass-produced through motion capture technology), animations can be directly generated. This not only avoids the storage overhead associated with large amounts of animation but also significantly reduces the overall animation production cycle. Furthermore, due to the generalization ability of machine learning, it can automatically generate suitable animations even for scenes not covered by the original animations.

[0155] The following is combined Figure 9 The flowchart illustrates the animation implementation scheme combining machine learning-generated animation and skeletal animation playback in this application embodiment:

[0156] Step 1: Input control signals.

[0157] It should be noted that the input control signals include the target velocity vector of the virtual character, terrain data (such as the current slope or the height of future trajectory points), etc., which are used to control the actions of the virtual character.

[0158] Step 2: Based on the control signal, determine whether to use the skeletal animation playback scheme.

[0159] For non-walking (Locomotion) animation scenes (the animation scene can be determined by control signals), the skeletal animation playback scheme is adopted; for Locomotion animation scenes, if it is necessary to generate high-frequency looping animation (such as walking or running in a straight line on flat ground), the skeletal animation playback scheme is also adopted, otherwise the machine learning animation generation scheme is adopted.

[0160] For non-Locomotion animation scenarios, such as skill actions (e.g., jump attacks) and stationary performance animations, while machine learning-generated animation solutions can achieve the same results, they increase the size of the machine learning model, thus increasing performance overhead. Furthermore, machine learning techniques are more prone to distortion when generating these types of animations, affecting the overall animation quality. Using skeletal animation, on the other hand, avoids these problems and is simple and effective. Therefore, when implementing machine learning-generated animation solutions in games, it's necessary to combine them with skeletal animation solutions. This allows each solution to leverage its strengths and gradually allow the machine learning approach to cover more and more actions.

[0161] For Locomotion animation scenarios, while machine learning can generate animations for high-frequency looping animations, this approach suffers from loss of detail and high performance overhead. Therefore, this application proposes using skeletal animation to improve both performance and visual quality for high-frequency looping animations. For example, in animations of walking or running on flat ground, NPCs move primarily through navigation logic, mostly in straight lines. Since most hotspot maps in the game are flat, NPCs primarily perform this straight-line walking / running action. While this action can be generated using machine learning, it can also be achieved by directly playing skeletal animation due to its fixed nature. Therefore, this application proposes directly playing straight-line walking / running animations instead of machine learning. This is because the overhead of playing skeletal animations is significantly lower than that of machine learning-generated animations, thus improving performance. Furthermore, playing skeletal animations avoids any distortion and provides fully controllable visual effects. For instance, playing a longer straight-line animation can prevent monotony and enhance the overall visual experience.

[0162] For other scenarios of Locomotion animation, a machine learning-based animation generation solution is adopted, which can better cover different scenarios and avoid storage issues.

[0163] It should be noted that a skeletal animation playback scheme is used for high-frequency looping animations. The following example illustrates this using a straight-line walking / running scene on flat ground. When using machine learning to generate animations, the transition to skeletal animation playback will occur when all of the following conditions are met:

[0164] (1) The current virtual character’s orientation is consistent with the current driving speed direction (used to determine whether the virtual character is turning or walking), that is, the angle difference between the current character’s orientation and the driving speed direction is less than the angle threshold 1;

[0165] (2) The current speed of the virtual character is close to the target speed (used to determine whether the virtual character is changing speed), that is, the speed difference between the current speed and the target speed is less than the speed threshold.

[0166] (3) The current terrain is close to flat ground (used to determine whether the virtual character is walking up or down a slope), that is, the slope is less than the slope threshold.

[0167] When transitioning to playing skeletal animation, you can choose an animation that matches the target speed or the target gait (such as walking, running, sprinting, etc.) as needed.

[0168] When playing skeletal animation, the transition to machine learning-generated animation will occur when any of the following conditions are met:

[0169] (1) The current driving speed direction is inconsistent with the current virtual character's facing direction, that is, the angle difference between the current character's facing direction and the driving speed direction is greater than the angle threshold 2. To avoid frequent switching, the angle threshold 2 is greater than the angle threshold 1, and other thresholds are similar.

[0170] (2) The target speed is inconsistent with the current speed, that is, the speed difference between the current speed and the target speed is greater than the speed threshold;

[0171] (3) The current terrain is not flat, that is, the slope is greater than the slope threshold.

[0172] When an animation is about to end, it can directly transition to the next animation without having to transition back to machine learning-generated animation, thus reducing the overhead of transitions.

[0173] For example, if the current driving speed direction is different from the current virtual character speed direction, but the error does not exceed the angle threshold of 2, a slow forced turn can be made to correct the current virtual character speed. This can reduce the probability of frequently transitioning back to machine learning-generated animation, thereby further optimizing performance.

[0174] Step 3: Determine the first frame node.

[0175] For the skeletal animation playback solution, when switching from machine learning-generated animation to skeletal animation playback, the current node is determined to be the first frame node (initially considered the first frame node, but after executing step 4, it is no longer the first frame node, and returns to the first frame node after the skeletal animation finishes playing or playback exits). When transitioning from machine learning-generated animation to skeletal animation playback, an appropriate animation and a suitable frame need to be selected at the first frame node for the transition.

[0176] Step 4: Match the appropriate animation frames.

[0177] When playing skeletal animation, if the left and right feet of the virtual character do not match during the transition (for example, the left foot is in front in the last frame of the source animation (the animation before the transition), but the right foot is in front in the first frame of the target animation (the animation after the transition), there will be a sudden change in the action, which looks very unnatural. Therefore, the embodiments of this application propose a phase matching-based solution, which can effectively solve the problem of mismatch between the left and right feet.

[0178] For scenarios transitioning from machine learning-generated animation to skeletal animation, the animation images generated by the machine learning model inherently possess left and right foot phase information. The left and right foot phase information in the skeletal animation is also easily obtained (through manual annotation or pre-calculation by the program). Therefore, the transition begins from the phase-matching point in the target animation, achieving a smooth transition. If there are multiple target animations, or if the target animation lacks left and right foot phase information, motion matching technology can be used to select the most matching target animation, and even the starting time point of the matching target animation's playback can be chosen.

[0179] like Figure 10 As shown, the last frame of the source animation is a running motion generated by machine learning, with the left foot forward, while the first frame of the skill animation (i.e., the animation generated by the skeletal animation scheme) has the right foot forward. If the animation starts playing directly from the first frame of the skill animation, the transition will be very unnatural. However, if the most suitable animation playback start time is found as the optimal entry point according to the above scheme (…),… Figure 10 The optimal entry point 1001 shown in the diagram is also the state where the left foot is forward. Then, the skeletal animation is played directly from the optimal entry point 1001 to achieve a natural transition.

[0180] Step 5: Output and play the animation generated by the skeletal animation scheme.

[0181] It should be noted that the target animation is played based on the target animation selected in step 4 and the animation playback start time point in order to output the animation's motion.

[0182] Step 6: Determine the first frame node.

[0183] For the machine learning-generated animation scheme, determine whether the current frame is the first frame node (it is the first frame node when switching from playing skeletal animation, but is no longer the first frame node after executing step 7). When transitioning from playing skeletal animation to machine learning-generated animation, the state parameters need to be written back at the first frame node to ensure a smooth transition between the machine learning-generated actions and the playing skeletal animation.

[0184] Step 7: Write back the status parameters.

[0185] For scenarios transitioning from playing skeletal animation (source animation) to machine learning-generated animation (target animation), initial parameters (i.e., state parameters) need to be input into the machine learning model to ensure that the output animation closely resembles the cut-out animation of the source animation, thus making the transition natural. The input to the machine learning model includes phase information, skeletal states (position, velocity, rotation, etc. of each bone), and control trajectories. Phase information can be obtained from the animation information (it needs to be generated through pre-annotation or program pre-calculation), skeletal states can be read from the current skeletal state, and control trajectories can be calculated based on the motion state at the time of the cut-out from the source animation.

[0186] Step 8: Output the animation generated by the machine learning animation generation scheme.

[0187] In summary, the embodiments of this application not only solve the storage resource consumption problem of the skeletal animation playback scheme, but also solve the animation distortion problem of the machine learning generated animation scheme; for the animation transition between the skeletal animation playback scheme and the machine learning generated animation scheme, phase matching and motion matching technologies are used to achieve a smooth and natural transition; for high-frequency scenes such as walking and running on flat ground, using the skeletal animation playback scheme instead of the machine learning generated animation scheme not only makes the performance effect more controllable, but also reduces the performance overhead caused by machine learning forward inference, especially in scenes with a large number of characters.

[0188] The exemplary application and implementation of the terminal or server provided in the embodiments of this application have been used to describe the animation processing method of the virtual scene provided in the embodiments of this application. The following describes the cooperation of the various modules in the animation processing device 465 of the virtual scene provided in the embodiments of this application to realize the animation processing scheme of the virtual scene.

[0189] The acquisition module 4651 is used to acquire the interaction mode corresponding to when the virtual object will interact in the virtual scene; the determination module 4652 is used to determine the target animation mode that matches the interaction mode among the machine learning-based animation mode and the skeletal animation playback mode; the simulation module 4653 is used to perform animation simulation processing on the virtual object based on the target animation mode that matches the interaction mode to obtain the animation of the virtual object; wherein, the animation is used to characterize the interaction mode of the virtual object in the virtual scene; and the playback module 4654 is used to play the animation of the virtual object based on the virtual scene.

[0190] In some embodiments, the determining module 4652 is further configured to, when determining that the interaction mode is a non-loop movement mode, use the machine learning-based animation mode as the target animation mode that matches the interaction mode; and when determining that the interaction mode is a non-movement mode or a loop movement mode, determine the animation mode based on playing skeletal animation as the target animation mode that matches the interaction mode; wherein, the types of non-movement modes include: skill interaction and performance interaction.

[0191] In some embodiments, before determining the target animation mode that matches the interaction mode, the determining module 4652 is further configured to determine the interaction mode as the non-cyclic movement mode when at least one of the interaction parameters in the interaction mode is greater than a first interaction parameter threshold; and to determine the interaction mode as the cyclic movement mode when all the interaction parameters in the interaction mode are less than or equal to the first interaction parameter threshold; wherein, the types of the interaction parameters include: the angle difference between the torso orientation of the virtual object and the direction of movement speed is less than a first angle threshold, the speed difference between the movement speed of the virtual object and the target movement speed, and the slope of the terrain on which the virtual object moves.

[0192] In some embodiments, when the previous interaction mode of the interaction mode is the animation mode based on playing skeletal animation, the determining module 4652 is further configured to adjust the interaction parameters in the interaction mode and determine the interaction mode as the cyclic movement mode when the interaction parameters in the interaction mode are greater than the first interaction parameter threshold and less than the second interaction parameter threshold; wherein, the second interaction parameter threshold is greater than the first interaction parameter threshold.

[0193] In some embodiments, the simulation module 4653 is further configured to, when the interaction method is the first interaction, perform animation simulation processing on the virtual object based on the interaction method and the target animation mode to obtain the animation of the virtual object; when the interaction method is not the first interaction, perform animation simulation processing on the virtual object based on the previous interaction method and the target animation mode to obtain the animation of the virtual object.

[0194] In some embodiments, the simulation module 4653 is further configured to perform machine learning-based animation generation processing on the virtual object based on the interaction method when the target animation mode is the machine learning-based animation mode, to obtain the animation of the virtual object; and to perform skeletal animation-based animation matching processing on the virtual object based on the interaction method when the target animation mode is the animation mode based on playing skeletal animation, to obtain the animation of the virtual object.

[0195] In some embodiments, the simulation module 4653 is further configured to, when the target animation mode is the machine learning-based animation mode, perform machine learning-based animation generation processing on the virtual object based on the previous interaction mode of the interaction mode to obtain the animation of the virtual object; and when the target animation mode is the animation mode based on playing skeletal animation, perform skeletal animation matching processing on the virtual object based on the previous interaction mode of the interaction mode to obtain the animation of the virtual object.

[0196] In some embodiments, the simulation module 4653 is further configured to acquire the last frame of the animation image of the previous interaction; call a machine learning model to perform animation prediction processing based on the state parameters of the last frame of the animation image to obtain the animation of the virtual object; wherein the first frame of the animation of the virtual object matches the last frame of the animation image of the previous interaction.

[0197] In some embodiments, the simulation module 4653 is further configured to acquire the last frame animation image of the previous interaction mode; determine the target skeletal animation that matches the interaction mode among the skeletal animations in multiple state machines; determine the target animation image that matches the last frame animation image among the multiple animation images of the target skeletal animation; and use the target animation image as the starting point of the animation to perform truncation processing on the target skeletal animation to obtain the animation of the virtual object.

[0198] In some embodiments, the simulation module 4653 is further configured to, when the plurality of animation images include phase parameters, determine the animation image that matches the phase parameters of the last frame animation image as the target animation image; and when the plurality of animation images do not include phase parameters, perform motion matching on the last frame animation image and the plurality of animation images to obtain a target animation image that matches the last frame animation image.

[0199] In some embodiments, the simulation module 4653 is further configured to: obtain the playback frequency of the animation of the virtual object when the target animation mode is the machine learning-based animation mode; determine the animation of the virtual object as the skeletal animation of the animation mode based on playing skeletal animation when the playback frequency is greater than the playback frequency threshold; and perform animation simulation processing on the virtual object based on the skeletal animation when the subsequent interaction mode matches the interaction mode, thereby obtaining the animation corresponding to the subsequent interaction mode.

[0200] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the virtual scene animation processing method described above in this application.

[0201] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the animation processing method for a virtual scene provided in this application. For example, ... Figures 3-5 The animation processing method for the virtual scene is shown.

[0202] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0203] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0204] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0205] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0206] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for animation processing of a virtual scene, characterized in that, The method includes: To obtain the interaction method corresponding to when a virtual object interacts with another virtual object in a virtual scene; In both machine learning-based animation modes and skeletal animation-based animation modes, when the interaction method is determined to be a non-cyclic movement method, the machine learning-based animation mode is used as the target animation mode that matches the interaction method; the non-cyclic movement method refers to the non-cyclic, fixed movement behavior of the virtual object in the virtual scene. When the interaction method is determined to be a non-movement method or a looping movement method, the animation mode based on playing skeletal animation is determined as the target animation mode that matches the interaction method; wherein, the types of non-movement methods include: skill interaction and performance interaction; the looping movement method refers to the looping and fixed movement behavior of the virtual object in the virtual scene; The virtual object is animated based on a target animation pattern that matches the interaction method to obtain the animation of the virtual object; wherein the animation is used to characterize the interaction method of the virtual object in the virtual scene; The animation of the virtual object is played based on the virtual scene.

2. The method according to claim 1, characterized in that, The method further includes: When at least one of the interaction parameters in the interaction method is greater than the first interaction parameter threshold, the interaction method is determined as the non-cyclic movement method; When all the interaction parameters in the interaction mode are less than or equal to the first interaction parameter threshold, the interaction mode is determined to be the cyclic movement mode; The types of interactive parameters include: the angle difference between the virtual object's torso orientation and the direction of its movement speed is less than a first angle threshold; the speed difference between the virtual object's movement speed and the target's movement speed; and the slope of the terrain on which the virtual object moves.

3. The method according to claim 2, characterized in that, When the previous interaction mode was the animation mode based on playing skeletal animation, the method further includes: When the interaction parameter in the interaction mode is greater than the first interaction parameter threshold and less than the second interaction parameter threshold, the interaction parameter in the interaction mode is adjusted, and the interaction mode is determined as the cyclic movement mode; Wherein, the second interaction parameter threshold is greater than the first interaction parameter threshold.

4. The method according to claim 1, characterized in that, The process of simulating animation on the virtual object based on a target animation pattern that matches the interaction method to obtain the animation of the virtual object includes: When the interaction method is the first interaction, the virtual object is subjected to animation simulation processing based on the interaction method and the target animation mode to obtain the animation of the virtual object; When the interaction method is not the first interaction, the virtual object is subjected to animation simulation processing based on the previous interaction method and the target animation mode to obtain the animation of the virtual object.

5. The method according to claim 4, characterized in that, The step of performing animation simulation processing on the virtual object based on the interaction method and the target animation mode to obtain the animation of the virtual object includes: When the target animation mode is the machine learning-based animation mode, the virtual object is subjected to machine learning-based animation generation processing based on the interaction method to obtain the animation of the virtual object; When the target animation mode is the animation mode based on playing skeletal animation, the virtual object is subjected to skeletal animation matching processing based on the interaction method to obtain the animation of the virtual object.

6. The method according to claim 4, characterized in that, The animation simulation process for the virtual object, based on the previous interaction method and the target animation mode, to obtain the animation of the virtual object includes: When the target animation mode is the machine learning-based animation mode, the virtual object is subjected to machine learning-based animation generation processing based on the previous interaction mode to obtain the animation of the virtual object; When the target animation mode is the animation mode based on playing skeletal animation, the virtual object is subjected to skeletal animation matching processing based on the previous interaction mode to obtain the animation of the virtual object.

7. The method according to claim 6, characterized in that, The process of generating an animation for the virtual object based on the previous interaction method, using machine learning, to obtain the animation of the virtual object includes: Obtain the last frame of the animation image from the previous interaction. Based on the state parameters of the last frame of the animation image, a machine learning model is invoked to perform animation prediction processing to obtain the animation of the virtual object; The first frame of the animation of the virtual object is matched with the last frame of the animation of the previous interaction.

8. The method according to claim 6, characterized in that, The process of performing skeletal animation matching on the virtual object based on the previous interaction method to obtain the animation of the virtual object includes: Obtain the last frame of the animation image from the previous interaction. In the skeletal animation across multiple state machines, determine the target skeletal animation that matches the interaction method; Among the multiple animated images of the target skeletal animation, determine the target animated image that matches the last frame of the animated image; Using the target animation image as the starting point, the target skeletal animation is cropped to obtain the animation of the virtual object.

9. The method according to claim 8, characterized in that, The process of determining the target animation image that matches the last frame of the animation image includes: When the plurality of animated images include phase parameters, the animated image that matches the phase parameters of the last animated image is determined as the target animated image; When the plurality of animation images do not include phase parameters, motion matching is performed on the last frame animation image and the plurality of animation images to obtain a target animation image that matches the last frame animation image.

10. The method according to claim 1, characterized in that, The method further includes: When the target animation mode is the machine learning-based animation mode, the playback frequency of the animation of the virtual object is obtained; When the playback frequency is greater than the playback frequency threshold, the animation of the virtual object is determined as the skeletal animation of the animation mode based on the playback skeletal animation. When a subsequent interaction matches the first interaction, the virtual object is animated based on the skeletal animation to obtain the animation corresponding to the subsequent interaction.

11. An animation processing device for a virtual scene, characterized in that, The device includes: The acquisition module is used to acquire the interaction methods corresponding to when virtual objects will interact in the virtual scene. A determination module is used to, in the context of machine learning-based animation modes and skeletal animation-based animation modes, when the interaction mode is determined to be a non-looping movement mode, select the machine learning-based animation mode as the target animation mode matching the interaction mode; the non-looping movement mode refers to the non-looping, fixed movement behavior performed by the virtual object in the virtual scene; when the interaction mode is determined to be a non-movement mode or a looping movement mode, the skeletal animation-based animation mode is determined as the target animation mode matching the interaction mode; wherein, the types of non-movement modes include: skill interaction and performance interaction; the looping movement mode refers to the looping, fixed movement behavior performed by the virtual object in the virtual scene; The simulation module is used to perform animation simulation processing on the virtual object based on a target animation mode that matches the interaction mode, so as to obtain the animation of the virtual object; wherein, the animation is used to characterize the interaction mode of the virtual object in the virtual scene; The playback module is used to play the animation of the virtual object based on the virtual scene.

12. The apparatus according to claim 11, characterized in that, The determining module is further configured to: When at least one of the interaction parameters in the interaction method is greater than the first interaction parameter threshold, the interaction method is determined as the non-cyclic movement method; When all the interaction parameters in the interaction mode are less than or equal to the first interaction parameter threshold, the interaction mode is determined to be the cyclic movement mode; The types of interactive parameters include: the angle difference between the virtual object's torso orientation and the direction of its movement speed is less than a first angle threshold; the speed difference between the virtual object's movement speed and the target's movement speed; and the slope of the terrain on which the virtual object moves.

13. The apparatus according to claim 12, characterized in that, The simulation module is also used for: When the interaction method is the first interaction, the virtual object is subjected to animation simulation processing based on the interaction method and the target animation mode to obtain the animation of the virtual object; When the interaction method is not the first interaction, the virtual object is subjected to animation simulation processing based on the previous interaction method and the target animation mode to obtain the animation of the virtual object.

14. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the animation processing method of the virtual scene according to any one of claims 1 to 10.

15. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the animation processing method of the virtual scene as described in any one of claims 1 to 10 when executed by a processor.

16. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the animation processing method for the virtual scene as described in any one of claims 1 to 10.

Images