A motion information determination method, device and computer readable storage medium

Abstract

The application provides a motion information determination method, device and computer readable storage medium; the method comprises the following steps: obtaining the i-th frame motion information of a motion object and target position information, i is a positive integer; based on the i-th frame motion information and the target position information, predicting the i+1-th frame motion update amount of the motion object and the estimated target position information based on the i+1-th frame motion update amount; based on the difference between the estimated target position information and the target position information, correcting the i+1-th frame motion update amount, and obtaining the i+1-th frame motion information according to the corrected i+1-th frame motion update amount; continuing iteration until the i+n-th frame motion information corresponding to the target position information is obtained, n is a positive integer greater than 1; obtaining a motion frame sequence comprising the i-th frame motion information to the i+n-th frame motion information, and the motion frame sequence is a motion information set of the motion object performing a target-oriented action for the target position information. Through the application, the accuracy of the motion frame sequence can be improved based on artificial intelligence.

Description

Technical Field

[0001] This application relates to information processing technology in the field of artificial intelligence, and more particularly to a method, device and computer-readable storage medium for determining motion information. Background Technology

[0002] With the rapid development of artificial intelligence, people have increasingly higher requirements for the movement of objects; for example, determining the movement information of objects performing goal-oriented actions (such as opening a door, sitting on a chair, or moving a box).

[0003] Generally, in order to determine the motion information of an object performing a target-oriented action, the current motion state information of the moving object and the target position are usually used as inputs to a neural network model, and then the motion information of the moving object performing the target-oriented action is determined based on the output of the neural network model. However, in the process of determining the motion information of the moving object performing the target-oriented action, the position of the last frame of motion information output by the neural network model often deviates from the target position, thus the accuracy of the determined motion information of the moving object performing the target-oriented action is low. Summary of the Invention

[0004] This application provides a motion information determination method, device, and computer-readable storage medium, which can improve the accuracy of motion information of a determined moving object performing a target-guided action.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a method for determining motion information, including:

[0007] Obtain the motion information and target position information of the moving object in the i-th frame, where i is a positive integer;

[0008] Based on the motion information of the i-th frame and the target position information, predict the motion update amount of the moving object in the (i+1)-th frame and the estimated target position information based on the motion update amount of the (i+1)-th frame;

[0009] Based on the difference between the estimated target location information and the target location information, the motion update amount of the (i+1)th frame is corrected, and the motion information of the (i+1)th frame is obtained according to the corrected motion update amount of the (i+1)th frame.

[0010] Continue iterating until the (i+n)th frame of motion information corresponding to the target position information is obtained, where n is a positive integer greater than 1;

[0011] Obtain a motion frame sequence including the motion information of the i-th frame to the motion information of the (i+n)-th frame, wherein the motion frame sequence is a set of motion information of the moving object performing a target-guided action in response to the target position information.

[0012] This application provides a motion information determination device, including:

[0013] The information acquisition module is used to acquire the motion information and target position information of the i-th frame of the moving object, where i is a positive integer;

[0014] The information prediction module is used to predict the motion update amount of the moving object in the (i+1)th frame and the estimated target position information based on the motion information of the i-th frame and the target position information.

[0015] The information correction module is used to correct the motion update amount of the (i+1)th frame based on the difference between the estimated target location information and the target location information, and to obtain the motion information of the (i+1)th frame based on the corrected motion update amount of the (i+1)th frame.

[0016] The information iteration module is used to continue iterating until the (i+n)th frame of motion information corresponding to the target position information is obtained, where n is a positive integer greater than 1;

[0017] The information determination module is used to obtain a motion frame sequence including the motion information of the i-th frame to the motion information of the (i+n)-th frame, wherein the motion frame sequence is a set of motion information of the moving object performing a target-guided action in response to the target position information.

[0018] In this embodiment, the motion update amount of the (i+1)th frame includes the object point position update amount of the (i+1)th frame, the motion information of the i-th frame includes the object point position information of the i-th frame, and the motion information of the (i+1)th frame includes the object point position information of the (i+1)th frame. The information correction module is further configured to perform vector difference calculation based on the estimated target position information and the target position information to obtain an initial correction vector; adjust the initial correction vector based on the correction coefficient to obtain a correction vector; use the correction vector to correct the object point position update amount of the (i+1)th frame, and then superimpose the corrected object point position update amount of the (i+1)th frame with the object point position information of the i-th frame to obtain the object point position information of the (i+1)th frame in the motion information of the (i+1)th frame.

[0019] In this embodiment of the application, the correction coefficient is positively correlated with the speed of the moving object.

[0020] In this embodiment of the application, when the motion distance between the object point position information in the i-th frame and the target position information falls within the distance range, the correction coefficient is calculated based on the motion distance and the object point position update amount in the (i+1)-th frame; when the motion distance falls outside the distance range, the correction coefficient is a constant.

[0021] In this embodiment, when the moving object is a virtual object, the motion information of the (i+1)th frame further includes the position information of the object part in the (i+1)th frame, and the motion update amount of the (i+1)th frame further includes the relative amount of the object part position in the (i+1)th frame. The information correction module is further configured to superimpose the relative amount of the object part position in the (i+1)th frame onto the object point position information in the (i+1)th frame to obtain the position information of the object part to be adjusted in the (i+1)th frame; superimpose both the relative amount of the object part position in the (i+1)th frame and the object point position update amount in the (i+1)th frame onto the object point position information in the (i)th frame to obtain the reference position information of the object part in the (i+1)th frame; and adjust the position information of the object part to be adjusted in the (i+1)th frame based on the reference position information of the object part in the (i+1)th frame to obtain the position information of the object part in the (i+1)th frame in the motion information.

[0022] In this embodiment, the reference position information of the object part in the (i+1)th frame includes ankle reference position information and toe reference position information, and the position information of the object part to be adjusted in the (i+1)th frame includes hip position information to be adjusted, knee position information to be adjusted, and ankle position information to be adjusted. The information correction module is further configured to adjust the ankle position information to be adjusted to the ankle reference position information by rotating the knee position information to be adjusted and the hip position information to be adjusted, thereby determining the knee position information in the (i+1)th frame; and to determine the knee position information, the hip position information to be adjusted, the ankle reference position information, and the toe reference position information in the (i+1)th frame motion information as the position information of the object part in the (i+1)th frame.

[0023] In this embodiment, the information correction module is further configured to: determine the knee rotation direction based on the direction from the knee position information to the ankle position information to be adjusted and the direction from the knee position information to the hip position information to be adjusted; rotate the knee position information to be adjusted about the knee rotation direction as the axis of rotation to adjust the ankle position information to be adjusted based on the distance between the hip position information to be adjusted and the ankle reference position information; determine the hip rotation direction based on the direction from the hip position information to the ankle reference position information and the direction from the hip position information to the adjusted ankle position information; and rotate the hip position information to be adjusted about the hip rotation direction as the axis of rotation to adjust the target ankle position information to the ankle reference position information, thereby determining the (i+1)th frame knee position information.

[0024] In this embodiment of the application, the information prediction module is further configured to use a motion prediction model to predict the motion information of the i-th frame and the target position information to obtain the motion update amount of the moving object in the (i+1)-th frame and the estimated target position information based on the motion update amount of the (i+1)-th frame, wherein the motion prediction model is used to predict the motion information of the moving object.

[0025] In this embodiment, the motion information determination device further includes a model training module for acquiring model training samples, wherein the model training samples include at least one frame of motion samples and a target position sample; using a motion prediction model to be trained to predict the j-th frame of motion samples and the target position sample in the at least one frame of motion samples to obtain the (j+1)-th frame of motion information, wherein the motion prediction model to be trained is a model to be trained for predicting motion information, and j is a positive integer greater than 1; based on the difference between the (j+1)-th frame of motion information and the (j+1)-th frame of motion samples, the motion prediction model to be trained is trained to obtain the motion prediction model.

[0026] In this embodiment, the motion information determination device further includes a model enhancement module, used to acquire the k-th frame motion information and training position information of the training object, where k is a positive integer; to predict the k-th frame motion information and the training position information using the motion prediction model, to obtain the (k+1)-th frame motion update amount and the estimated training position information based on the (k+1)-th frame motion update amount of the training object; to correct the (k+1)-th frame motion update amount based on the difference between the estimated training position information and the training position information, and to obtain the (k+1)-th frame motion information based on the corrected (k+1)-th frame motion update amount; and to train the motion prediction model based on the k-th frame motion information and the (k+1)-th frame motion information to obtain an enhanced motion prediction model.

[0027] In this embodiment of the application, the information prediction module is further configured to use the enhanced motion prediction model to predict the motion information of the i-th frame and the target position information.

[0028] In this embodiment of the application, the model enhancement module is further configured to add the motion information of the k-th frame and the motion information of the (k+1)-th frame to the dataset including the model training samples; delete motion information in the dataset that meets the deletion conditions to obtain an enhanced dataset; and train the motion prediction model based on the enhanced dataset to obtain the enhanced motion prediction model.

[0029] In this embodiment of the application, when the moving object is a physical object, the motion information determining device further includes a motion control module, which is used to determine the motion trajectory of the physical object based on the motion frame sequence; and control the physical object to move along the motion trajectory.

[0030] In this embodiment of the application, the information acquisition module is further configured to, in response to an animation generation request sent by the rendering device, acquire the i-th frame motion information and the target position information of the virtual object, wherein the animation generation request is generated by the rendering device when it receives a target-guided action execution operation.

[0031] In this embodiment of the application, the animation information determining device further includes an animation sending module, which is used to generate a target motion animation based on the motion frame sequence; send the target motion animation to the rendering device so that the rendering device plays the target motion animation and renders a virtual scene in which the virtual object performs the target guiding action in response to the target position information.

[0032] This application provides a motion information determination device, including:

[0033] Memory, used to store executable instructions;

[0034] The processor, when executing executable instructions stored in the memory, implements the motion information determination method provided in the embodiments of this application.

[0035] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the motion information determination method provided in this application.

[0036] The embodiments of this application have at least the following beneficial effects: Since the motion information of the (i+1)th frame predicted based on the motion information of the i-th frame of the moving object and the target position information is based on the difference between the estimated target position information and the target position information before being used as the basis for determining the motion information of subsequent frames; that is, the accuracy of the motion information of the (i+1)th frame is high, and thus, the accuracy of the motion information of subsequent frames iterated based on the motion information of the (i+1)th frame and the target position information is also high. Therefore, it is possible to reduce the deviation between the position corresponding to the last frame of motion information (the motion information of the (i+n)th frame) and the target position information, and thus, the accuracy of the determined motion frame sequence is high, which can improve the accuracy of the determined set of motion information of the moving object performing the target-guided action. Attached Figure Description

[0037] Figure 1 This is a schematic diagram illustrating an exemplary process for training a neural network model;

[0038] Figure 2 This is an exemplary schematic diagram illustrating the generation of virtual objects based on a neural network model to perform goal-oriented motion animations.

[0039] Figure 3 This is a schematic diagram of an optional architecture of the motion information determination system provided in the embodiments of this application;

[0040] Figure 4 This is provided by the embodiments of this application. Figure 3 A schematic diagram of the composition structure of a server;

[0041] Figure 5 This is an optional flowchart illustrating the motion information determination method provided in the embodiments of this application;

[0042] Figure 6 This is a schematic diagram of an exemplary iterative prediction correction process provided in an embodiment of this application;

[0043] Figure 7 This is another optional flowchart illustrating the motion information determination method provided in the embodiments of this application;

[0044] Figure 8 This is a schematic diagram illustrating an exemplary method for correcting the update amount of the object point position in the (i+1)th frame, provided in an embodiment of this application.

[0045] Figure 9 This is a schematic diagram of an exemplary reverse motion adjustment process provided in an embodiment of this application;

[0046] Figure 10 This is another optional flowchart illustrating the motion information determination method provided in the embodiments of this application;

[0047] Figure 11 This is another optional flowchart illustrating the motion information determination method provided in the embodiments of this application;

[0048] Figure 12 This is an exemplary motion information determination method provided in the embodiments of this application;

[0049] Figure 13 This is a schematic diagram illustrating an exemplary execution stage provided in an embodiment of this application;

[0050] Figure 14 This is a schematic diagram illustrating an exemplary annotation action type provided in an embodiment of this application;

[0051] Figure 15 This is a schematic diagram of an exemplary target motion animation provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] In the following description, the terms “first, second, third, fourth, fifth, and sixth” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first, second, third, fourth, fifth, and sixth” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0056] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0057] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0058] 1) Artificial Intelligence (AI): The theory, methods, technologies and application systems that use digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0059] 2) Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills; and how to reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning typically includes techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0060] 3) Artificial neural networks are mathematical models that mimic the structure and function of biological neural networks. Exemplary structures of artificial neural networks in the embodiments of this application include deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN), NSM (Neural State Machine), PFNN (Phase-Functioned Neural Network), etc.

[0061] 4) Data-driven animation generation technology is a technique that uses motion sequences to train neural network models and then generates animations using these models. Specifically, professional stunt performers wearing suits equipped with special sensors perform actions (such as walking, running, and jumping) to collect motion data. This motion data is then mapped onto virtual objects to obtain motion sequences. The mapping of motion data to virtual objects is necessary because professional stunt performers and virtual objects differ in factors such as height, body shape, and joint structure.

[0062] 5) An operation is a way to trigger a device to perform processing, such as a click operation, a double-click operation, a long-press operation, a swipe operation, a gesture operation, a received trigger command, etc. In addition, the various operations in the embodiments of this application can be a single operation or a collective term for multiple operations.

[0063] 6) In response to, used to indicate the conditions or states on which the performed processing depends, when the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay; unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0064] 7) Virtual objects: These are interactive images of people and objects within a virtual scene, or movable objects within the virtual scene. These movable objects can be virtual characters, virtual animals, cartoon characters, etc., such as people, animals, plants, oil drums, walls, and stones displayed in the virtual scene. Additionally, a virtual object can be a virtual avatar representing the user within the virtual scene. A virtual scene can include multiple virtual objects, each with its own shape and volume, occupying a portion of the space within the virtual scene.

[0065] It's important to note that artificial intelligence (AI) is a comprehensive technology within computer science. It attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0066] Furthermore, artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0067] With the research and advancement of artificial intelligence (AI) technology, AI has been studied and applied in various fields, such as smart homes, wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. As technology develops, AI will be applied in even more fields and play an increasingly important role. This application will describe the application of AI in the field of motion information determination.

[0068] Generally, when the moving object is a virtual object, to generate an animation of the virtual object performing a goal-oriented action, an animation asset library is typically built. Then, based on the motion state information and operation commands of the virtual object, the closest matching animation is selected from the library and played to complete the generation of the goal-oriented action animation. However, in the process of generating the goal-oriented action animation of the virtual object described above, the closest matching animation cannot correspond well to the goal-oriented action performed by the virtual object. Furthermore, the memory space and computational resources required increase linearly with the increase in the animation asset library.

[0069] In addition, to generate animations of virtual objects performing goal-oriented actions, data-driven animation generation technology can also be used. Figure 1 and Figure 2 .in, Figure 1 This is an exemplary flowchart of training a neural network model; such as Figure 1 As shown, the exemplary process for training a neural network model includes a data acquisition module 1-1, a data mapping module 1-2, and a model training module 1-3. Here, the data acquisition module 1-1 is used to collect motion data based on the actions performed by a professional stunt performer 1-11 wearing a costume 1-12 equipped with special sensors; the data mapping module 1-2 is used to map the motion data to a virtual object 1-21 to obtain a motion sequence; and the model training module 1-3 is used to train a neural network model 1-31 based on the motion sequence.

[0070] Figure 2 This is an exemplary diagram illustrating the generation of virtual objects based on a neural network model to perform goal-oriented motion animation; such as... Figure 2 As shown, Figure 1 The inputs to the neural network model 1-31 include operation instructions 2-1 and motion state information 2-2 of the current frame of the virtual object. The output of the neural network model 1-31 is the predicted motion state information 2-3 of the virtual object in the next frame.

[0071] However, in the process of generating animations where virtual objects perform target-oriented actions, the position of the last frame of motion information output by the neural network model often deviates from the target position. This is because the motion information of the moving object is constantly changing, and therefore the information input to the neural network model is also constantly changing, leading to the accumulation of errors. This accumulation of errors results in the last frame of motion information in the generated animation not falling on the given target position.

[0072] Based on this, embodiments of this application provide a motion information determination method, apparatus, device, and computer-readable storage medium, which can reduce the deviation between the position corresponding to the last frame of motion information and the target position information, thereby improving the accuracy of the generated motion information for performing target-guided actions.

[0073] The following describes exemplary applications of the motion information determination device provided in this application. This device can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or as a server. The following will describe exemplary applications of the motion information determination device when implemented as a server.

[0074] See Figure 3 , Figure 3 This is a schematic diagram of an optional architecture of the motion information determination system provided in this application embodiment; as shown... Figure 3 As shown, to support a motion information determination application, in the motion information determination system 100, terminal 200-1 (rendering device) and terminal 200-2 (entity object) are connected to server 400 (motion information determination device) via network 300. Network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. Additionally, the motion information determination system 100 also includes a database 500 for providing data support to server 400.

[0075] Server 400 is used to acquire the i-th frame motion information and target position information of the moving object (for terminal 200-1, the acquisition of the i-th frame motion information and target position information is triggered by responding to the animation generation request sent by terminal 200-1 through network 300), where i is a positive integer; based on the i-th frame motion information and target position information, predict the (i+1)-th frame motion update amount of the moving object and the estimated target position information based on the (i+1)-th frame motion update amount; based on the difference between the estimated target position information and the target position information, correct the (i+1)-th frame motion update amount, and obtain the (i+1)-th frame motion information according to the corrected (i+1)-th frame motion update amount; continue iterating until the (i+n)-th frame motion information corresponding to the target position information is obtained, where n is a positive integer greater than 1; obtain a motion frame sequence including the i-th frame motion information to the (i+n)-th frame motion information, where the motion frame sequence is a set of motion information of the moving object performing a target-guided action in response to the target position information. It is also used to send target motion animation generated based on motion frame sequence to terminal 200-1 via network 300, or to send motion control command determined based on motion frame sequence to terminal 200-2 via network 300.

[0076] Terminal 200-1 is used to respond to the received target-guided action and perform an operation by sending an animation generation request to server 400 via network 300. It is also used to receive the target action animation sent by server 400 via network 300, play the target action animation, and render a virtual scene 200-11 in which virtual objects perform target-guided actions based on target location information.

[0077] Terminal 200-2 is used to receive motion control commands sent by server 400 via network 300 and to perform motion based on the motion control commands.

[0078] In some embodiments, server 400 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Terminal 200 may be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, smart game console, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the invention.

[0079] See Figure 4 , Figure 4 This is provided by the embodiments of this application. Figure 3 A schematic diagram of the composition structure of a server. Figure 4 The server 400 shown includes at least one processor 410, memory 450, at least one network interface 420, and a user interface 430. The various components in server 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general labeled all buses as Bus System 440.

[0080] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0081] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0082] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0083] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0084] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0085] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0086] The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, Wi-Fi, and Universal Serial Bus (USB), etc.

[0087] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.).

[0088] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0089] In some embodiments, the motion information determination device provided in this application can be implemented in software. Figure 4 A motion information determination device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: information acquisition module 4551, information prediction module 4552, information correction module 4553, information iteration module 4554, information determination module 4555, model training module 4556, model enhancement module 4557, motion control module 4558, and animation sending module 4559. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0090] In other embodiments, the motion information determination device provided in this application can be implemented in hardware. As an example, the device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the motion information determination method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0091] The motion information determination method provided in this application will be described below with reference to exemplary applications and implementations of the server provided in the embodiments of this application.

[0092] See Figure 5 , Figure 5 This is an optional flowchart illustrating the motion information determination method provided in this application embodiment, which will be combined with... Figure 5 The steps shown are explained.

[0093] S501. Obtain the motion information of the i-th frame and the target position information of the moving object.

[0094] In this embodiment, in application scenarios where a target-guided action can be triggered, when a user triggers an operation to execute a target-guided action on a moving object (e.g., triggering a cover-avoiding control in a virtual scene, or triggering a motion button in a scene controlling the movement of a smart device), the motion information determining device responds to the operation by acquiring the motion state of the moving object, thus obtaining the i-th frame motion information of the moving object. Additionally, the motion information determining device can also acquire the position corresponding to the moving object executing the target-guided action, i.e., the target position information. Here, i is a positive integer.

[0095] It should be noted that the moving object is a movable object, which can be a virtual object in a virtual scene, such as a game character, an interactive object in a human-computer interaction scene, etc.; it can also be a physical object in a physical scene, such as a smart robot or other smart device; this application embodiment does not specifically limit this. The motion information of the i-th frame is the motion information of the moving object in the current frame, such as the position and speed information of the moving object, the position and speed information of each joint, the relevant information of the motion trajectory, and the surrounding terrain data, etc. The target position information corresponds to the target-guided action; for example, when the target-guided action is to take cover, the target position information is near the cover; or when the target-guided action is to sit on a chair, the target position information is the chair surface; usually, the target position information is given when the operation of triggering the execution of the target-guided action is performed. In addition, the application scenarios that can trigger the execution of the target-guided action are, for example, the application scenario where the virtual object is located between the cover (wall, statue, tree, etc. in the virtual scene) and the camera, or the application scenario where there is a preset distance between the smart device and the object corresponding to the target-guided action (express delivery, water cup, etc.).

[0096] S502. Based on the motion information of the i-th frame and the target position information, predict the motion update amount of the moving object in the (i+1)-th frame and estimate the target position information based on the motion update amount of the (i+1)-th frame.

[0097] In this embodiment, after obtaining the i-th frame motion information and target position information of the moving object, the motion information determining device predicts the motion state (position and posture, etc.) of the moving object in the (i+1)-th frame based on the i-th frame motion information and target position information, thus obtaining the motion update amount of the (i+1)-th frame. In addition, based on the i-th frame motion information and target position information, the motion information determining device can also predict the position corresponding to the moving object when performing the target-guided action, thus obtaining the estimated target position information, such as the predicted position of sitting on a chair, or the predicted position of hiding when taking cover.

[0098] It should be noted that the (i+1)th frame is the frame following the i-th frame.

[0099] Here, the motion information determination device can predict the motion update amount and the estimated target position information of the (i+1)th frame through a network model; it can also determine the motion update amount of the moving object in relation to the target position information of the (i+1)th frame by analyzing the motion information of the (i)th frame, thereby predicting the motion update amount and the estimated target position information of the (i+1)th frame; the embodiments of this application do not specifically limit this.

[0100] S503. Based on the difference between the estimated target position information and the target position information, correct the motion update amount of the (i+1)th frame, and obtain the motion information of the (i+1)th frame according to the corrected motion update amount of the (i+1)th frame.

[0101] In this embodiment, the estimated target position information represents the final position reached by the virtual object as it moves according to the motion trend corresponding to the motion information of the i-th frame and the motion update amount of the (i+1)-th frame. Therefore, after obtaining the estimated target position information, the motion information determining device compares the estimated target position information with the target position information to obtain the difference between the estimated target position information and the target position information. Thus, the motion information determining device corrects the motion update amount of the (i+1)-th frame based on the difference between the estimated target position information and the target position information, so that the motion trend corresponding to the corrected motion update amount of the (i+1)-th frame is closer to the target position information. Here, the corrected motion update amount of the next frame is superimposed on the motion information of the i-th frame, thus obtaining the target motion information of the (i+1)-th frame.

[0102] It should be noted that the motion information of the (i+1)th frame refers to the motion information of the determined moving object in the (i+1)th frame. Therefore, after obtaining the motion information of the (i+1)th frame, the motion information determining device can determine the motion state of the moving object in the (i+1)th frame based on the motion information of the (i+1)th frame. Here, when the moving object is a virtual object, the motion information of the (i+1)th frame is used to render the motion state of the virtual object in the (i+1)th frame. Furthermore, the motion information determining device can send the motion information of the (i+1)th frame to the rendering device for rendering, or it can perform rendering itself. This application embodiment does not specifically limit this. In addition, the difference between the estimated target position information and the target position information can be a distance value, a vector, or a direction. This application embodiment does not specifically limit this.

[0103] S504. Continue iterating until the motion information of the (i+n)th frame corresponding to the target position information is obtained.

[0104] In this embodiment, after obtaining the motion information of frame i+1, the motion information determining device, similar to the process of predicting the motion update amount of frame i+1 based on the motion information of frame i and the target position information, continues to predict the motion update amount of frame i+2 and the estimated target position information of frame i+2 based on the motion information of frame i+1 and the target position information. The motion update amount of frame i+2 is then corrected based on the difference between the estimated target position information and the target position information, resulting in the motion information of frame i+2. The motion information of frame i+2 is then used as input data for predicting the motion information of the moving object until the motion information of frame i+n (the last frame of motion information) corresponding to the target position information is obtained, at which point the iteration process ends. Specifically, the motion information of frame i+n corresponding to the target position information obtained by the motion information determining device has the smallest position information difference from the target position information, and the moving object can reach the target position information through the motion information of frame i+n; n is a positive integer greater than 1.

[0105] S505. Obtain a motion frame sequence including motion information from frame i to frame i+n.

[0106] In this embodiment, after the motion information determining device obtains the motion information of the (i+n)th frame, it combines the motion information of the ith frame, the (i+1)th frame, the (i+2)th frame, ..., the (i+n)th frame to obtain a motion frame sequence. The motion frame sequence is a set of motion information representing the motion information of a moving object performing a target-guided action in response to target position information.

[0107] It should be noted that if the motion information of the (i+1)th frame is the last frame of motion information, that is, if the motion information of the (i+1)th frame corresponds to the target position information, then the motion information determining device obtains the motion frame sequence after obtaining the motion information of the (i+1)th frame; at this time, the motion frame sequence includes the motion information of the ith frame and the motion information of the (i+1)th frame. However, if the motion information of the (i+1)th frame is not the last frame of motion information of the target, then the motion information determining device executes S504 and S505.

[0108] See Figure 6 , Figure 6 This is a schematic diagram illustrating an exemplary iterative process for acquiring motion information, provided in an embodiment of this application; as shown... Figure 6As shown, the motion information determination device predicts the motion update amount 6-121 and the estimated target position information 6-22 for the (i+1)th frame based on the motion information 6-11 of the i-th frame and the target position information 6-21. Based on the difference between the target position information 6-21 and the estimated target position information 6-22, the motion update amount 6-121 of the (i+1)th frame is corrected to obtain the motion information 6-122 of the (i+1)th frame. Then, the motion information determination device predicts the motion update amount for the (i+2)th frame based on the motion information 6-122 of the (i+1)th frame and the target position information 6-21. The motion update amount 6-131 in frame i-1 and the estimated target position information 6-23 in frame i+2 are used to correct the motion update amount 6-131 in frame i+2 based on the difference between the target position information 6-21 and the estimated target position information 6-23 in frame i+2, resulting in motion information 6-132 in frame i+2. This process is repeated until the motion information determining device completes the correction of the motion update amount 6-1m1 in the last frame (motion update amount in frame i+n), resulting in motion information 6-1m2 in the last frame (motion information in frame i+n); where m is an integer greater than 2.

[0109] It is understandable that the motion information of the (i+1)th frame predicted based on the motion information of the i-th frame of the moving object and the target position information is based on the difference between the estimated target position information and the target position information before it is used as the basis for determining the motion information of subsequent frames. In other words, the accuracy of the motion information of the (i+1)th frame is high. Therefore, the accuracy of the motion information of subsequent frames iterated based on the motion information of the (i+1)th frame and the target position information is also high. Thus, it can reduce the deviation between the position corresponding to the motion information of the last frame and the target position information, thereby improving the accuracy of the generated motion frame sequence that performs the target-guided action.

[0110] In this embodiment, the motion update amount in the (i+1)th frame includes the object point position update amount in the (i+1)th frame, and the motion information in the i-th frame includes the object point position information in the i-th frame; the object point position update amount in the (i+1)th frame is the position update amount of the point corresponding to the moving object (e.g., the center of gravity of the moving object, the center of the moving object, etc.) relative to the object point position information in the i-th frame, and the object point position information in the i-th frame is the position information of the point corresponding to the moving object in the i-th frame. See also... Figure 7 , Figure 7 This is another optional flowchart illustrating the motion information determination method provided in the embodiments of this application; as shown below. Figure 7 As shown, S503 can be implemented through S5031-S5033; that is, the motion information determining device corrects the motion update amount of the (i+1)th frame based on the difference between the estimated target position information and the target position information, and obtains the motion information of the (i+1)th frame based on the corrected motion update amount of the (i+1)th frame, including S5031-S5033. Each step is explained below.

[0111] S5031. Calculate the vector difference based on the estimated target position information and the target position information to determine the initial correction vector.

[0112] In this embodiment of the application, the difference between the estimated target location information and the target location information refers to the vector difference between the estimated target location information and the target location information, which is called the initial correction vector; and the direction corresponding to the initial correction vector is the direction from the estimated target location information to the target location information.

[0113] S5032. Adjust the initial correction vector based on the correction coefficient to obtain the correction vector.

[0114] It should be noted that the motion information determination device has a pre-set correction coefficient, or the motion information determination device can obtain the correction coefficient; the correction coefficient can be a coefficient threshold, or it can be dynamically changed based on the determined motion information, and the embodiments of this application do not specifically limit it in this way.

[0115] In this embodiment, the motion information determining device adjusts the initial correction vector based on a correction coefficient, and the adjusted initial correction vector is the correction vector. Here, the adjustment method used by the motion information determining device can be a calculation method such as multiplication or division to obtain a correction vector smaller than the initial correction vector. Furthermore, the correction vector and the initial correction vector are in the same direction. In this case, the motion information determining device adjusts the initial correction vector based on the correction coefficient, adjusting the magnitude of the initial correction vector.

[0116] S5033. Using a correction vector, after correcting the update amount of the object point position in the (i+1)th frame, the corrected update amount of the object point position in the (i+1)th frame is superimposed with the object point position information in the i-th frame to obtain the object point position information in the motion information of the (i+1)th frame.

[0117] It should be noted that the motion information determination device corrects the object point position update amount in the (i+1)th frame based on the correction vector, and then superimposes the corrected object point position update amount in the (i+1)th frame with the object point position information in the ith frame to obtain the object point position information in the (i+1)th frame. For example, the motion information determination device can multiply the correction vector by the object point position update amount in the (i+1)th frame, or it can use other combination methods, which are not specifically limited in this embodiment. Here, the object corrected by the motion information determination device based on the difference between the estimated target position information and the target position information is the object point position update amount in the (i+1)th frame of the motion information.

[0118] See also Figure 7In this embodiment of the application, when the moving object is a virtual object, the motion update amount of the (i+1)th frame also includes the relative position amount of the object part in the (i+1)th frame. The relative position amount of the object part in the (i+1)th frame is the relative position information of the object part (e.g., each joint of the moving object) relative to the object point position information of the (i+1)th frame. At this time, the motion information determining device also needs to adjust the lower body of the virtual object. Therefore, S5033 can be implemented by S50331-S50333. That is to say, the motion information determining device obtains the object point position information of the (i+1)th frame in the motion information of the (i+1)th frame, including S50331-S50333. The steps are described below.

[0119] S50331. Superimpose the relative position of the object part in the (i+1)th frame onto the position information of the object point in the (i+1)th frame to obtain the position information of the object part to be adjusted in the (i+1)th frame.

[0120] It should be noted that since the update amount of the object point position in frame i+1 and the relative amount of the object part position in frame i+1 are matched, and the relative amount of the object part position in frame i+1 is relative to the position determined by the update amount of the object point position in frame i+1; therefore, the corrected update amount of the object point position in frame i+1 obtained by correcting the update amount of the object point position in frame i+1 will no longer match with the relative amount of the object part position in frame i+1; when rendering the motion of the moving object in frame i+1 directly based on the corrected update amount of the object point position in frame i+1 and the relative amount of the object part position in frame i+1, a sliding phenomenon will occur (being dragged and translated, the phenomenon of the foot sliding on the ground); therefore, the motion information determination device also needs to adjust the relative amount of the object part position in frame i+1.

[0121] In this embodiment, the motion information determining device first determines the positions in the world coordinate system corresponding to the object point position update amount and the target object point position update amount in the (i+1)th frame, respectively. Then, based on the difference between the positions in the world coordinate system corresponding to the object point position update amount and the target object point position update amount in the (i+1)th frame, it adjusts the relative position of the object part in the (i+1)th frame to match the adjusted relative position of the object part in the (i+1)th frame with the target object point position update amount. Here, since the relative position of the object part in the (i+1)th frame is the relative information of the object part (e.g., each joint of the moving object) relative to the object point position information in the (i+1)th frame, the motion information determining device superimposes the relative position of the object part in the (i+1)th frame onto the object point position information in the (i+1)th frame, thus obtaining a set of corrected position information of the object part of the moving object in the (i+1)th frame. From this set of corrected position information of the object part of the moving object in the (i+1)th frame, the position information of the object part to be adjusted is obtained, thus obtaining the position information of the object part to be adjusted in the (i+1)th frame.

[0122] S50332. The relative position of the object part in frame i+1 and the update amount of the object point position in frame i+1 are both superimposed on the object point position information in frame i to obtain the reference position information of the object part in frame i+1.

[0123] It should be noted that the motion information determining device superimposes the object point position information of the (i+1)th frame onto the object point position information of the ith frame, thus obtaining the uncorrected object point position information of the (i+1)th frame. Next, the motion information determining device superimposes the relative position of the object part of the (i+1)th frame onto the uncorrected object point position information of the (i+1)th frame, thus obtaining another set of uncorrected position information of the object part of the moving object in the (i+1)th frame. From this set of uncorrected position information of the object part of the moving object in the (i+1)th frame, the position information of the footsteps is obtained, thus obtaining the reference position information of the object part in the (i+1)th frame.

[0124] S50333: Based on the reference position information of the object part in the (i+1)th frame, adjust the position information of the object part to be adjusted in the (i+1)th frame to obtain the position information of the object part in the (i+1)th frame in the motion information of the (i+1)th frame.

[0125] It should be noted that, to address the slippage problem, the motion information determination device needs to reverse-move the adjustment position information of the object part in frame i+1 to the reference position information of the object part in frame i+1. Thus, after the motion information determination device completes the adjustment, it obtains the position information of the object part in frame i+1. Here, when the adjustment position information of the object part in frame i+1 is the position information of a portion of the object part corresponding to the relative position of the object part in frame i+1, the adjusted adjustment position information of the object part in frame i+1, combined with the position information of the remaining unadjusted object parts, can be used to obtain the position information of the object part in frame i+1. In this case, the motion information of frame i+1 includes the object point position information and the object part position information.

[0126] It is understandable that by correcting the obtained object point position information in frame i+1, the relative position of the object part in frame i+1 is adjusted in reverse motion, so that the adjusted relative position of the object part in frame i+1 matches the object point position information in frame i+1; this solves the problem of slippage and improves the accuracy of the generated motion frame sequence.

[0127] In this embodiment of the application, S5034 is included before S5032; that is, before the motion information determining device determines the correction amount based on the distance between the current frame object point position information corresponding to the i-th frame motion information and the i+1 frame object point position information corresponding to the i+1 frame motion information, and the distance between the current frame object point position information and the target position information, the motion information determining method further includes S5034, which will be described below.

[0128] In this embodiment, the correction coefficient is positively correlated with the speed of the moving object. That is, the faster the moving object moves, the larger the correction coefficient; the slower the moving object moves, the smaller the correction coefficient.

[0129] In this embodiment, the correction coefficient is determined based on the motion distance between the object point position information and the target position information in the i-th frame, the object point position update amount in the (i+1)-th frame, and the motion speed. Furthermore, when the motion distance between the object point position information and the target position information in the i-th frame falls within the distance range, the correction coefficient is calculated based on the motion distance and the object point position update amount in the (i+1)-th frame. When the motion distance falls outside the distance range, the correction coefficient is a constant, and in some embodiments, the correction coefficient tends to 0.

[0130] It should be noted that the motion information determination device has a pre-set distance range, or the device is capable of acquiring a distance range, which is determined by the distance range corresponding to the minimum and maximum distance thresholds. When the motion distance between the object point position information and the target position information in frame i falls within this range, it indicates that at frame i, the distance between the moving object and the target position information is neither too far nor too close. In this case, the motion information determination device determines a correction coefficient based on the motion speed, motion distance, and the object point position update amount in frame i+1. Conversely, when the motion distance falls outside the preset range, it indicates that at frame i, the distance between the moving object and the target position information is either too far or too close. Since it is unnecessary to correct the object point position update amount in frame i+1 when the distance is too far, and correcting the object point position update amount in frame i+1 when the distance is too close would make the virtual object's movement unnatural, the motion information determination device determines a correction coefficient based on the motion speed.

[0131] Here, the correction factor obtained when the movement distance falls within the distance range is greater than the correction factor obtained when the movement distance falls outside the distance range.

[0132] For example, see Figure 8 , Figure 8 This is an exemplary schematic diagram of determining a correction coefficient provided in an embodiment of this application; as shown... Figure 8 As shown, when the target-guided action is to take cover, point G at cover 8-1 is the target position information, point P is the estimated target position information, point A is the object point position information in the i-th frame, and point F is the object point position information in the (i+1)-th frame; in addition, the direction pointed to by the dashed arrows at each point is the orientation of the moving object.

[0133] based on Figure 8 The correction coefficient can be obtained through equation (1), which is:

[0134] (1)

[0135] in, This is a correction factor; To estimate the distance difference between target location information and target location information; This is the value corresponding to the position update of the object point in frame i+1. Let be the distance between the object point position information and the target position information in the i-th frame. A parameter that is positively correlated with the speed of the moving object; where, When it falls within the distance range, k is , When it falls outside the distance range, k is It tends to 0. It is easy to see that the initial correction vector is... The corrected vector quantity is .

[0136] It should be noted that when the movement distance between the object point position information and the target position information in the i-th frame falls outside the distance range, the correction coefficient tends to zero; the obtained corrected object point position update amount in the (i+1)-th frame is .

[0137] Understandably, the motion information determination device dynamically determines the degree of correction for the object point position update in the (i+1)th frame based on the motion distance between the object point position information and the target position information in the i-th frame, so as to accurately execute the target-guided action while ensuring the animation quality.

[0138] In this embodiment, the reference position information of the object part in the (i+1)th frame includes ankle reference position information and toe reference position information. The position information of the object part to be adjusted in the (i+1)th frame includes hip position information to be adjusted, knee position information to be adjusted, and ankle position information to be adjusted. The ankle reference position information is the position information of the ankle to be referenced, the toe reference position information is the position information of the toe to be referenced, the hip position information to be adjusted is the position information of the hip to be adjusted, the knee position information to be adjusted is the position information of the knee to be adjusted, and the ankle position information to be adjusted is the position information of the ankle to be adjusted. At this time, S50333 can be achieved through S503331 and S503332; that is, the motion information determining device adjusts the position information of the object part to be adjusted in the (i+1)th frame based on the reference position information of the object part in the (i+1)th frame, to obtain the position information of the object part in the (i+1)th frame of the motion information, including S503331 and S503332. Each step is described below.

[0139] S503331. By rotating the knee position information and hip position information to be adjusted, the ankle position information to be adjusted is adjusted to the ankle reference position information, thereby determining the knee position information of the (i+1)th frame.

[0140] It should be noted that the motion information determination device adjusts the knee position information and hip position information to be adjusted by reverse motion adjustment, so as to adjust the ankle position information to be adjusted to the ankle reference position information, thus determining the target knee position information corresponding to the knee position information to be adjusted.

[0141] S503332, The knee position information, hip position information to be adjusted, ankle reference position information and toe reference position information of the (i+1)th frame are determined as the object part position information of the (i+1)th frame in the motion information of the (i+1)th frame.

[0142] In this embodiment, S503331 can be implemented by S5033311-S5033314; that is, the motion information confirmation device adjusts the ankle position information to be adjusted to the ankle reference position information by rotating the knee position information to be adjusted and the hip position information to be adjusted, thereby determining the knee position information of the (i+1)th frame, including S5033311-S5033314. Each step is described below.

[0143] S5033311. Determine the knee rotation direction based on the direction from the knee position information to the ankle position information to the hip position information to the knee position information to the hip position information.

[0144] It should be noted that the direction determined by the cross product of the direction from which the knee position information to be adjusted points to the ankle position information to be adjusted, and the direction from which the knee position information to be adjusted points to the hip position information to be adjusted, is determined as the knee rotation direction.

[0145] S5033312. Rotate the knee position information to be adjusted with the knee rotation direction as the axis of rotation, and adjust the ankle position information to be adjusted based on the distance between the hip position information to be adjusted and the ankle reference position information.

[0146] It should be noted that the motion information determination device first rotates the knee joint. Then, the motion information determination device rotates the knee position information to be adjusted based on the knee rotation direction: in the plane formed by the hip position information to be adjusted, the knee position information to be adjusted, and the ankle position information to be adjusted, the knee position information to be adjusted is rotated about the knee rotation direction as the axis of rotation, so as to adjust the ankle position information to be adjusted, so that the distance between the hip position information to be adjusted and the adjusted ankle position information is equal to the distance between the hip position information to be adjusted and the ankle reference position information.

[0147] In this embodiment of the application, when the sum of the distances between the hip position information to be adjusted and the knee position information to be adjusted, and the distances between the knee position information to be adjusted and the ankle position information to be adjusted, is less than the distance between the hip position information to be adjusted and the ankle reference position information, it is because the correction coefficient is too large. The correction coefficient can be lowered (for example, by adjusting α in formula (1)) so that the sum of the distances is greater than or equal to the distance between the hip position information to be adjusted and the ankle reference position information. At this time, S50333 is executed again.

[0148] S5033313, Determine the hip rotation direction based on the direction of the hip position information to be adjusted pointing to the ankle reference position information, and the direction of the hip position information to be adjusted pointing to the adjusted ankle position information.

[0149] In this embodiment, after the motion information determining device completes the adjustment of the knee position information to be adjusted, it begins to adjust the hip joint, that is, to adjust the hip position information to be adjusted. Here, the motion information determining device determines the direction of hip rotation by the cross product of the direction from the hip position information to be adjusted pointing to the ankle reference position information and the direction from the hip position information to be adjusted pointing to the target ankle position information to be adjusted.

[0150] S5033314. Rotate the hip position information to be adjusted with the hip rotation direction as the axis of rotation to adjust the target ankle position information to the ankle reference position information, thereby determining the knee position information of the (i+1)th frame.

[0151] It should be noted that the motion information determination device finally adjusts the ankle joint, that is, it rotates the hip position information to be adjusted in the direction of hip rotation to rotate the target ankle position information to the ankle reference position information. At this time, the position of the knee joint changes, thus determining the knee position information of the (i+1)th frame.

[0152] For example, see Figure 9 , Figure 9 This is a schematic diagram illustrating an exemplary reverse motion adjustment process provided in an embodiment of this application; as shown... Figure 9 As shown in position state 9-1, points B, C, D, E, H, and T represent the hip position information to be adjusted, the knee position information to be adjusted, the ankle position information to be adjusted, the toe position information to be adjusted, the ankle reference position information, and the toe reference position information, respectively. The motion information determination device adjusts points B, C, D, and E based on points H and T so that DE stops at HT. Furthermore, position state 9-1 is an oblique projection diagram, where any four points are not necessarily coplanar. Here, the adjustment process from position state 9-1 to position state 9-2 corresponds to the adjustment of the knee joint: in triangle BCD, the angle of angle BCD is determined using the law of cosines. Because point D is rotated around point C in the plane formed by BCD to point D' (the target ankle position information to be adjusted) in position state 9-2, and the distance between point B and point D' is equal to the distance between point B and point H (the distance between the hip position information to be adjusted and the ankle reference position information); at this time, the axis of rotation of point C is... The corresponding direction (knee rotation direction), the angle is ,in, To determine the angle of angle BCD' in position state 9-2 using the law of cosines; in addition, after the knee joint adjustment, point E rotates to point E'.

[0153] The adjustment process from position state 9-2 to position state 9-3 corresponds to the adjustment of the hip joint: in triangle BHD', the angle HBD' is determined using the law of cosines. Therefore, point B is... The direction is the axis of rotation. The angle is rotated so that D' is adjusted to point D”, i.e., point H, and point E' is adjusted to point E”. It should be noted that triangle BC'D” can also be rotated around axis BD” by an appropriate angle as needed. Furthermore, the reason for treating the knee and hip joints differently is that the knee joint moves as an axial joint, while the hip joint can move as a ball joint.

[0154] The adjustment process from position state 9-3 to position state 9-4 corresponds to the adjustment of the ankle joint: adjusting point E” to point E”’, i.e., point T. In addition, considering that the foot of the moving object has width, and the ankle joint can also be regarded as a ball joint, the effect of the sole of the foot coinciding with the ground can also be achieved by having D” E”’ rotate around D” E”’.

[0155] It is understandable that in each frame of the motion frame sequence, by adjusting the update amount of the object point position, and then performing inverse motion adjustment on the lower body joints (the position information of the object part to be adjusted in the (i+1)th frame), the footsteps of the moving object when performing the target-guided action can be automatically planned and adjusted, and finally the motion information for the moving object to accurately perform the target-guided action at the specified position (target position information) can be determined.

[0156] See Figure 10 , Figure 10 This is another optional flowchart illustrating the motion information determination method provided in the embodiments of this application; as shown below. Figure 10 As shown in the embodiment of this application, S502 can be implemented through S5021; that is, the motion information determining device predicts the motion update amount of the moving object in the (i+1)th frame and the estimated target position information based on the motion information of the i-th frame and the target position information, including S5021, which will be described below.

[0157] S5021. Use a motion prediction model to predict the motion information and target position information of the i-th frame, and obtain the motion update amount of the moving object in the (i+1)-th frame and the estimated target position information based on the motion update amount of the (i+1)-th frame.

[0158] It should be noted that when the motion information determination device predicts the motion information of the virtual object in the (i+1)th frame and estimates the target position information based on the motion information of the i-th frame and the target position information, it can be achieved through a network model; this network model is referred to here as the motion prediction model, and the motion prediction model is used to predict the motion information of the moving object.

[0159] In this embodiment, S5022-S5024 are included before S5021; that is, before the motion information determination device uses the motion prediction model to predict the motion information and target position information of the i-th frame, the motion information determination method further includes S5022-S5024, and each step is described below.

[0160] S5022. Obtain model training samples, wherein the model training samples include at least one frame of motion information samples and target position information samples.

[0161] It should be noted that the model training samples are the dataset used to train a network model for predicting motion information of moving objects.

[0162] S5023. Use the motion prediction model to be trained to predict the motion sample and target position sample of the jth frame in at least one frame of motion samples, and obtain the motion information of the (j+1)th frame.

[0163] It should be noted that the motion prediction model to be trained is a network model used to predict motion information; the motion information determination device inputs any frame of motion information sample (the j-th frame motion sample) from at least one frame of motion information into the motion prediction model to be trained, and the output obtained is the (j+1)-th frame of motion information.

[0164] S5024. Based on the difference between the motion information of frame j+1 and the motion sample of frame j+1, train the motion prediction model to be trained and obtain the motion prediction model.

[0165] It should be noted that, based on the difference between the motion information of frame j+1 and the motion sample of frame j+1, the motion prediction model to be trained is iteratively trained until the training cutoff condition is met, at which point training stops, thus obtaining the motion prediction model. The motion prediction model is the motion prediction model to be trained after iterative training; the training cutoff condition can be that the difference between the motion information sample of frame j+1 and the motion information of frame j+1 is less than a difference threshold, or it can be reaching a preset number of training iterations, etc., which are not specifically limited in this embodiment.

[0166] In this embodiment, S5022 can be implemented through S50221-S50223; that is, the motion information determination device acquires model training samples, including S50221-S50223, and each step is described below.

[0167] S50221. In the action execution phase based on goal-oriented actions, acquire goal-oriented action data.

[0168] It should be noted that the goal-oriented motion data refers to the collected motion information of performing the goal-oriented motion. The motion execution phase includes one or more of the following: pre-preparation phase, preparation phase, execution phase, mid-motion phase, and exit phase. Specifically, the pre-preparation phase is the phase before the moving object begins to lift its first foot to run; the preparation phase is the phase from when the moving object lifts its first foot to when the t-th step before performing the goal-oriented motion lands; the execution phase is the phase from when the moving object lifts its foot to when the last foot lands; the mid-motion phase is the phase in which the moving object maintains the goal-oriented motion; and the exit phase is the phase in which the moving object rises from a stationary position to enter the next motion. t is a positive integer greater than or equal to 2.

[0169] S50222: Map target-oriented motion data to motion information of moving objects.

[0170] It should be noted that, since professional stunt performers differ from the target of the movement in terms of height, body shape, number of joints, etc., the motion information determination device maps the collected motion sequence, i.e., the target-oriented motion data, onto the target of the movement, thus obtaining the motion information of the target of the movement.

[0171] S50223. Label the motion information of the moving object to obtain model training samples including at least one frame of motion information samples and target position information samples.

[0172] It should be noted that annotation refers to the process of converting the format of motion information of moving objects. By annotating, the standardization of data is improved, which in turn improves the accuracy of motion frame sequences.

[0173] In this embodiment, S50223 can be implemented through S502231-S502235; that is, the motion information determination device annotates the motion information of the moving object to obtain a model training sample including at least one frame of motion information sample and target position information sample, including S502231-S502235. Each step is described below.

[0174] S502231. Based on the motion information of each frame in the motion information of the moving object, determine the object part information, the initial motion trajectory information and the initial object point information, wherein the object part information includes ankle and toe position information and the initial object point information includes object point position information.

[0175] It should be noted that the motion information of the moving object is a frame sequence of motion information. For each frame of motion information, the corresponding object part information, initial motion trajectory information, and initial object point information can be determined. Among them, the object part information is the skeletal joint information of the moving object, such as position, speed, and direction; the initial motion trajectory information is the motion trajectory information of the moving object, such as position, terrain information, and direction; the initial object point information is the information of the moving object as a whole (e.g., the center of gravity of the moving object), such as position and direction.

[0176] It should also be noted that since the object part information is the skeletal joint information of the moving object, the motion information determination device can obtain the position representing the foot, i.e., the ankle and toe position information, from the object part information; while the object point position information is the position of the moving object when the moving object is regarded as a whole.

[0177] S502232. Based on the correspondence between object point location information and preset action points, mark the action type of the initial motion trajectory information and the initial object point information respectively, and obtain motion trajectory information including action type and object point information including action type.

[0178] It should be noted that the preset action points are determined based on the action execution stage. Each preset action point can be the boundary point of each stage in the action execution stage, or multiple preset action points can belong to one stage in the action execution stage, etc. This application embodiment does not make specific limitations in this regard.

[0179] S502233. Based on the correspondence between ankle and toe position information and preset phase segments, mark the phase information.

[0180] It should be noted that the preset phase segment is determined based on the motion cycle. Therefore, based on the labeled phase information, the position corresponding to the motion cycle in the current frame can be determined. For example, the phase information can be used to calibrate the left and right feet. Here, the motion cycle refers to the smallest unit of motion corresponding to the virtual object's movement. For example, when the moving object is a virtual character, the motion cycle is left foot right foot left foot or right foot left foot right foot.

[0181] S502234. Combine the object part information, motion trajectory information, object point information and phase information into a motion information sample to obtain at least one motion information sample.

[0182] It should be noted that each motion information sample in at least one frame includes object part information, motion trajectory information, object point information, and phase information.

[0183] S502235. Obtain target position information samples based on the motion information of the moving object, thereby obtaining model training samples including at least one frame of motion information samples and target position information samples.

[0184] It should be noted that since the motion information of the moving object is the training data for the moving object to perform a target-oriented action, and the target-oriented action corresponds to the endpoint position, the motion information determination device obtains the target position information sample by acquiring the endpoint position.

[0185] In this embodiment, S502232 can be implemented by S5022321-S5022323; that is, the motion information determining device marks the motion type of the initial motion trajectory information and the initial object point information based on the correspondence between the object point position information and the preset action point, including S5022321-S5022323. Each step is described below.

[0186] S5022321. When the object point position information corresponds to the first action point in the preset action points, or the object point position information corresponds to the sixth action point in the preset action points, the action type for marking the initial motion trajectory information and the initial object point information is the standing type.

[0187] It should be noted that the first action point is the dividing point between the pre-preparation stage and the preparation stage; the sixth action point is the stage after the exit stage.

[0188] S5022322. When the object point location information corresponds to the second action point in the preset action points, or the object point location information corresponds to the third action point in the preset action points, the action type for marking the initial motion trajectory information and the initial object point information is the running type.

[0189] It should be noted that the second action point belongs to the preparation stage; the third action point is the dividing point between the preparation stage and the execution stage.

[0190] S5022323. When the object point position information corresponds to the fourth action point in the preset action points, or the object point position information corresponds to the fifth action point in the preset action points, the action type for marking the initial motion trajectory information and the initial object point information is the execution type.

[0191] It should be noted that the fourth action point is the dividing point between the execution phase and the action phase; the fifth action point is the dividing point between the action phase and the exit phase.

[0192] In this embodiment, when the object point location information corresponds to the stage before the first action point, the action type is labeled as standing; when the object point location information corresponds to the stage between the first and second action points, the action type is labeled as standing and running; when the object point location information corresponds to the stage between the second and third action points, the action type is labeled as running; when the object point location information corresponds to the stage between the third and fourth action points, the action type is labeled as running and execution; when the object point location information corresponds to the stage between the fourth and fifth action points, the action type is labeled as execution; and when the object point location information corresponds to the stage between the fifth and sixth action points, the action type is labeled as standing.

[0193] In this embodiment of the application, S502233 can be implemented by S5022331 and S5022332; that is, the motion information determining device marks the phase information based on the correspondence between the ankle and toe position information and the preset phase segment, including S5022331 and S5022332. The steps are described below.

[0194] S5022331. When the ankle-toe position information corresponds to the phase from the left foot landing to the right foot landing, phase information including the phase of the first sub-movement cycle is obtained.

[0195] It should be noted that the phase of the first sub-motion cycle is determined based on the motion cycle, and the phase of the first sub-motion cycle corresponds to the first half of the motion cycle, such as from 0 to π.

[0196] S5022332: When the ankle-toe position information corresponds to the phase from the landing of the right foot to the landing of the left foot, phase information including the phase of the second sub-movement cycle is obtained.

[0197] It should be noted that the phase of the second sub-motion cycle is determined based on the motion cycle, and the phase of the second sub-motion cycle corresponds to the second half of the motion cycle, such as π to 2π.

[0198] In this embodiment of the application, S5024 is followed by S5025-S5028; that is, after the motion information determining device obtains the motion prediction model, the motion information determining method further includes S5025-S5028. Each step is described below.

[0199] S5025. Obtain the k-th frame motion information and training position information of the training object.

[0200] It should be noted that the description of the implementation process corresponding to S5025 is similar to that corresponding to S501, and will not be repeated here in the embodiments of this application; where k is a positive integer.

[0201] S5026. Use a motion prediction model to predict the motion information and training position information of the kth frame, and obtain the motion update amount of the training object in the (k+1)th frame and the estimated training position information based on the motion update amount of the (k+1)th frame.

[0202] It should be noted that the description of the implementation process corresponding to S5026 is similar to that corresponding to S502, and will not be repeated here in the embodiments of this application.

[0203] S5027. Based on the difference between the estimated training position information and the training position information, correct the motion update amount of the (k+1)th frame, and obtain the motion information of the (k+1)th frame according to the corrected motion update amount of the (k+1)th frame.

[0204] It should be noted that the description of the implementation process corresponding to S5027 is similar to that corresponding to S503, and will not be repeated here in the embodiments of this application.

[0205] S5028. Based on the motion information of the k-th frame and the motion information of the (k+1)-th frame, train the motion prediction model to obtain an enhanced motion prediction model.

[0206] It should be noted that the description of the implementation process corresponding to S5028 is similar to that corresponding to S504, and will not be repeated here in the embodiments of this application.

[0207] Accordingly, in the embodiments of this application, the motion information determination device in S5021 uses a motion prediction model to predict the motion information and target position information of the i-th frame, including S50211, which will be described below.

[0208] S50211. Use an enhanced motion prediction model to predict the motion information and target position information of the i-th frame.

[0209] It should be noted that the enhanced motion prediction model and the enhanced motion prediction model are used. After the motion information determination device completes the enhanced training of the motion prediction model and obtains the enhanced motion prediction model, it deploys the enhanced motion prediction model to predict the motion information of the moving object in the (i+1)th frame and estimate the target position information based on the enhanced motion prediction model.

[0210] In this embodiment, in step S5028, the motion information determining device trains a motion prediction model based on the motion information of frame k and frame (k+1) to obtain an enhanced motion prediction model. This includes: adding the motion information of frame k and frame (k+1) to a dataset containing model training samples; deleting motion information in the dataset that meets the deletion conditions to obtain an enhanced dataset; and training the motion prediction model based on the enhanced dataset to obtain the enhanced motion prediction model. The deletion conditions can be deleting the dataset obtained by the motion prediction model before obtaining the motion information of frame k and frame (k+1), or deleting the first part of the motion information in the dataset obtained by the motion prediction model before obtaining the motion information of frame k and frame (k+1), etc. This embodiment does not specifically limit this.

[0211] In other words, after acquiring the motion information of frame k and frame (k+1), the motion information determination device adds these two frames to the dataset consisting of the model training samples, and continues to update the weights of the motion prediction model. The updated motion prediction model is then deployed in the simulator to continue executing goal-oriented actions, repeating the above process. In the dataset pool, the model training samples, which have the highest quality, are retained; while data generated by the model and subsequently post-processed, if data from a newer version of the network is added, will be removed from the dataset due to the lower motion quality of data from older versions of the network.

[0212] It is understandable that by strengthening the motion prediction model and improving its prediction accuracy, the magnitude of corrections can be reduced, thereby improving the quality of the generated motion frame sequence.

[0213] In this embodiment of the application, S5024 is followed by S5029 and S50210; that is, after the motion information determining device obtains the motion prediction model, the motion information determining method further includes S5029 and S50210. Each step is described below.

[0214] S5029. Obtain new model training samples.

[0215] It should be noted that the new model training samples are obtained after the model training samples.

[0216] S50210: Optimize the motion prediction model based on the new model training samples.

[0217] In this embodiment of the application, the motion information determination device in S5021 uses a motion prediction model to predict the motion information and target position information of the i-th frame, which can also be achieved through S50212. The following describes each step separately.

[0218] S50212. Use the optimized motion prediction model to predict the motion information and target position information of the i-th frame.

[0219] It should be noted that after the motion information determination device completes the optimization of the motion prediction model and obtains the optimized motion prediction model, it deploys the optimized motion prediction model to predict the motion information of the moving object in the (i+1)th frame and estimate the target position information based on the optimized motion prediction model.

[0220] Understandably, after obtaining the motion prediction model, training the model again with new training samples optimizes it, improves the generalization ability of the optimized model, and thus improves the accuracy of the determined motion frame sequence.

[0221] See Figure 11 , Figure 11 This is another optional flowchart illustrating the motion information determination method provided in the embodiments of this application; as shown below. Figure 11 As shown in the embodiment of this application, when the moving object is a virtual object, S501 can be implemented through S5011; that is, the motion information determining device obtains the i-th frame motion information and target position information of the moving object, including S5011, which will be described below.

[0222] S5011, In response to the animation generation request sent by the rendering device, obtain the i-th frame motion information and target position information of the virtual object.

[0223] It should be noted that the animation generation request is generated when the rendering device receives the target-guided action execution operation.

[0224] Accordingly, see [link to relevant documentation] Figure 11 In this embodiment of the application, S505 is followed by S506 and S507; that is, after the motion information determining device obtains the motion frame sequence including the motion information of the i-th frame to the i+n-th frame, the motion information determining method further includes S506 and S507, which will be described below.

[0225] S506. Generate target motion animation based on motion frame sequence.

[0226] In this embodiment of the application, the motion information determining device generates an animation from a sequence of motion frames, thus obtaining an animation used to render a virtual object performing a target-guided action in response to the target position information.

[0227] S507. Send the target motion animation to the rendering device.

[0228] It should be noted that the motion information determination device sends the target motion animation to the rendering device, so that the rendering device plays the target motion animation and renders a virtual scene in which virtual objects perform target-guided actions in response to the target position information. Here, the motion information determination device can send each frame of the target motion animation to the rendering device in real time, or it can send multiple frames of the target motion animation to the rendering device in batches, or it can send the target motion animation as a whole to the rendering device, or it can send the target motion animation to the rendering device in the form of a "Feeds" stream, etc. The embodiments of this application do not specifically limit this.

[0229] In this embodiment of the application, when the moving object is a physical object, S505 is followed by S508 and S509; that is, after the motion information determining device determines the motion frame sequence including the motion information of the i-th frame to the motion information of the last frame of the target, the motion information determining method further includes S507 and S508. Each step is described below.

[0230] S508. Determine the motion trajectory of an entity object based on a sequence of motion frames.

[0231] It should be noted that the motion frame sequence is the motion information corresponding to each smallest unit when the entity object moves. Therefore, the motion information determination device can determine the motion trajectory of the entity object performing target-guided actions in response to the target position information based on the motion frame sequence.

[0232] S509. Control the entity object to move along the motion trajectory.

[0233] It should be noted that after the motion information determination device obtains the motion trajectory information, it can control the entity object to move along the motion trajectory, thereby controlling the entity object to complete the execution of the target-oriented action.

[0234] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0235] See Figure 12 , Figure 12 This application provides an exemplary method for determining motion information; such as... Figure 12 As shown, this exemplary motion information determination method includes: a data acquisition module 12-1, a data annotation module 12-2, a data preprocessing module 12-3, a model training module 12-4, a model enhancement module 12-5, and a model deployment module 12-6, wherein:

[0236] The data acquisition module 12-1 is used to collect motion capture data (target-oriented motion data) using motion capture equipment. Here, the motion capture equipment can be clothing with special sensors. Professional stunt performers wear the motion capture equipment to perform cover-dodging actions (target-oriented actions), and the motion capture data of the cover-dodging actions can be collected through the motion capture equipment; thus, the motion capture data includes standing data, walking data, running data, turning data, and cover-dodging data, etc. In addition, the action execution phases corresponding to the take-cover action include the pre-cover preparation phase, the cover preparation phase (preparation phase), the take-cover entry phase (execution phase), the cover-in phase (action phase), and the exit phase (exit phase). The pre-cover preparation phase is the phase from when the game character (moving object) stands up to when they start to lift their first foot and run. The cover preparation phase is the phase from when the game character starts to lift their first foot and run until the fifth-to-last foot before taking cover lands. The take-cover entry phase is the phase from when the game character lifts their fourth-to-last foot before taking cover until the last foot lands. The cover-in phase is the phase when the game character remains stationary in the cover. The exit phase is the phase from when the game character gets up from being stationary to standing up again and entering the next action.

[0237] It should also be noted that the four steps during the bunker entry phase can be either "left foot, right foot, left foot, right foot" or "right foot, left foot, right foot, left foot." See also Figure 13 , Figure 13 This is a schematic diagram illustrating an exemplary execution stage provided in an embodiment of this application; as shown... Figure 13 As shown, when performing the cover-dodging action for cover 13-1, the four steps corresponding to the cover-entry phase 13-2 are "right foot, left foot, right foot, left foot," and the four steps corresponding to the cover-entry phase 13-3 are "left foot, right foot, left foot, right foot." Here, the black circle represents the game character's cover-dodging position (target position information), the arrow indicates the game character's orientation when dodging, the box indicates the landing point of the last four steps, L is the left foot, R is the right foot, and the number in the white circle indicates the step number. It is easy to see that when the cover-dodging position is different from the game character's orientation, the order of the left and right feet in the corresponding cover-entry phase is different: for the cover-dodging action of "turn left, right shoulder against the wall," the right foot landing is the first step of the cover-entry phase. After the left foot lands in the fourth step, the game character remains still and enters the cover-entry phase; for the cover-dodging action of "turn right, left shoulder against the wall," the left foot landing is the first step of the cover-entry phase. After the right foot lands in the fourth step, the game character remains still and enters the cover-entry phase.

[0238] The data annotation module 12-2 is used to annotate motion capture data. The annotation includes phase annotation and motion type annotation. For phase annotation, the phase corresponding to the frame information of the left foot landing (one frame of motion information) is determined to be 0, the phase corresponding to the frame information of the right foot landing is π, the phase from the left foot landing to the right foot landing is an interpolation from 0 to π (the phase of the first sub-motion cycle), and the phase from the right foot landing to the left foot landing is an interpolation from π to 2π (the phase of the second sub-motion cycle).

[0239] For the annotation of action type, the action type is represented by m components, where m is the number of action types (here, action types include standing type, running type and cover-dodging type (execution type), so m takes the value of 3); and each component takes the value range of 0 to 1, each component corresponds to one action type, and the sum of the m components of each frame information is 1.

[0240] See Figure 14 , Figure 14 This is a schematic diagram illustrating an exemplary annotation action type provided in an embodiment of this application; as shown... Figure 14 As shown, the horizontal axis represents the number of frames in the motion capture data, including 1200 frames; this motion capture data sequentially represents the process of standing, running, taking cover, and standing again, represented by the broken line. Figure 14-1 The line represents the values ​​of the standing type 14-11 components in each frame. Figure 14-2 The line represents the values ​​of the components of running type 14-21 in each frame information. Figure 14-3 This indicates the values ​​of the cover type 14-31 components in each frame. Action points 14-41 to 14-46 (first action point to sixth action point) represent six key time nodes: when starting to take cover, the game character is in a standing position; when the first foot is lifted, the corresponding frame number is action point 14-41. Figure 14 (Frame 200 in the original text) In all frames prior to action point 14-41, the standing component is 1, while the running and cover-dodging components are 0. When the game character lands its first foot, the corresponding frame number is action point 14-42. Figure 14 (Frame 300 is shown in the image). Between action points 14-41 and 14-42, the value of the standing component decreases uniformly from 1 to 0, while the value of the running component increases uniformly to 1, representing the transition from standing to running. Additionally, the value of the cover-dodging component is 0. When the character lands the fifth-to-last step before entering cover, the corresponding frame number is action point 14-43. Figure 14(Frame 500 is shown in the image). Between action points 14-42 and 14-43, the running component has a value of 1, while the standing and cover-taking components have values ​​of 0. The game character performs four steps to enter cover, and the frame number corresponding to the landing of the last step is action point 14-44. Figure 14 (Frame 700 is shown in the image). Between action points 14-43 and 14-44, the value of the running component decreases uniformly from 1 to 0, while the value of the cover-taking component increases uniformly from 0 to 1. The standing component is 0. Between action points 14-44 and 14-45, the cover-taking component is 1, while both the standing and running components are 0. When the game character begins to stand up, the corresponding frame number is action point 14-45. Figure 14 (The middle frame is frame 900). When the game character returns to a standing position, the frame number corresponds to action point 14-46 (in...). Figure 14 (The middle frame is frame 1000). Based on action points 14-41 to 14-46, the action type of each frame can be labeled.

[0241] The data preprocessing module 12-3 is used to convert the format of the labeled motion capture data so that the converted data format is the input format corresponding to the neural network model, thus obtaining the model training samples. See Table 1, which shows the input and output data of the neural network model:

[0242] Table 1

[0243]

[0244] In Table 1, the skeletal joint information refers to the object part information in the embodiments of this application; the phase refers to the phase information in the embodiments of this application. The skeletal joint information (position, velocity, direction), motion trajectory information (position, direction, action type, terrain information), object point information (position, direction, action type), and phase refer to the motion information samples of each frame in the embodiments of this application.

[0245] Model training module 12-4 is used to train the NSM neural network model (the motion prediction model to be trained) based on the model training samples, and obtain the initial NSM neural network model (motion prediction model).

[0246] Model enhancement module 12-5 deploys the initial NSM neural network model to a game simulator for testing. During testing, the game character triggers a cover-avoidance action at different distances from the cover. After post-processing (see equation (1) and...), the model is then... Figure 8-9After implementing corrections and inverse motion adjustments, the game character accurately hides at the designated cover position (target position information), and the animation of hiding at that cover is captured. After processing by the data preprocessing module 12-3, it is added to the dataset composed of motion capture data, and the initial NSM neural network model is trained again, with NSM weights updated. The updated initial NSM neural network model is then redeployed to the game simulator, and the cover-hitting action is triggered at different distances from the cover by the game character, repeating the above iterative update process. It should be noted that in the dataset, the motion capture data has the highest motion quality, so the dataset corresponding to the motion capture data is always retained; while the dataset generated by the NSM neural network and then post-processed, when a new dataset is added, the historically generated dataset has poor motion quality, so the historically generated dataset is deleted. Finally, at the end of the iterative update, the enhanced NSM neural network model (enhanced motion prediction model) is obtained.

[0247] Model deployment module 12-6: After training, the NSM network will be deployed to the game client. In the game client, the motion state of the current game character (motion information of frame i) is extracted and processed by data preprocessing module 12-3 as input to the enhanced NSM neural network model. After forward propagation of the enhanced NSM neural network model, the network output (motion update of frame i+1) is obtained. The post-processed motion state of the network output (motion information of frame i+1) will be assigned to the state of the game character, generating the scene of frame i+1.

[0248] See Figure 15 , Figure 15 This is a schematic diagram of an exemplary target motion animation provided in an embodiment of this application; as shown... Figure 15 The image shown is a screenshot of the animation of a game character hiding behind cover, obtained based on a deployed reinforced NSM neural network model. Figure 15-1 To the end Figure 15-5 It describes the process of a game character turning around, running quickly towards cover, turning around and hiding in cover, and stopping precisely in a designated position.

[0249] It is understood that the motion information determination method provided in this application embodiment can meet the requirement of precise control of the endpoint for cover-hiding actions in shooting games. Furthermore, by using the motion information determination method provided in this application embodiment, a small amount of data can be collected to automatically generate cover-hiding motion animations for game characters under various conditions (different distances, angles, and orientations between the character and cover), reducing game development work, shortening the development process, and also reducing the storage memory occupied by the game.

[0250] The following description continues to illustrate the exemplary structure of the motion information determination device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 4 As shown, the software module stored in the motion information determining device 455 in the memory 450 may include:

[0251] Information acquisition module 4551 is used to acquire the motion information and target position information of the i-th frame of the moving object, where i is a positive integer;

[0252] Information prediction module 4552 is used to predict the motion update amount of the moving object in the (i+1)th frame and the estimated target position information based on the motion information of the i-th frame and the target position information.

[0253] The information correction module 4553 is used to correct the motion update amount of the (i+1)th frame based on the difference between the estimated target position information and the target position information, and to obtain the motion information of the (i+1)th frame based on the corrected motion update amount of the (i+1)th frame.

[0254] The information iteration module 4554 is used to continue iterating until the (i+n)th frame motion information corresponding to the target position information is obtained, where n is a positive integer greater than 1;

[0255] The information determination module 4555 is used to obtain a motion frame sequence including the motion information of the i-th frame to the motion information of the (i+n)-th frame, wherein the motion frame sequence is a set of motion information of the moving object performing a target-guided action in response to the target position information.

[0256] In this embodiment, the motion update amount of the (i+1)th frame includes the object point position update amount of the (i+1)th frame, the motion information of the i-th frame includes the object point position information of the i-th frame, and the motion information of the (i+1)th frame includes the object point position information of the (i+1)th frame; the information correction module 4553 is further configured to perform vector difference calculation based on the estimated target position information and the target position information to obtain an initial correction vector; adjust the initial correction vector based on the correction coefficient to obtain a correction vector; use the correction vector to correct the object point position update amount of the (i+1)th frame, and then superimpose the corrected object point position update amount of the (i+1)th frame with the object point position information of the i-th frame to obtain the object point position information of the (i+1)th frame in the motion information of the (i+1)th frame.

[0257] In this embodiment of the application, the correction coefficient is positively correlated with the speed of the moving object.

[0258] In this embodiment of the application, when the motion distance between the object point position information in the i-th frame and the target position information falls within the distance range, the correction coefficient is calculated based on the motion distance and the object point position update amount in the (i+1)-th frame; when the motion distance falls outside the distance range, the correction coefficient is a constant.

[0259] In this embodiment, when the moving object is a virtual object, the motion information of the (i+1)th frame further includes the position information of the object part in the (i+1)th frame, and the motion update amount of the (i+1)th frame further includes the relative amount of the object part position in the (i+1)th frame; the information correction module 4553 is further configured to superimpose the relative amount of the object part position in the (i+1)th frame onto the object point position information in the (i+1)th frame to obtain the position information of the object part to be adjusted in the (i+1)th frame; superimpose both the relative amount of the object part position in the (i+1)th frame and the object point position update amount in the (i+1)th frame onto the object point position information in the (i)th frame to obtain the reference position information of the object part in the (i+1)th frame; and adjust the position information of the object part to be adjusted in the (i+1)th frame based on the reference position information of the object part in the (i+1)th frame to obtain the position information of the object part in the (i+1)th frame in the motion information of the (i+1)th frame.

[0260] In this embodiment, the reference position information of the object part in the (i+1)th frame includes ankle reference position information and toe reference position information, and the position information of the object part to be adjusted in the (i+1)th frame includes hip position information to be adjusted, knee position information to be adjusted, and ankle position information to be adjusted. The information correction module 4553 is further configured to adjust the ankle position information to be adjusted to the ankle reference position information by rotating the knee position information to be adjusted and the hip position information to be adjusted, thereby determining the knee position information in the (i+1)th frame; and to determine the knee position information, the hip position information to be adjusted, the ankle reference position information, and the toe reference position information in the (i+1)th frame motion information as the position information of the object part in the (i+1)th frame.

[0261] In this embodiment, the information correction module 4553 is further configured to: determine the knee rotation direction based on the direction from the knee position information to the ankle position information to be adjusted and the direction from the knee position information to the hip position information to be adjusted; rotate the knee position information to be adjusted about the knee rotation direction as the axis of rotation to adjust the ankle position information to be adjusted based on the distance between the hip position information to be adjusted and the ankle reference position information; determine the hip rotation direction based on the direction from the hip position information to the ankle reference position information and the direction from the hip position information to the adjusted ankle position information; and rotate the hip position information to be adjusted about the hip rotation direction as the axis of rotation to adjust the target ankle position information to the ankle reference position information, thereby determining the (i+1)th frame knee position information.

[0262] In this embodiment of the application, the information prediction module 4552 is further configured to use a motion prediction model to predict the motion information of the i-th frame and the target position information to obtain the motion update amount of the moving object in the (i+1)-th frame and the estimated target position information based on the motion update amount of the (i+1)-th frame, wherein the motion prediction model is used to predict the motion information of the moving object.

[0263] In this embodiment, the motion information determination device 455 further includes a model training module 4556, used to acquire model training samples, wherein the model training samples include at least one frame of motion samples and target position samples; using a motion prediction model to be trained to predict the j-th frame of motion samples and the target position samples in the at least one frame of motion samples to obtain the (j+1)-th frame of motion information, wherein the motion prediction model to be trained is a model to be trained for predicting motion information, and j is a positive integer greater than 1; based on the difference between the (j+1)-th frame of motion information and the (j+1)-th frame of motion samples, the motion prediction model to be trained is trained to obtain the motion prediction model.

[0264] In this embodiment, the motion information determination device 455 further includes a model enhancement module 4557, used to acquire the k-th frame motion information and training position information of the training object, where k is a positive integer; to predict the k-th frame motion information and the training position information using the motion prediction model, to obtain the (k+1)-th frame motion update amount and the estimated training position information based on the (k+1)-th frame motion update amount of the training object; to correct the (k+1)-th frame motion update amount based on the difference between the estimated training position information and the training position information, and to obtain the (k+1)-th frame motion information based on the corrected (k+1)-th frame motion update amount; and to train the motion prediction model based on the k-th frame motion information and the (k+1)-th frame motion information to obtain an enhanced motion prediction model.

[0265] In this embodiment of the application, the information prediction module 4552 is further configured to use the enhanced motion prediction model to predict the motion information of the i-th frame and the target position information.

[0266] In this embodiment of the application, the model enhancement module 4557 is further configured to add the motion information of the k-th frame and the motion information of the (k+1)-th frame to the dataset including the model training samples; delete motion information in the dataset that meets the deletion conditions to obtain an enhanced dataset; and train the motion prediction model based on the enhanced dataset to obtain the enhanced motion prediction model.

[0267] In this embodiment of the application, when the moving object is a physical object, the motion information determining device 455 further includes a motion control module 4558, which is used to determine the motion trajectory of the physical object based on the motion frame sequence and control the physical object to move along the motion trajectory.

[0268] In this embodiment of the application, the information acquisition module 4551 is further configured to, in response to an animation generation request sent by the rendering device, acquire the i-th frame motion information and the target position information of the virtual object, wherein the animation generation request is generated when the rendering device receives a target guiding action execution operation.

[0269] In this embodiment of the application, the animation information determining device 455 further includes an animation sending module 4559, which is used to generate a target action animation based on the motion frame sequence; send the target action animation to the rendering device so that the rendering device plays the target action animation and renders a virtual scene in which the virtual object performs the target guiding action in response to the target position information.

[0270] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the motion information determination method described in this application.

[0271] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the motion information determination method provided in this application. For example, ... Figure 5 The method for determining motion information is shown.

[0272] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0273] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0274] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0275] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0276] In summary, through the embodiments of this application, since the motion information of the (i+1)th frame predicted based on the motion information of the moving object in the i-th frame and the target position information is corrected based on the difference between the estimated target position information and the target position information before being used as the basis for determining the motion information of subsequent frames; that is, the accuracy of the motion information of the target in the (i+1)th frame is high, and therefore, the accuracy of the motion information of subsequent frames predicted and corrected based on the motion information of the (i+1)th frame and the target position information is also high. Therefore, the deviation between the position corresponding to the last frame of motion information and the target position information can be reduced, and thus the accuracy of the determined motion frame sequence is high, which can improve the accuracy of the motion information of the moving object performing the target-guided action. In addition, when performing the target-guided action, automatic footwork planning and adjustment can also be realized, further improving the accuracy of the motion information.

[0277] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A motion information determination method characterized by comprising: The method comprises the following steps: obtaining the i-th frame motion information of a moving object and target position information, wherein i is a positive integer; based on the i-th frame motion information and the target position information, predicting the i+1-th frame motion update amount of the moving object and the estimated target position information based on the i+1-th frame motion update amount; based on the difference between the estimated target position information and the target position information, correcting the i+1-th frame motion update amount, and obtaining the i+1-th frame motion information according to the corrected i+1-th frame motion update amount; iterating until the i+n-th frame motion information corresponding to the target position information is obtained, wherein n is a positive integer greater than 1; obtaining a motion frame sequence comprising the i-th frame motion information to the i+n-th frame motion information, wherein the motion frame sequence is a set of motion information of the moving object performing a target-oriented action for the target position information.

2. The method of claim 1, wherein, The i+1-th frame motion update amount comprises an i+1-th frame object point position update amount, the i-th frame motion information comprises i-th frame object point position information, and the i+1-th frame motion information comprises i+1-th frame object point position information. The method comprises the following steps: performing vector difference calculation according to the estimated target position information and the target position information to obtain an initial correction vector; adjusting the initial correction vector based on a correction coefficient to obtain a correction vector; using the correction vector to correct the i+1-th frame object point position update amount, and then superimposing the corrected i+1-th frame object point position update amount and the i-th frame object point position information to obtain the i+1-th frame object point position information in the i+1-th frame motion information.

3. The method of claim 2, wherein, The correction coefficient is positively correlated with the motion speed of the moving object.

4. The method of claim 3, wherein: when the motion distance between the i-th frame object point position information and the target position information falls within a distance range, the correction coefficient is calculated according to the motion distance and the i+1-th frame object point position update amount; when the motion distance falls outside the distance range, the correction coefficient is a constant.

5. The method according to any one of claims 2 to 4, characterized in that, When the moving object is a virtual object, the i+1-th frame motion information further comprises i+1-th frame object part position information, and the i+1-th frame motion update amount further comprises i+1-th frame object part position relative amount. After obtaining the i+1-th frame object point position information in the i+1-th frame motion information, the method further comprises the following steps: superimposing the i+1-th frame object part position relative amount on the i+1-th frame object point position information to obtain i+1-th frame object part to-be-adjusted position information; superimposing the i+1-th frame object part position relative amount and the i+1-th frame object point position update amount on the i-th frame object point position information to obtain i+1-th frame object part reference position information; adjust the object part to-be-adjusted position information in the i+1th frame based on the object part reference position information in the i+1th frame, to obtain the object part position information in the i+1th frame in the i+1th frame motion information.

6. The method of claim 5, wherein, The object part reference position information in the i+1th frame includes ankle reference position information and toe reference position information, and the object part to-be-adjusted position information in the i+1th frame includes to-be-adjusted hip position information, to-be-adjusted knee position information and to-be-adjusted ankle position information. The adjusting of the object part to-be-adjusted position information in the i+1th frame based on the object part reference position information in the i+1th frame, to obtain the object part position information in the i+1th frame in the i+1th frame motion information, includes: adjusting the to-be-adjusted ankle position information to the ankle reference position information by rotating the to-be-adjusted knee position information and the to-be-adjusted hip position information, to determine the i+1th knee position information; determining the i+1th knee position information, the to-be-adjusted hip position information, the ankle reference position information and the toe reference position information as the object part position information in the i+1th frame in the i+1th frame motion information.

7. The method of claim 6, wherein, The adjusting of the to-be-adjusted ankle position information to the ankle reference position information by rotating the to-be-adjusted knee position information and the to-be-adjusted hip position information, to determine the i+1th knee position information, includes: determining a knee rotation direction based on a direction in which the to-be-adjusted knee position information points to the to-be-adjusted ankle position information, and a direction in which the to-be-adjusted knee position information points to the to-be-adjusted hip position information; rotating the to-be-adjusted knee position information about the knee rotation direction, to adjust the to-be-adjusted ankle position information based on a distance between the to-be-adjusted hip position information and the ankle reference position information; determining a hip rotation direction based on a direction in which the to-be-adjusted hip position information points to the ankle reference position information, and a direction in which the to-be-adjusted hip position information points to the adjusted to-be-adjusted ankle position information; rotating the to-be-adjusted hip position information about the hip rotation direction, to adjust the target to-be-adjusted ankle position information to the ankle reference position information, to determine the i+1th knee position information.

8. The method of claim 5, wherein, The predicting of the i+1th frame motion update amount of the moving object and the estimated target position information based on the i+1th frame motion update amount based on the i th frame motion information and the target position information, includes: predicting the i th frame motion information and the target position information by using a motion prediction model, to obtain the i+1th frame motion update amount of the moving object and the estimated target position information based on the i+1th frame motion update amount, wherein the motion prediction model is used to predict motion information of the moving object.

9. The method of claim 8, wherein, Before the predicting of the i th frame motion information and the target position information by using the motion prediction model, the method further includes: obtaining a model training sample, wherein the model training sample includes at least one frame of motion sample and target position sample; predicting, by using a to-be-trained motion prediction model, a (j+1)th frame of motion information based on a jth frame of motion sample in the at least one frame of motion sample and the target position sample, where the to-be-trained motion prediction model is a model to be trained for predicting motion information, and j is a positive integer greater than 1; training the to-be-trained motion prediction model based on a difference between the (j+1)th frame of motion information and a (j+1)th frame of motion sample, to obtain the motion prediction model.

10. The method of claim 9, wherein, After the motion prediction model is obtained, the method further includes: obtaining a kth frame of motion information of a training object and training position information, where k is a positive integer; predicting, by using the motion prediction model, the kth frame of motion information and the training position information, to obtain a (k+1)th frame of motion update amount of the training object and estimated training position information based on the (k+1)th frame of motion update amount; correcting the (k+1)th frame of motion update amount based on a difference between the estimated training position information and the training position information, and obtaining a (k+1)th frame of motion information according to the corrected (k+1)th frame of motion update amount; training the motion prediction model based on the kth frame of motion information and the (k+1)th frame of motion information, to obtain a reinforced motion prediction model; The prediction, by using the motion prediction model, of the ith frame of motion information and the target position information includes: predicting, by using the reinforced motion prediction model, the ith frame of motion information and the target position information.

11. The method of claim 10, wherein, The training, based on the kth frame of motion information and the (k+1)th frame of motion information, of the motion prediction model to obtain a reinforced motion prediction model includes: adding the kth frame of motion information and the (k+1)th frame of motion information to a data set including the model training sample; deleting motion information in the data set that meets a deletion condition, to obtain a reinforced data set; training the motion prediction model based on the reinforced data set, to obtain the reinforced motion prediction model.

12. The method according to any one of claims 1 to 4, characterized in that, When the motion object is a physical object, after the motion frame sequence including the ith frame of motion information to the (i+n)th frame of motion information is obtained, the method further includes: determining a motion trajectory of the physical object based on the motion frame sequence; controlling the physical object to move along the motion trajectory.

13. The method of claim 5, wherein, The obtaining of the ith frame of motion information of the motion object and the target position information includes: in response to an animation generation request sent by a rendering device, obtaining the ith frame of motion information of the virtual object and the target position information, where the animation generation request is generated when the rendering device receives a target guiding action execution operation; After the motion frame sequence including the ith frame of motion information to the (i+n)th frame of motion information is obtained, the method further includes: generating a target action animation based on the motion frame sequence; sending the target action animation to the rendering device, so that the rendering device plays the target action animation and renders a virtual scene in which the virtual object performs the target guiding action on the target position information.

14. A motion information determining apparatus, characterized by comprising: The device includes: The information acquisition module is used to acquire the motion information and target position information of the i-th frame of the moving object, where i is a positive integer; The information prediction module is used to predict the motion update amount of the moving object in the (i+1)th frame and the estimated target position information based on the motion information of the i-th frame and the target position information. The information correction module is used to correct the motion update amount of the (i+1)th frame based on the difference between the estimated target location information and the target location information, and to obtain the motion information of the (i+1)th frame based on the corrected motion update amount of the (i+1)th frame. The information iteration module is used to continue iterating until the (i+n)th frame of motion information corresponding to the target position information is obtained, where n is a positive integer greater than 1; The information determination module is used to obtain a motion frame sequence including the motion information of the i-th frame to the motion information of the (i+n)-th frame, wherein the motion frame sequence is a set of motion information of the moving object performing a target-guided action in response to the target position information.

15. The apparatus of claim 14, wherein, The motion update amount of the (i+1)th frame includes the object point position update amount of the (i+1)th frame, and the motion information of the i-th frame includes the object point position information of the i-th frame. The information correction module is further configured to perform vector difference calculation based on the estimated target location information and the target location information to obtain an initial correction vector; The initial correction vector is adjusted based on the correction coefficient to obtain the correction vector; After correcting the object point position update amount in the (i+1)th frame using the correction vector, the corrected object point position update amount in the (i+1)th frame is superimposed with the object point position information in the i-th frame to obtain the object point position information in the motion information of the (i+1)th frame.

16. The apparatus of claim 15, wherein, The correction factor is positively correlated with the speed of the moving object.

17. The apparatus of claim 16, wherein, When the movement distance between the object point position information in the i-th frame and the target position information falls within the distance range, the correction coefficient is calculated based on the movement distance and the object point position update amount in the (i+1)-th frame. When the movement distance falls outside the distance range, the correction coefficient is a constant.

18. The apparatus of any of claims 15-17, wherein, When the moving object is a virtual object, the motion information of the (i+1)th frame also includes the position information of the object part in the (i+1)th frame, and the motion update amount of the (i+1)th frame also includes the relative amount of the position of the object part in the (i+1)th frame. The information correction module is also used to superimpose the relative position of the object part in the (i+1)th frame onto the object point position information in the (i+1)th frame to obtain the position information of the object part to be adjusted in the (i+1)th frame. The relative position of the object part in the (i+1)th frame and the update amount of the object point position in the (i+1)th frame are both superimposed on the object point position information in the i-th frame to obtain the reference position information of the object part in the (i+1)th frame. Based on the reference position information of the object part in the (i+1)th frame, adjust the position information of the object part to be adjusted in the (i+1)th frame to obtain the position information of the object part in the (i+1)th frame in the motion information of the (i+1)th frame.

19. A motion information determining device, characterized by include: a memory for storing executable instructions; a processor for executing the executable instructions stored in the memory to implement the method of any one of claims 1 to 13.

20. A computer-readable storage medium, characterized in that, executable instructions stored in the memory for being executed by a processor to implement the method of any one of claims 1 to 13.

21. A computer program product comprising computer-executable instructions or a computer program, characterized in that, the computer executable instructions or computer program are executed by a processor to implement the method of any one of claims 1 to 13.

Images