Object action determination method and apparatus, storage medium, and electronic device

Abstract

This disclosure provides a method, apparatus, storage medium, and electronic device for determining object actions. The method includes: acquiring a current action information set and a target action information set matching a target object, wherein the current action information set indicates a first object action of the target object at a first moment, and the target action information set indicates a second object action of the target object at a second moment; determining a reference action information set based on the current action information set and the target action information set; inputting a subset of target action information associated with a target object part from the reference action information set, and a subset of current action information associated with a target object part from the current action information set, into a part action prediction network to obtain a subset of part action information matching the target object part; and determining a transition action information set based on the subset of part action information and the reference action information set, wherein the transition action information set indicates a transition object action of the target object at a third moment.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to a method and apparatus for determining the action of an object, a storage medium, and an electronic device. Background Technology

[0002] In the process of designing or rendering the object actions of virtual objects, the object actions of keyframes are usually obtained first, and then the transition actions between keyframes are determined based on the object actions of two keyframes.

[0003] Existing methods for determining or rendering transition animations typically rely on interpolation or animation curves to define the transition between two adjacent key actions. However, transition animations generated through interpolation or animation curves can sometimes appear unnatural, affecting the final visual effect. In other words, existing methods for determining object animations suffer from the technical problem of poor display quality.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method and apparatus for determining object actions, a storage medium, and an electronic device, to at least solve the technical problem of poor display effect of object actions in related fields.

[0006] According to one aspect of the present invention, a method for determining object actions is provided, comprising: acquiring a current action information set and a target action information set matching a target object, wherein the current action information set is used to indicate a first object action of the target object at a first time moment, and the target action information set is used to indicate a second object action of the target object at a second time moment, the second time moment being later than the first time moment; determining a reference action information set based on the current action information set and the target action information set, wherein the reference action information set includes action description information for determining a transitional object action of the target object; inputting a subset of target action information associated with a target object part in the reference action information set and a subset of current action information associated with the target object part in the current action information set into a part action prediction network to obtain a subset of part action information matching the target object part; and determining a transitional action information set based on the subset of part action information and the reference action information set, wherein the transitional action information set is used to indicate a transitional object action of the target object at a third time moment, the third time moment being the time between the first time moment and the second time moment.

[0007] According to another aspect of the present invention, an apparatus for determining object actions is also provided, comprising: an acquisition unit, configured to acquire a current action information set and a target action information set matching a target object, wherein the current action information set is used to indicate a first object action of the target object at a first time moment, and the target action information set is used to indicate a second object action of the target object at a second time moment, the second time moment being later than the first time moment; a first determination unit, configured to determine a reference action information set based on the current action information set and the target action information set, wherein the reference action information set includes action description information for determining a transitional object action of the target object; a prediction unit, configured to input a subset of target action information associated with a target object part from the reference action information set and a subset of current action information associated with the target object part from the current action information set into a part action prediction network to obtain a subset of part action information matching the target object part; and a second determination unit, configured to determine a transitional action information set based on the subset of part action information and the reference action information set, wherein the transitional action information set is used to indicate a transitional object action of the target object at a third time moment, the third time moment being the time between the first time moment and the second time moment.

[0008] Optionally, the prediction unit includes: a gating module, used to input the target action information subset and the current action information subset into the gating network of the part action prediction network to obtain weight coefficients corresponding to each of the plurality of action decoding modules in the part action prediction network; a decoding module, used to input at least one object information from the target action information subset and the current action information subset into at least one of the plurality of action decoding modules according to the weight coefficients; and a determining module, used to determine the part action information subset according to the output results of each of the at least one of the action decoding modules.

[0009] Optionally, the gating unit is further configured to: determine the predicted linear velocity information corresponding to each of the multiple root nodes in the reference motion information set as a subset of the target motion information associated with the target object part, wherein the target object is a virtual object controlled by multiple object nodes with a hierarchical control structure, and the root node is an object node at the first level among the multiple object nodes; determine the first current linear velocity information, the first current rotation information, the second current linear velocity information, and the second current rotation information corresponding to each of the multiple root nodes in the current motion information set as a subset of the current motion information associated with the target object part, wherein the multiple target object nodes are used to drive the target object part of the target object.

[0010] Optionally, the determining module is configured to: obtain the predicted linear velocity information and the predicted rotation information corresponding to each of the multiple target object nodes from the output results of at least one of the above-mentioned action decoding modules; and determine the predicted linear velocity information and the predicted rotation information corresponding to each of the multiple target object nodes as a subset of the above-mentioned part action information.

[0011] Optionally, the second determining unit includes: an acquisition module, configured to acquire a subset of reference action information from the reference action information set, wherein the subset of reference action information includes first predicted rotation information corresponding to each of a plurality of reference object nodes and second predicted rotation information corresponding to each of a plurality of root nodes, wherein the plurality of reference object nodes are used to drive reference object parts of the target object, and the reference object parts are object parts of the target object other than the target object parts; and a second determining module, configured to determine the transition action information set based on the subset of reference action information and the subset of part action information.

[0012] Optionally, the prediction unit is further configured to: when the reference action information set is a decoded information set obtained by decoding according to the long short-term memory network, obtain a subset of latent variable information from the reference action information set; and input at least one object information from the latent variable information subset, the target action information subset, and the current action information subset into at least one of the multiple action decoding modules according to the weight coefficients.

[0013] Optionally, the prediction unit is configured to: determine the target object part that matches the action type based on the action types of the first object action and the second object action; and determine the target object part based on the interaction position of the interaction element associated with the first object action and the second object action.

[0014] Optionally, the above-mentioned object action determination device further includes: a transition action sequence determination unit, configured to determine the transition action information set as the updated current action information set when the transition action information set satisfies the action sequence prediction condition, and to repeatedly execute the following steps until a target transition information set sequence is obtained, wherein the target transition information set sequence is used to indicate the transition object action sequence of the target object, the transition object action sequence is used to connect the initial object action and the second object action of the target object, and the target transition information set sequence includes the transition action information set: determining the next transition action information set included in the target transition information set sequence based on the updated current action information set and the target action information set; and determining the next transition action information set as the updated current action information set when the next transition action information set satisfies the action sequence prediction condition.

[0015] Optionally, the transition action sequence determination unit is configured to: determine at least one subset of state information based on the updated current action information set and the target action information set; obtain the state encoding result corresponding to each of the at least one subset of state information; determine the target fusion encoding result based on the at least one state encoding result; and determine the next transition action information set included in the target transition information set sequence based on the target fusion encoding result.

[0016] Optionally, the aforementioned transition action sequence determination unit is used for at least one of the following: when at least one of the aforementioned state information subsets includes the current node state information of multiple object nodes in the aforementioned current action information set, the encoding result of the aforementioned current state information subset is determined as the current state code, wherein the aforementioned state encoding result includes the aforementioned current state code, the aforementioned current node state information of the multiple object nodes includes linear velocity information corresponding to each of the multiple target object nodes, velocity information corresponding to each of the multiple root nodes, and rotation information corresponding to each of the multiple object nodes, and the aforementioned target object nodes are used to drive the target object parts of the target object; when at least one of the aforementioned state information subsets includes the target node state information of multiple object nodes in the aforementioned target action information set, the encoding result of the aforementioned current state information subset is determined as the target state code, wherein the aforementioned state encoding result includes the aforementioned target state code, the aforementioned target node state information of the multiple object nodes includes target position information corresponding to each of the multiple target object nodes, and multiple object The target rotation information corresponding to each node; in at least one of the above-mentioned state information subsets, the current state information subset includes node state difference information of multiple object nodes determined according to the current action information set and the target action information set, and the encoding result of the current state information subset is determined as offset state encoding, wherein the state encoding result includes the offset state encoding, and the node state difference information of multiple object nodes includes position difference information corresponding to multiple target object nodes, position difference information corresponding to multiple root nodes, and rotation difference information corresponding to multiple object nodes; in at least one of the above-mentioned state information subsets, the current state information subset includes velocity state information of multiple object nodes in the target action information set, and the node-aware convolution result of the current state information subset is obtained, and the rhythm state encoding is determined according to the node-aware convolution result, wherein the state encoding result includes the rhythm state encoding, and the velocity state information of multiple object nodes includes linear velocity information of multiple object nodes and rotation information of multiple object nodes.

[0017] Optionally, the transition action sequence determination unit is configured to: obtain a current position encoding result determined based on a subset of position information in the current action information set; superimpose the current position encoding result with at least one of the state encoding results to obtain a target encoding result corresponding to each of the at least one state encoding result; determine the target fusion encoding result based on at least one target encoding result; further be configured to: decode the target fusion encoding result using a long short-term memory network to obtain the reference action information set; input the subset of target action information associated with the target object part in the reference action information set and the subset of current action information associated with the target object part in the current action information set into the part action prediction network to obtain the subset of part action information matching the target object part; and determine the next transition action information set included in the target action information set sequence based on the subset of part action information and the reference action information set.

[0018] Optionally, the aforementioned transition action sequence determination unit further includes: a rendering module, configured to sequentially add the transition action information sets included in the target transition information set sequence to a rendering cache queue, wherein the transition action information in the rendering cache queue is used to render the transition object action of the target object; obtain from the rendering cache queue a plurality of first action information sets that have not yet been used to render the transition object action of the target object, and obtain compensation information sets corresponding to each of the plurality of first action information sets, and adjust the plurality of first action information sets according to the plurality of compensation information sets to obtain a plurality of second action information sets; and replace the plurality of first action information sets in the rendering cache queue with the plurality of second action information sets.

[0019] Optionally, the transition action sequence determination unit is configured to: determine a sequence of first action information sets and a target action information set as a sequence of first action information sets; determine a first state difference information set based on the first first action information set and the target action information set in the sequence of first action information sets, wherein the first state difference information set includes first node state differences corresponding to each of a plurality of object nodes; determine a second state difference information set corresponding to each of every two adjacent first action information sets in the sequence of first action information sets, wherein the second state difference information set includes second node state differences corresponding to each of a plurality of node objects; and determine a compensation information set corresponding to each of the plurality of first action information sets based on the first state difference information set and the second state difference information set.

[0020] Optionally, the aforementioned transition action sequence determination unit is further configured to: acquire playback indication information matching the aforementioned target transition information set sequence, wherein the playback indication information is used to indicate the playback speed of the transition object action between the aforementioned initial object action and the aforementioned second object action; determine at least one of the aforementioned transition action information sets matching the current time based on the aforementioned playback indication information; and determine a current transition action information set matching the aforementioned current time based on a weighted result among at least one of the aforementioned transition action information sets, wherein the current transition action information set is used to render the aforementioned transition object action of the aforementioned target object at the aforementioned current time.

[0021] Optionally, the aforementioned transition action sequence determination unit is further configured to: acquire a first action information set and a second action information set matching the target object, wherein the first action information set is used to indicate the initial object action of the target object at an initial time, and the second action information set is used to indicate the target object action of the target object at a target time; adjust the first action information set and the second action information set according to the model adjustment weight matching the target object to obtain an initial action information set and a target action information set, wherein the initial action information set and the target action information set are used to determine the target transition information set sequence; after determining the transition action information set as the updated current action information set and repeatedly executing the following steps until the target transition information set sequence is obtained, the unit is further configured to: adjust the weight according to the model adjustment weight matching the target object to restore multiple transition action information sets in the target transition information set sequence to obtain multiple target transition action information sets, wherein the multiple target transition action information sets are used to render the transition object action of the target object in the virtual scene.

[0022] Optionally, the above-mentioned object action determination device further includes: a training unit, configured to acquire an initial sample action information set and a target sample action information set matching the sample object, wherein the initial sample action information set is used to indicate the initial object action of the sample object at an initial time, and the target action information set is used to indicate the target object action of the sample object at a target time, the target time being later than the initial time; inputting the current action information set and the target action information set into a transitional action sequence prediction model to obtain a sample transitional action information set sequence, wherein the transitional action sequence prediction model includes a long short-term memory network and the above-mentioned part action prediction network, the long short-term memory network being used to determine a reference sample action information set based on the initial sample action information set and the target sample action information set. The aforementioned part action prediction network is used to determine a subset of sample part action information that matches the target object part based on a subset of target sample action information associated with the target object part in the aforementioned reference sample action information set and a subset of initial action information associated with the target object part in the aforementioned initial sample action information set. The subset of sample part action information and the aforementioned reference sample action information set are used to determine the sample transition action information set in the aforementioned sample transition action information set sequence. Based on the label transition action information set sequence that matches the aforementioned initial sample action information set and the aforementioned target sample action information set, and the aforementioned sample transition action information set sequence, the current model training loss is determined. Based on the aforementioned current model training loss, at least one model parameter in the aforementioned transition action sequence prediction model is adjusted.

[0023] Optionally, the training unit is configured to input the initial sample action information set and the target sample action information set into the long short-term memory network of the transition action sequence prediction model to obtain a reference sample action information set, wherein the reference sample action information set includes action description information for determining the transition object action of the sample object; input the subset of target sample action information associated with the target object part of the sample object in the reference sample action information set, and the subset of current sample action information associated with the target object part in the initial sample action information set, into the part action prediction network to obtain a part sample action information subset matching the target object part; and determine the transition sample action information set based on the part sample action information subset and the reference sample action information set, wherein the transition sample action information set is used to indicate the transition object action of the target object at a third time, the third time being the time between the first time and the second time.

[0024] Optionally, the training unit is configured to input the target sample action information subset and the current sample action information subset into the gating network of the part action prediction network to obtain sample weight coefficients corresponding to each of the multiple action decoding modules in the part action prediction network; input at least one object information from the target sample action information subset and the current sample action information subset into at least one of the multiple action decoding modules according to the sample weight coefficients; and determine the part sample action information subset according to the output results of each of the at least one action decoding module.

[0025] Optionally, the training unit is used for at least one of the following: determining a rotation information loss based on the label transition action information set sequence and the sample transition action information set sequence, wherein the rotation information loss is used to indicate a first information difference between the label rotation information corresponding to each object node included in the label transition action information set sequence and the sample rotation information corresponding to each object node included in the sample transition action information set sequence; determining a root node position loss based on the label transition action information set sequence and the sample transition action information set sequence, wherein the root node position loss is used to indicate a first information difference between the label rotation information corresponding to each root node included in the label transition action information set sequence and the sample rotation information set sequence. The location information is compared with the second information difference between the location information and the sample location information corresponding to each root node included in the sample transition action information set sequence; based on the label transition action information set sequence and the sample transition action information set sequence, the target object part information loss is determined, wherein the rotation information loss is used to indicate the third information difference between the label location information corresponding to each target object node included in the label transition action information set sequence and the sample location information corresponding to each target object node included in the sample transition action information set sequence, where the target object node is used to drive the target object part of the target object; the sliding information loss is determined based on the label transition action information set sequence. The aforementioned sliding information loss is used to indicate the degree of sliding of the target object on the virtual ground as indicated by the label transition action information set sequence; ground contact information loss is determined based on the aforementioned label transition action information set sequence and the aforementioned sample transition action information set sequence, wherein the aforementioned rotation information loss is used to indicate the fourth information difference between the label ground contact information indicated by the label transition action information set sequence and the sample ground contact information indicated by the aforementioned sample transition action information set sequence, wherein the aforementioned sample ground contact information is used to indicate the contact state between the virtual foot of the target object and the virtual ground; and object position information loss is determined based on the aforementioned label transition action information set sequence and the aforementioned sample transition action information set sequence, wherein... The aforementioned object position information loss is used to indicate the fifth information difference between the label position information corresponding to each of the multiple object nodes included in the label transition action information set sequence and the sample position information corresponding to each of the multiple object nodes included in the sample transition action information set sequence; based on the aforementioned label transition action information set sequence and the aforementioned sample transition action information set sequence, the object velocity information loss is determined, wherein the aforementioned object position information loss is used to indicate the sixth information difference between the label velocity information corresponding to each of the multiple object nodes included in the label transition action information set sequence and the sample velocity information corresponding to each of the multiple object nodes included in the sample transition action information set sequence.

[0026] Optionally, the training unit is further configured to: acquire multiple sets of weight coefficients included in the trained transition action sequence prediction model; determine at least one set of reference coefficients from the multiple sets of weight coefficients based on the amount of data corresponding to each set of weight coefficients; quantize the weight coefficients in the at least one set of reference coefficients to obtain at least one set of target coefficients after processing; update the transition action sequence prediction model based on the at least one set of target coefficients to obtain a target transition action sequence prediction model; and send the target transition action sequence prediction model to the target terminal, wherein the target transition action sequence prediction model running on the target terminal is used to determine the transition action information set of the target object in the game task during the game task execution.

[0027] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to perform the above-described method for determining object actions when running.

[0028] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for determining the object action as described above.

[0029] According to another aspect of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to perform the object action determination method through the computer program.

[0030] In the above-described method for determining object actions, after obtaining the current action information set and the target action information set for determining the transition actions of the target object, a reference action information set is first determined based on the current action information set and the target action information set. It is understood that the reference action information set can be used to initially determine the initial transition actions of the target object. Next, in this embodiment, the subset of target action information associated with the target object's parts in the reference action information set, and the subset of current action information associated with the target object's parts in the current action information set, can be input into a part action prediction network to obtain a subset of part action information matching the target object's parts. Then, based on the initially determined reference action information set, further prediction is performed to obtain a subset of part action information associated with the target object's parts, thereby enabling accurate prediction of transition actions related to specific object parts of the target object. Finally, based on the subset of part action information and the reference action information set, a transition action information set is determined, thereby further accurately determining the overall transition actions of the target object after determining the transition actions of specific object parts of the target object.

[0031] In the above embodiments of this application, a two-stage motion information prediction method is used to accurately determine the set of transitional motion information used to indicate transitional motions. More specifically, after initially determining the reference motion information set in the first stage, the information subset related to the target object part in the prediction results of the first stage, and the information subset related to the target object part in the current motion information set are further combined to accurately predict the transitional motion of the target object part. Finally, based on the accurate local transitional motion of the object part, the overall object motion of the target object is determined. This achieves accurate prediction and determination of the transitional object motion from local to global, solving the technical problem of inaccurate object motion determination in existing methods, which leads to poor motion display effects. Attached Figure Description

[0032] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0033] Figure 1 This is a schematic diagram of the hardware environment for an optional method for determining object actions according to an embodiment of the present invention;

[0034] Figure 2 This is a flowchart of an optional method for determining object actions according to an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of an optional method for determining object actions according to an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of another optional method for determining object actions according to an embodiment of the present invention;

[0037] Figure 5 This is a flowchart of another optional method for determining object actions according to an embodiment of the present invention;

[0038] Figure 6 This is a schematic diagram of object node relationships according to an optional method for determining object actions according to an embodiment of the present invention;

[0039] Figure 7 This is a schematic diagram of the structure of an optional object action determination model according to an embodiment of the present invention;

[0040] Figure 8 This is a schematic diagram of the structure of another optional object action determination model according to an embodiment of the present invention;

[0041] Figure 9 This is a schematic diagram of another optional method for determining object actions according to an embodiment of the present invention;

[0042] Figure 10 This is a flowchart of another optional method for determining object actions according to an embodiment of the present invention;

[0043] Figure 11 This is a flowchart of another optional method for determining object actions according to an embodiment of the present invention;

[0044] Figure 12 This is a flowchart of another optional method for determining object actions according to an embodiment of the present invention;

[0045] Figure 13 This is a schematic diagram of the structure of an optional object action determination device according to an embodiment of the present invention;

[0046] Figure 14 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. Detailed Implementation

[0047] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0048] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] It should be noted that in the relevant embodiments of this application, the acquisition and processing of relevant data information of the target account are all obtained in advance through methods that comply with the relevant normative legal documents, and the authorization permission of the corresponding account subject must be obtained before acquiring the above-mentioned relevant information and data.

[0050] According to one aspect of the present invention, a method for determining an object action is provided. This method for determining an object action can be applied, but is not limited to, to... Figure 1 The illustrated hardware environment includes a system for determining object actions, which may include, but is not limited to, a terminal device 102, a network 104, a server 106, and a database 108. The terminal device 102 runs a game client for displaying virtual scenes in game tasks. The terminal device 102 includes a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen in the terminal device 102 is used to display the virtual scene and also provides a human-computer interaction interface to receive human-computer interaction operations for controlling a controlled virtual character in the virtual scene, which will complete game tasks set in the virtual scene. The processor is used to generate interaction instructions in response to the aforementioned human-computer interaction operations and send the interaction instructions to the server. The memory is used to store relevant attribute data, such as object attribute information of the controlled virtual character and attribute information of the virtual items it holds. This attribute information may include, but is not limited to, information used to identify its identity and current location.

[0051] Furthermore, server 106 and database 108 can form a distributed network. Server 106 includes a processing engine used to perform store or read operations on database 108. Specifically, the processing engine reads virtual scene information and operation information for each virtual character from database 108.

[0052] Assumption Figure 1The terminal device 102 runs a client for displaying virtual scenes. The specific process of this embodiment is as follows: S102 is executed by the server 106 to train the transition action sequence prediction model; then S104 is executed to send the transition action sequence prediction model, wherein the transition action sequence prediction model is used to predict the transition action information set.

[0053] Next, the game application is run on terminal device 102, and if a target object in a virtual scene is displayed on the terminal interface, steps S106 to S114 are executed to obtain a current action information set and a target action information set that match the target object. The current action information set is used to indicate the first object action of the target object at a first moment, and the target action information set is used to indicate the second object action of the target object at a second moment, which is later than the first moment. Based on the current action information set and the target action information set, a reference action information set is determined. The reference action information set includes action description information used to determine the transitional object action of the target object. The subset of target action information associated with the target object parts in the reference action information set and the subset of current action information associated with the target object parts in the current action information set are input into the part action prediction network to obtain a subset of part action information that matches the target object parts. Based on the subset of part action information and the reference action information set, a transitional action information set is determined. The transitional action information set is used to indicate the transitional object action of the target object at a third moment, which is the moment between the first and second moments. The object is rendered based on the transitional action information set.

[0054] In another alternative implementation, if the network bandwidth of network 104 meets the requirements, the above steps S106 to S112 can be executed in server 106. After server 106 completes the determination of the transition action information set based on S106 to S112, it can send the transition action information set to terminal device 102 so that the terminal device 102 can perform subsequent virtual scene real-world operations.

[0055] Optionally, in this embodiment, the terminal device 102 can be a terminal device configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client can be a video client, instant messaging client, browser client, educational client, or other client that supports providing shooting game tasks. The network may include, but is not limited to, wired network and wireless network, wherein the wired network includes: local area network, metropolitan area network, and wide area network, and the wireless network includes: Bluetooth, WIFI, and other networks that enable wireless communication. The server may be a single server, a server cluster composed of multiple servers, or a cloud server. The above is only an example, and no limitation is made in this embodiment.

[0056] Optionally, in this embodiment, the operation sequence involved in the above-described method for determining object actions can be applied, but is not limited to, to determining the operation sequence of game object operations in a game-type terminal application (APP) that completes a predetermined interactive game task in a virtual scene. The aforementioned game application can be an open-world game application, in which players can control virtual characters to freely explore, interact, and complete tasks in a vast, open virtual environment. This provides players with a high degree of freedom in their control operations and a non-linear experience within the game world.

[0057] The method for determining object actions described above can also be applied to Multiplayer Online Battle Arena (MOBA) applications. In these MOBAs, players can control virtual characters to unleash character skills in a virtual environment, creating virtual objects within that environment. The interactive game tasks described above can be, but are not limited to, tasks completed by the current player through virtual social interaction between virtual characters in the virtual environment and virtual characters controlled by other players. This method can also be applied to massively multiplayer online role-playing games (MMORPGs). In these games, the current player can complete interactive game tasks from the first-person perspective of a virtual character, such as completing tasks together with other virtual characters. These interactive game tasks can, but are not limited to, run as plugins or mini-programs within an application (such as a non-standalone game app), or run within a game engine within an application (such as a standalone game app). The types of game applications mentioned above may include, but are not limited to, at least one of the following: two-dimensional (2D) game applications, three-dimensional (3D) game applications, virtual reality (VR) game applications, augmented reality (AR) game applications, and mixed reality (MR) game applications. This is merely an example, and this embodiment does not impose any limitations on it.

[0058] In the above-described method for determining object actions, after obtaining the current action information set and the target action information set for determining the transition actions of the target object, a reference action information set is first determined based on the current action information set and the target action information set. It is understood that the reference action information set can be used to initially determine the initial transition actions of the target object. Next, in this embodiment, the subset of target action information associated with the target object's parts in the reference action information set, and the subset of current action information associated with the target object's parts in the current action information set, can be input into a part action prediction network to obtain a subset of part action information matching the target object's parts. Then, based on the initially determined reference action information set, further prediction is performed to obtain a subset of part action information associated with the target object's parts, thereby enabling accurate prediction of transition actions related to specific object parts of the target object. Finally, based on the subset of part action information and the reference action information set, a transition action information set is determined, thereby further accurately determining the overall transition actions of the target object after determining the transition actions of specific object parts of the target object.

[0059] In the above embodiments of this application, a two-stage motion information prediction method is used to accurately determine the set of transitional motion information used to indicate transitional motions. More specifically, after initially determining the reference motion information set in the first stage, the information subset related to the target object part in the prediction results of the first stage, and the information subset related to the target object part in the current motion information set are further combined to accurately predict the transitional motion of the target object part. Finally, based on the accurate local transitional motion of the object part, the overall object motion of the target object is determined. This achieves accurate prediction and determination of the transitional object motion from local to global, solving the technical problem of inaccurate object motion determination in existing methods, which leads to poor motion display effects.

[0060] The above is merely an example, and no limitation is made in this embodiment.

[0061] As an optional implementation method, such as Figure 2 As shown, the method for determining the object's action described above can be applied to a server or terminal, and may specifically include the following steps:

[0062] S202, obtain the current action information set and the target action information set that match the target object, wherein the current action information set is used to indicate the first object action of the target object at the first moment, and the target action information set is used to indicate the second object action of the target object at the second moment, the second moment being later than the first moment;

[0063] S204, determine a reference action information set based on the current action information set and the target action information set, wherein the reference action information set includes action description information for determining the transition object action of the target object;

[0064] S206, input the target action information subset associated with the target object part in the reference action information set and the current action information subset associated with the target object part in the current action information set into the part action prediction network to obtain the part action information subset that matches the target object part;

[0065] S208, determine the transition action information set based on the subset of part action information and the reference action information set, wherein the transition action information set is used to indicate the transition object action of the target object at the third time, the third time being the time between the first time and the second time.

[0066] First, the application scenarios of S202 to S208 in the above embodiments will be explained.

[0067] In one alternative approach, S202 to S208 described above can be applied to a game application to determine and render the transition object actions of virtual game objects in a virtual scene within the game application. Furthermore, the method for determining the transition object actions can be a real-time determination method during the operation of the game application, or it can be used during the development of the game application to determine the transition actions in the various object display animations corresponding to different virtual game objects.

[0068] In another alternative approach, steps S202 to S208 described above can be applied to an animation development application. This application can be used to generate and render virtual animation objects in an animation scene. Having determined the object actions corresponding to two key animation frames of the virtual animation object, the transition object actions rendered and displayed in the transition animation frames between the key animation frames can be determined using the above implementation method.

[0069] In another alternative approach, the above implementation can be applied to a 3D modeling scene. This 3D modeling scene can be a process of generating CG (Computer-Generated Animation) based on multiple sequences of 3D coordinate signals realistically acquired using motion capture technology. Since the frame rate of the multiple 3D coordinate signal sequences acquired by motion capture technology is relatively low, the above implementation can supplement the transitional CG actions between multiple keyframe CG actions.

[0070] The above implementation methods are merely illustrative examples and do not limit the specific application scenarios of steps S202 to S208.

[0071] Furthermore, when the above-described embodiments are applied to a game scenario, steps S202 to S208 can be applied to a game client or a game server. When applied to a game client, the above steps can be implemented by an action generation thread that runs in parallel with the game screen rendering thread in the terminal; when applied to a game server, the terminal can obtain the determination result of the transition action information set of the game server in real time via the network, and perform screen rendering in the terminal based on the obtained transition action information set.

[0072] Furthermore, it should be noted that when the above-described implementation is applied to a game application, the target object in step S202 can be a virtual game object actually displayed in a virtual game scene within the game application. Correspondingly, the first object action and the second object action in step S202 can be two different object actions controlled and executed by the virtual game object in the virtual game scene.

[0073] The relationship between the first object action and the second object action will be further explained below based on the action types of the object actions. It should be noted that the aforementioned first object action and second object action can be two actions performed by a virtual game object during the execution of virtual object operations in a virtual game scene. For example, when a virtual game object triggers an attack operation, the aforementioned first object action and second object action can be two different attack actions during the attack process; when a virtual game object triggers a skill release operation, the aforementioned first object action and second object action can be two different skill actions during the skill release process; when a virtual game object is attacked by other virtual game objects, the aforementioned first object action and second object action can be two different hit actions performed in response to being hit by an attack operation from another virtual game object; when a virtual game object triggers a movement operation in a virtual game scene, the aforementioned first object action and second object action can be two movement actions during the movement process. The above examples are merely illustrative, and in this embodiment, the specific types of the aforementioned first object action and second object action are not limited.

[0074] It's worth noting that conventional games typically pre-define a complete set of animation sequences when designing object actions. For example, the entire process of a virtual character's attack might be fixed as a sequence of multiple attack actions, or the entire process of a virtual character being attacked might be fixed as a sequence of multiple hit actions. This approach means, on the one hand, that the animations corresponding to each action need to be stored separately for multiple virtual characters in the game, consuming significant storage space; on the other hand, because the animation responses of virtual game objects in the virtual game scene are identical every time they perform the same game action, the display effect lacks flexibility and realism.

[0075] In the above embodiments of this application, before the game task is run, the two object actions corresponding to each of the multiple virtual interactive operations of the virtual game object can be determined first. For example, the start action and the end action of the virtual interactive operation can be determined and rendered in real time through the above steps S202 to S208, thereby reducing the storage pressure of game resources.

[0076] On the other hand, since the aforementioned transition actions can be determined and rendered in real time, the transition actions can be further adjusted in real time by taking into account other scene influencing factors in the virtual scene during the determination process. This can improve the flexibility of the transition action determination process and enhance the display effect.

[0077] It should be noted that the current action information set and the target action information set in step S202 can be two types of reference information used to determine the transition action information set. The current action information set can be used to indicate the first object action, and the target action information set can be used to indicate the second object action. Furthermore, the aforementioned first object action can be a starting object action in the above embodiments, or it can be a transition object action.

[0078] The following combination Figure 3 The methods for determining the actions of the three optional transition objects are explained. For example... Figure 3 In this process, given the current action information set used to indicate the starting object action 302 and the target action information set used to indicate the target object action 304, a transition action information set for displaying the transition object action 306 can be determined based on the current action information set and the target action information set. In an optional implementation, the action sequence can be directly determined based on the starting object action 302, the transition object action 306, and the target object action 304. Figure 3(Not shown in the image). It is understood that, in this embodiment, based on repeating the above method, when a current action information set for indicating the starting object action 302 and a target action information set for indicating the target object action 304 are obtained, multiple transition action information sets can be sequentially determined based on the current action information set and the target action information set, and an action sequence can be determined based on the multiple transition action information sets, the current action information set, and the target action set.

[0079] In another alternative approach, after determining the transition action information set for indicating transition object action 306 based on the current action information set and the target action information set, the transition action information set for indicating transition object action 306 can be further determined as the current action information set, and a transition action information set for indicating transition object action 308 can be further determined based on the current action information set and the target action information set for indicating target object action 304. Finally, the starting object action, the target object action, and the multiple transition object actions determined through the above implementation method are determined as action sequence 310. It is understood that in this embodiment, newly determined transition action information sets can be continuously determined as the current action information set based on the above method, and the next transition action information set can be determined.

[0080] The following combination Figure 4 The processing flow of steps S202 to S208 above, and the relationships between the various information sets, are explained. For example... Figure 4 As shown, in step S204, a reference action information set 406 can be determined based on the current action information set 402 and the target action information set 404. This determination method can be a prediction method based on a neural network model. For example, one or more of the following can be used: Long Short-Term Memory (LSTM) network, Gated Recurrent Unit (GRU) network, and Convolutional Neural Network (CNN). The reference action information set is obtained by encoding and decoding the relevant information of the current action information set and the target action information set.

[0081] Understandable Figure 4The reference action information set 406 can be a preliminary prediction result, which may include a set of motion state information related to the target object. Next, in step S206, a second prediction operation can be performed based on two subsets of information related to the target part from the reference action information set 406 and the current action information set 402, to obtain the part action information subset 408. This second prediction operation can also be performed using one or more neural network models, such as Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Convolutional Neural Network (CNN). It is understood that, based on the first prediction operation, the transitional actions of specific parts of the target object can be further refined to obtain refined transitional actions for those specific parts.

[0082] Finally, in step S208, the predicted transitional motion information set 410 can be determined based on the subset 408 of part-body motion information and the reference motion information set 406. The method for determining the predicted transitional motion information set 410 based on the subset 408 of part-body motion information and the reference motion information set 406 can be a set operation between information sets, such as one or more of union or intersection operations. It can also be a weight-based set operation. This embodiment does not limit the method for determining the predicted transitional motion information set 410 described above.

[0083] In the above embodiments of this application, a two-stage motion information prediction method is used to accurately determine the set of transitional motion information used to indicate transitional motions. More specifically, after initially determining the reference motion information set in the first stage, the information subset related to the target object part in the prediction results of the first stage, and the information subset related to the target object part in the current motion information set are further combined to accurately predict the transitional motion of the target object part. Finally, based on the accurate local transitional motion of the object part, the overall object motion of the target object is determined. This achieves accurate prediction and determination of the transitional object motion from local to global, solving the technical problem of inaccurate object motion determination in existing methods, which leads to poor motion display effects.

[0084] The following provides a further explanation of the target object part in step S206 above. The target object part can be an object part that requires special attention during the generation of the transition object action. The type of the object part can have a specific association with the target object action and the current object action, or it can be determined based on other interaction factors of the target object in the virtual scene.

[0085] In one alternative approach, the target body part can be a pre-determined body part. For example, it could be a pre-determined lower body part that is likely to affect the target's movement performance; or it could be a pre-determined upper body part that is likely to affect the target's skill execution performance.

[0086] In another alternative implementation, the target object portion can be determined based on the action type of the first object action and the second object action.

[0087] For example, if the first object action and the second object action are reaction actions of the target object being attacked by other objects, the target object part can be identified as the lower body part.

[0088] For example, in the case where the first object action and the second object action are attack operations initiated by the target object against other objects, the aforementioned target object part can be determined as the upper body part.

[0089] For example, in cases where the first object action and the second object action are object interaction operations (such as handshake operations) initiated by the target object to other objects, the aforementioned target object part can be identified as the hand.

[0090] In the above implementation, the target object part can be determined in real time based on the target object's interaction in the virtual scene, thereby dynamically determining the focus of attention during the generation of transition actions based on the operation type of the object.

[0091] In another alternative implementation, the target object location can be determined based on the interaction position of the target object with interactive elements in the virtual scene.

[0092] For example, when a target is attacked, its location can be determined based on the specific location of the attack. Specifically, if the attacked location is a virtual hand, the target part can be dynamically determined as a virtual hand; if the attacked location is a virtual leg, the target part can be dynamically determined as a virtual hand.

[0093] For example, when a target releases a skill, the target's position can be determined based on the location of the virtual skill's corresponding effect. Specifically, if the target releases a first virtual skill via a virtual hand, the target's body part can be dynamically determined as a virtual hand; if the target releases a second virtual skill via a virtual head, the target's body part can be dynamically determined as a virtual head.

[0094] The above methods for determining the target object parts are merely illustrative examples and do not limit the specific methods for determining the target object parts or the correspondence between the target object parts and interactive elements in the virtual scene.

[0095] like Figure 5 As shown, this paper describes an optional method for dynamically determining the set of transition action information during the operation of a game application.

[0096] S502, Obtain the action types of the first object's action and the second object's action;

[0097] Optionally, the above action types may include response actions when attacked and attack actions.

[0098] Next, as in steps S504-1 and S504-2, the target body part is determined based on the action type. Specifically: if the action type of the first object action and the second object action is a response action when attacked, the lower body part is determined as the target body part; if the action type of the first object action and the second object action is an attack action, the upper body part is determined as the target body part.

[0099] Next, as in steps S506 and S508, a reference action information set is obtained, specifically including: obtaining the current action information set and the target action information set that match the target object; and determining the reference action information set based on the current action information set and the target action information set.

[0100] Next, during the prediction process based on the part-based motion prediction network, prediction can also be performed based on the part-based motion prediction networks corresponding to different object parts. In other words, in this embodiment, separate part-based motion prediction networks can be trained for different object parts to allow for flexible use of the part-based motion prediction networks corresponding to the object parts during game execution.

[0101] As in step S512-1 or S512-2, when the action type of the first object action and the second object action is a response action when attacked, the target action information subset associated with the lower body in the reference action information set and the current action information subset associated with the lower body in the current action information set are input into the upper body action prediction network to obtain the lower body action information subset that matches the target object part.

[0102] When the action types of the first object action and the second object action are attack actions, the target action information subset associated with the upper body in the reference action information set and the current action information subset associated with the upper body in the current action information set are input into the upper body action prediction network to obtain the upper body action information subset that matches the target object part.

[0103] Finally, as in S514, the transition motion information set is determined based on the subset of part motion information and the reference motion information set.

[0104] Through the above-described embodiments of this application, the target object part can be determined in at least two ways: determining the target object part that matches the action type based on the action type of the first object action and the second object action; determining the target object part based on the interaction position of the interactive elements associated with the first object action and the second object action. Thus, the target object part can be dynamically determined based on the interaction results during game operation, thereby dynamically determining the precision of different parts during the determination of transition actions and improving the accuracy of transition action determination.

[0105] The following details the information content that can be included in the current action information set and the target action information set in the above embodiments. In the above embodiments of this application, the current action information set and the target action information set can include state information of multiple object nodes used to drive the rendering of the target object. For example, they can include spatial position information, linear velocity information, and angular velocity information of the object nodes. The object nodes can be hierarchical object nodes included in the three-dimensional driving model used to render and display the target object in virtual space. The three-dimensional driving model can include a root node used to determine driving information according to control instructions, object nodes used to transmit driving information, and a set of skinned mesh objects for determining the spatial displacement of each point on the object surface of the target object according to the driving information. In the process of determining the spatial displacement of each skinned mesh according to the driving information, the spatial displacement of each skinned mesh object can be further determined by combining the skin weight information with the driving information.

[0106] The following combination Figure 6 An optional 3D driving model is described. For example... Figure 6 As shown, the 3D driving model includes at least an object node set 604 and a skinned mesh 606. The object node set 604 is used to transmit and transform driving information based on the connection relationships between object nodes. Multiple root nodes that can directly receive control commands 602 can transmit the control commands 602, combined with weight information, to the object nodes they are connected to. The skinned mesh 606 is used to determine the displacement information of the target skinned mesh rendered on the target object based on the driving information output by node 1. Due to the aforementioned current action information set, the target action information set can include the relevant state information of the various object nodes, and thus, the object actions of the target object in the virtual scene can be displayed based on the aforementioned information set.

[0107] In one specific implementation, the information included in the current action information set and the target action information set may be the displacement information and quaternion rotation information corresponding to each of the multiple object nodes in the object model used to render the target object. Then, subsequent operations are performed based on the aforementioned current action information set and target action information set.

[0108] The following describes the further implementation process of determining the reference action information set based on the current action information set and the target action information set in step S204 above. Step S204 further includes the following steps:

[0109] S1, determine at least one subset of state information based on the current action information set and the target action information set;

[0110] S2, obtain the state encoding results corresponding to at least one subset of state information;

[0111] S3, determine the target fusion coding result based on at least one state coding result;

[0112] S4. Determine the reference action information set based on the target fusion coding results.

[0113] It is understandable that, in this embodiment, after obtaining the current action information set and the target action information set, in order to improve the accuracy of transition action prediction, the node state information included in the current action information set and the target action information set can be classified first, and then at least one subset of state information can be determined.

[0114] Furthermore, for each subset of state information, the corresponding encoder can be obtained to encode the subset of state information, resulting in a state encoding result corresponding to each subset of state information, and then the target fusion encoding result can be determined based on at least one state encoding result.

[0115] In one specific implementation, the above-mentioned at least one subset of state information may be specifically four subsets of state information.

[0116] Among them, the first state information subset of the four state information subsets can be a current node state information subset obtained from the current action information set. Specifically, it can include the current node state information of multiple object nodes, including the linear velocity information corresponding to each of the multiple target object nodes, the velocity information corresponding to each of the multiple root nodes, and the rotation information corresponding to each of the multiple object nodes. The multiple target object nodes are used to drive the target object parts of the target object.

[0117] Furthermore, when the target object part is the lower body, the aforementioned target object node is an object node used to drive the lower body part of the target object; when the target object part is the upper body, the aforementioned target object node is an object node used to drive the upper body part of the target object.

[0118] Taking the lower body as an example, the first subset of state information can further include the linear velocities of multiple lower body nodes in the current frame. The speed of each root skeleton (i.e., a specific root node) 6D rotational velocity information of all bones R i It should be noted that the 6D rotational velocity information includes the three rotational velocities of each bone object around its corresponding three rotation axes (X, Y, Z), as well as the rotational acceleration of its corresponding rotation axes;

[0119] The second subset of state information in the four subsets can be a subset of target action information obtained from the target action information set. Specifically, it can include the normalized position information of multiple lower body nodes in the target frame (corresponding to the target action information set). 6D rotational velocity information of all bones in the target frame R t .

[0120] The third subset of state information in the four subsets can be the node state difference information of multiple object nodes determined according to the current action information set and the target action information set. Specifically, it can include multiple root bone position differences. The difference in 6-dimensional rotational velocity R of all bones t -R i .

[0121] The fourth subset of state information in the four subsets can be obtained from the target action information set, specifically including linear velocity V. t and angular velocity R t+1 -R t .

[0122] In one optional implementation, obtaining the state encoding results corresponding to at least one subset of state information includes at least one of the following:

[0123] Encoding Method 1: When the current state information subset in at least one subset of state information includes the current node state information of multiple object nodes in the current action information set, the encoding result of the current state information subset is determined as the current state code. The state code result includes the current state code. The current node state information of multiple object nodes includes the linear velocity information corresponding to each of the multiple target object nodes, the velocity information corresponding to each of the multiple root nodes, and the rotation information corresponding to each of the multiple object nodes. The multiple target object nodes are used to drive the target object parts of the target object.

[0124] Encoding Method 2: When the current state information subset in at least one subset of state information includes the target node state information of multiple object nodes in the target action information set, the encoding result of the current state information subset is determined as the target state code. The state encoding result includes the target state code, and the target node state information of multiple object nodes includes the target position information corresponding to each of the multiple target object nodes and the target rotation information corresponding to each of the multiple object nodes.

[0125] Encoding Method 3: In at least one subset of state information, the current state information subset includes, based on the node state difference information of multiple object nodes determined according to the current action information set and the target action information set, the encoding result of the current state information subset is determined as the offset state code. The state encoding result includes the offset state code, and the node state difference information of multiple object nodes includes the position difference information corresponding to each of the multiple target object nodes, the position difference information corresponding to each of the multiple root nodes, and the rotation difference information corresponding to each of the multiple object nodes.

[0126] Encoding Method 4: When the current state information subset in at least one subset of state information includes the velocity state information of multiple object nodes in the target action information set, obtain the node-aware convolution result of the current state information subset, and determine the rhythm state code based on the node-aware convolution result. The state code result includes the rhythm state code, and the velocity state information of multiple object nodes includes the linear velocity information of multiple object nodes and the rotation information of multiple object nodes.

[0127] Furthermore, after obtaining the above four subsets of state information, they can be encoded using four different encoding modules. Specifically, they can be encoded as follows: Figure 7 As shown, when the input data 702, which includes the current action information set and the target action information set, is obtained, the four state information subsets obtained through the above implementation method can be input into four different encoders for encoding respectively, to obtain the current state code 704, the target state code 706, the offset code 708, and the future code 710 (i.e., a specific rhythm state code).

[0128] It should be noted that the current state code 704 and the target state code 706 include a large amount of state information about the lower body. Therefore, by focusing on the lower body, the positive impact on leg movements during transition motion prediction can be enhanced, thereby improving the rendering effect of the legs during transition motion rendering. Furthermore, during training, by collecting the aforementioned information about the lower body, leg features can be extracted specifically for training.

[0129] The input to offset encoding 708 is the root bone position difference, 6D rotation difference, and normalized lower body position difference between the target frame and the starting frame. This encoding can be used to measure the distance between the current frame and the target frame.

[0130] The last encoding module is the future frame encoding module, which yields future encoding 710. The input to this future frame encoding module is the linear velocity and 6D velocity of the target frame. The overall generation speed can be controlled by the linear and angular velocities of the target frame. For example, a faster starting speed and a slower ending speed will result in a transition from fast to slow motion, improving the smoothness of the generated transition motion and the rendering effect.

[0131] It should be noted that during the encoding of relevant information in the aforementioned future encoding module, skeleton-aware convolution can be used. The core idea of ​​skeleton-aware convolution is to incorporate skeletal information during the convolution process, enabling the network to learn features related to the skeletal structure. When performing the above skeleton-aware convolution operation based on skeleton-guided convolution, geometric information of the bones, such as their position, orientation, and length, can be incorporated into the convolution operation to guide the sampling and weight allocation of the convolution kernel.

[0132] Introducing skeleton-aware convolutional methods into future coding can more accurately capture the dynamic changes of targets or objects during motion. This approach can better extract the spatial relationships between bones, further extracting the features of nearby bones, making the final generated result closer to the desired outcome.

[0133] It should be noted that the above-mentioned determination of the target fusion coding result based on at least one state coding result includes: obtaining the current position coding result determined based on the position information subset in the current action information set; superimposing the current position coding result and at least one state coding result to obtain the target coding result corresponding to each of the at least one state coding result; and determining the target fusion coding result based on at least one target coding result.

[0134] Continue as Figure 7As shown, in this embodiment of the application, after obtaining the current state code 704, target state code 706, offset code 708, and future code 710, position codes can be superimposed on each of them. Since the position codes of each object node on the target object can characterize the distance between the node object of the current action in the currently generated frame and the node object corresponding to the target action, the generated frames follow a continuous sequence. The superimposed features can be concatenated to determine the target fusion coding result, i.e. Figure 7 The encoded feature 712.

[0135] In one optional implementation, determining the reference action information set based on the target fusion coding result includes: decoding the target fusion coding result using a long short-term memory network to obtain the reference action information set.

[0136] In this embodiment, the superimposed features can be fed into the LSTM module for decoding calculation after concatenation. The LSTM module can memorize continuously input features and enhance short-term memory while appropriately forgetting long-term memory. The output features of the LSTM will be further decoded into the aforementioned reference action information set.

[0137] In one alternative implementation, a subset of target motion information, a subset of reference motion information, and a subset of latent variable information can be determined from the set of reference motion information as associated with the target object part.

[0138] The target object is a virtual object controlled by multiple object nodes with a hierarchical control structure, and the root node is the object node at the first level among the multiple object nodes. The reference motion information subset includes the first predicted rotation information corresponding to each of the multiple reference object nodes and the second predicted rotation information corresponding to each of the multiple root nodes. The multiple reference object nodes are used to drive the reference object parts of the target object, and the reference object parts are the object parts of the target object other than the target object parts.

[0139] The following section explains the further processing methods for the aforementioned subset of target action information, referencing subsets of action information and subsets of latent variable information.

[0140] It should be noted that, in an optional implementation, before inputting the target action information subset and the current action information subset into the gating network of the part-body action prediction network to obtain the weight coefficients corresponding to each of the multiple action decoding modules in the part-body action prediction network, the method further includes:

[0141] S1, the predicted linear velocity information corresponding to each of the multiple root nodes in the reference action information set is determined as a subset of target action information associated with the target object part, wherein the target object is a virtual object controlled by multiple object nodes with a hierarchical control structure, and the root node is the object node at the first level among the multiple object nodes.

[0142] S2, the first current linear velocity information, the first current rotation information, the second current linear velocity information, and the second current rotation information corresponding to the multiple root nodes in the current action information set are determined as the current action information subset associated with the target object part, wherein the multiple target object nodes are used to drive the target object part of the target object.

[0143] Specifically, the aforementioned subset of target motion information may include the linear velocity information of the root bones in the predicted transition motion, and the aforementioned current motion information set may include the linear velocity information and 6D rotational velocity of the lower body nodes, as well as the linear velocity information and 6D rotational information of the root nodes.

[0144] In one optional implementation, the above-mentioned inputting the target motion information subset associated with the target object part in the reference motion information set and the current motion information subset associated with the target object part in the current motion information set into the part motion prediction network to obtain the part motion information subset matching the target object part includes:

[0145] S1, input the target action information subset and the current action information subset into the gating network in the part action prediction network to obtain the weight coefficients corresponding to each of the multiple action decoding modules in the part action prediction network.

[0146] S2, based on the weighting coefficients, input at least one object information from the target action information subset and the current action information subset into at least one action decoding module among multiple action decoding modules;

[0147] S3, determine a subset of body part motion information based on the output results of at least one motion decoding module.

[0148] In this embodiment, the aforementioned part-body motion prediction network may include a gating network and multiple motion decoding modules. Furthermore, the part-body motion prediction network can dynamically select the most suitable motion decoding module for prediction based on the characteristics of the input data through a mechanism called a "gating network." This mechanism ensures that the model can flexibly adjust when processing different inputs, thereby improving overall performance.

[0149] Specifically, as in steps S2 and S3 above, the gating network can first analyze the input target action information subset and the current action information subset; then determine the three most relevant action decoding modules from multiple action decoding modules; and then use the selected action decoding modules to receive the target action information subset and the current action information subset and make predictions in parallel to obtain the above-mentioned part action information subset.

[0150] In one optional implementation, determining a subset of body part motion information based on the output of at least one motion decoding module includes:

[0151] S1, obtain the predicted linear velocity information and the predicted rotation information of each of the multiple target object nodes from the output results of at least one action decoding module.

[0152] S2, the predicted linear velocity information corresponding to each of the multiple target object nodes, and the predicted rotation information corresponding to each of the multiple target object nodes, are determined as a subset of part motion information.

[0153] After determining the subset of body part motion information, it also includes:

[0154] In one optional implementation, determining the transitional motion information set based on the subset of location motion information and the reference motion information set includes: determining the transitional motion information set based on the subset of reference motion information and the subset of location motion information.

[0155] Furthermore, the method of inputting at least one object information from the target action information subset and the current action information subset into at least one action decoding module among multiple action decoding modules according to weighted coefficients further includes:

[0156] S1, when the reference action information set is the decoded information set obtained by decoding according to the long short-term memory network, obtain the latent variable information subset from the reference action information set;

[0157] S2, based on the weighting coefficients, input at least one object information from the subset of latent variable information, the subset of target action information, and the subset of current action information into at least one of the multiple action decoding modules.

[0158] The following combination Figure 8 A specific implementation method will be described. For example... Figure 8 As shown, the encoded feature 802 can be another based on Figure 7The encoding method shown yields encoded feature 712. After the features are stacked, they are concatenated and then fed into an LSTM for computation. This module can memorize continuously input features and enhance short-term memory while appropriately forgetting long-term memory. The output features of the LSTM are further decoded into a reference action information set 804, which includes the linear velocity of the next frame from the root skeleton. 6D rotation speed of the root skeleton in the next frame The hidden variable Z and the 6D rotation speed of the upper body in the next frame Among them, the 6D rotation speed of the next frame And the 6D rotation speed of the root skeleton in the next frame Reference motion information subset 806, linear velocity of the root bone in the next frame. 808 is a specific subset of target action information, and 809 is a specific subset of latent variable information.

[0159] Since the lower body has a significant impact on the visual experience, the decoded latent variable information subset 809 and the linear velocity of the root skeleton in the next frame are used. It will further combine the current motion information subset 810, namely the lower body linear velocity of the current frame. and 6D rotation speed and the linear velocity of the root skeleton in the current frame and 6D rotation speed The part motion prediction network decoding structure is entered, and a subset 812 of part motion information is obtained, which is the linear velocity of the lower body in the next frame. And the 6D rotation speed of the lower body in the next frame And the predicted ground contact signal. After obtaining the incremental velocity information of the upper and lower body (i.e., the part motion information subset 812), it can be combined with the state information of the current frame (i.e., the reference motion information subset 806) to synthesize the transition motion information set 814.

[0160] Through the above-described embodiments of this application, the convolutional method based on skeleton perception enables the model to better learn the velocity features of the target frame, thereby generating rhythm-matched actions. For the sliding problem during the generation of transition actions, a joint decoding approach is adopted, combining upper body general decoding and lower body part-based action prediction network decoding, thus focusing more on the overall performance of the lower body and achieving accurate determination of the transition action information set used to indicate the transition action. More specifically, after initially determining the reference action information set in the first stage, the information subset related to the target object part in the prediction results of the first stage, as well as the information subset related to the target object part in the current action information set, are further combined to accurately predict the transition action of the target object part. Finally, based on the accurate local transition action of the object part, the overall object action of the target object is determined, achieving accurate prediction and determination of the transition object action from local to overall, solving the technical problem of inaccurate object action determination in existing methods, which leads to poor action display effects.

[0161] In an optional implementation, after determining the transitional motion information set based on the subset of part motion information and the reference motion information set, the method further includes:

[0162] S1, if the transition action information set satisfies the action sequence prediction condition, the transition action information set is determined as the updated current action information set, and the following steps are executed repeatedly until the target transition information set sequence is obtained. The target transition information set sequence is used to indicate the transition object action sequence of the target object, and the transition object action sequence is used to connect the initial object action and the second object action of the target object. The target transition information set sequence includes the transition action information set:

[0163] S2, based on the updated current action information set and target action information set, determine the next transition action information set included in the target transition information set sequence;

[0164] S3, if the next transition action information set meets the action sequence prediction conditions, the next transition action information set is determined as the updated current action information set.

[0165] It is understood that in this embodiment, the predicted transition action information set can be determined as the updated current action information set by combining the sequence prediction method of LSTM network, thereby realizing the prediction of the target transition information set sequence.

[0166] In the above embodiments, the transitional action information set satisfying the action sequence prediction condition may include, but is not limited to, the frame number corresponding to the transitional action information set being less than or equal to the target number, indicating that the prediction of the transitional action information set has not yet been completed, and therefore the above steps need to be continued to complete the prediction operation of subsequent transitional actions. The action sequence prediction condition may also be that the information difference between the transitional action information set and the target transitional information set is greater than or equal to a target threshold, thereby indicating that the prediction of the transitional action information set has not yet been completed through the information difference.

[0167] For example Figure 9 As shown in the above embodiments of this application, an autoregressive model can be used to predict the transition action sequence in the game. Specifically, in the embodiments of this application, the autoregressive model can generate transition frame actions according to the required number of frames, start frame, and target frame information. Further, when the initial action and end action of the target object are obtained, the action information sets corresponding to the initial action and end action are input into a multi-feature encoder; then, through the decoding and prediction operations of the LSTM network, general decoding information is obtained. Then, the information subset related to the lower body in the general decoding information is decoded using a specific part decoder to obtain the transition action of the lower body; then, it is combined with other remaining decoding information in the general decoding information to obtain the transition action.

[0168] Furthermore, in Figure 9 In the autoregressive model shown, the newly generated transition action is used as a new starting frame to guide the generation of the next transition action, and finally the action sequence to be rendered is obtained.

[0169] In an optional implementation, after determining the next transition action information set included in the target transition information set sequence based on the updated current action information set and the target action information set, the method further includes:

[0170] S1, add the transition action information set included in the target transition information set sequence to the rendering cache queue in sequence, wherein the transition action information in the rendering cache queue is used to render the transition object action of the target object;

[0171] S2, retrieve multiple first action information sets from the rendering cache queue that have not yet been used to render the transition object action of the target object, and retrieve the compensation information sets corresponding to each of the multiple first action information sets, and adjust the multiple first action information sets according to the multiple compensation information sets to obtain multiple second action information sets.

[0172] S3, replace multiple first action information sets in the rendering cache queue with multiple second action information sets.

[0173] It is understood that, in the embodiments of this application, when the above implementation method is applied to a game client and executed asynchronously with the main thread of the game client, the generated transition action information set sequence can be further optimized through the above steps.

[0174] In the above embodiments, the generation of transition frames mainly relies on the transition frames between the current frame prediction and the target frame prediction. While this may ensure a smooth transition between the generated frame and the previous frame, it cannot guarantee the smooth transition between the generated frame and the target frame. Although the final generated action is only slightly different from the target frame, a "frame skipping" phenomenon may still occur visually. Based on this, the embodiments of this application further optimize the above-mentioned transition action information set sequence.

[0175] In one optional implementation, the acquisition of compensation information sets corresponding to each of the plurality of first action information sets includes:

[0176] S1, determine the first action information set sequence by identifying multiple first action information sets and target action information sets;

[0177] S2, determine the first state difference information set based on the first first action information set and the target action information set in the first action information set sequence, wherein the first state difference information set includes the first node state differences corresponding to each of the multiple object nodes;

[0178] S3, based on every two adjacent first action information sets in the first action information set sequence, determine the second state difference information set corresponding to each of the two adjacent first action information sets, wherein the second state difference information set includes the second node state difference corresponding to each of the multiple node objects.

[0179] S4. Based on the first state difference information set and the second state difference information set, determine the compensation information set corresponding to each of the multiple first action information sets.

[0180] Specifically, in this embodiment, a smooth transition algorithm based on the rate of action change is used to post-process the generated frames. Since the model inference speed is much faster than the game animation frame tick cycle, after the transition action sequence is generated, the game screen continues to retrieve and play the transition action sequence sequentially. Before the last frame is retrieved from the cache for playback, the remaining unrendered actions can be processed after the entire transition action inference is completed.

[0181] The specific processing method is as follows: First, retrieve the current unrendered frame model inference result from the cache; then, calculate the change in world position between every two frames of the unrendered frame inference result and store it in the array ArrDiff;

[0182] In the steps described above, the change in the object's position in the world coordinate system between each two adjacent frames can be obtained from the generated transition sequence. These changes are stored in the array ArrDiff for subsequent calculations.

[0183] Simultaneously calculate the total change and divide the change value of each frame by the total value to calculate the change rate of each frame:

[0184] Where T is the remaining frame length

[0185] In the above calculation process, firstly, the sum of the position changes across all frames is calculated. Then, the position change of each frame is divided by this sum to obtain the rate of change for each frame. This rate of change represents the importance or influence of each frame in the overall animation, and subsequent offsets can be adjusted based on the calculated weight information.

[0186] Next, the difference ΔP between the target frame's rotation and position and the predicted frame is calculated. T With ΔR T ;

[0187] Finally, the position and rotation increments from each frame are calculated and compensated for into the inference prediction frame based on the rate of change ArrDiffRate and the last frame error value:

[0188] ΔP(t) = Ratio(t) * ΔP T ΔR(t) = Ratio(t) * ΔR T

[0189] P′(t)=P(t)+ΔP(t), R′(t)=R(t)+ΔR(t)

[0190] Where ΔP(t) is the positional difference, ΔR T For rotational differences, the increments in position and rotation for each frame are calculated using the rate of change (ArrDiffRate) per frame and the error value of the last frame. These increments are used to adjust the predicted frames to make them closer to the target frame.

[0191] Finally, the calculated position and rotation increments are applied to the predicted frame to obtain the adjusted position P′(t) and rotation R′(t). In this way, the predicted frame is compensated to a state that is closer to the target frame.

[0192] By implementing the above methods, when the model inference speed of the animation system is faster than the animation frame update speed, smoother and more accurate animation effects can be achieved by adjusting the position and rotation of each frame. This ensures the fluidity and naturalness of the animation, maintaining high-quality animation performance even under high-speed model inference.

[0193] In an optional implementation, after determining the next transition action information set included in the target transition information set sequence based on the updated current action information set and the target action information set, the method further includes:

[0194] S1, obtain playback indication information that matches the target transition information set sequence, wherein the playback indication information is used to indicate the playback speed of the transition object action between the initial object action and the second object action;

[0195] S2, determine at least one set of transition action information that matches the current moment based on the playback instruction information;

[0196] S3, determine the current transition action information set that matches the current moment based on the weighted result between at least one transition action information set, wherein the current transition action information set is used to render the transition object action of the target object at the current moment.

[0197] In this embodiment, when the model inference speed is much faster than the game animation frame tick cycle, a rhythm adjustment curve is further provided. Since model inference is much faster than the animation frame tick (typically, one frame tick can complete the inference of an entire animation segment), a rhythm adjustment curve is added to adjust the playback rate of the generated transition actions, thereby allowing for free adjustment of the playback rhythm of the transition actions. The current transition frame number is calculated as follows:

[0198] Where T is the length of the transition frame.

[0199] In the above formula, A i This could be the motion amplitude of the i-th frame determined by the aforementioned rhythm adjustment curve, or the playback speed or weight of the transition motion. Using the above formula, the proportion of the weighted cumulative sum of motion amplitudes to the total cumulative sum within time t can be determined, and then multiplied by the total number of frames T to obtain a normalized time index.

[0200] Since the inference sequence number takes values ​​in an integer space, the final rendered frame is calculated as follows:

[0201]

[0202] In the above formula, since the frame number needs to be an integer during actual rendering, the continuous transition frame number Index(t) needs to be converted into a discrete frame number. Further, through... The relative position of time t between two integer frames was determined. The integer part of t is used to indicate the current frame; It is the next integer after t, used to indicate the next frame.

[0203] The above formula This represents position interpolation between two keyframes, where Pi t It is the position of the object at time t. and These represent the positions in the current frame and the next frame, respectively. The precise position of the object at time t is calculated using linear interpolation based on the weight α. For rotation, a... Among them, R t This shows the rotation interpolation between two keyframes. It is the rotation at time t. This provides rotation information for the next frame. The Slerp() function specifies a method for interpolating between quaternions to obtain a smooth interpolation result of the rotation values.

[0204] Through the above-described embodiments of this application, after the model infers the transition action sequence, the playback and rendering process of the transition action sequence can be maintained naturally and smoothly at different playback rates, even when the model inference speed is much faster than the animation frame update speed. By adjusting the rhythm curve, the playback speed of the animation can be controlled, making the actions more in line with the expected rhythm.

[0205] In an optional implementation, before determining the transition action information set as the updated current action information set and repeatedly executing the following steps until the target transition information set sequence is obtained, the method further includes: acquiring a first action information set and a second action information set that match the target object, wherein the first action information set is used to indicate the initial object action of the target object at the initial time, and the second action information set is used to indicate the target object action of the target object at the target time; adjusting the weights according to the model matching the target object, and adjusting the first action information set and the second action information set respectively to obtain the initial action information set and the target action information set, wherein the initial action information set and the target action information set are used to determine the target transition information set sequence;

[0206] In another optional implementation, after determining the transition action information set as the updated current action information set and repeatedly executing the following steps until the target transition information set sequence is obtained, the method further includes: adjusting the weights according to the model matching the target object, restoring the multiple transition action information sets in the target transition information set sequence to obtain multiple target transition action information sets, wherein the multiple target transition action information sets are used to render the transition object actions of the target object in the virtual scene.

[0207] Understandably, since different virtual objects in the game have different shapes and sizes in the virtual scene, it is necessary to redirect the character skeleton to the model skeleton before determining the transition action using the above implementation method. After obtaining the calculation results, the model skeleton needs to be redirected back to the character skeleton.

[0208] Specifically, the height ratio of the initial model T-Pose (upright shape) and the target model T-Pose can be calculated:

[0209]

[0210] Furthermore, before determining the transition actions, the character skeleton needs to be redirected to the model skeleton, which can be done through P... target =P current *s is used for implementation; furthermore, after obtaining the transition action information, before performing the rendering operation, R can also be restored in the following way. current =R target / s.

[0211] Through the above-described embodiments of this application, a target transition action sequence prediction model can be obtained by training a standard object model once, thereby enabling the deduction and prediction of transition action sequences for virtual objects of different body types and categories in the game, and improving the versatility of the object action determination method in the above-described embodiments of this application.

[0212] The following combination Figure 10 The application process of the above-described embodiments of this application will be fully described. Figure 10 When the implementation shown is running on a game client, S1001 can be executed first to trigger an action transition event; then, target frame sampling is performed via S1003; model skeleton repositioning is performed via S1005; and target frame information is determined via S1007. The combination of these three steps determines the target frame information. It can be understood that the above step S1005 can configure the reorientation operation weights according to the model's height ratio.

[0213] On the other hand, S1002, current frame sampling component is executed; S1004, motion data preprocessing is performed; S1006, initial frame is buffered.

[0214] Thus, given the initial frame action information and the target frame action information, steps S1008 to S1016 are executed: thread inference (autoregressive generation); current frame information is updated; inference result sequence is cached; it is determined whether sequence inference is complete; if inference is complete, adjustment blending is executed to update the inference result; if inference is not complete, autoregressive generation of transition action information is returned; finally, the inference result sequence is obtained.

[0215] After obtaining the inference result sequence and storing it in the cache, during the rendering process, steps S1020, S1022, S1024, S1026, and S1028 are executed. The animation system ticks; fractional frames are generated based on the rhythm curve; fractional frames are retrieved from the inference result cache; character skeletons are repositioned; and the animation is rendered.

[0216] exist Figure 10 In the illustrated implementation, an ONNX middleware can be used within the game to expose a C++ interface for in-game calls. Within the game, a start-frame buffer component is first created to continuously cache motion information from the current and past frames. During calculations, these two frames are needed to calculate the velocity of the start frame. When the motion transition signal is invoked, the target frame is obtained from the target animation, specifically through a target sampling component. This component acquires information from two frames to calculate the linear velocity and 6D rotation velocity of the target frame. The outputs of the two components mentioned above correspond to... Figure 7 The model inputs the required information.

[0217] Since the model is an autoregressive model, all actions to be rendered can be calculated frame-by-frame by the model. Model calculation can be performed asynchronously with other programs on the main thread, as long as it is confirmed that the calculation of the current transition action frame can be completed before the current frame is rendered. During testing, the above-described implementation of this application can satisfy the requirement that the transition frame calculation speed is much faster than the rendering speed on both PC and mobile devices. Taking PC as an example, the model takes an average of 0.4ms to calculate one frame, while the game takes 16.67ms per frame at 60 frames per second. It can be seen that the model can complete the calculation one frame before the game tick and the calculated action output can be assigned to the character to complete the rendering. In addition, this implementation further considers the need to ensure a smooth transition between the generated last frame and the target frame, that is, post-processing is performed based on the adjustment blending algorithm.

[0218] The following is Figure 7 , Figure 8 The training process of a transition action sequence prediction model is explained.

[0219] In an optional implementation, before obtaining the current action information set and the target action information set matching the target object, the method further includes:

[0220] S1, obtain the initial sample action information set and the target sample action information set that match the sample object. The initial sample action information set is used to indicate the initial object action of the sample object at the initial time, and the target action information set is used to indicate the target object action of the sample object at the target time, which is later than the initial time.

[0221] S2, input the current action information set and the target action information set into the transition action sequence prediction model to obtain the sample transition action information set sequence. The transition action sequence prediction model includes a long short-term memory network and a part action prediction network. The long short-term memory network is used to determine the reference sample action information set based on the initial sample action information set and the target sample action information set. The part action prediction network is used to determine the sample part action information subset that matches the target object part based on the target sample action information subset associated with the target object part in the reference sample action information set and the initial action information subset associated with the target object part in the initial sample action information set. The sample part action information subset and the reference sample action information set are used to determine the sample transition action information set in the sample transition action information set sequence.

[0222] S3, determine the current model training loss based on the label transition action information set sequence that matches the initial sample action information set and the target sample action information set, and the sample transition action information set sequence;

[0223] S4. Adjust at least one model parameter in the transition action sequence prediction model based on the current model training loss.

[0224] Optionally, the above-mentioned inputting the current action information set and the target action information set into the transition action sequence prediction model to obtain a sample transition action information set sequence includes:

[0225] S1, input the initial sample action information set and the target sample action information set into the long short-term memory network in the transition action sequence prediction model to obtain the reference sample action information set, wherein the reference sample action information set includes action description information for determining the transition object action of the sample object;

[0226] S3, input the subset of target sample action information associated with the target object part of the reference sample action information set and the subset of current sample action information associated with the target object part of the initial sample action information set into the part action prediction network to obtain the subset of part sample action information that matches the target object part.

[0227] S3. Based on the subset of action information of the part sample and the set of action information of the reference sample, determine the set of action information of the transition sample. The set of action information of the transition sample is used to indicate the action of the target object at the third time. The third time is the time between the first time and the second time.

[0228] Optionally, the above-mentioned subset of target sample action information associated with the target object part of the reference sample action information set and the current subset of sample action information associated with the target object part of the initial sample action information set are input into the part action prediction network to obtain a subset of part sample action information matching the target object part, including:

[0229] S1, input the target sample action information subset and the current sample action information subset into the gating network in the part action prediction network to obtain the sample weight coefficients corresponding to each of the multiple action decoding modules in the part action prediction network.

[0230] S2, based on the sample weight coefficient, input at least one object information from the target sample action information subset and the current sample action information subset into at least one action decoding module among multiple action decoding modules;

[0231] S3, determine a subset of motion information for the part sample based on the output of at least one motion decoding module.

[0232] In one specific implementation, the aforementioned current motion information set and target motion information set can be information sets determined based on a BVH (Biovision Hierarchy Data Format) file. In the BVH file, motion data is stored based on a tree structure of virtual object joints. The file records the offsets of human skeleton nodes (i.e., a specific object node) and their parent nodes, and the motion information consists of Euler angle rotations in the local parent coordinate system and displacements of the root skeleton (i.e., a specific root node).

[0233] Before performing the above training steps, such as Figure 11 As shown, the information included in the acquired BVH file can be preprocessed first. Specifically, as in step S1102, the forward kinematics method is used to convert the displacement information and local rotation information of each object node included in the BVH file into displacement information and quaternion rotation information in the world coordinate system.

[0234] Furthermore, to make the final data suitable for batch training of neural network models, the action sequence can be uniformly divided into translation and quaternion tensors of length 52 frames, where the first 10 frames are buffer frames, and the 10th frame of the tensor data is used as the normalization reference frame (this frame is also the starting action used during inference). The root bone position is set to zero, and the quaternion rotation is set to the +x orientation, and the changing translation and rotation are recorded.

[0235] Next, S1104-1 and S1104-2 are executed. Data from other frames is adjusted accordingly based on the recorded normalized displacement and rotation. Finally, the quaternion rotations of all data are converted to 6D rotations. In addition, the positional information of the Foot (object nodes included in the foot) and Ankle (object nodes included in the ankle) in the world coordinate system can also be used to calculate whether the foot joint is in contact with the ground. Specifically, the square of the Euclidean distance (in cm) between the foot and the object nodes in the preceding and following frames is calculated to be less than a threshold of 0.02, and the square of the Euclidean distance (in cm) between the ankle and the object nodes in the preceding and following frames is calculated to be less than a threshold of 0.02. If both conditions are met, it can be determined whether the foot is on the ground. It should be noted that this criterion can be used to constrain phenomena such as sliding during subsequent training.

[0236] After determining the samples as described above, the above sample set can be used to... Figure 7 and Figure 8 The model shown is trained. Specifically, the model trained in this embodiment is an autoregressive model, which generates transition frame actions based on the number of frames generated, the start frame, and the target frame information. The newly generated transition frame then serves as the new start frame to guide the generation of actions in the next frame; this process is autoregressive. The multi-feature encoding part of the model is... Figure 7 As shown, this section consists of four sub-encoders. The current state encoder takes as input the linear velocity of the lower body skeleton in the starting frame, the root bone velocity, and the 6D rotation of all bones. The target frame encoder takes as input the 6D rotation of all bones in the target frame and the normalized lower body position. The current state and target encodings focus on the lower body because leg movements have the greatest impact on visual perception; therefore, leg features can be extracted specifically for training. The offset encoder takes as input the root bone position difference, 6D rotation difference, and normalized lower body position difference between the target and starting frames. This encoding is used to measure the distance between the current and target frames. The final encoding module is the future frame encoder. This encoder takes as input the linear velocity and 6D velocity of the target frame. This module primarily controls the overall generation rhythm using the linear and angular velocities of the target frame. For example, a faster starting speed and a slower ending speed will result in a transition from fast to slow motion. The aforementioned future frame encoding structure allows for control over the rhythm of this transitional motion generation.

[0237] In the specific implementation of the future encoding, the Skeleton-Aware Convolution method was used. This method can better extract the spatial relationships of bones and further extract the features of neighboring bones, so that the final generated effect is closer to the desired result.

[0238] The input features, encoded by four encoders, are superimposed with positional codes. These codes represent the distance of the currently generated frame from the target, ensuring a sequential order of generated frames. The superimposed features are then concatenated and fed into an LSTM for computation. This module can memorize continuously input features and enhance short-term memory while appropriately forgetting long-term memory. The LSTM output features are further decoded into the linear velocity and 6D rotational velocity of the next frame for the root skeleton, the latent variable z, and the 6D rotational velocity of the upper body in the next frame. Since the lower body significantly impacts visual perception, the decoded latent variable z and the linear velocity of the root skeleton in the next frame are further combined with the current frame's lower body linear velocity and 6D rotational velocity to enter the body part motion prediction network, obtaining the lower body's linear velocity and 6D rotational velocity in the next frame, as well as the predicted ground contact signal. After obtaining the incremental velocity information predicted for the upper and lower bodies, a new frame can be synthesized by combining it with the current frame's state information, and a new transition frame is generated iteratively through autoregression until it connects with the target frame. The decoding process described above is as follows: Figure 8 As shown.

[0239] The loss function involved in the above model training process is further explained below. In an optional implementation, the current model training loss is determined based on the sequence of label transition action information sets matching the initial sample action information set and the target sample action information set, and the sequence of sample transition action information sets, including at least one of the following:

[0240] The first loss determination method is to determine the rotation information loss based on the label transition action information set sequence and the sample transition action information set sequence. The rotation information loss is used to indicate the first information difference between the label rotation information corresponding to each object node included in the label transition action information set sequence and the sample rotation information corresponding to each object node included in the sample transition action information set sequence.

[0241] Specifically, the aforementioned rotational information loss can be determined in the following way:

[0242]

[0243] The second loss determination method is to determine the root node position loss based on the label transition action information set sequence and the sample transition action information set sequence. The root node position loss is used to indicate the second information difference between the label position information corresponding to each root node in the label transition action information set sequence and the sample position information corresponding to each root node in the sample transition action information set sequence.

[0244] Specifically, the root node position loss can be determined as follows:

[0245]

[0246] The third loss determination method is to determine the target object part information loss based on the label transition action information set sequence and the sample transition action information set sequence. The target object part information loss is used to indicate the third information difference between the label position information corresponding to each target object node included in the label transition action information set sequence and the sample position information corresponding to each target object node included in the sample transition action information set sequence. The target object node is used to drive the target object part of the target object.

[0247] Specifically, the loss of information about the target object's location can be determined in the following ways:

[0248]

[0249] The fourth loss determination method is to determine the slippage information loss based on the label transition action information set sequence, wherein the slippage information loss is used to indicate the degree of slippage of the target object indicated by the label transition action information set sequence on the virtual ground;

[0250] Specifically, the loss of the aforementioned sliding information can be determined in the following way:

[0251]

[0252] In the above loss function L footskating In the middle, when the foot contacts the ground (c i =0), loss function L footskating =0, because there should be no slippage at this time; while when the foot is not in contact with the ground (c i =1), the loss function will be based on The value of L is used for calculation, therefore the above loss function reflects the degree of foot slippage. By analyzing L... footskating By imposing constraints, the sliding phenomenon can be implicitly constrained by penalizing the speed at the moment of contact with the ground, thus improving the display effect of the generated transition animation.

[0253] The fifth loss determination method is to determine the ground contact information loss based on the label transition action information set sequence and the sample transition action information set sequence. The ground contact information loss is used to indicate the fourth information difference between the label ground contact information indicated by the label transition action information set sequence and the sample ground contact information indicated by the sample transition action information set sequence. The sample ground contact information is used to indicate the contact state between the virtual foot of the target object and the virtual ground.

[0254] Specifically, the loss of the above-mentioned ground contact information can be determined in the following ways:

[0255]

[0256] The above loss function Lcontact It is a cross-entropy loss function used for binary classification problems, which can be used to evaluate whether the foot of a target object has made contact with the ground.

[0257] The sixth loss determination method is to determine the object location information loss based on the label transition action information set sequence and the sample transition action information set sequence. The object location information loss is used to indicate the fifth information difference between the label location information corresponding to each of the multiple object nodes included in the label transition action information set sequence and the sample location information corresponding to each of the multiple object nodes included in the sample transition action information set sequence.

[0258] Specifically, the loss of the above-mentioned object location information can be determined in the following way:

[0259]

[0260] The seventh loss determination method is to determine the object velocity information loss based on the label transition action information set sequence and the sample transition action information set sequence. The object velocity information loss is used to indicate the sixth information difference between the label velocity information corresponding to each of the multiple object nodes included in the label transition action information set sequence and the sample velocity information corresponding to each of the multiple object nodes included in the sample transition action information set sequence.

[0261] Specifically, the loss of the above-mentioned object velocity information can be determined in the following way:

[0262]

[0263] In a preferred approach, the model can be trained using all seven loss functions mentioned above simultaneously, namely:

[0264] L = L rot +L root_pos +L lower_pos +L footskating +L contact +L fkpos +L global_vel

[0265] By using the loss function described above in the embodiments of this application to train the model, on the one hand, L footskating Sliding can be implicitly constrained by penalizing the speed at ground contact, while explicit sliding feature prediction can further reduce sliding. Since the action in this embodiment is calculated by predicting the position in the fk world coordinate space, there will be no inconsistency between the bone length and the character's height during training using the same target object. Meanwhile, the lower body position loss function L... lower_posIt can help the model converge to the appropriate rotation value prediction more quickly. Finally, the addition of global velocity loss can make the performance between frames of the generated transition action more continuous, reduce action abrupt changes, and improve the generation effect of transition actions.

[0266] In one alternative implementation, during model training, the Adam optimizer can be used with an initial learning rate of 5e-4 maintained for 300 epochs. The learning rate then linearly decreases to 5e-5 between 300 and 900 epochs, and further decreases to 1e-5 between 900 and 1600 epochs, maintaining this rate until training ends. By dynamically configuring the learning rate parameter during training, a higher learning rate can be used early on to prevent the model from getting stuck in local optima, while a smaller learning rate can be adjusted later to further reduce model loss, thereby improving the training performance.

[0267] In addition, to enable the model to generalize and generate transition actions of multiple lengths, positional encoding can be superimposed on features during training to help the model remember the current transition frame number. Furthermore, a scheme for randomly generating frames can be adopted during training. First, this scheme randomly generates 5 to N frames with a uniform distribution, where N is the maximum allowed number of frames, which gradually increases from 10 to 40 as the training rounds progress. Then, by dynamically adjusting the maximum number of frames for predicting transition actions according to the training phase, good results can be quickly achieved in terms of segment length generation in the early stages. Subsequently, gradually increasing the longest sequence length allows the model to gradually adapt to generating transition segments of different lengths, thereby further improving the model's training performance.

[0268] In an optional implementation, after adjusting at least one model parameter in the transition action sequence prediction model based on the current model training loss, the method further includes:

[0269] S1, obtain the set of multiple weight coefficients included in the completed training of the transition action sequence prediction model;

[0270] S2, Based on the amount of data corresponding to each of the multiple weight coefficient sets, determine at least one reference coefficient set from the multiple weight coefficient sets;

[0271] S3, quantize the weight coefficients in at least one set of reference coefficients to obtain at least one set of target coefficients after processing;

[0272] S4. Update the transition action sequence prediction model based on at least one set of target coefficients to obtain the target transition action sequence prediction model;

[0273] S5, send the target transition action sequence prediction model to the target terminal, wherein the target transition action sequence prediction model running on the target terminal is used to determine the transition action information set of the target object in the game task during the game task execution.

[0274] In the above embodiments of this application, in order to enable the model to be smoothly deployed on the device and run in real time, it is statistically determined that the weights with the largest proportion in the model are the weights in the two GEMM (General Matrix Multiplication) operators of the LSTM and the weights of the subsequent linear layers. Based on this, the weights stored in the two subsequent fully connected layers of the LSTM set can be quantized after training, so that the precision of the weight parameters in the two subsequent fully connected layers of the LSTM set is adjusted from float32 to int8. Through the above quantization process, the size of the trained model can be reduced from 15.72MB to 6.17MB, thereby making it easier to deploy to the terminal and run smoothly synchronously during game operation.

[0275] Through the above-described embodiments of this application, by quantizing the parameter weights of some fully connected layers of the model, in scenarios where the model is deployed to the terminal for operation, not only is the memory space occupied reduced, but the calculation speed is also greatly accelerated, thereby improving the efficiency of object action generation.

[0276] The following combination Figure 12 A complete implementation of this application will be described.

[0277] First, in the data processing stage, including step S1202, motion data preprocessing. This stage primarily involves processing the data in the original motion data BVH file into a format suitable for model training and converting it to the world coordinate system for unified processing. It should be noted that during data processing, the positional information of the foot and ankle is also used to calculate whether the foot joint is in contact with the ground, for subsequent calculations of slip loss, thereby constraining the slip phenomenon in the model's predicted transitional movements.

[0278] The model training phase, including S1204 and S1206, mainly comprises two stages: action data preprocessing, model training, and model compression. During model training, L... footskating The penalty for the speed at ground contact implicitly constrains the sliding, while explicit sliding feature prediction can further reduce sliding. Since the action in this embodiment is calculated by predicting the position in the fk world coordinate space, there will be no inconsistency between the bone length and the character's height during training using the same target object. Meanwhile, the lower body position loss function L... lower_posIt can help the model converge to the appropriate rotation value prediction more quickly. Finally, the addition of global velocity loss can make the performance between frames of the generated transition action more continuous, reduce action abrupt changes, and improve the generation effect of transition actions.

[0279] Furthermore, by compressing the model, when the model is deployed to the terminal for execution, not only is the memory space occupied reduced, but the calculation speed is also greatly accelerated, thus improving the efficiency of object action generation.

[0280] Deployment and post-processing mainly include S1208 to S1218, loading model weights;

[0281] During game execution, determine whether action generation is required. If action generation is required, call the model to generate actions frame by frame and write the generated actions to the generation sequence cache; then determine whether the sequence action generation is complete; if generation is complete, perform post-processing on the remaining unrendered action frames.

[0282] In the above implementation, the convolutional method based on skeleton perception enables the model to learn the speed features of the target frame better, thereby generating rhythm-matching actions. For the sliding problem, a general decoding method for the upper body and a specific model decoding method for the lower body are adopted to pay more attention to the overall performance of the lower body. In addition, the post-processing of frame connection and overall rhythm further optimizes the overall performance of the generated transition actions, improving the generation effect of transition actions in the game.

[0283] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0284] According to another aspect of the present invention, an apparatus for determining an object action for implementing the above-described method for determining an object action is also provided. For example... Figure 13 As shown, the device includes:

[0285] The acquisition unit 1302 is used to acquire a current action information set and a target action information set that match the target object. The current action information set is used to indicate the first object action of the target object at a first moment, and the target action information set is used to indicate the second object action of the target object at a second moment, wherein the second moment is later than the first moment.

[0286] The first determining unit 1304 is used to determine a reference action information set based on the current action information set and the target action information set, wherein the reference action information set includes action description information for determining the transition object action of the target object.

[0287] The prediction unit 1306 inputs the target action information subset associated with the target object part in the above reference action information set and the current action information subset associated with the target object part in the above current action information set into the part action prediction network to obtain the part action information subset that matches the target object part.

[0288] The second determining unit 1308 is used to determine a transitional action information set based on the aforementioned subset of part action information and the aforementioned reference action information set, wherein the aforementioned transitional action information set is used to indicate the transitional object action of the aforementioned target object at a third time, and the aforementioned third time is the time between the aforementioned first time and the aforementioned second time.

[0289] Optionally, the prediction unit 1306 includes: a gating module, used to input the target action information subset and the current action information subset into the gating network of the part action prediction network to obtain weight coefficients corresponding to each of the plurality of action decoding modules in the part action prediction network; a decoding module, used to input at least one object information from the target action information subset and the current action information subset into at least one of the plurality of action decoding modules according to the weight coefficients; and a determining module, used to determine the part action information subset according to the output results of each of the at least one of the action decoding modules.

[0290] Optionally, the gating unit is further configured to: determine the predicted linear velocity information corresponding to each of the multiple root nodes in the reference motion information set as a subset of the target motion information associated with the target object part, wherein the target object is a virtual object controlled by multiple object nodes with a hierarchical control structure, and the root node is an object node at the first level among the multiple object nodes; determine the first current linear velocity information, the first current rotation information, the second current linear velocity information, and the second current rotation information corresponding to each of the multiple root nodes in the current motion information set as a subset of the current motion information associated with the target object part, wherein the multiple target object nodes are used to drive the target object part of the target object.

[0291] Optionally, the determining module is configured to: obtain the predicted linear velocity information and the predicted rotation information corresponding to each of the multiple target object nodes from the output results of at least one of the above-mentioned action decoding modules; and determine the predicted linear velocity information and the predicted rotation information corresponding to each of the multiple target object nodes as a subset of the above-mentioned part action information.

[0292] Optionally, the second determining unit 1308 includes: an acquisition module, configured to acquire a subset of reference action information from the reference action information set, wherein the subset of reference action information includes first predicted rotation information corresponding to each of a plurality of reference object nodes and second predicted rotation information corresponding to each of a plurality of root nodes, wherein the plurality of reference object nodes are used to drive reference object parts of the target object, and the reference object parts are object parts of the target object other than the target object parts; and a second determining module, configured to determine the transition action information set based on the subset of reference action information and the subset of part action information.

[0293] Optionally, the prediction unit 1306 is further configured to: when the reference action information set is a decoded information set obtained by decoding according to the long short-term memory network, obtain a subset of latent variable information from the reference action information set; and input at least one object information from the latent variable information subset, the target action information subset, and the current action information subset into at least one of the multiple action decoding modules according to the weight coefficients.

[0294] Optionally, the above-mentioned object action determination device further includes: a transition action sequence determination unit, configured to determine the transition action information set as the updated current action information set when the transition action information set satisfies the action sequence prediction condition, and to repeatedly execute the following steps until a target transition information set sequence is obtained, wherein the target transition information set sequence is used to indicate the transition object action sequence of the target object, the transition object action sequence is used to connect the initial object action and the second object action of the target object, and the target transition information set sequence includes the transition action information set: determining the next transition action information set included in the target transition information set sequence based on the updated current action information set and the target action information set; and determining the next transition action information set as the updated current action information set when the next transition action information set satisfies the action sequence prediction condition.

[0295] Optionally, the transition action sequence determination unit is configured to: determine at least one subset of state information based on the updated current action information set and the target action information set; obtain the state encoding result corresponding to each of the at least one subset of state information; determine the target fusion encoding result based on the at least one state encoding result; and determine the next transition action information set included in the target transition information set sequence based on the target fusion encoding result.

[0296] Optionally, the prediction unit 1306 is configured to: determine the target object part that matches the action type based on the action types of the first object action and the second object action; and determine the target object part based on the interaction position of the interaction element associated with the first object action and the second object action.

[0297] Optionally, the aforementioned transition action sequence determination unit is used for at least one of the following: when at least one of the aforementioned state information subsets includes the current node state information of multiple object nodes in the aforementioned current action information set, the encoding result of the aforementioned current state information subset is determined as the current state code, wherein the aforementioned state encoding result includes the aforementioned current state code, the aforementioned current node state information of the multiple object nodes includes linear velocity information corresponding to each of the multiple target object nodes, velocity information corresponding to each of the multiple root nodes, and rotation information corresponding to each of the multiple object nodes, and the aforementioned target object nodes are used to drive the target object parts of the target object; when at least one of the aforementioned state information subsets includes the target node state information of multiple object nodes in the aforementioned target action information set, the encoding result of the aforementioned current state information subset is determined as the target state code, wherein the aforementioned state encoding result includes the aforementioned target state code, the aforementioned target node state information of the multiple object nodes includes target position information corresponding to each of the multiple target object nodes, and multiple object The target rotation information corresponding to each node; in at least one of the above-mentioned state information subsets, the current state information subset includes node state difference information of multiple object nodes determined according to the current action information set and the target action information set, and the encoding result of the current state information subset is determined as offset state encoding, wherein the state encoding result includes the offset state encoding, and the node state difference information of multiple object nodes includes position difference information corresponding to multiple target object nodes, position difference information corresponding to multiple root nodes, and rotation difference information corresponding to multiple object nodes; in at least one of the above-mentioned state information subsets, the current state information subset includes velocity state information of multiple object nodes in the target action information set, and the node-aware convolution result of the current state information subset is obtained, and the rhythm state encoding is determined according to the node-aware convolution result, wherein the state encoding result includes the rhythm state encoding, and the velocity state information of multiple object nodes includes linear velocity information of multiple object nodes and rotation information of multiple object nodes.

[0298] Optionally, the transition action sequence determination unit is configured to: obtain a current position encoding result determined based on a subset of position information in the current action information set; superimpose the current position encoding result with at least one of the state encoding results to obtain a target encoding result corresponding to each of the at least one state encoding result; determine the target fusion encoding result based on at least one target encoding result; further be configured to: decode the target fusion encoding result using a long short-term memory network to obtain the reference action information set; input the subset of target action information associated with the target object part in the reference action information set and the subset of current action information associated with the target object part in the current action information set into the part action prediction network to obtain the subset of part action information matching the target object part; and determine the next transition action information set included in the target action information set sequence based on the subset of part action information and the reference action information set.

[0299] Optionally, the aforementioned transition action sequence determination unit further includes: a rendering module, configured to sequentially add the transition action information sets included in the target transition information set sequence to a rendering cache queue, wherein the transition action information in the rendering cache queue is used to render the transition object action of the target object; obtain from the rendering cache queue a plurality of first action information sets that have not yet been used to render the transition object action of the target object, and obtain compensation information sets corresponding to each of the plurality of first action information sets, and adjust the plurality of first action information sets according to the plurality of compensation information sets to obtain a plurality of second action information sets; and replace the plurality of first action information sets in the rendering cache queue with the plurality of second action information sets.

[0300] Optionally, the transition action sequence determination unit is configured to: determine a sequence of first action information sets and a target action information set as a sequence of first action information sets; determine a first state difference information set based on the first first action information set and the target action information set in the sequence of first action information sets, wherein the first state difference information set includes first node state differences corresponding to each of a plurality of object nodes; determine a second state difference information set corresponding to each of every two adjacent first action information sets in the sequence of first action information sets, wherein the second state difference information set includes second node state differences corresponding to each of a plurality of node objects; and determine a compensation information set corresponding to each of the plurality of first action information sets based on the first state difference information set and the second state difference information set.

[0301] Optionally, the aforementioned transition action sequence determination unit is further configured to: acquire playback indication information matching the aforementioned target transition information set sequence, wherein the playback indication information is used to indicate the playback speed of the transition object action between the aforementioned initial object action and the aforementioned second object action; determine at least one of the aforementioned transition action information sets matching the current time based on the aforementioned playback indication information; and determine a current transition action information set matching the aforementioned current time based on a weighted result among at least one of the aforementioned transition action information sets, wherein the current transition action information set is used to render the aforementioned transition object action of the aforementioned target object at the aforementioned current time.

[0302] Optionally, the aforementioned transition action sequence determination unit is further configured to: acquire a first action information set and a second action information set matching the target object, wherein the first action information set is used to indicate the initial object action of the target object at an initial time, and the second action information set is used to indicate the target object action of the target object at a target time; adjust the first action information set and the second action information set according to the model adjustment weight matching the target object to obtain an initial action information set and a target action information set, wherein the initial action information set and the target action information set are used to determine the target transition information set sequence; after determining the transition action information set as the updated current action information set and repeatedly executing the following steps until the target transition information set sequence is obtained, the unit is further configured to: adjust the weight according to the model adjustment weight matching the target object to restore multiple transition action information sets in the target transition information set sequence to obtain multiple target transition action information sets, wherein the multiple target transition action information sets are used to render the transition object action of the target object in the virtual scene.

[0303] Optionally, the above-mentioned object action determination device further includes: a training unit, configured to acquire an initial sample action information set and a target sample action information set matching the sample object, wherein the initial sample action information set is used to indicate the initial object action of the sample object at an initial time, and the target action information set is used to indicate the target object action of the sample object at a target time, the target time being later than the initial time; inputting the current action information set and the target action information set into a transitional action sequence prediction model to obtain a sample transitional action information set sequence, wherein the transitional action sequence prediction model includes a long short-term memory network and the above-mentioned part action prediction network, the long short-term memory network being used to determine a reference sample action information set based on the initial sample action information set and the target sample action information set. The aforementioned part action prediction network is used to determine a subset of sample part action information that matches the target object part based on a subset of target sample action information associated with the target object part in the aforementioned reference sample action information set and a subset of initial action information associated with the target object part in the aforementioned initial sample action information set. The subset of sample part action information and the aforementioned reference sample action information set are used to determine the sample transition action information set in the aforementioned sample transition action information set sequence. Based on the label transition action information set sequence that matches the aforementioned initial sample action information set and the aforementioned target sample action information set, and the aforementioned sample transition action information set sequence, the current model training loss is determined. Based on the aforementioned current model training loss, at least one model parameter in the aforementioned transition action sequence prediction model is adjusted.

[0304] Optionally, the training unit is configured to input the initial sample action information set and the target sample action information set into the long short-term memory network of the transition action sequence prediction model to obtain a reference sample action information set, wherein the reference sample action information set includes action description information for determining the transition object action of the sample object; input the subset of target sample action information associated with the target object part of the sample object in the reference sample action information set, and the subset of current sample action information associated with the target object part in the initial sample action information set, into the part action prediction network to obtain a part sample action information subset matching the target object part; and determine the transition sample action information set based on the part sample action information subset and the reference sample action information set, wherein the transition sample action information set is used to indicate the transition object action of the target object at a third time, the third time being the time between the first time and the second time.

[0305] Optionally, the training unit is configured to input the target sample action information subset and the current sample action information subset into the gating network of the part action prediction network to obtain sample weight coefficients corresponding to each of the multiple action decoding modules in the part action prediction network; input at least one object information from the target sample action information subset and the current sample action information subset into at least one of the multiple action decoding modules according to the sample weight coefficients; and determine the part sample action information subset according to the output results of each of the at least one action decoding module.Optionally, the training unit is used for at least one of the following: determining a rotation information loss based on the label transition action information set sequence and the sample transition action information set sequence, wherein the rotation information loss is used to indicate a first information difference between the label rotation information corresponding to each object node included in the label transition action information set sequence and the sample rotation information corresponding to each object node included in the sample transition action information set sequence; determining a root node position loss based on the label transition action information set sequence and the sample transition action information set sequence, wherein the root node position loss is used to indicate a first information difference between the label rotation information corresponding to each root node included in the label transition action information set sequence and the sample rotation information set sequence. The location information is compared with the second information difference between the location information and the sample location information corresponding to each root node included in the sample transition action information set sequence; based on the label transition action information set sequence and the sample transition action information set sequence, the target object part information loss is determined, wherein the rotation information loss is used to indicate the third information difference between the label location information corresponding to each target object node included in the label transition action information set sequence and the sample location information corresponding to each target object node included in the sample transition action information set sequence, where the target object node is used to drive the target object part of the target object; the sliding information loss is determined based on the label transition action information set sequence. The aforementioned sliding information loss is used to indicate the degree of sliding of the target object on the virtual ground as indicated by the label transition action information set sequence; ground contact information loss is determined based on the aforementioned label transition action information set sequence and the aforementioned sample transition action information set sequence, wherein the aforementioned rotation information loss is used to indicate the fourth information difference between the label ground contact information indicated by the label transition action information set sequence and the sample ground contact information indicated by the aforementioned sample transition action information set sequence, wherein the aforementioned sample ground contact information is used to indicate the contact state between the virtual foot of the target object and the virtual ground; and object position information loss is determined based on the aforementioned label transition action information set sequence and the aforementioned sample transition action information set sequence, wherein... The aforementioned object position information loss is used to indicate the fifth information difference between the label position information corresponding to each of the multiple object nodes included in the label transition action information set sequence and the sample position information corresponding to each of the multiple object nodes included in the sample transition action information set sequence; based on the aforementioned label transition action information set sequence and the aforementioned sample transition action information set sequence, the object velocity information loss is determined, wherein the aforementioned object position information loss is used to indicate the sixth information difference between the label velocity information corresponding to each of the multiple object nodes included in the label transition action information set sequence and the sample velocity information corresponding to each of the multiple object nodes included in the sample transition action information set sequence.

[0306] Optionally, the training unit is further configured to: acquire multiple sets of weight coefficients included in the trained transition action sequence prediction model; determine at least one set of reference coefficients from the multiple sets of weight coefficients based on the amount of data corresponding to each set of weight coefficients; quantize the weight coefficients in the at least one set of reference coefficients to obtain at least one set of target coefficients after processing; update the transition action sequence prediction model based on the at least one set of target coefficients to obtain a target transition action sequence prediction model; and send the target transition action sequence prediction model to the target terminal, wherein the target transition action sequence prediction model running on the target terminal is used to determine the transition action information set of the target object in the game task during the game task execution.

[0307] Optionally, in this embodiment, the implementation of each of the above-mentioned unit modules can be referred to the above-mentioned method embodiments, which will not be repeated here.

[0308] According to another aspect of the present invention, an electronic device for implementing the above-described method for determining object actions is also provided. This electronic device may be... Figure 14 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 14 As shown, the electronic device includes a memory 1402 and a processor 1404. The memory 1402 stores a computer program, and the processor 1404 is configured to execute the steps of any of the above method embodiments via the computer program.

[0309] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0310] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0311] S1, obtain the current action information set and the target action information set that match the target object, wherein the current action information set is used to indicate the first object action of the target object at a first moment, and the target action information set is used to indicate the second object action of the target object at a second moment, wherein the second moment is later than the first moment.

[0312] S2, based on the current action information set and the target action information set, determine a reference action information set, wherein the reference action information set includes action description information for determining the transition object action of the target object;

[0313] S3, the target action information subset associated with the target object part in the above reference action information set, and the current action information subset associated with the target object part in the above current action information set are input into the part action prediction network to obtain the part action information subset that matches the target object part.

[0314] S4. Based on the above-mentioned subset of body part action information and the above-mentioned reference action information set, a transition action information set is determined, wherein the above-mentioned transition action information set is used to indicate the transition object action of the above-mentioned target object at the third time, and the above-mentioned third time is the time between the above-mentioned first time and the above-mentioned second time.

[0315] Alternatively, as those skilled in the art will understand, Figure 14 The structure shown is for illustrative purposes only. Electronic devices can also be in-vehicle terminals, smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 14 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 14 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 14 The different configurations shown.

[0316] The memory 1402 can be used to store software programs and modules, such as the program instructions / modules corresponding to the object action determination method and apparatus in this embodiment of the invention. The processor 1404 executes various functional applications and data processing by running the software programs and modules stored in the memory 1402, thereby realizing the aforementioned object action determination method. The memory 1402 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1402 may further include memory remotely located relative to the processor 1404, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1402 may be used, but is not limited to, to store file information such as target logical files. As an example, such as Figure 14 As shown, the memory 1402 may include, but is not limited to, the acquisition unit 1302, the first determination unit 1304, the prediction unit 1306, and the second determination unit 1308 in the object action determination device. Furthermore, it may include, but is not limited to, other module units in the object action determination device, which will not be described further in this example.

[0317] Optionally, the transmission device 1406 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1406 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1406 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0318] In addition, the aforementioned electronic device also includes a display 1408 and a connection bus 1410 for connecting various module components in the aforementioned electronic device.

[0319] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0320] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in embodiments of this application.

[0321] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0322] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of a computer device reads computer instructions from the computer-readable storage medium, executes the computer instructions, and causes the computer device to perform the aforementioned method for determining the object action.

[0323] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0324] S1, obtain the current action information set and the target action information set that match the target object, wherein the current action information set is used to indicate the first object action of the target object at a first moment, and the target action information set is used to indicate the second object action of the target object at a second moment, wherein the second moment is later than the first moment.

[0325] S2, based on the current action information set and the target action information set, determine a reference action information set, wherein the reference action information set includes action description information for determining the transition object action of the target object;

[0326] S3, the target action information subset associated with the target object part in the above reference action information set, and the current action information subset associated with the target object part in the above current action information set are input into the part action prediction network to obtain the part action information subset that matches the target object part.

[0327] S4. Based on the above-mentioned subset of body part action information and the above-mentioned reference action information set, a transition action information set is determined, wherein the above-mentioned transition action information set is used to indicate the transition object action of the above-mentioned target object at the third time, and the above-mentioned third time is the time between the above-mentioned first time and the above-mentioned second time.

[0328] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0329] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0330] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0331] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0332] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0333] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0334] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for determining the action of an object, characterized in that, include: Obtain a current action information set and a target action information set that match the target object, wherein the current action information set is used to indicate the first object action of the target object at a first moment, and the target action information set is used to indicate the second object action of the target object at a second moment, the second moment being later than the first moment; Based on the current action information set and the target action information set, a reference action information set is determined, wherein the reference action information set includes action description information for determining the transition object action of the target object; The target action information subset associated with the target object part in the reference action information set and the current action information subset associated with the target object part in the current action information set are input into the part action prediction network to obtain the part action information subset that matches the target object part. A transitional action information set is determined based on the subset of location action information and the reference action information set, wherein the transitional action information set is used to indicate the transitional object action of the target object at a third time, the third time being the time between the first time and the second time.

2. The method according to claim 1, characterized in that, The step of inputting the target action information subset associated with the target object part in the reference action information set and the current action information subset associated with the target object part in the current action information set into the part action prediction network to obtain the part action information subset matching the target object part includes: The target action information subset and the current action information subset are input into the gating network in the part action prediction network to obtain the weight coefficients corresponding to each of the multiple action decoding modules in the part action prediction network. Based on the weighting coefficient, at least one object information from the target action information subset and the current action information subset is input into at least one of the multiple action decoding modules. A subset of the body part motion information is determined based on the output of at least one of the motion decoding modules.

3. The method according to claim 2, characterized in that, Before inputting the target action information subset and the current action information subset into the gating network in the part-body action prediction network to obtain the weight coefficients corresponding to each of the multiple action decoding modules in the part-body action prediction network, the method further includes: The predicted linear velocity information corresponding to each of the multiple root nodes in the reference motion information set is determined as the target motion information subset associated with the target object part, wherein the target object is a virtual object controlled by multiple object nodes with a hierarchical control structure, and the root node is the object node at the first level among the multiple object nodes. The first current linear velocity information corresponding to each of the multiple root nodes, the first current rotation information corresponding to each of the multiple root nodes, the second current linear velocity information corresponding to each of the multiple target object nodes, and the second current rotation information corresponding to each of the multiple root nodes in the current action information set are determined as the current action information subset associated with the target object part, wherein the multiple target object nodes are used to drive the target object part of the target object.

4. The method according to claim 3, characterized in that, Determining the subset of motion information for the body part based on the output results of at least one of the motion decoding modules includes: The predicted linear velocity information and the predicted rotation information corresponding to each of the multiple target object nodes are obtained from the output results of at least one of the motion decoding modules. The predicted linear velocity information corresponding to each of the multiple target object nodes, and the predicted rotation information corresponding to each of the multiple target object nodes, are determined as the subset of the part motion information.

5. The method according to claim 3, characterized in that, The step of determining the transitional motion information set based on the subset of motion information of the body part and the reference motion information set includes: A subset of reference motion information is obtained from the reference motion information set, wherein the subset of reference motion information includes first predicted rotation information corresponding to each of multiple reference object nodes and second predicted rotation information corresponding to each of multiple root nodes, wherein the multiple reference object nodes are used to drive the reference object parts of the target object, and the reference object parts are the object parts of the target object other than the target object parts; The transitional action information set is determined based on the reference action information subset and the part action information subset.

6. The method according to claim 1, characterized in that, After determining the transitional motion information set based on the subset of motion information for the body part and the reference motion information set, the method further includes: If the transition action information set satisfies the action sequence prediction condition, the transition action information set is determined as the updated current action information set, and the following steps are executed repeatedly until a target transition information set sequence is obtained. The target transition information set sequence indicates the transition object action sequence of the target object, and the transition object action sequence connects the initial object action and the second object action of the target object. The target transition information set sequence includes the transition action information set. Based on the updated current action information set and the target action information set, determine the next transition action information set included in the target transition information set sequence; If the next set of transitional action information satisfies the action sequence prediction condition, the next set of transitional action information will be determined as the updated current action information set.

7. The method according to claim 6, characterized in that, The step of determining the next transition action information set included in the target transition information set sequence based on the updated current action information set and the target action information set includes: Based on the updated current action information set and the target action information set, at least one subset of state information is determined; Obtain the state encoding result corresponding to at least one subset of the state information; The target fusion coding result is determined based on at least one of the state coding results; Based on the target fusion coding result, determine the next transition action information set included in the target transition information set sequence.

8. The method according to claim 7, characterized in that, The step of obtaining the state encoding result corresponding to at least one subset of the state information includes at least one of the following: In the case where at least one subset of the state information includes the current node state information of multiple object nodes in the current action information set, the encoding result of the current state information subset is determined as the current state code, wherein the state encoding result includes the current state code, the current node state information of the multiple object nodes includes the linear velocity information corresponding to each of the multiple target object nodes, the velocity information corresponding to each of the multiple root nodes, and the rotation information corresponding to each of the multiple object nodes, and the multiple target object nodes are used to drive the target object part of the target object; In the case where the current state information subset in at least one of the state information subsets includes the target node state information of multiple object nodes in the target action information set, the encoding result of the current state information subset is determined as the target state code, wherein the state encoding result includes the target state code, and the target node state information of the multiple object nodes includes the target position information corresponding to each of the multiple target object nodes and the target rotation information corresponding to each of the multiple object nodes; In at least one of the state information subsets, the current state information subset includes, based on the node state difference information of each of the multiple object nodes determined according to the current action information set and the target action information set, the encoding result of the current state information subset is determined as an offset state code, wherein the state encoding result includes the offset state code, and the node state difference information of each of the multiple object nodes includes the position difference information corresponding to each of the multiple target object nodes, the position difference information corresponding to each of the multiple root nodes, and the rotation difference information corresponding to each of the multiple object nodes; When the current state information subset in at least one of the state information subsets includes the velocity state information of multiple object nodes in the target action information set, the node-aware convolution result of the current state information subset is obtained, and the rhythm state code is determined based on the node-aware convolution result, wherein the state code result includes the rhythm state code, and the velocity state information of the multiple object nodes includes the linear velocity information of the multiple object nodes and the rotation information of the multiple object nodes.

9. The method according to claim 7, characterized in that, Determining the target fusion coding result based on at least one of the state coding results includes: Obtain the current position encoding result determined based on the location information subset of the current action information set; superimpose the current position encoding result and at least one of the state encoding results to obtain the target encoding result corresponding to each of the at least one state encoding result; determine the target fusion encoding result based on at least one of the target encoding results; The step of determining the next transition action information set included in the target transition information set sequence based on the target fusion coding result includes: The target fusion encoding result is decoded using a long short-term memory network to obtain the reference action information set; The subset of target action information associated with the target object part in the reference action information set and the subset of current action information associated with the target object part in the current action information set are input into the part action prediction network to obtain the subset of part action information that matches the target object part. Based on the subset of body part action information and the reference action information set, determine the next transition action information set included in the target action information set sequence.

10. The method according to claim 6, characterized in that, After determining the next transition action information set included in the target transition information set sequence based on the updated current action information set and the target action information set, the method further includes: The transition action information set included in the target transition information set sequence is added to the rendering cache queue in sequence, wherein the transition action information in the rendering cache queue is used to render the transition object action of the target object; From the rendering cache queue, obtain multiple first action information sets that have not yet been used to render the transition object action of the target object, and obtain compensation information sets corresponding to each of the multiple first action information sets. Adjust the multiple first action information sets according to the multiple compensation information sets to obtain multiple second action information sets. Replace multiple sets of the first action information in the rendering cache queue with multiple sets of the second action information.

11. The method according to claim 6, characterized in that, After determining the next transition action information set included in the target transition information set sequence based on the updated current action information set and the target action information set, the method further includes: Obtain playback indication information that matches the target transition information set sequence, wherein the playback indication information is used to indicate the playback speed of the transition object action between the initial object action and the second object action; Determine at least one set of transition action information that matches the current moment based on the playback instruction information; Based on the weighted result among at least one set of transition action information, a current set of transition action information matching the current moment is determined, wherein the current set of transition action information is used to render the transition object action of the target object at the current moment.

12. The method according to claim 6, characterized in that, Before determining the transition action information set as the updated current action information set and repeatedly executing the following steps until the target transition information set sequence is obtained, the method further includes: Obtain a first action information set and a second action information set that match the target object, wherein the first action information set is used to indicate the initial object action of the target object at an initial time, and the second action information set is used to indicate the target object action of the target object at a target time; The weights are adjusted according to the model matching the target object, and the first action information set and the second action information set are adjusted respectively to obtain the initial action information set and the target action information set, wherein the initial action information set and the target action information set are used to determine the target transition information set sequence; After determining the transition action information set as the updated current action information set and repeatedly executing the following steps until the target transition information set sequence is obtained, the method further includes: The weights are adjusted according to the model matching the target object, and multiple transition action information sets in the target transition information set sequence are restored to obtain multiple target transition action information sets. The multiple target transition action information sets are used to render the transition object action of the target object in the virtual scene.

13. The method according to claim 1, characterized in that, Before obtaining the current action information set and the target action information set that match the target object, the method further includes: Obtain an initial sample action information set and a target sample action information set that match the sample object, wherein the initial sample action information set is used to indicate the initial object action of the sample object at an initial time, and the target action information set is used to indicate the target object action of the sample object at a target time, wherein the target time is later than the initial time; The initial sample action information set and the target sample action information set are input into the transition action sequence prediction model to obtain a sample transition action information set sequence. The transition action sequence prediction model includes a long short-term memory network and a part action prediction network. The long short-term memory network is used to determine a reference sample action information set based on the initial sample action information set and the target sample action information set. The part action prediction network is used to determine a sample part action information subset that matches the target object part based on the target sample action information subset associated with the target object part in the reference sample action information set and the initial action information subset associated with the target object part in the initial sample action information set. The sample part action information subset and the reference sample action information set are used to determine the sample transition action information set in the sample transition action information set sequence. The current model training loss is determined based on the sequence of label transition action information sets that match the initial sample action information set and the target sample action information set, and the sequence of sample transition action information sets. Based on the current model training loss, at least one model parameter in the transition action sequence prediction model is adjusted.

14. The method according to claim 13, characterized in that, The step of determining the current model training loss based on the label transition action information set sequence matching the initial sample action information set and the target sample action information set, and the sample transition action information set sequence, includes at least one of the following: Based on the label transition action information set sequence and the sample transition action information set sequence, a rotation information loss is determined, wherein the rotation information loss is used to indicate a first information difference between the label rotation information corresponding to each object node included in the label transition action information set sequence and the sample rotation information corresponding to each object node included in the sample transition action information set sequence. Based on the label transition action information set sequence and the sample transition action information set sequence, the root node position loss is determined, wherein the root node position loss is used to indicate the second information difference between the label position information corresponding to each root node included in the label transition action information set sequence and the sample position information corresponding to each root node included in the sample transition action information set sequence. Based on the label transition action information set sequence and the sample transition action information set sequence, the target object part information loss is determined, wherein the target object part information loss is used to indicate the third information difference between the label position information corresponding to each target object node included in the label transition action information set sequence and the sample position information corresponding to each target object node included in the sample transition action information set sequence, and the target object node is used to drive the target object part of the target object; The slip information loss is determined based on the label transition action information set sequence, wherein the slip information loss is used to indicate the degree of slip of the target object on the virtual ground as indicated by the label transition action information set sequence; Based on the label transition action information set sequence and the sample transition action information set sequence, a ground contact information loss is determined, wherein the ground contact information loss is used to indicate the fourth information difference between the label ground contact information indicated by the label transition action information set sequence and the sample ground contact information indicated by the sample transition action information set sequence, and the sample ground contact information is used to indicate the contact state between the virtual foot of the target object and the virtual ground. Based on the label transition action information set sequence and the sample transition action information set sequence, the object position information loss is determined, wherein the object position information loss is used to indicate the fifth information difference between the label position information corresponding to each of the multiple object nodes included in the label transition action information set sequence and the sample position information corresponding to each of the multiple object nodes included in the sample transition action information set sequence. Based on the label transition action information set sequence and the sample transition action information set sequence, the object velocity information loss is determined, wherein the object velocity information loss is used to indicate the sixth information difference between the label velocity information corresponding to each of the multiple object nodes included in the label transition action information set sequence and the sample velocity information corresponding to each of the multiple object nodes included in the sample transition action information set sequence.

15. The method according to claim 13, characterized in that, The initial sample action information set and the target sample action information set are input into the transition action sequence prediction model to obtain a sample transition action information set sequence, including: The initial sample action information set and the target sample action information set are input into the long short-term memory network in the transition action sequence prediction model to obtain a reference sample action information set, wherein the reference sample action information set includes action description information for determining the transition object action of the sample object; The target sample action information subset associated with the target object part of the sample object in the reference sample action information set, and the current sample action information subset associated with the target object part in the initial sample action information set are input into the part action prediction network to obtain the part sample action information subset that matches the target object part. Based on the subset of action information of the part sample and the set of action information of the reference sample, a set of action information of the transition sample is determined, wherein the set of action information of the transition sample is used to indicate the action of the target object at a third time, and the third time is the time between the first time and the second time; The current model training loss adjusts at least one model parameter of the Long Short-Term Memory network in the transition action sequence prediction model.

16. The method according to claim 15, characterized in that, The step of inputting the subset of target sample action information associated with the target object part of the reference sample action information set and the subset of current sample action information associated with the target object part of the initial sample action information set into the part action prediction network to obtain the subset of part sample action information matching the target object part includes: The target sample action information subset and the current sample action information subset are input into the gating network in the part action prediction network to obtain the sample weight coefficients corresponding to each of the multiple action decoding modules in the part action prediction network. Based on the sample weight coefficient, at least one object information from the target sample action information subset and the current sample action information subset is input into at least one of the multiple action decoding modules. Based on the output results of at least one of the action decoding modules, a subset of the action information of the body part sample is determined; The current model training loss adjusts at least one model parameter of the part action prediction network in the transition action sequence prediction model.

17. The method according to claim 13, characterized in that, After adjusting at least one model parameter in the transition action sequence prediction model based on the current model training loss, the method further includes: Obtain the set of multiple weight coefficients included in the completed training of the transition action sequence prediction model; Based on the amount of data corresponding to each of the multiple sets of weight coefficients, at least one set of reference coefficients is determined from the multiple sets of weight coefficients; The weight coefficients in at least one of the reference coefficient sets are quantized to obtain at least one target coefficient set after processing. The transition action sequence prediction model is updated based on at least one of the target coefficient sets to obtain the target transition action sequence prediction model; The target transition action sequence prediction model is sent to the target terminal, wherein the target transition action sequence prediction model running on the target terminal is used to determine the transition action information set of the target object in the game task during the game task execution.

18. A device for determining the action of an object, characterized in that, include: The acquisition unit is used to acquire a current action information set and a target action information set that match the target object, wherein the current action information set is used to indicate the first object action of the target object at a first moment, and the target action information set is used to indicate the second object action of the target object at a second moment, the second moment being later than the first moment; The first determining unit is configured to determine a reference action information set based on the current action information set and the target action information set, wherein the reference action information set includes action description information for determining the transition object action of the target object; The prediction unit inputs the subset of target action information associated with the target object part in the reference action information set and the subset of current action information associated with the target object part in the current action information set into the part action prediction network to obtain the subset of part action information that matches the target object part. The second determining unit is configured to determine a transitional action information set based on the subset of the part action information and the reference action information set, wherein the transitional action information set is used to indicate the transitional object action of the target object at a third time, and the third time is the time between the first time and the second time.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 17.

20. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 17.

Images