Training sample generation method and device, storage medium and electronic equipment

Abstract

The present disclosure provides a method and device for generating training samples, a storage medium and an electronic device. The method comprises: obtaining a set of action description information, wherein the set of action description information comprises a plurality of groups of action description information, each group of action description information comprises first action description information for indicating an action of a first object and second action description information for indicating an action of a second object; in a case where at least one subset of action description information is determined by performing a clustering operation based on the set of action description information, obtaining first sample action information and second sample action information from the at least one subset of action description information; performing an action collection operation based on the first sample action information and the second sample action information to obtain a plurality of frames of transition sample action information; and determining a target training sample according to the first sample action information, the second sample action information and the plurality of frames of transition sample action information. The present disclosure solves the technical problem of low quality of training samples obtained by related methods.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to a method and apparatus for generating training samples, a storage medium, and an electronic device. Background Technology

[0002] When designing and rendering the motion of virtual objects, the relevant technologies first capture motion data in keyframes. Then, using motion prediction models, the transition motion between keyframes is inferred based on these keyframe motions.

[0003] Before applying action prediction models to predict transitional actions, it is usually necessary to train the model with training samples to optimize its performance. To enhance the training effect, action samples are usually randomly selected from a regular action dataset for training. However, since these randomly selected action samples differ significantly from the transitional actions in real-world application scenarios, existing methods for obtaining training samples suffer from the technical problem of low sample quality.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method and apparatus for generating training samples, a storage medium, and an electronic device to at least address the technical problem of low training sample quality in related fields.

[0006] According to one aspect of the present invention, a method for generating training samples is provided, comprising: acquiring an action description information set, wherein the action description information set includes multiple action description information groups, wherein the action description information groups include first action description information for indicating a first object action and second action description information for indicating a second object action; acquiring first sample action information and second sample action information from at least one of the action description information subsets after performing a clustering operation based on the action description information set to determine at least one subset of action description information; performing an action acquisition operation based on the first sample action information and the second sample action information to obtain multiple frames of transition sample action information, wherein the multiple frames of transition sample action information are used to indicate multiple frames of transition object actions matching the first object action and the second object action; and determining a target training sample based on the first sample action information, the second sample action information, and the multiple frames of transition sample action information, wherein the target training sample is used to train a transition action prediction model.

[0007] According to another aspect of the present invention, an apparatus for generating training samples is also provided, comprising: a first acquisition unit, configured to acquire an action description information set, wherein the action description information set includes a plurality of action description information groups, wherein the action description information groups include first action description information for indicating a first object action and second action description information for indicating a second object action; a second acquisition unit, configured to acquire first sample action information and second sample action information from at least one subset of action description information after performing a clustering operation based on the action description information set to determine at least one subset of action description information; a collection unit, configured to perform an action collection operation based on the first sample action information and the second sample action information to obtain multiple frames of transition sample action information, wherein the multiple frames of transition sample action information are used to indicate multiple frames of transition object actions matching the first object action and the second object action; and a determination unit, configured to determine a target training sample based on the first sample action information, the second sample action information and the multiple frames of transition sample action information, wherein the target training sample is used to train a transition action prediction model.

[0008] Optionally, the above-mentioned training sample generation device further includes one of the following: a first clustering unit, configured to determine a reference action description matrix corresponding to each of the plurality of action description information groups in the action description information set; and to perform a clustering operation on the plurality of action description information groups according to the spatial distance relationship between the plurality of reference action description matrices to obtain at least one subset of the action description information; a second clustering unit, configured to determine at least one first action description information subset based on the first action description information included in each of the plurality of action description information groups, and to determine at least one second action description information subset based on the second action description information included in each of the plurality of action description information groups, wherein the first action description information subset includes a plurality of first action description information that satisfies a second clustering condition, the second action description information subset includes a plurality of second action description information that satisfies a third clustering condition, and at least one subset of the action description information includes the first action description information subset and the second action description information subset.

[0009] Optionally, the first clustering unit is configured to: obtain a first action description matrix corresponding to the first action description information in the current action description information group, and a second action description matrix corresponding to the second action description information in the current action description information group; determine an action difference matrix based on the first action description matrix and the second action description matrix, and determine a third action description matrix based on the action difference matrix and the first action description matrix, wherein the first action description matrix is ​​used to indicate the first object action performed by the virtual object at a first moment, and the second action description matrix is ​​used to indicate the second object action performed by the virtual object at a second moment, the second moment being later than the first moment; and determine the projection matrix obtained by spatially projecting the third action description matrix into the target vector space as the reference action description matrix, wherein the first matrix dimension of the reference action description matrix is ​​lower than the second matrix dimension of the third action description matrix.

[0010] Optionally, the first clustering unit is configured to: obtain multiple reference action description matrices and multiple current centroid matrices; determine clustering distance information corresponding to each of the multiple reference action description matrices based on the multiple reference action description matrices and the multiple current centroid matrices, wherein the clustering distance information includes multiple matrix distances between the multiple current centroid matrices and the reference action description matrices; update the clustering labels matched by each of the multiple reference action description matrices based on the clustering distance information corresponding to each of the multiple reference action description matrices, wherein the clustering labels are used for the action description information subset to which the action description information group currently belongs; update the multiple current centroid matrices based on the reference action description matrices matched by each of the multiple current centroid matrices to obtain updated multiple current centroid matrices; if the multiple current centroid matrices satisfy the first clustering condition, determine at least one subset of the action description information based on the clustering labels matched by each of the multiple reference action description matrices; if the multiple current centroid matrices do not satisfy the first clustering condition, repeat the above steps.

[0011] Optionally, the second acquisition unit includes a first acquisition module and a second acquisition module. The first acquisition module is configured to: after performing a clustering operation on multiple action description information groups based on the spatial distance relationship between multiple reference action description matrices to obtain at least one subset of action description information, the module further includes: acquiring the action description information group from the subset of action description information, and acquiring the first sample action information and the second sample action information from the subset of action description information. The second acquisition module is configured to: after determining at least one first action description information subset based on the first action description information included in each of the multiple action description information groups, and determining at least one second action description information subset based on the second action description information included in each of the multiple action description information groups, the module further includes: acquiring the first sample action information from the subset of first action description information, and acquiring the second sample action information from the subset of second action description information.

[0012] Optionally, the acquisition unit is configured to: display a first virtual object performing the first object action based on the first sample action information, and display a second virtual object performing the second object action based on the second sample action information, wherein the first virtual object is displayed according to a first style and the second virtual object is displayed according to a second style; in response to an action acquisition operation, if the action acquisition device acquires current sample action information, display a third virtual object performing the current sample object action based on the current sample action information, wherein the third virtual object is displayed according to a third style; and if the acquired multiple frames of current sample action information meet the acquisition conditions, determine multiple frames of transitional sample action information that match the first sample action information and the second sample action information based on the multiple frames of current sample action information.

[0013] Optionally, the acquisition unit is configured to: render the first sample object action performed by the reference virtual object in the target rendering engine according to the first sample action information, and render the second sample object action performed by the reference virtual object at the same virtual position according to the second sample action information; sample first reference action information and second reference action information according to the first sample object action and the second sample object action; initialize the root bone position information and rotation information in the first reference action information and the second reference action information to obtain updated first reference action information and second reference action information; and in the action acquisition application, display the first virtual object performing the first object action according to the updated first reference action information, and display the second virtual object performing the second object action according to the second reference action information.

[0014] Optionally, the acquisition unit is further configured to: in response to a first angle adjustment operation, display the first virtual object performing the first object action at the adjusted first display angle; in response to a second angle adjustment operation, display the second virtual object performing the second object action at the adjusted second display angle; and in response to a third angle adjustment operation, display the third virtual object performing the currently sampled object action at the adjusted third display angle.

[0015] Optionally, the acquisition unit is further configured to: determine that the current sampling action information of the multiple frames satisfies the acquisition conditions when the first matching degree between the starting sampling action information in the multiple frames of the current sampling action information and the first sample action information is greater than or equal to a first threshold, and the second matching degree between the ending sampling action information in the multiple frames of the current sampling action information and the second sample action information is greater than or equal to a second threshold.

[0016] Optionally, the training sample generation apparatus further includes at least one of the following: a first sample enhancement unit, configured to obtain a sequence of sample action information from the current training samples included in the plurality of target training samples, wherein the sequence of sample action information includes the first sample action information, the second sample action information, and multiple frames of transition sample action information; perform mirror transformation operations on the plurality of sample action information in the sequence of sample action information to obtain the first sample action information sequence; and use the first sample action information sequence as a newly added target training sample; and a second sample enhancement unit, configured to obtain a sequence of sample action information from the current training samples included in the plurality of target training samples, wherein the first sample action sequence includes the first sample action information, the second sample action information, and multiple frames of transition sample action information. The process involves: performing a velocity transformation operation on the aforementioned sample action information sequence to obtain a second sample action information sequence; using the second sample action information sequence as a newly added target training sample; and a third sample enhancement unit, configured to obtain a sample action information sequence from the current training samples included in the plurality of target training samples, wherein the first sample action sequence includes the first sample action information obtained from the first action description subset, the second sample action information obtained from the second action description subset, and multiple frames of transition sample action information; obtaining a first reference sample action information from the first action description subset and a second reference sample action information from the second action description subset; and using the first reference sample action information, the second reference sample action information, and multiple frames of transition sample action information as newly added target training samples.

[0017] Optionally, the first sample enhancement unit is configured to: obtain current sample action information from multiple sample action information; perform a first mirror symmetry operation on the target rotation information of the first part included in the current sample action information; perform a second mirror symmetry operation on the three-dimensional coordinate information of the second part included in the current sample action information; and determine the updated current sample action information as the first sample action information included in the first sample action information sequence.

[0018] Optionally, the second sample enhancement unit is configured to: acquire velocity description information corresponding to the velocity transformation operation; determine the number of second sample action information included in the second sample action information sequence and the sequence identifier corresponding to each of the multiple sample action information in the sample action information sequence based on the velocity description information; determine the weight information between every two adjacent second sample action information based on the velocity description information; and perform interpolation processing on the multiple sample action information based on the sequence identifier corresponding to each of the multiple sample action information and the weight information between every two adjacent second sample action information to obtain the second sample action information sequence.

[0019] Optionally, the second sample enhancement unit is configured to: perform coordinate transformation operations on the multiple sample action information to obtain multiple sample global action information in a global coordinate system; perform linear interpolation operations on the global position information in the multiple sample global action information to obtain multiple global position information corresponding to the second sample action information sequence; perform spherical linear interpolation operations on the global rotation information in the multiple sample global action information to obtain multiple global rotation information corresponding to the second sample action information sequence; and determine the second sample action information sequence based on the multiple global position information and the multiple global rotation information.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the above-described method for generating training samples at runtime.

[0021] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the training sample generation method as described above.

[0022] According to another aspect of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the training sample generation method described above through the computer program.

[0023] In the above method for generating training samples, firstly, an action description information set containing multiple action description information groups is obtained. Each action description information group contains both first action description information indicating the action of a first object and second action description information indicating the action of a second object. Next, by performing a clustering operation, at least one subset of action description information is determined from the action description information set. After obtaining at least one subset of action description information, first sample action information and second sample action information are extracted from these subsets. Further, an action acquisition operation is performed based on the first sample action information and the second sample action information to obtain multi-frame transition sample action information. Finally, the target training sample is determined based on the first sample action information, the second sample action information, and the multi-frame transition sample action information.

[0024] Through the above-described embodiments of this application, the combination of clustering and action acquisition operations not only improves the generalization effect and quality of the samples but also significantly reduces the number of action acquisition operations. On the one hand, by clustering action information, similar first and second action description information are grouped into one category, thereby avoiding repeated acquisition of similar actions and significantly improving the generalization effect of the samples. On the other hand, since the action acquisition operation is based only on the first and second action description information obtained from the clustering operation, without needing to acquire every group of action description information in the dataset, the generation efficiency of training samples is further improved while ensuring the quality of training samples, thus solving the technical problem of low training sample quality in the prior art. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0026] Figure 1 This is a schematic diagram of the hardware environment for an optional training sample generation method according to an embodiment of the present invention;

[0027] Figure 2 This is a flowchart of an optional method for generating training samples according to an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of an optional method for predicting transition actions according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of another optional method for predicting transition actions according to an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of an optional object node according to an embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of an optional clustering method according to an embodiment of the present invention;

[0032] Figure 7 This is a schematic diagram of another optional clustering method according to an embodiment of the present invention;

[0033] Figure 8 This is a flowchart of another optional method for generating training samples according to an embodiment of the present invention;

[0034] Figure 9 This is a flowchart of another optional method for generating training samples according to an embodiment of the present invention;

[0035] Figure 10 This is a schematic diagram of another optional object node according to an embodiment of the present invention;

[0036] Figure 11 This is a schematic diagram of another optional clustering method according to an embodiment of the present invention;

[0037] Figure 12 This is a flowchart of an optional motion capture method according to an embodiment of the present invention;

[0038] Figure 13 This is a schematic diagram of an optional motion capture method according to an embodiment of the present invention;

[0039] Figure 14 This is a schematic diagram of another optional motion capture method according to an embodiment of the present invention;

[0040] Figure 15 This is a scene diagram of an optional motion capture method according to an embodiment of the present invention;

[0041] Figure 16 This is a schematic diagram of an optional data augmentation method according to an embodiment of the present invention;

[0042] Figure 17 This is a schematic diagram of another optional data augmentation method according to an embodiment of the present invention;

[0043] Figure 18 This is a flowchart of another optional method for generating training samples according to an embodiment of the present invention;

[0044] Figure 19 This is a schematic diagram of an optional action prediction model training method according to an embodiment of the present invention;

[0045] Figure 20 A schematic diagram of the structure of an optional training sample generation device according to an embodiment of the present invention;

[0046] Figure 21 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. Detailed Implementation

[0047] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0048] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] It should be noted that in the relevant embodiments of this application, the acquisition and processing of relevant data information of the target account are all obtained in advance through methods that comply with the relevant normative legal documents, and the authorization permission of the corresponding account subject must be obtained before acquiring the above-mentioned relevant information and data.

[0050] The following explains the technical terms used in this application:

[0051] BVH format files: BVH files contain the character's skeleton and limb joint rotation data. It is a universal human feature animation file format that is widely supported by various popular animation software today.

[0052] FBX (Filmbox) format files: FBX files are a standard file format for 3D data exchange. They are widely used in game development, film and television production, and other 3D design fields, supporting the integration and transmission of various data types, including models, textures, and animations.

[0053] According to one aspect of the present invention, a method for generating training samples is provided, which may be applied, but is not limited to, methods for generating training samples such as... Figure 1 The training sample generation system in the hardware environment shown may include, but is not limited to, terminal device 102, network 104, server 106, and database 108. Terminal device 102 runs an information processing client for processing action description information sets. Terminal device 102 includes a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen in terminal device 102 can be used to display a virtual scene and also provides a human-computer interaction interface to receive human-computer interaction operations for controlling virtual objects in the virtual scene, which will complete game tasks set in the virtual scene. The processor is used to generate interaction instructions in response to the aforementioned human-computer interaction operations and send the interaction instructions to the server. The memory is used to store relevant attribute data, such as object attribute information of the controlled virtual object and attribute information of the virtual props it holds. This attribute information may include, but is not limited to, information used to identify its identity and current location.

[0054] Furthermore, server 106 and database 108 can form a distributed network. Server 106 includes a processing engine used to perform store or read operations on database 108. Specifically, the processing engine reads the virtual scene information of each virtual object and the operation information it performs from database 108.

[0055] Assumption Figure 1The terminal device 102 runs a client for displaying a virtual scene. The specific process of this embodiment is as follows: The terminal device 102 executes steps S102 to S108 to obtain an action description information set, wherein the action description information set includes multiple action description information groups, and each action description information group includes first action description information for indicating the action of a first object and second action description information for indicating the action of a second object; If at least one subset of action description information is determined by performing a clustering operation based on the action description information set, first sample action information and second sample action information are obtained from the at least one subset of action description information; Based on the first sample action information and the second sample action information, an action acquisition operation is performed to obtain multi-frame transition sample action information, wherein the multi-frame transition sample action information is used to indicate multi-frame transition object actions that match the actions of the first object and the second object; Based on the first sample action information, the second sample action information, and the multi-frame transition sample action information, a target training sample is determined, wherein the target training sample is used to train the transition action prediction model;

[0056] Next, terminal device 102 executes S110 to send the target training samples to server 106 via network 104. Finally, server 106 executes S112 to determine the target training sample set based on the target training samples.

[0057] Optionally, in this embodiment, the terminal device 102 can be a terminal device configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client can be a data operation client that provides operations on a set of action description information. The network may include, but is not limited to, wired networks and wireless networks, wherein the wired network includes: local area network, metropolitan area network, and wide area network, and the wireless network includes: Bluetooth, WIFI, and other networks that enable wireless communication. The server may be a single server, a server cluster composed of multiple servers, or a cloud server. The above is merely an example, and no limitation is made in this embodiment.

[0058] Optionally, in this embodiment, the action description information involved in the above-described training sample generation method can be, but is not limited to, applied to rendering based on object actions of a target virtual object in a virtual scene. The target virtual object can be a virtual object involved in a game-type terminal application (APP) used to complete a predetermined interactive game task.

[0059] The training samples obtained by the above-described method can be used to train a transitional action generation model. This transitional action generation model can generate transitional action information for at least one transitional object action between the first object action and the second object action, based on first action information indicating the first object action and second object action, and second action information indicating the second object action.

[0060] The aforementioned first action information, second action information, and transition action information can be applied to render the object actions of virtual objects in Multiplayer Online Battle Arena (MOBA) applications. In these MOBAs, players can control virtual objects to release character skills within a virtual environment, thereby creating virtual objects within that environment. The aforementioned interactive game tasks can be, but are not limited to, game tasks completed by the current player through virtual social interaction between virtual objects generated from human-computer interaction training samples and virtual objects controlled by other players. The method for generating these training samples can also be applied to massively multiplayer online role-playing games (MMORPGs) on mobile devices. In these games, the current player can complete interactive game tasks from the first-person perspective of a virtual object through role-playing, such as completing game tasks together with other virtual objects. These interactive game tasks can, but are not limited to, run as plugins or mini-programs within an application (such as a non-standalone game app), or run within a game engine within an application (such as a standalone game app). The types of game applications mentioned above may include, but are not limited to, at least one of the following: two-dimensional (2D) game applications, three-dimensional (3D) game applications, virtual reality (VR) game applications, augmented reality (AR) game applications, and mixed reality (MR) game applications. This is merely an example, and this embodiment does not impose any limitations on it.

[0061] In the above method for generating training samples, firstly, an action description information set containing multiple action description information groups is obtained. Each action description information group contains both first action description information indicating the action of a first object and second action description information indicating the action of a second object. Next, by performing a clustering operation, at least one subset of action description information is determined from the action description information set. After obtaining at least one subset of action description information, first sample action information and second sample action information are extracted from these subsets. Further, an action acquisition operation is performed based on the first sample action information and the second sample action information to obtain multi-frame transition sample action information. Finally, the target training sample is determined based on the first sample action information, the second sample action information, and the multi-frame transition sample action information.

[0062] Through the above-described embodiments of this application, the combination of clustering and action acquisition operations not only improves the generalization effect and quality of the samples but also significantly reduces the number of action acquisition operations. On the one hand, by clustering action information, similar first and second action description information are grouped into one category, thereby avoiding repeated acquisition of similar actions and significantly improving the generalization effect of the samples. On the other hand, since the action acquisition operation is based only on the first and second action description information obtained from the clustering operation, without needing to acquire every group of action description information in the dataset, the generation efficiency of training samples is further improved while ensuring the quality of training samples, thus solving the technical problem of low training sample quality in the prior art.

[0063] As an optional implementation method, such as Figure 2 As shown, the above method for generating training samples can be applied to servers or terminals, and specifically includes the following steps:

[0064] S202, Obtain an action description information set, wherein the action description information set includes multiple action description information groups, and the action description information groups include first action description information for indicating the action of the first object and second action description information for indicating the action of the second object.

[0065] S204, when at least one subset of action description information is determined by performing a clustering operation based on the action description information set, first sample action information and second sample action information are obtained from the at least one subset of action description information.

[0066] S206, based on the first sample action information and the second sample action information, perform an action acquisition operation to obtain multi-frame transition sample action information, wherein the multi-frame transition sample action information is used to indicate multi-frame transition object actions that match the first object action and the second object action.

[0067] S208, determine the target training sample based on the action information of the first sample, the action information of the second sample, and the action information of the multi-frame transition sample, wherein the target training sample is used to train the transition action prediction model.

[0068] First, the application scenarios of the target training samples obtained in S202 to S208 of the above embodiments will be explained. In an optional manner, the target training samples obtained in S202 to S208 can be used to train a transition action prediction model. The transition action prediction model can be used to predict the transition object actions between keyframes. It should be noted that when the keyframe is used to render a virtual scene, and the virtual scene includes virtual objects performing object actions, the keyframe can include action description information for indicating the object actions. For example, the keyframe can include scene rendering data for rendering, and object rendering data for rendering virtual objects. In the object rendering data, the data for indicating object actions can be determined as object description information, which may include, but is not limited to, the position information, velocity information, and rotation information of each driving node in the object-driven model corresponding to the virtual object.

[0069] Furthermore, in the two keyframes used to predict the action of the transition object, the keyframe with the earlier timestamp can be described as the start frame, in which the virtual object performs the action of the start object in the virtual scene indicated by the start frame; the keyframe with the later timestamp can be described as the end frame, in which the virtual object performs the action of the end object in the virtual scene indicated by the end frame.

[0070] Furthermore, the transition object action information predicted by the aforementioned transition action prediction model can be applied to a game application to determine and render the transition object actions of virtual game objects in the virtual scene of the game application. Furthermore, the method for determining the transition object actions can be a real-time determination method during the operation of the game application, or it can be used during the development of the aforementioned game application to determine the transition actions in the various object display animations corresponding to different virtual game objects.

[0071] In the case where the aforementioned target training samples are applied to the training process of the transitional action prediction model, and the aforementioned transitional action prediction model is used to predict object actions in the game application, the relationship between the first object action and the second object action in step S202 will be further explained below from the perspective of the action type of the object action.

[0072] It should be noted that the first object action and the second object action mentioned above can be two actions performed by the virtual game object in the process of performing virtual object operations in the virtual game scene. For example, when a virtual game object triggers an attack, the first object action and the second object action can be two different attack actions during the attack process; when a virtual game object triggers a skill release, the first object action and the second object action can be two different skill actions during the skill release process; when a virtual game object is attacked by other virtual game objects, the first object action and the second object action can be two different hit actions executed in response to being hit by an attack from another virtual game object; when a virtual game object triggers a movement operation in the virtual game scene, the first object action and the second object action can be two movement actions during the movement process; when it is detected that a virtual game object switches from a standby state to a hit state in the virtual scene, the first object action can be a standby action corresponding to the standby state, and the second object action can be a hit action corresponding to the hit state, wherein the standby state is a state in which an object does not receive a control command and remains stationary, and the hit state is a state in which it is attacked by other virtual game objects; optionally, when a virtual game object switches back from the hit state to the standby state, the first object action can also be a hit action, and the second object action can also be a standby action. The above examples are merely illustrative. In this embodiment, the specific types of the first object action and the second object action are not limited.

[0073] In another alternative approach, when the aforementioned transition motion prediction model is applied to an animation development application, the application can be used to generate and render virtual animation objects in an animation scene. After determining the object actions corresponding to the two key animation frames of the virtual animation object, the transition object actions rendered and displayed in the transition animation frames between the key animation frames can be determined using the transition motion prediction model trained based on the target training samples generated in steps S202 to S208.

[0074] In another alternative approach, when the aforementioned transitional motion prediction model is applied to a 3D modeling scene, this 3D modeling scene can generate CG (Computer-Generated Animation) based on multiple 3D coordinate signal sequences realistically acquired using motion capture technology. Since the frame rate of the multiple 3D coordinate signal sequences acquired by motion capture technology is relatively low, the aforementioned transitional motion prediction model can predict the transitional CG actions between multiple CG actions.

[0075] The following combination Figure 3The method for predicting the actions of transition objects based on a transition action prediction model in the above scenario is explained. For example... Figure 3 As shown, when the transition action prediction model 308 is trained based on the training samples obtained in steps S202 to S208, the starting object action 304 and ending object action 306 of the current virtual object can be obtained based on the interaction between virtual objects in the virtual scene 302. The starting object action 304 can be the object action executed by the current virtual object in a static state or a state where no operation command has been received, and the ending object action 306 can be the object action executed by the current virtual object to indicate the attacked state when attacked by other virtual objects.

[0076] Next, the transition action prediction model 308 can sequence a series of transition actions between the two actions based on the action description information of the starting object action 304 and the ending object action 306, thereby obtaining the object action sequence 309. Through the object action sequence 309, the smooth transition from the starting object action 304 to the ending object action 306 of the current virtual object can be displayed.

[0077] Finally, in the virtual scene 310, a series of object actions performed by the current virtual object when it is attacked by other virtual objects can be displayed based on the above object action sequence 309.

[0078] It should be noted that, in the above embodiments, the transition action prediction model 308 can be a neural network model among Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Convolutional Neural Network (CNN), or a neural network model obtained by combining multiple neural networks in a certain way. In this embodiment, the specific model type of the transition action prediction model 308 is not limited.

[0079] In an optional implementation, the transition action prediction model 308 can directly predict the object action sequence 309 based on the action description information of the ending object action 306 and the starting object action 304.

[0080] In another alternative implementation, such as Figure 4As shown, when the transition object action 406 is predicted and determined based on the starting object action 402 and the ending object action 404 according to the aforementioned transition action prediction model 308, the transition object action 408 can be further determined based on the transition object action 406 and the ending object action 404. Finally, the starting object action 402, the transition object action 406, the transition object action 408, and the ending object action 404 are determined as the object action sequence 410. It can be understood that in the above embodiment, the above object actions can all be represented as action description information in the above transition action prediction model, thereby realizing the prediction of each transition action in the transition action sequence according to the transition action prediction model 308.

[0081] The above implementation methods are merely illustrative examples and do not limit the specific application scenarios of the transition action prediction model determined in step S208.

[0082] Furthermore, when the above-described embodiments are applied to a game scenario, steps S202 to S208 can be applied to a game client or a game server. When applied to a game client, the above steps can be implemented by an action generation thread that runs in parallel with the game screen rendering thread in the terminal; when applied to a game server, the terminal can obtain the determination result of the transition action information set of the game server in real time via the network, and perform screen rendering in the terminal based on the obtained transition action information set.

[0083] Furthermore, it should be noted that the aforementioned action description information set can be a pre-acquired sample dataset, including a limited number of sample data. Through the subsequent steps in the above embodiments of this application, high-quality training samples can be generated based on the limited number of sample data. It is understood that by reusing the methods described in steps S204 to S208, or by recombining one or more steps in S204 to S208, a large number of high-quality training samples can be obtained, thereby solving the technical problem of low-quality training samples obtained by existing methods.

[0084] In one optional approach, the aforementioned action description information set can be a sample dataset obtained from a publicly available dataset, including first object actions and second object actions corresponding to multiple virtual objects in multiple virtual scenes. In another optional approach, the aforementioned action description information set can be a sample dataset designed based on a specific game scene. For example, when the virtual object is specifically a game character, the aforementioned first object actions and second object actions can be a set of first object actions and second object actions designed for each of multiple game characters such as character a, character b, character c, etc. in game A. The embodiments of this application do not limit the method of obtaining the aforementioned action description information set.

[0085] In step S202 above, the action description information group may include first action description information for indicating the first object action and second action description information for the second object action. When the above embodiment is applied to a game application, the first object action and the second object action in step S202 may be two different object actions performed by a virtual game object displayed in a virtual game scene. Optionally, the first object action may be a starting object action, for example, an object action performed by the current virtual object in a static state or a state where no operation command has been received; the second object action may be a ending object action, for example, an object action performed by the current virtual object to indicate a hit state when attacked by other virtual objects.

[0086] The following details the information content that may be included in the first action description information used to indicate the action of the first object and the second action description information used to indicate the action of the second object in the above embodiments. In the above embodiments of this application, the first action description information and the second action description information may include state information of multiple object nodes used to drive the rendering of the target object, such as spatial position information, linear velocity information, and angular velocity information of the object nodes. The object nodes may be hierarchical object nodes included in a three-dimensional driving model used to render and display the target object in virtual space. The three-dimensional driving model may include a root node used to determine driving information according to control instructions, object nodes used to transmit driving information, and a set of skinned mesh objects for determining the spatial displacement of each point on the object surface of the target object according to the driving information. In the process of determining the spatial displacement of each skinned mesh according to the driving information, the spatial displacement of each skinned mesh object can be further determined by combining the skin weight information with the driving information.

[0087] The following combination Figure 5 An optional 3D driving model is described. For example... Figure 5As shown, the 3D driving model includes at least an object node set 504 and a skinned mesh 505. The object node set 504 receives control commands 502 and transmits and transforms driving information according to the connection relationships between object nodes. Multiple root nodes that can directly receive control commands can transmit control commands combined with weight information to the object nodes they are connected to. The skinned mesh 505 determines the displacement information of the target skinned mesh rendered on the target object based on the driving information output by node 1. Due to the aforementioned first action description information, the second action description information can include the relevant state information of the various object nodes, thereby displaying the object actions of the target object in the virtual scene based on the aforementioned information set.

[0088] In one specific implementation, the information included in the first and second motion description information may be the displacement information and quaternion rotation information corresponding to each of the multiple object nodes in the object model used to render the target object. Optionally, the aforementioned first and second motion description information may be a set of information determined based on a BVH (Biovision Hierarchy Data Format) file. In the aforementioned BVH file, motion data is stored based on a tree structure of virtual object joints. The file records the offset of human skeleton nodes (i.e., a specific object node) from their parent nodes, and the motion information consists of Euler angle rotation in the local parent coordinate system and displacement of the root bone (i.e., a specific root node).

[0089] Furthermore, having obtained the action description information and the action description information group included in step S202, one of the following steps can be performed before executing step S204:

[0090] Clustering Method 1: Determine the reference action description matrix corresponding to each of the multiple action description information groups in the action description information set; perform clustering operation on the multiple action description information groups according to the spatial distance relationship between the multiple reference action description matrices to obtain at least one subset of action description information;

[0091] In the first clustering method described above, multiple action description information groups can be clustered directly based on their respective reference action description matrices. In other words, in this method, clustering can be performed directly using action description information groups as units, where the reference action description matrix indicates the relevant features contained within an action description information group. Then, a clustering operation is performed based on the spatial distance relationships between the reference action description matrices to obtain a clustering result containing multiple subsets of action description information, where each subset is still considered an action description information group.

[0092] The following combination Figure 6 The first clustering method described above will be explained. For example... Figure 6 As shown, action description information set 602 contains multiple action description information groups, among which action description information group 603 is included. It can be understood that each of the multiple action description information groups includes first action description information and second action description information (identified by square and elliptical graphics, respectively). During clustering, each action description information group indicates the relevant features it contains based on its corresponding reference action description matrix.

[0093] The above clustering method one can cluster based on action description information groups. For example... Figure 6 As shown, after clustering the action description information set 602, multiple action description information subsets can be obtained, including action description information subset 604, action description information subset 606 and action description information subset 608. The units included in the above action description information subsets are still considered as action description information groups.

[0094] Clustering Method 2: Determine at least one subset of first action description information based on the first action description information included in each of the multiple action description information groups, and determine at least one subset of second action description information based on the second action description information included in each of the multiple action description information groups. The first subset of action description information includes multiple first action description information that satisfy the second clustering condition, the second subset of action description information includes multiple second action description information that satisfy the third clustering condition, and at least one subset of action description information includes both the first subset of action description information and the second subset of action description information.

[0095] In the second clustering method described above, the first and second action description information included in multiple action description information groups can be clustered separately. In other words, in this second clustering method, clustering can be performed using action description information as the unit. During the clustering process, clustering operations can be performed based on the spatial distance relationship between the information matrices corresponding to each of the aforementioned action description information, resulting in a clustering result containing multiple subsets of action description information. The units included in each subset of action description information are either the first or second action description information.

[0096] The following combination Figure 7 The second clustering method described above will be explained. For example... Figure 7 As shown, the action description information set 702 contains multiple action description information groups, among which action description information group 703 is included. It can be understood that each of the multiple action description information groups includes first action description information and second action description information (identified by a square graphic and an ellipse graphic, respectively).

[0097] The second clustering method described above can cluster based on action description information as units. In other words, the first action description information and the second action description information can be clustered separately. For example... Figure 7 As shown, after clustering the action description information in action description information set 702, multiple first action description information subsets and multiple second action description information subsets can be obtained. The multiple first action description information subsets include action description information subsets 704, 706, and 708; the multiple second action description information subsets include action description information subsets 710, 712, and 714. Through the above-mentioned clustering method two, the units included in the above multiple action description information subsets are updated to individual action description information.

[0098] Furthermore, if multiple subsets of action description information are obtained based on the above-mentioned clustering method one or clustering method two, step S204 as described above can be continued to obtain the first sample action information and the second sample action information from at least one subset of action description information. It is understood that the methods for obtaining the first sample action information and the second sample action information from at least one subset of action description information may differ depending on the clustering method.

[0099] For example, regarding the first clustering method described above, since the units included in the subset of action description information obtained by clustering constitute an action description information group, an action description information group can be directly obtained from one subset of action description information, and the first sample action information and the second sample action information can be determined based on the action description information group. For example, in... Figure 6 In this process, the first sample action information and the second sample action information can be determined based on one action description information group in the action description information subset 604.

[0100] For example, regarding the second clustering method mentioned above, since the action description information subsets obtained from clustering include two types: a first action description information subset and a second action description information subset, each unit in the first action description information subset is a first action description information, and each unit in the second action description information subset is a second action description information. Therefore, a first sample action information can be obtained from a first action description information subset, and a second sample action information can be obtained from a second action description information subset. For example, in... Figure 7 In this process, a first action description information can be obtained based on the action description information subset 704, and a second sample action information can be obtained from the action description information subset 714.

[0101] Furthermore, after obtaining the first sample action information and the second sample action information through the above method, the action acquisition operation can be further performed based on the above step S206 to obtain multi-frame transition sample action information for indicating the action of multi-frame transition objects.

[0102] In an optional implementation, the aforementioned transition sample action information can be action information determined based on the transition action design draft. Specifically, the aforementioned transition action design draft can be the action design result obtained based on the art design operation.

[0103] In another alternative implementation, the aforementioned transitional sample motion information can also be obtained based on 3D video information output by a generative video model. For example, the first frame corresponding to the first object action indicated by the first sample motion information and the second frame corresponding to the second object action indicated by the second sample motion information can be input into the generative video model. The generative video model can then be prompted by relevant prompts to output a target video generated based on the first and second frames. Subsequently, the transitional sample motion information can be extracted from multiple intermediate frames included in the target video.

[0104] In another optional implementation, the aforementioned transitional sample motion information can also be motion information obtained based on motion capture technology. Correspondingly, in this implementation, the motion acquisition operation can specifically be a motion capture operation. It should be noted that motion capture technology can capture the performer's movements and map these movement data onto virtual objects, thereby determining the aforementioned multi-frame transitional sample motion information based on the motion description information of the virtual objects. In other words, in this implementation, while the performer demonstrates a smooth transition between the first object's movement and the second object's movement according to actual motion performance operations, relevant motion information of the performer can be collected to obtain the aforementioned multi-frame transitional sample motion information.

[0105] Having obtained multi-frame transition sample information through the above step S206, the target training sample is further obtained by combining the above first sample action information and the second sample action information as described in step S208.

[0106] In an optional implementation, if the target training samples are obtained based on the above steps S202 to S208, data augmentation operations can be performed based on the target training samples to obtain a larger number of training samples.

[0107] The following combination Figure 8 The main steps involved in a complete embodiment of this application are described below. A complete embodiment of this application may include the following main steps:

[0108] S800, export game resources;

[0109] First, by exporting game resources in step S800, an initial dataset, namely the action description information set, can be obtained. This step provides the foundational data for subsequent data processing and analysis.

[0110] S802, Data Analysis and Filtering;

[0111] The above steps mainly involve clustering operations to conduct in-depth analysis of the initial dataset and to effectively filter the data through clustering methods in order to identify and extract key action features.

[0112] S804, video motion capture;

[0113] The above steps capture the process of the performer executing transitional movements, thereby determining the sequence of transitional movements in the training samples and providing real motion sampling data for model training.

[0114] S806, enhanced motion data;

[0115] By implementing relevant data augmentation operations, a larger number of high-quality training samples can be generated, thereby enriching the training dataset and improving the model's generalization ability.

[0116] S808, model training.

[0117] Finally, the transition action prediction model is trained using the data acquired and processed in the previous steps, enabling it to accurately predict and generate transition actions.

[0118] The embodiments described above in this application provide a systematic method for processing and utilizing motion data to generate high-quality transition animations for applications such as 3D fighting games. This method not only improves the efficiency and quality of motion generation but also enhances the predictive ability and generalization performance of the model through data augmentation and model training.

[0119] Specifically, in the sample generation method based on the above clustering method one, by performing clustering operations on action information groups, similar combinations of first action description information and second action description information are grouped into one category. Thus, an action description information group can be obtained from a cluster to represent the combination of first action description information and second action description information of that category. By performing subsequent action acquisition operations through the obtained action description information group, the acquired samples can demonstrate high generalization, without having to repeatedly perform action acquisition operations on other action description information groups in the same category. Thus, while ensuring the quality of training samples, the sample acquisition efficiency can be significantly improved.

[0120] For example, in clustering results such as Figure 6 As shown, an action description information group can be obtained from action description information subsets 604, 606, and 608 respectively to perform subsequent action acquisition operations, thereby obtaining three high-quality target training samples and improving the acquisition efficiency of training samples.

[0121] In the sample generation method based on the above-mentioned clustering method two, by performing clustering operations on the first action description information and the second action description information respectively, a first object action representation of the first action cluster can be obtained from a first object action cluster, and a second object action representation of the second action cluster can be obtained from a second object action cluster. On the one hand, it is not necessary to obtain other object actions in the same action cluster to perform subsequent action collection operations, thus improving the sample collection efficiency; on the other hand, when the first object action and the second object action are clustered separately, the richness of the samples can be improved by permuting and combining different first object action clusters and second object action clusters, thereby increasing the number of high-quality training samples.

[0122] For example, in clustering results such as Figure 7 In the scenario shown, a first sample action information can be obtained from action description information subsets 704, 706, and 708 respectively; and a second sample action information can be obtained from action description information subsets 710, 712, and 714 respectively. Then, based on the three first sample action information and the three second sample action information, nine sample action information groups are obtained. Subsequent action acquisition operations are performed based on the nine sample action information groups, resulting in nine high-quality target training samples. This improves the efficiency of training sample acquisition and further enriches the number of training samples.

[0123] The following are Figure 8 Further details on the implementation of each step involved will be provided.

[0124] In the clustering process, taking the above-mentioned clustering method one as an example, the method for determining at least one subset of action description information is explained. First, the reference action description matrix corresponding to each of the multiple action description information groups in the action description information set is determined, including:

[0125] S1, obtain the first action description matrix corresponding to the first action description information in the current action description information group, and the second action description matrix corresponding to the second action description information in the current action description information group;

[0126] S2, determine the action difference matrix based on the first action description matrix and the second action description matrix, and determine the third action description matrix based on the action difference matrix and the first action description matrix, wherein the first action description matrix is ​​used to indicate the first object action performed by the virtual object at the first moment, and the second action description matrix is ​​used to indicate the second object action performed by the virtual object at the second moment, which is later than the first moment.

[0127] S3, the projection matrix obtained by spatially projecting the third action description matrix onto the target vector space is determined as the reference action description matrix, wherein the first matrix dimension of the reference action description matrix is ​​lower than the second matrix dimension of the third action description matrix.

[0128] It is understood that, in the embodiments of this application, since the matrices corresponding to the first action description information and the second action description information are both high-dimensional matrices, in the above embodiments, the principal component analysis method can be used to perform dimensionality reduction processing on the third action description matrix used to characterize the information features of the current action description information group to obtain the above reference action description matrix.

[0129] The following provides a detailed explanation of the dimensionality reduction method described above, which may include the following steps:

[0130] S1, principal component learning of samples;

[0131] Specifically, when the action description information set includes n1 action description information groups, n1 sample matrices can be determined based on the bias result between the first action description matrix of the first action description information corresponding to each of the n1 action description information groups, and the second action description matrix of the second action description information, i.e., the third action description matrix. Each sample is represented by x. i The entire sample set can then be represented as:

[0132]

[0133] First, the entire sample set is subjected to mean removal. The mean of the sample set is defined as follows:

[0134]

[0135] The result after removing the mean from the entire sample set is as follows:

[0136]

[0137] right By performing the SVD decomposition in mathematics, we can obtain:

[0138]

[0139] where U P and V P are two orthogonal matrices, and Σ P is a diagonal matrix, with singular values on the diagonal arranged in descending order;

[0140] assume:

[0141] Σ P =diag(σ1,σ2,…,σ m )

[0142] acquire a principal component coefficient a, where 0<a<1, which is used to indicate the feature proportion of the spatial feature indicated by the principal component matrix in the original feature space, and may generally be selected as 0.8 or 0.9. Then, find such a positive integer L p that makes both of the following inequalities hold:

[0143]

[0144] It can be understood that in the above inequality, after arranging the singular values in Σ P in descending order by size, the minimum number of singular values is found such that the sum of squares occupied by these singular values is a times the sum of squares of all singular values. Since the value of a is usually close to 1, that is, this part of the singular values can approximately replace all the singular values.

[0145] Further, the matrix dimension of the principal component matrix that can represent the original sample space can be determined correspondingly through this value, that is:

[0146] =the first L P columns of U p

[0147] sample projection calculation;

[0148] For the third action description matrix corresponding to each action description information group, its projection in the sample principal component space is calculated based on the above principal component analysis result. The calculation method for the projection of the sample x i in the positive sample principal component space is as follows: firstly subtract the mean of the sample space, and then multiply by the transpose of the sample principal component space matrix, that is:

[0149]

[0150] Through the above embodiments of the present application, dimensionality reduction processing can be performed on high-dimensional action description information while retaining main information features, thereby significantly reducing the amount of calculation in the subsequent clustering process and improving data processing efficiency.

[0151] ​Furthermore, after performing dimensionality reduction and obtaining the reference action description matrix corresponding to each action description information group, the method based on the reference action description matrix can be a clustering operation performed when the number of clusters is predetermined. When the number of clusters is determined, it can be a hard clustering method (i.e., a sample is only allowed to belong to one cluster) or a soft clustering method (i.e., a sample is allowed to belong to multiple clusters). Optionally, when the number of clusters cannot be predetermined, clusters can be formed based on the sample density.

[0152] In an optional implementation, given a predetermined number of clusters, the above-mentioned clustering method one performs a clustering operation on multiple action description information groups based on the spatial distance relationship between multiple reference action description matrices to obtain at least one subset of action description information, including:

[0153] S1, obtain multiple reference action description matrices and multiple current centroid matrices;

[0154] S2, based on multiple reference action description matrices and multiple current centroid matrices, determine the clustering distance information corresponding to each of the multiple reference action description matrices, wherein the clustering distance information includes multiple matrix distances between the multiple current centroid matrices and the reference action description matrices;

[0155] In the above steps, given the current centroid matrix, multiple matrix distances can be determined based on the multiple current centroid matrices and the reference action description matrix. Then, based on the matrix distance threshold, the reference action description matrix included in the cluster corresponding to each centroid matrix can be updated.

[0156] S3, based on the clustering distance information corresponding to each of the multiple reference action description matrices, update the clustering label matched by each reference action description matrix, where the clustering label is used for the action description information subset to which the action description information group currently belongs;

[0157] Understandably, in the current step, the cluster to which each reference action description matrix belongs can be re-determined by combining the distance threshold and the matrix distance between each reference action description matrix and each current centroid matrix, and then the corresponding cluster labels can be updated.

[0158] S4. Update the multiple current centroid matrices according to the reference action description matrices matched by each current centroid matrix to obtain the updated multiple current centroid matrices.

[0159] S4-1, If ​​multiple current centroid matrices satisfy the first clustering condition, determine at least one subset of action description information based on the clustering labels matched by each reference action description matrix.

[0160] S4-2, If multiple current centroid matrices do not satisfy the first clustering condition, repeat steps S1 to S4 above.

[0161] Optionally, the first clustering condition mentioned above may include, but is not limited to, one of the following: the number of iterations of the current centroid matrix is ​​greater than or equal to the target number; the spatial distance between the current centroid matrix and the historical centroid matrix corresponding to the previous round is less than or equal to the target threshold.

[0162] Through the above-described embodiments of this application, by calculating the spatial distance between the reference action description matrix and the current centroid matrix, the information group cluster to which each action description information group belongs can be accurately determined; at the same time, the clustering labels corresponding to each action description information group and the centroid matrix corresponding to each information group cluster will be dynamically updated according to the latest clustering distance information, thereby continuously approaching the optimal clustering result through the clustering labels, improving the flexibility and adaptability of clustering.

[0163] It should be noted that the implementation of the above-described clustering method two can also refer to the above implementation method. First, perform dimensionality reduction processing on the matrix used to represent action description information, and then perform clustering operations on the first action description information and the second action description information respectively based on the above implementation method. The specific implementation method can be implemented with reference to the above implementation method, and will not be repeated here.

[0164] In one specific embodiment of this application, the action combinations between the actions of the first object and the actions of the second object can be classified based on data feature analysis. Then, clustering is performed in the feature space to group actions with similar features into the same group. During action acquisition, sampling a certain number of actions within different groups allows for more efficient coverage of more similar actions with a smaller number of actions. The following is combined with... Figure 9 The specific implementation process of the above clustering method one will be explained.

[0165] S900, Euler angle conversion 6D representation;

[0166] S902, Calculate the bias of the first object action and the second object action;

[0167] In this embodiment, a bias operation is performed on the first and second object actions of the character. Specifically, this can be achieved by subtracting the action description information corresponding to the first and second object actions, and then summing the difference with the first object action to determine the bias result. The lower body nodes include nodes 1, 2, 3, 4, 5, 6, 7, and 8, while the remaining nodes are upper body nodes, such as... Figure 10 As shown.

[0168] When the first object action is a standby action and the second object action is a hit action, the Euler angle data (α,β,γ) in the action is first converted into a six-dimensional two-axis rotation matrix vector V based on the standby actions and hit action biases of all characters. 6D This represents a three-dimensional vector that includes an upward and a forward vector:

[0169] V 6D =(R 11 ,R 12 ,R 13 ,R 21 ,R 22 ,R 23 )

[0170] in,

[0171]

[0172] This embodiment can use a skeleton model with 22 joints, so the dimensions of the upper and lower body pose models are 84 (14*6) and 54 (9*6) respectively (both the upper and lower body include the hip joint).

[0173] S904 performs dimensionality reduction on the bias result;

[0174] During the dimensionality reduction process, principal component analysis can be performed first, followed by dimensionality reduction. Specifically, the data can be reduced to three dimensions for visualization. For details on the implementation, please refer to the above embodiments.

[0175] S906, Clustering Processing;

[0176] S908, classify data based on clustering results;

[0177] In this embodiment, the K-means clustering method can be used to classify the action combinations of different roles. Figure 11 The diagram illustrates a classification result in a feature space, where different 3D points represent the positions of different human poses after dimensionality reduction in the feature space, and different filling methods are used to indicate different clustering results.

[0178] Understandably, during the data acquisition phase, motion capture operations are performed on the transitional movements of some characters based on the above classification in order to obtain target sample data.

[0179] Through the above-described embodiments of this application, in the sample generation method based on the above-described clustering method one, by performing clustering operations on action information groups, similar combinations of first action description information and second action description information are grouped into one category. Thus, an action description information group can be obtained from a cluster to represent the combination of first action description information and second action description information of that category. By performing subsequent action acquisition operations through the obtained action description information group, the acquired samples can exhibit high generalization, without having to repeatedly perform action acquisition operations on other action description information groups in the same category. Thus, while ensuring the quality of training samples, the sample acquisition efficiency can be significantly improved.

[0180] The following are Figure 8 Step S804 in the document further explains the implementation of video motion capture. For example... Figure 12 As shown in the embodiments of this application, the implementation process of the above-mentioned video motion capture may include three steps: exporting motion keyframe assets from the game engine, importing the motion keyframe assets into the motion capture software, and acquiring the motion capture data of the actor object. The step of exporting motion keyframe assets from the game engine and importing the motion keyframe assets into the motion capture software corresponds to S1 in the above embodiments, and the step of acquiring the motion capture data of the actor object corresponds to steps S2 and S3. The following provides further explanation of each of the three steps.

[0181] In one optional implementation, the above-mentioned action acquisition operation based on the first sample action information and the second sample action information to obtain multi-frame transition sample action information includes:

[0182] S1, based on the first sample action information, display the first virtual object performing the first object action, and based on the second sample action information, display the second virtual object performing the second object action, wherein the first virtual object is displayed according to the first style, and the second virtual object is displayed according to the second style;

[0183] S2, in response to the action acquisition operation, based on the action acquisition device acquiring the current sampled action information, displays the third virtual object to perform the action of the current sampled object, wherein the third virtual object is displayed according to the third style;

[0184] S3, if the collected multi-frame current sampling action information meets the collection conditions, determine the multi-frame transition sample action information that matches the first sample action information and the second sample action information based on the multi-frame current sampling action information.

[0185] In the above steps, two virtual objects performing different actions (the first object's action and the second object's action) can be displayed in the display interface according to preset styles. This makes it easier for the actor object to determine the currently aligned action version based on different display styles during the performance of transition actions. This improves the matching degree between the actions performed by the actor object and the actions indicated by the action description information, thereby ensuring that the acquisition of transition action sequences is completed more efficiently based on the actor object.

[0186] The first style and the second style mentioned above can be different color styles. For example, the first style is red and the second style is blue. The current action of the red virtual object indicates the action of the first object, and the current action of the blue virtual object indicates the action of the second object.

[0187] For example, the first style and the second style can also be a dashed stroke style and a solid stroke style. Specifically, the first object action of a virtual object with a dashed stroke can indicate that the first object action is the starting action; the second object action of a virtual object with a solid stroke can indicate that the second object action is the ending action.

[0188] For example, the first style and the second style can also be a transparent display style and an opaque display style. Specifically, the first object action of the virtual object with the transparent display style can indicate that the first object action is the starting action; the second object action of the virtual object with the opaque display style can indicate that the second object action is the ending action.

[0189] Furthermore, during motion capture operations, in response to motion acquisition, a third virtual object can be displayed. This third virtual object performs the action of the currently sampled object and is displayed according to a third style. It is understood that the action display of the third virtual object is a simulation or mapping of the user's actual actions, and the third style may differ from the first and second styles to distinguish actions acquired in real time.

[0190] Furthermore, the acquisition conditions satisfied in S3 above can be either the first type of conditions used to indicate the continuity of the transition action, or the second type of conditions used to indicate the stability of the transition action.

[0191] Through the above-described embodiments of this application, a highly efficient and intuitive motion capture and matching method is achieved by displaying virtual objects of different styles in a virtual environment to perform specific actions. In step S1, based on preset first and second sample motion information, the actions of two virtual objects are displayed in first and second styles respectively. This visual distinction helps the motion capture actor clearly identify and understand the characteristics of each sample action. Next, in step S2, in response to the motion capture operation, the motion capture device captures the motion information of the motion capture actor in real time, and displays the current sampled action performed by the third virtual object in a third style. This not only provides the motion capture actor with immediate motion feedback, but also facilitates the motion capture actor to compare its own actions with sample actions. Finally, in step S3, when the acquired multi-frame motion information meets preset conditions, the system can accurately identify transitional motion information that matches the sample actions from the acquired actions, thereby significantly improving the efficiency and quality of motion capture and processing.

[0192] The following section further explains the process of exporting motion keyframe assets from a game engine and importing those assets into motion capture software.

[0193] In one optional implementation, displaying a first virtual object performing a first object action based on first sample action information, and displaying a second virtual object performing a second object action based on second sample action information, includes:

[0194] S1, in the target rendering engine, the first sample object action executed by the reference virtual object is rendered according to the first sample action information, and the second sample object action executed by the reference virtual object is rendered according to the second sample action information at the same virtual position.

[0195] S2, based on the actions of the first sample object and the second sample object, sample the first reference action information and the second reference action information respectively;

[0196] S3, initialize the root bone position information and rotation information in the first reference motion information and the second reference motion information to obtain the updated first reference motion information and the second reference motion information.

[0197] S4, in the motion acquisition application, the first virtual object is displayed to perform the first object action according to the updated first reference motion information, and the second virtual object is displayed to perform the second object action according to the second reference motion information.

[0198] The following combination Figure 13The above implementation method will be described. For each transition action to be collected, the object action of two frames, namely the start frame and the end frame, needs to be exported. The object action of the start frame can be the object action corresponding to the initial time, and the object action of the end frame can be the object action corresponding to the end time, that is, they correspond to the first sample action information and the second sample action information, respectively.

[0199] Since directly using fixed art assets may not accurately reflect the real game performance—for example, knockback animations created by the art team may not include displacement data—in the embodiments described above, the motion assets are all exported from the target rendering engine. This allows the program to control the relevant data during game runtime, making the displacement of the motion capture data closer to the display effect in the virtual scene, thereby improving the training results of the transition motion prediction model.

[0200] Specifically, such as Figure 13 As shown, when the first object movement is a hit action (i.e., receiving an attack command) and the second object movement is a standby action (i.e., not receiving a control operation), as in the first virtual scene 1302, the hit-ending frame object movement can be selected as the current hit action. Then, in the second virtual scene 1303, the standby action of the character is sampled based on the game engine-rendered image of the current character position. Next, the root bone positions and rotations of these two action poses are cleared to zero and redirected to a general character skeleton. Finally, the image is exported as an FBX format file, as shown in the action description information group 1304.

[0201] The following section further explains how to obtain motion capture data for actor objects.

[0202] In one optional implementation, after displaying the first virtual object performing a first object action based on the first sample action information and displaying the second virtual object performing a second object action based on the second sample action information, the method further includes at least one of the following:

[0203] The first adjustment method, in response to the first angle adjustment operation, displays the first virtual object and performs the first object action at the adjusted first display angle;

[0204] The second adjustment method, in response to the second angle adjustment operation, displays the second virtual object and performs the second object action at the adjusted second display angle;

[0205] The third adjustment method, in response to the third angle adjustment operation, displays the third virtual object performing the current sampling object action at the adjusted third display angle.

[0206] It is understood that, in the above embodiments, during the motion capture data acquisition process, the display angles of the first virtual object, the second virtual object, and the third virtual object can be adjusted respectively in response to different adjustment methods, so as to facilitate the alignment of the motion capture actor's movements with the movements of different objects during the performance.

[0207] For example, during the motion capture data acquisition process, the exported motion pose data can be imported into the motion capture software. These poses will retain the original position and rotation information and can be displayed from four different perspectives (front, back, left, and right) so that motion capture actors can observe and align their movements from multiple angles.

[0208] Furthermore, to help motion capture actors distinguish between the start and end frames of their poses, different poses can be marked with different colors and different fill methods. Assuming... Figure 14 The first virtual object 1404 is a virtual object marked according to the first filling method, the second virtual object 1406 is a virtual object marked according to the second filling method, and the third virtual object 1406 is a virtual object marked according to the third filling method. The first virtual object 1404 is used to indicate the starting action, i.e., the first object action; the third virtual object 1408 is used to indicate the ending action, i.e., the second object action; and the second virtual object 1406 is used to indicate the current object action performed by the actor object 1402.

[0209] When actors actually perform transitional movements, they can start from the pre-placed markers, assume the starting frame pose, and then naturally transition to the ending frame pose. By displaying different object movements according to different display styles, and providing a flexible method to adjust the display angle, the motion alignment process during motion capture is greatly simplified, and the accuracy is significantly improved.

[0210] In an optional implementation, before determining the multi-frame transition sample action information that matches the first sample action information and the second sample action information based on the multi-frame current sampling action information, the method further includes: determining that the multi-frame current sampling action information meets the acquisition conditions when the first matching degree between the starting sampling action information in the multi-frame current sampling action information and the first sample action information is greater than or equal to a first threshold, and the second matching degree between the ending sampling action information in the multi-frame current sampling action information and the second sample action information is greater than or equal to a second threshold.

[0211] In the above embodiments, a method is provided for determining whether multi-frame motion information meets specific acquisition conditions during motion capture and analysis. The multi-frame currently sampled motion information is used to indicate a series of motion data continuously acquired during motion capture.

[0212] By defining the acquisition conditions as described above, motion data highly similar to preset sample actions can be automatically filtered out, thereby improving the accuracy and reliability of motion capture. Specifically, by setting a matching threshold, the acquired motion information is filtered and verified to ensure that only motion data highly matching the preset sample actions is used. This method can improve the accuracy of motion capture, reduce the impact of erroneous data, and thus improve the quality of the acquired motion capture samples.

[0213] like Figure 15 The diagram illustrates a possible motion capture scenario setup, including at least two devices: a motion capture device and a motion preview device. It also provides the actor's range of motion for motion capture. The motion capture device may include multiple high-speed cameras that capture markers on the actor from different angles, recording their movement trajectories to capture the actor's actions within their range of motion. The motion preview device can be used to display or play back the captured motion data in real time, allowing the motion-captured actor to instantly check the effectiveness of the motion capture and ensure the accuracy and quality of the motion data.

[0214] In one alternative approach, during motion capture, the actor performs actions within a pre-defined range of motion, with markers on their body captured by multiple motion capture devices. The captured motion data is recorded by a script supervisor and displayed and reviewed in real-time on a motion preview device. The actor can also observe the content displayed on the motion preview device in real-time and adjust their movements accordingly. Ultimately, this data is used to generate high-quality motion assets, serving as an important component of the target training samples.

[0215] The following are Figure 8 Step S806 in the document will be explained in detail, further illustrating the implementation of motion data augmentation.

[0216] In one optional implementation, after determining the target training sample based on the first sample action information, the second sample action information, and the multi-frame transition sample action information, the method further includes at least one of the following:

[0217] Enhancement Method 1: Obtain the sequence of sample action information from the current training samples, which include multiple target training samples. The sequence of sample action information includes the first sample action information, the second sample action information, and the multi-frame transition sample action information.

[0218] Perform mirror transformation operation on multiple sample action information in the sample action information sequence to obtain the first sample action information sequence;

[0219] The action information sequence of the first sample is used as the new target training sample.

[0220] It is understandable that in the above enhancement method one, a mirror transformation operation can be used on the already generated target training samples to obtain new training samples, which can then be added to the sample set.

[0221] Enhancement Method 2: Obtain sample action information sequence from the current training sample including multiple target training samples, wherein the first sample action sequence includes first sample action information, second sample action information and multi-frame transition sample action information;

[0222] A velocity transformation operation is performed on the sample action information sequence to obtain a second sample action information sequence;

[0223] The action information sequence of the second sample is used as the new target training sample.

[0224] Understandably, in the above-mentioned enhancement method two, a velocity transformation operation can be applied to the already generated target training samples to obtain new training samples, which can then be added to the sample set.

[0225] Enhancement Method 3: Obtain sample action information sequence from the current training samples including multiple target training samples, wherein the first sample action sequence includes first sample action information obtained from the first action description subset, second sample action information obtained from the second action description subset, and multi-frame transition sample action information.

[0226] Obtain first reference sample action information from the first action description subset, and obtain second reference sample action information from the second action description subset;

[0227] The action information of the first reference sample, the action information of the second reference sample, and the action information of the multi-frame transition sample are used as new target training samples.

[0228] It should be noted that the above three enhancement methods can be used in combination. For example, when the first training sample is obtained based on enhancement method one, enhancement method two can also be executed based on the first training sample to obtain the second training sample. The embodiments of this application do not limit the specific combination of the above three enhancement methods.

[0229] It is understandable that in the above enhancement method three, when the target training sample is obtained according to clustering method two, a new training sample can be obtained by combining the collected transition action sequence and the clustering results of the first object action and the second object action.

[0230] The following combination Figure 16 This paper describes a data augmentation method based on clustering method two. Figure 16As shown, the first action description information subset 1602 includes multiple first action description information obtained from clustering operations, and the second action description information subset 1604 includes multiple second action description information obtained from clustering operations. When a motion capture operation is performed based on the first sample action information a1 in the first action description information subset 1602 and the second sample action information b1 in the second action description information subset 1604 to obtain a transitional action sequence 1606, the target training sample can be determined based on the first sample action information a1, the second sample action information b1 in the second action description information subset 1604, and the transitional action sequence 1606.

[0231] Furthermore, the newly added target training sample can be determined based on the first sample action information a2 in the first action description information subset 1602 and the second sample action information b1 in the second action description information subset 1604, combined with the aforementioned transition action sequence 1606.

[0232] Furthermore, the newly added target training sample can be determined based on the first sample action information a3 in the first action description information subset 1602 and the second sample action information b2 in the second action description information subset 1604, combined with the aforementioned transition action sequence 1606.

[0233] Understandably, in the second clustering method described above, since the first object action and the second object action can be clustered separately, the similarity between the first object actions in the same cluster can be higher than a certain threshold. Based on this, in the obtained target training samples, the first object action and the second object action can be replaced by the first object action or the second object action in the same cluster, thereby quickly obtaining new samples and improving the efficiency of training sample generation.

[0234] In the above enhancement method one, a mirror transformation operation is performed on multiple sample action information in the sample action information sequence to obtain the first sample action information sequence, including:

[0235] S1, Obtain the current sample action information from multiple sample action information;

[0236] S2, perform the first mirror symmetry operation on the target rotation information of the first part included in the current sample action information;

[0237] S3, perform the second mirror symmetry operation on the three-dimensional coordinate information of the second part included in the current sample action information;

[0238] S4, the updated current sample action information is determined as the first sample action information included in the first sample action information sequence.

[0239] In the first enhancement method described above, to ensure that the enhanced movement does not disrupt the rhythm of the original movement, mirror-symmetric operations can be performed on the left and right lateral joints (first part) and the hip joint (second part). Specifically, it can be done as follows: Figure 17 As shown, first, the character's frontal orientation in the starting frame is adjusted to the positive y-axis. Then, based on the storage method of BVH data, the rotation of the left and right joints and the 3D coordinates of the hip joint are mirrored symmetrically along the yz plane to obtain the mirrored motion, as shown. Figure 17 In the process, after performing mirroring operations on each object action in the first action sequence 1702 in the manner described above, the newly added training sample, namely the second action sequence 1704, can be obtained.

[0240] In the above enhancement method two, a velocity transformation operation is performed on the sample action information sequence to obtain a second sample action information sequence, including:

[0241] S1, Obtain the velocity description information corresponding to the velocity transformation operation;

[0242] Optionally, the speed description information described above can be described by an acceleration or deceleration factor δ.

[0243] S2, determine the number of second sample action information included in the second sample action information sequence and the sequence identifier corresponding to each of the multiple sample action information in the sample action information sequence based on the speed description information;

[0244] S3, determine the weight information between every two adjacent second sample action information based on the velocity description information;

[0245] S4. Based on the sequence identifiers corresponding to each of the multiple sample action information and the weight information between every two adjacent second sample action information, interpolation processing is performed on the multiple sample action information to obtain the second sample action information sequence.

[0246] Since the animation playback speed is the same, either 30 frames per second or 60 frames per second, in the above implementation, interpolation can be used to reduce / increase the total number of frames of the original animation while ensuring the consistency of the animation performance, thereby achieving acceleration / deceleration effects.

[0247] In this embodiment, an acceleration or deceleration factor δ is first defined according to requirements. Then, the number of frames after acceleration / deceleration is determined by division based on the acceleration / deceleration factor, and the sequence number i of each new frame after interpolation is determined accordingly. inter (corresponding to the original sequence number), and the interpolation weight corresponding to each newly generated frame.

[0248]

[0249] Where i represents the original frame number, i inter Indicates the frame number after acceleration / deceleration. Indicates the i-th value after interpolation inter Frame and i inter +1 frame weighting coefficient This indicates rounding down to the nearest integer.

[0250] Furthermore, based on the sequential identifiers corresponding to each of the multiple sample action information and the weight information between every two adjacent second sample action information, the above-mentioned interpolation processing is performed on the multiple sample action information to obtain the second sample action information sequence, including:

[0251] S1, Perform coordinate transformation on multiple sample action information to obtain global action information of multiple samples in the global coordinate system;

[0252] S2, perform linear interpolation based on the global position information in the global action information of multiple samples to obtain multiple global position information corresponding to the second sample action information sequence;

[0253] S3, perform spherical linear interpolation based on the global rotation information in the global motion information of multiple samples to obtain multiple global rotation information corresponding to the second sample motion information sequence;

[0254] S4. Determine the second sample action information sequence based on multiple global position information and multiple global rotation information.

[0255] In this embodiment of the application, each frame is determined by the root node position and the local joint rotation R. local Composition. Since interpolation calculations need to be performed in the global coordinate system, it is necessary to first use FK (Forward Kinematics) to convert the original joint rotation data R, stored in local coordinates, into a single coordinate system. local Convert to global rotation R global and use R global Perform interpolation.

[0256] Wherein, the Root position (P) root The global rotation R can be obtained using linear interpolation. global The Slerp method, a spherical linear interpolation technique, is used to obtain the result. The specific calculation process is as follows:

[0257]

[0258] Where p and q are quaternion representations, and t is the weighting coefficient. This is the root position after interpolation. This is the rotation after interpolation.

[0259] Through the above-described embodiments of this application, by adjusting the magnitude of different acceleration / deceleration factors δ, action sequences with different rhythms can be generated.

[0260] The following combination Figure 18 A complete implementation of this application will be described. S1800, Obtain the action description information set; then, perform a judgment, S1802, whether to adopt clustering method one; if it is determined that clustering method one will be adopted, firstly, according to... Figure 6 The clustering method shown clusters a group of action description information, and then executes S1804 to obtain the first sample action information and the second sample action information from the action description information group.

[0261] If clustering method one is not executed, i.e., clustering method two is executed, firstly based on Figure 7 The second clustering method shown clusters the action description information to obtain multiple subsets of first action descriptions and multiple subsets of second action description information. Then, S1804-1 is executed to obtain the first sample action information and the second sample action information from the two clusters respectively.

[0262] Next, step S1806 is executed to acquire multi-frame transition motion sample information through motion capture. In this embodiment of the application, the above-mentioned acquisition of multi-frame transition motion sample information through motion capture may include three steps: exporting motion keyframe assets from the game engine, importing the motion keyframe assets into motion capture software, and acquiring motion capture data of the actor object.

[0263] The first number of target training samples can be obtained through the above step S1806.

[0264] Next, execute judgment S1808 to determine whether clustering method one was used. If clustering method one was used, execute S1810 to perform data augmentation on the training samples using data augmentation method one and data augmentation method two; if clustering method one was not used, i.e., clustering method two was used, execute S1810-1 to perform data augmentation on the training samples using data augmentation method one, data augmentation method two, and data augmentation method three.

[0265] In the first enhancement method, a mirror transformation operation can be applied to the already generated target training samples to obtain new training samples, which are then added to the sample set. In the second enhancement method, a velocity transformation operation can be applied to the already generated target training samples to obtain new training samples, which are then added to the sample set. In the third enhancement method, when the target training samples are obtained according to the second clustering method, new training samples can be obtained by combining the collected transition action sequences and the clustering results of the first object action and the second object action.

[0266] S1812, Determine the target sample set. Through the above implementation method, a second number of target training samples can be obtained, wherein the second number is greater than the first number.

[0267] After obtaining the target sample set, the following set Figure 19 A training method for an action prediction model is described. During model training, a complete action segment is randomly sampled from the dataset. The beginning and end of the action are selected as the first action description information corresponding to the start frame and the second action description information corresponding to the end frame, respectively. These are used as inputs to the transition action prediction model. The model is optimized by performing difference learning between the model's prediction results and existing high-quality transition actions in the dataset.

[0268] like Figure 19 First, a complete action sequence is randomly sampled from the original dataset. Then, the beginning and end of the action are selected as the first action description information for the start frame and the second action description information for the end frame, respectively. These two frames serve as input to the AI ​​model, training it to understand and predict action transitions. Next, the first and second action description information are input into the transition action prediction model. The model learns the action transition between these two frames to predict the entire transition action. During model training, the model's predictions are compared with existing high-quality transition actions in the dataset to optimize the model through difference learning.

[0269] The whole process is as follows Figure 19 As shown, through continuous data sampling, model training, and optimization learning, a transition motion prediction model that can accurately predict motion transitions is finally trained. This method not only improves the efficiency of motion generation but also ensures the naturalness and realism of the motion.

[0270] The above-described embodiments of this application can reduce the amount of data required to construct a dataset for transition actions in fighting games, improve the motion capture efficiency of transition actions in 3D fighting games, thereby saving human resources and time costs. At the same time, it can ensure the high quality of model training results and is more suitable for training specific tasks than public datasets.

[0271] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0272] According to another aspect of the present invention, an apparatus for generating training samples for implementing the above-described method for generating training samples is also provided. For example... Figure 20 As shown, the device includes:

[0273] The first acquisition unit 2002 is used to acquire an action description information set, wherein the action description information set includes multiple action description information groups, and the action description information groups include first action description information for indicating the action of a first object and second action description information for indicating the action of a second object.

[0274] The second acquisition unit 2004 is used to acquire first sample action information and second sample action information from at least one subset of action description information when performing clustering operation based on the above action description information set to determine at least one subset of action description information.

[0275] The acquisition unit 2006 is used to perform an action acquisition operation based on the first sample action information and the second sample action information to obtain multiple frames of transition sample action information, wherein the multiple frames of the transition sample action information are used to indicate multiple frames of transition object actions that match the first object action and the second object action.

[0276] The determining unit 2008 is used to determine a target training sample based on the first sample action information, the second sample action information and the multiple frames of transition sample action information, wherein the target training sample is used to train the transition action prediction model.

[0277] Optionally, the above-mentioned training sample generation device further includes one of the following: a first clustering unit, configured to determine a reference action description matrix corresponding to each of the plurality of action description information groups in the action description information set; and to perform a clustering operation on the plurality of action description information groups according to the spatial distance relationship between the plurality of reference action description matrices to obtain at least one subset of the action description information; a second clustering unit, configured to determine at least one first action description information subset based on the first action description information included in each of the plurality of action description information groups, and to determine at least one second action description information subset based on the second action description information included in each of the plurality of action description information groups, wherein the first action description information subset includes a plurality of first action description information that satisfies a second clustering condition, the second action description information subset includes a plurality of second action description information that satisfies a third clustering condition, and at least one subset of the action description information includes the first action description information subset and the second action description information subset.

[0278] Optionally, the first clustering unit is configured to: obtain a first action description matrix corresponding to the first action description information in the current action description information group, and a second action description matrix corresponding to the second action description information in the current action description information group; determine an action difference matrix based on the first action description matrix and the second action description matrix, and determine a third action description matrix based on the action difference matrix and the first action description matrix, wherein the first action description matrix is ​​used to indicate the first object action performed by the virtual object at a first moment, and the second action description matrix is ​​used to indicate the second object action performed by the virtual object at a second moment, the second moment being later than the first moment; and determine the projection matrix obtained by spatially projecting the third action description matrix into the target vector space as the reference action description matrix, wherein the first matrix dimension of the reference action description matrix is ​​lower than the second matrix dimension of the third action description matrix.

[0279] Optionally, the first clustering unit is configured to: obtain multiple reference action description matrices and multiple current centroid matrices; determine clustering distance information corresponding to each of the multiple reference action description matrices based on the multiple reference action description matrices and the multiple current centroid matrices, wherein the clustering distance information includes multiple matrix distances between the multiple current centroid matrices and the reference action description matrices; update the clustering labels matched by each of the multiple reference action description matrices based on the clustering distance information corresponding to each of the multiple reference action description matrices, wherein the clustering labels are used for the action description information subset to which the action description information group currently belongs; update the multiple current centroid matrices based on the reference action description matrices matched by each of the multiple current centroid matrices to obtain updated multiple current centroid matrices; if the multiple current centroid matrices satisfy the first clustering condition, determine at least one subset of the action description information based on the clustering labels matched by each of the multiple reference action description matrices; if the multiple current centroid matrices do not satisfy the first clustering condition, repeat the above steps.

[0280] Optionally, the second acquisition unit 2004 includes a first acquisition module and a second acquisition module. The first acquisition module is used to: after performing a clustering operation on multiple action description information groups based on the spatial distance relationship between multiple reference action description matrices to obtain at least one subset of action description information, the module further includes: acquiring the action description information group from the subset of action description information, and acquiring the first sample action information and the second sample action information from the subset of action description information. The second acquisition module is used to: after determining at least one first action description information subset based on the first action description information included in each of the multiple action description information groups, and determining at least one second action description information subset based on the second action description information included in each of the multiple action description information groups, the module further includes: acquiring the first sample action information from the subset of first action description information, and acquiring the second sample action information from the subset of second action description information.

[0281] Optionally, the acquisition unit 2006 is configured to: display a first virtual object performing the first object action based on the first sample action information, and display a second virtual object performing the second object action based on the second sample action information, wherein the first virtual object is displayed according to a first style and the second virtual object is displayed according to a second style; in response to an action acquisition operation, when the action acquisition device acquires current sample action information, display a third virtual object performing the current sample object action based on the current sample action information, wherein the third virtual object is displayed according to a third style; and when the acquired multiple frames of current sample action information meet the acquisition conditions, determine multiple frames of transitional sample action information that match the first sample action information and the second sample action information based on the multiple frames of current sample action information.

[0282] Optionally, the acquisition unit 2006 is configured to: render the first sample object action performed by the reference virtual object in the target rendering engine according to the first sample action information, and render the second sample object action performed by the reference virtual object at the same virtual position according to the second sample action information; sample first reference action information and second reference action information according to the first sample object action and the second sample object action; initialize the root bone position information and rotation information in the first reference action information and the second reference action information to obtain updated first reference action information and second reference action information; and in the action acquisition application, display the first virtual object performing the first object action according to the updated first reference action information, and display the second virtual object performing the second object action according to the second reference action information.

[0283] Optionally, the acquisition unit 2006 is further configured to: in response to a first angle adjustment operation, display the first virtual object performing the first object action at the adjusted first display angle; in response to a second angle adjustment operation, display the second virtual object performing the second object action at the adjusted second display angle; and in response to a third angle adjustment operation, display the third virtual object performing the currently sampled object action at the adjusted third display angle.

[0284] Optionally, the acquisition unit 2006 is further configured to: determine that the current sampling action information in multiple frames satisfies the acquisition conditions when the first matching degree between the starting sampling action information in the current sampling action information in multiple frames and the first sample action information is greater than or equal to a first threshold, and the second matching degree between the ending sampling action information in the current sampling action information in multiple frames and the second sample action information is greater than or equal to a second threshold.

[0285] Optionally, the training sample generation apparatus further includes at least one of the following: a first sample enhancement unit, configured to obtain a sequence of sample action information from the current training samples included in the plurality of target training samples, wherein the sequence of sample action information includes the first sample action information, the second sample action information, and multiple frames of transition sample action information; perform mirror transformation operations on the plurality of sample action information in the sequence of sample action information to obtain the first sample action information sequence; and use the first sample action information sequence as a newly added target training sample; and a second sample enhancement unit, configured to obtain a sequence of sample action information from the current training samples included in the plurality of target training samples, wherein the first sample action sequence includes the first sample action information, the second sample action information, and multiple frames of transition sample action information. The process involves: performing a velocity transformation operation on the aforementioned sample action information sequence to obtain a second sample action information sequence; using the second sample action information sequence as a newly added target training sample; and a third sample enhancement unit, configured to obtain a sample action information sequence from the current training samples included in the plurality of target training samples, wherein the first sample action sequence includes the first sample action information obtained from the first action description subset, the second sample action information obtained from the second action description subset, and multiple frames of transition sample action information; obtaining a first reference sample action information from the first action description subset and a second reference sample action information from the second action description subset; and using the first reference sample action information, the second reference sample action information, and multiple frames of transition sample action information as newly added target training samples.

[0286] Optionally, the first sample enhancement unit is configured to: obtain current sample action information from multiple sample action information; perform a first mirror symmetry operation on the target rotation information of the first part included in the current sample action information; perform a second mirror symmetry operation on the three-dimensional coordinate information of the second part included in the current sample action information; and determine the updated current sample action information as the first sample action information included in the first sample action information sequence.

[0287] Optionally, the second sample enhancement unit is configured to: acquire velocity description information corresponding to the velocity transformation operation; determine the number of second sample action information included in the second sample action information sequence and the sequence identifier corresponding to each of the multiple sample action information in the sample action information sequence based on the velocity description information; determine the weight information between every two adjacent second sample action information based on the velocity description information; and perform interpolation processing on the multiple sample action information based on the sequence identifier corresponding to each of the multiple sample action information and the weight information between every two adjacent second sample action information to obtain the second sample action information sequence.

[0288] Optionally, the second sample enhancement unit is configured to: perform coordinate transformation operations on the multiple sample action information to obtain multiple sample global action information in a global coordinate system; perform linear interpolation operations on the global position information in the multiple sample global action information to obtain multiple global position information corresponding to the second sample action information sequence; perform spherical linear interpolation operations on the global rotation information in the multiple sample global action information to obtain multiple global rotation information corresponding to the second sample action information sequence; and determine the second sample action information sequence based on the multiple global position information and the multiple global rotation information.

[0289] Optionally, in this embodiment, the implementation of each of the above-mentioned unit modules can be referred to the above-mentioned method embodiments, which will not be repeated here.

[0290] According to another aspect of the present invention, an electronic device for implementing the above-described training sample generation method is also provided, the electronic device being... Figure 21 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 21 As shown, the electronic device includes a memory 2102 and a processor 2104. The memory 2102 stores a computer program, and the processor 2104 is configured to execute the steps of any of the above method embodiments via the computer program.

[0291] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0292] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0293] S1, Obtain an action description information set, wherein the action description information set includes multiple action description information groups, and the action description information groups include first action description information for indicating the action of the first object and second action description information for indicating the action of the second object.

[0294] S2, when at least one subset of action description information is determined by performing clustering operations based on the action description information set, first sample action information and second sample action information are obtained from the at least one subset of action description information.

[0295] S3, based on the first sample action information and the second sample action information, perform action acquisition operation to obtain multi-frame transition sample action information, wherein the multi-frame transition sample action information is used to indicate the multi-frame transition object action that matches the first object action and the second object action.

[0296] S4. Determine the target training sample based on the action information of the first sample, the action information of the second sample, and the action information of the multi-frame transition sample. The target training sample is used to train the transition action prediction model.

[0297] Alternatively, as those skilled in the art will understand, Figure 21 The structure shown is for illustrative purposes only. Electronic devices can also be in-vehicle terminals, smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 21 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 21 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 21 The different configurations shown.

[0298] The memory 2102 can be used to store software programs and modules, such as the program instructions / modules corresponding to the training sample generation method and apparatus in this embodiment of the invention. The processor 2104 executes various functional applications and data processing by running the software programs and modules stored in the memory 2102, thereby realizing the aforementioned training sample generation method. The memory 2102 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 2102 may further include memory remotely located relative to the processor 2104, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 2102 may be used, but is not limited to, for storing file information such as target logical files. As an example, such as Figure 21 As shown, the memory 2102 may include, but is not limited to, the first acquisition unit 2002, the second acquisition unit 2004, the acquisition unit 2006, and the determination unit 2008 in the training sample generation device. Furthermore, it may include, but is not limited to, other module units in the training sample generation device, which will not be elaborated upon in this example.

[0299] Optionally, the transmission device 2106 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 2106 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 2106 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0300] In addition, the aforementioned electronic device also includes a display 2108 and a connection bus 2110 for connecting various module components in the aforementioned electronic device.

[0301] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0302] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in embodiments of this application.

[0303] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0304] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of a computer device reads computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the above-described method for generating training samples.

[0305] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0306] S1, Obtain an action description information set, wherein the action description information set includes multiple action description information groups, and the action description information groups include first action description information for indicating the action of the first object and second action description information for indicating the action of the second object.

[0307] S2, when at least one subset of action description information is determined by performing clustering operations based on the action description information set, first sample action information and second sample action information are obtained from the at least one subset of action description information.

[0308] S3, based on the first sample action information and the second sample action information, perform action acquisition operation to obtain multi-frame transition sample action information, wherein the multi-frame transition sample action information is used to indicate the multi-frame transition object action that matches the first object action and the second object action.

[0309] S4. Determine the target training sample based on the action information of the first sample, the action information of the second sample, and the action information of the multi-frame transition sample. The target training sample is used to train the transition action prediction model.

[0310] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0311] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0312] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0313] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0314] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0315] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0316] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for generating training samples, characterized in that, include: Obtain an action description information set, wherein the action description information set includes multiple action description information groups, and the action description information groups include first action description information for indicating the action of a first object and second action description information for indicating the action of a second object; When performing a clustering operation based on the action description information set to determine at least one subset of action description information, first sample action information and second sample action information are obtained from at least one subset of action description information. Based on the first sample action information and the second sample action information, an action acquisition operation is performed to obtain multiple frames of transition sample action information, wherein the multiple frames of transition sample action information are used to indicate multiple frames of transition object actions that match the first object action and the second object action. Based on the first sample action information, the second sample action information, and the transition sample action information of multiple frames, a target training sample is determined, wherein the target training sample is used to train the transition action prediction model.

2. The method according to claim 1, characterized in that, Before obtaining the first sample action information and the second sample action information from at least one of the at least one subset of action description information when performing a clustering operation based on the action description information set to determine at least one subset of action description information, the method further includes one of the following: Determine the reference action description matrix corresponding to each of the multiple action description information groups in the action description information set; perform a clustering operation on the multiple action description information groups according to the spatial distance relationship between the multiple reference action description matrices to obtain at least one subset of the action description information; At least one first action description information subset is determined based on the first action description information included in each of the plurality of action description information groups, and at least one second action description information subset is determined based on the second action description information included in each of the plurality of action description information groups. The first action description information subset includes a plurality of first action description information that satisfies a second clustering condition, and the second action description information subset includes a plurality of second action description information that satisfies a third clustering condition. At least one action description information subset includes the first action description information subset and the second action description information subset.

3. The method according to claim 2, characterized in that, Determining the reference action description matrix corresponding to each of the multiple action description information groups in the action description information set includes: Obtain the first action description matrix corresponding to the first action description information in the current action description information group, and the second action description matrix corresponding to the second action description information in the current action description information group; An action difference matrix is ​​determined based on the first action description matrix and the second action description matrix, and a third action description matrix is ​​determined based on the action difference matrix and the first action description matrix. The first action description matrix is ​​used to indicate the first object action performed by the virtual object at a first moment, and the second action description matrix is ​​used to indicate the second object action performed by the virtual object at a second moment, which is later than the first moment. The projection matrix obtained by spatially projecting the third action description matrix onto the target vector space is determined as the reference action description matrix, wherein the first matrix dimension of the reference action description matrix is ​​lower than the second matrix dimension of the third action description matrix.

4. The method according to claim 2, characterized in that, The step of performing a clustering operation on multiple action description information groups based on the spatial distance relationship between multiple reference action description matrices to obtain at least one subset of the action description information includes: Obtain the reference action description matrices and the current centroid matrices; Based on the plurality of reference action description matrices and the plurality of current centroid matrices, clustering distance information corresponding to each of the plurality of reference action description matrices is determined, wherein the clustering distance information includes multiple matrix distances between the plurality of current centroid matrices and the reference action description matrices; Based on the clustering distance information corresponding to each of the multiple reference action description matrices, update the clustering label matched by each of the reference action description matrices, wherein the clustering label is used for the action description information subset to which the action description information group currently belongs; Based on the reference action description matrix matched by each of the multiple current centroid matrices, the multiple current centroid matrices are updated to obtain the multiple updated current centroid matrices; If multiple current centroid matrices satisfy the first clustering condition, at least one subset of action description information is determined based on the clustering labels matched by each of the reference action description matrices. If multiple current centroid matrices do not satisfy the first clustering condition, repeat the above steps.

5. The method according to claim 2, characterized in that, After performing a clustering operation on multiple action description information groups based on the spatial distance relationship between multiple reference action description matrices to obtain at least one subset of action description information, the step includes: obtaining the action description information group from the subset of action description information, and obtaining the first sample action information and the second sample action information from the action description information group. After determining at least one subset of first action description information based on the first action description information included in each of the plurality of action description information groups, and determining at least one subset of second action description information based on the second action description information included in each of the plurality of action description information groups, the process includes: obtaining first sample action information from the first subset of action description information, and obtaining second sample action information from the second subset of action description information.

6. The method according to claim 1, characterized in that, The step of performing an action acquisition operation based on the first sample action information and the second sample action information to obtain multi-frame transition sample action information includes: According to the first sample action information, the first virtual object is displayed to perform the first object action, and according to the second sample action information, the second virtual object is displayed to perform the second object action, wherein the first virtual object is displayed according to the first style, and the second virtual object is displayed according to the second style; In response to the motion acquisition operation, based on the current sampled motion information acquired by the motion acquisition device, a third virtual object is displayed to perform the current sampled object's action according to the current sampled motion information, wherein the third virtual object is displayed according to a third style; If the current sampling action information of the collected multiple frames meets the collection conditions, the transition sample action information of the multiple frames that matches the first sample action information and the second sample action information is determined based on the current sampling action information of the multiple frames.

7. The method according to claim 6, characterized in that, The step of displaying a first virtual object performing the first object action based on the first sample action information, and displaying a second virtual object performing the second object action based on the second sample action information, includes: In the target rendering engine, the first sample object action executed by the reference virtual object is rendered according to the first sample action information, and the second sample object action executed by the reference virtual object is rendered according to the second sample action information at the same virtual position. Based on the actions of the first sample object and the actions of the second sample object, first reference action information and second reference action information are sampled respectively. The root bone position information and rotation information in the first reference motion information and the second reference motion information are initialized to obtain the updated first reference motion information and the second reference motion information. In motion capture applications, the first virtual object is displayed to perform the first object action based on the updated first reference motion information, and the second virtual object is displayed to perform the second object action based on the second reference motion information.

8. The method according to claim 6, characterized in that, After displaying the first virtual object performing the first object action based on the first sample action information, and displaying the second virtual object performing the second object action based on the second sample action information, the method further includes at least one of the following: In response to the first angle adjustment operation, the first virtual object is displayed at the adjusted first display angle to perform the first object action; In response to the second angle adjustment operation, the second virtual object is displayed at the adjusted second display angle to perform the second object action; In response to the third angle adjustment operation, the third virtual object is displayed at the adjusted third display angle to perform the action of the currently sampled object.

9. The method according to claim 6, characterized in that, Before determining the transitional sample action information of multiple frames that matches the first sample action information and the second sample action information based on the current sample action information of multiple frames, the method further includes: If the first matching degree between the starting sampling action information in the current sampling action information of multiple frames and the first sample action information is greater than or equal to a first threshold, and the second matching degree between the ending sampling action information in the current sampling action information of multiple frames and the second sample action information is greater than or equal to a second threshold, then the current sampling action information of multiple frames is determined to satisfy the acquisition condition.

10. The method according to claim 2, characterized in that, After determining the target training sample based on the first sample action information, the second sample action information, and the transition sample action information of multiple frames, the method further includes at least one of the following: From the current training samples including multiple target training samples, a sequence of sample action information is obtained, wherein the sequence of sample action information includes the first sample action information, the second sample action information, and multiple frames of transition sample action information; a mirror transformation operation is performed on multiple sample action information in the sequence of sample action information to obtain a first sample action information sequence; the first sample action information sequence is used as a newly added target training sample. From the current training samples including multiple target training samples, a sequence of sample action information is obtained, wherein the first sample action sequence includes the first sample action information, the second sample action information, and the transition sample action information of multiple frames; a velocity transformation operation is performed on the sample action information sequence to obtain a second sample action information sequence; the second sample action information sequence is used as a newly added target training sample. From the current training samples including multiple target training samples, a sequence of sample action information is obtained, wherein the first sample action sequence includes the first sample action information obtained from the first action description subset, the second sample action information obtained from the second action description subset, and multiple frames of transition sample action information; a first reference sample action information is obtained from the first action description subset, and a second reference sample action information is obtained from the second action description subset; the first reference sample action information, the second reference sample action information, and the multiple frames of transition sample action information are used as newly added target training samples.

11. The method according to claim 10, characterized in that, The step of performing a mirror transformation operation on multiple sample action information in the sample action information sequence to obtain a first sample action information sequence includes: Obtain the current sample action information from multiple sample action information; Perform a first mirror symmetry operation on the target rotation information of the first part included in the current sample action information; Perform a second mirror symmetry operation on the three-dimensional coordinate information of the second part included in the current sample action information; The updated current sample action information is determined as the first sample action information included in the first sample action information sequence.

12. The method according to claim 10, characterized in that, The step of performing a velocity transformation operation on the sample action information sequence to obtain a second sample action information sequence includes: Obtain the velocity description information corresponding to the velocity transformation operation; The number of second sample action information included in the second sample action information sequence and the sequence identifier corresponding to each of the multiple sample action information in the sample action information sequence are determined based on the speed description information. The weight information between every two adjacent second sample action information is determined based on the velocity description information; Based on the sequence identifiers corresponding to each of the multiple sample action information and the weight information between every two adjacent second sample action information, interpolation processing is performed on the multiple sample action information to obtain the second sample action information sequence.

13. The method according to claim 10, characterized in that, The step of performing interpolation processing on the multiple sample action information according to the order identifiers corresponding to each of the multiple sample action information and the weight information between every two adjacent second sample action information to obtain the second sample action information sequence includes: The coordinate transformation operation is performed on the multiple sample action information to obtain the multiple sample global action information in the global coordinate system; A linear interpolation operation is performed based on the global position information in the global action information of the multiple samples to obtain multiple global position information corresponding to the second sample action information sequence; Perform a spherical linear interpolation operation based on the global rotation information in the multiple sample global action information to obtain multiple global rotation information corresponding to the second sample action information sequence; The second sample action information sequence is determined based on multiple global position information and multiple global rotation information.

14. A training sample generation apparatus, characterized in that, include: The first acquisition unit is used to acquire an action description information set, wherein the action description information set includes multiple action description information groups, and the action description information groups include first action description information for indicating the action of a first object and second action description information for indicating the action of a second object. The second acquisition unit is used to acquire first sample action information and second sample action information from at least one subset of action description information when performing a clustering operation based on the action description information set to determine at least one subset of action description information. The acquisition unit is used to perform an action acquisition operation based on the first sample action information and the second sample action information to obtain multiple frames of transition sample action information, wherein the multiple frames of transition sample action information are used to indicate multiple frames of transition object actions that match the first object action and the second object action. A determining unit is configured to determine a target training sample based on the first sample action information, the second sample action information, and the transition sample action information of multiple frames, wherein the target training sample is used to train the transition action prediction model.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 13.

16. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 13.

17. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 13 through the computer program.

Images