Combat action skeletal animation generation method and system based on text feature extraction

Abstract

The present invention discloses a method and system for generating combat action skeletal animation based on text feature extraction, comprising the steps of: constructing a text dataset that precisely describes skeletal animation features in multiple dimensions; receiving text input and generating combat action skeletal animation via model reasoning; aligning the text skeletal animation with the model output feedback results; iteratively optimizing the generated results based on the feedback results; and outputting the generated results in accordance with the standard. Therefore, the present invention improves the matching degree between the input text description and the generated corresponding skeletal animation by optimizing the text input received by the model and iteratively optimizing the generated results through feedback, thereby achieving the effect of improving the robustness of the model and lowering the threshold for use. The method also improves the quality of skeletal animation generated by the generative model that receives text input, making the generated combat action skeletal animation more consistent with expectations.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning, and in particular relates to a method and system for generating combat action skeletal animation based on text feature extraction. Background Art

[0002] Skeletal animation separates the skeleton and skin of a character, allowing them to smoothly perform complex movements in games, significantly enhancing the visual quality and interactivity of the game. In recent years, significant progress has been made in skeletal animation generation technology based on generative models. Traditional skeletal animation production typically relies on motion capture or manual annotation and manual adjustments by animators, but generative models, through deep learning and data-driven methods, can automatically generate high-quality animations.

[0003] Variational Auto-Encoders (VAEs) and Diffusion Models are two important generative models for generating skeletal animation, each with its own unique characteristics. VAEs learn the key features of motion by mapping skeletal animation data into a low-dimensional latent space. During the generation process, the VAE's encoder transforms the input animation into a distribution of latent variables, and the decoder reconstructs the animation sequence from these latent variables. During training, the VAE minimizes the reconstruction error to optimize the generation. This approach performs well in generating smooth transitions and diverse animations, but can face challenges with complex and high-frequency dynamics. In contrast, diffusion models gradually add noise to skeletal animation data and then generate animation sequences through a reverse denoising process. During training, diffusion models compare real and noisy animation sequences to learn how to effectively remove noise, thereby generating high-quality skeletal animations. While diffusion models can capture detailed and complex dynamic features and generate highly realistic animations, they are computationally expensive and the generation process is relatively complex. In general, VAE is suitable for generating smooth action sequences, while the diffusion model excels in generating high details and complex dynamics.

[0004] Text-based skeletal animation generation is relatively simple to operate. Users can describe their needs in natural language and generate skeletal animation sequences that meet their expectations. This is of great significance for reducing animation production costs and improving work efficiency in the gaming and film industries. However, the overall level of skeletal animation generation still faces several prominent issues. For example, existing skeletal animation generation models often perform poorly, with the generated animation sequences often inconsistent with the user's expectations, making it difficult to meet user needs. Furthermore, existing skeletal animation generation models mostly generate everyday movements such as walking, running, and jumping. For action games, which require the most diverse skeletal animation, the generated combat movements often lack the variety needed to meet user needs. Furthermore, existing skeletal animation generation models lack detailed differentiation between skeletal animation types, often struggle to respond effectively to users' more detailed text descriptions, and often perform poorly for inputs not previously included in the dataset.

[0005] Therefore, in order to address problems such as the generation results of the generative model not matching the input text well enough, and the generated actions being closer to daily actions rather than combat actions, a method is needed to optimize the text input and the output results of the generative model, improve the matching degree between the generated results and the input text, and enhance the model's ability to cope with unseen text descriptions. Summary of the Invention

[0006] The purpose of the present invention is to address the problems and shortcomings of the existing technology and propose a combat skeleton animation generation method and system based on text feature extraction.

[0007] To achieve the above object, the present invention provides a method for generating combat action skeletal animation based on text feature extraction, the method comprising the following steps:

[0008] S1: Construct a text dataset that describes skeletal animation features in a multi-dimensional and detailed manner;

[0009] S2: Receive text input and generate combat skeletal animation through model reasoning;

[0010] S3: Align the generated skeletal animation with the original text description and output feedback results;

[0011] S4: Iteratively optimize the generated results based on the feedback results; analyze the problems and deficiencies in the generated animation based on the feedback from the alignment model; determine the category that best matches the generated animation by comparing the similarity between the generated animation and the descriptions of each category, and then optimize the original text input to make it more accurately reflect the required action characteristics and details;

[0012] S5: For the generated results that meet the standards, the corresponding combat action skeleton animation is output as the output result.

[0013] Furthermore, in step S2, the formula for generating the skeletal animation for the first time according to the text description is expressed as follows:

[0014]

[0015] in represents the k-th description of category y, It represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and Animation-Gen represents the reasoning process of the generative model generating skeletal animation by processing the corresponding description.

[0016] Furthermore, the formula for aligning the skeletal animation generated in step S3 with the text is expressed as follows:

[0017]

[0018] Among them, p asess Represents the generated skeletal animation The probability distribution of the degree of alignment with the text description of each category, T represents the transpose operator in matrix operations, MotionCLIP anim MotionCLIP is a skeletal animation encoder that represents a skeletal animation-text pair model. text represents the text encoder, D represents the set of all description texts describing the skeletal animation of this category, represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and k represents the k-th description.

[0019] Furthermore, in step S4, when comparing the similarity between the generated animation and the descriptions of each category, feature alignment and similarity measurement methods are used to evaluate the fit between the generated animation and the category description.

[0020] Furthermore, in step S4, the alignment formula for the initially generated skeletal animation and the corresponding text category is expressed as follows:

[0021]

[0022] where p MotionCLIP represents the probability derived from the MotionCLIP model, which is expressed as test , the animation encoding module of the MotionCLIP model is matched and aligned with all pre-stored text category features. The formula represents the matching probability calculation process for one of the features.

[0023] Furthermore, in step S5, the generation result that meets the standards specifically includes: the generated animation meets the requirements of the text description and meets the animation quality standards, including fluency, realism and accuracy of movement.

[0024] To achieve the above objectives, the present invention also provides a combat action skeletal animation generation system based on text feature extraction, which includes the following modules:

[0025] Text description dataset construction module: used to collect and organize data pairs containing text descriptions and their corresponding skeletal animations, annotate the data, and ensure accurate correspondence between text descriptions and animations;

[0026] Text feature extraction module: used to extract useful information from text descriptions, including word embedding, part-of-speech tagging, and syntactic structure; and convert the extracted information into data information that the model can process and understand;

[0027] Skeletal animation feature extraction module: used to extract key motion features from skeletal animation, including keyframes, joint positions, velocities, and accelerations; and convert the extracted features into data information that the model can process and understand in order to match them with text features;

[0028] Skeletal animation-text feature matching module: responsible for matching and matching the extracted text features with skeletal animation features, and learning the mapping relationship between text descriptions and skeletal animations;

[0029] Matching result feedback optimization module: This module evaluates the consistency between the generated animation and the text description based on the matching results. For generated results that deviate significantly from the text description, the module identifies the animation category and provides relevant information and directions for optimizing the text description based on the feedback. The module then transmits the optimized text description for iterative generation of results.

[0030] Animation processing and output module: used to receive the generated skeletal animation data and perform post-processing, including smoothing motion and adjusting time series; converting the processed animation into a usable format, including video files or pre-stored skeletal motion data.

[0031] The beneficial effects of the present invention are as follows:

[0032] 1. The present invention improves the matching degree between the generated skeletal animation and the input text by introducing a combat skeletal animation generation method based on text feature extraction, and improves the conformity degree between the generated skeletal animation and the user's expectations.

[0033] 2. The present invention can accurately identify the deficiencies in the generated animation by analyzing the degree of match between the generated animation and the description of the skeletal animation category, thereby finding the description that best fits the target action. Optimizing text input can improve the guidance accuracy of the generated model and ensure that the generated animation is more in line with the expected action characteristics. This method avoids the high cost and complexity of retraining the model and improves the results by adjusting the input text, so that the quality and accuracy of the generated animation are significantly improved. In addition, by repeatedly optimizing and adjusting the text input, the details and performance of the generated animation can be gradually refined, the naturalness and smoothness of the animation can be enhanced, and ultimately a better generation effect can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are incorporated into the specification and serve as an integral part of the specification, illustrating the implementation of the various modules involved in the present invention in accordance with the principles of the present invention. The focus of the drawings is not to limit the invention, but to explain the principles of the invention. In the drawings,

[0035] Figure 1 This is a flow chart of a combat skeleton animation generation method based on text feature extraction according to the present invention;

[0036] Figure 2 This is a schematic diagram of the training process of the skeleton animation generator in the present invention;

[0037] Figure 3 This is a feedback and optimization flow chart of the present invention;

[0038] Figure 4 Schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be described in detail below based on the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become more apparent. The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0040] The combat action skeletal animation generation method based on text feature extraction of the present invention optimizes the matching degree between user input and generated skeletal animation results, optimizes the production process of combat action skeletal animation, and improves the production efficiency of combat action skeletal animation.

[0041] like Figure 1 As shown, the present invention provides a method for generating combat skeleton animation based on text feature extraction, and the specific implementation steps include:

[0042] Step 1: Construct a text dataset that provides multi-dimensional, detailed descriptions of skeletal animation features. Collect and create a text dataset containing rich descriptions of combat actions. This dataset includes detailed text descriptions of various combat actions, describing the corresponding combat actions from multiple perspectives. The descriptions include characteristics of multiple dimensions of the actions, including movement speed, direction, force, weapon held, and the starting and ending positions of the attack. These descriptions can be constructed using professional combat action literature, annotations of motion capture data, and the knowledge of domain experts. Ensure that the dataset covers a wide range of action types and variations to improve the model's generalization ability.

[0043] Step 2: Receive text input and generate combat skeleton animation through model reasoning. In actual applications, users input specific combat action description text. The model receives these text inputs and converts the text into a format that the model can understand through the built-in natural language processing module. Then, the generation model (such as the VAE model) uses this text information for reasoning to generate the corresponding combat skeleton animation sequence. Among them, the training process structure of the generation model is as follows: Figure 2 The process of generating skeletal animation from a trained model involves converting text descriptions into feature vectors in a latent space and generating skeletal animations through a decoder or generator. The formula for generating skeletal animations from text descriptions for the first time is as follows:

[0044]

[0045] in represents the k-th description of category y, It represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and Animation-Gen represents the reasoning process of the generative model generating skeletal animation by processing the corresponding description.

[0046] Step 3: The text-skeleton animation alignment model outputs feedback results. The generated skeletal animation needs to be aligned with the original text description to ensure the accuracy and consistency of the movement. This step involves using the alignment model to match the generated skeletal animation with the text description. The alignment model evaluates whether the movement meets the expectations of the description by comparing the degree of match between the generated animation and the description. The feedback results include the gaps and deficiencies between the generated animation and the expected movement. The formula for aligning the generated skeletal animation with the text is expressed as follows:

[0047]

[0048] This formula describes how to use the skeletal animation-text alignment capability of MotionCLIP to generate skeletal animation. The process of aligning with D. Among them, p asess Represents the generated skeletal animation The probability distribution of the degree of alignment with the text description of each category, T represents the transpose operator in matrix operations, MotionCLIP anim MotionCLIP is a skeletal animation encoder that represents a skeletal animation-text pair model. text represents the text encoder, D represents the set of all description texts describing the skeletal animation of this category, represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and k represents the k-th description.

[0049] Step 4: Iteratively optimize the generated results based on the feedback results. According to the feedback from the alignment model, analyze the problems and deficiencies in the generated animation. By comparing the similarity between the generated animation and the descriptions of each category, determine the category that best matches the generated animation; this process usually uses feature alignment and similarity measurement methods to evaluate the fit between the generated animation and the category description. Then, according to the results obtained by comparing the similarity between the generated animation and the descriptions of each category, and determining the category that best matches the generated animation, optimize the original text input to make it more accurately reflect the required action features and details. This optimization may include adjusting the specific details of the action description, increasing the accuracy of the description, or restating the action requirements. The optimized text input will be provided to the generation model as a new input to generate skeletal animation again. In this way, the accuracy and quality of the generated animation can be improved by precisely adjusting the text description without retraining the model. This process is repeated, and the specific execution process is as follows. Figure 3 As shown, until the generated skeletal animation better matches the optimized text description. This process is iterative until the generated combat animation meets the expected standards. During the optimization iteration process, the alignment optimization process of the skeletal animation is shown in the following formula:

[0050]

[0051] where p MotionCLIP represents the probability derived from the MotionCLIP model, which is expressed as test , the animation encoding module of the MotionCLIP model is matched and aligned with all pre-stored text category features. The formula represents the matching probability calculation process for one of the features.

[0052] Step 5: Ensure the generated results meet the required output standards. The optimized combat skeletal animation is exported and subjected to final verification. This verification step ensures that the generated animation not only meets the requirements described in the text but also meets animation quality standards such as smoothness, realism, and accuracy of movement. If the output meets the expected standards, it can be used in actual games or applications.

[0053] like Figure 4 As shown, the present invention also provides a system for generating combat skeleton animation based on text feature extraction, which includes the following modules:

[0054] Text description dataset construction module: collects and organizes data pairs containing text descriptions and their corresponding skeletal animations, annotates the data, and ensures an accurate correspondence between the text descriptions and the animations.

[0055] Text feature extraction module: Extracts useful information from text descriptions, such as word embedding, part-of-speech tagging, and syntactic structure, and converts the extracted information into data that the model can process and understand.

[0056] Skeletal animation feature extraction module: Extracts key motion features from skeletal animation, such as keyframes, joint positions, velocity, and acceleration. The extracted features are converted into data information that the model can process and understand, so as to match them with text features.

[0057] Skeletal animation-text feature matching module: responsible for matching the extracted text features with skeletal animation features, and learning the mapping relationship from text description to skeletal animation.

[0058] Matching Result Feedback Optimization Module: Based on the matching results, the generated animation is evaluated for consistency with the text description. For generated results that deviate significantly from the text description, the module determines the animation's category and, based on the feedback, provides information and directions for optimizing the text description. The module then transmits the optimized text description for iteration.

[0059] Animation processing and output module: Receives the generated skeletal animation data and performs necessary post-processing, such as smoothing motion, adjusting time series, etc. Converts the processed animation into a usable format, such as a video file or pre-stored skeletal motion data.

[0060] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solutions disclosed in the present invention without the need for creative work should be included in the scope of protection of the present invention.

Claims

1. A method for generating combat action skeleton animation based on text feature extraction, characterized in that: The method comprises the following steps: S1: Construct a text dataset that describes skeletal animation features in a multi-dimensional and detailed manner; S2: Receive text input and generate combat skeletal animation through model reasoning; S3: Align the generated skeletal animation with the original text description and output feedback results; S4: Iteratively optimize and generate results based on the feedback results; Analyze problems and deficiencies in the generated animation based on feedback from the alignment model; By comparing the similarity between the generated animation and the descriptions of each category, the category that best matches the generated animation is determined, and the original text input is optimized to make it more accurately reflect the required action characteristics and details; When comparing the similarity between the generated animation and the description of each category, feature alignment and similarity measurement methods are used to evaluate the fit between the generated animation and the category description; S5: For the generated results that meet the standards, the corresponding combat action skeleton animation is output as the output result.

2. The method for generating combat action skeleton animation based on text feature extraction according to claim 1, characterized in that: In step S2, the formula for generating the skeletal animation for the first time according to the text description is expressed as follows: in represents the k-th description of category y, It represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and Animation-Gen represents the reasoning process of the generative model generating skeletal animation by processing the corresponding description.

3. The method for generating combat action skeleton animation based on text feature extraction according to claim 1, wherein: The formula for aligning the skeletal animation generated in step S3 with the text is as follows: Among them, p asess Represents the generated skeletal animation The probability distribution of the degree of alignment with the text description of each category, T represents the transpose operator in matrix operations, MotionCLIP anim Skeletal animation encoder representing the skeletal animation-text alignment model, MotionCLIP text represents the text encoder, D represents the set of all description texts describing the skeletal animation of this category, represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and k represents the k-th description.

4. The method for generating combat action skeleton animation based on text feature extraction according to claim 1, wherein: The alignment formula of the initially generated skeletal animation and the corresponding text category in step S4 is expressed as follows: where p MotionCLIP represents the probability derived from the MotionCLIP model, which is expressed as test , the animation encoding module of the MotionCLIP model is matched and aligned with all pre-stored text category features. The formula represents the matching probability calculation process for one of the features.

5. The method for generating combat action skeleton animation based on text feature extraction according to claim 1, characterized in that: In step S5, the generation result that meets the standards specifically includes: the generated animation meets the requirements of the text description and meets the animation quality standards, including fluency, realism and accuracy of movements.

6. A combat action skeletal animation generation system based on text feature extraction that implements the method as claimed in claim 1, characterized in that: The system consists of the following modules: Text description dataset construction module: used to collect and organize data pairs containing text descriptions and their corresponding skeletal animations, annotate the data, and ensure accurate correspondence between text descriptions and animations; Text feature extraction module: used to extract useful information from text descriptions, including word embedding, part-of-speech tagging, and syntactic structure; and convert the extracted information into data information that the model can process and understand; Skeletal animation feature extraction module: used to extract key motion features from skeletal animation, including keyframes, joint positions, velocities, and accelerations; and convert the extracted features into data information that the model can process and understand in order to match them with text features; Skeletal animation-text feature matching module: responsible for matching and matching the extracted text features with skeletal animation features, and learning the mapping relationship between text descriptions and skeletal animations; Matching result feedback optimization module: This module evaluates the consistency between the generated animation and the text description based on the matching results. For generated results that deviate significantly from the text description, the module identifies the animation category and provides relevant information and directions for optimizing the text description based on the feedback. The module then transmits the optimized text description for iterative generation of results. Animation processing and output module: used to receive the generated skeletal animation data and perform post-processing, including smoothing motion and adjusting time series; converting the processed animation into a usable format, including video files or pre-stored skeletal motion data.

Images