Character feature extraction-based combat action skeletal animation generation method and system

By constructing a multi-dimensional text dataset and iterative optimization generation process, the shortcomings of the skeleton animation generation model in combat action diversity and unseen description processing are solved, and the matching degree and animation quality of the generation results and input text are improved, and the More efficient combat skeleton animation generation.

CN118967895AActive Publication Date: 2024-11-15ZHEJIANG UNIV

Patent Information

Application Number
CN202411050737.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-15
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

The existing skeleton animation generation model is insufficient in generating combat actions, which is difficult to meet user needs, and has poor performance on unseen text descriptions, resulting in low matching between the generation results and the input text, making it difficult to meet complex and high-detailed actions. Requirements.

Method used

By constructing a text dataset that describes skeletal animation features in a multi-dimensional way, receiving text input and generating skeletal animations, aligning animations with animations and text descriptions, iterative optimization generates results, using feature alignment and similarity measurement methods to evaluate the matching degree, and adjust text input To improve the accuracy and detailed performance of generated results.

Benefits of technology

Improve the matching degree of skeleton animation and input text, enhance the model's processing ability for unseen text descriptions, and significantly improve the quality and accuracy of generated animations, avoiding the high cost and complexity of retraining the model, and achieving a more natural and smooth animation performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967895A_ABST
    Figure CN118967895A_ABST
Patent Text Reader

Abstract

The invention discloses a combat action skeletal animation generation method and system based on character feature extraction. Comprising the steps of constructing a character data set for multi-dimensional fine description of skeletal animation features, receiving character input, generating a combat action skeletal animation through model reasoning, aligning the character skeletal animation with a model to output a feedback result, iteratively optimizing a generation result according to the feedback result, and outputting a result when the generation result meets a standard. Therefore, the matching degree of the input character description and the generated corresponding skeletal animation is improved through the character input received by the optimization model and the iterative feedback optimization generation result, and the effects of improving the robustness of the model and reducing the use threshold are achieved. The quality of the skeletal animation generated by the generative model for receiving character input is improved, so that the generated skeletal animation of the combat action is more in line with expectation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of machine learning, and in particular relates to a method and system for generating combat action skeleton animation based on text feature extraction. Background Art

[0002] Skeletal animation processes the skeleton and skin of a character separately, allowing the character to perform complex movements smoothly in the game, greatly improving the visual effects and interactivity of the game. In recent years, skeletal animation generation technology based on generative models has made significant progress. Traditional skeletal animation production usually relies on motion capture or manual annotation and manual adjustment by animators, but generative models can automatically generate high-quality animations through deep learning and data-driven methods.

[0003] Variational Auto-Encoder (VAE) and Diffusion Models are two important generative models for generating skeletal animation, each with its own characteristics. VAE learns the main features of the action by mapping the skeletal animation data into a low-dimensional latent space. During the generation process, the encoder of VAE converts the input animation into a distribution of latent variables, and the decoder reconstructs the animation sequence from these latent variables. During training, VAE minimizes the reconstruction error to optimize the generation effect. This method performs well in generating smooth transitions and varied animations, but may be challenging when dealing with complex and high-frequency dynamics. In contrast, the diffusion model gradually adds noise to the skeletal animation data and then generates the animation sequence through the reverse denoising process. During the training process, the diffusion model learns how to effectively remove noise by comparing the real and noisy animation sequences, thereby generating high-quality skeletal animation. Although the diffusion model can capture delicate and complex dynamic features and generate highly realistic animations, it has a high computational cost and a complex generation process. In general, VAE is suitable for generating smooth action sequences, while the diffusion model excels in generating high details and complex dynamics.

[0004] The skeletal animation generation based on text input is relatively simple in operation. Users can describe their needs in natural language and generate skeletal animation sequences that meet expectations. It is of great significance to reduce the animation production costs of the game industry and the film and television industry and improve work efficiency. Looking at the overall level of skeletal animation generation work, there are still several prominent problems: for example, the effect of the existing skeletal animation generation model is not as good as expected, and the generated animation sequence is often inconsistent with the user's ideas, which is difficult to meet the user's work needs; for example, the existing skeletal animation generation model mostly generates people's daily actions, such as walking, running and jumping. For action games that require the strongest diversity of skeletal animation, the diversity of generated combat actions often cannot meet the needs of users. For example, the existing skeletal animation generation model is not detailed enough to distinguish the types of skeletal animation, and it is often difficult to respond well to the user's more detailed text descriptions, and it often performs poorly for inputs that have not appeared in the data set.

[0005] Therefore, in order to address the problems such as the generation results of the generative model not matching the input text sufficiently and the generated actions being closer to daily actions rather than combat actions, a method is needed to optimize the text input and the output results of the generative model, improve the matching degree between the generation results and the input text, and enhance the model's ability to deal with unprecedented text descriptions. Summary of the invention

[0006] The purpose of the present invention is to address the problems and shortcomings of the prior art and to propose a combat skeleton animation generation method and system based on text feature extraction.

[0007] To achieve the above object, the present invention provides a method for generating combat action skeleton animation based on text feature extraction, the method comprising the following steps:

[0008] S1: Construct a text dataset that describes the characteristics of skeletal animation in multiple dimensions;

[0009] S2: receiving text input and generating combat skeletal animation through model reasoning;

[0010] S3: align the generated skeletal animation with the original text description and output feedback results;

[0011] S4: Iteratively optimize the generated results according to the feedback results; analyze the problems and deficiencies in the generated animation according to the feedback of the alignment model; determine the category that best matches the generated animation by comparing the similarity between the generated animation and the description of each category, and then optimize the original text input to make it more accurately reflect the required action characteristics and details;

[0012] S5: For the generated results that meet the standards, the corresponding combat action skeleton animation is output as the output result.

[0013] Furthermore, in step S2, the formula for generating the skeletal animation for the first time according to the text description is expressed as follows:

[0014]

[0015] in represents the kth description of category y, It represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and Animation-Gen represents the reasoning process of the generative model generating skeletal animation by processing the corresponding description.

[0016] Furthermore, the formula for aligning the skeletal animation and text generated in step S3 is expressed as follows:

[0017]

[0018] Among them, p asess Represents the generated skeletal animation The probability distribution of the degree of alignment with the text description of each category. T represents the transpose operator in matrix operations. MotionCLIP anim MotionCLIP is a skeletal animation encoder that represents skeletal animation-text to its model. text represents the text encoder, D represents the set of all description texts describing the skeletal animation of this category, represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and k represents the kth description.

[0019] Furthermore, in step S4, when comparing the similarity between the generated animation and each category description, feature alignment and similarity measurement methods are used to evaluate the fit between the generated animation and the category description.

[0020] Furthermore, in step S4, the alignment formula between the initially generated skeletal animation and the corresponding text category is expressed as follows:

[0021]

[0022] where p MotionCLIP represents the probability derived from the MotionCLIP model. This formula represents the output skeletal animation i test , the animation encoding module of the MotionCLIP model is matched and aligned with all pre-stored text category features. The formula represents the matching probability calculation process for one of the features.

[0023] Furthermore, in step S5, the generation result that meets the standards is specifically: the generated animation meets the requirements of the text description and meets the animation quality standards, including fluency, realism and accuracy of movements.

[0024] To achieve the above object, the present invention also provides a combat action skeleton animation generation system based on text feature extraction, which comprises the following modules:

[0025] Text description dataset construction module: used to collect and organize data pairs containing text descriptions and their corresponding skeletal animations, annotate the data, and ensure that there is an accurate correspondence between the text descriptions and the animations;

[0026] Text feature extraction module: used to extract useful information from text descriptions, including word embedding, part-of-speech tagging, and syntactic structure; convert the extracted information into data information that the model can process and understand;

[0027] Skeletal animation feature extraction module: used to extract key motion features from skeletal animation, including key frames, joint positions, speeds, and accelerations; convert the extracted features into data information that the model can process and understand, so as to match them with text features;

[0028] Skeletal animation-text feature matching module: responsible for matching and matching the extracted text features with the skeletal animation features, and learning the mapping relationship from text description to skeletal animation;

[0029] Matching result feedback optimization module: used to evaluate the consistency between the generated animation and the text description based on the matching results; for the generated results that deviate too much from the text description, determine the category to which the animation belongs, and provide relevant information and directions for optimizing the text description based on the feedback information, and then transmit the optimized text description for iterative generation of results;

[0030] Animation processing and output module: used to receive the generated skeletal animation data and perform post-processing, including smoothing motion and adjusting time series; converting the processed animation into a usable format, including video files or pre-stored skeletal motion data.

[0031] The beneficial effects of the present invention are as follows:

[0032] 1. The present invention improves the matching degree between the generated skeletal animation and the input text, and improves the conformity degree between the generated skeletal animation and the user's expectations by introducing a combat skeletal animation generation method based on text feature extraction.

[0033] 2. The present invention can accurately identify the deficiencies in the generated animation by analyzing the degree of match between the generated animation and the description of the skeletal animation category, thereby finding the description that best fits the target action. Optimizing text input can improve the guidance accuracy of the generated model and ensure that the generated animation is more in line with the expected action characteristics. This method avoids the high cost and complexity of retraining the model, and improves the results by adjusting the input text, so that the quality and accuracy of the generated animation are significantly improved. In addition, by repeatedly optimizing and adjusting the text input, the details and performance of the generated animation can be gradually refined, the naturalness and fluency of the animation can be enhanced, and ultimately a better generation effect can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The attached figures are incorporated into the specification and serve as an integral part of the specification, illustrating the implementation methods of the various modules involved in the present invention in accordance with the principles of the present invention. The focus of the drawings is not to limit but to explain the principles of the invention. In the drawings,

[0035] Figure 1 It is a flow chart of the combat skeleton animation generation method based on text feature extraction of the present invention;

[0036] Figure 2 This is a schematic diagram of the training process of the skeleton animation generator in the present invention;

[0037] Figure 3 It is a feedback and optimization flow chart in the present invention;

[0038] Figure 4 It is a schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be described in detail below according to the accompanying drawings and preferred embodiments, and the purpose and effect of the present invention will become more clear. The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] The combat action skeletal animation generation method based on text feature extraction of the present invention optimizes the matching degree between the user input and the generated skeletal animation result, optimizes the production process of the combat action skeletal animation, and improves the production efficiency of the combat action skeletal animation.

[0041] like Figure 1 As shown, the present invention provides a method for generating combat skeleton animation based on text feature extraction, and the specific implementation steps include:

[0042] Step 1: Construct a text dataset that describes the characteristics of skeletal animation in multiple dimensions. Collect and create a text dataset containing rich descriptions of combat actions. This dataset includes detailed text descriptions of various combat actions, describing the corresponding combat actions from multiple angles, and describing the characteristics of multiple dimensions of the actions, including movement speed, movement direction, sense of strength, holding weapons, starting and ending positions of attacks, etc. These descriptions can be constructed through professional combat action literature, annotations of motion capture data, and knowledge of domain experts. Ensure that the dataset covers a wide range of action types and variations to improve the generalization ability of the model.

[0043] Step 2: Receive text input and generate combat skeletal animations through model reasoning. In actual applications, users input specific combat action description text. The model receives these text inputs and converts the text into a format that the model can understand through the built-in natural language processing module. Then, the generation model (such as the VAE model) uses these text information for reasoning to generate the corresponding combat skeletal animation sequence. Among them, the training process structure of the generation model is as follows: Figure 2 The process of generating skeletal animation from the trained model includes converting the text description into a feature vector in the latent space and generating skeletal animation through a decoder or generator. The formula for generating skeletal animation for the first time based on the text description is as follows:

[0044]

[0045] in represents the kth description of category y, It represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and Animation-Gen represents the reasoning process of the generative model generating skeletal animation by processing the corresponding description.

[0046] Step 3: The text-skeleton animation alignment model outputs feedback results. The generated skeletal animation needs to be aligned with the original text description to ensure the accuracy and consistency of the action. This step includes using the alignment model to match the generated skeletal animation with the text description. The alignment model evaluates whether the action meets the expectations of the description by comparing the degree of match between the generated animation and the description. The feedback results include the gap and shortcomings between the generated animation and the expected action. The formula for aligning the generated skeletal animation with the text is expressed as follows:

[0047]

[0048] This formula describes the generated skeletal animation, using the skeletal animation-text alignment capability of MotionCLIP. The process of aligning with D. Among them, p asess Represents the generated skeletal animation The probability distribution of the degree of alignment with the text description of each category. T represents the transpose operator in matrix operations. MotionCLIP anim MotionCLIP is a skeletal animation encoder that represents skeletal animation-text to its model. text represents the text encoder, D represents the set of all description texts describing the skeletal animation of this category, represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and k represents the kth description.

[0049] Step 4: Iteratively optimize the generated results based on the feedback results. According to the feedback from the alignment model, analyze the problems and deficiencies in the generated animation. By comparing the similarity between the generated animation and the description of each category, determine the category that best matches the generated animation; this process usually uses feature alignment and similarity measurement methods to evaluate the fit between the generated animation and the category description. Next, according to the results of comparing the similarity between the generated animation and the description of each category, and determining the category that best matches the generated animation, optimize the original text input to make it more accurately reflect the required action features and details. This optimization may include adjusting the specific details of the action description, increasing the accuracy of the description, or restating the action requirements. The optimized text input will be provided to the generation model as a new input to generate skeletal animation again. In this way, the accuracy and quality of the generated animation can be improved by precisely adjusting the text description without retraining the model. This process is repeated, and the specific execution flow is as follows Figure 3 As shown, until the generated skeletal animation better matches the optimized text description. This process is iterative until the generated combat animation meets the expected standards. During the optimization iteration process, the alignment optimization process of the skeletal animation is shown in the following formula:

[0050]

[0051] where p MotionCLIP represents the probability derived from the MotionCLIP model. This formula represents the output skeletal animation i test , the animation encoding module of the MotionCLIP model is matched and aligned with all pre-stored text category features. The formula represents the matching probability calculation process for one of the features.

[0052] Step 5: The generated results meet the standard output results. The optimized combat skeleton animation is output and finally verified. This verification step ensures that the generated animation not only meets the requirements of the text description, but also meets the animation quality standards such as smoothness, realism, and accuracy of action. If the output results meet the expected standards, they can be used in actual games or applications.

[0053] like Figure 4 As shown, the present invention also provides a system for generating combat skeleton animation based on text feature extraction, and the system includes the following modules:

[0054] Text description dataset construction module: collect and organize data pairs containing text descriptions and their corresponding skeletal animations, annotate the data, and ensure that there is an accurate correspondence between the text descriptions and the animations.

[0055] Text feature extraction module: extracts useful information from text descriptions, such as word embedding, part-of-speech tagging, syntactic structure, etc., and converts the extracted information into data information that the model can process and understand.

[0056] Skeletal animation feature extraction module: Extract key motion features from skeletal animation, such as key frames, joint positions, speed and acceleration, etc. Convert the extracted features into data information that the model can process and understand so as to match them with text features.

[0057] Skeletal animation-text feature matching module: responsible for matching the extracted text features with the skeletal animation features, and learning the mapping relationship from text description to skeletal animation.

[0058] Matching result feedback optimization module: Based on the matching results, the consistency between the generated animation and the text description is evaluated. For the generated results that deviate too much from the text description, the category of the animation is determined, and the relevant information and direction for optimizing the text description are given based on the feedback information, and then the optimized text description is transmitted for the iteration of the generated results.

[0059] Animation processing and output module: Receives the generated skeletal animation data and performs necessary post-processing, such as smoothing motion, adjusting time series, etc. Converts the processed animation into a usable format, such as a video file or pre-stored skeletal motion data.

[0060] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative work on the basis of the technical solution disclosed in the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating combat action skeleton animation based on text feature extraction, characterized in that: The method comprises the following steps: S1: Construct a text dataset that describes the characteristics of skeletal animation in multiple dimensions; S2: receiving text input and generating combat skeletal animation through model reasoning; S3: align the generated skeletal animation with the original text description and output feedback results; S4: Iteratively optimize and generate results based on the feedback results; Analyze the problems and deficiencies in the generated animation based on the feedback from the alignment model; By comparing the similarity between the generated animation and the descriptions of each category, the category that best matches the generated animation is determined, and the original text input is optimized to more accurately reflect the required action features and details; S5: For the generated results that meet the standards, the corresponding combat action skeleton animation is output as the output result.

2. The method for generating combat action skeleton animation based on text feature extraction according to claim 1, characterized in that: In step S2, the formula for generating the skeletal animation for the first time according to the text description is as follows: in represents the kth description of category y, It represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and Animation-Gen represents the reasoning process of the generative model generating skeletal animation by processing the corresponding description.

3. The method for generating combat action skeleton animation based on text feature extraction according to claim 1, characterized in that: The formula for aligning the skeletal animation and text generated in step S3 is as follows: Among them, p asess Represents the generated skeletal animation The probability distribution of the degree of alignment with the text description of each category. T represents the transpose operator in matrix operations. MotionCLIP anim MotionCLIP is a skeletal animation encoder that represents skeletal animation-text to its model. text represents the text encoder, D represents the set of all description texts describing the skeletal animation of this category, represents the corresponding skeletal animation generated by the skeletal animation generative model based on the description, and k represents the kth description.

4. The method for generating combat action skeleton animation based on text feature extraction according to claim 1, characterized in that: In the step S4, when comparing the similarity between the generated animation and each category description, feature alignment and similarity measurement methods are used to evaluate the fit between the generated animation and the category description.

5. The method for generating combat action skeleton animation based on text feature extraction according to claim 4, characterized in that: The alignment formula of the initially generated skeletal animation and the corresponding text category in step S4 is expressed as follows: where p MotionCLIP represents the probability derived from the MotionCLIP model. This formula represents the output skeletal animation i test , the animation encoding module of the MotionCLIP model is matched and aligned with all pre-stored text category features. The formula represents the matching probability calculation process for one of the features.

6. The method for generating combat action skeleton animation based on text feature extraction according to claim 1, characterized in that: In step S5, the generation result that meets the standards specifically includes: the generated animation meets the requirements of the text description and satisfies the animation quality standards, including fluency, realism and accuracy of movements.

7. A combat action skeleton animation generation system based on text feature extraction, characterized in that: The system consists of the following modules: Text description dataset construction module: used to collect and organize data pairs containing text descriptions and their corresponding skeletal animations, annotate the data, and ensure that there is an accurate correspondence between the text descriptions and the animations; Text feature extraction module: used to extract useful information from text descriptions, including word embedding, part-of-speech tagging, and syntactic structure; convert the extracted information into data information that the model can process and understand; Skeletal animation feature extraction module: used to extract key motion features from skeletal animation, including key frames, joint positions, speed and acceleration; convert the extracted features into data information that the model can process and understand, so as to match them with text features; Skeletal animation-text feature matching module: responsible for matching and matching the extracted text features with the skeletal animation features, and learning the mapping relationship from text description to skeletal animation; Matching result feedback optimization module: used to evaluate the consistency between the generated animation and the text description based on the matching results; for the generated results that deviate too much from the text description, determine the category to which the animation belongs, and provide relevant information and directions for optimizing the text description based on the feedback information, and then transmit the optimized text description for iterative generation of results; Animation processing and output module: used to receive the generated skeletal animation data and perform post-processing, including smoothing motion and adjusting time series; converting the processed animation into a usable format, including video files or pre-stored skeletal motion data.

Citation Information

Patent Citations

  • Semantic-driven two-dimensional animation automatic generation method

    CN111340920A

  • Animation generation method and system based on comparative learning

    CN116485962A

  • Human body action generation method, device and product based on natural language description

    CN118196242A

Cited By

  • Digital character skeleton animation generation method and device suitable for Unity development

    CN120807727A