A method and device for real-time generation of cartoon actions

Abstract

The application discloses a cartoon action real-time generation method and device, and relates to the technical field of animation production. The method comprises the following steps: collecting action data through an action capture engine; processing the action data according to the requirements of different fields to form cartoon action templates suitable for various fields, wherein all the cartoon action templates form a model library; selecting corresponding cartoon action templates in the model library according to the field selected by a user; obtaining real-time data input by the user, converting the real-time data into corresponding action parameters; applying the action parameters in the cartoon action templates; and rendering the cartoon action templates in sequence to generate cartoon actions. The method can not only generate cartoon actions with high expressiveness in real time, but also automatically optimize action details according to scene requirements, thereby significantly improving the animation production efficiency and user experience.

Description

Technical Field

[0001] This application relates to the field of animation production technology, and in particular to a method and apparatus for real-time generation of cartoon motion. Background Technology

[0002] Existing real-time cartoon animation generation technologies (such as the Unity engine and motion capture devices) are widely used in games, film and television, and virtual character interaction, enabling basic motion design through preset animation libraries or manual adjustments. However, there is still a significant technical gap in dynamically generating coherent and logical cartoon motion sequences based on real-time user input (such as voice commands, gesture capture, and emotion recognition data). Existing technologies struggle to automatically call or generate suitable motion models (such as walking, jumping, and facial expression changes) based on real-time input data and to achieve intelligent integration of actions with the scene, resulting in low animation generation efficiency and a lack of flexibility. These shortcomings are specifically reflected in the following aspects:

[0003] The disconnect between real-time data parsing and motion generation. Existing technologies cannot parse multimodal data (such as voice, gestures, and biosignals) input by users in real time and dynamically generate high-precision animation sequences based on this data, still requiring manual intervention or pre-programmed scripts.

[0004] The system lacks cross-domain scenario adaptability. Existing systems struggle to integrate the needs of different scenarios such as games, film and television, and education, and cannot automatically invoke appropriate action models or scene components based on specific application scenarios.

[0005] The operation is highly complex. Mainstream technologies (such as Unity + motion capture + real-time rendering) still rely on cumbersome manual modeling and data mapping processes, which require high technical skills from users and limit the popularization and application of the technology. Summary of the Invention

[0006] This application provides a method and apparatus for real-time cartoon motion generation, which addresses the problems of insufficient utilization of real-time multimodal data, insufficient cross-domain scene adaptation capability, and high usage requirements in existing real-time cartoon motion generation technologies.

[0007] On the one hand, embodiments of this application provide a method for real-time generation of cartoon animations, including:

[0008] Motion data is collected using a motion capture engine;

[0009] The motion data is processed according to the requirements of different fields to form scenario-based cartoon motion templates applicable to various fields. All scenario-based cartoon motion templates form a model library.

[0010] Select the appropriate scenario-based cartoon action template from the model library according to the user's chosen field;

[0011] Acquire real-time data from user input and convert the real-time data into corresponding action parameters;

[0012] Apply motion parameters to scenario-based cartoon motion templates;

[0013] Render multiple scene-based cartoon action templates in sequence to generate cartoon actions.

[0014] In one possible implementation, the real-time data is voice data. After converting the voice data into corresponding text data, the action parameters are obtained by extracting keywords from the text data.

[0015] In one possible implementation, the real-time data is a three-dimensional motion, which includes the position and movement data of multiple key points. The position and movement data of each key point in the three-dimensional motion are extracted to obtain the motion parameters.

[0016] In one possible implementation, the 3D motion is the action input by the user through a 3D sensor that corresponds to a part of the cartoon motion to be generated.

[0017] In one possible implementation, 3D motion refers to actions input by the user via a 3D sensor to adjust the state of a specific part of a scened cartoon motion template.

[0018] In one possible implementation, UE (Unreal Engine) real-time rendering technology is used to render multiple scene-based cartoon action templates.

[0019] On the other hand, embodiments of this application also provide a real-time cartoon motion generation device, including:

[0020] Motion capture engine, used to collect motion data;

[0021] The model library building module is used to process motion data according to the requirements of different fields, forming scenario-based cartoon motion templates applicable to various fields. All scenario-based cartoon motion templates constitute the model library.

[0022] The template selection module is used to select appropriate scene-based cartoon action templates from the model library according to the user's selected field.

[0023] The parameter acquisition module is used to acquire real-time data input by the user and convert the real-time data into corresponding action parameters.

[0024] The motion application module is used to apply motion parameters to scene-based cartoon motion templates;

[0025] The motion generation module is used to render multiple scene-based cartoon motion templates in sequence to generate cartoon motions.

[0026] The cartoon motion real-time generation method and apparatus of this application have the following advantages:

[0027] The method described in this application not only enables the real-time generation of highly expressive cartoon animations but also automatically optimizes animation details (such as physical collisions and lighting adaptation) according to scene requirements, significantly improving animation production efficiency and user experience. When combined with AI (artificial intelligence) technology, it can be extended to emerging fields such as virtual idols and metaverse interaction, promoting the democratization and intelligentization of animation generation technology. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart illustrating a method for real-time generation of cartoon animations provided in this application embodiment. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] Figure 1 A flowchart illustrating a real-time cartoon motion generation method provided in this application embodiment. This application embodiment provides a real-time cartoon motion generation method, including the following steps:

[0032] The S100 collects motion data through a motion capture engine.

[0033] For example, users can pre-create 3D cartoon models through 3D modeling. These cartoon models can simulate the images of people, animals, plants, etc. After creating the 3D cartoon model, users also need to set motion matching points at each key position on it. These motion matching points are usually the movable positions of the 3D cartoon model. For example, in a 3D cartoon model that simulates a person, motion matching points can be set on the head, limbs, body, and face.

[0034] Then, the user wears multiple 3D sensors on their body, such as a three-axis gyroscope. These 3D sensors can monitor the position and movement data of specific parts of the user's body, such as limbs, torso, and even the face. This position and movement data represents the state of multiple key points on the user's body, i.e., the positions of the 3D sensors. After mapping and binding these key points to action matching points, the user can perform various actions in 3D space, such as walking, running, jumping, and making different facial expressions. After the 3D sensors collect data, they extract the position and movement data of each key point and apply it to the corresponding action matching point of the 3D cartoon model. This causes each action matching point in the 3D cartoon model to change its state according to the position and movement data of the key points, thus forming basic motion data.

[0035] S110 processes motion data according to the requirements of different fields to form scene-based cartoon motion templates applicable to various fields. All scene-based cartoon motion templates form a model library.

[0036] For example, the requirements for 3D cartoon models vary in different fields such as games, film and television, and education. Therefore, in order to meet the needs of different fields, it is necessary to generate corresponding scene-based cartoon action templates for different fields.

[0037] The process of generating action templates for different domains from basic action data is as follows:

[0038] 1. Data preprocessing.

[0039] 1.1 Denoising the raw motion data collected by the motion capture engine, such as filtering to eliminate sensor jitter noise.

[0040] 1.2 Standardize the coordinate system and time axis of motion data to ensure compatibility with data from different sources.

[0041] 1.3 Segment continuous motion data into independent motion units, such as “walking”, “waving”, and “jumping”.

[0042] 2. Extract feature parameters from the action unit.

[0043] 2.1 Spatial characteristics: joint angles, motion trajectory, and key point displacement.

[0044] 2.2 Time characteristics: duration of motion, velocity curve, peak acceleration.

[0045] 2.3 Physical characteristics: collision volume, center of gravity offset.

[0046] 3. Feature parameter adjustment.

[0047] 3.1 Store feature parameters as structured data, such as JSON (JavaScript Object Notation) or matrix form.

[0048] 3.2 An interpolation algorithm is used to smooth the motion curve and optimize the transition of micro-expressions.

[0049] 3.3 Add lighting and shadow matching parameters, such as real-time interaction between actions and scene light sources.

[0050] 4. Bind the adjusted motion parameters to the preset 3D model skeleton system to generate a standardized template.

[0051] 4.1 Scene element integration and template encapsulation.

[0052] 4.2 Based on the needs of the domain scenario, dynamically associate the action template with scene components, such as game effects, teaching icons, and film and television backgrounds.

[0053] 4.3 The encapsulated template is a callable module, containing the following metadata:

[0054] Range of motion parameters (such as speed threshold, angle limit);

[0055] Compatibility indicators (such as support for UE engine rendering and Unity plugin interfaces).

[0056] 5. Generate model actions.

[0057] 6. Template Validation and Iterative Optimization

[0058] 6.1 Performance of the simulated environment test template in the target domain.

[0059] 6.2 Collect user feedback and use regression analysis to optimize and adjust the rule base.

[0060] 6.3 Establish an automated update mechanism to synchronize the optimized templates to the model library.

[0061] Through the above process, basic motion data can be efficiently converted into scenario-based templates that are adapted to multiple domains, achieving "one-time capture, multiple-scenario reuse" and meeting the core goal of "cross-domain intelligent adaptation".

[0062] S120: Select the corresponding scene-based cartoon action template from the model library according to the user's chosen field.

[0063] For example, after establishing the model library, users can begin working on specific development tasks. Once the development task is determined, the domain to which the cartoon actions belong is also determined. Therefore, users can select a specific domain, and the computer will then select a matching scenario-based cartoon action template according to the selected domain. It should be understood that during the model library establishment process, users only input basic actions through the motion capture engine. However, in actual development tasks, the cartoon actions to be generated may be more complex. Therefore, it is necessary to further adjust the actions of the scenario-based cartoon action templates by collecting user input data to adapt to the needs of the development task.

[0064] S130: Obtain real-time data input by the user and convert the real-time data into corresponding action parameters.

[0065] For example, real-time data can be divided into voice data and 3D motion data depending on the user access API (Application Programming Interface). Voice data can be input by the user through a microphone, while 3D motion data is input by 3D sensors worn on the user's body, such as a three-axis gyroscope.

[0066] When using a microphone to collect voice data, speech recognition technology can be used to convert the voice data into corresponding text data. Then, the action parameters can be obtained by extracting keywords from the text data. These keywords typically contain the names of objects and actions. Objects are the names of specific parts of the scenario-based cartoon action template that the user needs to control, such as feet or hands. Actions are the instructions to adjust the state of these specific parts, such as rotation or upward movement.

[0067] When using 3D sensors to acquire 3D motion data, the 3D motion includes the position and movement data of multiple key points. Extracting the position and movement data of each key point in the 3D motion yields the motion parameters. Users can choose one of two control methods to adjust the scene-based cartoon motion template using 3D motion as needed.

[0068] The first method is similar to the method of collecting motion data through a motion capture engine. In this process, the part of the user wearing the 3D sensor is the same as the part in the scene-based cartoon motion model, such as the hand. The motion input by the user through the 3D sensor that corresponds to the part in the cartoon motion to be generated can be used as the 3D motion. By extracting the position and movement data of the 3D motion, the motion parameters can be obtained.

[0069] The second method involves the user inputting motion parameters through 3D sensors worn on a specific part of the body, typically the hand. These parameters are then used to adjust the state of the scene-based cartoon action template. In this case, the position and movement data of the 3D motion input by the user through a glove or other device integrating multiple 3D sensors constitute the motion parameters. While the first method can simulate the user's body movements to adjust the scene-based cartoon action template, cartoon actions are not entirely the same as real human movements. Many cartoon actions cannot be performed by the human body. In such cases, the user needs to wear gloves to adjust the state of specific parts of the 3D cartoon model in the scene-based cartoon action template using AR (Augmented Reality) or VR (Virtual Reality) to enable those parts to perform actions that are impossible for the human body to perform.

[0070] S140 applies motion parameters to a scene-based cartoon motion template.

[0071] For example, when using voice data, select the part of the scene-based cartoon action template that corresponds to the object in the keyword, and adjust the position of that part according to the action instructions to achieve the effect of generating cartoon actions based on voice data.

[0072] When using the first method of 3D animation, although the scene-based cartoon animation template already has some basic actions, these basic actions either do not meet the needs of the actual development task, or the number and types are too few to meet all the needs. Therefore, some actions in the scene-based cartoon animation template can be adjusted by the user inputting 3D actions to meet the needs of the actual development task, or actions that are not available in the scene-based cartoon animation template can be regenerated, such as connecting the actions before and after the scene-based cartoon animation template, so that the final generated scene-based cartoon animation template can be spliced ​​to form a coherent cartoon animation.

[0073] The second method of 3D animation requires displaying a scene-based cartoon animation template and the 3D animation on the user interface. This allows the user to understand the relative positions of the two and adjust the animation based on these positions, so that a specific part of the scene-based cartoon animation template adjusts its state according to the user's 3D movements. For example, when a user wears gloves integrating a total of 10 3D sensors, these 10 points will be displayed on the user interface. The positions and distances of these 10 points correspond to the user's actual movements. By adjusting the positions of these 10 points to rotate or move specific parts of the scene-based cartoon animation template, the user achieves the desired state adjustment.

[0074] S150 renders multiple scene-based cartoon action templates in sequence to generate cartoon actions.

[0075] For example, after generating multiple scenario-based cartoon action templates according to the development task, these scenario-based cartoon action templates are usually continuous in time, such as walking, running and jumping actions that occur one after another in the same scene. Moreover, after applying the action parameters input by the user to the scenario-based cartoon action templates, the scenario-based cartoon action templates of two adjacent actions are also continuous. Therefore, these scenario-based cartoon action templates can be spliced ​​together in sequence to form preliminary cartoon actions, and then rendered to obtain the final cartoon action.

[0076] Specifically, UE real-time rendering technology can be used to render multiple scene-based cartoon action templates.

[0077] This application also provides an apparatus for a real-time cartoon motion generation method, comprising the following modules:

[0078] Motion capture engine, used to collect motion data;

[0079] The model library building module is used to process motion data according to the requirements of different fields, forming scenario-based cartoon motion templates applicable to various fields. All scenario-based cartoon motion templates constitute the model library.

[0080] The template selection module is used to select appropriate scene-based cartoon action templates from the model library according to the user's selected field.

[0081] The parameter acquisition module is used to acquire real-time data input by the user and convert the real-time data into corresponding action parameters.

[0082] The motion application module is used to apply motion parameters to scene-based cartoon motion templates;

[0083] The motion generation module is used to render multiple scene-based cartoon motion templates in sequence to generate cartoon motions.

[0084] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0085] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for real-time generation of cartoon action, characterized in that, The method comprises the following steps: capturing motion data by a motion capture engine; processing the motion data according to requirements of different fields to form scenario-based cartoon motion templates suitable for multiple different fields, all the scenario-based cartoon motion templates forming a model library; selecting corresponding scenario-based cartoon motion templates in the model library according to a field selected by a user; obtaining real-time data input by the user, and converting the real-time data into corresponding motion parameters; applying the motion parameters in the scenario-based cartoon motion templates; rendering multiple scenario-based cartoon motion templates in sequence to generate cartoon motion. In the step of processing the motion data according to requirements of different fields to form scenario-based cartoon motion templates suitable for multiple different fields, the method comprises the following steps: data preprocessing: performing denoising processing on original motion data captured by the motion capture engine, standardizing the coordinate system and time axis of the motion data, and dividing continuous motion data into independent motion units; extracting feature parameters in the motion units: extracting spatial features, time features and physical features; adjusting the feature parameters: storing the feature parameters as structured data, smoothing motion curves by using an interpolation algorithm, and adding light and shadow matching parameters; binding the adjusted motion parameters to a preset three-dimensional model skeleton system to generate a standardized template; generating model motion; template verification and iterative optimization: simulating an environment to test the performance of the template in a target field, collecting user feedback, optimizing and adjusting a rule library by using regression analysis, establishing an automatic updating mechanism, and synchronizing the optimized template to the model library.

2. The method according to claim 1, wherein, The real-time data is voice data, and the motion parameters are obtained by extracting keywords in text data converted from the voice data.

3. The method of claim 1, wherein, The real-time data is three-dimensional motion, and the motion parameters are obtained by extracting position and movement data of multiple key points in the three-dimensional motion.

4. The method of claim 3, wherein, The three-dimensional motion is an action corresponding to a part in a cartoon motion to be generated, which is input by a user through a three-dimensional sensor.

5. The method of claim 3, wherein, The three-dimensional motion is an action for adjusting the state of a specific part in the scenario-based cartoon motion template, which is input by a user through a three-dimensional sensor.

6. The method of claim 1, wherein, The multiple scenario-based cartoon motion templates are rendered by using UE real-time rendering technology.

7. A device for applying the method for real-time generation of cartoon motion according to any one of claims 1 to 6, characterized in that, The method comprises the following steps: capturing motion data by a motion capture engine; processing the motion data according to requirements of different fields to form scenario-based cartoon motion templates suitable for multiple different fields, all the scenario-based cartoon motion templates forming a model library; selecting corresponding scenario-based cartoon motion templates in the model library according to a field selected by a user; obtaining real-time data input by the user, and converting the real-time data into corresponding motion parameters; applying the motion parameters in the scenario-based cartoon motion templates; rendering multiple scenario-based cartoon motion templates in sequence to generate cartoon motion.

Images