Cartoon action real-time generation method and device

Abstract

The invention discloses a cartoon action real-time generation method and device, and relates to the technical field of animation production, and the method comprises the steps: collecting action data through an action capture engine; processing the action data according to requirements of different fields to form scenarized cartoon action templates suitable for various different fields, and forming a model library by all the scenarized cartoon action templates; selecting a corresponding scenarized cartoon action template in a model library according to the field selected by the user; acquiring real-time data input by a user, and converting the real-time data into corresponding action parameters; the action parameters are applied to a scenarized cartoon action template; and rendering the plurality of scenarized cartoon action templates according to a sequence to generate cartoon actions. By adopting the method provided by the invention, the cartoon action with high expressive force can be generated in real time, the action details can be automatically optimized according to scene requirements, and the animation production efficiency and the user experience are remarkably improved.

Description

Technical Field

[0001] This application relates to the technical field of animation production, and particularly relates to a method and device for real-time generation of cartoon actions. Background Art

[0002] Existing real-time cartoon animation generation technologies (such as Unity engine, motion capture devices, etc.) are widely used in games, movies, and virtual character interactions, and can achieve basic action design through preset animation libraries or manual adjustment. However, there are still significant technical gaps in dynamically generating coherent and logical cartoon action sequences based on real-time user inputs (such as voice commands, gesture capture, emotion recognition, etc.). Existing technologies are difficult to automatically call or generate adapted action models (such as walking, jumping, expression changes, etc.) based on real-time input data and achieve intelligent integration of actions and scenes, resulting in low animation generation efficiency and lack of flexibility. These defects are specifically reflected in the following aspects: The disconnection between real-time data parsing and action generation. Existing technologies cannot parse multi-modal data (such as voice, gestures, biological signals) input by users in real time and dynamically generate high-precision animation sequences, and still rely on manual intervention or pre-programmed scripts.

[0003] Insufficient cross-domain scene adaptation ability. Existing systems are difficult to integrate the requirements of different scenarios such as games, movies, and education, and cannot automatically call adapted action models or scene components according to specific application scenarios.

[0004] High operation complexity. Mainstream technologies (such as Unity + motion capture + real-time rendering) still rely on cumbersome manual modeling and data mapping processes, which require high technical capabilities from users and limit the popularization and application scope of the technologies. Summary of the Invention

[0005] Embodiments of this application provide a method and device for real-time generation of cartoon actions to solve the problems in existing real-time cartoon action generation technologies, such as insufficient utilization of real-time multi-modal data, insufficient cross-domain scene adaptation ability, and high usage requirements.

[0006] On the one hand, embodiments of this application provide a method for real-time generation of cartoon actions, including: Collect action data through a motion capture engine; Process the action data according to the requirements of different fields to form scene-based cartoon action templates applicable to multiple different fields, and all the scene-based cartoon action templates form a model library; Select the corresponding scene-based cartoon action template from the model library according to the field selected by the user; Obtain the real-time data input by the user and convert the real-time data into corresponding action parameters; Apply the action parameters to the scene-based cartoon action templates; Render multiple scene-based cartoon action templates in sequence to generate cartoon actions.

[0007] In a possible implementation, the real-time data is voice data. After converting the voice data into corresponding text data, the action parameters are obtained by extracting keywords from the text data.

[0008] In a possible implementation, the real-time data is a 3D action. The 3D action includes the positions and movement data of multiple key points. The positions and movement data of each key point in the 3D action are extracted to obtain the action parameters.

[0009] In a possible implementation, the 3D action is an action corresponding to the part in the cartoon action to be generated input by the user through a 3D sensor.

[0010] In a possible implementation, the 3D action is an action for adjusting the state of a specific part in the scene-based cartoon action template input by the user through a 3D sensor.

[0011] In a possible implementation, the UE (Unreal Engine) real-time rendering technology is used to render multiple scene-based cartoon action templates.

[0012] On the other hand, an embodiment of the present application also provides a device for real-time generation of cartoon actions, including: An action capture engine for collecting action data; A model library construction module for processing the action data according to the requirements of different fields to form scene-based cartoon action templates applicable to multiple different fields, and all the scene-based cartoon action templates constitute a model library; A template selection module for selecting corresponding scene-based cartoon action templates in the model library according to the field selected by the user; A parameter acquisition module for acquiring the real-time data input by the user and converting the real-time data into corresponding action parameters; An action application module for applying the action parameters to the scene-based cartoon action templates; An action generation module for rendering multiple scene-based cartoon action templates in sequence to generate cartoon actions.

[0013] A method and device for real-time generation of cartoon actions in the present application have the following advantages: Through the method of this application, not only can highly expressive cartoon actions be generated in real time, but also the action details can be automatically optimized according to the scene requirements (such as physical collisions and lighting adaptation), significantly improving the animation production efficiency and user experience. After combining with AI (Artificial Intelligence) technology, it can be extended to emerging fields such as virtual idols and metaverse interactions, promoting the popularization and intelligence of animation generation technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0015] Figure 1 It is a flowchart of a method for real-time generation of cartoon actions provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0017] Figure 1 It is a flowchart of a method for real-time generation of cartoon actions provided by an embodiment of the present application. An embodiment of the present application provides a method for real-time generation of cartoon actions, including the following steps: S100, collecting action data through an action capture engine.

[0018] Exemplarily, the user can pre-establish a three-dimensional cartoon model through three-dimensional modeling. These cartoon models can be established by simulating the images of people, animals, plants, etc. After establishing the three-dimensional cartoon model, the user also needs to set action matching points at each key position on it. These action matching points are usually the positions where the three-dimensional cartoon model can move. For example, in a three-dimensional cartoon model established by simulating a person, the action matching points can be set on the head, limbs, body, and face.

[0019] Then the user wears multiple 3D sensors on the body, such as a three-axis gyroscope, etc. These 3D sensors can monitor the position and movement data of specific parts of the user's body, such as limbs, torso and even face. These position and movement data represent the status of multiple key points on the user's body, that is, the position of the 3D sensor. After the key points and action matching points are matched and bound one by one, the user can make various actions in the 3D space, such as walking, running, jumping and different expressions. After the 3D sensor collects data, the position and movement data of each key point are extracted and applied to the corresponding action matching points of the 3D cartoon model, so that each action matching point in the 3D cartoon model changes its state according to the position and movement data of the key point, and a basic action data can be formed.

[0020] S110, processing the action data according to the requirements of different fields to form scenario-based cartoon action templates applicable to a variety of different fields, and all scenario-based cartoon action templates constitute a model library.

[0021] For example, in different fields such as games, film and television, and education, the requirements for three-dimensional cartoon models are also different. Therefore, in order to meet the needs of different fields, it is necessary to generate corresponding scenario-based cartoon action templates according to the needs of different fields.

[0022] During the processing, the process of generating action templates in different fields through basic action data is as follows: 1. Data preprocessing.

[0023] 1.1 Perform denoising on the raw motion data collected by the motion capture engine, such as filtering to eliminate sensor jitter noise.

[0024] 1.2 Standardize the coordinate system and time axis of motion data to ensure the compatibility of data from different sources.

[0025] 1.3 Divide continuous action data into independent action units, such as "walking", "waving", and "jumping".

[0026] 2. Extract feature parameters from action units.

[0027] 2.1 Spatial features: joint angles, motion trajectories, and key point displacements.

[0028] 2.2 Time characteristics: action duration, speed curve, acceleration peak.

[0029] 2.3 Physical characteristics: collision volume, center of gravity offset.

[0030] 3. Feature parameter adjustment.

[0031] 3.1 Store the characteristic parameters as structured data, such as JSON (JavaScript Object Notation) or matrix form.

[0032] 3.2 Adopt an interpolation algorithm to smooth the action curve and optimize the micro-expression transition.

[0033] 3.3 Add light and shadow matching parameters, such as real-time interaction between actions and scene light sources.

[0034] 4. Bind the adjusted action parameters to the preset 3D model bone system to generate a standardized template.

[0035] 4.1 Scene element fusion and template encapsulation.

[0036] 4.2 According to the requirements of the domain scene, dynamically associate the action template with scene components, such as game special effects, teaching icons, and film and television backgrounds.

[0037] 4.3 Encapsulate the template into a callable module, including the following metadata: Action parameter range (such as speed threshold, angle limit); Compatibility identifier (such as supporting UE engine rendering, Unity plugin interface).

[0038] 5. Generate model actions.

[0039] 6. Template verification and iterative optimization 6.1 Simulate the performance of the template in the target domain in a test environment.

[0040] 6.2 Collect user feedback and use regression analysis to optimize and adjust the rule base.

[0041] 6.3 Establish an automated update mechanism to synchronize the optimized template to the model library.

[0042] Through the above process, the basic action data can be efficiently converted into a scene-based template adaptable to multiple domains, achieving "one capture, multi-scene reuse" and meeting the core goal of "cross-domain intelligent adaptation".

[0043] S120. Select the corresponding scene-based cartoon action template in the model library according to the domain selected by the user.

[0044] Exemplarily, after the model library is established, the user can engage in the work of specific development tasks. After the development task is determined, the field to which the cartoon action belongs is also determined. Therefore, the user can select a specific field, and then the computer will select a matching scenario-based cartoon action template according to the selected field. It should be understood that during the process of establishing the model library, what the user inputs through the motion capture engine is only the basic actions. However, in actual development tasks, the generated cartoon actions may be relatively complex. Therefore, it is also necessary to further adjust the actions of the scenario-based cartoon action template by collecting the data input by the user to meet the requirements of the development task.

[0045] S130, obtain the real-time data input by the user, and convert the real-time data into corresponding action parameters.

[0046] Exemplarily, according to the different APIs (Application Programming Interfaces) accessed by the user, the real-time data can be divided into voice data and three-dimensional actions. The voice data can be input by the user through a microphone, while the three-dimensional actions are input by three-dimensional sensors worn on the user's body, such as a three-axis gyroscope, etc.

[0047] When collecting voice data using a microphone, speech recognition technology can be used to convert the voice data into corresponding text data, and then the action parameters can be obtained by extracting the keywords in the text data. These keywords usually include the names of the object and the action. The object is the name of a specific part of the scenario-based cartoon action template that the user needs to control, such as the foot, hand, etc., and the action is an instruction to adjust the state of these specific parts, such as rotation, upward movement, etc.

[0048] When collecting three-dimensional actions using a three-dimensional sensor, the three-dimensional actions include the position and movement data of multiple key points. Extract the position and movement data of each key point in the three-dimensional actions to obtain the action parameters. The user can select one of the two control methods according to the needs to adjust the scenario-based cartoon action template using the three-dimensional actions.

[0049] The first of them is similar to the method of collecting action data through the motion capture engine. In this process, the part of the user wearing the three-dimensional sensor is the same as the part in the scenario-based cartoon action model. For example, it is both the hand. The action corresponding to the part in the cartoon action to be generated input by the user through the three-dimensional sensor can be used as the three-dimensional action. Extract the position and movement data in this three-dimensional action to obtain the action parameters.

[0050] The second method is to input motion parameters through a 3D sensor worn by the user at a specific part, usually the hand, and use these motion parameters to adjust the state of the scene-based cartoon motion template. At this time, the position and movement data in the 3D motion input by the user through a device such as a glove integrated with multiple 3D sensors are the motion parameters. Although the first method can simulate the user's body movements to adjust the scene-based cartoon motion template, the cartoon movements and real human movements are not exactly the same. Many cartoon movements cannot be made by the human body. At this time, the user needs to wear a glove to adjust the state of a specific part of the 3D cartoon model in the scene-based cartoon motion template in the form of AR (augmented reality) or VR (virtual reality) so that this part can make movements that cannot be achieved by the human body.

[0051] S140, Apply the motion parameters to the scene-based cartoon motion template.

[0052] Exemplarily, when using voice data, select the part in the scene-based cartoon motion template corresponding to the object in the keyword, and adjust the position of this part according to the motion instruction, then the effect of generating cartoon motions according to the voice data can be achieved.

[0053] When using the first method of 3D motion, although the scene-based cartoon motion template already has some basic motions, these basic motions either do not meet the requirements of the actual development task or cannot meet all the requirements due to the small number and types. Therefore, the user can input 3D motions to adjust some of the motions in the scene-based cartoon motion template to meet the requirements of the actual development task, or regenerate the motions that are not available in the scene-based cartoon motion template, such as connecting the motions before the start and after the end of the scene-based cartoon motion template, so that the finally generated scene-based cartoon motion template can be spliced to form coherent cartoon motions.

[0054] When using the second method of 3D motion, it is necessary to display the scene-based cartoon motion template and the 3D motion on the user interface so that the user can understand the relative position between the two and adjust the motion based on this relative position, so that a specific part of the scene-based cartoon motion template adjusts its state according to the user's 3D motion. For example, after the user wears gloves integrated with a total of 10 3D sensors on both hands, 10 points will be displayed on the user interface, and the positions and distances of these 10 points correspond to the user's actual motions. After the user adjusts the positions of these 10 points to perform operations such as rotation or movement on a specific part of the scene-based cartoon motion template, the purpose of adjusting the state is achieved.

[0055] S150, Render multiple scene-based cartoon motion templates in sequence to generate cartoon motions.

[0056] Exemplarily, after generating multiple scenario-based cartoon action templates according to development tasks, these scenario-based cartoon action templates are usually continuous in time. For example, actions such as walking, running, and jumping that occur successively in the same scenario. Moreover, after applying the action parameters input by the user to the scenario-based cartoon action templates, the scenario-based cartoon action templates of two adjacent actions before and after are also coherent. Therefore, these scenario-based cartoon action templates can be spliced in sequence to form a preliminary cartoon action, and then rendered to obtain the final cartoon action.

[0057] Specifically, the UE real-time rendering technology can be used to render multiple scenario-based cartoon action templates.

[0058] The embodiment of the present application also provides a device for the method of real-time generation of cartoon actions, including the following modules: An action capture engine for collecting action data; A model library construction module for processing action data according to the requirements of different fields to form scenario-based cartoon action templates applicable to multiple different fields, and all the scenario-based cartoon action templates constitute a model library; A template selection module for selecting corresponding scenario-based cartoon action templates in the model library according to the field selected by the user; A parameter acquisition module for acquiring the real-time data input by the user and converting the real-time data into corresponding action parameters; An action application module for applying the action parameters to the scenario-based cartoon action templates; An action generation module for rendering multiple scenario-based cartoon action templates in sequence to generate cartoon actions.

[0059] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0060] Obviously, those skilled in the art can make various changes and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A method for real-time generation of cartoon actions, characterized in that: include: Collect motion data through motion capture engine; Processing the action data according to the requirements of different fields to form scenario-based cartoon action templates applicable to a variety of different fields, all of which constitute a model library; Selecting the corresponding scenario-based cartoon action template in the model library according to the field selected by the user; Acquire real-time data input by the user, and convert the real-time data into corresponding action parameters; Applying the action parameters to the scenario-based cartoon action template; Rendering a plurality of the scenario-based cartoon action templates in sequence to generate cartoon actions.

2. A method for real-time generation of cartoon actions according to claim 1, characterized in that: The real-time data is voice data. After the voice data is converted into corresponding text data, the action parameters are obtained by extracting keywords from the text data.

3. A method for real-time generation of cartoon actions according to claim 1, characterized in that: The real-time data is a three-dimensional action, and the three-dimensional action includes position and movement data of multiple key points. The position and movement data of each key point in the three-dimensional action are extracted to obtain the action parameters.

4. A method for real-time generation of cartoon actions according to claim 3, characterized in that: The three-dimensional action is an action input by the user through a three-dimensional sensor and corresponds to the part in the cartoon action to be generated.

5. A method for real-time generation of cartoon actions according to claim 3, characterized in that: The three-dimensional action is an action input by a user through a three-dimensional sensor to adjust the state of a specific part in the scenario-based cartoon action template.

6. A method for real-time generation of cartoon actions according to claim 1, characterized in that: UE real-time rendering technology is used to render the plurality of scenario-based cartoon action templates.

7. A device using the method for real-time generation of cartoon actions according to any one of claims 1 to 6, characterized in that: include: Motion capture engine, used to collect motion data; A model library construction module is used to process the action data according to the requirements of different fields to form scenario-based cartoon action templates applicable to a variety of different fields, and all the scenario-based cartoon action templates constitute a model library; A template selection module, used to select the corresponding scenario-based cartoon action template in the model library according to the field selected by the user; A parameter acquisition module is used to acquire real-time data input by a user and convert the real-time data into corresponding action parameters; An action application module, used for applying the action parameters to the scenario-based cartoon action template; The action generation module is used to render the plurality of scenario-based cartoon action templates in sequence to generate cartoon actions.

Images