Role animation-oriented AI-driven facial expression migration and enhancement method

Abstract

The invention belongs to the technical field of character animation, and discloses an AI-driven facial expression migration and enhancement method for character animation. Comprising the steps of obtaining source actor facial expression sequence information, and performing preprocessing and feature extraction to obtain an expression feature data set; performing expression analysis and sentiment analysis to obtain a high-dimensional expression description vector and a sentiment-driven expression model; obtaining a role expression mapping rule and an initial migration expression sequence based on facial anatomy analysis and topology corresponding construction; carrying out stylized enhancement and detail optimization on the initial migration expression sequence to obtain an enhanced expression sequence; and constructing a rendering parameter strategy, and carrying out illumination rendering and frame rate synthesis to obtain a movie expression animation. According to the method, the micro-expression features are accurately captured, and collaborative optimization of expression reality and role personalization is also realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of character animation, more particularly, the present application relates to an AI-driven facial expression migration and enhancement method for character animation. BACKGROUND

[0002] In the field of character animation, facial expressions are key elements for conveying emotions and shaping character personalities. Traditional character expression animation mainly relies on animators to manually design key frames and intermediate frames, which is labor-intensive and the results often depend on the experience and skills of animators. With the increasing demand for realism in character expressions in the film and game industries, expression migration technology based on motion capture has gradually become a hot topic in the industry.

[0003] Existing expression capture and migration technologies mainly include marker-based capture methods and markerless capture methods based on computer vision. Marker-based methods track specific marker points attached to the actor's face, which have high accuracy but limited number of marker points, making it difficult to capture micro-expression details, and the marker points themselves may affect the actor's natural performance. Markerless methods based on computer vision do not interfere with the actor's performance, but have poor stability under complex lighting conditions and insufficient capture accuracy for small expression areas such as eye corners and laugh lines. In terms of expression migration, existing technologies mainly use direct mapping or parameterized mapping methods. Direct mapping applies the source actor's facial deformation directly to the target character, which is simple but ignores the character's individual characteristics. Parameterized mapping methods achieve migration by extracting expression parameters, but often lack a deep understanding of expression semantics, resulting in distorted emotional expression or insufficient exaggeration. In particular, when the source actor and the target character have significant differences in facial structure, existing migration methods struggle to ensure the naturalness of the expression and the accuracy of the emotional expression.

[0004] In addition, existing expression migration systems rarely consider the individual characteristics of the character and the artistic style requirements of the film. When the same expression is applied to characters with different personality characteristics, there is a lack of targeted adjustment, making the expression performance mismatched with the character setting, reducing the credibility and expressiveness of the character. And usually use uniform rendering parameters, it is difficult to optimize for different expression states, it is difficult to effectively enhance the visual expressiveness of the expression, especially in the rendering of micro-expression details and the emphasis on key areas of emotional expression.

[0005] In view of the above, the present application proposes an AI-driven facial expression migration and enhancement method for character animation to solve the above problems. SUMMARY

[0006] In order to overcome the above-mentioned defects of the prior art, in order to achieve the above-mentioned purpose, the present application provides the following technical solutions: AI-driven facial expression transfer and enhancement methods for character animation include: Step 1: Obtain the facial expression sequence information corresponding to the source actor, and preprocess and extract features from the obtained facial expression sequence information to obtain the corresponding expression feature dataset; Step 2: Based on the obtained facial expression feature dataset, perform facial expression analysis and sentiment analysis to obtain the corresponding high-dimensional facial expression description vectors, and perform semantic mapping and temporal correlation to obtain the corresponding sentiment-driven facial expression model; Step 3: Based on the emotion-driven expression model, perform facial anatomy analysis and topological correspondence construction to obtain the corresponding character expression mapping rules, and perform personalized feature extraction and expression adaptive transfer to obtain the corresponding initial transferred expression sequence; Step 4: Stylize and refine the initial transferred expression sequence to obtain the corresponding enhanced expression sequence; Step 5: Construct a rendering parameter strategy; and based on it, perform lighting rendering and frame rate synthesis on the enhanced facial expression sequence to obtain the corresponding movie facial expression animation.

[0007] Furthermore, the process of acquiring facial expression sequence information includes: The system acquires the facial structure of the source actors during film production and configures facial feature markers accordingly. Based on a pre-set expression capture unit, it performs multi-view expression capture of the source actors' facial expressions with configured facial feature markers to obtain corresponding multi-view expression image sequences. Simultaneously, it performs three-dimensional structural scanning of the source actors' facial structure to obtain corresponding facial depth information. Based on facial feature markers, the collected facial depth information and multi-view expression image sequences are fused and aligned to obtain the corresponding initial three-dimensional expression data. The head posture data and neck movement data of the source actor are captured, and the captured head posture data and neck movement data are spatiotemporally aligned with the corresponding initial 3D facial expression data. The spatiotemporally aligned initial 3D facial expression data, head posture data and neck movement data are then summarized to obtain the corresponding facial expression sequence information.

[0008] Furthermore, the process of obtaining the facial expression feature dataset includes: The collected facial expression sequence information is preprocessed; and based on the preprocessed facial expression sequence information, the anatomical features of the source actor's facial structure are analyzed to obtain the corresponding facial key points and the distribution of facial muscles. Facial segmentation was performed on the source actor's facial structure based on facial key points to obtain several anatomical structural regions. Based on the distribution of facial muscles, the anatomical features corresponding to each anatomical structural region and the connection relationships between different anatomical structural regions are obtained. Construct corresponding facial feature maps based on the obtained anatomical features and connectivity relationships; Based on the optical flow algorithm, the corresponding facial key points are tracked to obtain the corresponding key point temporal data; based on the key point temporal data, the facial feature map is optimized and analyzed to obtain a simplified facial feature map. Temporal features are extracted from the obtained key point temporal data to obtain the corresponding facial temporal features; Based on the aforementioned facial temporal features, and combined with the simplified facial feature map, key facial expression recognition and feature extraction are performed on the facial expression sequence information to obtain the corresponding expression feature vector set; The obtained facial expression feature vector set is decomposed into basic facial expression units to obtain the corresponding facial expression primitive set; the facial expression primitive set includes several basic facial expression units. A 3D facial model corresponding to the source actor is constructed based on facial expression sequence information and facial feature maps; Based on the aforementioned expression primitive set and 3D facial model, expression description models at different levels are generated; and based on the expression description models, corresponding multi-scale expression sets are constructed. Redundant information is filtered and key information is enhanced from the obtained multi-scale expression set to obtain the corresponding expression feature dataset.

[0009] Furthermore, the process of obtaining the high-dimensional facial expression description vector includes: Based on the facial expression feature dataset, basic emotion decomposition and complex emotion recognition are performed on the facial expressions of the source actors to obtain the corresponding emotion expression mapping relationship; the emotion expression mapping relationship includes basic emotion units and expression combination patterns; Based on the emotion expression mapping relationship, the basic expression units in the expression feature dataset are annotated with emotional semantics to obtain the corresponding emotionally annotated expression set; Based on the simplified facial features and the collinear relationship between various expression elements in the emotionally labeled expression set; the expression elements include basic expression units and basic emotion units; An initial facial expression semantic graph is constructed based on the obtained co-occurrence relationships and sentiment-annotated facial expression set; and temporal and sentiment analyses are performed on the constructed initial facial expression semantic graph to obtain the corresponding high-dimensional facial expression description vectors.

[0010] Furthermore, the process of constructing an emotion-driven facial expression model includes: We perform affective semantic space mapping and affective parameterization on high-dimensional facial expression description vectors to obtain the corresponding facial expression control model; based on the facial expression control model, we perform facial expression temporal analysis and affective coherence evaluation to obtain the corresponding facial expression temporal model. Text script alignment and character sentiment analysis are performed on the obtained expression time sequence model to obtain the corresponding emotion-driven expression model.

[0011] Furthermore, the process of obtaining the character expression mapping rules includes: Obtain the 3D mesh model of the target character, and construct a subdivided mesh model of the target character's face based on it; Based on the subdivided mesh model, the anatomical features of the target character's facial structure are analyzed to obtain the key feature points and deformable areas of the target character's face; Based on the anatomical feature analysis results of the source actor and the target character, key rules are mined to obtain the corresponding feature response set; Based on the emotion-driven expression model, structural difference analysis and expression compatibility assessment are performed between the subdivided mesh model and the 3D facial model to obtain the corresponding expression transfer mapping strategy. Based on the expression transfer mapping strategy and feature response set, the expression space is transformed to obtain the corresponding character expression mapping rule set.

[0012] Furthermore, the process of obtaining the initial transferred facial expression sequence includes: The skeletal structure of the target character's subdivided network model is analyzed and parametrically modeled to generate a character skeletal control model; Based on the character skeleton control model, establish the expression-driven skeleton linkage relationship and weight allocation to obtain the corresponding expression-skeleton mapping matrix; based on the expression-skeleton mapping matrix, design fine control parameters for the deformation area within the target character's face, and construct the expression skeleton control framework based on it. Based on the character expression mapping rule set and expression skeleton control framework, the expression mapping transformation from the source actor to the target character is performed to obtain the basic transferred expression; The obtained basic transferred expressions are adjusted to adapt to character features and transition frames are generated to obtain the corresponding initial transferred expression sequence.

[0013] Furthermore, the process of enhancing the acquisition of facial expression sequences includes: The obtained facial expression sequence data were subjected to expression semantic feature extraction and expression dynamic feature extraction respectively to obtain the corresponding expression semantic features and expression dynamic features; An enhanced objective function is constructed based on the aforementioned facial expression semantic features and facial expression dynamic features; and the intensity of the facial expression control parameters within the initial facial expression transfer strategy is adjusted based on the enhanced objective function. Simultaneously, based on facial expression semantic features, the facial expressions within the initial expression transfer strategy are enhanced in terms of expression details, and based on facial expression dynamic features, the temporal changes of facial expressions within the initial expression transfer strategy are optimized in terms of rhythm. The results of enhancement adjustments, facial detail enhancements, and rhythm optimizations are weighted and fused to obtain the corresponding preliminary enhanced transfer facial expression sequence. The initial enhanced facial expression sequence is processed with facial expression stylization and character personalization enhancement to obtain the corresponding enhanced facial expression sequence.

[0014] Furthermore, the process of rendering parameter strategy includes: The enhanced emoji sequence is rendered with layered emojis and detailed control marking is performed to generate multi-level detailed enhanced emoji packs; based on the enhanced emoji packs, fine-grained division of rendering parameters and strategy generation are performed to obtain an initial set of rendering parameter rules; Based on the initial rendering parameter rule set and scene requirements, the state machine modeling and transformation rule definition of the rendering process are carried out to form a dynamic rendering state diagram. Based on the dynamic rendering state diagram, the quality of rendering operations is evaluated and thresholds are set to obtain the corresponding rendering quality control parameters. Rendering resources are segmented and distributed for computation based on rendering quality control parameters, forming a distributed rendering task set. The protocol design and quality verification of the rendering result synthesis process based on the distributed rendering task set are used to ultimately generate the rendering parameter strategy.

[0015] Furthermore, the process of acquiring animated facial expressions in movies includes: The facial expression control parameters within the obtained enhanced facial expression sequence are mapped to the constructed facial expression skeleton control framework to generate corresponding skeletal animation data; Physical constraint checks are performed on the skeletal animation data; simultaneously, deformation weight mapping is constructed based on the subdivided mesh model and the facial bone control framework. Based on the skeletal animation data after deformation weight mapping and physical constraint checking, obtain the mesh deformation animation corresponding to the target character; Subsurface scattering and displacement mapping techniques are applied within the corresponding mesh deformation animation to obtain the corresponding facial deformation data; and the final facial animation is generated based on the facial orientation data and the subdivided network model of the target character. Based on the obtained rendering parameter strategy, the generated final facial animation is rendered and a rendering test is performed; the corresponding movie facial animation is obtained.

[0016] The technical effects and advantages of the AI-driven facial expression transfer and enhancement method for character animation described in this invention are as follows: 1. Through high-precision multi-view facial expression capture and anatomical feature analysis, a precise mapping relationship between the source actor's facial expressions and the target character was effectively established, enabling the AI ​​system to deeply understand and retain the subtle details and emotional expressions of facial expressions; 2. By decomposing facial expression primitives and constructing multi-scale facial expression sets, the system achieves structured expression and hierarchical control of dynamic facial expression features, significantly enhancing its semantic understanding and facial expression generation capabilities in complex emotional expressions; 3. By combining adaptive adjustments to character traits and stylized facial expressions, the generated facial expression transfer sequence can dynamically adjust its expression according to the character settings, achieving a synergistic optimization of facial realism and character personalization; 4. Through layered rendering technology and physical constraint checking mechanisms, the final animated film facial expressions are provided with multi-dimensional quality assurance in terms of clarity of emotional expression, smoothness of facial expression changes, and prominence of artistic expression, thus achieving precise control of animated facial expressions and enhanced visual effects. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the AI-driven facial expression transfer and enhancement method for character animation according to the present invention. Figure 2 This is a schematic diagram of the AI-driven facial expression transfer and enhancement system for character animation according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1; please refer to Figure 1 As shown in this embodiment, the AI-driven facial expression transfer and enhancement method for character animation includes: Step 1: Obtain the facial expression sequence information corresponding to the source actor, and preprocess and extract features from the obtained facial expression sequence information to obtain the corresponding expression feature dataset; Step 2: Based on the obtained facial expression feature dataset, perform facial expression analysis and sentiment analysis to obtain the corresponding high-dimensional facial expression description vectors, and perform semantic mapping and temporal correlation to obtain the corresponding sentiment-driven facial expression model; Step 3: Based on the emotion-driven expression model, perform facial anatomy analysis and topological correspondence construction to obtain the corresponding character expression mapping rules, and perform personalized feature extraction and expression adaptive transfer to obtain the corresponding initial transferred expression sequence; Step 4: Stylize and refine the initial transferred expression sequence to obtain the corresponding enhanced expression sequence; Step 5: Construct a rendering parameter strategy; and based on it, perform lighting rendering and frame rate synthesis on the enhanced facial expression sequence to obtain the corresponding movie facial expression animation.

[0020] It should be further explained that, in the specific implementation process, the acquisition of facial expression sequence information includes: Set up an expression capture unit, which includes a high-precision camera array and motion capture equipment; deploy the expression capture unit in a pre-defined expression capture area, which is also equipped with a lighting control terminal and an environmental monitoring terminal to ensure lighting stability and environmental consistency during the expression capture process; The process involves acquiring the facial structure of source actors during film production and configuring facial feature markers based on this data. These markers include visible light markers and infrared reflection markers. Visible light markers are mainly distributed along the eyebrow contour, eye contour, sides of the nose, lip contour, and facial contour of the source actors. Infrared reflection markers are mainly distributed in the micro-expression areas of the source actors, including the corners of the eyes, nasolabial folds, and forehead wrinkles. The source actors refer to performers whose facial expressions require transfer during film production. Furthermore, based on a high-precision camera array, multi-view facial expressions of the source actor, which is equipped with facial feature markers, are captured to obtain the corresponding multi-view facial expression image sequence; and simultaneously, the facial structure of the source actor is scanned in three dimensions to obtain the corresponding facial depth information. Based on facial feature markers, the collected facial depth information and multi-view expression image sequences are fused and aligned to obtain the corresponding initial three-dimensional expression data. Furthermore, based on motion capture equipment, head pose data and neck movement data of the source actor are captured, and the captured head pose data and neck movement data are spatiotemporally aligned with the corresponding initial 3D facial expression data. The spatiotemporally aligned initial 3D facial expression data, head pose data, and neck movement data are then summarized to obtain the corresponding facial expression sequence information. The spatiotemporal alignment process includes time synchronization and spatial alignment. Time synchronization is used to unify the data within the facial expression sequence information to the same time reference to ensure the temporal consistency between different data frames. Spatial alignment is to perform a unified coordinate system transformation and position adjustment on the data within the facial expression sequence information to ensure data consistency and comparability. It should be further explained that, in the specific implementation process, the acquisition process of the facial expression feature dataset includes: The collected facial expression sequence information is preprocessed, including noise filtering and data cleaning. Noise filtering uses the Kalman filter algorithm to identify and remove jitter and outliers generated during the capture process of the expression capture unit. Data cleaning includes steps such as filling in missing data, correcting erroneous data, and standardizing data formats to ensure data integrity. Based on the preprocessed facial expression sequence information, anatomical feature analysis of the source actor's facial structure is performed, and combined with the facial feature markers of the source actor's face, the corresponding facial key points and facial muscle distribution are obtained; the facial key points include anatomically significant feature points such as the brow arch point, inner canthus point, outer canthus point, tip of the nose point, and corner of the mouth point; the facial muscle distribution is used to target the position and range of action of each muscle group within the facial structure. Facial segmentation was performed on the source actor's facial structure based on facial key points to obtain several anatomical structural regions, including the eye region, eyebrow region, etc. Furthermore, based on the distribution of facial muscles, the anatomical features corresponding to each anatomical structural region and the connection relationships between different anatomical structural regions are obtained; the anatomical features include the morphological features corresponding to each key facial point; the connection relationships include adjacent relationships, motion linkage relationships, and muscle transmission relationships; for example, the morphological features corresponding to the eye region include eye size, shape, and the shape and thickness of the eyelids, etc. Based on the obtained anatomical features and connections, a corresponding facial feature map is constructed. The facial feature map is represented by a three-dimensional mesh structure, consisting of several nodes, edges, and edge attributes. Nodes represent facial key points, edges represent the connection relationships between facial key points, and edge attributes include connection type, motion correlation, etc. Based on the optical flow algorithm, the corresponding facial key points are tracked to obtain the corresponding key point time series data. The key point tracking refers to tracking the running trajectory of facial key points in each data frame of facial expression sequence information and generating corresponding time series data based on it. Based on the obtained key point time-series data, feature optimization analysis is performed on the facial feature map to obtain a simplified facial feature map. Feature optimization analysis refers to identifying and filtering redundant features, low-information connections, and non-expression-related motions within the constructed facial feature map to reduce computational complexity and improve expression capture accuracy. For example, for the constructed facial feature map, facial key point connections with motion correlation below T0 are identified and marked as redundant connections; facial key points with displacement less than T1 during expression changes are identified and marked as static points; and based on this, the graph structure of the facial feature map is optimized, including removing redundant connections and filtering static points to obtain a simplified expression feature representation; T0 and T1 represent the pre-set correlation threshold and displacement threshold, respectively. Furthermore, temporal features are extracted from the obtained key point temporal data to obtain corresponding facial temporal features; facial temporal features include expression change rate, expression duration, and expression transition pattern, etc. Furthermore, based on facial temporal features and combined with a simplified facial feature map, key expression recognition and feature extraction are performed on facial expression sequence information to obtain the corresponding expression feature vector set. Key expression recognition refers to identifying extreme points and transition points in facial expression sequence information based on facial temporal features. For example, cluster analysis is used to segment the expression sequence and extract key frames expressing emotional transitions. Feature extraction involves extracting low-dimensional expression feature representations from key expressions. For example, principal component analysis (PCA) is used to reduce the dimensionality of high-dimensional facial deformation features to expression control parameters, including key expression features such as the degree of eyebrow raising, the angle of mouth corner raising, and the degree of eye opening and closing. The obtained facial feature vector set is decomposed into basic facial expression units to obtain the corresponding set of facial expression primitives. The basic facial expression unit decomposition uses the facial action unit theory to decompose the facial expressions of the source actor into several basic facial expression units, such as "eyebrows raised" and "corners of the mouth pulled down". The basic facial expression units can express complex facial expressions through linear combination. Furthermore, a three-dimensional facial model corresponding to the source actor is constructed based on facial expression sequence information and facial feature maps. The construction process of the three-dimensional facial model adopts multi-view stereo reconstruction technology, which performs feature matching and depth estimation through multi-view expression image sequences and facial depth information to generate a high-precision facial mesh model (i.e., a three-dimensional facial model). Furthermore, based on the expression primitive set and the 3D facial model, expression description models of different levels are generated. The expression description models include a base layer, an intermediate layer, and a high-level layer. The base layer is used to describe the displacement changes of key facial points. The intermediate layer is used to describe the regional deformation of anatomical structural regions. The high-level layer is used to describe the muscle activity in different anatomical structural regions. The complexity of the expression information described by the expression description models of different levels is: base layer < intermediate layer < high-level layer. Based on different levels of facial expression description models, a corresponding multi-scale facial expression set is constructed; at the same time, a mapping relationship between different levels of facial expression description models is established to realize the conversion of facial expression information between different description granularities. The obtained multi-scale facial expression dataset is subjected to redundant information filtering and key information enhancement to obtain the corresponding facial expression feature dataset. Redundant information filtering includes three steps: non-expression-related motion suppression, random noise removal, and data dimensionality reduction. Key information enhancement highlights the feature contribution of key areas of emotional expression through facial saliency analysis and feature weight adjustment. For example, principal component analysis (PCA) is used to reduce the dimensionality of facial features, and facial region importance analysis is used to increase the feature weights of anatomical structures corresponding to key facial points such as the eyes and mouth.

[0021] It should be further explained that, in the specific implementation process, the acquisition process of the high-dimensional facial expression description vector includes: Based on the facial expression feature dataset, basic emotion decomposition and complex emotion recognition are performed on the facial expressions of the source actors to obtain the corresponding emotion expression mapping relationship. The emotion expression mapping relationship includes basic emotion units and expression combination patterns. Among them, the basic emotion decomposition process adopts the Ekman six basic emotion model to decompose facial expressions into six basic emotion units: joy, sadness, anger, fear, disgust, and surprise. The complex emotion recognition identifies more complex emotion expressions, such as embarrassment, doubt, and thinking, by analyzing the combination patterns of basic emotion units. Furthermore, based on the emotion expression mapping relationship, the basic expression units in the expression feature dataset are subjected to emotional semantic annotation to obtain the corresponding emotionally annotated expression set; emotional semantic annotation refers to configuring semantic attributes for the basic expression units in the corresponding expression feature dataset; for example, raising eyebrows is configured with the semantic meaning of "surprise"; raising the corners of the mouth is configured with the semantic meaning of "joy"; and the corresponding emotional expression intensity is assigned simultaneously; Based on the simplified facial features and the collinear relationships between various expression elements in the emotionally labeled expression set; the collinear relationships include synchronous activation relationships (e.g., frowning and downturned corners of the mouth), sequential triggering relationships (e.g., surprise followed by fear), and mutually exclusive inhibition relationships (e.g., laughing and frowning at the same time); expression elements include basic expression units and basic emotion units; Furthermore, an initial facial expression semantic graph is constructed based on the obtained co-occurrence relationships and sentiment-annotated facial expression set. The initial facial expression semantic graph is represented by a directed weighted graph structure, where nodes represent facial expression elements and edges represent co-occurrence relationships between facial expression elements. Node attributes include basic emotion units and semantic attribute configuration results; edge attributes include co-occurrence relationship type, co-occurrence probability, and temporal relationship, etc. Once the initial facial expression semantic map is constructed, temporal and sentiment analysis are performed on it to obtain the corresponding high-dimensional facial expression description vectors. Temporal analysis refers to obtaining the dynamic characteristics of facial expressions by analyzing the complete process of facial expression occurrence, development, climax, and decay during the change of facial expressions. Sentiment analysis deepens the understanding of facial expressions by analyzing the emotional meaning and social communication intention behind the expressions. It should be further explained that, in the specific implementation process, the construction process of the emotion-driven facial expression model includes: This study utilizes emotional semantic space mapping and parameterized expression to map high-dimensional facial expression description vectors, resulting in an expression control model with emotional driving capabilities. Emotional semantic space mapping refers to constructing a three-dimensional emotional space based on emotional psychology theory and mapping facial expressions, which are linearly combined from basic expression units, into this three-dimensional emotional space. Parameterized expression transforms facial expressions into a set of controllable parameters. For example, based on the Valence-Arousal-Dominance three-dimensional emotional space model, each basic expression unit in a facial expression is quantified into a parameter representation, such as determining the angle of eyebrow raising and the degree of mouth corner stretching, thereby achieving precise control and description of facial expressions. Based on the facial expression control model, facial expression temporal analysis and emotional coherence assessment are performed to obtain the corresponding facial expression temporal model. Among them, the facial expression temporal analysis is based on key point temporal data to obtain the development process of facial expressions, including the onset, climax and decay stages of the expression; the emotional coherence assessment examines the consistency and fluency of facial expressions in emotional expression within continuous data frames. The obtained expression time sequence model is aligned with the text script and subjected to character sentiment analysis to obtain the corresponding emotion-driven expression model. The emotion-driven expression model can generate corresponding expression sequences based on the emotional state of the characters in the script, realizing the mapping from emotion description to expression generation. Among them, text script alignment refers to matching the expression data with the emotional description of the performance script to establish the correspondence between text emotion and expression performance. Character sentiment analysis, based on character setting and plot analysis, predicts the emotional state of the characters in specific situations.

[0022] It should be further explained that, in the specific implementation process, the process of obtaining the character expression mapping rules includes: Obtain a 3D mesh model of the target character, which includes the target character's geometric information, topology, and material information; and construct a subdivided mesh model of the target character's face based on it; improve the geometric accuracy of the target character's facial region; for example, use the Loop subdivision algorithm to perform three levels of subdivision on the target character's facial region, so that the facial mesh density reaches a level that can be used to represent subtle changes in facial expressions; Anatomical feature analysis of the facial structure of the target character is performed based on a subdivided mesh model to obtain key feature points and deformation areas of the target character's face; among them, the transition threshold needs to be determined according to the character design style and performance requirements of the film; it usually includes key expression areas such as the eyebrow area, eye area, and mouth area; Key rule mining is performed based on the anatomical feature analysis results of the source actor and the target character to obtain the corresponding feature response set; management rule mining is performed by identifying the correspondence between the facial key points of the source actor's face and the key feature points of the target character's face, and obtaining the mapping pattern between the two based on this; for example, the "left outer corner of the eye" of the source actor is mapped to the key feature point at the corresponding position of the target character, and the relative positional relationship and influence weight between the two are defined. Furthermore, based on the emotion-driven expression model, structural difference analysis and expression compatibility assessment are performed on the subdivided mesh model and the 3D facial model to obtain the corresponding expression transfer mapping strategy. Structural difference analysis refers to quantifying the facial differences in morphology between the source actor's face and the target character's face. Facial differences include bone proportions, muscle distribution, and skin characteristics. Expression compatibility assessment analyzes whether the source actor's facial expressions can be reasonably mapped to the target character and identifies potential inconsistencies. Based on the expression transfer mapping strategy and feature response set, expression space transformation is performed to obtain the corresponding character expression mapping rule set; the character expression mapping rules include global mapping rules, region mapping rules and local mapping rules; among them, the global mapping rules control the overall facial expression mapping style; the region mapping rules set specific mapping parameters for different anatomical structure regions (such as eyes and mouth); and the local mapping rules handle the precise mapping relationship between specific facial key points and key feature points. It should be further explained that, in the specific implementation process, the acquisition process of the initial transferred facial expression sequence includes: Skeletal structure analysis and parametric modeling are performed on the subdivided network model of the target character to generate a character skeleton control model. Skeletal structure analysis identifies the relationship between the facial bones of the target character and the various basic expression units within the facial expressions through geometric information and topological structure. Parametric modeling is used to quantify facial skeleton control into a kind of skeleton control parameters, including skeleton rotation angle and scaling ratio. Based on the character skeleton control model, the expression-driven skeleton linkage relationship and weight allocation are established to obtain the corresponding expression-skeleton mapping matrix. The expression-driven skeleton linkage relationship refers to describing how the basic expression unit drives the movement of the facial bones through positive kinematic modeling. The weight allocation is implemented by using a linear hybrid skinning algorithm to determine the degree of influence of different basic expression units on the movement of facial bones. Based on the expression-skeleton mapping matrix, fine control parameters are designed for different deformation areas of the target character's face, and an expression skeleton control framework is constructed based on it; this framework is used to achieve comprehensive control over the target character's face. Based on the character expression mapping rule set and expression skeleton control framework, the expression mapping transformation from the source actor to the target character is carried out to obtain the basic transferred expression; the expression mapping transformation generates preliminary expression transfer results by applying the facial expression features of the source actor to the character facial control model of the target character. The obtained basic transferred expressions are subjected to character feature adaptive adjustment and transition frame generation to obtain the corresponding initial transferred expression sequence. The character feature adaptive adjustment is carried out to make targeted adjustments to the transferred expressions according to the personalized character features so that the expressions conform to the character settings. The personalized character features are obtained by performing actor individual feature separation on the obtained expression feature dataset. Actor individual feature separation refers to identifying and removing the unique facial features of the source actor, such as individual differences in facial proportions and skin elasticity. Transition frame generation uses interpolation algorithms to generate intermediate states during facial expression changes, ensuring smooth and natural transitions. For example, based on the definition of the target character in a film, details of facial wrinkles and muscle changes are refined to enhance realism. High-quality transition frames are generated using curve interpolation algorithms to ensure continuous and smooth facial expression changes.

[0023] It should be further explained that, in the specific implementation process, the process of obtaining enhanced facial expression sequences includes: The obtained facial expression sequence data were subjected to semantic feature extraction and dynamic feature extraction to obtain corresponding semantic and dynamic features. Semantic features include emotion category, emotion intensity, and emotion complexity. Emotion category refers to the basic or complex emotion expressed by the expression; emotion intensity refers to the degree of emotion expression; and emotion complexity reflects the number and mixing degree of emotional components contained in the expression. Dynamic features include rate of change, change pattern, and prosodic characteristics. Rate of change describes how quickly the expression control parameters change over time; change pattern reflects the path and manner of expression change; and prosodic characteristics describe the rhythm and intensity pattern of expression change. An enhancement objective function is constructed based on facial expression semantic features and facial expression dynamic features. The enhancement objective function is used to define the facial expression enhancement objectives of the desired facial expression. The facial expression enhancement objectives include aspects such as the clarity of emotional expression, the smoothness of facial expression changes, and the prominence of artistic expression. The intensity of the expression control parameters within the initial expression transfer strategy is adjusted based on the enhancement objective function. The intensity adjustment is achieved by using the enhancement objective function to differentiate the expression control parameters. For example, key parameters that express the main emotions are adaptively enhanced, while non-key parameters are kept at their original intensity or slightly adjusted. Simultaneously, facial expressions within the initial expression transfer strategy are enhanced based on semantic features of facial expressions. This enhancement refers to adding micro-expression details that match the emotional state of the target character based on the film. Micro-expression details include fine lines around the eyes, changes in nasolabial folds, and subtle movements of facial muscles. For example, when expressing the emotion of "thinking," details such as slight wrinkles between the eyebrows and slightly narrowed eyes are automatically added; when expressing the emotion of "surprise," details such as pupil dilation and slight forehead elevation are added. Furthermore, the temporal changes of facial expressions within the initial expression transfer strategy are optimized based on the dynamic features of facial expressions. The rhythm optimization identifies key moments of emotional expression by analyzing the acceleration curve of expression changes, and adjusts the speed curve and keyframe distribution of expression changes accordingly. For example, it identifies emotional outbursts (such as sudden surprise or anger) and increases the speed and amplitude of expression changes to create more dramatic performances. Furthermore, the results of the enhancement adjustments, facial detail enhancements, and rhythm optimizations are weighted and fused to obtain the corresponding preliminary enhanced transfer facial expression sequence; The initial enhanced facial expression sequence is processed with facial expression stylization and character personalization enhancement to obtain the corresponding enhanced facial expression sequence. Facial expression stylization uses art style transfer technology to adjust the expressions to conform to the overall style of the film. Character personalization enhancement adds specific facial features according to the character setting to highlight the character's personality. For example, gradient tones and soft edges are used to enhance the facial expression style or strengthen the facial features of determination and decisiveness, such as a slightly furrowed brow and a firm gaze.

[0024] It should be further explained that, in the specific implementation process, the rendering parameter strategy includes: The enhanced facial expression sequence is rendered in layers and marked with detail control tags to generate a multi-level enhanced facial expression pack. Layered rendering refers to dividing the facial expressions in the enhanced facial expression sequence into layers according to different levels of detail and setting corresponding control tags for each layer of expression, thereby achieving multi-level detail control of the expression. Fine-grained partitioning and strategy generation of rendering parameters based on enhanced emojis yield an initial set of rendering parameter rules. Fine-grained partitioning ensures that each emoji detail receives appropriate rendering processing, improving the realism and expressiveness of the emojis. Based on the initial rendering parameter rule set and scene requirements, state machine modeling and transition rule definition of the rendering process are performed to form a dynamic rendering state diagram. State machine modeling is used to describe each state and its transition rules in the rendering process, while the dynamic rendering state diagram can show each state and its transition path. Transition rule definition is used to define the transition conditions and behaviors between each state. The rendering operation quality is evaluated and thresholds are set based on the dynamic rendering state diagram to obtain the corresponding rendering quality control parameters; the quality evaluation assesses the rendering quality through predefined quality indicators (such as accuracy and realism); the threshold setting is used to clearly limit the acceptable range of the quality indicators. Rendering resources are segmented and distributed based on rendering quality control parameters to form a distributed rendering task set. Segmented rendering of rendering resources divides the complete rendering task into several segments, while distributed computing distributes these tasks to different computing nodes, thereby improving the speed and efficiency of rendering. The protocol design and quality verification of the rendering result compositing process based on the distributed rendering task set ensure the consistency and high quality of the rendering results, and finally generate the rendering parameter strategy; the protocol design is used to specify the compositing method and process of the rendering results; the quality verification is to check the compositing results to ensure that they meet the quality requirements; the protocol design specifications and verification rules corresponding to the protocol design and quality verification are all pre-defined. It should be further explained that, in the specific implementation process, the acquisition of movie facial animations includes: The obtained enhanced facial expression sequence's facial expression control parameters are mapped to the constructed facial expression skeleton control framework to generate corresponding skeletal animation data; the mapping process is based on the facial expression-skeleton mapping matrix; for example, the "eyebrows raised" parameter is mapped to a combination of rotation and displacement of the eyebrow bones; Physical constraints are checked on the skeletal animation data to ensure that facial deformation conforms to physical rules and anatomical principles. Physical constraints include restrictions on bone spacing, rotation angle, and speed. For example, the maximum allowable distance between adjacent facial bones is limited to prevent facial tearing. The range of rotation angles of facial bones is limited to ensure that deformation conforms to the anatomical possibilities of the face. A deformation weight mapping is constructed based on a subdivided mesh model and an expression skeleton control framework. The deformation weight mapping defines the degree of influence of facial skeleton motion on key feature points within the subdivided mesh model. Based on the skeletal animation data after deformation weight mapping and physical constraint checks, the mesh deformation animation corresponding to the target character is obtained; for example, for each frame of data, according to the current skeletal animation data and deformation weight mapping, the new positions of each key point in the subdivided network model are obtained, and a continuous facial deformation sequence (i.e., mesh deformation animation) is generated in this way. Simultaneously, subsurface scattering and displacement mapping techniques are applied within the corresponding mesh deformation animation to obtain the corresponding facial deformation data. Subsurface scattering simulates the scattering effect of light inside the skin, enhancing the realism of the skin. Displacement mapping dynamically changes facial details, such as wrinkles and pores, under specific expressions to enhance the visual expressiveness of facial expressions. For example, the subsurface scattering parameters of the skin are dynamically adjusted according to the expression state, so that emotional changes are reflected in the skin texture. Displacement mapping associated with expressions is applied to display forehead wrinkles when the character frowns and smile lines at the corners of the eyes when smiling. Furthermore, the final facial animation is generated based on facial-oriented data and a segmented network model of the target character; Then, based on the obtained rendering parameter strategy, the generated final facial animation is rendered and rendered to ensure efficient rendering performance and high-quality visual effects; and corresponding cinematic facial animation is obtained; among which, the rendering test includes effect evaluation under different lighting conditions, different viewing angles and different resolutions.

[0025] This invention captures and extracts the facial expressions of the source actors, models facial expressions and emotions, constructs character expression mapping rules based on anatomy, enhances and optimizes the stylization of expressions, and designs rendering parameter strategies and generates movie expression animations. It accurately captures the micro-expression features of performers, achieves high-quality semantic transfer of expressions, and provides adaptive expression enhancement based on the personalized characteristics of the characters. It effectively integrates the authenticity of performing arts with the expressiveness of digital characters, providing an advanced AI-driven expression transfer solution for animated film production.

[0026] Example 2; please refer to Figure 2 As shown, parts not described in detail in this embodiment are described in Embodiment 1. An AI-driven facial expression transfer and enhancement system for character animation is provided, including: The data processing module is used to acquire the facial expression sequence information corresponding to the source actor, and to preprocess and extract features from the acquired facial expression sequence information to obtain the corresponding expression feature dataset. Sentiment analysis module: Based on the obtained facial expression feature dataset, facial expression and sentiment analysis are performed to obtain the corresponding high-dimensional facial expression description vectors, and semantic mapping and temporal correlation are performed on them to obtain the corresponding emotion-driven facial expression model; The expression transfer module performs facial anatomy analysis and topological correspondence construction based on the emotion-driven expression model to obtain the corresponding character expression mapping rules, and performs personalized feature extraction and adaptive expression transfer to obtain the corresponding initial transferred expression sequence. Style matching module: Stylizes and optimizes the details of the initial transferred expression sequence to obtain the corresponding enhanced expression sequence; The module generates animated works; it constructs rendering parameter strategies; and based on these strategies, it performs lighting rendering and frame rate synthesis on enhanced facial expression sequences to obtain corresponding cinematic facial animations.

[0027] The modules are connected via wired and / or wireless means to enable data transmission between them.

[0028] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0029] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0030] In the description of this invention, it should be understood that the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0031] In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0032] In the description of this invention, "several" means one or more, and "a large number" means two or more.

[0033] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0034] All formulas in this manual are dimensionless and calculated numerically. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0035] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. An AI-driven facial expression transfer and enhancement method for character animation, characterized in that, include: Step 1: Obtain the facial expression sequence information corresponding to the source actor, and preprocess and extract features from the obtained facial expression sequence information to obtain the corresponding expression feature dataset; Step 2: Based on the obtained facial expression feature dataset, perform facial expression analysis and sentiment analysis to obtain the corresponding high-dimensional facial expression description vectors, and perform semantic mapping and temporal correlation to obtain the corresponding sentiment-driven facial expression model; Step 3: Based on the emotion-driven expression model, perform facial anatomy analysis and topological correspondence construction to obtain the corresponding character expression mapping rules, and perform personalized feature extraction and expression adaptive transfer to obtain the corresponding initial transferred expression sequence; Step 4: Stylize and refine the initial transferred expression sequence to obtain the corresponding enhanced expression sequence; Step 5: Construct rendering parameter strategy; Based on this, lighting rendering and frame rate synthesis are performed on the enhanced facial expression sequence to obtain the corresponding movie facial expression animation.

2. The AI-driven facial expression transfer and enhancement method for character animation according to claim 1, characterized in that, The process of acquiring facial expression sequence information includes: The system acquires the facial structure of the source actors during film production and configures facial feature markers accordingly. Based on a pre-set expression capture unit, it performs multi-view expression capture of the source actors' facial expressions with configured facial feature markers to obtain corresponding multi-view expression image sequences. Simultaneously, it performs three-dimensional structural scanning of the source actors' facial structure to obtain corresponding facial depth information. Based on facial feature markers, the collected facial depth information and multi-view expression image sequences are fused and aligned to obtain the corresponding initial three-dimensional expression data. The head posture data and neck movement data of the source actor are captured, and the captured head posture data and neck movement data are spatiotemporally aligned with the corresponding initial 3D facial expression data. The spatiotemporally aligned initial 3D facial expression data, head posture data and neck movement data are then summarized to obtain the corresponding facial expression sequence information.

3. The AI-driven facial expression transfer and enhancement method for character animation according to claim 2, characterized in that, The process of obtaining the facial expression feature dataset includes: The collected facial expression sequence information is preprocessed; and based on the preprocessed facial expression sequence information, the anatomical features of the source actor's facial structure are analyzed to obtain the corresponding facial key points and the distribution of facial muscles. Facial segmentation was performed on the source actor's facial structure based on facial key points to obtain several anatomical structural regions. Based on the distribution of facial muscles, the anatomical features corresponding to each anatomical structural region and the connection relationships between different anatomical structural regions are obtained. Based on the obtained anatomical features and connectivity relationships, a corresponding facial feature map is constructed; and the corresponding facial key points are tracked to obtain the corresponding key point temporal data; based on the key point temporal data, feature optimization analysis is performed on the facial feature map to obtain a simplified facial feature map; Temporal features are extracted from the obtained key point temporal data to obtain corresponding facial temporal features; key expression recognition and feature extraction are performed on the facial expression sequence information in combination with the simplified facial feature map to obtain the corresponding expression feature vector set. The obtained facial expression feature vector set is decomposed into basic facial expression units to obtain the corresponding facial expression primitive set; the facial expression primitive set includes several basic facial expression units; the basic facial expression units include raised eyebrows and downturned corners of the mouth; A 3D facial model corresponding to the source actor is constructed based on facial expression sequence information and facial feature maps; Based on the aforementioned expression primitive set and 3D facial model, expression description models at different levels are generated; and based on the expression description models, corresponding multi-scale expression sets are constructed. Redundant information is filtered and key information is enhanced from the obtained multi-scale expression set to obtain the corresponding expression feature dataset.

4. The AI-driven facial expression transfer and enhancement method for character animation according to claim 3, characterized in that, The process of obtaining high-dimensional facial expression description vectors includes: Based on the facial expression feature dataset, basic emotion decomposition and complex emotion recognition are performed on the facial expressions of the source actors to obtain the corresponding emotion expression mapping relationship. The emotion expression mapping relationship includes basic emotion units and expression combination patterns. The basic emotion units include joy, sadness, anger, fear, disgust, and surprise. Based on the emotion expression mapping relationship, the basic expression units in the expression feature dataset are annotated with emotional semantics to obtain the corresponding emotionally annotated expression set; Based on the simplified facial features and the collinear relationship between various expression elements in the emotionally labeled expression set; the expression elements include basic expression units and basic emotion units; An initial facial expression semantic graph is constructed based on the obtained co-occurrence relationships and sentiment-annotated facial expression set; and temporal and sentiment analyses are performed on the constructed initial facial expression semantic graph to obtain the corresponding high-dimensional facial expression description vectors.

5. The AI-driven facial expression transfer and enhancement method for character animation according to claim 4, characterized in that, The process of building an emotion-driven facial expression model includes: We perform affective semantic space mapping and affective parameterization on high-dimensional facial expression description vectors to obtain the corresponding facial expression control model; based on the facial expression control model, we perform facial expression temporal analysis and affective coherence evaluation to obtain the corresponding facial expression temporal model. Text script alignment and character sentiment analysis are performed on the obtained expression time sequence model to obtain the corresponding emotion-driven expression model.

6. The AI-driven facial expression transfer and enhancement method for character animation according to claim 5, characterized in that, The process of obtaining character expression mapping rules includes: Obtain the 3D mesh model of the target character, and construct a subdivided mesh model of the target character's face based on it; Based on the subdivided mesh model, the anatomical features of the target character's facial structure are analyzed to obtain the key feature points and deformable areas of the target character's face; Based on the anatomical feature analysis results of the source actor and the target character, key rules are mined to obtain the corresponding feature response set; Based on the emotion-driven expression model, structural difference analysis and expression compatibility assessment are performed between the subdivided mesh model and the 3D facial model to obtain the corresponding expression transfer mapping strategy. Based on the expression transfer mapping strategy and feature response set, the expression space is transformed to obtain the corresponding character expression mapping rule set.

7. The AI-driven facial expression transfer and enhancement method for character animation according to claim 6, characterized in that, The process of obtaining the initial transferred facial expression sequence includes: The skeletal structure of the target character's subdivided network model is analyzed and parametrically modeled to generate a character skeletal control model; Based on the character skeleton control model, establish the expression-driven skeleton linkage relationship and weight allocation to obtain the corresponding expression-skeleton mapping matrix; based on the expression-skeleton mapping matrix, design fine control parameters for the deformation area within the target character's face, and construct the expression skeleton control framework based on it. Based on the character expression mapping rule set and expression skeleton control framework, the expression mapping transformation from the source actor to the target character is performed to obtain the basic transferred expression; The obtained basic transferred expressions are adjusted to adapt to character features and transition frames are generated to obtain the corresponding initial transferred expression sequence.

8. The AI-driven facial expression transfer and enhancement method for character animation according to claim 7, characterized in that, The process of obtaining enhanced facial expression sequences includes: The obtained facial expression sequence data were subjected to expression semantic feature extraction and expression dynamic feature extraction respectively to obtain the corresponding expression semantic features and expression dynamic features; An enhanced objective function is constructed based on the aforementioned facial expression semantic features and facial expression dynamic features; and the intensity of the facial expression control parameters within the initial facial expression transfer strategy is adjusted based on the enhanced objective function. Simultaneously, based on facial expression semantic features, the facial expressions within the initial expression transfer strategy are enhanced in terms of expression details, and based on facial expression dynamic features, the temporal changes of facial expressions within the initial expression transfer strategy are optimized in terms of rhythm. The results of enhancement adjustments, facial detail enhancements, and rhythm optimizations are weighted and fused to obtain the corresponding preliminary enhanced transfer facial expression sequence. The initial enhanced facial expression sequence is processed with facial expression stylization and character personalization enhancement to obtain the corresponding enhanced facial expression sequence.

9. The AI-driven facial expression transfer and enhancement method for character animation according to claim 8, characterized in that, The process of rendering parameter strategy includes: The enhanced emoji sequence is rendered with layered emojis and detailed control marking is performed to generate multi-level detailed enhanced emoji packs; based on the enhanced emoji packs, fine-grained division of rendering parameters and strategy generation are performed to obtain an initial set of rendering parameter rules; Based on the initial rendering parameter rule set and scene requirements, the state machine modeling and transformation rule definition of the rendering process are carried out to form a dynamic rendering state diagram. Based on the dynamic rendering state diagram, the quality of rendering operations is evaluated and thresholds are set to obtain the corresponding rendering quality control parameters. Rendering resources are segmented and distributed for computation based on rendering quality control parameters, forming a distributed rendering task set. The protocol design and quality verification of the rendering result synthesis process based on the distributed rendering task set are used to ultimately generate the rendering parameter strategy.

10. The AI-driven facial expression transfer and enhancement method for character animation according to claim 9, characterized in that... The process of obtaining animated movie expressions includes: The facial expression control parameters within the obtained enhanced facial expression sequence are mapped to the constructed facial expression skeleton control framework to generate corresponding skeletal animation data; Physical constraint checks are performed on the skeletal animation data; simultaneously, deformation weight mapping is constructed based on the subdivided mesh model and the facial bone control framework. Based on the skeletal animation data after deformation weight mapping and physical constraint checking, obtain the mesh deformation animation corresponding to the target character; Subsurface scattering and displacement mapping techniques are applied within the corresponding mesh deformation animation to obtain the corresponding facial deformation data; and the final facial animation is generated based on the facial orientation data and the subdivided network model of the target character. Based on the obtained rendering parameter strategy, the generated final facial animation is rendered and a rendering test is performed; the corresponding movie facial animation is obtained.

Images