Facial expression parameter generation method based on two-parameter space

Abstract

The present application provides a kind of face expression parameter generation method based on two-parameter space, comprising: obtaining single face input image, and preprocessing;Obtain the reference parameter set corresponding to face;According to the semantic determination of 52-dimensional first expression parameter vector of facial action;Obtain the initial population of the form of first expression parameter vector;Then, multiple generations of evolution are carried out to candidate individual, and multiple different high-score candidate individuals are output;Perform fitting conversion of first expression parameter vector to second expression driving parameter vector, to obtain corresponding second expression driving parameter vector;The generated different high-score candidate individual and / or second expression driving parameter vector are used as candidate expression parameter, and visualized and displayed to user;When meeting termination trigger condition, output final result.The method of the present application can realize fine expression customization with lower use threshold, and keep face identity stable and expression natural in iteration process.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and digital human technology, specifically relating to a method for generating facial expression parameters based on a dual-parameter space. Background Technology

[0002] With the development of digital human creation and film and television content production, facial expression customization technology, as a core technology for virtual character content production, has received widespread attention in recent years. Prior to this, parametric 3D face editing technology had achieved high geometric accuracy, but it relied on manual parameter tuning by professionals, resulting in a high barrier to entry and low production efficiency for ordinary users. While end-to-end generative expression editing technology can automatically output results, it has limitations in terms of expression controllability, identity consistency, and result reproducibility.

[0003] In contrast, parameter optimization methods based on interactive evolution can drive parameter-oriented iteration through user selection, balancing the accuracy of parameterization schemes with the ease of use of interactive schemes. However, existing methods still face technical bottlenecks: First, they cannot quickly complete personalized parameter initialization based on a single arbitrary face, relying on general preset templates, resulting in poor adaptability to different faces; second, the evolutionary mode of a single parameter space struggles to balance search efficiency and rendering accuracy, with insufficient semantic space search accuracy and slow convergence in geometric space search; third, the evolutionary process does not perform partition optimization, requiring multiple iterations by the user to obtain satisfactory results. Furthermore, the lack of built-in facial physiological constraints leads to a large number of invalid and deformed expressions in the population that do not conform to muscle movement patterns, resulting in severe homogenization of candidate results and failing to provide sufficiently diverse expression selection directions, increasing the user's screening cost.

[0004] For details on existing 3D face parameter fitting techniques, please refer to the following literature: [S. Qian, VHAP: Versatile HeadAlignment with Adaptive Appearance Priors, 2024, DOI: 10.5281 / zenodo.14988309], [T. Li, T. Bolkart, M. Black, J. Romero, et al. Learning a Model of Facial Shape and Expression from 4D Scans. ACM Transactions on Graphics, 2017, 36(6): 194:1-194:17]. For details on existing digital human rendering modules, please refer to the literature [Y.He, X.Gu, X.Ye, et al. LAM: Large Avatar Model for One-shot Animatable Gaussian Head. Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Papers, 2025: 1-13]. While these technologies have achieved the generation of facial expression parameters and digital human rendering to some extent, they struggle to adjust expressions semantically, thus failing to achieve refined expression customization with a low barrier to entry. Summary of the Invention

[0005] The purpose of this invention is to propose a method for generating facial expression parameters based on a dual-parameter space, which achieves refined expression customization with a low barrier to entry and maintains stable facial identity and natural expressions during the iteration process.

[0006] To achieve the above objectives, the present invention provides a method for generating facial expression parameters based on a two-parameter space, comprising:

[0007] Step S1: Obtain a single face input image, and perform preprocessing on the input image sequentially to obtain a preprocessed standardized face image;

[0008] Step S2: Using the existing 3D face parameter fitting module, perform single-image 3D parameter fitting on the standardized face image to obtain the benchmark parameter set corresponding to the face;

[0009] Step S3: Determine the 52-dimensional first expression parameter vector according to the semantics of facial movements. The first expression parameter vector is divided into different facial regions according to physiological structure.

[0010] Step S4: Obtain the initial population in the form of the first expression parameter vector; then perform multiple generations of evolution on the candidate individuals to output multiple differentiated high-scoring candidate individuals;

[0011] Step S5: For each differentiated high-scoring candidate individual, perform a fitting transformation from the first expression parameter vector to the second expression driving parameter vector to obtain the corresponding second expression driving parameter vector;

[0012] Step S6: The generated differentiated high-scoring candidate individuals and / or the corresponding second expression driving parameter vectors are used as candidate expression parameters and visualized and displayed to the user;

[0013] Step S7: Receive the user's preference selection instruction for the displayed results and obtain the user's preference information;

[0014] Step S8: When the termination trigger condition is met, output the final result, which includes the face image or its dynamic result corresponding to the second expression driving parameter.

[0015] The reference parameter set includes facial identity shape parameters, neutral expression reference parameters, spatial pose parameters, camera parameters, and portrait foreground mask. The neutral expression reference parameters are composed of 100-dimensional expression base coefficients and 3-dimensional jaw pose parameters.

[0016] The process involves single-image 3D parameter fitting of standardized face images, specifically: first, detecting facial key points in the input face image and establishing a parameterized 3D face mesh; then, projecting the 3D face mesh onto the input face image; and iteratively adjusting the face identity shape parameters, neutral expression baseline parameters, spatial pose parameters, and camera parameters to reduce the facial key point error between the projected key points and the image key points, as well as the image pixel error between the rendered result and the input image. After the error converges, a baseline parameter set corresponding to the face is output. When establishing the baseline parameter set corresponding to the face, the 100-dimensional expression basis coefficients and the 3-dimensional jaw pose parameters in the neutral expression baseline parameters are set to zero.

[0017] The different facial regions, classified according to physiological structure, include the eyebrow, eye, nose and cheek, mouth and jaw, and tongue regions. The 52-dimensional first expression parameter vector includes: 5 dimensions for the eyebrow, namely, lowering the eyebrows, raising the center of the eyebrows, and raising the outer sides of the eyebrows; 14 dimensions for the eye, namely, closing the eyes, looking down, looking inward, looking outward, looking up, squinting, and widening the eyes; 5 dimensions for the nose and cheek, namely, puffing out the cheeks, tightening the cheeks, and raising the nostrils; 4 dimensions for the jaw, namely, protruding the jaw, moving it to the left, opening it, and moving it to the right; 23 dimensions for the mouth, namely, closing the mouth, concave corners of the mouth, lowering the corners of the mouth, pursing the lips into a funnel shape, moving the mouth to the left, pulling down the lower lips, pressing the lips together, pouting the lips, moving the mouth to the right, curling the upper and lower lips inward, shrugging the upper and lower lips, smiling, stretching the corners of the mouth, and raising the upper lips; and 1 dimension for the tongue.

[0018] Step S3 also includes setting an independent upper bound for the physiological natural intensity of the first expression parameter vector for each dimension.

[0019] In step S4, an initial population in the form of a first expression parameter vector is obtained, specifically including:

[0020] Step S401: Based on whether user preference information exists, select the random exploration mode or the preference continuation mode, and determine the corresponding seed vector;

[0021] If no user preference information exists, select the random exploration mode. In the random exploration mode, the seed vector is a neutral vector with all zeros. If user preference information exists, select the preference continuation mode, and use the first expression parameter vector selected by the user as the seed vector.

[0022] Step S402: Superimpose random perturbation on the seed vector and restrict each dimension component of the first expression parameter vector to between zero and the corresponding upper bound of natural intensity to obtain an initial population with multiple candidate individuals.

[0023] In step S4, the candidate individuals undergo multiple generations of evolution to output multiple differentiated high-scoring candidate individuals. Specifically, for each evolutionary generation, the following steps are performed:

[0024] Step S41: Calculate the fitness score of the candidate individual. The fitness score includes candidate novelty reward, facial expression humidity activation reward, and facial physiological naturalness penalty.

[0025] Step S42: Select parent individuals based on fitness scores, and generate offspring populations through crossover and mutation;

[0026] Step S43: Sort by fitness score in descending order and combine with the distance threshold parameter to select high-scoring candidate individuals with differentiated characteristics.

[0027] Step S5 specifically includes:

[0028] Step S51: Input the fixed face identity shape parameters obtained in step S2 and the 52-dimensional first expression parameter vector into the first three-dimensional face driving function to generate the first three-dimensional mesh vertices of the target expression, and extract the coordinates of the first facial key points from them as the fitting target;

[0029] Step S52: Input the same face identity shape parameters and the neutral expression reference parameters to be optimized into the second three-dimensional face driving function to generate the second three-dimensional mesh vertices to be fitted, and extract the coordinates of the second facial key points from them;

[0030] Step S53: With the optimization objective of minimizing the errors of the first and second mesh vertices and the errors of the second and second facial key point coordinates, the optimal neutral expression reference parameters are obtained and used as the second expression driving parameter vector.

[0031] In the first display mode, differentiated high-scoring candidate individuals are visualized. The display results include candidate number, thumbnail of the original face image, active component name and numerical intensity of the 52-dimensional first expression parameter vector;

[0032] In the second display mode, the second expression-driven parameter vector is visualized. Specifically, the second expression-driven parameter vector obtained in step S5, the face identity shape parameters, spatial pose parameters, and camera parameters obtained in step S2 are input into the existing digital human rendering module to generate corresponding face images or their dynamic results in batches and display them to the user.

[0033] In step S8, if the termination trigger condition is not met, return to step S4 to continue iterating; or, when the result is a face image or its dynamic result, take the selected second expression driving parameter vector as the center, use the 100-dimensional expression basis parameters and the 3-dimensional jaw pose parameters as variables to be mutated, perform local refinement to generate new candidate individuals, and return to step S6 to re-render the preview.

[0034] The termination trigger conditions include: the user clicks the confirmation button, the number of iterations in step S7 reaches the preset maximum number, or the parameter change of the candidate individuals selected in two consecutive rounds is less than the preset threshold.

[0035] The method of this invention constructs a two-layer evolutionary architecture that combines a first expression parameter space and a second expression-driven parameter space: the first expression parameter space consists of 52-dimensional interpretable facial motion parameters, mainly used for quickly generating candidates, displaying candidate differences, and receiving user filtering; the second expression-driven parameter space consists of 100-dimensional expression basis parameters and 3-dimensional jaw pose parameters, mainly used for realistic rendering and local refinement; in addition, this invention establishes a geometric fitting transformation mechanism from the first expression parameters to the second expression-driven parameters, enabling interpretable candidate search results to be converted into renderable high-precision expression-driven parameters. Since a single RGB image cannot be directly used as a benchmark for expression evolution, this method first performs three-dimensional parameter calculation on the input image through a three-dimensional face parameter fitting module to obtain a benchmark parameter set for identity feature locking. Then, it performs candidate search in the first expression parameter space and converts to the second expression driving parameter space for fitting and display when realistic rendering is required. The final output is expression parameters that can directly drive digital human rendering. It can achieve refined expression customization with a low usage threshold and maintain the stability of face identity and natural expression during the iteration process. It not only achieves personalized expression customization under single image input, but also solves the problems of high parameter tuning threshold and uncontrollable generation results in traditional expression editing schemes. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the specific execution of the two-parameter space interactive facial expression evolution method. Detailed Implementation

[0037] The method of the present invention is further described below: This embodiment is implemented under the premise of the technical solution of the present invention, and provides detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to the following embodiment.

[0038] Example 1: A method for generating facial expression parameters based on a two-parameter space

[0039] like Figure 1 As shown, the method for generating facial expression parameters based on dual-parameter space of the present invention includes: Step S1: acquiring a single facial input image, and performing preprocessing on the input image in sequence, including face detection, key point alignment, foreground segmentation and size normalization, to obtain a preprocessed standardized facial image;

[0040] In this embodiment, the input face image is in RGB format. Face detection refers to locating the face region using a face detection model; keypoint alignment refers to detecting 68 facial keypoints and performing rigid alignment; foreground segmentation refers to obtaining a foreground mask using a portrait segmentation model; and size normalization preprocessing refers to cropping and scaling the face region to a standard size of 256×256 pixels to obtain a preprocessed standardized face image. The preprocessed standardized face image conforms to the input specifications of the 3D face parameter fitting module.

[0041] The input specifications for the 3D face parameter fitting module include: a standardized face image containing a single clear face, a complete face region with detectable key points, an RGB format image, which meets the module's input size requirements after cropping, alignment, and scaling, and has a usable portrait foreground mask.

[0042] Step S2: Using the existing 3D face parameter fitting module, perform single-image 3D parameter fitting on the standardized face image to obtain the benchmark parameter set corresponding to the face;

[0043] The baseline parameter set is a collection of parameters supporting the complete rendering of a 3D face. It includes face identity shape parameters, neutral expression baseline parameters, spatial pose parameters, camera parameters, and a foreground mask. Single-image 3D parameter fitting is performed on the standardized face image, specifically: first, facial key points in the input face image are detected and a parameterized 3D face mesh is established; then, the 3D face mesh is projected onto the input face image. The face identity shape parameters, neutral expression baseline parameters, spatial pose parameters, and camera parameters are iteratively adjusted to reduce the facial key point error (i.e., key point position error) between the projected key points and the image key points, as well as the image pixel error between the rendered result and the input image. After the error converges, the baseline parameter set corresponding to the face is output.

[0044] The expression for the specific set of reference parameters as follows:

[0045]

[0046] Where, β shape β is a facial identity shape parameter, 300-dimensional in size, used to characterize inherent facial features such as face width, size and relative position of facial features, skeletal contour, and nasal bridge height; expr0 The neutral facial expression baseline parameter has a size of 103 dimensions, composed of 100-dimensional neutral facial expression baseline coefficients and 3-dimensional neutral jaw posture parameters, all of which take zero values ​​in the neutral facial expression baseline parameter set; R head The parameters for head rotation are 3D; R neck The parameters for neck rotation are 3D; T globalThe global translation parameter has a 3D dimension; E pose These are binocular gaze posture parameters, with a total dimension of 5, and 3-dimensional rotation parameters for the left and right eyeballs respectively; K cam For camera intrinsic and extrinsic parameters; M fg Foreground mask for human portraits.

[0047] To establish a unified, expressionless starting point for subsequent expression evolution, after completing the 3D parameter fitting of a single image, when establishing the baseline parameter set corresponding to the face, the obtained face identity shape parameters, spatial pose parameters, camera parameters, and portrait foreground mask are retained, while the 100-dimensional expression basis coefficients and 3-dimensional jaw pose parameters in the neutral expression baseline parameters are set to zero. All subsequent steps only modify the 100-dimensional expression basis coefficients and 3-dimensional jaw pose parameters in the neutral expression baseline parameters. In other words, the face identity shape parameters, head rotation parameters, neck rotation parameters, global translation parameters, binocular gaze pose parameters, and camera parameters all maintain the baseline values ​​determined in step S2, thereby avoiding deviations in character identity, gaze direction, or overall pose during expression editing.

[0048] The existing 3D face parameter fitting module iteratively adjusts the face identity shape parameters, neutral expression baseline parameters, spatial pose parameters, and camera parameters of the baseline parameter set through facial key point errors and image pixel errors (the foreground mask directly reuses the previous image segmentation results). It outputs 300-dimensional face identity shape parameters, neutral expression baseline parameters (including 100-dimensional neutral expression base coefficients and 3-dimensional neutral jaw pose parameters), spatial pose parameters (including 3-dimensional head rotation parameters, 3-dimensional neck pose parameters, 6-dimensional binocular gaze pose parameters, and 3-dimensional global translation parameters), camera parameters (including camera intrinsic and extrinsic parameters), and a foreground mask. This 3D face parameter fitting module belongs to existing 3D face parameter fitting technology. This invention does not improve its model structure, training method, or parameter fitting method itself, but rather uses its output as a fixed basis for subsequent expression candidate generation, parameter conversion, and digital human rendering.

[0049] Step S3: Determine the 52-dimensional first expression parameter vector according to the semantics of facial movements. The first expression parameter vector is divided into different facial regions according to physiological structure to construct the 52-dimensional first expression parameter space.

[0050] Therefore, the first expression parameter space serves as the carrier for expression candidate search and user interaction selection. The resulting first expression parameter space only includes the semantics of facial movements, excluding person identity and camera information, and cannot independently complete realistic 3D face rendering. The first stage only displays the original face thumbnail, candidate number, and activity parameter intensity for the user to judge the direction of the movement. When a realistic preview is needed, it is then fitted into 103-dimensional second expression driving parameters through subsequent step S5. Compared to directly searching for 103-dimensional parameters, searching in 52-dimensional space first reduces the difficulty of interactive selection, makes candidate differences have interpretable facial movement meanings, and then uses 103-dimensional parameters to complete fine fitting and realistic rendering.

[0051] Specifically, the first 52-dimensional expression parameter vector is determined according to the semantics of facial movements. This includes filtering based on whether the semantics of facial movements are clear, whether they can be independently controlled, whether they can be converted into three-dimensional geometric deformations, and whether they are independent of identity and camera parameters, to obtain the dimensions corresponding to facial movements, and removing dimensions with semantic repetition or that cannot be stably fitted.

[0052] In this embodiment, the different facial regions divided according to physiological structure include the eyebrow, eye, nose and cheek, mouth and jaw, and tongue regions; the 52-dimensional first expression parameter vector includes: 5 dimensions for the eyebrow, namely, lowering the eyebrows, raising the center of the eyebrows, and raising the outer sides of the eyebrows; 14 dimensions for the eye, namely, closing the eyes, looking down, looking inward, looking outward, looking up, squinting, and widening the eyes; 5 dimensions for the nose and cheek, namely, puffing out the cheeks, tightening the cheeks, and raising the nostrils; 4 dimensions for the jaw, namely, protruding the jaw, moving it to the left, opening it, and moving it to the right; 23 dimensions for the mouth, namely, closing the mouth, concave corners of the mouth, lowering the corners of the mouth, pursing the lips in a funnel shape, moving the mouth to the left, pulling down the lower lips, pressing the lips together, pouting the lips, moving the mouth to the right, curling the upper and lower lips inward, shrugging the upper and lower lips, smiling, stretching the corners of the mouth, and raising the upper lips; and 1 dimension for the tongue.

[0053] Step S3 further includes setting an independent upper bound for the physiological natural intensity of the first expression parameter vector for each dimension. The upper bound is determined based on a combination of the range of common facial movements, the stable interval of historical generated results, and a human experience threshold.

[0054] Step S4: Obtain the initial population in the form of the first expression parameter vector; then perform multiple generations of evolution on the candidate individuals to output multiple differentiated high-scoring candidate individuals;

[0055] In step S4, an initial population in the form of a first expression parameter vector is obtained, specifically including:

[0056] Step S401: Based on whether user preference information exists, select the random exploration mode or the preference continuation mode, and determine the corresponding seed vector;

[0057] If no user preference information is available, select the random exploration mode. In the random exploration mode, the seed vector is a neutral vector with all zeros. If user preference information is available, select the preference continuation mode. Then, the first expression parameter vector selected by the user (often the mean value) is used as the seed vector.

[0058] Step S402: Superimpose random perturbation on the seed vector and restrict each dimension component of the first expression parameter vector to between zero and the corresponding upper bound of natural intensity to obtain an initial population with multiple candidate individuals.

[0059] The initial population size can be set from 16 to 128 candidates depending on the interaction speed and candidate richness; in this embodiment, 64 candidates are selected. As the number of iterations increases, the perturbation intensity gradually decreases to narrow the search range and ensure that the candidates continuously match user preferences.

[0060] The initial population is randomly generated from the 52-dimensional first expression parameter vector, and the parameter values ​​are constrained by the upper bound of the natural intensity of each region, so that the candidate parameter combinations maintain diversity while avoiding obviously unnatural facial movements. When obtaining the initial population, most individuals are generated by superimposing random perturbations on the all-zero neutral vector to expand the search range; the remaining small number of individuals are retained as those near the seed vector, or only the changes in the first expression parameter vector in a face region are superimposed to provide a stable benchmark and local exploration direction.

[0061] In the initial population, the initialization formula for a single candidate individual is:

[0062] ,

[0063] Where i is the candidate individual number in the initial population. Let be the first expression parameter vector of the i-th candidate individual in the initial population. This is the seed vector for the i-th candidate individual in the initial population, and its value is determined by the selected initialization mode. Let be a random perturbation vector that follows a Gaussian distribution. The initial mutation strength, Let be a 52-dimensional identity matrix, u be the upper bound of physiological natural intensity, Clip be the clamping clipping function, which restricts the components of the first expression parameter vector to the range of 0 to the upper bound of physiological natural intensity, and N be the initial population size. It follows a Gaussian distribution.

[0064] In this embodiment, a random exploration mode is used to initialize the population, with a population size of 64 and an initial mutation intensity of 0.09. The initialization formula for a single candidate individual is as follows:

[0065] .

[0066] In this embodiment, the maximum number of evolution generations is set to 45.

[0067] In step S4, the candidate individuals undergo multiple generations of evolution to output multiple differentiated high-scoring candidate individuals. Specifically, for each evolutionary generation, the following steps are performed:

[0068] Step S41: Calculate the fitness score of the candidate individual. The fitness score includes candidate novelty reward, facial expression humidity activation reward, and facial physiological naturalness penalty.

[0069] Therefore, the fitness score F(x) is:

[0070] F(x) = D nov (x) + R act (x) - P nat (x)

[0071] Where x is a 52-dimensional first expression parameter vector, which serves as a candidate individual, D nov (x) represents the candidate novelty reward, which is equal to the minimum of the mean absolute difference of the 52-dimensional parameters between the current candidate and the historical candidates; R act (x) represents the reward for appropriate facial expression activation, corresponding to areas such as the mouth, eyes, eyebrows, and cheeks; P nat (x) represents the penalty for facial physiological naturalness.

[0072] Facial physiological naturalness penalties include overall sparsity constraints, left-right coordination constraints, and mutually exclusive action constraints. Left-right coordination constraints only reduce scores for significantly unbalanced combinations, while preserving reasonable asymmetrical actions such as blinking and unilateral upturn of the corner of the mouth; mutually exclusive action constraints are used to suppress the simultaneous high intensity of mutually exclusive facial actions such as jaw opening and closing, smiling and frowning, and closing and widening the eyes; overall sparsity constraints are used to limit the total activation intensity of all facial parameters.

[0073] Therefore, facial physiological naturalness penalty P nat (x) is:

[0074] P nat (x) = 0.08∑ j x j + 0.08∑ (l,r)∈S |x l - x r | + ∑ (p,q)∈E w pq x p x q ,

[0075] Where j is the ordinal number of the first expression parameter vector, x jLet x be the i-th dimension component of the 52-dimensional first facial expression parameter vector. l x r These are the components of the left and right facial movement dimensions in the first expression parameter vector, where l and r represent the left and right facial movement dimensions, respectively. p x q Let p and q be the components of the first facial expression parameter vector in the first and second mutually exclusive facial action dimensions, respectively, and w be the components of the first and second mutually exclusive facial action dimensions. pq Let S be the weight of mutually exclusive actions, where S is the set of corresponding left and right actions, and E is the set of mutually exclusive actions.

[0076] The set of mutually exclusive actions E includes three pairs of actions: opening and closing the jaw, smiling and frowning, and closing and opening the eyes. The weights of these mutually exclusive actions are w. pq We take values ​​of 0.55, 0.22, and 0.22 respectively. Thus, the first term limits the overall activation intensity, the second term limits the obvious left-right imbalance, and the third term inhibits the simultaneous high intensity of contradictory actions.

[0077] Step S42: Select parent individuals based on fitness scores, and generate offspring populations through crossover and mutation;

[0078] Therefore, new candidates are generated by combining two preferred individuals according to parameter dimensions, while random small variations are added, so that the new candidates produce new expression differences while retaining the better expression direction.

[0079] In this embodiment, the first facial expression parameter vector of the candidate individuals in the (t+1)th generation offspring population for:

[0080]

[0081] in, and Let be the first expression parameter vector of the two parent individuals in generation t; m is a dimension-wise cross mask composed of 0s and 1s. ε is the crossover operator; ε is the Gaussian mutation perturbation vector, corresponding to the mutation operator; The upper bound of the physiological natural intensity is defined by , and Clip is the clipping function that restricts the components of the first expression parameter vector to the range of 0 to the upper bound of the physiological natural intensity.

[0082] Step S43: Sort by fitness score in descending order and combine with the distance threshold parameter to select high-scoring candidate individuals with differentiated characteristics.

[0083] The selected high-scoring candidates are the K candidates with the highest fitness scores and whose parameters differ from each other by a preset threshold. This ensures both naturalness and diversity in the results. K can be 8, or set according to your needs.

[0084] During the screening process, the system first sorts candidate individuals in descending order according to their fitness scores, then extracts the top-ranked candidate individual in each order and calculates the average absolute difference of the 52-dimensional parameters between this candidate individual and all previously screened candidates. If the average absolute difference is not less than a preset difference threshold, the candidate individual is added to the Top-K set. In this embodiment, the difference threshold is set to 0.025. If fewer than K candidate individuals meet the preset difference threshold, they are supplemented according to their fitness scores.

[0085] Step S5: For each differentiated high-scoring candidate individual, perform a fitting transformation from the first expression parameter vector to the second expression driving parameter vector to obtain the corresponding second expression driving parameter vector;

[0086] Since the 52-dimensional first expression parameter vector cannot be directly rendered into a face image, it needs to be converted into a 103-dimensional second expression driving parameter (100-dimensional expression basis coefficients + 3-dimensional jaw pose parameters) to render the face image.

[0087] Step S5 specifically includes:

[0088] Step S51: Input the fixed face identity shape parameters obtained in step S2 and the 52-dimensional first expression parameter vector into the first three-dimensional face driving function to generate the first three-dimensional mesh vertices of the target expression, and extract the coordinates of the first facial key points from them as the fitting target;

[0089] Here, the 52-dimensional first expression parameter is obtained by the evolutionary algorithm in step S4, which is different from the initial first expression parameter in step S3.

[0090] The first 3D face driving function is pre-generated by triggering 52-dimensional facial movements of the first expression parameter vector and collecting the face mesh deformation offset when each movement is activated individually as a facial deformation basis. The first 3D face driving function linearly combines the neutral face mesh according to the 52 pre-set facial deformation basis to obtain the first 3D mesh vertices of the target expression, and obtains the coordinates of the first facial key points from the mesh according to the pre-fixed key point index.

[0091] Step S52: Input the same face identity shape parameters and the neutral expression baseline parameters to be optimized (including 100-dimensional neutral expression baseline coefficients and 3-dimensional neutral jaw pose parameters) into the second three-dimensional face driving function to generate the second three-dimensional mesh vertices to be fitted, and extract the coordinates of the second facial key points from them;

[0092] Therefore, the first and second 3D face driving functions use the same mesh topology and identity shape parameters, so the positional differences between corresponding vertices and key points can be directly compared.

[0093] This step does not use training labels or retrain the model. Instead, it uses the 3D geometric state generated by the first expression parameter vector as the fitting target to perform parameter optimization on a single candidate individual. The 3D state corresponding to the first expression parameter vector provides the fitting target, while the 3D state corresponding to the second expression-driven parameters generates the renderable result. The two types of parameters do not require a one-to-one dimensional correspondence; instead, they are linked by the proximity of 3D mesh vertices and facial keypoints within the same topology. Since both use the same set of identity shape parameters, the optimization process only changes the expression basis parameters and jaw pose parameters, without altering the individual's identity.

[0094] Step S53: With the optimization objective of minimizing the errors of the first and second mesh vertices and the errors of the second and second facial key point coordinates, the optimal neutral expression reference parameters are obtained and used as the second expression driving parameter vector.

[0095] When the optimization objective is to minimize the errors of the first and second grid vertices and the errors of the coordinates of the second and third facial key points, the optimization loss function used includes a vertex error term, a key point error term, and a regularization constraint term.

[0096] The optimized loss function is as follows:

[0097] ,

[0098] in, The vertex error term is used to calculate the coordinate deviation between the vertices of the 3D mesh generated by the second expression driving parameters and the vertices of the target expression mesh, so as to ensure the fit of the overall facial contour and deformation amplitude. This is the key point error term, used to calculate the coordinate deviation between the feature points of key parts such as eyes, eyebrows, nose, and mouth and the target key points, so as to prioritize the position of facial features and facial expression details. This is a regularization term used to constrain the amplitudes of the facial expression base parameters and jaw posture parameters, preventing the parameters from deviating excessively from a reasonable range. , The first and second loss balancing weights are used to adjust the proportion of vertex error, key point error, and regularization term in the optimization.

[0099] The optimal neutral expression baseline parameters are obtained by solving the problem. The expression is:

[0100] ,

[0101] in, The expression basis coefficients are 100-dimensional. These are 3D mandibular posture parameters. 100-dimensional facial expression basis coefficients are used to describe the main components of facial muscle deformation, while 3D mandibular posture parameters are used to describe the opening and closing, forward and backward extension and lateral displacement of the mandible.

[0102] It is worth noting that the fixed facial shape parameters and camera parameters from step S2 are reused during the conversion process, and only the expression and jaw pose dimensions are updated to ensure that the facial identity is consistent before and after the conversion.

[0103] In this embodiment, the Adam optimizer is used to obtain the optimal neutral facial expression baseline parameters, with a learning rate of 0.04 and 260 iterations; the vertex loss weight is 1.0, the keypoint loss weight is 10.0, and the regularization coefficient is 0.0008. The facial expression baseline parameters are limited to [-4.0, 4.0], and the jaw pose parameters are limited to [-0.8, 0.8]. After fitting, a 20-frame continuous motion sequence is generated according to the easing curve, and the result is output as a standard parameter file readable by the digital human rendering process.

[0104] Step S6: The generated differentiated high-scoring candidate individuals and / or the corresponding second expression driving parameter vectors are used as candidate expression parameters and visualized and displayed to the user;

[0105] In the first display mode, differentiated high-scoring candidate individuals (in the form of a first expression parameter vector) are visualized. The display results include the candidate number, a thumbnail of the original face image, and the names and numerical strengths of the active components of the 52-dimensional first expression parameter vector, which are used by the user for initial screening. At this time, the displayed content is not a real rendered face image, but an explanatory preview of the candidate individuals in the first expression parameter vector.

[0106] In the second display mode, the second expression-driven parameter vector is visualized. Specifically, this includes inputting the second expression-driven parameter vector obtained in step S5, along with the face shape parameters, spatial pose parameters, and camera parameters obtained in step S2, into an existing digital human rendering module to batch generate corresponding face images or their dynamic results and display them to the user. The digital human rendering module is existing technology; this invention does not improve its network structure or training method. The improvements focus on candidate generation before rendering, two-parameter space fitting transformation, and a user preference-driven iterative optimization process.

[0107] Step S7: Receive the user's preference selection instruction for the displayed results and obtain the user's preference information; In this process, the user selects at least one differentiated high-scoring candidate individual. The user preference information includes at least one of the following: candidate number, selection round, selection time, confirmation or elimination flag, and optional scoring information for the differentiated high-scoring candidate individual. Step S7 further includes recording at least one of the following: a first expression parameter vector, a second expression driving parameter, a visualization result, and user preference information corresponding to the differentiated high-scoring candidate individual selected by the user, so as to use the user preference information as a reference for subsequent iterations.

[0108] Step S8: When the termination trigger condition is met, output the final result, which includes the face image or its dynamic result corresponding to the second expression driving parameter.

[0109] In this embodiment, if the termination trigger condition is not met, the process returns to step S4 and continues iterating.

[0110] In other embodiments, if the termination trigger condition is not met, the step of returning to step S4 can be replaced by: when the displayed result is a face image or its dynamic result (i.e., when realistic rendering is enabled), using the selected second expression-driven parameter vector as the center, taking the 100-dimensional expression basis parameters and the 3-dimensional jaw pose parameters as variables to be mutated, performing local refinement to generate new candidate individuals, and returning to step S6 to re-render the preview. Thus, by rendering and displaying the result and continuing to receive user preference feedback, a cyclical human-computer interactive expression optimization process is formed.

[0111] The configuration for local refinement is as follows: population size 48, maximum number of generations 40, and different variation lengths are set for the 100-dimensional facial expression parameters and the 3-dimensional jaw posture parameters, generating new candidate populations through small perturbations. The preferred variation lengths for the facial expression parameters are 0.02 to 0.12, and for the jaw posture parameters, they are 0.01 to 0.06, to optimize facial expression details and jaw movements respectively. In each round, several dimensions are randomly selected for small-scale mutations according to a preset ratio, while retaining a small number of overall perturbation candidates, avoiding large-scale modifications to all 103 dimensions simultaneously. For the selected dimensions to be mutated, one or more perturbation values ​​can be generated for each dimension, and the perturbations from different dimensions are combined to form a new candidate population. In this embodiment, 48 candidates are generated at once based on the population size, rather than generating a fixed number of candidates for each dimension separately.

[0112] Termination trigger conditions include: the user clicks the confirmation button, the number of iterations in step S7 reaches the preset maximum number, or the parameter change of the candidate individuals selected in two consecutive rounds is less than the preset threshold.

[0113] The final results include the second expression-driven parameters of the selected high-scoring differential candidates, the corresponding face image or its dynamic results, the entire evolutionary history, and user preference logs. All output parameters can be directly integrated into the digital human rendering pipeline without secondary conversion or adaptation.

[0114] It should be noted that if step S6 only visualizes the differentiated high-scoring candidate individuals (in the form of a first expression parameter vector) in the first display mode, then step S8 obtains the face image or its dynamic result in the final result by inputting the second expression driving parameter vector corresponding to the user's preference selection instruction for the display result, the face identity shape parameter, spatial pose parameter, and camera parameter obtained in step S2 into the existing digital human rendering module.

[0115] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the invention. Various variations can be made to the above embodiments of the present invention. That is, all simple and equivalent changes and modifications made based on the claims and description of this invention fall within the protection scope of the claims of this patent. All aspects not described in detail in this invention are conventional technical content.

Claims

1. A method for generating facial expression parameters based on a two-parameter space, characterized in that, include: Step S1: Obtain a single face input image, and perform preprocessing on the input image sequentially to obtain a preprocessed standardized face image; Step S2: Using the existing 3D face parameter fitting module, perform single-image 3D parameter fitting on the standardized face image to obtain the benchmark parameter set corresponding to the face; Step S3: Determine the 52-dimensional first expression parameter vector according to the semantics of facial movements. The first expression parameter vector is divided into different facial regions according to physiological structure. Step S4: Obtain the initial population in the form of the first expression parameter vector; then perform multiple generations of evolution on the candidate individuals to output multiple differentiated high-scoring candidate individuals; Step S5: For each differentiated high-scoring candidate individual, perform a fitting transformation from the first expression parameter vector to the second expression driving parameter vector to obtain the corresponding second expression driving parameter vector; Step S6: The generated differentiated high-scoring candidate individuals and / or the corresponding second expression driving parameter vectors are used as candidate expression parameters and visualized and displayed to the user; Step S7: Receive the user's preference selection instruction for the displayed results and obtain the user's preference information; Step S8: When the termination trigger condition is met, output the final result, which includes the face image or its dynamic result corresponding to the second expression driving parameter.

2. The method for generating facial expression parameters based on a dual-parameter space according to claim 1, characterized in that, The reference parameter set includes facial identity shape parameters, neutral expression reference parameters, spatial pose parameters, camera parameters, and portrait foreground mask. The neutral expression reference parameters are composed of 100-dimensional expression base coefficients and 3-dimensional jaw pose parameters. The standard face image is fitted with three-dimensional parameters in a single image. Specifically, the process includes: first, detecting facial key points in the face input image and establishing a parameterized three-dimensional face mesh; then, projecting the three-dimensional face mesh onto the face input image; and iteratively adjusting the face identity shape parameters, neutral expression reference parameters, spatial pose parameters, and camera parameters to reduce the facial key point error between the projected key points and the image key points, as well as the image pixel error between the rendering result and the input image. After the error converges, the baseline parameter set corresponding to the face is output; when establishing the baseline parameter set corresponding to the face, the 100-dimensional expression basis coefficients and the 3-dimensional mandibular pose parameters in the neutral expression baseline parameters are set to zero.

3. The method for generating facial expression parameters based on a dual-parameter space according to claim 1, characterized in that, The different facial regions, classified according to physiological structure, include the eyebrow, eye, nose and cheek, mouth and jaw, and tongue regions. The 52-dimensional first expression parameter vector includes: 5 dimensions for the eyebrow, namely, lowering the eyebrows, raising the center of the eyebrows, and raising the outer sides of the eyebrows; 14 dimensions for the eye, namely, closing the eyes, looking down, looking inward, looking outward, looking up, squinting, and widening the eyes; 5 dimensions for the nose and cheek, namely, puffing out the cheeks, tightening the cheeks, and raising the nostrils; 4 dimensions for the jaw, namely, protruding the jaw, moving it to the left, opening it, and moving it to the right; 23 dimensions for the mouth, namely, closing the mouth, concave corners of the mouth, lowering the corners of the mouth, pursing the lips into a funnel shape, moving the mouth to the left, pulling down the lower lips, pressing the lips together, pouting the lips, moving the mouth to the right, curling the upper and lower lips inward, shrugging the upper and lower lips, smiling, stretching the corners of the mouth, and raising the upper lips; and 1 dimension for the tongue.

4. The method for generating facial expression parameters based on a dual-parameter space according to claim 1, characterized in that, Step S3 also includes setting an independent upper bound for the physiological natural intensity of the first expression parameter vector for each dimension.

5. The method for generating facial expression parameters based on a dual-parameter space according to claim 1, characterized in that, In step S4, an initial population in the form of a first expression parameter vector is obtained, specifically including: Step S401: Based on whether user preference information exists, select the random exploration mode or the preference continuation mode, and determine the corresponding seed vector; If no user preference information exists, select the random exploration mode. In the random exploration mode, the seed vector is a neutral vector with all zeros. If user preference information exists, select the preference continuation mode, and use the first expression parameter vector selected by the user as the seed vector. Step S402: Superimpose random perturbation on the seed vector and restrict each dimension component of the first expression parameter vector to between zero and the corresponding upper bound of natural intensity to obtain an initial population with multiple candidate individuals.

6. The method for generating facial expression parameters based on a dual-parameter space according to claim 5, characterized in that, In step S4, the candidate individuals undergo multiple generations of evolution to output multiple differentiated high-scoring candidate individuals. Specifically, for each evolutionary generation, the following steps are performed: step S41: Calculate the fitness score of the candidate individual, which includes the candidate novelty reward, the expression humidity activation reward, and the facial physiological naturalness penalty; Step S42: Select parent individuals based on fitness scores, and generate offspring populations through crossover and mutation; Step S43: Sort by fitness score in descending order and combine with the distance threshold parameter to select high-scoring candidate individuals with differentiated characteristics.

7. The method for generating facial expression parameters based on a dual-parameter space according to claim 1, characterized in that, Step S5 specifically includes: Step S51: Input the fixed face identity shape parameters obtained in step S2 and the 52-dimensional first expression parameter vector into the first three-dimensional face driving function to generate the first three-dimensional mesh vertices of the target expression, and extract the coordinates of the first facial key points from them as the fitting target; Step S52: Input the same face identity shape parameters and the neutral expression reference parameters to be optimized into the second three-dimensional face driving function to generate the second three-dimensional mesh vertices to be fitted, and extract the coordinates of the second facial key points from them; Step S53: With the optimization objective of minimizing the errors of the first and second mesh vertices and the errors of the second and second facial key point coordinates, the optimal neutral expression reference parameters are obtained and used as the second expression driving parameter vector.

8. The method for generating facial expression parameters based on a dual-parameter space according to claim 1, characterized in that, In the first display mode, differentiated high-scoring candidate individuals are visualized. The display results include candidate number, thumbnail of the original face image, active component name and numerical intensity of the 52-dimensional first expression parameter vector; In the second display mode, the second expression-driven parameter vector is visualized. Specifically, the second expression-driven parameter vector obtained in step S5, the face identity shape parameters, spatial pose parameters, and camera parameters obtained in step S2 are input into the existing digital human rendering module to generate corresponding face images or their dynamic results in batches and display them to the user.

9. The method for generating facial expression parameters based on a dual-parameter space according to claim 1, characterized in that, In step S8, if the termination trigger condition is not met, return to step S4 to continue iterating; or, when the result is a face image or its dynamic result, take the selected second expression driving parameter vector as the center, use the 100-dimensional expression basis parameters and the 3-dimensional jaw pose parameters as variables to be mutated, perform local refinement to generate new candidate individuals, and return to step S6 to re-render the preview.

10. The method for generating facial expression parameters based on a dual-parameter space according to claim 1, characterized in that, The termination trigger conditions include: the user clicks the confirmation button, the number of iterations in step S7 reaches the preset maximum number, or the parameter change of the candidate individuals selected in two consecutive rounds is less than a preset threshold.

Images