Unsupervised-based facial expression capture model training method, system and medium
By using an unsupervised facial expression capture model training method, a first two-dimensional face image corresponding to a preset prompt word is generated. By combining the facial expression score vector and face parameters, a model that closely resembles a real face is trained, which solves the problems of difficult and costly data collection in existing technologies and achieves efficient and low-cost facial expression capture.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAMEN MEITUZHIJIA TECH
- Filing Date
- 2023-09-27
- Publication Date
- 2026-06-23
AI Technical Summary
Existing facial expression capture technology suffers from difficulties in data collection and high costs.
An unsupervised facial expression capture model training method is adopted. By generating a first two-dimensional face image corresponding to a preset prompt word, and combining the face expression score vector and face parameters, the model is trained using massive unsupervised data. The preset loss is calculated and gradient backpropagation training is performed.
It achieves low-cost and high-efficiency facial expression capture, improves the stability, accuracy, precision and generalization ability of expression capture, and reduces data collection costs.
Smart Images
Figure CN117218492B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of facial motion capture, and particularly relates to a facial expression capture model training method and system based on unsupervised learning and a medium. BACKGROUND
[0002] Facial motion capture, also known as facial expression capture, is part of motion caption technology, which refers to the process of recording human facial expressions and movements using external devices such as cameras or mechanical devices and converting them into parameters that can be used by animation or film.
[0003] In the prior art, there are two main solutions for facial expression capture. One is a head-mounted device-based solution, which usually requires some additional marker points and uses special high-speed camera equipment and sensors to record human facial expression movements and changes. These devices can capture subtle muscle movements, wrinkles, and eye changes, providing high-quality data for subsequent processing, but such devices are generally expensive, ranging from several thousand yuan to hundreds of thousands of yuan. The other is a general RGB camera-based solution, which can capture facial expression movements at very low cost. This type of solution usually trains a neural network model first. Training the model usually requires labeled facial expression data, which often has some problems. First, the training data is small, because it is very difficult to obtain labeled data with expression parameters, and the size of the collected or synthesized data is very limited. Second, these data are strongly associated with the dimensions of blendshape, and the change in blendshape dimensions makes the labeled data no longer matchable.
[0004] It can be seen that the existing facial expression capture solution has the problems of difficult data collection and high cost. SUMMARY
[0005] The main purpose of the present application is to provide a facial expression capture model training method and system based on unsupervised learning and a storage medium, which aims to solve the technical problems of difficult data collection and high cost in the existing facial expression capture model training method based on unsupervised learning.
[0006] To achieve the above objectives, this invention provides a method for training an unsupervised facial expression capture model, comprising the following steps: acquiring a set of preset prompt words for image generation, and randomly selecting a preset prompt word from the set to generate a corresponding first two-dimensional face image; inputting the first two-dimensional face image into a pre-trained emotion regression network to obtain a first facial expression score vector; inputting the first two-dimensional face image into an expression-driven network to obtain face ID coefficients, expression-driven coefficients, and face pose and texture coefficients; calculating three-dimensional facial key points based on the face ID coefficients and expression-driven coefficients; projecting the three-dimensional facial key points onto the first two-dimensional face image, and rendering the image based on the pixel values to obtain a second two-dimensional face image; inputting the second two-dimensional face image into the emotion regression network to obtain a second facial expression score vector; calculating a preset loss according to a preset loss calculation method, and training the facial expression capture model using gradient backpropagation.
[0007] Optionally, generating the corresponding first two-dimensional face image includes the following steps: inputting a randomly selected preset prompt word into a pre-trained first encoding model to obtain the prompt word encoding; then inputting the prompt word encoding into a pre-trained second encoding model to obtain the image encoding features of the preset prompt word; randomly sampling a Gaussian noise image and an expression driving coefficient, and iteratively calculating according to the following formula:
[0008] Where, x t-1 x represents the calculation result at time t-1. t This represents the calculation result at time t, α t The value represents the decay over time; text represents the image encoding features of the preset prompt word; and exp represents the expression-driven coefficient. The expression-driving coefficient represents the product of α from time t to the current time, where z is the Gaussian noise image, and σ is the expression-driving coefficient. t Let t be the mixing ratio; perform n iterations to generate the corresponding first two-dimensional face image, where n = t.
[0009] Optionally, the preset loss is calculated according to a preset loss calculation formula, including at least: calculating a first preset loss according to a preset first loss calculation method, specifically the expression consistency loss; calculating a second preset loss according to a preset second loss calculation method, specifically the key point loss; calculating a third preset loss according to a preset third loss calculation method, specifically the training parameter direction control loss; calculating a fourth preset loss according to a preset fourth loss calculation method, specifically the texture loss; and calculating a fifth loss according to a preset fifth loss calculation method, the first preset loss, the second preset loss, the third preset loss, and the fourth preset loss, specifically the comprehensive loss.
[0010] Optionally, the feature is that: the first preset loss is calculated according to the preset first loss calculation method, specifically according to the following formula: loss_expression=l1(p_e-t_e); where loss_expression represents the expression consistency loss, l1 represents the norm loss function, p_e represents the second facial expression score vector, and t_e represents the first facial expression score vector.
[0011] Optionally, the second preset loss is calculated according to a preset second loss calculation method, which includes at least the following steps: inputting the second two-dimensional face image into a pre-trained 2D keypoint regression network to obtain two-dimensional face keypoints; indexing the projection points of the three-dimensional face keypoints onto the first two-dimensional face image to obtain the points corresponding to the projection points and the two-dimensional face keypoints; determining the two-dimensional face keypoints that need to participate in the loss calculation based on the positive and negative values of the yaw angle in the face pose; and calculating the keypoint loss based on the MSE loss function and the two-dimensional face keypoints that need to participate in the loss calculation.
[0012] Optionally, a third preset loss is calculated according to a preset third loss calculation method, specifically including the following steps: inputting the second two-dimensional face image into the expression-driven network to obtain face expression-driven parameters; obtaining a set of random numbers within a preset range to represent direction coefficients, and obtaining the total number of generated random numbers to represent the expression-driven dimension; and performing random processing on the face expression-driven parameters according to a preset random processing formula, the specific preset random processing formula being as follows:
[0013] new_expression=d_expression+direction*(random(0,0.3*d_expression);
[0014] Where new_expression represents the facial expression driving parameters obtained after random processing, d_expression represents the facial expression driving parameters, direction represents the direction coefficient, and random(0,0.3*d_expression) represents a randomly generated value, with the value ranging from [0,0.3*d_expression]. Based on the feedforward network, feature encoding is performed on the direction coefficients, and facial expression encoding is performed on the randomly processed facial expression driving parameters. The feature encoding results and facial expression encoding results are then input into the expression driving network. Based on the second facial expression score vector and the facial expression driving parameters, the training parameter direction control loss is calculated, and the specific calculation formula is as follows:
[0015] loss_expression_direction = sum(abs(sign(p_e-new_expression))); where loss_expression_direction represents the training parameter direction control loss, sum represents the summation function, abs represents the absolute value function, sign represents the sign function, p_e represents the second face expression score vector, and new_expression represents the face expression driving parameters obtained after random processing.
[0016] Optionally, the fourth preset loss is calculated according to the preset fourth loss calculation method, which specifically includes the following steps: rendering the face ID coefficient, expression driving coefficient, face pose and texture coefficient to obtain an image with texture information; calculating the pixel loss between the image with texture information and the first two-dimensional face image to obtain the texture loss.
[0017] Optionally, the fifth loss is calculated based on the preset fifth loss calculation method, the first preset loss, the second preset loss, the third preset loss, and the fourth preset loss. Specifically, it is calculated according to the following formula: loss=α*loss_expression+β*loss_lmk+δ*loss_expression_direction+σ*loss_texture; where α, β, δ, and σ represent the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient, and the fourth adjustment coefficient, respectively; loss_expression represents the expression consistency loss; loss_lmk represents the keypoint loss; loss_expression_direction represents the training parameter direction control loss; and loss_texture represents the texture loss.
[0018] Corresponding to the aforementioned unsupervised facial expression capture model training method, this invention provides an unsupervised facial expression capture model training system, comprising: a first two-dimensional face image generation module, used to acquire a set of preset prompt words for image generation, and randomly select a preset prompt word from the set to generate a corresponding first two-dimensional face image; a first face expression score vector acquisition module, used to input the first two-dimensional face image into a pre-trained emotion regression network to obtain a first face expression score vector; and a face parameter acquisition module, used to input the first two-dimensional face image into an expression-driven network to obtain face ID coefficients, expression-driven coefficients, and face pose. The system includes: a texture coefficient module; a 3D facial keypoint calculation module, used to calculate 3D facial keypoints based on facial ID coefficients and expression-driven coefficients; a projection and rendering processing module, used to project the 3D facial keypoints onto a first 2D facial image and perform rendering processing based on the pixel values in the first 2D facial image to obtain a second 2D facial image; a second facial expression score vector acquisition module, used to input the second 2D facial image into an emotion regression network to obtain a second facial expression score vector; a loss calculation module, used to calculate a preset loss according to a preset loss calculation method; and a training module, used to train the facial expression capture model using gradient backpropagation based on the loss calculation results.
[0019] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a training program for an unsupervised facial expression capture model, wherein the training program, when executed by a processor, implements the steps of the unsupervised facial expression capture model training method described above.
[0020] The beneficial effects of this invention are:
[0021] (1) Compared with the prior art, the present invention generates a first two-dimensional face image corresponding to the preset prompt word, and combines the face expression score vector, face parameters (including the face expression score vector), and unsupervised massive data to train a model with a driving effect closer to the real face. It does not require expensive data acquisition, making the data collection process simple and the cost low. At the same time, the stability of face expression capture is significantly improved compared with supervised training.
[0022] (2) Compared with the prior art, the facial expression capture model trained by the unsupervised facial expression capture model training method of the present invention can estimate the facial motion capture coefficients by simply inputting the image to be estimated, which can greatly improve the efficiency of facial expression capture.
[0023] (3) Compared with the prior art, the first two-dimensional face image generated by the present invention through iterative calculation has a higher matching degree with the preset prompt words, which can lay the foundation for subsequent high-accuracy facial expression capture.
[0024] (4) Compared with the prior art, the present invention can improve the ability of facial expression capture to capture emotional actions by calculating the expression consistency loss;
[0025] (5) Compared with the prior art, the present invention can improve the consistency of facial expression capture and key points by calculating key point loss, thereby improving the stability and accuracy of facial capture;
[0026] (6) Compared with the prior art, the present invention can improve the generalization ability and sensitivity of facial expression capture by calculating the direction control loss of training parameters;
[0027] (7) Compared with the prior art, the present invention can improve the precision of facial expression capture by calculating texture loss, and can also achieve very high accuracy in some subtle expressions;
[0028] (8) Compared with the prior art, the present invention calculates the third preset loss according to the preset third loss calculation method, and also considers the information of the previous frame in the facial expression capture in the video sequence. In actual use, the direction vector is calculated by the relationship between the facial points of the previous and previous frames. By passing the direction vector to the network, the network can learn the relationship between the previous and previous frames and the current expression capture value more clearly, thereby improving the accuracy of facial expression capture. Attached Figure Description
[0029] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:
[0030] Figure 1 This is a simplified flowchart of an embodiment of the unsupervised facial expression capture model training method of the present invention;
[0031] Figure 2 This is a framework diagram of an embodiment of the facial expression capture model training system of the present invention. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0033] like Figure 1 As shown, the present invention provides a method for training an unsupervised facial expression capture model, comprising the following steps: acquiring a set of preset prompt words for image generation, and randomly selecting a preset prompt word from the set to generate a corresponding first two-dimensional face image; inputting the first two-dimensional face image into a pre-trained emotion regression network to obtain a first facial expression score vector; inputting the first two-dimensional face image into an expression-driven network to obtain face ID coefficients, expression-driven coefficients, and face pose and texture coefficients; calculating three-dimensional facial key points based on the face ID coefficients and expression-driven coefficients; projecting the three-dimensional facial key points onto the first two-dimensional face image, and rendering the image based on the pixel values to obtain a second two-dimensional face image; inputting the second two-dimensional face image into the emotion regression network to obtain a second facial expression score vector; calculating a preset loss according to a preset loss calculation method, and training the facial expression capture model using gradient backpropagation.
[0034] This invention generates a first two-dimensional face image corresponding to a preset prompt word, and combines it with a face expression score vector, face parameters (including the face expression score vector), and massive amounts of unsupervised data to train a model whose driving effect is closer to that of a real face. It does not require expensive data acquisition, making the data collection process simple and cost-effective. At the same time, the stability of face expression capture is significantly improved compared to supervised training.
[0035] Furthermore, the facial expression capture model trained by the unsupervised facial expression capture model training method of this invention can estimate facial motion capture coefficients simply by inputting the image to be estimated, which can greatly improve the efficiency of facial expression capture.
[0036] In this embodiment, generating the corresponding first two-dimensional face image specifically includes the following steps: inputting a randomly selected preset prompt word into a pre-trained first encoding model to obtain the prompt word encoding; then inputting the prompt word encoding into a pre-trained second encoding model to obtain the image encoding features of the preset prompt word; randomly sampling a Gaussian noise image and an expression driving coefficient, and iteratively calculating according to the following formula:
[0037]
[0038] Where, x t-1 x represents the calculation result at time t-1. t This represents the calculation result at time t, α t The value represents the decay over time; text represents the image encoding features of the preset prompt word; and exp represents the expression-driven coefficient. The expression-driving coefficient represents the product of α from time t to the current time, where z is the Gaussian noise image, and σ is the expression-driving coefficient. t Let t be the mixing ratio; perform n iterations to generate the corresponding first two-dimensional face image, where n = t.
[0039] Preferably, α t α is a preset value; the mixing ratio can be set according to actual needs.
[0040] This invention generates a first two-dimensional face image with a higher degree of matching with preset prompts through iterative calculations, which lays the foundation for achieving high-accuracy facial expression capture in the future.
[0041] In this embodiment, calculating the preset loss according to the preset loss calculation formula includes at least: calculating a first preset loss according to a preset first loss calculation method, specifically the expression consistency loss; calculating a second preset loss according to a preset second loss calculation method, specifically the key point loss; calculating a third preset loss according to a preset third loss calculation method, specifically the training parameter direction control loss; calculating a fourth preset loss according to a preset fourth loss calculation method, specifically the texture loss; and calculating a fifth loss according to a preset fifth loss calculation method, the first preset loss, the second preset loss, the third preset loss, and the fourth preset loss, specifically the comprehensive loss.
[0042] In this embodiment, the first preset loss is calculated according to the preset first loss calculation method, specifically according to the following formula: loss_expression=l1(p_e-t_e); where loss_expression represents the expression consistency loss, l1 represents the norm loss function, p_e represents the second facial expression score vector, and t_e represents the first facial expression score vector.
[0043] This invention improves the ability of facial expression capture to capture emotional movements by calculating the loss of facial expression consistency.
[0044] In this embodiment, calculating the second preset loss according to the preset second loss calculation method includes at least the following steps: inputting the second two-dimensional face image into the pre-trained 2D keypoint regression network to obtain two-dimensional face keypoints; indexing the projection points of the three-dimensional face keypoints onto the first two-dimensional face image to obtain the points corresponding to the projection points and the two-dimensional face keypoints; determining the two-dimensional face keypoints that need to participate in the loss calculation based on the positive and negative values of the yaw angle in the face pose; and calculating the keypoint loss based on the MSE loss function and the two-dimensional face keypoints that need to participate in the loss calculation.
[0045] This invention improves the consistency between facial expression capture and key points by calculating key point loss, thereby enhancing the stability and accuracy of facial capture.
[0046] Preferably, the two-dimensional facial key points that need to be included in the loss calculation are those that are not occluded.
[0047] In this embodiment, the third preset loss is calculated according to the preset third loss calculation method, which specifically includes the following steps: inputting the second two-dimensional face image into the expression driving network to obtain face expression driving parameters; obtaining a set of random numbers within a preset range to represent the direction coefficients, and obtaining the total number of generated random numbers to represent the expression driving dimension;
[0048] Preferably, the preset range is three values: 1, 0, and -1, which represent different directions. Any one of these values can be generated, and the total number of random numbers generated is the number of -1, 0, and 1 generated.
[0049] Preferably, the facial expression driving parameters are specific numerical values.
[0050] The facial expression driving parameters are randomly processed according to a preset random processing formula, which is as follows:
[0051] new_expression=d_expression+direction*(random(0,0.3*d_expression);
[0052] Where new_expression represents the facial expression driving parameters obtained after random processing, d_expression represents the facial expression driving parameters, direction represents the direction coefficient, and random(0,0.3*d_expression) represents a randomly generated value, with the value ranging from [0,0.3*d_expression]. Based on a feedforward network (FFN), feature encoding is performed on the direction coefficients, and facial expression encoding is performed on the randomly processed facial expression driving parameters. The feature encoding results and facial expression encoding results are then input into the expression driving network. Based on the second facial expression score vector and the facial expression driving parameters, the training parameter direction control loss is calculated, and the specific calculation formula is as follows:
[0053] loss_expression_direction = sum(abs(sign(p_e-new_expression))); where loss_expression_direction represents the training parameter direction control loss, sum represents the summation function, abs represents the absolute value function, sign represents the sign function, p_e represents the second face expression score vector, and new_expression represents the face expression driving parameters obtained after random processing.
[0054] This invention improves the generalization ability and sensitivity of facial expression capture by calculating the direction control loss of training parameters.
[0055] Furthermore, the present invention calculates the third preset loss according to the preset third loss calculation method, and also considers the information of the previous frame in the facial expression capture in the video sequence. In actual use, the direction vector is calculated by the relationship between the facial points of the previous and next frames. By passing the direction vector to the network, the network can learn the relationship between the previous and next frames and the definite value of the current expression capture more clearly, thereby improving the accuracy of facial expression capture.
[0056] In this embodiment, the fourth preset loss is calculated according to the preset fourth loss calculation method, which specifically includes the following steps: rendering the face ID coefficient, expression driving coefficient, face pose and texture coefficient to obtain an image with texture information; calculating the pixel loss between the image with texture information and the first two-dimensional face image to obtain the texture loss.
[0057] This invention improves the precision of facial expression capture by calculating texture loss, achieving very high accuracy even for subtle facial expressions.
[0058] In this embodiment, the fifth loss is calculated based on the preset fifth loss calculation method, the first preset loss, the second preset loss, the third preset loss, and the fourth preset loss. Specifically, the calculation is performed according to the following formula:
[0059] loss=α*loss_expression+β*loss_lmk+δ*loss_expression_direction+σ*loss_texture;
[0060] Where α, β, δ, and σ represent the first, second, third, and fourth adjustment coefficients, respectively; loss_expression represents the expression consistency loss; loss_lmk represents the keypoint loss; loss_expression_direction represents the training parameter direction control loss; and loss_texture represents the texture loss.
[0061] like Figure 2 As shown, the present invention also provides a training system for an unsupervised facial expression capture model, comprising: a first two-dimensional face image generation module 10, used to acquire a set of preset prompt words for image generation, and randomly select a preset prompt word from the set to generate a corresponding first two-dimensional face image; a first face expression score vector acquisition module 20, used to input the first two-dimensional face image into a pre-trained emotion regression network to obtain a first face expression score vector; a face parameter acquisition module 30, used to input the first two-dimensional face image into an expression-driven network to obtain face ID coefficients, expression-driven coefficients, face pose and texture coefficients; and a three-dimensional face key... The point calculation module 40 is used to calculate the three-dimensional facial key points based on the face ID coefficient and expression driving coefficient; the projection and rendering processing module 50 is used to project the three-dimensional facial key points onto the first two-dimensional face image and perform rendering processing based on the pixel values in the first two-dimensional face image to obtain the second two-dimensional face image; the second facial expression score vector acquisition module 60 is used to input the second two-dimensional face image into the emotion regression network to obtain the second facial expression score vector; the loss calculation module 70 is used to calculate the preset loss according to the preset loss calculation method; and the training module 80 is used to train the facial expression capture model based on the loss calculation result using gradient backpropagation.
[0062] This invention also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement... Figure 1The method shown is based on an unsupervised facial expression capture model training method. The computer-readable storage medium can be a read-only memory, a disk, or an optical disk, etc.
[0063] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the device embodiments, equipment embodiments, and storage medium embodiments, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments.
[0064] Furthermore, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0065] The foregoing description illustrates and describes preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept by means of the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.
Claims
1. A method for training an unsupervised facial expression capture model, characterized in that, Includes the following steps: Obtain a set of preset prompt words for image generation, and randomly select one preset prompt word from the set to generate the corresponding first two-dimensional face image; The first two-dimensional face image is input into the pre-trained emotion regression network to obtain the first face expression score vector; the first two-dimensional face image is input into the expression-driven network to obtain face ID coefficient, expression-driven coefficient, face pose and texture coefficient; Based on the face ID coefficient and expression-driven coefficient, calculate the 3D face key points; The three-dimensional facial key points are projected onto the first two-dimensional facial image, and the second two-dimensional facial image is obtained by rendering based on the pixel values in the first two-dimensional facial image. The second two-dimensional face image is input into the emotion regression network to obtain the second face expression score vector; The preset loss is calculated according to the preset loss calculation method, and the facial expression capture model is trained by gradient backpropagation. Generating the corresponding first two-dimensional face image includes the following steps: A randomly selected pre-defined prompt word is input into the first pre-trained encoding model to obtain the prompt word encoding. Then, the prompt word encoding is input into the second pre-trained encoding model to obtain the image encoding features of the pre-defined prompt word. A random Gaussian noise image and an expression-driven coefficient are sampled, and the results are iteratively calculated according to the following formula: ； Where, x t-1 x represents the calculation result at time t-1. t This represents the calculation result at time t. The value represents the decay over time; text represents the image encoding features of the preset prompt word; and exp represents the expression-driven coefficient. express The product from time t to the current time, The image contains Gaussian noise. For mixing ratio; Perform n iterations to generate the corresponding first two-dimensional face image, where n=t; The preset loss is calculated according to the preset loss calculation method, and includes at least: The first preset loss is calculated according to the preset first loss calculation method. The first preset loss is specifically the expression consistency loss. The second preset loss is calculated according to the preset second loss calculation method. The second preset loss is specifically the key point loss. The third preset loss is calculated according to the preset third loss calculation method. The third preset loss is specifically the training parameter direction control loss. The fourth preset loss is calculated according to the preset fourth loss calculation method. The fourth preset loss is specifically the texture loss. The fifth loss is calculated based on the preset fifth loss calculation method, the first preset loss, the second preset loss, the third preset loss, and the fourth preset loss. The fifth loss is specifically a comprehensive loss.
2. The method for training an unsupervised facial expression capture model according to claim 1, characterized in that: The first preset loss is calculated according to the preset first loss calculation method, specifically according to the following formula: ;in, Let l_e represent the expression consistency loss, l1 represent the norm loss function, p_e represent the second face expression score vector, and t_e represent the first face expression score vector.
3. The method for training an unsupervised facial expression capture model according to claim 1, characterized in that: The calculation of the second preset loss according to the preset second loss calculation method includes at least the following steps: The second two-dimensional face image is input into the pre-trained 2D keypoint regression network to obtain two-dimensional face keypoints; Index the projection points of the 3D facial key points onto the first 2D facial image to obtain the points corresponding to the 2D facial key points; Based on the sign of the yaw angle in the face pose, determine the two-dimensional face key points that need to participate in the loss calculation; Calculate the keypoint loss based on the MSE loss function and the two-dimensional facial keypoints that need to be involved in the loss calculation.
4. The method for training an unsupervised facial expression capture model according to claim 1, characterized in that: The third preset loss is calculated according to the preset third loss calculation method, specifically including the following steps: The second two-dimensional face image is input into the expression-driven network to obtain the face expression driving parameters; Obtain a set of random numbers within a preset range to represent the direction coefficients, and obtain the total number of generated random numbers to represent the expression-driven dimension; The facial expression driving parameters are randomly processed according to a preset random processing formula, which is as follows: ； Where new_expression represents the facial expression driving parameters obtained after random processing, d_expression represents the facial expression driving parameters, direction represents the direction coefficient, and random(0,0.3) represents the direction coefficient. `d_expression` represents a randomly generated value, and the value range is [0, 0.3]. d_expression]; Based on the feedforward network, feature encoding is performed on the directional coefficients, and facial expression encoding is performed on the facial expression driving parameters obtained after random processing. The feature encoding results and facial expression encoding results are then input into the expression driving network. Based on the second facial expression score vector and the facial expression driving parameters, the training parameter orientation control loss is calculated. The specific calculation formula is as follows: _expression_direction=sum(abs(sign(p_e - new_expression))) ； Where loss_expression_direction represents the training parameter direction control loss, sum represents the summation function, abs represents the absolute value function, sign represents the sign function, p_e represents the second face expression score vector, and new_expression represents the face expression driving parameters obtained after random processing.
5. The method for training an unsupervised facial expression capture model according to claim 1, characterized in that: The fourth preset loss is calculated according to the preset fourth loss calculation method, specifically including the following steps: The face ID coefficient, expression-driven coefficient, face pose, and texture coefficient are rendered to obtain an image with texture information; The pixel loss is calculated by comparing the image with texture information with the first two-dimensional face image to obtain the texture loss.
6. The method for training an unsupervised facial expression capture model according to claim 1, characterized in that: The fifth loss is calculated based on the preset fifth loss calculation method, the first preset loss, the second preset loss, the third preset loss, and the fourth preset loss, specifically according to the following formula: ； in, , , , These represent the first adjustment coefficient, the second adjustment coefficient, the third adjustment coefficient, and the fourth adjustment coefficient, respectively. The loss represents the expression consistency loss, loss_lmk represents the keypoint loss, loss_expression_direction represents the training parameter direction control loss, and loss_texture represents the texture loss.
7. A system for training an unsupervised facial expression capture model, using the unsupervised facial expression capture model training method according to any one of claims 1-6, characterized in that, include: The first two-dimensional face image generation module is used to obtain a set of preset prompt words for image generation, and randomly select a preset prompt word from the set to generate the corresponding first two-dimensional face image; The first facial expression score vector acquisition module is used to input the first two-dimensional facial image into the pre-trained emotion regression network to obtain the first facial expression score vector. The face parameter acquisition module is used to input the first two-dimensional face image into the expression-driven network to obtain face ID coefficients, expression-driven coefficients, face pose and texture coefficients; The 3D facial landmark calculation module is used to calculate 3D facial landmarks based on the face ID coefficient and expression driving coefficient. The projection and rendering processing module is used to project the three-dimensional facial key points onto the first two-dimensional facial image, and to perform rendering processing based on the pixel values in the first two-dimensional facial image to obtain the second two-dimensional facial image. The second facial expression score vector acquisition module is used to input the second two-dimensional facial image into the emotion regression network to obtain the second facial expression score vector; The loss calculation module is used to calculate the preset loss according to the preset loss calculation method; The training module is used to train the facial expression capture model using gradient backpropagation based on the loss calculation results.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a training method program for an unsupervised facial expression capture model, which, when executed by a processor, implements the steps of the training method for an unsupervised facial expression capture model as described in any one of claims 1 to 6.