A facial image intelligent binding method, device and computer equipment
Intelligent binding of facial images through neural network technology solves the authenticity and fluency of facial expression binding in the existing technology, and realizes efficient and real dynamic binding of facial expressions and postures in animations or games.
Patent Information
- Application Number
- CN202311543676.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-17
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2043-11-17
AI Technical Summary
In the prior art, the authenticity of facial expression binding is reduced, making it difficult to accurately adjust facial expressions in animations or games, resulting in lag and poor results.
Using neural network technology, through time-series prediction network, feature extraction network, facial image restoration model and parameter calculation network, obtain facial videos for intelligent binding of facial images, and generate binding parameters to achieve automatic binding of dynamic facial expressions and postures. .
It improves the timing fluency and spatial authenticity of facial binding, improves the facial expression performance in animation or games, reduces production time and improves binding efficiency.
Smart Images

Figure CN117974852B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a facial image intelligent binding method, device and computer equipment. Background Art
[0002] Facial binding is an important step in game or animation production, which can make game characters or animation characters have facial expressions. For example, in the existing chess and war mobile games on the market, when making character expressions, the usual practice is to create a lot of bone points and skin in 3ds max or Maya, and use the scaling, rotation, and displacement of the bones to change the shape of the face.
[0003] However, when using 3ds max or Maya to create facial expressions, due to manual control, firstly, during the early design stage, the subjective judgment of the production staff makes it difficult to accurately grasp the actual effect of facial expression binding, which directly leads to a decrease in the authenticity of facial expression binding. Secondly, during post-production, it is difficult to accurately adjust the continuous expressions in the game or animation to have good time continuity, resulting in stuttering and poor effects in facial expression binding in game animations. If each expression is adjusted in place for time continuity, it is necessary to adjust the face point by point frame by frame, which not only wastes a lot of time, but also may not produce good results.
[0004] Therefore, in the prior art, due to the subjective effect judgment of the production staff, the authenticity of facial expression binding is reduced, and it is difficult to accurately adjust continuous expressions in games or animations to have good time continuity, resulting in lag in facial expression binding in game animations. Summary of the invention
[0005] The purpose of the present invention is to provide a facial image intelligent binding method to solve the technical problems of reduced authenticity of facial expression binding in the prior art and lag in facial expression binding in game animation.
[0006] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:
[0007] A facial image intelligent binding method comprises the following steps:
[0008] Acquire a face video, wherein the face video includes a set of first face images connected in time sequence;
[0009] According to a set of first face images, a set of second face images is obtained through a time series prediction network, wherein each second face image is located at the time series of each first face image, and the second face image corresponds to a time series prediction result of the set of first face images, and the time series prediction network is a neural network;
[0010] Performing feature extraction on the first face image and the second face image at each time sequence through a feature extraction network, and correspondingly obtaining bound facial features at each time sequence, wherein the feature extraction network is a neural network;
[0011] Perform facial image analysis on the bound facial features at each time sequence by using a facial image restoration model to obtain a facial image of a third person, wherein the facial image restoration model is a neural network;
[0012] The facial image binding parameters are calculated based on the facial image of the third person at each time sequence through a parameter calculation network to obtain the facial image binding parameters at each time sequence. The parameter calculation network is a neural network. The facial image binding parameters correspond to the Blender Shape binding parameters applied to the FLAME face model. The FLAME face model is used to realize automatic binding of the facial model to the facial image according to the Blender Shape binding parameters.
[0013] As a preferred solution of the present invention, a set of second face images is obtained through a time series prediction network according to a set of first face images, including:
[0014] Obtain the time sequence of each first face image and use them as the prediction time sequence in turn;
[0015] The facial images of the first person at the preceding time sequence of the prediction time sequence are subjected to deep learning by using the LSTM network to obtain a time sequence prediction model of the facial image of the second person at the predicted time sequence;
[0016] The second face facial image at each prediction time sequence is predicted using the time sequence prediction model at each prediction time sequence to obtain a group of second face facial images.
[0017] As a preferred solution of the present invention, the expression of the time series prediction network is:
[0018] G2 t =LSTM({G11,G12,G13,…,G1 t-1});
[0019] t∈[3,N];
[0020] G21=G11,G22=G12;
[0021] In the formula, G2 t is the facial image of the second person at the tth time sequence, G11, G12, G13, …, G1 t-1are the facial images of the first face at the 1st, 2nd, 3rd, and t-1 time sequences respectively, N is the total number of time sequences in the facial video, G21 is the facial image of the second face at the 1st time sequence, G22 is the facial image of the second face at the 2nd time sequence, and t is a counting variable.
[0022] As a preferred solution of the present invention, feature extraction is performed on the first face image and the second face image at each time sequence through a feature extraction network, and corresponding binding facial features at each time sequence are obtained, including:
[0023] The facial image of the first face and the facial image of the second face at each time sequence are simultaneously input into the feature extraction network, and the feature extraction network outputs the bound facial features at each time sequence.
[0024] As a preferred solution of the present invention, the construction of the feature extraction network includes:
[0025] Randomly selecting a first face image and a second face image at a plurality of time sequences as a first sample image and a second sample image, respectively;
[0026] Inputting the first sample image at each time sequence into the first CNN neural network, and the first CNN neural network outputs the facial features of the first sample image;
[0027] Inputting the second sample image at each time sequence into the second CNN neural network, and the second CNN neural network outputs the facial features of the second sample image;
[0028] The difference between the facial features of the first sample image and the facial features of the second sample image is used as the loss function;
[0029] Based on minimization of the loss function, the first CNN neural network and the second CNN neural network are trained to obtain the feature extraction network;
[0030] The expression of the feature extraction network is:
[0031]
[0032] H3 t =H1 t orH2 t ;
[0033] Loss = MSE(H1 t ,H2 t );
[0034] Where H3 t is the bound facial feature at the tth time sequence, H1 t is the facial feature of the first sample image at the tth time sequence, H2t is the facial feature of the second sample image at the tth time sequence, G1 t is the first sample image at the tth time sequence, G2 t is the second sample image at the tth time sequence, CNN1 is the first CNN neural network, CNN 2 is the second CNN neural network, Loss is the loss function value, MSE(H1 t ,H2 t ) is H1 t and H2 t is the mean square error between , t is a counting variable, and or is the mathematical identifier of or.
[0035] As a preferred solution of the present invention, facial image analysis is performed on the bound facial features at each time sequence by a facial image restoration model to obtain a facial image of a third person, including:
[0036] The bound facial features at each time sequence are input into the facial image restoration model, and the facial image restoration model outputs the facial image of the third person at each time sequence.
[0037] As a preferred solution of the present invention, the construction of the facial image restoration model includes:
[0038] Randomly selecting a first face image and a second face image at a plurality of time sequences as a first sample image and a second sample image, respectively;
[0039] Using a feature extraction network to obtain bound facial features corresponding to the first sample image and the second sample image at each time sequence as bound sample facial features at each time sequence;
[0040] Performing mean fusion on the first sample image and the second sample image at each time sequence to obtain a third sample image;
[0041] The facial features of each time-series binding sample are used as the input item of the SRCNN neural network, and the third sample image at each time sequence is used as the output item of the SRCNN neural network;
[0042] The SRCNN neural network is used to learn and train the input items and output items of the SRCNN neural network to obtain a facial image restoration model;
[0043] The expression of the facial image restoration model is:
[0044] G3 t =SRCNN(H3 t );
[0045] In the formula, G3 t is the third sample image at the tth time sequence, H3t is the facial feature of the bound sample at the tth time sequence, SRCNN is the SRCNN neural network, H3 t =H1 t orH2 t , H1 t is the facial feature of the first sample image at the tth time sequence, H2 t is the facial feature of the second sample image at the tth time sequence, and t is a counting variable.
[0046] As a preferred solution of the present invention, facial image binding parameters are calculated based on the facial images of the third person at each time sequence through a parameter calculation network to obtain facial image binding parameters at each time sequence, including:
[0047] Inputting the facial image of the third person at each time sequence into a parameter calculation network, and the parameter calculation network outputs the facial image binding parameters at each time sequence;
[0048] The construction of the parameter calculation network includes:
[0049] Marking the first sample image with facial image binding parameters to obtain the facial image binding parameters of the first sample image;
[0050] Using the first sample image as an input item of the BP neural network, and using the facial image binding parameter of the first sample image as an output item of the BP neural network;
[0051] Using the BP neural network to learn and train the input items and output items of the BP neural network to obtain the parameter measurement network;
[0052] S t =BP(G3 t );
[0053] In the formula, S t is the facial image binding parameter at the tth time sequence, G3 t is the facial image of the third person at the tth time sequence, BP is the BP neural network, and t is the counting variable.
[0054] As a preferred solution of the present invention, the present invention provides a facial image intelligent binding device, comprising:
[0055] A data acquisition module, used to acquire a face video of a person, wherein the face video of the person includes a set of first face images connected in time sequence;
[0056] A data processing module, used to obtain a set of second face images based on a set of first face images through a time series prediction network;
[0057] Used to extract features from the first face image and the second face image at each time sequence through a feature extraction network, and obtain corresponding binding facial features at each time sequence, wherein the feature extraction network is a neural network;
[0058] Used to perform facial image analysis on the bound facial features at each time sequence through a facial image restoration model to obtain a facial image of a third person, wherein the facial image restoration model is a neural network; and
[0059] Used to calculate facial image binding parameters according to the third face facial image at each time sequence through a parameter calculation network to obtain facial image binding parameters at each time sequence, wherein the parameter calculation network is a neural network, the facial image binding parameters correspond to the Blender Shape binding parameters applied to the FLAME face model, and the FLAME face model is used to realize automatic binding of the face model to the face facial image according to the Blender Shape binding parameters;
[0060] The data storage module is used to store the time series prediction network, feature extraction network, facial image restoration model and parameter measurement network.
[0061] As a preferred solution of the present invention, the present invention provides a computer device,
[0062] at least one processor; and
[0063] a memory communicatively coupled to the at least one processor;
[0064] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the computer device to execute the facial image intelligent binding method.
[0065] Compared with the prior art, the present invention has the following beneficial effects:
[0066] The present invention performs temporal prediction on facial images to obtain facial image prediction values that characterize the temporal regularity of facial expressions, performs feature extraction based on the facial image prediction values and the facial image true values, and obtains facial image binding parameters through binding parameter measurement, so that facial binding takes into account both temporal fluency and spatial authenticity, thereby improving the facial binding effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.
[0068] Figure 1 A flow chart of a facial image intelligent binding method provided by an embodiment of the present invention;
[0069] Figure 2 A block diagram of a facial image intelligent binding device provided by an embodiment of the present invention;
[0070] Figure 3 An internal structure diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0072] like Figure 1 As shown, the present invention provides a facial image intelligent binding method, comprising the following steps:
[0073] Acquire a face video, the face video comprising a set of first face images connected in time sequence;
[0074] According to a set of first face images, a set of second face images is obtained through a time series prediction network, wherein each second face image is located at the time series of each first face image, and the second face image corresponds to the time series prediction result of the set of first face images, and the time series prediction network is a neural network;
[0075] Performing feature extraction on the first face image and the second face image at each time sequence through a feature extraction network, and correspondingly obtaining bound facial features (such as facial expression features and facial posture features) at each time sequence, wherein the feature extraction network is a neural network;
[0076] The facial image restoration model is used to analyze the facial features bound at each time sequence to obtain a facial image of a third person, and the facial image restoration model is a neural network;
[0077] The facial image binding parameters are calculated according to the facial image of the third person at each time sequence through the parameter calculation network to obtain the facial image binding parameters at each time sequence. The parameter calculation network is a neural network. The facial image binding parameters correspond to the Blender Shape binding parameters applied to the FLAME face model. The FLAME face model is used to realize automatic binding of the face model to the facial image according to the Blender Shape binding parameters.
[0078] In order to realize face binding, the present invention comprehensively grasps the facial expressions and posture features of the face through the face video, and then performs face binding of the face model through the facial expressions and posture features grasped in the face video, so that the face image binding is more detailed and the display effect on the face model is better.
[0079] The present invention uses facial videos of human faces for facial binding, which can more fully grasp the expression and posture features required for binding, and performs binding in a time sequence, which can change the facial binding of the human face model from static binding to dynamic binding, that is, the human face model has dynamic facial expressions and facial postures, which is more suitable for use in animation and game characters, meets the dynamic expression requirements of animation games, can obtain dynamic facial binding at one time, and does not need to be made from static expressions and postures into dynamic expressions and postures, which greatly improves the efficiency of facial binding in characters.
[0080] Furthermore, in order to make the dynamic expressions and dynamic postures of the face model have a better binding effect, the present invention analyzes the face video on the one hand, uses the LSTM neural network to learn the changing rules of the face expressions and postures, and predicts the face expressions and postures in a single time series based on the learned changing rules of the face expressions and postures. The prediction of the face expressions and postures in a single time series conforms to the changing rules. Therefore, by using the predicted images of the face expressions and postures in a single time series, face binding is performed, which will make the expressions and postures in each time series in the face model obtained by face binding have time continuity, that is, the dynamic expressions and dynamic postures on the face model are smoother.
[0081] On the other hand, although the prediction of facial expressions and postures of a human face in a single time sequence is obtained based on the real facial expressions and postures in the face video and has a certain authenticity, the prediction of facial expressions and postures of a human face in a single time sequence is still virtual in nature. Therefore, if facial binding is performed only based on the predicted images of facial expressions and postures of the human face in a single time sequence, the authenticity of the dynamic expressions and dynamic postures on the face model will be reduced. In order to maintain the authenticity of the dynamic expressions and dynamic postures on the face model, the present invention performs facial binding based on the facial images of the human face with real facial expressions and postures in the face video, so that the expressions and postures at each time sequence in the face model obtained by facial binding have spatial authenticity, that is, the dynamic expressions and dynamic postures on the face model are more realistic.
[0082] The present invention performs facial binding based on a face image of a human face with real facial expressions and postures and a predicted image with virtual facial expressions and postures, and can maintain the spatial authenticity of the dynamic expressions and postures on the face model while maintaining the temporal continuity of the dynamic expressions and postures on the face model, so that the facial binding takes into account both authenticity and smoothness, and the facial binding effect is better.
[0083] The present invention analyzes human face videos, uses LSTM neural network to learn the changing rules of human face expressions and postures, and predicts human face expressions and postures in a single time series based on the learned changing rules of human face expressions and postures, as follows:
[0084] According to a set of first face images, a set of second face images is obtained through a time series prediction network, including:
[0085] Obtain the time sequence of each first face image and use them as the prediction time sequence in turn;
[0086] The facial images of the first person at the preceding time sequence of the prediction time sequence are subjected to deep learning by using the LSTM network to obtain a time sequence prediction model of the facial image of the second person at the predicted time sequence;
[0087] The second face facial image at each prediction time sequence is predicted using the time sequence prediction model at each prediction time sequence to obtain a group of second face facial images.
[0088] The expression of the time series prediction network is:
[0089] G2 t =LSTM({G11,G12,G13,…,G1 t-1});
[0090] t∈[3,N];
[0091] G21=G11,G22=G12;
[0092] In the formula, G2 t is the facial image of the second person at the tth time sequence, G11, G12, G13, …, G1 t-1 are the facial images of the first face at the 1st, 2nd, 3rd, and t-1 time sequences respectively, N is the total number of time sequences in the facial video, G21 is the facial image of the second face at the 1st time sequence, G22 is the facial image of the second face at the 2nd time sequence, and t is a counting variable.
[0093] The present invention performs facial binding based on a human face image with real facial expressions and postures and a predicted image with virtual facial expressions and postures, and performs facial binding together, which can maintain the spatial authenticity of dynamic expressions and dynamic postures on the human face model while maintaining the temporal continuity of dynamic expressions and dynamic postures on the human face model, as follows:
[0094] The feature extraction network is used to extract features from the first face image and the second face image at each time sequence, and corresponding binding facial features at each time sequence are obtained, including:
[0095] The facial image of the first face and the facial image of the second face at each time sequence are simultaneously input into the feature extraction network, and the feature extraction network outputs the bound facial features at each time sequence.
[0096] The construction of feature extraction network includes:
[0097] Randomly selecting a first face image and a second face image at a plurality of time sequences as a first sample image and a second sample image, respectively;
[0098] Inputting the first sample image at each time sequence into the first CNN neural network, and the first CNN neural network outputs the facial features of the first sample image;
[0099] Inputting the second sample image at each time sequence into the second CNN neural network, and the second CNN neural network outputs the facial features of the second sample image;
[0100] The difference between the facial features of the first sample image and the facial features of the second sample image is used as the loss function;
[0101] Based on minimization of the loss function, the first CNN neural network and the second CNN neural network are trained to obtain a feature extraction network;
[0102] The expression of the feature extraction network is:
[0103]
[0104] H3 t =H1 t orH2 t ;
[0105] Loss = MSE(H1 t ,H2 t );
[0106] Where H3 t is the bound facial feature at the tth time sequence, H1 t is the facial feature of the first sample image at the tth time sequence, H2 t is the facial feature of the second sample image at the tth time sequence, G1 t is the first sample image at the tth time sequence, G2 t is the second sample image at the tth time sequence, CNN1 is the first CNN neural network, CNN 2 is the second CNN neural network, Loss is the loss function value, MSE(H1 t ,H2 t ) is H1 t and H2 t is the mean square error between , t is a counting variable, and or is the mathematical identifier of or.
[0107] When extracting facial features (facial expression features, facial posture features, etc.), the present invention uses a constructed feature extraction network to perform extraction, wherein the network structure of the feature extraction network includes two CNN neural networks, which are respectively used to extract facial features in a face image (a first face image) with real facial expressions and postures, and to extract facial postures in a predicted image (a second face image) with virtual facial expressions and postures. During training, the output difference of the two CNN neural networks is used as a loss function to perform network training. The output difference of the two CNN neural networks is used as a loss function to perform network training, which makes the facial features in the face image (the first face image) with real facial expressions and postures outputted from the feature extraction network have the highest similarity with the facial features in the face image (the first face image) with real facial expressions and postures, so that the facial features outputted from the feature extraction network have both the first face image and the second face image, that is, the facial features outputted from the feature extraction network have both temporal fluency and spatial authenticity.
[0108] The present invention restores the facial image (third person face image) based on the facial features output from the feature extraction network, and also has both temporal fluency and spatial authenticity. Therefore, the facial image binding parameters obtained based on the third person face image can make the facial expressions and postures in the bound face model have temporal fluency and spatial authenticity.
[0109] The facial image restoration model is used to analyze the facial features bound at each time sequence to obtain the facial image of the third person, including:
[0110] The bound facial features at each time sequence are input into the facial image restoration model, and the facial image restoration model outputs the facial image of the third person at each time sequence.
[0111] The construction of the facial image restoration model includes:
[0112] Randomly selecting a first face image and a second face image at a plurality of time sequences as a first sample image and a second sample image, respectively;
[0113] Using a feature extraction network to obtain bound facial features corresponding to the first sample image and the second sample image at each time sequence as bound sample facial features at each time sequence;
[0114] Performing mean fusion on the first sample image and the second sample image at each time sequence to obtain a third sample image;
[0115] The facial features of each time-series binding sample are used as the input item of the SRCNN neural network, and the third sample image at each time sequence is used as the output item of the SRCNN neural network;
[0116] The SRCNN neural network is used to learn and train the input items and output items of the SRCNN neural network to obtain a facial image restoration model;
[0117] The expression of the facial image restoration model is:
[0118] G3 t =SRCNN(H3 t );
[0119] In the formula, G3 t is the third sample image at the tth time sequence, H3 t is the facial feature of the bound sample at the tth time sequence, SRCNN is the SRCNN neural network, H3 t =H1 t orH2 t , H1 t is the facial feature of the first sample image at the tth time sequence, H2 t is the facial feature of the second sample image at the tth time sequence, and t is a counting variable.
[0120] The facial image binding parameters are calculated based on the facial image of the third person at each time sequence through the parameter calculation network to obtain the facial image binding parameters at each time sequence, including:
[0121] Inputting the facial image of the third person at each time sequence into a parameter calculation network, and the parameter calculation network outputs the facial image binding parameters at each time sequence;
[0122] The construction of parameter estimation network includes:
[0123] Marking the first sample image with facial image binding parameters to obtain the facial image binding parameters of the first sample image;
[0124] Using the first sample image as an input item of the BP neural network, and using the facial image binding parameter of the first sample image as an output item of the BP neural network;
[0125] The BP neural network is used to learn and train the input items and output items of the BP neural network to obtain a parameter measurement network;
[0126] S t =BP(G3 t );
[0127] In the formula, S t is the facial image binding parameter at the tth time sequence, G3 t is the facial image of the third person at the tth time sequence, BP is the BP neural network, and t is the counting variable.
[0128] like Figure 2 As shown, the present invention provides a facial image intelligent binding device, comprising:
[0129] A data acquisition module, used for acquiring a face video, wherein the face video includes a set of first face images connected in time sequence;
[0130] A data processing module, used to obtain a set of second face images based on a set of first face images through a time series prediction network;
[0131] Used to extract features from the first face image and the second face image at each time sequence through a feature extraction network, and obtain corresponding bound facial features at each time sequence, wherein the feature extraction network is a neural network;
[0132] Used to perform facial image analysis on the bound facial features at each time sequence through a facial image restoration model to obtain a facial image of a third person, wherein the facial image restoration model is a neural network; and
[0133] Used to calculate facial image binding parameters according to the facial image of the third person at each time sequence through a parameter calculation network to obtain facial image binding parameters at each time sequence, the parameter calculation network is a neural network, the facial image binding parameters correspond to the Blender Shape binding parameters applied to the FLAME face model, and the FLAME face model is used to realize automatic binding of the face model to the facial image according to the Blender Shape binding parameters;
[0134] The data storage module is used to store the time series prediction network, feature extraction network, facial image restoration model and parameter measurement network.
[0135] like Figure 3 As shown, the present invention provides a computer device,
[0136] at least one processor; and
[0137] a memory communicatively coupled to the at least one processor;
[0138] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the computer device to perform the facial image intelligent binding method.
[0139] The present invention performs temporal prediction on facial images to obtain facial image prediction values that characterize the temporal regularity of facial expressions, performs feature extraction based on the facial image prediction values and the facial image true values, and obtains facial image binding parameters through binding parameter measurement, so that facial binding takes into account both temporal fluency and spatial authenticity, thereby improving the facial binding effect.
[0140] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.
Claims
1. A facial image intelligent binding method, characterized by: The following steps are involved: Acquire a face video, wherein the face video includes a set of first face images connected in time sequence; Obtaining a set of second facial images from a set of first facial images through a time series prediction network, wherein each second facial image is located at a time series position corresponding to each first facial image, and the second facial images correspond to time series prediction results of the set of first facial images, and the time series prediction network is a neural network; Performing feature extraction on the first and second facial images at each time sequence through a feature extraction network, thereby obtaining corresponding bound facial features at each time sequence, wherein the feature extraction network is a neural network; Performing facial image analysis on the bound facial features at each time sequence using a facial image restoration model, to obtain a facial image of a third person, wherein the facial image restoration model is a neural network; Calculating facial image binding parameters based on the facial image of the third person at each time sequence using a parameter calculation network to obtain facial image binding parameters at each time sequence, wherein the parameter calculation network is a neural network, and the facial image binding parameters correspond to Blender Shape binding parameters applied to a FLAME face model, wherein the FLAME face model is used to automatically bind the facial model to the facial image according to the Blender Shape binding parameters; The construction of feature extraction network includes: Randomly selecting a first face image and a second face image at a plurality of time sequences as a first sample image and a second sample image, respectively; Inputting the first sample image at each time sequence into the first CNN neural network, and having the first CNN neural network output the facial features of the first sample image; Inputting the second sample image at each time sequence into the second CNN neural network, and having the second CNN neural network output the facial features of the second sample image; The difference between the facial features of the first sample image and the facial features of the second sample image is used as the loss function; Based on the minimization of the loss function, the first CNN neural network and the second CNN neural network are trained to obtain a feature extraction network; The expression of the feature extraction network is: ; H3 t =H1 t or H2 t ; Loss=MSE(H1 t ,H2 t ); Where H3 t is the bound facial feature at the tth time sequence, H1 t is the facial feature of the first sample image at the tth time sequence, H2 t is the facial feature of the second sample image at the tth time sequence, G1 t is the first sample image at the tth time sequence, G2 t is the second sample image at the t-th time sequence, CNN1 is the first CNN neural network, CNN2 is the second CNN neural network, Loss is the loss function value, MSE(H1 t ,H2 t ) is H1 t and H2 t The mean square error between , t is a counting variable, and or is the mathematical identifier of or.
2. The facial image intelligent binding method according to claim 1, characterized in that: According to a set of first face images, a set of second face images is obtained through a time series prediction network, including: Obtain the time sequence of each first face image and use them as the prediction time sequence in turn; The facial images of the first person at the time preceding the prediction time sequence are subjected to deep learning using the LSTM network to obtain a time series prediction model for the facial images of the second person at the prediction time sequence; The time series prediction model at each prediction time series is used to predict the second face facial image at each prediction time series to obtain a group of second face facial images.
3. The facial image intelligent binding method according to claim 2, characterized in that: The expression of the time series prediction network is: G2 t =LSTM({G11,G12, G13,…,G1 t-1 }); ; G21= G11, G22= G12; Where G2 t is the second face image at the tth time sequence, G11, G12, G13,…, G1 t-1 are the facial images of the first face at the 1st, 2nd, 3rd, and t-1 time sequences respectively, N is the total number of time sequences in the facial video, G21 is the facial image of the second face at the 1st time sequence, G22 is the facial image of the second face at the 2nd time sequence, and t is a counting variable.
4. The facial image intelligent binding method according to claim 3, characterized in that: The feature extraction network is used to extract features from the first face image and the second face image at each time sequence, and corresponding bound facial features at each time sequence are obtained, including: The facial image of the first face and the facial image of the second face at each time sequence are simultaneously input into the feature extraction network, and the feature extraction network outputs the bound facial features at each time sequence.
5. The facial image intelligent binding method according to claim 4, characterized in that: The facial image restoration model is used to analyze the facial features bound at each time sequence to obtain the third person's facial image, including: The bound facial features at each time sequence are input into the facial image restoration model, and the facial image restoration model outputs the facial image of the third person at each time sequence.
6. The facial image intelligent binding method according to claim 5, characterized in that: The construction of the facial image restoration model includes: Randomly selecting a first face image and a second face image at a plurality of time sequences as a first sample image and a second sample image, respectively; Using a feature extraction network to obtain bound facial features corresponding to the first sample image and the second sample image at each time sequence as bound sample facial features at each time sequence; Performing mean fusion on the first sample image and the second sample image at each time sequence to obtain a third sample image; The facial features of each time-series binding sample are used as the input of the SRCNN neural network, and the third sample image at each time sequence is used as the output of the SRCNN neural network; The SRCNN neural network is used to learn and train the input items and output items of the SRCNN neural network to obtain a facial image restoration model; The expression of the facial image restoration model is: G3 t =SRCNN(H3 t ); Where G3 t is the third sample image at the tth time sequence, H3 t is the facial feature of the bound sample at the tth time sequence, SRCNN is the SRCNN neural network, H3 t =H1 t or H2 t , H1 t is the facial feature of the first sample image at the tth time sequence, H2 t is the facial feature of the second sample image at the tth time sequence, and t is a counting variable.
7. The facial image intelligent binding method according to claim 6, characterized in that: The facial image binding parameters are calculated based on the third person's facial image at each time sequence through the parameter calculation network, and the facial image binding parameters at each time sequence are obtained, including: Inputting the facial image of the third person at each time sequence into the parameter calculation network, and the parameter calculation network outputs the facial image binding parameters at each time sequence; The construction of the parameter calculation network includes: Marking the first sample image with facial image binding parameters to obtain the facial image binding parameters of the first sample image; Using the first sample image as an input item of the BP neural network and using the facial image binding parameter of the first sample image as an output item of the BP neural network; Using a BP neural network to learn and train the input items and output items of the BP neural network to obtain the parameter measurement network; S t =BP(G3 t ); Where S t is the facial image binding parameter at the tth time sequence, G3 t is the facial image of the third person at the tth time sequence, BP is the BP neural network, and t is the counting variable.
8. A facial image intelligent binding device, characterized in that: include: A data acquisition module is used to acquire a face video of a human face, wherein the face video of the human face includes a set of first face images connected in time sequence; a data processing module configured to obtain, based on a set of first facial images, a set of second facial images through a time series prediction network, wherein each second facial image is located at a time sequence corresponding to each first facial image, and the second facial images correspond to time series prediction results of the set of first facial images, and the time series prediction network is a neural network; for performing feature extraction on the first face image and the second face image at each time sequence through a feature extraction network, and obtaining corresponding bound facial features at each time sequence, wherein the feature extraction network is a neural network; Used to perform facial image analysis on the bound facial features at each time sequence by using a facial image restoration model to obtain a facial image of a third person, wherein the facial image restoration model is a neural network; and for calculating facial image binding parameters based on a third face image at each time sequence using a parameter calculation network to obtain facial image binding parameters at each time sequence, wherein the parameter calculation network is a neural network, the facial image binding parameters correspond to Blender Shape binding parameters applied to a FLAME face model, and the FLAME face model is used to automatically bind the face model to the face image according to the Blender Shape binding parameters; Data storage module, used to store time series prediction network, feature extraction network, facial image restoration model and parameter measurement network; The construction of the feature extraction network includes: Randomly selecting a first face image and a second face image at a plurality of time sequences as a first sample image and a second sample image, respectively; Inputting the first sample image at each time sequence into the first CNN neural network, and having the first CNN neural network output the facial features of the first sample image; Inputting the second sample image at each time sequence into the second CNN neural network, and having the second CNN neural network output the facial features of the second sample image; The difference between the facial features of the first sample image and the facial features of the second sample image is used as the loss function; Based on the minimization of the loss function, the first CNN neural network and the second CNN neural network are trained to obtain a feature extraction network; The expression of the feature extraction network is: ; H3 t =H1 t or H2 t ; Loss=MSE(H1 t ,H2 t ); Where H3 t is the bound facial feature at the tth time sequence, H1 t is the facial feature of the first sample image at the tth time sequence, H2 t is the facial feature of the second sample image at the tth time sequence, G1 t is the first sample image at the tth time sequence, G2 t is the second sample image at the t-th time sequence, CNN1 is the first CNN neural network, CNN2 is the second CNN neural network, Loss is the loss function value, MSE(H1 t ,H2 t ) is H1 t and H2 t The mean square error between , t is a counting variable, and or is the mathematical identifier of or.
9. A computer device, characterized in that: at least one processor; as well as a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the computer device to perform the method according to any one of claims 1 to 7.
