Face key point detection method based on generated network generated data in complex environment

Abstract

The application provides a method for complex environment face key point detection based on generated data of a generation network, comprising: obtaining corresponding three-dimensional synthetic face texture structure information files by using self-defined different face attribute parameter settings in a FaceGen model, and then constructing two-dimensional synthetic image data sets under different angles and occlusion conditions through three-dimensional reconstruction, and uniformly cutting into a fixed size; for a real data set, removing complex background interference by using an optimized DeepLab segmentation algorithm; converting the synthetic face data set obtained by the model into a simulated synthetic image data set with real face feature information by means of an optimized conversion model S->R based on UGATIT; and using the sum of the real data set with complex background interference removed and the simulated synthetic image data set converted by the S->R model as a model training data set, and training an improved Hourglass face key point detection model.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and computer vision, specifically to a method for detecting facial key points in complex environments based on data generated by generative networks. Background Technology

[0002] Face-related technologies have matured over the past few decades. However, the lack of real-world training data and issues such as privacy violations and data misuse caused by face applications have sparked global controversy. Intelligent models, in order to achieve higher accuracy, often desire more diverse and richer training data. In reality, relying on manual collection of training data and annotation of large datasets is becoming increasingly difficult. Different applications of face-related research have different data requirements, and the requirements for training images depend on the complexity of the scene, the number of project categories, and accuracy. Currently, facial landmark detection tasks have achieved high accuracy in detecting frontal and near-frontal face images. However, face datasets based on complex scenes face challenges such as difficulty in training data annotation, data imbalance, and blurred or even invisible landmarks. This makes existing landmark detection tasks unable to solve the problem of accurate localization under large-angle planar rotations, and also makes it difficult to train a unified model for them.

[0003] With the rapid development and advancement of image imaging, researchers are also using 3D models to create synthetic datasets. Many tasks are attempting to use synthetic datasets to minimize the cost and risk of training network models. Synthetic datasets can effectively reduce the development cost of computer vision applications while achieving privacy protection. This is because privacy limits the availability of real-world data or dictates how the dataset is used. When real-world data may not exist or does not meet specific conditions or requirements, synthetic datasets compensate for many of the shortcomings of manually collecting and labeling real-world data. However, existing synthetic techniques face challenges such as training instability, vanishing gradients, and pattern collapse. Especially when dealing with small sample data, they cannot learn all the features of the modeled image, overemphasizing a few image features, which easily leads to artifact drift and generation. Furthermore, synthetic images lack realism, and models trained using synthetic images perform poorly when generalizing to real-world test scenarios. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method for detecting facial landmarks in complex environments based on data generated by a generative network, comprising the following steps:

[0005] Step 1: Use the FaceGen Modeller model to customize different face attribute parameter settings to obtain the corresponding 3D synthetic face texture structure information file, and then construct a 2D synthetic face image dataset under different angles and occlusion conditions through 3D reconstruction, and uniformly crop it to a fixed size;

[0006] Step 2: For the real face image dataset, use the optimized DeepLab segmentation algorithm to remove complex background interference;

[0007] Step 3: Using the S→R conversion model optimized based on the UGATIT generative network structure, the two-dimensional synthetic face image dataset obtained in Step 1 is converted into a simulated synthetic face image dataset with real face feature information.

[0008] Step 4: Use the real face image dataset obtained in Step 2 (with complex background interference removed) and the simulated synthetic face image dataset obtained in Step 3 as the model training dataset to train the improved Hourglass face landmark detection model.

[0009] Furthermore, in step 1, the custom facial attribute parameters include: age, race, gender, hairstyle, and expression style. By adjusting the above attribute parameters, different facial texture structure information files are randomly generated.

[0010] Furthermore, in step 2, the background interference removal based on the optimized DeepLab segmentation algorithm is specifically as follows: In the feature encoding stage, feature extraction is performed on the image to generate deep features and shallow features respectively. The resolution of the output feature map is controlled by a dilated convolutional network to give the network receptive fields of different features. Then, the number of channels is adjusted using convolutional layers. In the feature decoding stage, the shallow features generated by the backbone feature extraction network and the output features after feature encoding are transformed into the same shape for feature fusion and then upsampling is performed to gradually restore spatial information.

[0011] Furthermore, in step 3,

[0012] The transformation model S→R, optimized based on the UGATIT generative network structure, contains two generators (G... x G y ) model and two discriminators (D x D y The model takes the real face image (x∈X) obtained in step 2 and the synthetic face image (y∈Y) obtained in step 1 as input;

[0013] Generator (G) x G y The model employs two hourglass modules before its encoder and after its decoder, adds an attention mechanism (SE) to the ResNet module for feature extraction in its encoder, and introduces a class activation map (CAM) between its encoder and decoder.

[0014] Generator (G) x G yIn the model, the decoder design is symmetrical to the encoder;

[0015] Adaptive layer instance normalization (Soft-AdaLIN) is added to each layer of the generator and discriminator to normalize and adjust the feature maps;

[0016] Discriminator (D) x D y The design employs a combination of global discriminators and local discriminators.

[0017] Furthermore, the cost function is modified to obtain a better optimization objective to balance the loss between the generative and discriminative models in the transformation model S→R optimized based on the UGATIT generative network structure.

[0018] Furthermore, modifying the cost function specifically includes:

[0019] 6.1 Replace the cross-entropy loss function with one based on least squares (L) LsGAN Optimization of )

[0020]

[0021] Where x represents the synthesized face image data, P represents the distribution of x from the synthetic face image dataset. x Probability, D x (x) represents the discrimination of synthetic face image data, G x (x) represents the use of generator G x The generated synthetic face image data, D x (G x (x) represents the discrimination of the generated synthetic face image data. P represents the distribution of the synthetic face image data x from the real face image dataset y. y probability;

[0022] 6.2 Employing the Cyclic Consistency Loss Function (L) Cyc This ensures that the image can be converted back to the original target domain.

[0023]

[0024] Among them, G y (G x (x) represents the reconstructed synthetic face image data, and y represents the real face image dataset. P represents the distribution of the real face image data from a real face image dataset y. y Probability, G x (G y (y) represents the reconstructed real face image dataset;

[0025] 6.3 Utilizing the ontological function (L) Ide Ensure that the color distribution of the input and output images is similar:

[0026]

[0027] Among them, G y (x) represents inputting the synthesized face image data x into G. y Reconstruction is performed in the generator, G x (y) represents inputting the real face image data y into G. x Reconstruction is performed in the generator;

[0028] 6.4 The CAM loss utilizes the generator-aided classifier η x Discriminator-assisted classifier ηD x Information:

[0029]

[0030] Where, η x (x) represents inputting the synthetic face image data x into the generator-aided classifier for classification, obtaining the predicted category of the generated image, and log(·) is the conversion of the classification model's prediction into a probability distribution form; ηD x (x) represents the classification of the input synthetic face image data x using a discriminator-assisted classifier, ηD x (G x (x) represents the use of generator G. x The generated synthetic face image data G x To determine the category of (x);

[0031] 6.5 Face Recognition Identity Loss Function (L) Facei d) uses pre-trained face recognition models like MobileFaceNets, or M-models, to extract facial landmark features from the input and generate the output:

[0032]

[0033] Where cos(·) represents the calculation of the cosine, M(x) and M(y) represent the input of x and y into the M model and the output of the corresponding facial key point features, respectively, and M(G x (x)),M(G y (y) respectively represent the generated data G x (x),G y (y) is input into the M model and outputs the corresponding facial key point features;

[0034] 6.6 pixel loss function (LPixel ) is used to preserve lower-level features after style transformation:

[0035]

[0036] Furthermore, in step 4, the improvement to the Hourglass face landmark detection model is as follows: Based on the Hourglass model, residual blocks optimized using an attention mechanism are added to reduce the dimensionality of the image data and store important information about the image; multiple Hourglass blocks are then directly concatenated to utilize the complementarity between features; the output of the residual blocks optimized using the attention mechanism is obtained by the following formula:

[0037] H(x n )=F(x n ,w n )+x n (7)

[0038] Wherein, H(x) n ) represents the output of the residual block, x n For a specific input part, w n Let F(·) be the weight, and F(·) be the function that extracts and transforms the features of the input.

[0039] The loss function L of the Hourglass face landmark detection model is:

[0040]

[0041] Where p(i,j) and g(i,j) represent the predicted heatmap and the ground reality heatmap at pixel position (i,j), respectively; N is the number of keypoints.

[0042] The method of this invention can reduce the domain difference between synthetic data and real datasets, enhancing the realism of synthetic images. The model is jointly trained with the generated simulated synthetic data and real-world data, achieving high performance even with limited real-world data, exhibiting high accuracy and strong generalization ability. Attached Figure Description

[0043] Figure 1 This is a flowchart of the facial landmark detection method in this invention;

[0044] Figure 2 This is a schematic diagram of the face synthesis process using FaceGen Modeller, as described in this invention.

[0045] Figure 3 This is a schematic diagram of the image background interference removal preprocessing procedure of the present invention;

[0046] Figure 4This is a diagram of the improved generative network model generator architecture of the present invention;

[0047] Figure 5 This is a diagram of the improved Hourglass face landmark detection model of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Figure 1 This invention illustrates a flowchart of a method for detecting facial landmarks in complex environments based on data generated by a generative network; as shown. Figure 1 As shown, the method includes the following:

[0050] 1. Acquisition of synthetic face image data

[0051] This invention uses FaceGen Modeller to generate synthetic face images. FaceGen Modeller has the function of customizing face attributes, and can define the attributes that make up the face using a parameterized method. This model allows users to edit facial information through multiple control parameters, such as age, ethnicity, gender, hairstyle, expression style, and other accessories like glasses, to randomize, differentiate, and standardize the face. Based on different parameter settings, corresponding 3D texture structure information files related to the synthetic face are obtained. This 3D information is used to reconstruct the face, thereby further extracting 2D synthetic face datasets from different angles and under different occlusions.

[0052] Figure 2 Examples of randomly generated faces are given by adjusting the gender parameter (male or female), race parameter (South Africa, East Asia, Europe), and expression parameter (neutral, surprised, smiling, angry, disgusted) of FaceGen Modeller.

[0053] 2. Background Removal Preprocessing

[0054] Considering the excessive interference from complex backgrounds in real-world datasets, image transformation under such backgrounds often fails. Therefore, preprocessing steps are performed on the training dataset before input to limit the complexity of the data distribution and avoid poor generation quality. However, in previous semantic segmentation networks, removing complex background interference often resulted in coarse segmentation results, mainly due to two reasons: information loss caused by pooling and failure to utilize the probabilistic relationships between labels. To address these issues, this invention uses a pre-trained DeepLab model for preprocessing. Figure 3 This is a schematic diagram illustrating the image background interference removal preprocessing procedure of the present invention. Figure 3 The face images shown are from the Helen subset of the 300-W dataset; download path:

[0055] (http: / / www.ifp.illinois.edu / ~vuongle2 / helen / ), First, DeepLab further improves performance by utilizing image-level features encoded by global context. In the Encoder stage, Atrous convolutions are used to extract image features and avoid information loss caused by pooling layers. Simultaneously, a 1x1 convolution, three 3x3 dilated convolution kernels, and a global pooling layer are used to effectively capture multi-scale information, followed by a 1x1 convolution to reduce the number of channels. In the Decoder stage, the intermediate outputs of the backbone network and the feature-encoded outputs are transformed into the same shape and then concatenated together for a 3x3 convolution. After preprocessing, the image discards background interference information and focuses only on the facial region.

[0056] Furthermore, real-world face datasets suffer from inconsistent sizes and small sample sizes, making image augmentation preprocessing a crucial step. The training set is augmented and resized to a uniform 256x256 pixels.

[0057] 3. S→R transformation model based on UGATIT

[0058] The biggest challenge in image synthesis is the lack of realism in synthesized images; there is a deviation between ideal synthesized data and real data. The lack of realism in synthesized images also means that the model only learns the details of the synthesized image but cannot generalize well to real images.

[0059] The proposed generative network-based transformation model S→R converts the synthetic face dataset generated by FaceGen Modeller into a simulated synthetic image dataset with real facial feature information. The S→R transformation model further improves upon the traditional UGATIT network, taking preprocessed real face images (x∈X) and initial synthetic images (y∈Y) generated by the FaceGen model as input. The network structure includes two generators (G... x G y ) and two discriminators (D x D y ). Figure 4 The improved generative network model generator architecture of this invention is presented. The generator's function is to acquire the input image and perform transformations to produce the target image, while the discriminator's function is to better distinguish between real and synthetic data. Throughout the objective function optimization process, least squares, cycle consistency functions, ontology mapping functions, CAM loss, face recognition identity loss, and pixel loss functions are introduced to achieve a better optimization objective and balance the losses between the generative and discriminative models.

[0060] This invention proposes an improved S→R conversion model based on UGATIT, comprising a generator and a discriminator. The generator's encoder feature extraction module utilizes a ResNet module (Res_SEBlock2) with an added Squeeze-and-Excitation (SE) attention mechanism to emphasize effective information and suppress ineffective information through channel weighting. A convolutional layer (Conv) and an hourglass module are added before encoding to improve model performance. The encoder gradually reduces the spatial resolution of the feature maps while enhancing semantic information; the output feature map of each encoder layer is also used as input to the decoder. Between the encoder and decoder, a class activation map (CAM) is introduced, with the input being the feature representation of the source image encoded by the encoder. This mechanism utilizes attention calculations between the encoder and decoder feature maps to improve attention to specific regions and details, enabling the model to adaptively adjust the details and local structures in the generated image. The decoder decodes and reconstructs the feature maps extracted by the encoder, generating an image matching the target domain. Its design is symmetrical to the encoder; the decoder gradually increases the spatial resolution of the feature maps, recovering more detailed generated results. To enhance the diversity of generated images, adaptive layer instance normalization (Soft-AdaLIN) is added after each layer in the decoder. This normalizes and adjusts the feature maps to balance the diversity of generated images with the preservation of detail clarity. The discriminator is designed with a combination of global and local discriminators. The difference between the global and local discriminators is that the global discriminator compresses the input image at a deeper level. This combination ensures both local and global consistency in the generated images.

[0061] At the same time, the loss between the generative and discriminative models is balanced by modifying the cost function to obtain a better optimization objective. Specifically:

[0062] (1) Replace the cross-entropy loss function with a least squares-based (L) loss function. LsGAN The optimization attempts to use different distance metrics to build a more stable and faster-converging network.

[0063] (1)

[0064] Where x represents the synthesized face image data, P represents the distribution of x from the synthetic face image dataset. x Probability, D x (x) represents the discrimination of synthetic face image data, G x (x) represents the use of generator G x The generated synthetic face image data, D x (Gx (x) represents the discrimination of the generated synthetic face image data. P represents the distribution of the synthetic face image data x from the real face image dataset y. y Probability.

[0065] (2) To ensure that the samples generated by the generator have the same distribution as the samples in the real world, the cycle consistency loss function (L) is used. Cyc This ensures that the image can be converted back to the original target domain.

[0066]

[0067] Among them, G y (G x (x) represents the reconstructed synthetic face image data, and y represents the real face image dataset. P represents the distribution of the real face image data from a real face image dataset y. y Probability, G x (G y (y) represents the reconstructed real face image dataset.

[0068] (3) When optimizing the generator, utilize the ontology projection function (L) Ide Ensure that the color distribution of the input and output images is similar.

[0069]

[0070] Among them, G y (x) represents inputting the synthesized face image data x into G. y Reconstruction is performed in the generator, G x (y) represents inputting the real face image data y into G. x Reconstruction is performed in the generator.

[0071] (4) The CAM loss utilizes the generator-aided classifier η. x Discriminator-assisted classifier ηD x This information. Such losses can be better concentrated in areas where the source and target domains differ.

[0072]

[0073] Where, η x (x) represents inputting the synthetic face image data x into the generator-aided classifier for classification, obtaining the predicted category of the generated image, and log(·) is the conversion of the classification model's prediction into a probability distribution form; ηD x (x) represents the classification of the input synthetic face image data x using a discriminator-assisted classifier, ηDx (G x (x) represents the use of generator G. x The generated synthetic face image data G x The category of (x) is determined.

[0074] (5) Face recognition identity loss function (L Faceid The method uses a pre-trained face recognition model, MobileFaceNets(M), to extract ID features from the input and generate the output. This loss uses cosine distance to constrain the ID information of the generated output to be as similar as possible to the input.

[0075]

[0076] Where cos(·) represents the calculation of the cosine, M(x) and M(y) represent the input of x and y into the M model and the output of the corresponding facial key point features, respectively, and M(G x (x)),M(G y (y) respectively represent the generated data G x (x),G y (y) is input into the M model and outputs the corresponding facial key point features.

[0077] (6) Pixel loss function (L) Pixe l) is used to preserve lower-level features after style transformation.

[0078]

[0079] 4. Face landmark detection model based on Hourglass

[0080] The training set for the keypoint detection model in this invention consists of a simulated synthetic dataset after S→R transformation and a small amount of real-world data. These two datasets are jointly trained to learn the detection model. Combining synthetic and real-world data achieves high performance even with limited real-world data. This design not only minimizes privacy issues in face recognition tasks but also introduces new variations from real-world data.

[0081] The facial landmark detection model is based on the structure of the HourGlass network to estimate facial landmark heatmap information. For example... Figure 5As shown, this model utilizes convolutional layers and residual blocks (Res_SEBlock1) optimized with SE attention in the feature extraction module to store important information about the image, better understanding and representing the input image. Then, multiple Hourglass modules are directly concatenated. Hourglass is an asymmetric top-down and bottom-up fully convolutional network, which includes upsampling using deconvolution to increase the resolution of the input feature map. After a series of convolutional layers, the features are further processed and extracted to capture information at different spatial scales. In the Hourglass module, the feature map is divided into a top fine-grained output branch and a bottom coarse-grained output branch, allowing for processing at the next level based on the features from the previous level. The Hourglass module spans multiple scales to extract features at different levels, while residual connections add the bottom coarse-grained output to the input of the top fine-grained branch, thus preserving and transmitting detailed information at different levels. Stacking multiple Hourglasses allows information to connect between different resolutions and levels, better utilizing the complementarity between features to obtain contextual and detailed information. The subsequent stacked convolutional layers and Res_SEBlock1 module can fuse and integrate the extracted features, improving the model's ability to perceive key points.

[0082] The model of this invention designs an optimized residual block (Res_SEBlock1), adds an attention SE channel to the connection, and provides attention to a specific input part (x). n Different weights (w) are learned through F(·). n The mechanism of ) for input x n The resulting feature map is processed so that the model focuses only on the local salient features of the target-related image. H(x) n The output of the residual block can be represented as follows:

[0083] H(x n )=F(x n ,w n )+x n (7)

[0084] Wherein, H(x) n ) represents the output of the residual block, x n For a specific input part, w n Let F(·) be the weight, and F(·) be the function that extracts and transforms the features of the input.

[0085] During training, the following loss function is optimized:

[0086]

[0087] Where p(i,j) and g(i,j) represent the predicted heatmap and the ground reality heatmap at pixel position (i,j), respectively, and N is the number of keypoints.

[0088] Based on a face landmark detection model jointly trained using a simulated synthetic dataset after S→R transformation and a small amount of real data, the evaluation process directly inputs real test faces in complex environments into the face landmark detection model for testing and evaluation. This method achieves higher detection accuracy than models trained directly with synthetic data or a small amount of real data. This approach reduces the model's dependence on real-world datasets, further exploring the complementarity and usability of synthetic datasets. Furthermore, it provides new research ideas for data protection and small-sample intelligent tasks in complex environments.

[0089] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for complex environment facial landmark detection based on generative network generated data, characterized in that, The method comprises the following steps: Step 1, using the FaceGen Modeller model to obtain the corresponding three-dimensional synthetic face texture structure information file by customizing different face attribute parameter settings, and then constructing a two-dimensional synthetic face image dataset under different angles and occlusion conditions through three-dimensional reconstruction, and uniformly cropping to a fixed size; Step 2, for the real face image dataset, the optimized DeepLab segmentation algorithm is used to remove complex background interference; Step 3, with the aid of the conversion model S→R based on the UGATIT generation network structure optimization, the two-dimensional synthetic face image dataset obtained in step 1 is converted into a simulated synthetic face image dataset with real face feature information; Step 4, using the real face image dataset obtained by removing complex background interference in step 2 and the simulated synthetic face image dataset obtained in step 3 as the model training dataset, the improved Hourglass face key point detection model is trained; In step 2, the background interference is removed based on the optimized DeepLab segmentation algorithm: the image performs feature extraction in the feature encoding stage to generate deep features and shallow features respectively, the deep features control the resolution of the output feature map through the empty convolution network, so that the network has different feature receptive fields, and then the channel number is adjusted through the convolution layer; In the feature decoding stage, the shallow features generated by the backbone feature extraction network and the output features after feature encoding are transformed into the same shape for feature fusion and then up-sampling to gradually restore the spatial information; In step 3, the conversion model S→R optimized in network structure based on UGATIT includes two generators ( , ) and two discriminators ( , ), and the real human face image data obtained by removing the complex background interference in step 2 and the synthetic human face image data obtained in step 1 are taken as inputs; Generator , ) model adopts two hourglass modules before its encoder and after its decoder, adds attention mechanism SE in the feature extraction ResNet module in its encoder, and introduces class activation map CAM between its encoder and decoder; Generator , ) models, the design of the decoder is symmetric to the encoder; The adaptive layer instance normalization Soft-AdaLIN is added to each layer of the generator and the discriminator to normalize and adjust the feature map; discriminator , ) is designed with a combination of global and local discriminators; In step 4, the improvement of the Hourglass face key point detection model is as follows: on the basis of the Hourglass model, the residual block optimized by the attention mechanism is added to reduce the dimension of the image data and store important information about the image; then multiple Hourglasses are directly connected in series to utilize the complementarity between features; the output of the residual block optimized by the attention mechanism is obtained by the following formula: wherein, represents the output of the residual block, is a particular input portion, is a weight, is a function that extracts and transforms features of the input; The loss function L of the Hourglass face key point detection model is: wherein, and respectively represent the predicted heat map and the ground truth heat map at the pixel position ; is the number of key points.

2. The method for complex environment facial landmark detection based on the data generated by the generative network according to claim 1, wherein, In step 1, the face attribute parameters include age, race, gender, hairstyle and expression style, and different face texture structure information files are randomly generated by adjusting the above attribute parameters.

3. The method for complex environment facial landmark detection based on the data generated by the generative network of claim 1, wherein, Change the cost function to obtain a better optimization goal to balance the loss between the generation model and the discrimination model in the conversion model S→R based on the UGATIT generation network structure optimization.

4. The method for complex environment facial landmark detection based on the data generated by the generative network of claim 3, wherein, The change of the cost function specifically includes: 6.1 Replace the cross-entropy loss function with an optimization based on least squares minimization: of the optimization: in, x This represents synthetic face image data. express x P from the distribution of synthetic face image dataset x probability, This indicates the process of discriminating between synthetic face image data. This indicates the use of generator G x The generated synthetic face image data, This indicates that the generated synthetic face image data is being analyzed. Represents synthetic face image data x From real face image dataset y distribution P y probability; 6.2 Cycle-consistency loss function to ensure that the image can be converted back to the original target domain: wherein, denotes the reconstructed synthetic face image data, y denotes the real face image data set, denotes that the real face image data is from the real face image data set y distributed P y probability, denotes the reconstructed real face image data set; 6.3 Using the body homographic function Ensuring that the color distribution of the input and output images is similar: wherein, representing reconstructing the synthetic face image data x input to the generator, representing reconstructing the real face image data y input to the generator; 6.4 CAM loss is utilizing information from the generator-aided classifier and the discriminator-aided classifier ​ wherein, represents inputting the generated human face image data x to the generator-assisted classifier for classification to obtain a predicted class of the generated image, is to convert the prediction of the classification model into a probability distribution form; represents using the discriminator-assisted classifier to discriminate the class of the input generated human face image data x , represents discriminating the class of the generated human face image data x generated by the generator G ; 6.5 Face recognition identity loss function is a pre-trained face recognition model MobileFaceNets, namely M model is used to extract the input and generate the output face key point features: wherein, denotes calculating the cosine, respectively denote inputting the generated data x,y to the M model to output the corresponding facial key point features, respectively denote inputting the generated data to the M model to output the corresponding facial key point features; 6.6 Pixel loss function are used to maintain low-level features after style transfer: 。

Images