Face generation method and device, electronic equipment and storage medium

Abstract

The present application provides a kind of face generation method, device, electronic equipment and storage medium, wherein the face generation method includes: the initial face image is preprocessed, generates first face semantic segmentation image;Determine the first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and the first generative adversarial network (GAN) model;The first face semantic segmentation image is edited based on the first latent code, and target face semantic segmentation image is generated;Initial face image and first face semantic segmentation image are input into second GAN model, and the second latent code corresponding to initial face image and first face semantic segmentation image is obtained;Second latent code and target face semantic segmentation image are input into second GAN model, and target face image is obtained.Through the above method, high-quality target face image can be generated, and the robustness of the face recognition model is improved by training the face recognition model based on the target face image.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for generating a human face. Background Technology

[0002] Facial recognition technology has a wide range of applications, including identity verification in public places such as airports, high-speed trains, and banks, and unlocking of smart devices such as mobile phones and tablets. In practical applications, facial recognition models are typically used to identify faces. However, facial recognition models are often affected by factors such as facial pose, lighting, age, expression, and accessories in the scene, resulting in low accuracy. Augmenting the training data of facial recognition models can effectively improve their robustness. For example, new facial images can be generated by editing existing facial images to augment the training data of the facial recognition model.

[0003] In related technologies, the quality of new face images generated by editing face images is generally low. Both the image resolution and the accuracy of image editing are significantly different from the real images. Face images edited by the above methods have very limited effect on improving the robustness of face recognition models.

[0004] Therefore, how to generate high-quality facial images to improve the robustness of facial recognition models is an urgent problem to be solved. Summary of the Invention

[0005] To address the problems existing in the prior art, embodiments of the present invention provide a face generation method, apparatus, electronic device, and storage medium.

[0006] This invention provides a face generation method, comprising:

[0007] The initial face image is preprocessed to generate a first face semantic segmentation image; the initial face image is used to represent face texture features, and the first face semantic segmentation image is used to represent face shape features.

[0008] Based on the first face semantic segmentation image and the first generative adversarial network (GAN) model, a first latent code corresponding to the first face semantic segmentation image is determined; the first GAN model is trained based on face semantic segmentation image samples.

[0009] Edit the first face semantic segmentation image based on the first latent code to generate the target face semantic segmentation image;

[0010] The initial face image and the first face semantic segmentation image are input into the second GAN model to obtain the second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image; the second GAN model is trained based on the initial face image samples and the face semantic segmentation image samples;

[0011] The second latent code and the target face semantic segmentation image are input into the second GAN model to obtain the target face image output by the second GAN model; the target face image is used to characterize the face texture features and the face shape features.

[0012] Optionally, the step of editing the first face semantic segmentation image based on the first latent code to generate the target face semantic segmentation image includes:

[0013] Obtain the attribute labels corresponding to the initial face image;

[0014] The first latent code and the attribute label are input into the Support Vector Machine (SVM) model to obtain the boundary vector output by the SVM model; the boundary vector is used to edit the first face semantic segmentation image; the SVM model is trained based on the latent code corresponding to the face semantic segmentation image sample and the attribute label corresponding to the initial face image sample;

[0015] The first face semantic segmentation image is edited based on the first latent code and the boundary vector to generate the target face semantic segmentation image.

[0016] Optionally, determining the first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and the first generative adversarial network (GAN) model includes:

[0017] Based on the first face semantic segmentation image, initialize the first vector corresponding to the first face semantic segmentation image;

[0018] The first vector is input into the first GAN model to obtain the second face semantic segmentation image output by the first GAN model.

[0019] The first vector is adjusted based on the second face semantic segmentation image until the first GAN model converges to obtain the target first vector; the target first vector is the first latent code.

[0020] Optionally, the step of preprocessing the initial face image to generate a first face semantic segmentation image includes:

[0021] Based on a face detection algorithm, the location of the face in the initial face image is determined;

[0022] Based on a key point detection algorithm, facial key points in the face location are determined; the facial key points are used to perform face alignment and cropping on the initial face image.

[0023] Based on the facial key points, the initial facial image is processed using a facial semantic segmentation algorithm to generate the first facial semantic segmentation image.

[0024] Optionally, after obtaining the target face image output by the second GAN model, the method further includes:

[0025] At least one of the target face images is input into the face recognition model for training until the face recognition model converges, thus obtaining the target face recognition model.

[0026] Optionally, the first GAN model and the second GAN model include multiple residual layers.

[0027] Optionally, the first GAN model and the second GAN model are trained using a triplet loss function;

[0028] The triplet loss function is expressed by the following formula (1):

[0029]

[0030] Among them, L m Represents the triplet loss function; l r The latent code representing any human face image; l p This represents the latent code of the face image that has the greatest similarity to any given face image during the Kth face image sampling process; This indicates that the face image sampled in the Kth time is different from l. p The latent code of the i-th face image corresponding to the face image.

[0031] The present invention also provides a face generation device, comprising:

[0032] The first generation module is used to preprocess the initial face image to generate a first face semantic segmentation image; the initial face image is used to represent face texture features, and the first face semantic segmentation image is used to represent face shape features.

[0033] The determination module is used to determine a first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and the first generative adversarial network (GAN) model; the first GAN model is trained based on face semantic segmentation image samples.

[0034] The second generation module is used to edit the first face semantic segmentation image based on the first latent code to generate a target face semantic segmentation image;

[0035] The first input module is used to input the initial face image and the first face semantic segmentation image into the second GAN model to obtain the second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image; the second GAN model is trained based on the initial face image samples and the face semantic segmentation image samples.

[0036] The second input module is used to input the second latent code and the target face semantic segmentation image into the second GAN model to obtain the target face image output by the second GAN model; the target face image is used to characterize the face texture features and the face shape features.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the face generation method as described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the face generation method as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the face generation method as described above.

[0040] The face generation method, apparatus, electronic device, and storage medium provided by this invention determine a first latent code corresponding to the first face semantic segmentation image based on a first face semantic segmentation image and a first generative adversarial network (GAN) model. Since the first face semantic segmentation image is used to represent the facial shape features of the original face image, the first face semantic segmentation image is edited based on the first latent code, and the generated target face semantic segmentation image can also represent the facial shape features of the original face image. The initial face image and the first face semantic segmentation image are input into a second GAN model to obtain the output of the second GAN model, which is consistent with the initial face image. The second latent code corresponding to the initial face image and the first face semantic segmentation image can transfer the face texture features and face shape features of the original face image to the second latent code, since the initial face image is used to represent the face texture features of the original face image. Inputting the second latent code and the target face semantic segmentation image into the second GAN model can generate a high-quality target face image while preserving the face texture features and face shape features of the original face image. Furthermore, using the high-quality target face image to recognize the face recognition model can effectively improve the robustness of the face recognition model. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is one of the flowcharts illustrating the face generation method provided by the present invention;

[0043] Figure 2 This is the second flowchart illustrating the face generation method provided by the present invention;

[0044] Figure 3 This is a schematic diagram of the face generation device provided by the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] To facilitate a clearer understanding of the various embodiments of this application, some relevant background knowledge will be introduced as follows.

[0048] Facial recognition technology has a wide range of applications, including identity verification in public places such as airports, high-speed trains, and banks, and unlocking of smart devices such as mobile phones and tablets. However, in practical applications, facial recognition models are often affected by factors such as facial pose, lighting, age, expression, and accessories in the scene, resulting in unstable facial recognition performance and posing certain risks to the application of this technology. Because augmenting training data is highly targeted and efficient, it has been widely used to improve the robustness of facial recognition models.

[0049] In related technologies, the following methods are commonly used to augment facial data, including:

[0050] Method 1: General editing method.

[0051] In Method 1, the face image is mapped to another space through affine transformation. This is mainly used as a general augmentation method in the training of deep learning networks, including cropping, blurring, rotation, and scaling.

[0052] Method 2: Component editing method.

[0053] In Method 2, a graph-to-graph mapping is established using Generative Adversarial Networks (GANs). The first part of the mapping maps the image into a latent space through an encoder, while the second part maps the image information in the latent space into the target space through a decoder that produces specific editing effects, thereby achieving the purpose of image editing.

[0054] Method 3: Attribute editing method.

[0055] In Method 3, the image is mapped to a latent code with certain semantics using a GAN model. The image editing purpose is achieved by changing the values ​​at the corresponding positions of the latent code and remapping the modified latent code back to the image.

[0056] However, in Method 1, the types of editing that can be performed by general editing methods are relatively simple. As a versatile editing method, it hardly changes the basic content of the image, so it has very limited improvement on the robustness of the face recognition model.

[0057] In Method 2, the component editing method can perform more complex image editing and meet the personalized needs in face image editing. However, the quality of the edited image is generally low. Both the image resolution and the accuracy of content editing are significantly different from the real image.

[0058] In Method 3, although the attribute editing method can achieve both generality and personalization, and the quality of the generated images can reach the level of real photos, it cannot guarantee that the identity information preservation and attribute decoupling effects are good at the same time during the inverse mapping and editing process. These two points are very important for the augmentation of face recognition and directly affect the face recognition effect.

[0059] In summary, in order to generate high-quality facial images and improve the robustness of facial recognition models, embodiments of the present invention provide a facial generation method, apparatus, electronic device, and storage medium.

[0060] The following is combined Figure 1 The face generation method provided by this invention will be described in detail. Figure 1 This is one of the flowcharts illustrating the face generation method provided by this invention. See [link / reference]. Figure 1 As shown, the method includes steps 101-105, wherein:

[0061] Step 101: Preprocess the initial face image to generate a first face semantic segmentation image; the initial face image is used to represent face texture features, and the first face semantic segmentation image is used to represent face shape features.

[0062] First, it should be noted that the subject of this invention can be any electronic device capable of editing facial images to generate new facial images, such as smartphones, smartwatches, desktop computers, laptops, etc.

[0063] In order to generate high-quality face images and improve the robustness of the face recognition model, in this embodiment, the initial face image needs to be preprocessed first to generate the first face semantic segmentation image.

[0064] The initial face image is an RGB image that includes a face portion. The purpose of this invention is to edit the initial face image while retaining its identity information, thereby generating a new target face image, which can augment the training data of the face recognition model.

[0065] In this embodiment, the first face semantic segmentation image is obtained by preprocessing the initial face image used to characterize face texture features, wherein face texture features include, for example, skin color and texture features of the face.

[0066] The first face semantic segmentation image includes at least one segmentation part, and each segmentation part has semantic information, which represents the shape features of the face; for example, the face image is segmented into 5 parts, and their semantic information is: glasses, nose, mouth, ears, and hair.

[0067] Step 102: Based on the first face semantic segmentation image and the first GAN model, determine the first latent code corresponding to the first face semantic segmentation image; the first GAN model is trained based on face semantic segmentation image samples.

[0068] In this embodiment, after generating the first face semantic segmentation image, it is necessary to transfer the face shape features represented by the first face semantic segmentation image to the first latent code corresponding to the first face semantic segmentation image.

[0069] The first latent code is a vector with length and direction, which is used to assist in editing the semantic segmentation image of the first face.

[0070] In practical applications, the first GAN model is trained based on face semantic segmentation image samples. The specific process of training the first GAN model is as follows:

[0071] Face semantic segmentation image samples are input into the initial face semantic segmentation generation model (initial GAN ​​model) for training. The generator of the initial face semantic segmentation generation model outputs fake face semantic segmentation images (e.g., 128x128 pixels in size), and the fake face semantic segmentation images are interpolated and enlarged to 512x512 pixels. The discriminator outputs the confidence score (numerical range 0-1) that the face semantic segmentation image sample is real. The generator and discriminator are optimized alternately.

[0072] When optimizing the generator, the parameters of the discriminator are fixed, and the loss function is calculated using the likelihood loss function and the triplet loss function. The optimal parameters of the generator are found using the gradient descent method (the optimizer is Adam).

[0073] When optimizing the discriminator, the generator parameters are fixed, the loss is calculated using a true / false judgment loss function, and the optimal parameters of the discriminator are found using gradient descent (with Adam as the optimizer). This process of alternating optimization continues until all parameters of the initial face semantic segmentation generation model converge, thus obtaining the first GAN model.

[0074] Step 103: Edit the first face semantic segmentation image based on the first latent code to generate the target face semantic segmentation image.

[0075] In this embodiment, after transferring the facial shape features represented by the first facial semantic segmentation image to the first latent code, the first facial semantic segmentation image can be edited based on the first latent code to generate the target facial semantic segmentation image. Since the first latent code can represent the facial shape features of the initial facial image, the target facial semantic segmentation image generated based on the first latent code can also represent the facial shape features of the initial facial image, thereby ensuring that the identity information of the initial facial image can be well preserved during the editing process of the first facial semantic segmentation image.

[0076] Step 104: Input the initial face image and the first face semantic segmentation image into the second GAN model to obtain the second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image; the second GAN model is trained based on the initial face image samples and the face semantic segmentation image samples.

[0077] In this embodiment, it is also necessary to transfer the facial texture features represented by the initial facial image to the second latent code corresponding to the initial facial image and the first facial semantic segmentation image; wherein, the second latent code is used to assist in generating the target facial image corresponding to the target facial semantic segmentation image.

[0078] In practical applications, the second GAN model is trained based on initial face image samples and face semantic segmentation image samples. The specific process of training the second GAN model is as follows:

[0079] The initial face image sample and the corresponding face semantic segmentation image sample are input in pairs into the initial face semantic segmentation generation model (initial GAN ​​model) for training. The initial GAN ​​model is a conditional generation model that can generate face images (e.g., 512x512 pixels in size) that simultaneously meet the texture features in the face image and the shape features of the semantic segmentation image. At the same time, there is a discriminator that outputs the confidence score (numerical range 0-1) that the semantic segmentation image sample is true.

[0080] The generator and discriminator are optimized alternately. When optimizing the generator, the discriminator's parameters are fixed, and the loss function is calculated using the likelihood loss function and the triplet loss function. Gradient descent (using Adam as the optimizer) is then used to find the optimal parameters for the generator. When optimizing the discriminator, the generator's parameters are fixed, and the loss function is calculated using the true / false judgment loss function. Gradient descent (using Adam as the optimizer) is then used to find the optimal parameters for the discriminator. This alternating optimization process continues until all parameters of the model converge.

[0081] In the above implementation, the GAN model is used to link the first latent code, the second latent code, and the first face semantic segmentation image, which further alleviates the problems of identity loss and attribute entanglement that occur during the inverse mapping and editing process.

[0082] Optionally, in one possible implementation of the present invention, the first GAN model and the second GAN model are trained using a triplet loss function;

[0083] The triplet loss function is expressed by the following formula (1):

[0084]

[0085] Among them, L m Represents the triplet loss function; l r The latent code representing any human face image; l p This represents the latent code of the face image that has the greatest similarity to any given face image during the Kth face image sampling process; This indicates that the face image sampled in the Kth time is different from l. p The latent code of the i-th face image corresponding to the face image.

[0086] During the training of the generators for the first and second GAN models, a triplet loss function was added. The triplet loss function can transfer the semantic information of facial features into the latent code, which helps to preserve the identity information of the facial image and decouple the attributes, thereby improving the quality of the subsequently generated target facial images.

[0087] Optionally, in one possible implementation of the present invention, the first GAN model and the second GAN model include multiple residual layers.

[0088] It should be noted that the first and second GAN models use a mapping network composed of multiple residual layers, which can map random noise into the latent space. This latent space is a unified latent space for face image generation, inverse mapping, and editing, which alleviates the identity loss problem that occurs in the initial face image during inverse mapping and editing. Furthermore, in order to improve the speed of semantic segmentation image editing, the first GAN model uses only 4 residual layers to form the mapping network and 12 style modules to form the synthesis network.

[0089] Correspondingly, unlike the input of the first GAN model, since the input of the second GAN model is the initial face image and the first face semantic segmentation image, in order to improve the image editing capability of the second GAN model, the second GAN model needs to use 8 residual layers to form a mapping network and 16 style modules to form a synthesis network; and the second latent code output by the second GAN model is a vector of size 8×512.

[0090] Step 105: Input the second latent code and the target face semantic segmentation image into the second GAN model to obtain the target face image output by the second GAN model; the target face image is used to characterize the face texture features and the face shape features.

[0091] In this embodiment, after obtaining the second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image, since the second latent code can characterize the face texture features and face shape features of the original face image, inputting the second latent code and the target face semantic segmentation image into the second GAN model can generate a high-quality target face image while preserving the face texture features and face shape features of the original face image.

[0092] The face generation method provided by this invention, based on a first face semantic segmentation image and a first generative adversarial network (GAN) model, determines a first latent code corresponding to the first face semantic segmentation image. Since the first face semantic segmentation image is used to represent the facial shape features of the original face image, editing the first face semantic segmentation image based on the first latent code generates a target face semantic segmentation image that can also represent the facial shape features of the original face image. The initial face image and the first face semantic segmentation image are input into a second GAN model to obtain a second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image. Since the initial face image is used to represent the facial texture features of the original face image, the facial texture features and facial shape features of the original face image can be transferred to the second latent code. Inputting the second latent code and the target face semantic segmentation image into the second GAN model can generate a high-quality target face image while preserving the facial texture features and facial shape features of the original face image. Furthermore, using the high-quality target face image to recognize a face recognition model can effectively improve the robustness of the face recognition model.

[0093] Optionally, the step of preprocessing the initial face image to generate a first face semantic segmentation image can be implemented in the following ways, specifically including steps 1) to 3):

[0094] Step 1) Determine the face location in the initial face image based on the face detection algorithm;

[0095] Step 2) Based on the key point detection algorithm, determine the facial key points in the face location; the facial key points are used to perform face alignment and cropping on the initial face image;

[0096] Step 3) Based on the facial key points, process the initial facial image using a facial semantic segmentation algorithm to generate the first facial semantic segmentation image.

[0097] First, it should be noted that after acquiring the initial face image in RGB format, it needs to be manually labeled according to the type of attribute editing of the initial face image.

[0098] For example, if age is to be labeled, the youngest 30% of data can be labeled as 0, the oldest 30% as 1, and the rest of the data can be left unlabeled, based on the age range of the initial face images.

[0099] If you want to indicate whether someone is wearing glasses or not, you can use 0 (no glasses) and 1 (wearing glasses) to mark it.

[0100] After labeling the initial face image to carry attribute tags, a face detection algorithm is needed to locate the face in the initial face image.

[0101] In practical applications, there are various face detection algorithms, such as Haar, Hog, CNN, SSD, MTCNN, etc. This invention does not limit the face detection algorithm.

[0102] After detecting the position of the face in the initial face image, a key point detection algorithm is needed to obtain the positions of 68 key points in each initial face image based on the position of the face in the initial face image; the key point detection algorithm can be, for example, dlib.

[0103] It should be noted that the purpose of extracting the location of key points in the initial face image is to perform face alignment and cropping, thereby filtering out the influence of key point location for subsequent facial feature extraction in the model.

[0104] After detecting facial key points in the face location, the initial face image is cropped and scaled to 512x512 pixels based on the location of the key points.

[0105] Finally, based on facial landmarks, the initial face image is processed using a face semantic segmentation algorithm to generate a first face semantic segmentation image of the same size (512x512 pixels). The face semantic segmentation algorithm can, for example, utilize a fully convolutional neural network (FCN).

[0106] In the above implementation, the face detection algorithm can accurately determine the position of the face in the initial face image, the key point detection algorithm can obtain the key points of the face in the face image, and the face semantic segmentation algorithm can be used to perform semantic segmentation on the initial face image, so as to accurately obtain the first face semantic segmentation image used to characterize the face shape features of the initial face image, which provides data support for the subsequent generation of the target face image.

[0107] Optionally, the step of determining the first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and the first GAN model can be implemented in the following way, specifically including steps [1]-[3]:

[0108] Step [1]: Based on the first face semantic segmentation image, initialize the first vector corresponding to the first face semantic segmentation image;

[0109] Step [2]: Input the first vector into the first GAN model to obtain the second face semantic segmentation image output by the first GAN model;

[0110] Step [3]: Adjust the first vector based on the second face semantic segmentation image until the first GAN model converges to obtain the target first vector; the target first vector is the first latent code.

[0111] In this embodiment, after preprocessing the initial face image to generate the first face semantic segmentation image, it is necessary to randomly generate (i.e. initialize) a first vector of size 4x512 based on the first face semantic segmentation image.

[0112] Then the first vector is input into the trained first GAN model to obtain a second face semantic segmentation image with a size of 512×512 pixels output by the first GAN model.

[0113] The difference between the second face semantic segmentation image output by the first GAN model and the first face semantic segmentation image is compared; the mean squared error function and the perceptual loss function are used to calculate the loss, and the first vector is adjusted using the gradient descent method (the optimizer is Adam) until the first GAN model converges, that is, the optimal input vector (i.e. the target first vector) is found; the target first vector is the first latent code corresponding to the first face semantic segmentation image.

[0114] In the above implementation, a first vector corresponding to the first face semantic segmentation image is generated based on the first face semantic segmentation image. Then, the first vector is input into the first GAN model to obtain the second face semantic segmentation image. The first vector is adjusted based on the second face semantic segmentation image until the first GAN model converges. This transfers the face shape features represented by the first face semantic segmentation image to the first latent code corresponding to the first face semantic segmentation image, thus mitigating the identity loss and attribute entanglement problems that occur during the inverse mapping and editing of the original face image.

[0115] Optionally, the step of editing the first face semantic segmentation image based on the first latent code to generate the target face semantic segmentation image is specifically achieved through the following steps a)-c):

[0116] Step a) Obtain the attribute labels corresponding to the initial face image;

[0117] Step b) Input the first latent code and the attribute label into the Support Vector Machine (SVM) model to obtain the boundary vector output by the SVM model; the boundary vector is used to edit the first face semantic segmentation image; the SVM model is trained based on the latent code corresponding to the face semantic segmentation image sample and the attribute label corresponding to the initial face image sample.

[0118] Step c) Edit the first face semantic segmentation image based on the first latent code and the boundary vector to generate the target face semantic segmentation image.

[0119] In this embodiment, after determining the first latent code based on the first face semantic segmentation image and the first GAN model, the first face semantic segmentation image needs to be edited based on the first latent code to generate the target face semantic segmentation image. The specific implementation process is as follows:

[0120] First, we need to obtain the attribute labels corresponding to the initial face image. Attribute labels can be, for example, age, whether or not the person is wearing glasses, gender, etc.

[0121] Then, the first latent code and attribute labels are input into the support vector machine (SVM) model, and the normal vector of the SVM model is the boundary vector.

[0122] After obtaining the boundary vector, the first face semantic segmentation image is edited based on the first latent code and the boundary vector using formula (2) to generate the target face semantic segmentation image.

[0123] Specifically, formula (2) can be expressed as:

[0124] L edit =l+α·n

[0125] Among them, L edit Let l be the semantic segmentation image of the target face, l be the first latent code, α be the edit coefficient, and n be the boundary vector of the attribute.

[0126] In the above implementation, based on the boundary vector, the first face semantic segmentation image can be edited to generate a target face semantic segmentation image that can characterize the shape features of the original face image, thus mitigating the identity loss problem that occurs during the inverse mapping and editing process of face images.

[0127] Optionally, in one possible implementation of this invention, after obtaining the target face image output by the second GAN model, the following steps are further performed:

[0128] At least one of the target face images is input into the face recognition model for training until the face recognition model converges, thus obtaining the target face recognition model.

[0129] In practical applications, training a face recognition model using a target face image can be achieved in the following ways:

[0130] Step 1: Test the face data test set using the initial face recognition model.

[0131] Step 2: Analyze the test results.

[0132] Specifically, for samples with poor recognition performance, their identification numbers and included attributes are recorded and statistically analyzed to identify several target attributes that appear frequently as the basis for data augmentation.

[0133] Step 3: Using the face generation method provided by this invention, augment the target attributes one by one on the training set to generate target face images. After each augmentation is completed, use the augmented data (i.e., the generated target face images) to train the initial face recognition model. Moreover, during each augmentation of a target attribute, the number of augmented samples needs to be gradually increased until the training results no longer change significantly, and then augmentation of other attributes is performed.

[0134] Step 4: After all the test data in the test set has been augmented, the initial face recognition model is trained using the augmented test set until the face recognition model converges, thus obtaining the target face recognition model.

[0135] In the above embodiments, for images with poor face recognition performance, analysis is performed, and the distribution of training data is considered to identify attributes that require augmentation. Then, according to the face generation method provided by this invention, multiple face data of this attribute type are generated for the training set and used in the training of the face recognition model. The face generation method provided by this invention can augment the training dataset of the face recognition model, thereby improving the robustness of the face recognition model and enhancing face recognition performance.

[0136] Figure 2 This is the second flowchart illustrating the face generation method provided by this invention. See also... Figure 2 As shown, the method includes steps 201-212, wherein:

[0137] Step 201: Obtain the attribute labels corresponding to the initial face image.

[0138] Step 202: Based on the face detection algorithm, determine the face location in the initial face image.

[0139] Step 203: Based on the key point detection algorithm, determine the facial key points in the face location.

[0140] Step 204: Based on facial key points, process the initial facial image using a facial semantic segmentation algorithm to generate the first facial semantic segmentation image.

[0141] Step 205: Based on the first face semantic segmentation image, initialize the first vector corresponding to the first face semantic segmentation image.

[0142] Step 206: Input the first vector into the first GAN model to obtain the second face semantic segmentation image output by the first GAN model.

[0143] It should be noted that the first GAN model was trained based on facial semantic segmentation image samples.

[0144] Step 207: Adjust the first vector based on the second face semantic segmentation image until the first GAN model converges to obtain the target first vector; wherein, the target first vector is the first latent code.

[0145] Step 208: Input the first latent code and attribute label into the Support Vector Machine (SVM) model to obtain the boundary vector output by the SVM model.

[0146] Specifically, the boundary vector is used to edit the first face semantic segmentation image; and the SVM model is trained based on the latent code corresponding to the face semantic segmentation image sample and the attribute label corresponding to the initial face image sample.

[0147] Step 209: Edit the first face semantic segmentation image based on the first latent code and boundary vector to generate the target face semantic segmentation image.

[0148] Step 210: Input the initial face image and the first face semantic segmentation image into the second GAN model to obtain the second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image.

[0149] It should be noted that the second GAN model was trained based on the initial face image samples and face semantic segmentation image samples.

[0150] Step 211: Input the second latent code and the target face semantic segmentation image into the second GAN model to obtain the target face image output by the second GAN model; wherein, the target face image is used to represent the face texture features and face shape features.

[0151] Step 212: Input at least one target face image into the face recognition model for training until the face recognition model converges, thus obtaining the target face recognition model.

[0152] The face generation apparatus provided by the present invention is described below. The face generation apparatus described below and the face generation method described above can be referred to in correspondence. Figure 3 This is a structural schematic diagram of the face generation device provided by the present invention, as shown below. Figure 3 As shown, the face generation device 300 includes: a first generation module 301, a determining module 302, a second generation module 303, a first input module 304, and a second input module 305, wherein:

[0153] The first generation module 301 is used to preprocess the initial face image to generate a first face semantic segmentation image; the initial face image is used to represent face texture features, and the first face semantic segmentation image is used to represent face shape features.

[0154] The determining module 302 is used to determine a first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and the first generative adversarial network (GAN) model; the first GAN model is trained based on face semantic segmentation image samples.

[0155] The second generation module 303 is used to edit the first face semantic segmentation image based on the first latent code to generate a target face semantic segmentation image;

[0156] The first input module 304 is used to input the initial face image and the first face semantic segmentation image into the second GAN model to obtain the second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image; the second GAN model is trained based on the initial face image samples and the face semantic segmentation image samples.

[0157] The second input module 305 is used to input the second latent code and the target face semantic segmentation image into the second GAN model to obtain the target face image output by the second GAN model; the target face image is used to characterize the face texture features and the face shape features.

[0158] The face generation device provided by this invention determines a first latent code corresponding to the first face semantic segmentation image based on a first face semantic segmentation image and a first generative adversarial network (GAN) model. Since the first face semantic segmentation image is used to represent the facial shape features of the original face image, editing the first face semantic segmentation image based on the first latent code generates a target face semantic segmentation image that can also represent the facial shape features of the original face image. The initial face image and the first face semantic segmentation image are input into a second GAN model to obtain a second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image. Since the initial face image is used to represent the facial texture features of the original face image, the facial texture features and facial shape features of the original face image can be transferred to the second latent code. Inputting the second latent code and the target face semantic segmentation image into the second GAN model can generate a high-quality target face image while preserving the facial texture features and facial shape features of the original face image. Furthermore, using the high-quality target face image to recognize a face recognition model can effectively improve the robustness of the face recognition model.

[0159] Optionally, the second generation module 303 is further configured to:

[0160] Obtain the attribute labels corresponding to the initial face image;

[0161] The first latent code and the attribute label are input into the Support Vector Machine (SVM) model to obtain the boundary vector output by the SVM model; the boundary vector is used to edit the first face semantic segmentation image; the SVM model is trained based on the latent code corresponding to the face semantic segmentation image sample and the attribute label corresponding to the initial face image sample;

[0162] The first face semantic segmentation image is edited based on the first latent code and the boundary vector to generate the target face semantic segmentation image.

[0163] Optionally, the determining module 302 is further configured to:

[0164] Based on the first face semantic segmentation image, initialize the first vector corresponding to the first face semantic segmentation image;

[0165] The first vector is input into the first GAN model to obtain the second face semantic segmentation image output by the first GAN model.

[0166] The first vector is adjusted based on the second face semantic segmentation image until the first GAN model converges to obtain the target first vector; the target first vector is the first latent code.

[0167] Optionally, the first generation module 301 is further configured to:

[0168] Based on a face detection algorithm, the location of the face in the initial face image is determined;

[0169] Based on a key point detection algorithm, facial key points in the face location are determined; the facial key points are used to perform face alignment and cropping on the initial face image.

[0170] Based on the facial key points, the initial facial image is processed using a facial semantic segmentation algorithm to generate the first facial semantic segmentation image.

[0171] Optionally, the device further includes:

[0172] The training module is used to input at least one of the target face images into the face recognition model for training until the face recognition model converges, thereby obtaining the target face recognition model.

[0173] Optionally, the first GAN model and the second GAN model include multiple residual layers.

[0174] Optionally, the first GAN model and the second GAN model are trained using a triplet loss function;

[0175] The triplet loss function is expressed by the following formula (1):

[0176]

[0177] Among them, L m Represents the triplet loss function; l r The latent code representing any human face image; l p This represents the latent code of the face image that has the greatest similarity to any given face image during the Kth face image sampling process; This indicates that the face image sampled in the Kth time is different from l. p The latent code of the i-th face image corresponding to the face image.

[0178] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a face generation method, which includes: preprocessing an initial face image to generate a first face semantic segmentation image; the initial face image is used to represent face texture features, and the first face semantic segmentation image is used to represent face shape features; determining a first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and a first generative adversarial network (GAN) model; the first GAN model is trained based on face semantic segmentation image samples; and encoding the first face semantic segmentation image based on the first latent code. The process involves: generating a target face semantic segmentation image; inputting the initial face image and the first face semantic segmentation image into a second GAN model to obtain a second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image; training the second GAN model based on the initial face image samples and the face semantic segmentation image samples; inputting the second latent code and the target face semantic segmentation image into the second GAN model to obtain the target face image output by the second GAN model; and using the target face image to characterize the face texture features and the face shape features.

[0179] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0180] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the face generation method provided by the above methods. The method includes: preprocessing an initial face image to generate a first face semantic segmentation image; the initial face image is used to characterize face texture features, and the first face semantic segmentation image is used to characterize face shape features; determining a first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and a first generative adversarial network (GAN) model; the first GAN model is based on the face semantic segmentation image sample. The initial face image and the first face semantic segmentation image are trained together; the initial face image and the first face semantic segmentation image are edited based on the first latent code to generate a target face semantic segmentation image; the initial face image and the first face semantic segmentation image are input into a second GAN model to obtain a second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image; the second GAN model is trained based on the initial face image samples and the face semantic segmentation image samples; the second latent code and the target face semantic segmentation image are input into the second GAN model to obtain the target face image output by the second GAN model; the target face image is used to represent the face texture features and the face shape features.

[0181] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the face generation method provided by the above methods. The method includes: preprocessing an initial face image to generate a first face semantic segmentation image; the initial face image is used to characterize face texture features, and the first face semantic segmentation image is used to characterize face shape features; determining a first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and a first generative adversarial network (GAN) model; the first GAN model is trained based on face semantic segmentation image samples; based on the first... The latent code edits the first face semantic segmentation image to generate a target face semantic segmentation image; the initial face image and the first face semantic segmentation image are input into a second GAN model to obtain a second latent code output by the second GAN model corresponding to the initial face image and the first face semantic segmentation image; the second GAN model is trained based on the initial face image samples and the face semantic segmentation image samples; the second latent code and the target face semantic segmentation image are input into the second GAN model to obtain the target face image output by the second GAN model; the target face image is used to characterize the face texture features and the face shape features.

[0182] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0183] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A face generation method, characterized in that, The method comprises the following steps: preprocessing an initial face image to generate a first face semantic segmentation image; the initial face image is used to represent face texture features, and the first face semantic segmentation image is used to represent face shape features; determining a first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and a first generative adversarial network (GAN) model; the first GAN model is obtained by training based on face semantic segmentation image samples; editing the first face semantic segmentation image based on the first latent code to generate a target face semantic segmentation image; the editing of the first face semantic segmentation image based on the first latent code to generate a target face semantic segmentation image comprises the following steps: obtaining an attribute label corresponding to the initial face image; inputting the first latent code and the attribute label into a support vector machine (SVM) model to obtain a boundary vector output by the SVM model; the boundary vector is used to edit the first face semantic segmentation image; the SVM model is obtained by training based on latent codes corresponding to the face semantic segmentation image samples and attribute labels corresponding to the initial face image samples; editing the first face semantic segmentation image based on the first latent code and the boundary vector to generate the target face semantic segmentation image; inputting the initial face image and the first face semantic segmentation image into a second GAN model to obtain a second latent code output by the second GAN model and corresponding to the initial face image and the first face semantic segmentation image; the second GAN model is obtained by training based on initial face image samples and the face semantic segmentation image samples; inputting the second latent code and the target face semantic segmentation image into the second GAN model to obtain a target face image output by the second GAN model; the target face image is used to represent the face texture features and the face shape features.

2. The face generation method of claim 1, wherein, The determination of the first latent code corresponding to the first face semantic segmentation image based on the first face semantic segmentation image and the first generative adversarial network (GAN) model comprises the following steps: initializing a first vector corresponding to the first face semantic segmentation image based on the first face semantic segmentation image; inputting the first vector into the first GAN model to obtain a second face semantic segmentation image output by the first GAN model; adjusting the first vector based on the second face semantic segmentation image until the first GAN model converges to obtain a target first vector; the target first vector is the first latent code.

3. The face generation method of claim 1, wherein, The preprocessing of the initial face image to generate the first face semantic segmentation image comprises the following steps: determining a face position in the initial face image based on a face detection algorithm; determining face key points in the face position based on a key point detection algorithm; the face key points are used for face alignment and cropping of the initial face image; processing the initial face image based on the face key points by using a face semantic segmentation algorithm to generate the first face semantic segmentation image.

4. The face generation method of claim 1, wherein, After the target face image output by the second GAN model is obtained, the method further comprises: inputting at least one target face image into a face recognition model for training until the face recognition model converges, to obtain a target face recognition model.

5. The face generation method of claim 1, wherein, The first GAN model and the second GAN model comprise a plurality of residual layers.

6. The face generation method of any one of claims 1-5, wherein, The first GAN model and the second GAN model are trained by using a triplet loss function; The triplet loss function is represented by the following formula (1): wherein L m represents the triplet loss function; l r represents the latent code of an arbitrary face image; l p represents the latent code of the face image with the maximum similarity to the arbitrary face image in the Kth face image sampling process; represents the latent code of the face image different from l p the latent code of the i-th face image corresponding to the face image.

7. A face generation apparatus, comprising: comprise: A first generation module is configured to preprocess an initial face image to generate a first face semantic segmentation image; the initial face image is used to represent a face texture feature, and the first face semantic segmentation image is used to represent a face shape feature; A determination module is configured to determine, based on the first face semantic segmentation image and a first generative adversarial network (GAN) model, a first latent code corresponding to the first face semantic segmentation image; The first GAN model is trained based on face semantic segmentation image samples; A second generation module is configured to edit the first face semantic segmentation image based on the first latent code to generate a target face semantic segmentation image; The editing of the first face semantic segmentation image based on the first latent code to generate a target face semantic segmentation image comprises: obtaining an attribute label corresponding to the initial face image; inputting the first latent code and the attribute label into a support vector machine (SVM) model to obtain a boundary vector output by the SVM model; the boundary vector is used to edit the first face semantic segmentation image; the SVM model is trained based on a latent code corresponding to the face semantic segmentation image sample and an attribute label corresponding to the initial face image sample; and the first face semantic segmentation image is edited based on the first latent code and the boundary vector to generate the target face semantic segmentation image; A first input module is configured to input the initial face image and the first face semantic segmentation image into a second GAN model to obtain a second latent code output by the second GAN model and corresponding to the initial face image and the first face semantic segmentation image; the second GAN model is trained based on initial face image samples and the face semantic segmentation image samples; A second input module is configured to input the second latent code and the target face semantic segmentation image into the second GAN model to obtain a target face image output by the second GAN model; the target face image is used to represent the face texture feature and the face shape feature.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the face generation method of any one of claims 1 to 6 when executing the program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the face generation method of any one of claims 1 to 6 when executed by the processor.

Images