Generator model training and face reconstruction generation method and device, and storage medium
By training a generator model and utilizing local feature consistency loss and teacher network feature extraction, the generator model reconstructs and generates face images, thus solving the impact of adversarial attacks on the accuracy and robustness of face recognition technology and achieving a highly efficient defense effect in adversarial attack environments.
Patent Information
- Application Number
- CN202211225684.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-09
- Publication Date
- 2026-01-13
- Estimated Expiration
- 2042-10-09
AI Technical Summary
Existing adversarial defense techniques are insufficient to effectively eliminate the impact of adversarial attacks on the accuracy and robustness of facial recognition technology, especially since traditional image preprocessing techniques are unable to completely eliminate the interference features of adversarial attack samples.
By training a generator model and utilizing loss functions such as local feature consistency loss, pixel difference loss, and identity preservation loss, combined with the feature extraction capabilities of the teacher network, the generator model reconstructs and generates face images, eliminates adversarial attack elements, and improves the robustness of face recognition.
It achieves accuracy and stability of face recognition technology under adversarial attack environments. The generator model can effectively filter adversarial attacks and improve the robustness and defense capabilities of face recognition.
Smart Images
Figure CN115880746B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of face recognition technology and deep learning technology, and in particular to generator model training and face reconstruction generation methods, devices and storage media. Background Technology
[0002] Facial recognition technology plays a crucial role in applications such as turnstile access control and quick payment. Due to the rise of deep learning technology, deep neural network-based facial recognition technology boasts advantages such as high accuracy, fast speed, and the ability to recognize a large number of people, gradually gaining dominance in these applications. The robustness of deep learning-based facial recognition technology has always been one of the most pressing concerns in the field of artificial intelligence security. However, the emergence of adversarial attack techniques has further impacted the robustness of facial recognition technology. Adversarial attack techniques can add imperceptible interference pixels to the original image, leading to facial recognition errors. In recent years, the advent of adversarial attack techniques has severely hampered the improvement of facial recognition technology's robustness, significantly hindering its deployment in applications such as turnstile access control and quick payment. Researchers have begun exploring proactive defenses against adversarial attacks—that is, adversarial defense techniques—to improve the recognition capabilities of facial recognition applications under the influence of adversarial attacks.
[0003] Existing adversarial defense techniques mainly fall into two categories. The first type requires retraining and re-deploying the model, necessitating changes to the parameters and structure of the currently deployed model, resulting in significant training overhead. The second type focuses on the data preprocessing stage of face recognition, where attack samples are reconstructed to filter out their interfering features, ensuring the high accuracy of the deployed face recognition model. Regarding the second type, existing traditional image preprocessing techniques, such as image compression, image resizing, image binarization, and random image cropping, can eliminate the influence of interfering pixels to some extent. However, due to the high semantic transferability of the interfering features in adversarial attack samples, these traditional image preprocessing techniques are insufficient to completely eliminate the influence of interfering pixels.
[0004] Therefore, how to effectively overcome the impact of adversarial attack techniques on the accuracy and robustness of facial recognition technology is a technical problem that urgently needs to be solved. Summary of the Invention
[0005] In view of this, embodiments of the present invention provide a model training and face reconstruction generation method, apparatus and storage medium, which trains a generator model and reconstructs face images based on this generator model to effectively overcome the impact of adversarial attack techniques on the accuracy and robustness of face recognition technology.
[0006] One aspect of the present invention provides a method for training an adversarial generative network generator model for face recognition, the method comprising the following steps:
[0007] Image reconstruction generation steps: Obtain real face images from the training set and input them into a pre-established generator model. The generator model reconstructs the input real face images to generate face reconstruction images.
[0008] The image local consistency enhancement step includes:
[0009] The reconstructed face image is randomly cropped into multiple random image blocks and paired to obtain multiple image block pairs. The teacher model is used to extract features from each random image block. Based on the extracted features, the feature distance within each image block pair is calculated. The constructed local feature consistency loss function is used to calculate the local feature consistency loss based on the feature distance within each image block pair.
[0010] The parameters of the generator model are updated at least based on the local feature consistency loss;
[0011] Image discrimination steps: Input the real face image and the face reconstruction image output by the current generator into the pre-established discriminator model, so that the discriminator model outputs a binary classification result indicating whether the input face reconstruction image is a real face image, calculate the true / false discrimination loss based on the binary classification result using the constructed discriminator loss function, and update the parameters of the discriminator model based on the true / false discrimination loss;
[0012] The image reconstruction generation step, the image local consistency enhancement step, and the image discrimination step are repeated to alternately update the parameters of the generator model and the discriminator model using stochastic gradient descent. The training process ends when the fluctuation of the true / false discrimination loss is lower than a preset threshold, thereby obtaining the trained generator model.
[0013] In some embodiments of the present invention, the image reconstruction generation step further includes:
[0014] The constructed pixel difference loss function is used to calculate the pixel difference loss between the input real face image and the output face reconstruction image based on the pixel differences between each pixel of the input real face image and the output face reconstruction image of the generator model; and / or
[0015] The identity preservation loss function is used to calculate the identity preservation loss between the input real face image and the output face reconstruction image based on the distance between the identity feature vectors of the input real face image and the output face reconstruction image of the generator model.
[0016] The step of updating the parameters of the generator model based at least on the local feature consistency loss includes updating the parameters of the generator model based on the local feature consistency loss, pixel difference loss, and / or identity preservation loss.
[0017] In some embodiments of the present invention, the pixel difference loss function is expressed as:
[0018]
[0019] Where W and H are the width and height of the image, respectively. and The pixel values at each pixel location for the corresponding real face image and the reconstructed face image;
[0020] The identity preservation loss function is expressed as:
[0021]
[0022] in, and The identity feature vectors are the real face image and the reconstructed face image. and Feature maps extracted from the input real face image and the face reconstruction image.
[0023] In some embodiments of the present invention, the local feature consistency loss function is expressed as:
[0024]
[0025] in, and , respectively, are the feature maps of two randomly paired image patches extracted by the teacher model, N is the total number of pairs of image patches randomly cropped from a face reconstruction image, and KL is the information divergence calculation formula used to calculate feature distance.
[0026] In some embodiments of the present invention, both the real face image and the face reconstruction image can be divided into key regions and non-key regions. The key regions include facial feature regions, and the non-key regions are non-facial feature regions. The step of randomly cropping the face reconstruction image into multiple random image blocks and pairing them to obtain multiple image block pairs includes: randomly cropping the face reconstruction image into multiple random image blocks and pairing image blocks belonging to the key regions of the face reconstruction image with image blocks belonging to the secondary key regions of the face reconstruction image to obtain multiple image block pairs.
[0027] In some embodiments of the present invention, in the step of using the teacher model to extract features from each random image block, the extracted features are feature maps containing two-dimensional spatial features of the face.
[0028] In some embodiments of the present invention, the teacher network is a face recognition neural network with prior knowledge that has been trained on a face dataset.
[0029] Another aspect of the present invention provides a face reconstruction generation method, the method comprising:
[0030] The generator model acquires face images;
[0031] The encoder of the generator model extracts feature vectors from real face images, and the decoder of the generator model reconstructs and generates faces based on the feature vectors.
[0032] The generator model outputs a reconstructed face image;
[0033] The generator model is the generator model obtained by any of the generator model training methods described in the above embodiments of the adversarial generative network generator model training method.
[0034] Another aspect of the present invention provides a face reconstruction generation apparatus, including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the apparatus implements the steps of the generator model training method or the steps of the face reconstruction generation method as described in any of the above embodiments.
[0035] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the generator model training method or the steps of the face reconstruction generation method as described in any of the above embodiments.
[0036] The generator model training and face reconstruction generation method, apparatus and storage medium of the present invention can analyze the feature information of local image blocks by leveraging the feature extraction capability of the teacher network. Through the designed random pairing distance measurement method, it can analyze the significant differences between adversarial attack samples and original sensory images at the feature level. The adversarial generative network generator model for face recognition obtained by training can reconstruct the input face image to eliminate possible adversarial attacks and achieve a good adversarial defense effect.
[0037] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.
[0038] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description
[0039] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:
[0040] Figure 1 This is a flowchart of a generator model training method in one embodiment of the present invention.
[0041] Figure 2 This is a schematic diagram illustrating the feature differences between a normal image and an attacked image in one embodiment of the present invention. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.
[0043] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.
[0044] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.
[0045] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.
[0046] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.
[0047] To effectively overcome the impact of adversarial attack techniques on the accuracy and robustness of face recognition technology, this invention provides a training method for an adversarial generative network (GNN) generator model for face recognition. The generator model, trained based on the feature consistency principle, reconstructs and generates face images, thereby filtering out face images containing adversarial attack elements. Furthermore, it analyzes and extracts undisturbed high-dimensional semantic features based on a powerful teacher network, and reconstructs the image based on these high-dimensional semantic features. This invention comprehensively considers both key and non-key regions of the face to ensure the ability to defend against adversarial attacks, thereby ensuring the accuracy and robustness of face recognition technology.
[0048] The core idea of this invention is to utilize the difference in feature distance between ordinary images and attacked images. Experiments have shown that there are differences in feature distance between different regions of the attacked image, especially between critical and non-critical regions.
[0049] Figure 2 This is a schematic diagram illustrating the feature differences between a normal image and an attacked image in one embodiment of the present invention. Research shows that there are subtle, visually imperceptible differences in interfering pixels between the original image and the attacked image after an adversarial attack. However, by randomly sampling local portions of both the original and attacked images, and then using a deep neural network (i.e., a teacher network) to extract the semantic features of the high-dimensional semantic space of the sampled blocks in both images, the semantic feature distance between sampled blocks within the same image can be calculated. The semantic distance of an image block is calculated using the semantic features of the high-dimensional semantic space of each sampled block within the same image. The semantic features of an image block are in the form of a feature map (two-dimensional), and the semantic distance is measured using Euclidean distance or cosine distance. Experiments show that the attacked image has a larger semantic feature distance between its sampled blocks, which is used to determine the difference between the original and attacked images.
[0050] In another embodiment of the present invention, the sampling block can be replaced by an image block obtained by randomly cropping the original image and the attacked image, and the image blocks are randomly paired.
[0051] Figure 1 This is a flowchart of a generator model training method in one embodiment of the present invention. The method includes the following steps:
[0052] Step S110, the image reconstruction generation step, includes: obtaining real face images from the training set and inputting them into a pre-established generator model; reconstructing the input real face images using the generator model to generate a reconstructed face image. The training set contains a large number of real face images, and the generator model is a generative adversarial network with an auto-encoder-decoder structure. The purpose of this invention is to train the generator model to reconstruct the input image to eliminate adversarial attacks.
[0053] Step S120, namely the image local consistency enhancement step, includes: randomly cropping the face reconstruction image into multiple random image patches and pairing them to obtain multiple image patch pairs; extracting features from each random image patch using a teacher model; calculating the feature distance within each image patch pair based on the extracted features; and calculating the local feature consistency loss based on the feature distance within each image patch pair using a constructed local feature consistency loss function. Furthermore, the parameters of the generator model are updated at least based on the local feature consistency loss.
[0054] In another embodiment of the present invention, step S120, namely the image local consistency enhancement step, further includes: calculating the pixel difference loss between the input real face image and the output face reconstruction image based on the pixel differences between each pixel point of the input real face image and the output face reconstruction image of the generator model using a constructed pixel difference loss function; and / or calculating the identity preservation loss between the input real face image and the output face reconstruction image based on the distance between the identity feature vectors of the input real face image and the output face reconstruction image of the generator model using a constructed identity preservation loss function. Updating the parameters of the generator model at least according to the local feature consistency loss includes: updating the parameters of the generator model according to the local feature consistency loss, the pixel difference loss, and / or the identity preservation loss.
[0055] The teacher network is a face recognition neural network that has been trained on a face dataset and possesses prior knowledge.
[0056] Step S130, namely the image discrimination step, includes: inputting a real face image and a face reconstruction image output by the current generator into a pre-established discriminator model, so that the discriminator model outputs a binary classification result indicating whether the input face reconstruction image is a real face image, calculating a true / false discrimination loss based on the binary classification result using a constructed discriminator loss function, and updating the parameters of the discriminator model based on the true / false discrimination loss.
[0057] The image reconstruction process repeats the above generation steps, image local consistency enhancement steps, and image discrimination steps, and alternately updates the parameters of the generator model and the discriminator model using stochastic gradient descent. The training process ends when the fluctuation of the true / false discrimination loss is lower than a preset threshold, thereby obtaining the trained generator model.
[0058] In one embodiment of the present invention, the pixel difference loss function is expressed as follows:
[0059]
[0060] Where W and H are the width and height of the image, respectively. and The pixel value at each pixel location for the corresponding real face image and the reconstructed face image.
[0061] In one embodiment of the present invention, the identity preservation loss function is expressed as follows:
[0062]
[0063] in, and The identity feature vectors are the real face image and the reconstructed face image. and Feature maps extracted from the input real face image and the face reconstruction image.
[0064] In one embodiment of the present invention, the local feature consistency loss function is expressed as follows:
[0065]
[0066] in, and , respectively, are the feature maps of two randomly paired image patches extracted by the teacher model, N is the total number of pairs of image patches randomly cropped from a face reconstruction image, and KL is the information divergence calculation formula used to calculate feature distance.
[0067] In embodiments of the present invention, both the real face image and the reconstructed face image can be divided into key regions and non-key regions. The key regions include facial features, and the non-key regions are non-facial feature regions. Key and non-key regions are divisions of different areas of the face. Key regions possess obvious facial recognition features, such as the eyes, nose, and mouth, while non-key regions lack obvious features, such as the forehead and cheeks. Because the images generated by adversarial attacks are covert—that is, the proportion of added perturbation pixels is smaller than the range perceptible to the human eye (absolute color value change not exceeding 3%)—and experimental statistics show that under covert conditions, most of the limited perturbation elements are concentrated in the key regions of the face image, this invention, based on local feature consistency, aims to narrow the semantic feature distance between key and non-key regions for targeted adversarial defense. Specifically, step S120, which involves randomly cropping the face reconstruction image into multiple random image blocks and pairing them to obtain multiple image block pairs, includes: randomly cropping the face reconstruction image into multiple random image blocks and pairing image blocks belonging to the key region of the face reconstruction image with image blocks belonging to the secondary key region of the face reconstruction image to obtain multiple image block pairs.
[0068] This invention presents a training method based on local feature consistency optimization, which enhances the generator's feature extraction capability for secondary critical regions. Adversarial attacks severely damage the identity information of key regions in facial images, making it difficult for the generator model to reconstruct the facial image based on the identity information of these regions. Therefore, it is necessary to utilize the features of secondary critical regions. However, the features of secondary critical regions are not significant and are difficult to extract. Therefore, a teacher model is needed to assist the training process.
[0069] In one embodiment of the present invention, in step S120, the feature extracted from each random image block using the teacher model is a feature map containing two-dimensional spatial features of the face.
[0070] In one embodiment of this invention, the teacher network is a face recognition neural network with prior knowledge, already trained on a face dataset. It should be noted that teacher networks are typically neural networks with a large number of parameters, complex model structures, and are already trained. Due to their advantages in parameters and structure, they possess strong feature extraction capabilities, and the extracted feature vectors can serve as pseudo-labels to guide the training process of weaker networks. In this invention, a pre-trained teacher network is introduced to extract identity feature vectors from face images, and these identity feature vectors are used as supervisory information during the training process. Similar modules with training guidance functions can replace the function of the teacher network, such as manually specified face recognition rules, but using a teacher network to provide supervisory information yields the best training results.
[0071] Combination Figure 1The content shown here provides a detailed explanation of the training process of the adversarial generative network generator model training method for face recognition in this embodiment:
[0072] In step S110:
[0073] First, a real face image x is input. Then, a generator G (i.e., the generator model) encodes the real face image x into a feature vector (equivalent to high-dimensional semantic features), and then decodes the feature vector into a reconstructed face image y. The generative adversarial network (GAN) structure of the generator G is an auto-encoder-decoder structure. The encoder in the GAN extracts identity feature information from the secondary key regions of the face, and the corresponding decoder generates the face, effectively eliminating interference pixels generated by adversarial attacks on the face image and improving the robustness of face recognition applications.
[0074] Furthermore, at this stage, the pixel difference loss and identity preservation loss between the real face image x and the reconstructed face image y are calculated. This identity preservation loss measures the loss of identity features between the real face image x and the reconstructed face image y, training the generator G to maintain the identity features of the real face image x during image reconstruction, thus better integrating with face recognition technology. This invention introduces a teacher network as supervisory information for identity feature preservation and designs a multi-level identity feature distance optimization function (i.e., one-dimensional feature vectors and two-dimensional feature maps) to improve the original model's ability to preserve identity features during image reconstruction, ensuring that the reconstructed face image y maintains the identity features of the original input face image as much as possible. The introduction of the teacher network effectively analyzes the feature information of image patches. Through the designed random pairing and feature distance measurement methods, significant differences in features between adversarial attack samples and the original perceived image can be analyzed.
[0075] The teacher network is a pre-trained face recognition neural network with prior knowledge, trained on a face dataset. It should be noted that teacher networks are typically large-parameter, complex, and pre-trained neural networks. Due to their advantages in parameters and structure, they possess strong feature extraction capabilities, and the extracted feature vectors can serve as pseudo-labels to guide the training process of weaker networks. In this invention, a pre-trained teacher network is introduced to extract identity feature vectors from face images, and these identity feature vectors are used as supervisory information during training. Similar modules with training guidance functions can replace the function of the teacher network, such as manually specified face recognition rules, but using a teacher network to provide supervisory information yields the best training results.
[0076] In step S120:
[0077] First, the reconstructed face image y is cropped into multiple image patches through a random cropping operation, and then these patches are randomly paired. Furthermore, the feature extraction capabilities of the teacher network, which possesses prior knowledge, are used to extract features from the image patches, calculate the feature distance between image pairs, and then calculate the local feature consistency loss. After completing step S110, the parameters of the generator G are updated using the calculated pixel difference loss, identity preservation loss, and local feature consistency loss.
[0078] In another embodiment of the present invention, the feature extraction capability of the teacher network is used to extract features from the image block to obtain a feature map. The feature map is a two-dimensional feature and is an intermediate result when the face recognition model extracts identity features. Compared with the one-dimensional feature vector, it has a lower degree of semantic abstraction and has certain two-dimensional spatial features of the face.
[0079] Regarding local feature consistency loss, experiments show that images containing adversarial attacks have a larger semantic feature distance between randomly paired image patches. This invention aims to reduce the semantic distance between image patches, enabling images containing adversarial attacks to retain more features of the original image, thus eliminating pixel interference from adversarial attacks. In this invention, the problem of reducing the semantic feature distance between image patches is defined as a local consistency optimization problem. A local consistency enhancement stage is added to the face image reconstruction generation process, allowing the generator model to better extract identity information from secondary key regions that has not been corrupted by adversarial attack pixels, thereby strengthening the local consistency of the generated image and improving the adversarial defense capability of face recognition technology.
[0080] This invention leverages the characteristic that interference pixels in adversarial attack samples tend to cluster in key facial regions. It employs local random sampling techniques to obtain random local images of the face and utilizes a powerful teacher network (a face recognition model trained on a publicly available large-scale face dataset) to analyze and extract its high-dimensional semantic features. The feature distance calculation formulas used include Euclidean distance and / or cosine distance to calculate the distances between image patches and statistically analyze the distance differences between adversarial attack samples and ordinary images. Due to the limitations of the stealth inherent in adversarial attacks, the appearance differences between adversarial attack samples and ordinary images are small, lacking significant texture or shape features for differentiation. Therefore, the above method is used to explore the feature differences between adversarial attack samples and ordinary images.
[0081] Specifically, in step S120, the step of randomly cropping the face reconstruction image into multiple random image blocks and pairing them to obtain multiple image block pairs includes: randomly cropping the face reconstruction image into multiple random image blocks and pairing image blocks belonging to the key region of the face reconstruction image with image blocks belonging to the secondary key region of the face reconstruction image to obtain multiple image block pairs.
[0082] In step S130:
[0083] In the training of discriminator D (i.e., the discriminator model), after one round of training (a), we obtain a batch of reconstructed images generated by G. These reconstructed images and real images are input into discriminator D. Discriminator D outputs a binary classification result indicating whether the input face reconstructed image is a real face image. It also outputs a classification of the input image, distinguishing whether it is a real image. Then, a true / false discrimination loss, measuring the discriminator D's discrimination ability, is calculated, and the model parameters in discriminator D are updated based on this loss. Here, discriminator D is a small convolutional neural network classification model that uses either natural or generated images as input and outputs a binary classification result indicating whether the input image is a natural image to judge the training effect of generator G.
[0084] Repeat steps S110, S120, and S130 to alternately update the parameters of the generator G and the discriminator D using stochastic gradient descent. The training process ends when the fluctuation of the true / false discrimination loss is lower than a preset threshold, thereby obtaining the trained generator model.
[0085] Throughout the training process, the generator G and discriminator D alternately update and act as adversaries. The generator G needs to generate images that the discriminator D cannot distinguish between real and fake images, while the discriminator D needs to differentiate between natural and generated images as clearly as possible. G and D alternately update their internal parameters in this adversarial game until the generator G can generate images that the discriminator D cannot distinguish between real and fake images. The metric for not being able to distinguish between real and fake images is the discriminator D's real / fake discrimination loss. When the value of the real / fake discrimination loss tends to reach a balanced state during training (i.e., the fluctuation of the real / fake discrimination loss is below a preset threshold), the generator's training is considered complete.
[0086] The trained generator G can serve as a data preprocessing module for adversarial defense, particularly in the field of face recognition technology, effectively ensuring the accuracy and stability of face recognition technology in environments containing adversarial attacks.
[0087] Furthermore, this training method uses only natural images (i.e., real human face images) as the training dataset throughout the entire training process, eliminating the need to incur additional costs associated with generating adversarial attack samples for training. Experiments show that generating adversarial attack samples is 20 times more expensive than using real human face images. Moreover, using only natural images as training input effectively avoids the generator model losing its generalization ability to unknown adversarial attack samples, which is caused by using adversarial attack samples (i.e., attacked images).
[0088] The calculation methods of the loss function in the embodiments of the present invention are listed below:
[0089] (1) The pixel difference loss function can be expressed as:
[0090]
[0091] Where W and H are the width and height of the image, and This represents the pixel value at each pixel location in the corresponding image. The pixel difference loss function measures the effectiveness of image reconstruction by calculating the one-dimensional distance between the pixel values of the input image x and the reconstructed image y.
[0092] (2) The multi-level identity preservation loss function is expressed by the Euclidean distance between them as follows:
[0093]
[0094] in, and Let x represent the identity feature vectors of the input image x and the reconstructed image y. and Let represent the feature maps extracted from the input image x and the reconstructed image y.
[0095] (3) The local feature consistency loss function can be expressed as:
[0096]
[0097] in, and The image patch features are extracted by the teacher model, N is the total number of image patch pairs in an image, and KL is the information divergence calculation formula, used to calculate the distance between two features.
[0098] (4) The loss for true / false judgment is expressed as follows:
[0099] Loss 判别器 =logD(x)+log(1-D(G(x)))
[0100] For part (b) of training the discriminator D, the binary classification loss for the two images is calculated and summed using the input image x and the reconstructed image y = G(x) to obtain the true / false discrimination loss.
[0101] Corresponding to the above methods, the present invention also provides a face reconstruction generation method, based on a generator model trained according to any of the above embodiments, the method comprising the following steps:
[0102] (1) The generator model obtains the face image;
[0103] (2) The encoder of the generator model extracts the feature vector of the real face image, and the decoder of the generator model performs face reconstruction based on the feature vector;
[0104] (3) The generator model outputs a reconstructed face image;
[0105] This generator model serves as a lightweight, non-replaceable adversarial defense module for face recognition. It is simple to deploy, can be trained offline in the cloud, and then embedded as a data preprocessing module into existing face recognition models to enhance their adversarial defense capabilities. The trained generator G acts as a data preprocessing module for adversarial defense, particularly in the field of face recognition technology, effectively ensuring the accuracy and stability of face recognition technology in environments containing adversarial attacks. This invention can be deployed in numerous applications requiring face recognition, such as turnstile access control and quick payment systems, demonstrating strong generalization capabilities.
[0106] Corresponding to the above method, the present invention also provides a face reconstruction generation apparatus, which includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus implements the steps of the generator model training method or the face reconstruction generation method as described above.
[0107] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method as described above. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art. When the program is executed, it implements the steps of the generator model training method or the face reconstruction generation method as described above.
[0108] The generator model training and face reconstruction generation method, apparatus, and storage medium of this invention can analyze the feature information of local image patches by leveraging the feature extraction capabilities of the teacher network. Through a designed random pairing distance measurement method, it can analyze the significant differences in features between adversarial attack samples and the original perceived image. The trained adversarial generative network generator model for face recognition can reconstruct the input face image to eliminate potential adversarial attacks and achieve good adversarial defense. The generator model training method of this invention has the advantage of low training overhead, as it does not require the assistance of adversarial attacks during training, greatly reducing additional training costs.
[0109] Furthermore, the face reconstruction generation method based on the generator model can serve as a preprocessing module for face recognition to remove adversarial attack elements that may exist in the image, effectively ensuring the accuracy and stability of face recognition technology. This generator model can be embedded into current face recognition models as a lightweight, non-replacement-based adversarial defense module, and can be deployed in numerous applications requiring face recognition, including gate access control and quick payment systems, demonstrating strong generalization and application value.
[0110] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.
[0111] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.
[0112] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.
[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for training a generative adversarial network generator model for face recognition, characterized in that, The method comprises the following steps: an image reconstruction generation step: obtaining a real face image from the training set and inputting the real face image into a pre-established generator model, and generating a face reconstruction image by reconstructing the input real face image via the generator model; an image local consistency strengthening step, which comprises: randomly cropping the face reconstruction image into a plurality of random image blocks and pairing the random image blocks to obtain a plurality of image block pairs, extracting features of each random image block using a teacher model, calculating feature distances within each image block pair based on the extracted features, and calculating a local feature consistency loss based on the feature distances within each image block pair using a constructed local feature consistency loss function; updating parameters of the generator model according to at least the local feature consistency loss; an image discrimination step: inputting a real face image and a face reconstruction image output by the current generator into a pre-established discriminator model to output a binary classification result indicating whether the input face reconstruction image is a real face image from the discriminator model, calculating a true-false discrimination loss based on the binary classification result using a constructed discriminator loss function, and updating parameters of the discriminator model based on the true-false discrimination loss; repeating the image reconstruction generation step, the image local consistency strengthening step, and the image discrimination step to alternately update parameters of the generator model and the discriminator model by a stochastic gradient descent method, ending the training process when fluctuations of the true-false discrimination loss are lower than a preset threshold, and thereby obtaining a trained generator model; the local feature consistency loss function is represented as: wherein, and are the feature maps of the randomly paired two image blocks extracted by the teacher model, is the total number of pairs of image blocks randomly cropped from a reconstructed image of a face, is the information divergence formula used to calculate the feature distance.
2. The method of claim 1, wherein, the image reconstruction generation step further comprises: calculating a pixel difference loss between the input real face image and the output face reconstruction image based on pixel differences between respective pixel points of the input real face image and the output face reconstruction image using a constructed pixel difference loss function; and / or calculating an identity preservation loss between the input real face image and the output face reconstruction image based on distances between identity feature vectors of the input real face image and the output face reconstruction image using a constructed identity preservation loss function; the updating of the parameters of the generator model according to at least the local feature consistency loss comprises updating the parameters of the generator model according to the local feature consistency loss, the pixel difference loss, and / or the identity preservation loss.
3. The method of claim 2, wherein, the pixel difference loss function is represented as: wherein, and W and H are width and height of the image, respectively, and and are pixel values of the corresponding real face image and the face reconstructed image at each pixel position, respectively. the identity preservation loss function is represented as: ； wherein, and are identity feature vectors of the real face image and the face reconstructed image, and are feature maps extracted from the input real face image and the face reconstructed image.
4. The method of claim 1, wherein, the real face image and the face reconstruction image can be divided into a key region and a non-key region, the key region contains a facial feature region, and the non-key region is a non-facial feature region; the step of randomly cropping the face reconstruction image into a plurality of random image blocks and pairing the random image blocks to obtain a plurality of image block pairs comprises randomly cropping the face reconstruction image into a plurality of random image blocks and pairing image blocks belonging to the key region of the face reconstruction image with image blocks belonging to the sub-key region of the face reconstruction image to obtain a plurality of image block pairs.
5. The method of claim 1, wherein, The feature extraction step using the teacher model extracts a feature map containing two-dimensional spatial features of the face.
6. The method of claim 1, wherein, The teacher model is a face recognition neural network trained by a face dataset and having prior knowledge.
7. A face reconstruction generation method, characterized in that, The method comprises: The generator model acquires a face image; An encoder of the generator model extracts a feature vector of the real face image, and a decoder of the generator model generates face reconstruction based on the feature vector; The generator model outputs a face reconstruction image; The generator model is obtained by the generator model training method according to any one of claims 1-6.
8. A face reconstruction generation apparatus comprising a processor and a memory, characterized in that, The memory stores computer instructions, and the processor is configured to execute the computer instructions stored in the memory, so that the device implements the steps of the generator model training method according to any one of claims 1-6 or the steps of the face reconstruction generation method according to claim 7.
9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the generator model training method according to any one of claims 1-6 or the steps of the face reconstruction generation method according to claim 7.