A face recognition method and apparatus

Abstract

This invention discloses a face recognition method and apparatus, relating to the field of computer technology. One specific embodiment of the method includes: acquiring a face image to be recognized; inputting the face image to be recognized into an adversarial example detection model to determine the sample region corresponding to the adversarial example; clearing the sample region in the face image to be recognized to obtain a target face image; and inputting the target face image into a face recognition model for face recognition processing. This embodiment reduces the complexity of the process, improves the security and recognition accuracy of the face recognition model, and enhances the user experience.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a face recognition method and apparatus. Background Technology

[0002] With the development of artificial intelligence technology, object detection and recognition models have been applied in various real-life scenarios. However, attackers, through adversarial attack techniques, can perform escape or substitution attacks on the detected objects, deceiving the detection and recognition models and causing them to make incorrect judgments, thus significantly threatening the security of object detection and recognition models. How to effectively defend against adversarial attacks targeting object detection and recognition models, especially facial recognition models, and improve the security and accuracy of these models, is a pressing issue that needs to be addressed.

[0003] The existing technology has at least the following problems:

[0004] Existing technologies primarily target general object detection and recognition models, offering defense only against attacks at the image domain level. They are ill-suited for real-world scenarios, especially against escape and substitution attacks targeting facial recognition models, resulting in low accuracy and security, and a poor user experience. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a face recognition method and apparatus that can effectively defend against adversarial attacks against face recognition systems, improve the security and recognition accuracy of face recognition models, and enhance user experience.

[0006] To achieve the above objectives, according to one aspect of the present invention, a face recognition method is provided, comprising:

[0007] Acquire the face image to be identified, and input the face image to be identified into the adversarial example detection model to determine the sample region corresponding to the adversarial example;

[0008] The sample regions in the face image to be identified are cleared to obtain the target face image;

[0009] The target face image is input into the face recognition model for face recognition processing.

[0010] Furthermore, prior to the step of acquiring the image of the face to be identified, the method also includes:

[0011] Acquire multiple face images;

[0012] Multiple adversarial samples are generated using an adversarial sample generation model, and then added to face images to obtain multiple face images with adversarial samples.

[0013] Multiple training sample data are obtained by piecing together face images with adversarial examples;

[0014] Based on multiple training sample data and loss functions, an adversarial example detection model is trained and optimized; the adversarial example detection model is used to identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

[0015] Furthermore, the steps for training and optimizing the adversarial example detection model based on multiple training sample data and a loss function include:

[0016] The training sample data is input into the adversarial example detection model for image feature extraction, multi-scale fusion processing, and convolution processing. The detection value is determined by combining the loss function. The detection value indicates whether there is an adversarial example and the sample region corresponding to the adversarial example.

[0017] Backpropagation is performed based on actual and detected values ​​to train and optimize the adversarial example detection model.

[0018] Furthermore, before the step of stitching together the face image with face images containing adversarial examples to obtain multiple training sample data, the method also includes:

[0019] Data augmentation processing is performed on face images and face images with adversarial examples, respectively.

[0020] Further, the step of stitching together the face image with a face image containing adversarial examples to obtain multiple training sample data includes:

[0021] The face image and the face image with adversarial examples are resized to the same size to obtain the initial sample set;

[0022] Multiple images are randomly selected from the initial sample set for mosaic processing to obtain training sample data. The coordinate positions of the adversarial samples are then transformed to determine their coordinate positions within the training sample data.

[0023] Furthermore, after acquiring the face image to be identified and inputting the face image to be identified into the adversarial example detection model, the method also includes:

[0024] Determine whether adversarial examples exist in the face image to be identified;

[0025] If there are no adversarial examples in the face image to be identified, the face image to be identified is input into the face recognition model for face recognition processing.

[0026] According to another aspect of the present invention, a face recognition device is provided, comprising:

[0027] The acquisition module is used to acquire the face image to be identified and input the face image to be identified into the adversarial example detection model to determine the sample region corresponding to the adversarial example;

[0028] The target face image determination module is used to remove sample regions in the face image to be identified, thereby obtaining the target face image;

[0029] The face recognition module is used to input the target face image into the face recognition model for face recognition processing.

[0030] Furthermore, it also includes a model training and optimization module, which is used before the step of acquiring the face image to be recognized to:

[0031] Acquire multiple face images;

[0032] Multiple adversarial samples are generated using an adversarial sample generation model, and then added to face images to obtain multiple face images with adversarial samples.

[0033] Multiple training sample data are obtained by piecing together face images with adversarial examples;

[0034] Based on multiple training sample data and loss functions, an adversarial example detection model is trained and optimized; the adversarial example detection model is used to identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

[0035] According to another aspect of the present invention, an electronic device for face recognition is provided, comprising:

[0036] One or more processors;

[0037] Storage device for storing one or more programs.

[0038] When one or more programs are executed by one or more processors, the one or more processors implement any of the face recognition methods described above.

[0039] According to another aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon that, when executed by a processor, implements any of the face recognition methods described above.

[0040] One embodiment of the above invention has the following advantages or beneficial effects: Because it employs the technical means of acquiring a face image to be recognized, inputting the face image to be recognized into an adversarial example detection model to determine the sample region corresponding to the adversarial example; clearing the sample region in the face image to be recognized to obtain the target face image; and inputting the target face image into a face recognition model for face recognition processing, it overcomes the technical problem that existing technologies mainly target detection and recognition models for general objects, only defending against attacks at the image domain level. This makes them difficult to apply in real-world scenarios, especially against escape and substitution attacks against face recognition models, resulting in low accuracy and security of the recognition model and a poor user experience. Therefore, it achieves the technical effect of effectively defending against adversarial attacks against face recognition systems, improving the security and recognition accuracy of face recognition models, and enhancing the user experience.

[0041] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0042] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0043] Figure 1 This is a schematic diagram of the main flow of a face recognition method provided according to an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the main flow of a face recognition method provided according to another embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of the main modules of a face recognition device provided according to an embodiment of the present invention;

[0046] Figure 4 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0047] Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0048] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0049] Figure 1 This is a schematic diagram of the main flow of a face recognition method provided according to an embodiment of the present invention; as shown. Figure 1 As shown, the face recognition method provided in this embodiment of the invention mainly includes:

[0050] Step S101: Obtain the face image to be identified and input the face image to be identified into the adversarial example detection model to determine the sample region corresponding to the adversarial example.

[0051] Specifically, since adversarial attacks against face recognition models mainly involve adding adversarial samples to the face image to be recognized, before using the face recognition model for face recognition processing, the adversarial samples in the face image to be recognized and the sample regions corresponding to the adversarial samples are first determined by the adversarial sample detection model. This allows the adversarial samples in the corresponding sample regions to be removed before performing face recognition processing, which can effectively improve the security and accuracy of the face recognition model.

[0052] Furthermore, according to an embodiment of the present invention, before the step of acquiring the face image to be identified, the above method further includes:

[0053] Acquire multiple face images;

[0054] Multiple adversarial samples are generated using an adversarial sample generation model, and then added to face images to obtain multiple face images with adversarial samples.

[0055] Multiple training sample data are obtained by piecing together face images with adversarial examples;

[0056] Based on multiple training sample data and loss functions, an adversarial example detection model is trained and optimized; the adversarial example detection model is used to identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

[0057] Specifically, according to specific embodiments of the present invention, multiple face images with different angles and different numbers (face user IDs) can be obtained from a publicly available face dataset as original face images. Then, an adversarial example generation model is used to generate adversarial examples. These adversarial examples can be regions cropped from other face images, blank strips, etc. After deforming the adversarial examples using STN (Spatial Transformer Networks), they are added to the original face images to obtain face images with adversarial examples. The face images and the face images with adversarial examples are then stitched together to obtain multiple training sample data. Finally, based on these training sample data and a loss function, the adversarial example detection model is trained and optimized.

[0058] Because the loss function optimized during the training of adversarial example detection models is calculated separately for each input image (i.e., training sample data), directly inputting the original face image (excluding adversarial example face images) and the face image with adversarial examples into the model for training will only calculate the loss function on the original face image and the face image with adversarial examples separately. The model trained in this way can only learn how to perform localization, but cannot distinguish whether the localized region is an adversarial example. Therefore, during training, detection boxes will be output separately for the original face image and the face image with adversarial examples, rendering the model ineffective. This is because, during the training process, for face images with adversarial examples, the model can only recognize the adversarial stickers, but cannot recognize the real face image in the non-adversarial sticker area.

[0059] By using the above settings, face images are stitched together with face images containing adversarial examples. This involves randomly combining some face images with adversarial examples with face images without adversarial examples (the original face images) as training sample data. This ensures that the same training sample data includes both face images and face images with adversarial examples. During the training process, when calculating the loss, for a single input sample, the adversarial example detection model can detect both face images with adversarial examples and the original face image. This allows it to distinguish whether the detected region is an adversarial example, thus providing support for the subsequent removal of sample regions corresponding to adversarial examples, avoiding impact on subsequent face recognition, and improving the accuracy of face recognition.

[0060] According to another embodiment of the present invention, face images with adversarial examples can also be obtained directly in real-world scenarios. For example, an actual user can wear (or attach) adversarial examples to their face and take photos from different angles to obtain face images with adversarial examples.

[0061] Preferably, according to an embodiment of the present invention, the step of training and optimizing the adversarial example detection model based on multiple training sample data and a loss function includes:

[0062] The training sample data is input into the adversarial example detection model for image feature extraction, multi-scale fusion processing, and convolution processing. The detection value is determined by combining the loss function. The detection value indicates whether there is an adversarial example and the sample region corresponding to the adversarial example.

[0063] Backpropagation is performed based on actual and detected values ​​to train and optimize the adversarial example detection model.

[0064] Specifically, according to embodiments of the present invention, the adversarial example detection model can select an existing target detection model, such as the RetinaNet model (a single-stage target detection model), input training sample data, and combine it with a loss function to train and optimize the model, so that the adversarial example data can effectively identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

[0065] Optionally, according to an embodiment of the present invention, before the step of stitching together a face image with a face image containing adversarial examples to obtain multiple training sample data, the method further includes:

[0066] Data augmentation processing is performed on face images and face images with adversarial examples, respectively.

[0067] For example, data augmentation processing mainly includes random horizontal flipping, random translation, random color changes, etc. By performing data augmentation processing on face images and face images with adversarial examples respectively, it helps to increase the diversity of training data and thus improve the generalization ability of the model.

[0068] Furthermore, according to an embodiment of the present invention, the step of performing a mosaic process on a face image and a face image with adversarial examples to obtain multiple training sample data includes:

[0069] The face image and the face image with adversarial examples are resized to the same size to obtain the initial sample set;

[0070] Multiple images are randomly selected from the initial sample set for mosaic processing to obtain training sample data. The coordinate positions of the adversarial samples are then transformed to determine their coordinate positions within the training sample data.

[0071] The image dimensions mentioned above refer to the width and height of each face image in the training sample data. The size of the training data is fixed, and the specific values ​​are determined by the subsequent face recognition model. The coordinate positions indicate the coordinates of the adversarial example in the image. Since the origin of the coordinate system changes after the image is stitched together, the coordinate positions of the adversarial example are transformed to correspond to its actual coordinates in the training sample data.

[0072] Step S102: The sample region in the face image to be identified is cleared to obtain the target face image.

[0073] Specifically, according to embodiments of the present invention, the sample region corresponding to the adversarial sample in the face image to be identified can be whitened to remove the adversarial sample in the face image to be identified, so as to avoid the adversarial sample affecting face recognition during subsequent face recognition.

[0074] Step S103: Input the target face image into the face recognition model for face recognition processing.

[0075] According to an embodiment of the present invention, adversarial samples in the image to be identified are determined using the above-mentioned adversarial sample detection model. After the adversarial samples are removed to obtain the target face image, it can be directly input into the face recognition model for processing. Since the adversarial samples have been removed, adversarial attacks against face recognition are effectively avoided, improving the security and accuracy of the face recognition model and enhancing the user experience.

[0076] Furthermore, according to embodiments of the present invention, after the steps of acquiring the face image to be identified and inputting the face image to be identified into the adversarial example detection model, the above method further includes:

[0077] Determine whether adversarial examples exist in the face image to be identified;

[0078] If there are no adversarial examples in the face image to be identified, the face image to be identified is input into the face recognition model for face recognition processing.

[0079] If the adversarial sample detection model finds that there are no adversarial samples in the face image to be identified, according to the embodiments of the present invention, the face image to be identified can be directly input into the face recognition model for processing.

[0080] According to the technical solution of this invention, by acquiring a face image to be recognized, inputting the face image to be recognized into an adversarial example detection model to determine the sample region corresponding to the adversarial example; clearing the sample region in the face image to be recognized to obtain the target face image; and inputting the target face image into a face recognition model for face recognition processing, this method overcomes the technical problem that existing technologies mainly target detection and recognition models for general objects, which only defend against attacks at the image domain level. They are difficult to apply in real-world scenarios, especially against escape and substitution attacks against face recognition models, resulting in low accuracy and security of the recognition model and a poor user experience. Therefore, this method effectively defends against adversarial attacks against face recognition systems, improves the security and recognition accuracy of face recognition models, and enhances the user experience.

[0081] Figure 2 This is a schematic diagram of the main flow of a face recognition method provided according to another embodiment of the present invention; as shown Figure 2 As shown, the face recognition method provided in this embodiment of the invention mainly includes:

[0082] Step S201: Acquire multiple face images; generate multiple adversarial samples using an adversarial sample generation model, and add the adversarial samples to the face images to obtain multiple face images with adversarial samples.

[0083] Specifically, according to the specific implementation of the present invention, multiple face data with different angles and different numbers (face user IDs) can be obtained from a publicly available face dataset as original face images, and then adversarial examples can be generated using an adversarial example generation model. The adversarial examples can be partial regions cropped from other face images, blank stickers, etc.

[0084] According to another embodiment of the present invention, face images with adversarial examples can also be obtained directly in real-world scenarios. For example, an actual user can wear (or attach) adversarial examples to their face and take photos from different angles to obtain face images with adversarial examples.

[0085] Optionally, according to an embodiment of the present invention, before the step of stitching together a face image with a face image containing adversarial examples to obtain multiple training sample data, the method further includes:

[0086] Data augmentation processing is performed on face images and face images with adversarial examples, respectively.

[0087] For example, data augmentation processing mainly includes random horizontal flipping, random translation, random color changes, etc. By performing data augmentation processing on face images and face images with adversarial examples respectively, it helps to increase the diversity of training data and thus improve the generalization ability of the model.

[0088] Step S202: The face image is stitched together with the face image containing adversarial examples to obtain multiple training sample data.

[0089] Furthermore, according to an embodiment of the present invention, the step of performing a mosaic process on a face image and a face image with adversarial examples to obtain multiple training sample data includes:

[0090] The face image and the face image with adversarial examples are resized to the same size to obtain the initial sample set;

[0091] Multiple images are randomly selected from the initial sample set for mosaic processing to obtain training sample data. The coordinate positions of the adversarial samples are then transformed to determine their coordinate positions within the training sample data.

[0092] The image dimensions mentioned above refer to the width and height of each face image in the training sample data. The size of the training data is fixed, and the specific values ​​are determined by the subsequent face recognition model. The coordinate positions indicate the coordinates of the adversarial example in the image. Since the origin of the coordinate system changes after the image is stitched together, the coordinate positions of the adversarial example are transformed to correspond to its actual coordinates in the training sample data.

[0093] Because the loss function optimized during the training of adversarial example detection models is calculated separately for each input image (i.e., training sample data), directly inputting the original face image (excluding adversarial example face images) and the face image with adversarial examples into the model for training will only calculate the loss function on the original face image and the face image with adversarial examples separately. The model trained in this way can only learn how to perform localization, but cannot distinguish whether the localized region is an adversarial example. Therefore, during training, detection boxes will be output separately for the original face image and the face image with adversarial examples, rendering the model ineffective. This is because, during the training process, for face images with adversarial examples, the model can only recognize the adversarial stickers, but cannot recognize the real face image in the non-adversarial sticker area.

[0094] Specifically, adversarial examples can be deformed using Spatial Transformer Networks (STNs) and added to the original face image to obtain a face image with adversarial examples. This face image is then stitched together with the face image containing adversarial examples to obtain multiple training sample data. Based on these training sample data and the loss function, the adversarial example detection model is trained and optimized. This setup, where face images with and without adversarial examples are randomly stitched together as training sample data, ensures that the same training sample data includes both face images and face images with adversarial examples. This guarantees that during loss calculation during training, for a single input sample, the adversarial example detection model can detect both face images with and without adversarial examples, allowing it to distinguish whether a detected region is an adversarial example. This supports the subsequent removal of sample regions corresponding to adversarial examples, avoiding impact on subsequent face recognition and improving the accuracy of face recognition.

[0095] Step S203: Input the training sample data into the adversarial example detection model for image feature extraction, multi-scale fusion processing and convolution processing, and determine the detection value by combining the loss function; wherein the detection value indicates whether there is an adversarial example and the sample region corresponding to the adversarial example; backpropagation is performed based on the actual value and the detection value to train and optimize the adversarial example detection model.

[0096] Specifically, according to embodiments of the present invention, the adversarial example detection model can select an existing target detection model, such as the RetinaNet model (a single-stage target detection model), input training sample data, and combine it with a loss function to train and optimize the model, so that the adversarial example data can effectively identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

[0097] According to an embodiment of the present invention, a specific implementation method for training and optimizing an adversarial example detection model is provided (it should be noted that the numerical values ​​in the specific implementation method are merely examples and are not intended to limit this application; the image size indicated by the numerical values ​​refers to the width and height of each face image in the training sample data; the size of the training sample data is fixed, and the specific value is determined based on the subsequent face recognition model):

[0098] First, determine the training sample data. After determining the original face image and the face image with adversarial examples, adjust both to a size of 120×120 (length and width). Then, randomly select four images (including both the original face image and the face image with adversarial examples) and stitch them together to form a 240×240 training sample image.

[0099] Secondly, feature extraction was performed. The RetinaNet model was selected as the adversarial example detection model, and the lightweight MobileNet was used as the backbone network model. Input data of size 240×240×3 (where 3 represents the number of channels in the feature map) was fed into the backbone network model for feature extraction. At different stages of the model, three feature maps of different sizes were obtained: P1 (30×30×128), P2 (16×16×256), and P3 (8×8×512). Then, three 1×1 convolutional layers were used to transform the number of channels in the three feature maps, adjusting the number of channels to 64, resulting in P1' (30×30×64), P2' (16×16×64), and P3' (8×8×64).

[0100] Then, multi-scale fusion processing is performed. Feature map P3' has the smallest size and does not need to be fused with other feature maps. It is directly sent to the context module for convolution processing. Feature map P2' is fused with feature map P3'. The specific operation is as follows: first, the size dimension of P3' (8×8×64) is adjusted to 16×16×64 by upsampling. Then, it is directly added to the corresponding position of P2' (16×16×64) to obtain the fused P2" (16×16×64). The fused P2" is then input to the context module for convolution processing (the image size of the feature map that performs multi-scale fusion does not change. The features of other feature maps are directly added to it). Similarly, P1' is also multi-scale fusion processed to obtain C1.

[0101] Then, the feature map after multi-scale fusion is input into the context module, which consists of several convolutional layers with larger kernel sizes (e.g., 5×5), and the principle of keeping the size of the output feature map unchanged is maintained. The purpose is to introduce more contextual information by expanding the receptive field of view.

[0102] Finally, the detected values ​​determined by the loss function and the actual values ​​are combined to perform backpropagation on the model for further optimization. The detected values ​​indicate the presence of adversarial examples and their corresponding sample regions. The actual values ​​indicate the presence of adversarial examples in the face image and their actual coordinates.

[0103] This invention also provides a specific implementation method for determining the loss function of an adversarial example detection model:

[0104] The loss function includes a bounding box location loss function and a bounding box classification loss function. Before calculating the loss function, the number of channels output by the context module needs to be adjusted. Taking P2” as an example, the feature map size of P2” is 16×16×64, and it remains 16× after passing through the context module.

[0105] To calculate the bounding box location loss, the feature map size is reduced to 16×16×4 using a detection head (a 1×1 convolution operation), with 4 channels representing the bounding box location information. Similarly, to calculate the bounding box classification loss, the feature map size is reduced to 16×16×2 using a classification head (a 1×1 convolution operation), with 2 channels representing the bounding box classification information (whether the region is an adversarial example or not). The final loss function is the sum of the weighted bounding box location loss function and the bounding box classification loss function.

[0106]

[0107] Where i represents the index of the regression box, p i This represents the probability that the bounding box is an adversarial example. These are the labels for the regression bounding box; 1 represents an adversarial example, and 0 represents a non-adversarial example. t i It is the location information of the regression box. This is the location information of the adversarial example regression bounding box. L cls Two-class (adversarial and non-adversarial) softmax loss functions are used, L reg The smooth L1 loss function is used, and Multiplication ensures that the location loss function is calculated only for adversarial bounding boxes. λ is used to balance the location loss function and the classification loss function of the bounding boxes.

[0108] The adversarial example detection model trained by this invention through steps S201 to S203 is lightweight, has a high success rate in defending against adversarial attacks, and is robust. It is also plug-and-play and can be attached to any face recognition model to effectively detect adversarial examples in real-world scenarios. This helps defend against identity impersonation and substitution attacks targeting face recognition models, ensuring the security of the face recognition system and making it highly practical.

[0109] Step S204: Obtain the face image to be identified, input the face image to be identified into the adversarial example detection model, and determine whether there is an adversarial example. If yes, that is, there is an adversarial example in the face image to be identified, proceed to step S205; if no, that is, there is no adversarial example in the face image to be identified, proceed to step S207.

[0110] Specifically, since adversarial attacks against face recognition models mainly involve adding adversarial samples to the face image to be recognized, before using the face recognition model for face recognition processing, the adversarial samples in the face image to be recognized and the sample regions corresponding to the adversarial samples are first determined by the adversarial sample detection model. This allows the adversarial samples in the corresponding sample regions to be removed before performing face recognition processing, which can effectively improve the security and accuracy of the face recognition model.

[0111] Step S205: Remove the sample region in the face image to be identified to obtain the target face image.

[0112] Specifically, according to embodiments of the present invention, the sample region corresponding to the adversarial sample in the face image to be identified can be whitened to remove the adversarial sample in the face image to be identified, so as to avoid the adversarial sample affecting face recognition during subsequent face recognition.

[0113] Step S206: Input the target face image into the face recognition model for face recognition processing.

[0114] According to an embodiment of the present invention, adversarial samples in the image to be identified are determined using the above-mentioned adversarial sample detection model. After the adversarial samples are removed to obtain the target face image, it can be directly input into the face recognition model for processing. Since the adversarial samples have been removed, adversarial attacks against face recognition are effectively avoided, improving the security and accuracy of the face recognition model and enhancing the user experience.

[0115] Step S207: The face image to be identified, which does not contain adversarial examples, is directly input into the face recognition model for face recognition processing.

[0116] According to the technical solution of this invention, by acquiring a face image to be recognized, inputting the face image to be recognized into an adversarial example detection model to determine the sample region corresponding to the adversarial example; clearing the sample region in the face image to be recognized to obtain the target face image; and inputting the target face image into a face recognition model for face recognition processing, this method overcomes the technical problem that existing technologies mainly target detection and recognition models for general objects, which only defend against attacks at the image domain level. They are difficult to apply in real-world scenarios, especially against escape and substitution attacks against face recognition models, resulting in low accuracy and security of the recognition model and a poor user experience. Therefore, this method effectively defends against adversarial attacks against face recognition systems, improves the security and recognition accuracy of face recognition models, and enhances the user experience.

[0117] Figure 3 This is a schematic diagram of the main modules of a face recognition device provided according to an embodiment of the present invention; as shown. Figure 3 As shown, the face recognition device 300 provided in this embodiment of the invention mainly includes:

[0118] The acquisition module 301 is used to acquire the face image to be identified and input the face image to be identified into the adversarial example detection model to determine the sample region corresponding to the adversarial example.

[0119] Specifically, since adversarial attacks against face recognition models mainly involve adding adversarial samples to the face image to be recognized, before using the face recognition model for face recognition processing, the adversarial samples in the face image to be recognized and the sample regions corresponding to the adversarial samples are first determined by the adversarial sample detection model. This allows the adversarial samples in the corresponding sample regions to be removed before performing face recognition processing, which can effectively improve the security and accuracy of the face recognition model.

[0120] Furthermore, the aforementioned face recognition device 300 also includes a model training and optimization module. Before the step of acquiring the face image to be recognized, the model training and optimization module is used for:

[0121] Acquire multiple face images;

[0122] Multiple adversarial samples are generated using an adversarial sample generation model, and then added to face images to obtain multiple face images with adversarial samples.

[0123] Multiple training sample data are obtained by piecing together face images with adversarial examples;

[0124] Based on multiple training sample data and loss functions, an adversarial example detection model is trained and optimized; the adversarial example detection model is used to identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

[0125] Specifically, according to specific embodiments of the present invention, multiple face images with different angles and different numbers (face user IDs) can be obtained from a publicly available face dataset as original face images. Then, an adversarial example generation model is used to generate adversarial examples. These adversarial examples can be regions cropped from other face images, or blank stickers, etc. After deforming the adversarial examples using STN (Spatial Transformer Networks), they are added to the original face images to obtain face images with adversarial examples. The face images and the face images with adversarial examples are then stitched together to obtain multiple training sample data. Finally, based on these training sample data and a loss function, the adversarial example detection model is trained and optimized.

[0126] By using the above settings, face images are stitched together with face images containing adversarial examples. This involves randomly combining some face images with adversarial examples with face images without adversarial examples (the original face images) as training sample data. This ensures that the same training sample data includes both face images and face images with adversarial examples. During the training process, when calculating the loss, for a single input sample, the adversarial example detection model can detect both face images with adversarial examples and the original face image. This allows it to distinguish whether the detected region is an adversarial example, thus providing support for the subsequent removal of sample regions corresponding to adversarial examples, avoiding impact on subsequent face recognition, and improving the accuracy of face recognition.

[0127] According to another embodiment of the present invention, face images with adversarial examples can also be obtained directly in real-world scenarios. For example, an actual user can wear (or attach) adversarial examples to their face and take photos from different angles to obtain face images with adversarial examples.

[0128] Preferably, according to an embodiment of the present invention, the above-described model training and optimization module is further used for:

[0129] The training sample data is input into the adversarial example detection model for image feature extraction, multi-scale fusion processing, and convolution processing. The detection value is determined by combining the loss function. The detection value indicates whether there is an adversarial example and the sample region corresponding to the adversarial example.

[0130] Backpropagation is performed based on actual and detected values ​​to train and optimize the adversarial example detection model.

[0131] Specifically, according to embodiments of the present invention, the adversarial example detection model can select an existing target detection model, such as the RetinaNet model (a single-stage target detection model), input training sample data, and combine it with a loss function to train and optimize the model, so that the adversarial example data can effectively identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

[0132] Optionally, according to an embodiment of the present invention, the face recognition device 300 further includes a data augmentation module, which, before the step of stitching together the face image with a face image containing adversarial examples to obtain multiple training sample data, is used to:

[0133] Data augmentation processing is performed on face images and face images with adversarial examples, respectively.

[0134] For example, data augmentation processing mainly includes random horizontal flipping, random translation, random color changes, etc. By performing data augmentation processing on face images and face images with adversarial examples respectively, it helps to increase the diversity of training data and thus improve the generalization ability of the model.

[0135] Furthermore, according to embodiments of the present invention, the above-mentioned model training and optimization module is also used for:

[0136] The face image and the face image with adversarial examples are resized to the same size to obtain the initial sample set;

[0137] Multiple images are randomly selected from the initial sample set for mosaic processing to obtain training sample data. The coordinate positions of the adversarial samples are then transformed to determine their coordinate positions within the training sample data.

[0138] The image dimensions mentioned above refer to the width and height of each face image in the training sample data. The size of the training data is fixed, and the specific values ​​are determined by the subsequent face recognition model. The coordinate positions indicate the coordinates of the adversarial example in the image. Since the origin of the coordinate system changes after the image is stitched together, the coordinate positions of the adversarial example are transformed to correspond to its actual coordinates in the training sample data.

[0139] The target face image determination module 302 is used to clear sample regions in the face image to be identified to obtain the target face image.

[0140] Specifically, according to embodiments of the present invention, the sample region corresponding to the adversarial sample in the face image to be identified can be whitened to remove the adversarial sample in the face image to be identified, so as to avoid the adversarial sample affecting face recognition during subsequent face recognition.

[0141] The face recognition module 303 is used to input the target face image into the face recognition model for face recognition processing.

[0142] According to an embodiment of the present invention, adversarial samples in the image to be identified are determined using the above-mentioned adversarial sample detection model. After the adversarial samples are removed to obtain the target face image, it can be directly input into the face recognition model for processing. Since the adversarial samples have been removed, adversarial attacks against face recognition are effectively avoided, improving the security and accuracy of the face recognition model and enhancing the user experience.

[0143] Furthermore, according to an embodiment of the present invention, the face recognition device 300 further includes a judgment module. After acquiring the face image to be recognized and inputting the face image to be recognized into the adversarial example detection model, the judgment module is used to:

[0144] Determine whether adversarial examples exist in the face image to be identified;

[0145] If there are no adversarial examples in the face image to be identified, the face image to be identified is input into the face recognition model for face recognition processing.

[0146] If the adversarial sample detection model finds that there are no adversarial samples in the face image to be identified, according to the embodiments of the present invention, the face image to be identified can be directly input into the face recognition model for processing.

[0147] According to the technical solution of this invention, by acquiring a face image to be recognized, inputting the face image to be recognized into an adversarial example detection model to determine the sample region corresponding to the adversarial example; clearing the sample region in the face image to be recognized to obtain the target face image; and inputting the target face image into a face recognition model for face recognition processing, this method overcomes the technical problem that existing technologies mainly target detection and recognition models for general objects, which only defend against attacks at the image domain level. They are difficult to apply in real-world scenarios, especially against escape and substitution attacks against face recognition models, resulting in low accuracy and security of the recognition model and a poor user experience. Therefore, this method effectively defends against adversarial attacks against face recognition systems, improves the security and recognition accuracy of face recognition models, and enhances the user experience.

[0148] Figure 4 An exemplary system architecture 400 for which the face recognition method or face recognition device of the present invention can be applied is shown.

[0149] like Figure 4As shown, system architecture 400 may include terminal devices 401, 402, and 403, network 404, and server 405 (this architecture is merely an example; the components included in a specific architecture may be adjusted according to the specific application). Network 404 serves as the medium for providing a communication link between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0150] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, and 403, such as facial recognition applications, payment applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0151] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0152] Server 405 can be a server that provides various services, such as a server for users to perform face recognition / data processing using terminal devices 401, 402, and 403 (for example only). This server can analyze and process the received data such as the face image to be recognized, and feed back the processing results (such as face recognition results - for example only) to the terminal device.

[0153] It should be noted that the face recognition method provided in the embodiments of the present invention is generally executed by server 405, and correspondingly, the face recognition device is generally set in server 405.

[0154] It should be understood that Figure 4 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0155] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing terminal devices or servers of the present invention. Figure 5 The terminal device or server shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0156] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0157] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0158] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0159] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0161] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including an acquisition module, a target face image determination module, and a face recognition module. The names of these modules do not necessarily limit the module itself; for example, the acquisition module can also be described as "a module for acquiring a face image to be recognized, inputting the face image to be recognized into an adversarial example detection model, and determining the sample region corresponding to the adversarial example."

[0162] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: acquiring a face image to be recognized; inputting the face image to be recognized into an adversarial example detection model to determine a sample region corresponding to an adversarial example; clearing the sample region in the face image to be recognized to obtain a target face image; and inputting the target face image into a face recognition model for face recognition processing.

[0163] According to the technical solution of this invention, by acquiring a face image to be recognized, inputting the face image to be recognized into an adversarial example detection model to determine the sample region corresponding to the adversarial example; clearing the sample region in the face image to be recognized to obtain the target face image; and inputting the target face image into a face recognition model for face recognition processing, this method overcomes the technical problem that existing technologies mainly target detection and recognition models for general objects, which only defend against attacks at the image domain level. They are difficult to apply in real-world scenarios, especially against escape and substitution attacks against face recognition models, resulting in low accuracy and security of the recognition model and a poor user experience. Therefore, this method effectively defends against adversarial attacks against face recognition systems, improves the security and recognition accuracy of face recognition models, and enhances the user experience.

[0164] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A face recognition method, characterized by, include: Acquire a face image to be identified, and input the face image to be identified into an adversarial example detection model to determine the sample region corresponding to the adversarial example; The adversarial sample is added to the face image to be identified; The sample region in the image of the face to be identified is cleared to obtain the target face image; The target face image is input into the face recognition model for face recognition processing; Prior to the step of acquiring the face image to be identified, the method further includes: Acquire multiple face images; Multiple adversarial samples are generated using an adversarial sample generation model, and these adversarial samples are added to the face image to obtain multiple face images with adversarial samples. The face image is stitched together with the face image containing adversarial examples to obtain multiple training sample data. Based on the multiple training sample data and loss function, an adversarial example detection model is trained and optimized; wherein, the adversarial example detection model is used to identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

2. The face recognition method of claim 1, wherein, The step of training and optimizing the adversarial example detection model based on the multiple training sample data and the loss function includes: The training sample data is input into the adversarial example detection model for image feature extraction, multi-scale fusion processing, and convolution processing. The detection value is determined by combining the loss function. The detection value indicates whether an adversarial example exists and the sample region corresponding to the adversarial example. Backpropagation is performed based on the actual values ​​and the detected values ​​to train and optimize the adversarial example detection model.

3. The face recognition method of claim 1, wherein, Before the step of stitching the face image with the face image containing adversarial examples to obtain multiple training sample data, the method further includes: Data augmentation processing is performed on the face image and the face image with adversarial examples, respectively.

4. The face recognition method of claim 1, wherein, The step of stitching the face image with the face image containing adversarial examples to obtain multiple training sample data includes: The face image and the face image with adversarial examples are resized to the same size to obtain an initial sample set; Multiple images are randomly selected from the initial sample set and stitched together to obtain the training sample data. The coordinates of the adversarial samples are then transformed to determine their coordinates within the training sample data.

5. The face recognition method of claim 1, wherein, After the steps of acquiring the face image to be identified and inputting the face image to be identified into the adversarial example detection model, the method further includes: Determine whether adversarial examples exist in the face image to be identified; If there are no adversarial examples in the face image to be identified, the face image to be identified is input into the face recognition model for face recognition processing.

6. A face recognition device, characterized in that, include: The acquisition module is used to acquire a face image to be identified, input the face image to be identified into an adversarial example detection model to determine the sample region corresponding to the adversarial example; the adversarial example is added to the face image to be identified; The target face image determination module is used to clear sample regions in the face image to be identified to obtain the target face image; The face recognition module is used to input the target face image into the face recognition model for face recognition processing; It also includes a model training and optimization module, which is used before the step of acquiring the face image to be recognized to: Acquire multiple face images; Multiple adversarial samples are generated using an adversarial sample generation model, and these adversarial samples are added to the face image to obtain multiple face images with adversarial samples. The face image is stitched together with the face image containing adversarial examples to obtain multiple training sample data. Based on the multiple training sample data and loss function, an adversarial example detection model is trained and optimized; wherein, the adversarial example detection model is used to identify adversarial examples in face images and determine the sample regions corresponding to the adversarial examples.

7. An electronic device for facial recognition, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer readable medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

Images