Face adversarial sample defense method, face adversarial sample defense device and medium

Abstract

The present application provides a face adversarial sample defense method, a face adversarial sample defense device and a storage medium. The face adversarial sample defense method comprises: a feature dictionary construction step, obtaining a sample image, aligning the sample image, dividing the sample image into a plurality of picture blocks according to a plurality of key points, extracting features of the plurality of picture blocks respectively to generate feature vectors, and saving the feature vectors of the sample image about each key point as a feature dictionary of each key point; a test sample feature extraction step, obtaining a test image about a test sample, aligning the test image, dividing the test image into a plurality of picture blocks according to a plurality of key points, and extracting features of the plurality of picture blocks respectively to generate feature vectors; and an adversarial sample detection step, about the test sample, calculating distances between the feature vectors of the test image about each key point and the feature vectors in the feature dictionary of the corresponding key point, and judging whether the test sample is an adversarial sample by comparing the distances with a threshold value.

Description

Technical Field

[0001] This invention relates to image recognition technology, and in particular to a method, device, and storage medium for defending against adversarial examples of faces. Background Technology

[0002] In real life, face detection technology has been applied to various industries. However, commonly used face detection algorithms are usually based on neural networks, which brings a significant problem: adversarial examples. Adversarial examples involve adding noise that is imperceptible to the human eye to the original sample. This noise does not affect human recognition but can easily deceive the neural network, causing it to make a completely different judgment. For example, an attacker can add subtle changes to the original sample that are difficult for humans to perceive, yet still cause the face recognition model to make an incorrect classification decision.

[0003] For example, simply printing a patterned strip of paper with a regular printer and sticking it on the forehead can cause the industry's leading public Face ID system to malfunction.

[0004] Figure 1 (a) is a normal picture of the test subject. However, as... Figure 1 As shown in (b), even though the paper did not obscure the face after the test subject placed a strip of paper on their forehead, the facial recognition model still misidentified them. This real-world perturbation is devastating to the system's operation.

[0005] Although these stickers, for the sake of concealment, may resemble human foreheads, such as... Figure 2 As shown in (a) and (b), the perturbation is unpredictable due to the invisibility of the inside of the attacked neural network.

[0006] In real-world applications, adversarial attacks can sometimes pose significant security risks. For example, in facial recognition, attackers can use adversarial examples to bypass verification and gain privileges they wouldn't otherwise have.

[0007] As mentioned above, the existence of adversarial examples weakens neural networks, becoming one of the main risks to neural networks in many critical security environments. Therefore, techniques to defend against adversarial examples are needed to address this problem.

[0008] In practical applications, there are numerous unknown methods for generating adversarial examples. Traditional methods often require developing defense strategies based on specific adversarial examples, which is difficult to implement in reality. Furthermore, the natural environment presents numerous disturbances, necessitating different defense methods for different disturbance scenarios. Therefore, traditional defense methods suffer from poor generalization ability. Summary of the Invention

[0009] In view of this, the purpose of the present invention is to provide a method, device and storage medium for defending against adversarial examples of faces, so as to be able to determine whether a test sample is an adversarial example without the need to prepare various adversarial examples.

[0010] According to one aspect of the present invention, a method for defending against adversarial examples of faces is provided, comprising:

[0011] The feature dictionary construction steps are as follows: For normal samples, obtain a face image of a normal sample as a sample image, align the sample image so that multiple key points of the face in the sample image are aligned, divide the sample image into multiple image blocks according to the multiple key points, extract features for each key point of the image block of the sample image to generate a feature vector, and save the feature vectors of each key point of the sample image as the feature dictionary of each key point.

[0012] The test sample feature extraction steps involve: acquiring a face image of the test sample as the test image; aligning the test image to ensure that multiple key points of the face in the test image are aligned; segmenting the test image into multiple image patches according to the multiple key points; and extracting features from each image patch for each key point to generate a feature vector; and...

[0013] The adversarial example detection step involves, for the test sample, calculating the distance between the feature vectors of each key point of the test image and the feature vectors in the feature dictionary of the corresponding key points, and comparing this distance with a threshold to determine whether the test sample is an adversarial example.

[0014] According to the above-mentioned face adversarial example defense method, the key points are selected as the facial features, or 29 key points labeled by the LFPW face database, or 21 key points labeled by the AFLW face database, or multiple custom key points.

[0015] According to the above-mentioned face adversarial example defense method, in the feature dictionary construction step, for each key point of the sample image, features are extracted using the corresponding key point autoencoder to generate feature vectors. In the test sample feature extraction step, for each key point of the test image, features are extracted using the corresponding key point autoencoder to generate feature vectors.

[0016] According to the above-mentioned face adversarial example defense method, in the feature dictionary construction step, for normal samples, m face images are obtained as m sample images, where m is an integer greater than 1. The m sample images are aligned respectively, and the m sample images are divided into multiple image blocks according to the multiple key points. For each key point of the m sample images, features are extracted from the m image blocks to generate m feature vectors. The m feature vectors of each key point of the m sample images are saved as the feature dictionary of each key point.

[0017] According to the above-mentioned face adversarial example defense method, in the feature dictionary construction step, for normal samples, the m feature vectors of each key point of m sample images are further clustered into k classes using the k-means clustering method to obtain k feature vectors. The k feature vectors after clustering of each key point of multiple key points of m sample images are saved as the feature dictionary of each key point, where 1≤k <m。

[0018] According to the above-mentioned face adversarial example defense method, in the adversarial example detection step, regarding the test sample, when the average distance between the feature vector of at least one key point of the test image and the L nearest feature vectors in the feature dictionary of the corresponding key point is greater than the threshold corresponding to the corresponding key point, the test image is determined to be an adversarial example, where L is an integer greater than 1.

[0019] According to another aspect of the present invention, a face adversarial sample defense device is provided, comprising:

[0020] The acquisition unit acquires face images of normal or test samples.

[0021] An alignment unit aligns the face image acquired by the acquisition unit so that multiple key points of the face in the face image are aligned.

[0022] The segmentation unit divides the face image, which has been aligned by the alignment unit, into multiple image blocks according to the multiple key points.

[0023] The feature extraction unit extracts features from image patches for each key point of the face image, generating feature vectors for each key point; and

[0024] The feature dictionary storage unit, for normal samples, saves the feature vectors of each key point in the face image of a normal sample as the feature dictionary for each key point.

[0025] The adversarial example detection unit, for a test sample, calculates the distance between the feature vectors of each key point of the face image of the test sample and the feature vectors in the feature dictionary of the corresponding key points stored in the feature dictionary storage unit, and determines whether the test sample is an adversarial example by comparing the distance with a threshold.

[0026] According to the aforementioned face adversarial sample defense device, among which,

[0027] The acquisition unit acquires multiple face images based on normal samples and outputs them to the alignment unit.

[0028] The face adversarial sample defense device also includes a clustering unit.

[0029] The clustering unit clusters multiple feature vectors of each key point extracted by the feature extraction unit from multiple face images, and outputs the clustered feature vectors of each key point to the feature dictionary storage unit.

[0030] The feature dictionary storage unit saves the clustered feature vectors of each key point of a normal sample face image as a feature dictionary for each key point.

[0031] According to the aforementioned face adversarial sample defense device, among which,

[0032] The feature extraction unit uses an autoencoder corresponding to each key point of multiple key points to extract the features of the image blocks of the corresponding key points to generate the feature vector of the corresponding key points.

[0033] According to another aspect of the present invention, a face adversarial sample defense device is provided, comprising:

[0034] A memory, on which instructions are stored; and

[0035] The processor is configured to execute instructions stored in the memory to perform the aforementioned face adversarial sample defense method.

[0036] According to another aspect of the present invention, a computer-readable storage medium is provided, comprising computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the above-described face adversarial sample defense method.

[0037] In this invention, face images are aligned for block segmentation, making feature extraction at specific locations easier and significantly reducing the false negative rate against adversarial example perturbations at specific locations. Normal face images are used as the normal sample set input into the face adversarial example defense device, eliminating the need for pre-preparation of various adversarial examples, thus resulting in strong generalization ability. This invention constructs a feature vector dictionary of normal samples, storing their main features, and uses this as a benchmark for adversarial example defense, increasing the success rate of defense. When constructing the feature dictionary, this invention utilizes k-means clustering to intelligently merge various similar features, reducing the size of the feature dictionary and facilitating adversarial example detection. Attached Figure Description

[0038] Figure 1 (a) is a normal photograph of the test subject. Figure 1 (b) is an image of the test subject after the paper strip was stuck to their forehead.

[0039] Figure 2 (a) and (b) are sticker images of adversarial examples of human foreheads.

[0040] Figure 3 This is a flowchart of the face adversarial sample defense method involved in the embodiments of the present invention.

[0041] Figure 4 This is a flowchart of establishing a feature dictionary for normal samples according to an embodiment of the present invention.

[0042] Figure 5 This illustrates a specific embodiment of building a feature dictionary using normal samples.

[0043] Figure 6 This is a flowchart illustrating the judgment of test samples according to an embodiment of the present invention.

[0044] Figure 7 This is a diagram illustrating how distance comparison is used to determine whether a test sample is an adversarial sample.

[0045] Figure 8 This is a structural diagram of the face adversarial sample defense device involved in the embodiments of the present invention.

[0046] Figure 9 This is a structural diagram of a computing device. Detailed Implementation

[0047] The specific implementation method will now be described in detail with reference to the accompanying drawings.

[0048] In the face adversarial example defense method according to the embodiments of the present invention, it is necessary to first establish a feature dictionary of normal samples using normal samples, and then, regarding the test sample, after obtaining the feature vector of the test sample, use the established feature dictionary to determine whether the test sample is an adversarial example. Figure 3 As shown, the face adversarial example defense method involved in the embodiments of the present invention includes the following steps:

[0049] Step S301: For normal samples, obtain the feature vectors of normal samples to build a feature dictionary for normal samples;

[0050] Step S302: Regarding the test sample, obtain the feature vector of the test sample;

[0051] Step S303: Determine whether the test sample is an adversarial sample by comparing the feature vector of the test sample with the feature vector in the feature dictionary.

[0052] The following is combined with Figure 4 This section details the steps involved in building a feature dictionary for normal samples.

[0053] Step S401: Obtain multiple face images as normal samples.

[0054] In this embodiment, for ease of description, m face images are obtained as normal samples, where m is an integer greater than 1.

[0055] Here, face detection can be used to obtain normal sample face images. Face detection refers to searching any given image using a certain strategy to determine whether it contains a face. If a face is found, its position, size, and pose are returned. Commonly used face detection algorithms include MTCNN and Dlib, but this invention does not limit the face detection algorithm.

[0056] Typically, before defense begins, a dataset of normal facial samples needs to be obtained to build a feature dictionary for these samples. The facial dataset can vary depending on the ethnicity being identified; it can be composed of existing online datasets or obtained through web scraping tools—no restrictions are placed here. Because the datasets are not standardized, they need to be inspected to obtain facial images. The face detection method is not limited here; algorithms such as MTCNN and Dlib can be used.

[0057] Step S402: Align the face image so that multiple key points of the face in the face image are aligned.

[0058] Faces detected by face detection algorithms may have misaligned angles, so face image alignment is necessary. This involves finding matching points on the face to improve the accuracy of subsequent face recognition and related tasks. During face image alignment, key facial points, such as the eyes, nose tip, corners of the mouth, eyebrows, and contour points of various facial features, are automatically located based on the input face image. Then, the face images are rotated and translated to ensure that the key points of each face image are in the same position.

[0059] Here, there is no limit to the number of key points. For example, you can choose facial features as key points, or you can choose 29 key points labeled by the LFPW (Labeled Face Parts in the Wild) face database, or you can choose 21 key points labeled by the AFLW (Annotated Facial Landmarks in the Wild) face database, or you can choose multiple custom key points.

[0060] Step S403: The face image is segmented into multiple image blocks according to the multiple key points.

[0061] In this embodiment, the face image has been aligned through step S402, and multiple key points in the face image have been aligned. Therefore, in step S403, the face image can be segmented into different image blocks according to multiple key points. For example, when the facial features are used as key points, the face image can be segmented into five image blocks, including eyes, nose, eyebrows, mouth, and ears respectively. When segmenting the face image into image blocks according to key points, the size of each image block can be customized according to the number of key points, and is not limited here.

[0062] Traditional segmentation typically uses a sliding window, but this invention takes into account the special characteristics of human faces and uses a segmentation method based on key points, which facilitates subsequent feature extraction.

[0063] Step S404: Extract features from the image patch for each key point and generate a feature vector.

[0064] In this step, feature extraction is performed on each image patch for each keypoint using its own autoencoder. In other words, feature extraction can be achieved using an autoencoder. Here, a corresponding autoencoder is set up for each of the multiple keypoints.

[0065] Since m face images were acquired as normal samples in step S401, in step S402, each face image in the m face images is aligned. In step S403, each face image in the m face images is segmented into multiple image blocks according to the multiple key points. Therefore, m image blocks are obtained for each key point. In this case, in step S404, feature vectors are generated by extracting features from each of the m image blocks for each key point using their respective autoencoders.

[0066] Assuming that in step S403, the face image is segmented according to the key points of the facial features to obtain image blocks, i.e., there are 5 key points, and each face image is segmented into 5 image blocks including eyes, nose, eyebrows, mouth, and ears, then in this step S404, 5 autoencoders corresponding to the 5 key points are used to extract features from the m image blocks corresponding to each key point, thereby generating a total of 5m feature vectors for the 5 key points. When the size of each feature vector is set to n, the total data size of the feature vectors extracted from the m face images used as normal samples is 5nm.

[0067] If the dimension of n is too large, it will result in an excessively large feature dictionary. Therefore, in this embodiment, features are extracted using an autoencoder, which can represent the original image with a relatively small dimension. The size n of the feature vector can be determined by computer performance and accuracy requirements, and is not limited here.

[0068] When m is too large, the size of the feature dictionary will also be very large. In this case, in this embodiment, after extracting feature vectors from m image blocks for each key point, the clustering step shown in step S405 is further performed. If m is not large, clustering of the feature vectors may not be performed, and in this case, step S405 is skipped and the process proceeds directly to step S406. Whether or not step S405 is performed depends on the size of m.

[0069] In step S405, the feature vectors of m image patches targeting the same key point are clustered into k categories using the k-means clustering method. The feature vector of each category is represented by the center vector of that category, where k <m。

[0070] The center vector of each cluster represents the core of that cluster. By performing this step S405, the amount of data for the feature vectors extracted from the m face images used as normal samples is reduced from 5nm to 5nk. The closer k is to m, the clearer the category is, and the more accurate the adversarial example detection. Therefore, a suitable k can be selected for k-means clustering, taking into account memory consumption and detection accuracy. In other words, the size of k can be determined by computer performance and accuracy requirements.

[0071] Step S406: For each key point of the face in the face image, the feature vector generated in step S404 is saved as the feature dictionary of each key point, or the clustered feature vector in step S405 is saved as the feature dictionary of each key point.

[0072] In this embodiment, feature dictionaries for each key point of a normal sample are established through the above steps S401 to S406.

[0073] Figure 5 This illustrates a specific example of building a feature dictionary using normal samples.

[0074] In S5012, multiple face images are acquired as normal samples, and these face images are then aligned. S5012 corresponds to... Figure 4 Steps S401 and S402 in the process.

[0075] In S5031, multiple face images are segmented into multiple image blocks according to multiple key points, resulting in the image blocks shown in S5032. S5031 and S5032 correspond to... Figure 4 Step S403 in the process.

[0076] In S5041, feature extraction is performed on the segmented image patches for each keypoint using their respective autoencoders, resulting in multiple feature vectors for each keypoint, as shown in S5042. Here, S5041 and S5042 correspond to... Figure 4 Step S404 in the process.

[0077] In this embodiment, in S505, multiple feature vectors for each key point are clustered separately, and in S506, the clustered feature vectors for each key point are saved as a feature dictionary for each key point. S505 corresponds to... Figure 4 Steps S405 and S506 in the text correspond to Figure 4 Step S406 in the process.

[0078] The following is combined with Figure 6 Explain the judgment process for test samples.

[0079] In step S601, regarding the test sample, the face image of the test sample is obtained as the test image.

[0080] In step S602, the test image is aligned so that multiple key points of the face in the test image are aligned.

[0081] In step S603, the test image is segmented into multiple image blocks according to the multiple key points.

[0082] In step S604, features are extracted from each key point of the image patch of the test image to generate a feature vector.

[0083] In step S605, regarding the test sample, the distance between the feature vector of each key point of the test image and the feature vector in the feature dictionary of the corresponding key point is calculated, and the test sample is determined to be an adversarial sample by comparing the distance with a preset threshold.

[0084] More specifically, regarding the test sample, the average distance between the feature vector of each keypoint of the test image and the nearest L (where L is an integer greater than 1) feature vectors in the feature dictionary of the corresponding keypoint is calculated. When the average distance between the feature vector of at least one keypoint of the test image and the nearest L feature vectors in the feature dictionary of the corresponding keypoint is greater than the threshold corresponding to the corresponding keypoint, the test image is determined to be an adversarial example. Here, a threshold is preset corresponding to each keypoint of the multiple keypoints, and the threshold can be set empirically. In specific implementation, the average distance between the feature vector of the test image and the nearest L feature vectors in the feature dictionary can be calculated one by one for each keypoint of the multiple keypoints, and the average distance can be compared with the threshold of the corresponding keypoint. When the average distance of a certain keypoint exceeds the threshold of the corresponding keypoint, the test image can be determined to be an adversarial example, and the process ends.

[0085] Here, the distance between the feature vector of the test image and the feature vector in the feature dictionary can be Euclidean distance, but is not limited to this. Furthermore, L can be changed according to the actual situation; it can be 3, 5, or other values, without restriction.

[0086] The above steps S601 to S604 can be respectively adopted with... Figure 4 The same method is used in steps S401 to S404. The difference is that in step S601, only one test image needs to be selected for the test sample.

[0087] Figure 7 This is a diagram illustrating how distance comparison is used to determine whether a test sample is an adversarial sample. Figure 7 The ellipse in the diagram represents the feature dictionary of a normal sample for a specific keypoint. The solid pentagrams 701, 702, 703, etc., inside the ellipse represent the feature dictionary of the normal sample for that keypoint, respectively. The hollow pentagram 700 outside the ellipse represents the feature vector of the test sample for that keypoint. For example... Figure 7As shown, since the three feature vectors represented by the solid pentagrams 701, 702, and 703 in the feature dictionary are closest to the feature vector of the hollow pentagram 700 outside the ellipse, when judging the test sample, the average distance between the hollow pentagram 700 outside the ellipse and the feature vectors represented by the solid pentagrams 701, 702, and 703 in the feature dictionary is calculated. If the average distance is greater than the threshold of the key point, the test sample is judged to be an adversarial sample.

[0088] In this invention, the feature vectors of each key point of the test sample are processed one by one. Figure 7 The judgment described above is as follows: if the distance between the feature vectors of all keypoints and the feature vectors in the feature dictionary of the corresponding keypoint does not exceed the threshold of the corresponding keypoint, then the test sample is judged not to be an adversarial example. If the distance between the feature vector of at least one keypoint and the feature vector in the feature dictionary of the corresponding keypoint exceeds the threshold of the corresponding keypoint, then the test sample can be judged to be an adversarial example. In this case, once the distance between the feature vector of any keypoint and the feature vector in the feature dictionary of the corresponding keypoint exceeds the threshold of the corresponding keypoint, the test sample can be judged to be an adversarial example, so there is no need to continue judging the remaining keypoints.

[0089] This disclosure can also be achieved through a face adversarial example defense device. For example... Figure 8 As shown, the face adversarial example defense device includes: an acquisition unit 801, which acquires face images of normal samples or test samples; an alignment unit 802, which aligns the face images acquired by the acquisition unit so that multiple key points of the face in the face images are aligned; a segmentation unit 803, which segments the face images aligned by the alignment unit into multiple image blocks according to the multiple key points; a feature extraction unit 804, which extracts features for each key point of the face image image and generates feature vectors for each key point; a feature dictionary storage unit 806, which, for normal samples, stores the feature vectors of each key point of the face image of a normal sample as a feature dictionary for each key point; and an adversarial example detection unit 807, which, for test samples, calculates the distance between the feature vectors of each key point of the face image of a test sample and the feature vectors in the feature dictionary of the corresponding key point stored in the feature dictionary storage unit, and determines whether the test sample is an adversarial example by comparing the distance with a preset threshold. The feature extraction unit 804 can use an autoencoder corresponding to each key point of multiple key points to extract the features of the image blocks of the corresponding key points to generate the feature vector of the corresponding key points.

[0090] In addition, the acquisition unit 801 acquires multiple face images of normal samples and outputs them to the alignment unit 802 respectively. Considering that the large number of acquired face images leads to a large feature dictionary, the face adversarial example defense device may further include a clustering unit 805. The clustering unit 805 clusters multiple feature vectors of each key point extracted by the feature extraction unit 804 from the multiple face images, and outputs the clustered feature vectors of each key point to the feature dictionary storage unit 806. The feature dictionary storage unit 806 stores the clustered feature vectors of each key point of the face images of normal samples as the feature dictionary of each key point.

[0091] Figure 8 The specific implementation of each unit in the face adversarial example defense device shown can be found in the description of the corresponding steps in the face adversarial example defense method, and the specific description is omitted here.

[0092] In this invention, normal face images are used as a set of normal samples and input into the face adversarial sample defense device to construct a feature dictionary of normal samples. There is no need to prepare various adversarial samples in advance, so the generalization ability is strong.

[0093] Figure 8 The face adversarial example defense device shown is one implementation of this disclosure. This disclosure can also be implemented through... Figure 9 The computing device shown is implemented.

[0094] Computing device 900 is an example of a hardware device capable of applying the foregoing aspects of this disclosure. Computing device 900 can be any machine configured to perform processing and / or computation. Computing device 900 can be, but is not limited to, a workstation, server, desktop computer, laptop computer, tablet computer, personal data assistant (PDA), smartphone, in-vehicle computer, or a combination thereof.

[0095] like Figure 9As shown, computing device 900 may include one or more components that can be connected to or communicate with bus 902 via one or more interfaces. Bus 902 may include, but is not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus. Computing device 900 may include, for example, one or more processors 904, one or more input devices 906, and one or more output devices 908. The one or more processors 904 may be any type of processor and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). Processor 904 may, for example, be configured to implement Figure 3 , Figure 4 , Figure 5 , Figure 6 The steps shown can also be configured to implement Figure 8 The illustrated device functions as follows: Input device 906 can be any type of input device capable of inputting information to a computing device, and may include, but is not limited to, a mouse, keyboard, touchscreen, microphone, and / or remote controller. Output device 908 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer.

[0096] The computing device 900 may also include or be connected to a non-transitory storage device 914, which may be any non-transitory storage device capable of storing data, and may include, but is not limited to, disk drives, optical storage devices, solid-state storage, floppy disks, flexible disks, hard disks, magnetic tapes or any other magnetic media, compressed disks or any other optical media, cache memory and / or any other storage chip or module, and / or any other medium from which a computer may read data, instructions and / or code. The computing device 900 may also include random access memory (RAM) 910 and read-only memory (ROM) 912. ROM 912 may store executable programs, utilities, or processes in a non-volatile manner. RAM 910 provides volatile data storage and stores instructions related to the operation of the computing device 900. The computing device 900 may also include a network / bus interface 916 coupled to a data link 918. The network / bus interface 916 can be any kind of device or system capable of enabling communication with external devices and / or networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication devices and / or chipsets (such as Bluetooth). TMEquipment, 802.11 equipment, WiFi equipment, WiMax equipment, cellular communication facilities, etc.

[0097] This disclosure can be implemented as any combination of apparatus, system, integrated circuit, and computer program on a non-transitory computer-readable medium. One or more processors can be implemented as integrated circuits (ICs), application-specific integrated circuits (ASICs), or large-scale integrated circuits (LSIs), system LSIs, super LSIs, or ultra LSI components that perform some or all of the functions described in this disclosure.

[0098] This disclosure includes the use of software, application programs, computer programs, or algorithms. Software, application programs, computer programs, or algorithms may be stored on a non-transitory computer-readable medium to cause a computer, such as one or more processors, to perform the steps described above and in the accompanying drawings. For example, one or more memories may store the software or algorithm in executable instructions, and one or more processors may be associated with executing a set of instructions of the software or algorithm to provide various functionalities according to embodiments described in this disclosure.

[0099] Software and computer programs (also referred to as programs, software applications, applications, components, or code) include machine instructions for programmable processors and can be implemented in high-level procedural languages, object-oriented programming languages, functional programming languages, logic programming languages, assembly languages, or machine languages. The term "computer-readable medium" means any computer program product, apparatus, or device used to provide machine instructions or data to a programmable data processor, such as magnetic disks, optical disks, solid-state storage devices, memories, and programmable logic devices (PLDs), including computer-readable media that receive machine instructions as computer-readable signals.

[0100] For example, computer-readable media may include dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store required computer-readable program code in the form of instructions or data structures, and that can be accessed by a general-purpose or special-purpose computer or a general-purpose or special-purpose processor. As used herein, a disk or disc includes compact discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks, and Blu-ray discs, wherein a disk typically copies data magnetically, while a disc copies data optically using a laser. Combinations of the above are also included within the scope of computer-readable media.

[0101] The subject matter of this disclosure is provided as examples of apparatus, systems, methods, and procedures for performing the features described herein. However, other features or variations are contemplated in addition to those described above. It is anticipated that components and functions of this disclosure may be implemented using any emerging techniques that may replace any of the above-described implementations.

[0102] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to market technology of the embodiments, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for defending against adversarial examples in face recognition, comprising: For normal samples, a face image of a normal sample is obtained as a sample image. The sample image is aligned so that multiple key points of the face in the sample image are aligned. The sample image is divided into multiple image blocks according to the multiple key points. For each key point of the image block of the sample image, features are extracted using an autoencoder corresponding to each key point to generate a feature vector. The feature vectors of each key point of the sample image are saved as the feature dictionary of each key point. Regarding the test sample, a face image of the test sample is obtained as the test image. The test image is aligned so that multiple key points of the face in the test image are aligned. The test image is divided into multiple image blocks according to the multiple key points. For each key point of the test image image block, the features are extracted using the autoencoder corresponding to each key point to generate a feature vector. as well as Regarding the test sample, the distance between the feature vector of each key point of the test image and the feature vector in the feature dictionary of the corresponding key point is calculated. The test sample is then compared with a threshold to determine whether it is an adversarial sample. This includes: if the average distance between the feature vector of at least one key point of the test image and the L nearest feature vectors in the feature dictionary of the corresponding key point is greater than the threshold corresponding to the corresponding key point, the test image is determined to be an adversarial sample; if the average distance between the feature vectors of all key points of the test image and the L nearest feature vectors in the feature dictionary of the corresponding key point is less than or equal to the threshold corresponding to the corresponding key point, the test image is determined not to be an adversarial sample, where L is an integer greater than 1.

2. The face adversarial example defense method according to claim 1, wherein, As multiple key points, you can choose facial features, or 29 key points calibrated by the LFPW face database, or 21 key points calibrated by the AFLW face database, or multiple custom key points.

3. The face adversarial example defense method according to claim 1, wherein, For normal samples, obtain m face images as m sample images, where m is an integer greater than 1. Align the m sample images respectively, and segment the m sample images into multiple image blocks according to the multiple key points. For each key point of the m sample images, extract features to generate m feature vectors. Save the m feature vectors of each key point of the m sample images as the feature dictionary of each key point.

4. The face adversarial example defense method according to claim 3, wherein, For normal samples, the m feature vectors of each keypoint in the m sample images are further clustered into k classes using the k-means clustering method to obtain k feature vectors. These k clustered feature vectors of each keypoint in the m sample images are then saved as the feature dictionary for each keypoint, where 1 ≤ k. <m。 5. A facial adversarial sample defense device, comprising: The acquisition unit acquires face images of normal or test samples. An alignment unit aligns the face image acquired by the acquisition unit so that multiple key points of the face in the face image are aligned. The segmentation unit divides the face image, which has been aligned by the alignment unit, into multiple image blocks according to the multiple key points. The feature extraction unit extracts features from image patches of a face image for each key point using an autoencoder corresponding to each key point, generating feature vectors for each key point; and The feature dictionary storage unit, for normal samples, saves the feature vectors of each key point in the face image of a normal sample as the feature dictionary for each key point. The adversarial example detection unit, regarding the test sample, calculates the distance between the feature vectors of each key point of the face image of the test sample (which serves as the test image) and the feature vectors in the feature dictionary of the corresponding key point stored in the feature dictionary storage unit. It then compares this distance with a threshold to determine whether the test sample is an adversarial example. This includes: determining that the test image is an adversarial example when the average distance between the feature vector of at least one key point of the test image and the L nearest feature vectors in the feature dictionary of the corresponding key point is greater than the threshold corresponding to the corresponding key point; and determining that the test image is not an adversarial example when the average distance between the feature vectors of all key points of the test image and the L nearest feature vectors in the feature dictionary of the corresponding key point is less than or equal to the threshold corresponding to the corresponding key point, where L is an integer greater than 1.

6. The face adversarial sample defense device according to claim 5, wherein, The acquisition unit acquires multiple face images based on normal samples and outputs them to the alignment unit. The face adversarial sample defense device also includes a clustering unit. The clustering unit clusters multiple feature vectors of each key point extracted by the feature extraction unit from multiple face images, and outputs the clustered feature vectors of each key point to the feature dictionary storage unit. The feature dictionary storage unit saves the clustered feature vectors of each key point of a normal sample face image as a feature dictionary for each key point.

7. A facial adversarial sample defense device, comprising: A memory that stores instructions; as well as The processor is configured to execute instructions stored in the memory to perform the method according to any one of claims 1 to 4.

8. A computer-readable storage medium comprising computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 4.

Images