Methods, devices, readable storage media and electronic devices for facial recognition anomaly detection

Abstract

This invention relates to a method, apparatus, readable storage medium, and electronic device for detecting anomalies in face recognition. It can determine the comprehensive attention features of a captured face image and the comprehensive attention features of a face image without interference features, and based on the difference in the distribution of activation values ​​of these two comprehensive attention features in the face recognition model, determine whether the captured face image is an image maliciously constructed by an attacker, thereby detecting the occurrence of anomalies caused by an attacker and improving the security of the face recognition system.

Description

Technical Field

[0001] This invention relates to the field of information technology security technology, and in particular to a method, apparatus, readable storage medium, and electronic device for detecting anomalies in facial recognition. Background Technology

[0002] With the development of deep learning technology, facial recognition technology has been widely used in many fields. However, while bringing many conveniences, this technology also faces many security risks. Among them, the existence of adversarial examples allows attackers to successfully impersonate others, leading to privacy leaks, financial security threats, and other problems. Existing research shows that attackers can forge the target's feature information by wearing carefully crafted glasses or affixing distracting stickers to the face, thus successfully launching an attack. Therefore, defense against such adversarial attacks is crucial in facial recognition applications.

[0003] The inventors of this application have discovered that attackers may maliciously wear elaborately crafted glasses, affix disturbing stickers or tattoos to their faces to achieve their adversarial attack objectives, thereby interfering with normal facial recognition results and causing them to be identified as the attacker's desired outcome.

[0004] The above situation may lead to a successful malicious attack on the facial recognition system, allowing attackers to impersonate others and bypass the detection system. Summary of the Invention

[0005] This invention provides a method, apparatus, readable storage medium, and electronic device for detecting anomalies in face recognition, so as to detect the security of face recognition systems during recognition and effectively defend against adversarial attacks by malicious attackers.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for detecting anomalies in facial recognition includes:

[0008] Obtain the face recognition result of the first face image captured by the image acquisition device;

[0009] A second face image matching the face recognition result is obtained from a preset face image library, wherein all face images in the preset face image library are face images without interfering features;

[0010] Obtain the channel attention weight coefficients and spatial attention weight coefficients of the first face image, and based on the channel attention weight coefficients and spatial attention weight coefficients of the first face image, obtain the comprehensive attention features of the first face image;

[0011] The channel attention weight coefficients and spatial attention weight coefficients of the second face image are obtained. Based on the channel attention weight coefficients and spatial attention weight coefficients of the second face image, the comprehensive attention features of the second face image are obtained.

[0012] The distribution differences of the activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model are determined, and based on the distribution differences, it is determined whether the first face image is an image maliciously constructed by an attacker.

[0013] Optionally, the process of obtaining the channel attention weight coefficients of the first face image includes:

[0014] Extract the first image features from the first face image; perform max pooling on the first image features to obtain the first weight vector coefficients; perform average pooling on the first image features to obtain the second weight vector coefficients; perform convolution on the first weight vector coefficients to obtain the third weight vector coefficients; perform convolution on the second weight vector coefficients to obtain the fourth weight vector coefficients; add the third weight vector coefficients and the fourth weight vector coefficients to obtain the fifth weight vector coefficients; process the fifth weight vector coefficients using an activation function to obtain the channel attention weight coefficients of the first face image.

[0015] And / or,

[0016] The process of obtaining the channel attention weight coefficients of the second face image includes:

[0017] Extract the second image features from the second face image; perform max pooling on the second image features to obtain the sixth weight vector coefficient; perform average pooling on the second image features to obtain the seventh weight vector coefficient; perform convolution on the sixth weight vector coefficient to obtain the eighth weight vector coefficient; perform convolution on the seventh weight vector coefficient to obtain the ninth weight vector coefficient; add the eighth weight vector coefficient and the ninth weight vector coefficient to obtain the tenth weight vector coefficient; process the tenth weight vector coefficient using an activation function to obtain the channel attention weight coefficients of the second face image.

[0018] Optionally, the process of obtaining the spatial attention weight coefficients of the first face image includes:

[0019] Obtain the third image feature of the first face image; perform max pooling on the third image feature to obtain the eleventh weight vector coefficient; perform average pooling on the third image feature to obtain the twelfth weight vector coefficient; stack the eleventh weight vector coefficient and the twelfth weight vector coefficient to obtain the thirteenth weight vector coefficient; perform convolution on the thirteenth weight vector coefficient to obtain the fourteenth weight vector coefficient; and process the fourteenth weight vector coefficient using an activation function to obtain the spatial attention weight coefficient of the first face image.

[0020] And / or,

[0021] The process of obtaining the spatial attention weight coefficients of the second face image includes:

[0022] Obtain the fourth image feature of the second face image; perform max pooling on the fourth image feature to obtain the fifteenth weight vector coefficient; perform average pooling on the fourth image feature to obtain the sixteenth weight vector coefficient; stack the fifteenth and sixteenth weight vector coefficients to obtain the seventeenth weight vector coefficient; perform convolution on the seventeenth weight vector coefficient to obtain the eighteenth weight vector coefficient; and process the eighteenth weight vector coefficient using an activation function to obtain the spatial attention weight coefficient of the second face image.

[0023] Optionally, the third image feature is the image feature obtained by multiplying the channel attention weight coefficient of the first face image with the first image feature of the first face image;

[0024] And / or,

[0025] The fourth image feature is the image feature obtained by multiplying the channel attention weight coefficient of the second face image with the second image feature of the second face image.

[0026] Optionally, obtaining the comprehensive attention features of the first face image based on the channel attention weight coefficients and spatial attention weight coefficients of the first face image includes:

[0027] The result of multiplying the first image feature of the first face image, the channel attention weight coefficient of the first face image, and the spatial attention weight coefficient of the first face image is determined as the comprehensive attention feature of the first face image.

[0028] And / or,

[0029] The comprehensive attention features of the second face image are obtained based on the channel attention weight coefficients and spatial attention weight coefficients of the second face image, including:

[0030] The comprehensive attention feature of the second face image is determined by multiplying the second image feature of the second face image, the channel attention weight coefficient of the second face image, and the spatial attention weight coefficient of the second face image.

[0031] Optionally, determining the distribution difference of activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model includes:

[0032] The comprehensive attention features of the first face image are input into the face recognition model to obtain the first distribution of activation values ​​in the last convolutional layer of the face recognition model;

[0033] The comprehensive attention features of the second face image are input into the face recognition model to obtain the second distribution of activation values ​​in the last convolutional layer of the face recognition model;

[0034] Obtain the distribution difference between the first distribution and the second distribution.

[0035] Optionally, determining whether the first face image is a maliciously constructed image by an attacker based on the distribution difference includes:

[0036] If the distribution difference is greater than a preset threshold, the first face image is determined to be an image maliciously constructed by an attacker; otherwise, the first face image is determined to be a normal image that has not been maliciously constructed by an attacker.

[0037] A facial recognition anomaly detection device includes: a result acquisition unit, an image acquisition unit, a first feature acquisition unit, a second feature acquisition unit, and a difference determination unit.

[0038] The result acquisition unit is used to obtain the face recognition result of the first face image acquired by the image acquisition device;

[0039] The image acquisition unit is used to obtain a second face image that matches the face recognition result from a preset face image library, wherein the face images in the preset face image library are all face images without interfering features;

[0040] The first feature acquisition unit is used to obtain the channel attention weight coefficient and spatial attention weight coefficient of the first face image, and to obtain the comprehensive attention feature of the first face image based on the channel attention weight coefficient and spatial attention weight coefficient of the first face image.

[0041] The second feature acquisition unit is used to obtain the channel attention weight coefficient and spatial attention weight coefficient of the second face image, and to obtain the comprehensive attention feature of the second face image based on the channel attention weight coefficient and spatial attention weight coefficient of the second face image.

[0042] The difference determination unit is used to determine the distribution difference of the activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model, and to determine whether the first face image is an image maliciously constructed by an attacker based on the distribution difference.

[0043] A storage medium having a program stored thereon, wherein the program, when executed by a processor, implements any of the above-described methods for detecting facial recognition anomalies.

[0044] An electronic device includes at least one processor, at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute any of the above-mentioned face recognition anomaly detection methods.

[0045] As can be seen from the above technical solutions, the present invention discloses a face recognition anomaly detection method, device, readable storage medium, and electronic device. It can determine the comprehensive attention features of the acquired face image and the comprehensive attention features of the face image without interference features, and based on the difference in the distribution of activation values ​​of these two comprehensive attention features in the face recognition model, determine whether the acquired face image is an image constructed by an attacker, thereby detecting the occurrence of anomalies caused by the attacker and improving the security of the face recognition system. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of a face recognition anomaly detection method provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the structure of a face recognition anomaly detection device provided in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] like Figure 1 As shown, an embodiment of the present invention provides a face recognition anomaly detection method, which may include:

[0052] S100: Obtain the face recognition result of the first face image acquired by the image acquisition device.

[0053] Among them, the image acquisition device can be an electronic device with image acquisition function, such as a computer, mobile phone, wearable device, access control device, and standalone camera.

[0054] The first face image could be a normal image or an image constructed by an attacker.

[0055] The aforementioned face recognition result can be obtained by recognizing the first face image using a face recognition model. This face recognition model can be any existing face recognition model, such as PaddleDetection, DeepFace, InsightFace, OpenFace, Eigen Face, Face++, etc., and this application does not impose any limitations on it. This application also does not limit the model structure or type of the face recognition model.

[0056] The face recognition result described above can indicate which person's face is in the first face image. For example, the face recognition result could indicate that the face in the first face image is Zhang San's face. The face recognition result can include the person's identifier, such as their name or ID number.

[0057] S200. Obtain a second face image that matches the face recognition result from a preset face image library, wherein all face images in the preset face image library are face images without interfering features.

[0058] The preset face image library contains pre-collected, compliant face images that do not contain interfering features. Interfering features are image features that can interfere with face recognition results, such as perturbation stickers, tattoos, and glasses that can interfere with face recognition. Images containing interfering features can be called interfering images, which are maliciously constructed by attackers and can be understood as images after being attacked.

[0059] The aforementioned pre-set facial image library can store facial images of multiple individuals. For example, a company can pre-collect facial images of its employees before activating its access control system and store these images along with corresponding personal identifiers, thus establishing a pre-set facial image library. After obtaining the facial recognition result, the system can then use the personal identifier in the result to find the facial image in the pre-set library that corresponds to that identifier; this matching facial image is the second facial image that matches the facial recognition result.

[0060] S300. Obtain the channel attention weight coefficients and spatial attention weight coefficients of the first face image. Based on the channel attention weight coefficients and spatial attention weight coefficients of the first face image, obtain the comprehensive attention features of the first face image.

[0061] To fully extract the features of facial images, an attention mechanism is introduced in this invention to refine the processing of input image features and improve the accuracy of extracting salient features of facial images.

[0062] like Figure 1 As shown, step S300 only needs to be executed before step S500. The execution order of step S300, S100, S200, and S400 is not limited in this application.

[0063] The channel attention weight coefficient refers to the weight assigned to features from different channels during feature selection. A larger channel attention weight coefficient indicates greater importance of the channel feature for tasks such as classification or detection; therefore, the corresponding feature will be given a higher weight when calculating classification or detection scores. In face recognition tasks, features from different channels may contain different information, such as facial features like eyes, mouth, and nose. By using the channel attention weight coefficient, these features can be better selected and weighted, thereby improving the accuracy and performance of face recognition tasks.

[0064] Spatial attention weight coefficients refer to the weights assigned to features at different spatial locations during spatial feature selection. A larger spatial attention weight coefficient indicates that the features at that spatial location are more important to tasks such as classification or detection, and these features will be given higher weights when calculating classification or detection scores. In face recognition tasks, features at different spatial locations may contain different information, such as facial expressions, eye positions, and lip movements. By using spatial attention weight coefficients, the model can better select and weight these features, thereby improving the accuracy and performance of face recognition tasks.

[0065] This application can first use conventional face feature extraction methods (such as histogram of orientation (HOG) feature extraction, Dlib feature extraction, convolutional neural network feature extraction) to extract features from the first face image, and then multiply the extracted image features with the channel attention weight coefficient and the spatial attention weight coefficient to obtain the comprehensive attention features of the first face image.

[0066] S400. Obtain the channel attention weight coefficients and spatial attention weight coefficients of the second face image. Based on the channel attention weight coefficients and spatial attention weight coefficients of the second face image, obtain the comprehensive attention features of the second face image.

[0067] Optionally, steps S300 and S400 can be performed by a convolutional attention module (CBAM). Because CBAM is "plug-and-play," it can be added to the first layer of the face recognition model to implement the attention mechanism, thereby providing more refined feature processing without increasing overhead. Of course, in other embodiments, CBAM can also operate independently of the face recognition model.

[0068] The process of obtaining the comprehensive attention features of the second face image in step S400 is similar to the process of obtaining the comprehensive attention features of the first face image in step S300, and will not be described again.

[0069] S500: Determine the distribution difference of the activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model, and determine whether the first face image is an image maliciously constructed by the attacker based on the distribution difference.

[0070] The distribution of activation values ​​in a face recognition model is calculated according to a specific activation function, such as sigmoid and ReLU.

[0071] The distribution of activation values ​​in a face recognition model represents the output intensity of each neuron in the neural network. Differences in the distribution of activation values ​​in a face recognition model can characterize the differences in various factors used to determine the recognition result when the face recognition model recognizes two different images. This application can determine whether the first face image is an image maliciously constructed by an attacker based on these differences.

[0072] The face recognition anomaly detection method provided in this invention can separately determine the comprehensive attention features of the acquired face image and the comprehensive attention features of the face image without interference features. Based on the difference in the distribution of activation values ​​of these two comprehensive attention features in the face recognition model, it determines whether the acquired face image is an image maliciously constructed by an attacker, thereby detecting anomalies caused by attacks and improving the security of the face recognition system. Because an attention mechanism is introduced, the comprehensive attention features extracted in this application are more effective and feature redundancy is reduced.

[0073] Optional, Figure 1 The process of obtaining the channel attention weight coefficients of the first face image in the method shown may include:

[0074] Extract the first image features from the first face image; perform max pooling on the first image features to obtain the first weight vector coefficients; perform average pooling on the first image features to obtain the second weight vector coefficients; perform convolution on the first weight vector coefficients to obtain the third weight vector coefficients; perform convolution on the second weight vector coefficients to obtain the fourth weight vector coefficients; add the third and fourth weight vector coefficients to obtain the fifth weight vector coefficients; process the fifth weight vector coefficients using an activation function to obtain the channel attention weight coefficients of the first face image.

[0075] Optionally, the convolution performed on the coefficients of the first weight vector can be a one-dimensional convolution. In this way, the present application can use a one-dimensional convolution in the channel attention part to replace the original multilayer perceptron (MLP), thereby reducing costs.

[0076] The coefficients of the first weight vector are processed by one-dimensional convolution and mapped to the weights of each channel. One-dimensional convolution has the function of non-fully connected, and only operates on a portion of the channels in each convolution process, realizing cross-channel interaction.

[0077] The first image feature is the feature extracted using conventional face feature extraction methods.

[0078] Max pooling is a local pooling method used to reduce the complexity of neural networks and decrease training time. The operation of max pooling is as follows:

[0079] Extract a local receptive field from the input data, typically a square or rectangle; within the local receptive field, take the pixel with the largest value as the output of the max pooling layer; repeat this process to cover the entire input data.

[0080] Average pooling divides an image into a fixed-size grid, and the pixel value within each grid is the average of all pixels within that grid. Average pooling is a typical pooling operation in convolutional neural networks and can effectively reduce the dimensionality of an image.

[0081] The fifth weight vector coefficient is obtained by adding the third and fourth weight vector coefficients. Therefore, the fifth weight vector coefficient combines the weight vector coefficients obtained by max pooling and the weight vector coefficients obtained by average pooling.

[0082] The activation function in this application can be Sigmoid.

[0083] Optional, Figure 1 The process of obtaining the channel attention weight coefficients of the second face image in the method shown may include:

[0084] Extract the second image features from the second face image; perform max pooling on the second image features to obtain the sixth weight vector coefficient; perform average pooling on the second image features to obtain the seventh weight vector coefficient; perform convolution on the sixth weight vector coefficient to obtain the eighth weight vector coefficient; perform convolution on the seventh weight vector coefficient to obtain the ninth weight vector coefficient; add the eighth and ninth weight vector coefficients to obtain the tenth weight vector coefficient; process the tenth weight vector coefficient using an activation function to obtain the channel attention weight coefficients of the second face image.

[0085] The process of obtaining the channel attention weight coefficients of the second face image is similar to that of obtaining the channel attention weight coefficients of the first face image, and will not be described in detail here.

[0086] Optional, Figure 1 The method shown includes the following steps to obtain the spatial attention weight coefficients of the first face image:

[0087] Obtain the third image feature of the first face image; perform max pooling on the third image feature to obtain the eleventh weight vector coefficient; perform average pooling on the third image feature to obtain the twelfth weight vector coefficient; stack the eleventh and twelfth weight vector coefficients to obtain the thirteenth weight vector coefficient; perform convolution on the thirteenth weight vector coefficient to obtain the fourteenth weight vector coefficient; process the fourteenth weight vector coefficient using an activation function to obtain the spatial attention weight coefficient of the first face image.

[0088] The process of performing max pooling and average pooling on the third image features involves pooling all channels of the same feature point.

[0089] The above convolution processing of the thirteenth weight vector coefficients can be specifically described as follows: convolution the thirteenth weight vector coefficients with a 7×7 convolutional layer.

[0090] Optionally, the third image feature is the image feature obtained by multiplying the channel attention weight coefficients of the first face image with the first image feature of the first face image. In this way, the third image feature is influenced by the channel attention weight coefficients, which can improve the accuracy and performance of the face recognition task.

[0091] The thirteenth weight vector coefficient is obtained by stacking the eleventh and twelfth weight vector coefficients, which shows that the thirteenth weight vector coefficient integrates the functions of max pooling and average pooling.

[0092] Optional, Figure 1 The method shown includes the following steps to obtain the spatial attention weight coefficients of the second face image:

[0093] Obtain the fourth image feature of the second face image; perform max pooling on the fourth image feature to obtain the fifteenth weight vector coefficient; perform average pooling on the fourth image feature to obtain the sixteenth weight vector coefficient; stack the fifteenth and sixteenth weight vector coefficients to obtain the seventeenth weight vector coefficient; perform convolution on the seventeenth weight vector coefficient to obtain the eighteenth weight vector coefficient; process the eighteenth weight vector coefficient using an activation function to obtain the spatial attention weight coefficients of the second face image.

[0094] The process of obtaining the spatial attention weight coefficients of the second face image is similar to the process of obtaining the spatial attention weight coefficients of the first face image, and will not be described in detail here.

[0095] Optionally, the fourth image feature is the image feature obtained by multiplying the channel attention weight coefficient of the second face image with the second image feature of the second face image.

[0096] Optionally, the comprehensive attention features of the first face image can be obtained based on the channel attention weight coefficients and spatial attention weight coefficients of the first face image, which may include:

[0097] The comprehensive attention feature of the first face image is determined by multiplying the first image feature of the first face image, the channel attention weight coefficient of the first face image, and the spatial attention weight coefficient of the first face image.

[0098] The combination of spatial attention and channel attention compensates for the shortcomings of channel attention to a certain extent, enabling the acquisition of more detailed feature information. This application utilizes both channel and spatial dimensions, employing two independent attention mechanisms to obtain weight coefficients for image features, achieving adaptive refinement of image features without significantly increasing computational load. This application achieves precise feature extraction, removing redundant information while preserving salient information, effectively improving the accuracy of the difference judgment in step S500.

[0099] Optionally, the comprehensive attention features of the second face image can be obtained based on the channel attention weight coefficients and spatial attention weight coefficients of the second face image, which may include:

[0100] The comprehensive attention feature of the second face image is determined by multiplying the second image features of the second face image, the channel attention weight coefficient of the second face image, and the spatial attention weight coefficient of the second face image.

[0101] Optional, Figure 1 The process of determining the distribution difference of activation values ​​of the integrated attention features of the first face image and the integrated attention features of the second face image in the face recognition model in step S500 may include:

[0102] The comprehensive attention features of the first face image are input into the face recognition model to obtain the first distribution of activation values ​​in the last convolutional layer of the face recognition model;

[0103] The comprehensive attention features of the second face image are input into the face recognition model to obtain the second distribution of activation values ​​in the last convolutional layer of the face recognition model;

[0104] Obtain the distribution difference between the first distribution and the second distribution.

[0105] Attackers can disguise themselves using methods such as adversarial stickers and glasses, resulting in abnormal activation value distributions in the face recognition model when their facial images are processed. Therefore, this invention detects anomalies in face recognition by comparing the activation distribution of a captured facial image with that of a face without interfering features in the last convolutional layer of the model. The Pearson correlation coefficient can be used to measure the difference in activation distribution.

[0106] Let X be the captured face image. in The distribution in the last convolutional layer is f. in Face image X without interfering features ori The distribution in the last convolutional layer is f. ori f in with f ori Differences Expressed using the following formula:

[0107]

[0108] Where E represents the expected value, μ in and μ ori σ represents the average value of each neuron in the last convolutional layer. in and σ ori denoted as standard deviation.

[0109] Optionally, determining whether the first face image is a maliciously constructed image by an attacker based on the distribution difference includes:

[0110] If the distribution difference is greater than a preset threshold, the first face image is determined to be an image maliciously constructed by an attacker; otherwise, the first face image is determined to be a normal image that has not been maliciously constructed by an attacker.

[0111] Specifically, this application can set a certain threshold ε, if If the value is not greater than ε, it indicates that the facial recognition system is secure, the captured image for recognition has not been compromised, and subsequent identity verification can proceed smoothly; if... If the value is greater than ε, it indicates that the image collected for recognition has been compromised by an attacker, the facial recognition system is under threat, and the captured facial image may be an adversarial sample, with an illegal attacker attempting to deceive the facial recognition model to achieve illegal purposes. In this case, this application can transmit the information that the first facial image is a maliciously constructed image by the attacker to the backend, interrupt the facial recognition operation, and issue a warning.

[0112] The advantages of this invention lie in its ability to address the problems of insufficient feature extraction for adversarial perturbations and high defense costs in previous defense methods. Leveraging the "plug-and-play" nature of CBAM, it improves feature extraction accuracy without incurring additional costs. Furthermore, the activation distribution difference judgment effectively addresses situations where different inputs result in different distributions within the model, improving the accuracy of anomaly detection in face recognition. In addition, this detection method possesses good compatibility and can be integrated with any defense method to enhance the model's defense capabilities against adversarial attacks.

[0113] Corresponding to the above method embodiments, this invention also provides a face recognition anomaly detection device, such as... Figure 2 As shown, the device may include: a result acquisition unit 100, an image acquisition unit 200, a first feature acquisition unit 300, a second feature acquisition unit 400, and a difference determination unit 500.

[0114] The result acquisition unit 100 is used to obtain the face recognition result of the first face image acquired by the image acquisition device;

[0115] The image acquisition unit 200 is used to obtain a second face image that matches the face recognition result from a preset face image library, wherein the face images in the preset face image library are all face images without interference features;

[0116] The first feature acquisition unit 300 is used to obtain the channel attention weight coefficient and spatial attention weight coefficient of the first face image, and to obtain the comprehensive attention features of the first face image based on the channel attention weight coefficient and spatial attention weight coefficient of the first face image.

[0117] The second feature acquisition unit 400 is used to obtain the channel attention weight coefficient and spatial attention weight coefficient of the second face image, and to obtain the comprehensive attention features of the second face image based on the channel attention weight coefficient and spatial attention weight coefficient of the second face image.

[0118] The difference determination unit 500 is used to determine the distribution difference of the activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model, and to determine whether the first face image is an image maliciously constructed by an attacker based on the distribution difference.

[0119] Optionally, the first feature acquisition unit 300 acquires the channel attention weight coefficients of the first face image, specifically set as follows:

[0120] Extract the first image features from the first face image; perform max pooling on the first image features to obtain the first weight vector coefficients; perform average pooling on the first image features to obtain the second weight vector coefficients; perform convolution on the first weight vector coefficients to obtain the third weight vector coefficients; perform convolution on the second weight vector coefficients to obtain the fourth weight vector coefficients; add the third and fourth weight vector coefficients to obtain the fifth weight vector coefficients; process the fifth weight vector coefficients using an activation function to obtain the channel attention weight coefficients of the first face image.

[0121] And / or,

[0122] The second feature acquisition unit 400 obtains the channel attention weight coefficients of the second face image, specifically set as follows:

[0123] Extract the second image features from the second face image; perform max pooling on the second image features to obtain the sixth weight vector coefficient; perform average pooling on the second image features to obtain the seventh weight vector coefficient; perform convolution on the sixth weight vector coefficient to obtain the eighth weight vector coefficient; perform convolution on the seventh weight vector coefficient to obtain the ninth weight vector coefficient; add the eighth and ninth weight vector coefficients to obtain the tenth weight vector coefficient; process the tenth weight vector coefficient using an activation function to obtain the channel attention weight coefficients of the second face image.

[0124] Optionally, the first feature acquisition unit 300 acquires the spatial attention weight coefficients of the first face image, specifically set as follows:

[0125] Obtain the third image feature of the first face image; perform max pooling on the third image feature to obtain the eleventh weight vector coefficient; perform average pooling on the third image feature to obtain the twelfth weight vector coefficient; stack the eleventh and twelfth weight vector coefficients to obtain the thirteenth weight vector coefficient; perform convolution on the thirteenth weight vector coefficient to obtain the fourteenth weight vector coefficient; and process the fourteenth weight vector coefficient using an activation function to obtain the spatial attention weight coefficient of the first face image.

[0126] And / or,

[0127] The second feature acquisition unit 400 obtains the spatial attention weight coefficients of the second face image, specifically set as follows:

[0128] Obtain the fourth image feature of the second face image; perform max pooling on the fourth image feature to obtain the fifteenth weight vector coefficient; perform average pooling on the fourth image feature to obtain the sixteenth weight vector coefficient; stack the fifteenth and sixteenth weight vector coefficients to obtain the seventeenth weight vector coefficient; perform convolution on the seventeenth weight vector coefficient to obtain the eighteenth weight vector coefficient; process the eighteenth weight vector coefficient using an activation function to obtain the spatial attention weight coefficients of the second face image.

[0129] Optionally, the third image feature is the image feature obtained by multiplying the channel attention weight coefficient of the first face image with the first image feature of the first face image;

[0130] And / or,

[0131] The fourth image feature is the image feature obtained by multiplying the channel attention weight coefficient of the second face image with the second image feature of the second face image.

[0132] Optionally, the first feature acquisition unit 300 obtains the comprehensive attention features of the first face image based on the channel attention weight coefficients and spatial attention weight coefficients of the first face image, specifically set as follows:

[0133] The result of multiplying the first image features of the first face image, the channel attention weight coefficient of the first face image, and the spatial attention weight coefficient of the first face image is determined as the comprehensive attention feature of the first face image.

[0134] And / or,

[0135] The second feature acquisition unit 400 obtains the comprehensive attention features of the second face image based on the channel attention weight coefficients and spatial attention weight coefficients of the second face image, specifically set as follows:

[0136] The comprehensive attention feature of the second face image is determined by multiplying the second image features of the second face image, the channel attention weight coefficient of the second face image, and the spatial attention weight coefficient of the second face image.

[0137] Optionally, the difference determination unit 500 determines the distribution difference of the activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model, specifically set as follows:

[0138] The comprehensive attention features of the first face image are input into the face recognition model to obtain the first distribution of activation values ​​in the last convolutional layer of the face recognition model; the comprehensive attention features of the second face image are input into the face recognition model to obtain the second distribution of activation values ​​in the last convolutional layer of the face recognition model; the distribution difference between the first distribution and the second distribution is obtained.

[0139] Optionally, the difference determination unit 500 determines whether the first face image is an image maliciously constructed by an attacker based on the distribution difference, specifically configured as follows:

[0140] If the distribution difference is greater than a preset threshold, the first face image is determined to be an image maliciously constructed by an attacker; otherwise, the first face image is determined to be a normal image that has not been constructed by an attacker.

[0141] The face recognition anomaly detection device includes a processor and a memory. The aforementioned result acquisition unit 100, image acquisition unit 200, first feature acquisition unit 300, second feature acquisition unit 400, and difference determination unit 500 are all stored as program units in the memory. The processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0142] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and kernel parameters can be adjusted to detect maliciously crafted images by attackers.

[0143] This invention provides a readable storage medium storing a program that, when executed by a processor, implements a face recognition anomaly detection method.

[0144] This invention provides a processor for running a program, wherein the program executes a face recognition anomaly detection method during runtime.

[0145] like Figure 3 As shown, this embodiment of the invention provides an electronic device 70, which includes at least one processor 701, at least one memory 702 connected to the processor 701, and a bus 703. The processor 701 and the memory 702 communicate with each other via the bus 703. The processor 701 is used to call program instructions in the memory 702 to execute the aforementioned face recognition anomaly detection method. The electronic device in this document can be a server, PC, PAD, mobile phone, etc.

[0146] This application also provides a computer program product that, when executed on a data processing device, is adapted to execute a program that initializes the steps of the above-described face recognition anomaly detection method.

[0147] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0148] In a typical configuration, the device includes one or more processors (CPUs), memory, and a bus. The device may also include input / output interfaces, network interfaces, etc.

[0149] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM, and memory includes at least one memory chip. Memory is an example of computer-readable media.

[0150] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0151] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0152] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting anomalies in face recognition, characterized in that, include: Obtain the face recognition result of the first face image captured by the image acquisition device; A second face image matching the face recognition result is obtained from a preset face image library, wherein all face images in the preset face image library are face images without interfering features; Obtain the channel attention weight coefficients and spatial attention weight coefficients of the first face image, and based on the channel attention weight coefficients and spatial attention weight coefficients of the first face image, obtain the comprehensive attention features of the first face image; The channel attention weight coefficients and spatial attention weight coefficients of the second face image are obtained. Based on the channel attention weight coefficients and spatial attention weight coefficients of the second face image, the comprehensive attention features of the second face image are obtained. The distribution differences of the activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model are determined, and based on the distribution differences, it is determined whether the first face image is an image maliciously constructed by an attacker.

2. The method according to claim 1, characterized in that, The process of obtaining the channel attention weight coefficients of the first face image includes: Extract the first image features from the first face image; perform max pooling on the first image features to obtain the first weight vector coefficients; perform average pooling on the first image features to obtain the second weight vector coefficients; perform convolution on the first weight vector coefficients to obtain the third weight vector coefficients; perform convolution on the second weight vector coefficients to obtain the fourth weight vector coefficients; add the third weight vector coefficients and the fourth weight vector coefficients to obtain the fifth weight vector coefficients; process the fifth weight vector coefficients using an activation function to obtain the channel attention weight coefficients of the first face image. And / or, The process of obtaining the channel attention weight coefficients of the second face image includes: Extract the second image features from the second face image; perform max pooling on the second image features to obtain the sixth weight vector coefficient; perform average pooling on the second image features to obtain the seventh weight vector coefficient; perform convolution on the sixth weight vector coefficient to obtain the eighth weight vector coefficient; perform convolution on the seventh weight vector coefficient to obtain the ninth weight vector coefficient; add the eighth weight vector coefficient and the ninth weight vector coefficient to obtain the tenth weight vector coefficient; process the tenth weight vector coefficient using an activation function to obtain the channel attention weight coefficients of the second face image.

3. The method according to claim 2, characterized in that, The process of obtaining the spatial attention weight coefficients of the first face image includes: Obtain the third image feature of the first face image; perform max pooling on the third image feature to obtain the eleventh weight vector coefficient; perform average pooling on the third image feature to obtain the twelfth weight vector coefficient; stack the eleventh weight vector coefficient and the twelfth weight vector coefficient to obtain the thirteenth weight vector coefficient; perform convolution on the thirteenth weight vector coefficient to obtain the fourteenth weight vector coefficient; and process the fourteenth weight vector coefficient using an activation function to obtain the spatial attention weight coefficient of the first face image. And / or, The process of obtaining the spatial attention weight coefficients of the second face image includes: Obtain the fourth image feature of the second face image; perform max pooling on the fourth image feature to obtain the fifteenth weight vector coefficient; perform average pooling on the fourth image feature to obtain the sixteenth weight vector coefficient; stack the fifteenth and sixteenth weight vector coefficients to obtain the seventeenth weight vector coefficient; perform convolution on the seventeenth weight vector coefficient to obtain the eighteenth weight vector coefficient; and process the eighteenth weight vector coefficient using an activation function to obtain the spatial attention weight coefficient of the second face image.

4. The method according to claim 3, characterized in that, The third image feature is the image feature obtained by multiplying the channel attention weight coefficient of the first face image with the first image feature of the first face image; And / or, The fourth image feature is the image feature obtained by multiplying the channel attention weight coefficient of the second face image with the second image feature of the second face image.

5. The method according to claim 2, characterized in that, The method of obtaining the comprehensive attention features of the first face image based on the channel attention weight coefficients and spatial attention weight coefficients of the first face image includes: The result of multiplying the first image feature of the first face image, the channel attention weight coefficient of the first face image, and the spatial attention weight coefficient of the first face image is determined as the comprehensive attention feature of the first face image. And / or, The comprehensive attention features of the second face image are obtained based on the channel attention weight coefficients and spatial attention weight coefficients of the second face image, including: The comprehensive attention feature of the second face image is determined by multiplying the second image feature of the second face image, the channel attention weight coefficient of the second face image, and the spatial attention weight coefficient of the second face image.

6. The method according to claim 1, characterized in that, Determining the distribution difference of activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model includes: The comprehensive attention features of the first face image are input into the face recognition model to obtain the first distribution of activation values ​​in the last convolutional layer of the face recognition model; The comprehensive attention features of the second face image are input into the face recognition model to obtain the second distribution of activation values ​​in the last convolutional layer of the face recognition model; Obtain the distribution difference between the first distribution and the second distribution.

7. The method according to claim 1, characterized in that, The step of determining whether the first face image is a maliciously constructed image by an attacker based on the distribution difference includes: If the distribution difference is greater than a preset threshold, the first face image is determined to be an image maliciously constructed by an attacker; otherwise, the first face image is determined to be a normal image that has not been maliciously constructed by an attacker.

8. A facial recognition anomaly detection device, characterized in that, include: The system comprises a result acquisition unit, an image acquisition unit, a first feature acquisition unit, a second feature acquisition unit, and a difference determination unit. The result acquisition unit is used to obtain the face recognition result of the first face image acquired by the image acquisition device; The image acquisition unit is used to obtain a second face image that matches the face recognition result from a preset face image library, wherein the face images in the preset face image library are all face images without interfering features; The first feature acquisition unit is used to obtain the channel attention weight coefficient and spatial attention weight coefficient of the first face image, and to obtain the comprehensive attention feature of the first face image based on the channel attention weight coefficient and spatial attention weight coefficient of the first face image. The second feature acquisition unit is used to obtain the channel attention weight coefficient and spatial attention weight coefficient of the second face image, and to obtain the comprehensive attention feature of the second face image based on the channel attention weight coefficient and spatial attention weight coefficient of the second face image. The difference determination unit is used to determine the distribution difference of the activation values ​​of the comprehensive attention features of the first face image and the comprehensive attention features of the second face image in the face recognition model, and to determine whether the first face image is an image maliciously constructed by an attacker based on the distribution difference.

9. A readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the face recognition anomaly detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, The electronic device includes at least one processor, and at least one memory and bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the face recognition anomaly detection method according to any one of claims 1 to 7.

Images