Anti-photo-attack face recognition method and device, equipment and storage medium

Abstract

The invention discloses an anti-photo-attack face recognition method and device, equipment and a storage medium, and relates to the field of face recognition, and the method comprises the steps: cutting a depth image based on a recognized and cut color face image in a color image, and obtaining a depth face image; inputting the deep face image into a binary classifier for photo attack detection to obtain a photo attack detection result; when the detection result indicates that the color image and the depth image are not photo attacks, extracting a face feature map of the color image, and performing feature restoration on the depth face image based on the face feature map; and feature extraction and face recognition detection are carried out on the color image based on the deep restoration face image. The depth map is self-adaptively repaired through the neural network, so that the model performance can be improved, and the model can adapt to different environments and scenes. Face recognition is performed by combining the color image and the depth image, and the complementary features of the color image and the depth image are combined, so that the face recognition precision can be further improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of face recognition, and in particular to a face recognition method, device, equipment and storage medium that are resistant to photo attacks. Background Art

[0002] Face recognition technology is a method of identifying people by comparing facial features. It is widely used in security monitoring, border security, and smart payment. The popularity of face recognition scenarios has led to higher and higher security requirements for face recognition. Photo attacks are a major security challenge for face recognition technology. Liveness detection of faces is required to resist face attacks from photos and videos. The current methods for face liveness detection mainly include the following:

[0003] (1) Action-based liveness detection: By requiring the user to perform specific actions, such as blinking, opening the mouth, shaking the head, etc., to determine whether the face is real and alive. This method is simple and easy to implement, low cost, and suitable for most scenarios, but it has high requirements on user operation, which may affect the user experience and take a long time to recognize.

[0004] (2) Liveness detection based on near-infrared light: Liveness face detection is achieved using the principle of near-infrared imaging. This method is less affected by light and does not require user cooperation, but requires an additional near-infrared sensor to acquire images.

[0005] (3) Liveness detection based on 3D structured light: By projecting structured light onto the face and using corresponding sensors to capture and analyze the returned light reflection information, the three-dimensional shape of the face can be obtained for liveness detection. This method is highly robust and can effectively prevent attacks using flat photos or videos, but it requires professional equipment and technical support, and the cost of data collection is relatively high.

[0006] (4) Liveness detection based on deep learning: Using deep learning algorithms to learn a large number of real live faces and photo attack samples, deep learning methods can learn the deeper differences between the two, thereby improving the accuracy and robustness of liveness detection, but it requires a large amount of training data and computing resources.

[0007] The above-mentioned liveness detection methods are difficult to strike a balance between time, cost, security and user experience, and only use single modality data in subsequent recognition, which may limit the efficiency and performance of face recognition. Summary of the invention

[0008] The embodiments of the present application provide a face recognition method, device, equipment and storage medium that are resistant to photo attacks, which solve the problems of insufficient security and recognition accuracy of face recognition.

[0009] On the one hand, the present application provides a face recognition method that is resistant to photo attacks, the method comprising:

[0010] Acquire a color image and a depth image containing a face picture, and crop the depth image based on the color face image identified and cropped in the color image to obtain a depth face image;

[0011] Inputting the deep face image into a binary classifier to perform photo attack detection to obtain a photo attack detection result;

[0012] In response to the detection result indicating that the color image and the depth image are not photo attacks, extracting a facial feature map of the color image, and performing feature restoration on the depth face image based on the facial feature map;

[0013] Based on the deep restored face image after feature restoration, feature extraction and face recognition detection are performed on the color image.

[0014] Specifically, the binary classifier is based on ResNet18, the input is a single-channel deep face image, and the output is a live face identifier, including 0 and 1, 0 represents a live face, and 1 represents a face photo; the loss function of the binary classifier is expressed as follows:

[0015] L c =-[ylogp+(1-y)log(1-p)]

[0016] Here y represents the sample label, which can be 0 or 1; p represents the probability that the model predicts the sample label to be 1.

[0017] Specifically, the performing feature restoration on the deep face image based on the face feature map includes:

[0018] Perform upsampling and convolution operations on the face feature map to convert it into a feature map consistent with the size and number of channels of the deep face image;

[0019] The depth face image and the convolution feature map are spliced ​​in the channel dimension, and the color face features are fused into the depth face image to form the depth repaired face map.

[0020] Specifically, the deep face image and the post-convolution feature map are jointly input into a fast short training convolutional neural network FSRCNN, and a mean square error loss function and a multi-scale structural similarity loss function are constructed. According to the values ​​of the mean square error loss function and the multi-scale structural similarity loss function, the low-resolution deep face image is converted into a high-resolution deep repaired face image; the mean square error loss function is the error between the output face image and the target face image, and the multi-scale structural similarity loss function is used to evaluate the pixel information provided by the mean square error loss function, and repair the structure and perceptual quality of the image.

[0021] Specifically, the multi-scale structural similarity loss function L msssim It is expressed as follows:

[0022]

[0023] Among them, a, b are the original depth face image and the target face image, l M (a, b) represent the brightness comparison function at the maximum scale M, c j (a, b) represents the contrast comparison function at scale j, s j (a, b) represents the structural comparison function at scale j, α M , β j , γ j is the weight parameter;

[0024] Brightness comparison function l M (a, b) is represented as follows:

[0025]

[0026] Among them, μ a , μ b is the mean of images a and b at the maximum scale M, C1 is a constant;

[0027] Contrast comparison function c j (a, b) is represented as follows:

[0028]

[0029] Among them, σ a , σ b is the standard deviation of images a and b at scale j, and C2 is a constant;

[0030] Structural comparison function s j (a, b) is represented as follows:

[0031]

[0032] Among them, σ abis the covariance of images a and b at scale j, and C3 is a constant.

[0033] Specifically, the feature extraction of the color image based on the deep restored face image after feature restoration includes:

[0034] Extracting a first feature map of the color face image and a second feature map of the deep restoration face image respectively;

[0035] The first feature map and the second feature map are weighted and concatenated in the channel dimension to obtain a concatenated channel map;

[0036] The spliced ​​image is average pooled and flattened to be converted into a face feature vector of the target dimension to be recognized.

[0037] Specifically, the face recognition detection process includes:

[0038] The Euclidean distance between the face feature vector to be identified and all face feature vectors entered into the matching library is calculated; when the calculated Euclidean distance is less than a set threshold, it is determined that the real face recognition is successful; when the calculated Euclidean distance is not less than the set threshold, it is determined that the real face recognition fails.

[0039] On the other hand, the present application provides a face recognition device that is resistant to photo attacks, the device comprising:

[0040] A cropping module, used to obtain a color image and a depth image containing a face picture, and to crop the depth image based on the color face image identified and cropped in the color image to obtain a depth face image;

[0041] A detection module, used for inputting the deep face image into a binary classifier to perform photo attack detection and obtain a photo attack detection result;

[0042] A restoration module, configured to extract a face feature map of the color image in response to a detection result indicating that the color image and the depth image are not photo attacks, and perform feature restoration on the depth face image based on the face feature map;

[0043] The recognition module is used to extract features from the color image and perform face recognition detection based on the deep repaired face image after feature repair.

[0044] On the other hand, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the face recognition method against photo attacks described in the above aspects.

[0045] On the other hand, the present application provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the face recognition method against photo attacks described in the above aspects.

[0046] The beneficial effects brought about by the technical solution provided by the embodiment of the present application include at least the following: By using a depth camera to collect data, the present application can speed up the collection speed and is more portable and less expensive to install compared to a high-precision three-dimensional scanner. By recognizing the depth map with a simple binary classifier, it is possible to resist photo and video attacks without the user performing any unnecessary actions. Adaptive repair of the depth map by a neural network can improve the performance of the model and adapt it to different environments and scenarios. By combining color images and depth images for face recognition, the complementary features of the two are combined to further improve the accuracy of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a face recognition method against photo attacks provided by an embodiment of the present application;

[0048] Figure 2 is a flow chart of a face recognition method against photo attacks provided by an embodiment of the present application;

[0049] Figure 3 This is a flowchart of the convolutional autoencoder training process;

[0050] Figure 4 A schematic diagram of the restoration process of a deep face image is shown;

[0051] Figure 5 The complete algorithm flow chart of anti-photo attack face recognition is shown;

[0052] Figure 6 A possible structural diagram of a SE module is shown;

[0053] Figure 7 It is a schematic diagram of the process of extracting feature maps and fusing them to obtain facial feature vectors;

[0054] Figure 8 The structure block diagram of the face recognition device against photo attacks provided by the embodiment of the present application is shown;

[0055] Fig. 9 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0057] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0058] Figure 1 : is a flow chart of a face recognition method against photo attacks provided by an embodiment of the present application, comprising the following steps:

[0059] S1, obtaining a color image and a depth image containing a face picture, and cropping the depth image based on the color face image identified and cropped in the color image to obtain a depth face image;

[0060] The color image and depth image mentioned in this application are obtained by the corresponding camera module, and the image needs to contain a face image. The color image specifically refers to the RGB three-color image, and the depth image is an image containing depth of field data. The two together constitute the RGB-D data image.

[0061] The first step is to preprocess the image data. You can set RGB-D samples specifically for model training. The faces in the images have different postures and expressions. For the sample data, convert the size of each image to a fixed size such as 224×224, set a corresponding label for each face image, and delete the depth image and its corresponding color image if the pixel value is 0 and exceeds 3 / 4 of the total size.

[0062] For all collected color images and depth images, face detection is first performed on the color image, and then cropping is performed after the face is detected, that is, the color face image is cropped from the color image, and then the depth image is cropped and cut based on the color face image to extract the corresponding depth face image.

[0063] The depth map in the RGB-D data collected by the depth camera usually has missing information and cannot accurately express the structural information of the face. Using these depth maps directly for feature extractor training may cause the model to learn wrong features, resulting in performance degradation, and even making it difficult for the model to converge or overfit, affecting the training effect of the model. Color images contain rich texture and color information, which can provide additional reference and supplement for the depth map. By using convolutional autoencoders to extract facial features from color images, this additional information can be used to assist in the restoration of the depth map, thereby improving the accuracy and completeness of the restoration.

[0064] S2, input the deep face image into the binary classifier for photo attack detection, and obtain the photo attack detection result:

[0065] The binary classifier used in this application is designed based on ResNet1 8. Its input is a single-channel deep face image, and its output is a live face identifier, including 0 and 1. Output 0 indicates that it is detected as a live face, and output 1 indicates a face photo (photo attack). The loss function used in binary classifier training is the cross entropy loss function, which is expressed as follows:

[0066] L c =-[ylogp+(1-y)log(1-p)]

[0067] Here y represents the sample label, which can be 0 or 1; p represents the probability that the model predicts the sample label to be 1.

[0068] S3, in response to the detection result indicating that the color image and the depth image are not photo attacks, extracting a face feature map of the color image, and performing feature restoration on the depth face image based on the face feature map;

[0069] When the detection result shows that it is a photo attack, no subsequent steps will be performed to prevent the system from being maliciously attacked. When the detection result shows a normal living face, it means that it is a normal face recognition verification scenario, and the special diagnosis extraction model is used to extract the face feature map (face feature data) of the color image. Considering that the information contained in a simple color face image or a depth image is relatively limited, this application uses the face feature map of the color face image to perform feature repair on the depth face image, that is, to fuse the face structure features of the color image into the depth face image, because the color image and the depth image belong to different perceptual modes, they contain complementary information. By extracting facial features from the color image and fusing it with the depth map, effective fusion of cross-modal information can be achieved, and the ability to repair the depth map can be improved. The obtained deep repaired face map can also repair the missing parts of the depth map to a certain extent to improve the accuracy of the detection.

[0070] Figure 2 It is a comparative diagram before and after using the extracted facial features to repair the depth face image. In the depth map before repair, there are obviously some missing features caused by factors such as viewing angle and resolution. After repairing with the facial features extracted from the color face image, some missing features are supplemented to achieve depth repair, and a more complete face depth image is obtained.

[0071] S4, extracting features from the color image and performing face recognition detection based on the deep restored face image after feature restoration.

[0072] The previous step is to extract the facial features of the color face image to repair the deep face image, while this step is to extract the color face image and the deep repaired face image at the same time, deeply fuse the feature map data of the two, and use the deeply fused feature vector (data) for face recognition detection. In contrast, the feature vector of feature fusion has all the characteristics of the original color face image and the deep repaired face image. The precision and accuracy of face matching recognition using this feature vector are greatly improved.

[0073] In summary, this application uses a depth camera to collect data, which can speed up the acquisition speed and is more portable and cheaper to install than a high-precision 3D scanner. By recognizing the depth map with a simple binary classifier, it is possible to resist photo and video attacks without the user performing any unnecessary actions. Adaptive repair of the depth map through a neural network can improve the performance of the model and adapt it to different environments and scenarios. By combining color images and depth images for face recognition, the complementary features of the two are combined to further improve the accuracy of face recognition.

[0074] In some embodiments, applying this method to actual system equipment requires preliminary data processing and model training processes. Figure 3 This is a flow chart of convolutional autoencoder training. The input of the convolutional autoencoder is a 3-channel original color face image, and the output is a 3-channel restored color face image and a 112×112×64 face feature map. The restored color face image is used to calculate the loss optimization model with the original color face image, and the face feature map is used to combine with the original depth face image for depth map restoration. The convolutional autoencoder consists of two parts: the encoder and the decoder. The first half of the encoder downsamples the color face based on ResNet50. After the first convolution layer, the image size becomes 112×112×64. Next, when passing through the following four convolutional layers in sequence, the length and width of the feature map of each layer are reduced to 1 / 2 of the previous one, and the number of channels becomes twice the original, resulting in a 7×7×1024 feature map. In the decoder part, the feature map is reversed through five layers of transposed convolution with a convolution kernel of 2×2 and a stride of 2 to restore the feature map to the original image size. The color feature map with a size of 112×112 and 64 channels during the upsampling process is saved for subsequent depth map restoration. The loss function used in training the convolutional autoencoder is the mean square error (MSE) loss, which is the error between the restored color face image and the original color face image.

[0075]

[0076] Among them, N is the number of pixels, x i The pixel value of a certain point in the image output by the convolutional self-encoder, is the pixel value of the corresponding point in the original image.

[0077] When using a binary classifier to detect photo attacks and determining that it is a real living person portrait, the face feature map ( Figure 3 The feature restoration of the deep face image is performed in advance or after the liveness detection is confirmed. Figure 4 The depth map repair schematic diagram shown includes the following steps:

[0078] A, upsample and convolve the face feature map to convert it into a feature map with the same size and number of channels as the deep face image;

[0079] B. Concatenate the deep face image and the convolutional feature map in the channel dimension, and fuse the color face features into the deep face image to form a deep repaired face map.

[0080] The upsampling and convolution operations are mainly for the unification of image size and number of channels. This application constructs a depth map restoration network, which specifically includes two inputs: a 112×112×64 color face feature map and a single-channel original depth face image. The feature map is upsampled and convolved with a convolution kernel size of 3×3, a step size of 1, and a convolution kernel number of 1, and its size is changed to 224×224×1. It is concatenated with the depth map in the channel dimension and then input into FSRCNN for depth map restoration.

[0081] Fast short training convolutional neural network FSRCNN is a neural network model for image super-resolution. This application uses this model mainly to convert low-resolution images into high-resolution images through the model to improve the clarity and details of the image. FSRCNN contains 7 convolutional layers and 1 transposed convolutional layer. The first convolution kernel size is 5×5, the step size is 1, and the number of convolution kernels is 56. This convolution layer is used to extract image features. The second convolution kernel size is 1×1, the step size is 1, and the number of convolution kernels is 12. This convolution layer is used to reduce the dimension of the features to avoid the problem of excessive nonlinear mapping calculation complexity caused by excessive dimensions after feature extraction. The third to sixth convolution kernel sizes are 3×3, the step size is 1, and the number of convolution kernels is 12. These convolution layers are used for nonlinear mapping to enhance the nonlinear expression ability of the network. The seventh convolution kernel size is 1×1, the step size is 1, and the number of convolution kernels is 56. This convolution layer is used to expand the feature map. If the low-dimensional feature map is directly used for image reconstruction, the quality of image reconstruction will be greatly reduced. Each convolutional layer is followed by a PreLU layer with a kernel size of 9×9, a stride of 1, and a transposed number of convolution kernels of 1. The convolutional layer can adaptively enlarge the image.

[0082] The loss functions constructed by training the deep image restoration network are the mean square error (MSE) loss function and the multi-scale structural similarity (MS-SSIM) loss function. The low-resolution deep face image is converted into the high-resolution deep restoration face image according to the mean square error loss function and the multi-scale structural similarity loss function. The mean square error calculates the error between the output face image and the target face image. When the restored output face image is slightly offset from the target face image, using only the mean square error will cause a large loss value, causing the network to believe that the restoration effect is poor, making it difficult for the network to learn the correct parameters. Therefore, the multi-scale structural similarity loss is introduced here to weaken the pixel information provided by the mean square error, so that the network pays more attention to the structure and perceptual quality of the restored image, and hyperparameters 0.001 and 0.999 are added before the mean square error loss and the multi-scale structural similarity loss.

[0083] The multi-scale structural similarity loss function L msssim It is expressed as follows:

[0084]

[0085] Among them, a, b are the original depth face image and the target face image, l M (a, b) represent the brightness comparison function at the maximum scale M, c j (a, b) represents the contrast comparison function at scale j, s j (a, b) represents the structural comparison function at scale j, α M , β j , γ j is the weight parameter;

[0086] Brightness comparison function l M (a, b) is represented as follows:

[0087]

[0088] Among them, μ a , μ b is the mean of images a and b at the maximum scale M, C1 is a constant used to control the denominator not to be 0;

[0089] Contrast comparison function c j (a, b) is represented as follows:

[0090]

[0091] Among them, σ a , σ b is the standard deviation of images a and b at scale j, C2 is a constant used to control the denominator not to be 0;

[0092] Structural comparison function sj (a, b) is represented as follows:

[0093]

[0094] Among them, σ ab is the covariance of images a and b at scale j, and C3 is a constant used to control the denominator not to be zero.

[0095] The depth image restoration method based on the above neural network can learn complex rules from a large amount of data, automatically learn the features and structural information in the depth image and perform restoration, can adapt to different scenes and data distributions, and has certain robustness and generalization capabilities. Figure 2 As shown in Figure 3, the network can repair some defects in the original low-precision depth map, automatically fill in the holes and make the surrounding contours more natural, making the repaired depth map more complete.

[0096] Figure 5 The complete algorithm flow chart of anti-photo attack face recognition is shown. First, the acquired color image and depth image are cropped to obtain the color face image and the depth face image, and then the color face image is extracted to the face feature map using the trained convolutional autoencoder. Correspondingly, the depth face image is judged by a binary classifier, and whether it is a photo attack is determined based on the classification result, because the photo attack cannot detect the depth information. After determining that it is a photo attack behavior, the process is terminated directly to prevent the system from being maliciously attacked. After determining that it is a living face, the face feature map and the depth face image output by the convolutional autoencoder are input into the trained depth map repair network together, and the depth repaired face map is output.

[0097] Further, a feature extraction network is used to extract features from the original color face image and the deep restoration face image to obtain a face feature vector. In some embodiments, the process of obtaining a face feature vector and face recognition can be expressed as follows:

[0098] A, extract the first feature map of the color face image and the second feature map of the deep restoration face image respectively;

[0099] B, weighting the first feature map and the second feature map, and splicing them in the channel dimension to obtain a spliced ​​channel map;

[0100] C, average pooling and flattening the spliced ​​image to convert it into a face feature vector of the target dimension to be recognized.

[0101] D, calculate the Euclidean distance between the face feature vector to be identified and all face feature vectors entered into the matching library;

[0102] E. When the calculated Euclidean distance is less than the set threshold, it is judged that the real face recognition is successful; when the calculated Euclidean distance is not less than the set threshold, it is judged that the real face recognition fails.

[0103] In the above process, the feature extraction network needs to be trained in advance. The SE module is used in the feature extraction network to make the network pay more attention to important features. Figure 6 A schematic diagram of the structure of a possible SE module is shown. First, the input is globally average pooled, and the weight of each channel is learned using the structure of a fully connected layer, a ReLU layer, a fully connected layer, and a Sigmoid layer. The learned weight is multiplied by the original feature map to obtain a weighted feature map.

[0104] Figure 7 This is a flowchart of using a feature extraction network to extract feature maps and fuse them to obtain facial feature vectors; the input of the feature extraction network is a three-channel color face image and a single-channel deep face image, which includes two branches, ResNet50 and ResNet18. ResNet50 is used to extract the first feature map of 7×7 size and 2048 channels in the color face; ResNet1 8 is used to extract the second feature map of 7×7 size and 512 channels in the deep repaired face image. The two feature maps are respectively input into the SE module for weighting and splicing in the channel dimension to obtain a spliced ​​channel map of 2560 channels. After the SE module, it is converted into a 2560-dimensional feature vector through average pooling and flattening. The Dropout layer is used to prevent the network from over-relying on certain features during training. Finally, a 128-dimensional face feature vector is obtained through the fully connected layer.

[0105] In addition to the cross entropy loss, the center loss is also used when training the feature extraction network. Hyperparameters 0.9 and 0.1 are added before the cross entropy loss and the center loss. The cross entropy loss is used to ensure the accuracy of the model in the classification task, and the center loss is used to improve the discriminability of the feature representation, so that the features are better clustered around the center of their respective categories. center and center loss It is expressed as follows:

[0106]

[0107]

[0108] Where n is the number of samples, f i is the 128-dimensional feature vector obtained by the fully connected layer, y i is the feature vector f i The corresponding category, For classification y i The center is obtained by finding the center of the class y iThe average characteristics of m data are obtained.

[0109] In summary, the solution provided by this application has the following advantages compared with the related art:

[0110] (1) By using a depth camera to collect data, the acquisition speed can be accelerated compared to a high-precision 3D scanner, and the installation is more portable and the cost is lower.

[0111] (2) By recognizing the depth map using a simple binary classifier, it can resist photo and video attacks without the user performing any unnecessary actions.

[0112] (3) Adaptive repair of depth maps through neural networks can improve model performance and make it adaptable to different environments and scenarios.

[0113] (4) By combining color images and depth images for face recognition, the complementary features of the two are combined, and the SE attention module is added to the feature extraction network to automatically learn the weights between features, which can further improve the accuracy of face recognition.

[0114] Figure 8 The structure block diagram of the face recognition device against photo attacks provided by an embodiment of the present application is shown, and the device includes:

[0115] The cropping module 810 is used to obtain a color image and a depth image containing a face picture, and to crop the depth image based on the color face image identified and cropped in the color image to obtain a depth face image;

[0116] A detection module 820, configured to input the deep face image into a binary classifier for photo attack detection to obtain a photo attack detection result;

[0117] A restoration module 830 is configured to extract a facial feature map of the color image in response to the detection result indicating that the color image and the depth image are not photo attacks, and perform feature restoration on the depth face image based on the facial feature map;

[0118] The recognition module 840 is used to extract features from the color image and perform face recognition detection based on the deep restored face image after feature restoration.

[0119] The face recognition device resistant to photo attacks provided in the embodiment of the present application can be applied to the face recognition method resistant to photo attacks provided in the above embodiment. For relevant details, refer to the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0120] It should be noted that the face recognition device resistant to photo attacks provided in the embodiments of the present application is only illustrated by the division of the above-mentioned functional modules / functional units. In practical applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the face recognition device resistant to photo attacks is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the face recognition method resistant to photo attacks provided in the above method embodiment and the implementation method of the face recognition device resistant to photo attacks provided in this embodiment belong to the same concept. The specific implementation process of the face recognition device resistant to photo attacks provided in this embodiment is detailed in the above method embodiment, which will not be repeated here.

[0121] Fig. 9 The block diagram of a computer device provided by an exemplary embodiment of the present application is shown. It is a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Among them, the processor and the memory may be connected by a bus or in other ways. Among them, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips.

[0122] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0123] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0124] In some embodiments, the computer device may further optionally include: a peripheral device interface and at least one peripheral device. The processor, the memory and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit, a display screen and a keyboard.

[0125] The peripheral device interface can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor and the memory. In some embodiments, the processor, the memory, and the peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory, and the peripheral device interface can be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0126] The display screen is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen is a touch screen, the display screen also has the ability to collect touch signals on the surface or above the surface of the display screen. The touch signal can be input to the processor as a control signal for processing. At this time, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen can be one, arranged on the front panel of the computer device; in other embodiments, the display screen can be at least two, respectively arranged on different surfaces of the computer device or in a folding design; in other embodiments, the display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the computer device. Even, the display screen can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0127] The power supply is used to power various components in the computer device. The power supply can be AC, DC, a disposable battery, or a rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0128] Those skilled in the art will appreciate that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0129] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art can understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the process of the implementation of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0130] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A face recognition method against photo attacks, characterized in that: The method comprises: Acquire a color image and a depth image containing a face picture, and crop the depth image based on the color face image identified and cropped in the color image to obtain a depth face image; Inputting the deep face image into a binary classifier to perform photo attack detection to obtain a photo attack detection result; In response to the detection result indicating that the color image and the depth image are not photo attacks, extracting a facial feature map of the color image, and performing feature restoration on the depth face image based on the facial feature map; Based on the deep restored face image after feature restoration, feature extraction and face recognition detection are performed on the color image.

2. The method according to claim 1, characterized in that: The binary classifier is based on ResNet18, with a single-channel deep face image as input and a live face identifier as output, including 0 and 1, where 0 represents a live face and 1 represents a face photo; the loss function of the binary classifier is expressed as follows: L c =-[ylogp+(1-y)log(1-p)] Here y represents the sample label, which can be 0 or 1; p represents the probability that the model predicts the sample label as 1.

3. The method according to claim 2, characterized in that The performing feature restoration on the deep face image based on the face feature map comprises: Perform upsampling and convolution operations on the face feature map to convert it into a feature map consistent with the size and number of channels of the deep face image; The depth face image and the convolution feature map are spliced ​​in the channel dimension, and the color face features are fused into the depth face image to form the depth repaired face map.

4. The method according to claim 3, characterized in that The deep face image and the convolutional feature map are jointly input into the fast short training convolutional neural network FSRCNN, and a mean square error loss function and a multi-scale structural similarity loss function are constructed. The low-resolution deep face image is converted into a high-resolution deep repaired face image according to the values ​​of the mean square error loss function and the multi-scale structural similarity loss function; the mean square error loss function is the error between the output face image and the target face image, and the multi-scale structural similarity loss function is used to evaluate the pixel information provided by the mean square error loss function, and repair the structure and perceptual quality of the image.

5. The method according to claim 4, characterized in that The multi-scale structural similarity loss function L msssim It is expressed as follows: Among them, a, b are the original depth face image and the target face image, l M (a, b) represents the brightness comparison function at the maximum scale M, c j (a, b) represents the contrast comparison function at scale j, s j (a, b) represents the structural comparison function at scale j, α M ,β j ,γ j is the weight parameter; Brightness comparison function l M (a,b) is represented as follows: Among them, μ a ,μ b is the mean of images a and b at the maximum scale M, C1 is a constant; Contrast comparison function c j (a,b) is represented as follows: Among them, σ a ,σ b is the standard deviation of images a and b at scale j, and C2 is a constant; Structural comparison function s j (a,b) is represented as follows: Among them, σ ab is the covariance of images a and b at scale j, and C3 is a constant.

6. The method according to claim 3, characterized in that The step of extracting features from the color image based on the feature-restored deep repaired face image comprises: Extracting a first feature map of the color face image and a second feature map of the deep restoration face image respectively; The first feature map and the second feature map are weighted and concatenated in the channel dimension to obtain a concatenated channel map; The spliced ​​image is average pooled and flattened to be converted into a face feature vector to be recognized of a target dimension.

7. The method according to claim 6, characterized in that The face recognition detection process includes: The Euclidean distance between the face feature vector to be identified and all face feature vectors entered into the matching library is calculated; when the calculated Euclidean distance is less than a set threshold, it is determined that the real face recognition is successful; when the calculated Euclidean distance is not less than the set threshold, it is determined that the real face recognition fails.

8. A face recognition device resistant to photo attacks, characterized in that: The device comprises: A cropping module, used to obtain a color image and a depth image containing a face picture, and to crop the depth image based on the color face image identified and cropped in the color image to obtain a depth face image; A detection module, used for inputting the deep face image into a binary classifier to perform photo attack detection and obtain a photo attack detection result; A restoration module, configured to extract a face feature map of the color image in response to a detection result indicating that the color image and the depth image are not photo attacks, and perform feature restoration on the depth face image based on the face feature map; The recognition module is used to extract features from the color image and perform face recognition detection based on the deep repaired face image after feature repair.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the face recognition method resistant to photo attacks as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the face recognition method resistant to photo attacks as described in any one of claims 1 to 7.

Images