Face anti-fraud detection method based on multi-scale texture features and pixel-level depth supervision

Abstract

The invention discloses a face anti-fraud detection method based on multi-scale texture features and pixel-level depth supervision. The method mainly aims at ignoring the difference between a real face and an attack face in texture and structure in a deep learning-based method. Firstly, a central difference convolution module is designed to extract detail texture information of a human face; secondly, encoding the image blocks by using a Swin Transform with four stages, so as to extract microscopic features; then, designing a dynamic feature fusion module to combine different depth layer feature information of the four stages so as to construct texture features and a depth map; then, designing a deep feature information enhancement module for deep reasoning, and improving the network feature expression capability; and finally, performing face anti-fraud detection on the input image in combination with the local texture features and the depth map. The method is high in recognition precision, and effectively solves the problem of face anti-fraud detection in a complex scene.

Description

Technical Field

[0001] The present invention relates to the technical field of face liveness detection, and in particular to a face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision. Background Art

[0002] Nowadays, face recognition technology is constantly developing. Its advantages such as non-contact, high efficiency, and low cost make it gradually integrated into people's lives, but risks also come with it. Criminals can obtain facial photos or video clips of legitimate users through illegal means, and even use facial data to make 3D masks to attack face recognition systems, which undoubtedly seriously threatens people's lives, property and other safety. Therefore, face liveness detection technology is particularly important. In the existing technology, face liveness detection mainly identifies and classifies real faces and forged faces by analyzing facial movements, color textures, spectral analysis, features, etc. These methods can be roughly divided into two categories: traditional methods and deep learning-based methods.

[0003] Traditional methods mainly perform detection by analyzing facial motion, color texture and spectrum. For example, Pan et al. and Cech et al. proposed to start from the movements of the eyes and lips of the face, and set it in a specific environment to obtain and analyze images to determine true and false faces. However, this type of method requires active cooperation from users and is time-consuming. In addition: Li et al. proposed a method based on Fourier spectrum analysis to extract the spectrum of true and false faces and analyze them. It was found that the high-frequency information of fake faces is less than that of real faces, but this method is easily affected by lighting and image resolution; Maatta et al. proposed to use the LBP binary method to extract the organ contours and other information of true and false faces and identify them. It can cope with lighting changes well, but it is more sensitive to changes in the position of the face; Pereira et al. subsequently proposed to introduce LBP-TOP, which has made greater progress, but the generalization of this type of method is limited. In recent years, scholars have been using traditional methods to conduct research in the field of face liveness detection. For example, Feng Yun et al. proposed the Differential Quantization Adjacent Local Binary Pattern (DQ_CoALBP) operator, which integrates the differences between the local center point and the surrounding points of the image in different directions to more fully describe the color and texture information. However, the ability of this method to extract feature information is still limited.

[0004] With the continuous development of deep learning, many efficient structures and networks have been proposed. Among them, the representative one is the Inception-v1 network proposed by Szegedy et al., whose Inception classic unit makes the network wider and makes better use of computer resources; the deep residual network proposed by He et al. is still the backbone network of major computer vision. In addition, people have introduced the attention mechanism by studying the ability of the human eye to find significant areas in complex environments. At present, there are a large number of literatures that can verify that the attention mechanism can promote the performance of convolutional neural networks.

[0005] In addition, due to the imperfection of the medium used for attacking faces, there are large differences between common attacking faces and real faces. For example, printed attacking face images show lattice effects in texture cues, which is caused by wood grain or screen pixels. Moreover, the medium of paper or electronic screen commonly used for attacking faces is a flat structure, which does not contain any depth information, which is different from the three-dimensional structural information of real faces. Therefore, many methods based on analyzing facial texture and structural information have been proposed. Liang et al. proposed a depth map-guided triple network for deep face liveness detection, and its deep detection network can effectively reflect the differences between real and fake faces in discontinuity, inconsistent lighting, and blur. Liu et al. used depth map supervision and GAN model to generate more fake face samples. Wang et al. proposed a transform-based face anti-spoofing method that integrates texture and depth cues to improve detection performance. Summary of the invention

[0006] In view of this, the purpose of the present invention is that the traditional face liveness detection network is easily affected by lighting and image quality, and the method based on deep learning ignores the differences in texture and structure between the real face and the attacking face. In order to improve the network feature expression ability, a method of face liveness detection network is proposed, which is a face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] Step 1: Preprocess the video stream dataset to obtain face images with the same posture and size.

[0009] Step 2: Design an improved central difference convolution module to extract detailed texture information of the face to enhance the network feature expression capability.

[0010] Step 3: Encode these image patches using a Swin Transformer with four stages to extract micro features.

[0011] Step 4: Design a dynamic feature fusion module to combine the feature information of different depth layers in the four stages to construct texture feature descriptors and pixel-level depth descriptors.

[0012] Step 5: Design a deep feature information enhancement module for deep reasoning, using accurate depth maps as auxiliary supervision to improve the network feature expression capability.

[0013] Step 6: Combine local texture features and pixel-level depth information into the fully connected layer, and use two loss functions for supervision to achieve liveness detection of the face.

[0014] In step 1 of the present invention, data preprocessing is performed on the video stream data set to obtain face images of the same posture and size. The specific method is:

[0015] (1) The three face liveness datasets of Replay-Attack, CASIA-FASD and OULU-NPU were cropped to eliminate the background influence and obtain face images. Then, the Dlib face detector was used to detect and align the face images in the dataset; finally, the face images were normalized and resized to 224×224×3. The dataset was divided into a training set: test set: validation set ratio of 6:2:2.

[0016] In step 2 of the present invention, center difference convolution is used to extract detailed texture information of the face to enhance the network feature expression ability. Since there is a difference in the input data dimension between the Swin Transformer model and the traditional CNN network, the data dimension needs to be adjusted accordingly. First, the data dimension input to the module is transformed from (B, H×W, C); then, the data is input into a 1×1 convolution layer to reduce the number of channels to C / 4; then, local features are extracted through a 3×3 center difference convolution; finally, a 1×1 convolution layer is used to restore the dimension, and a residual connection is introduced to prevent model degradation, and a batch normalization layer and a ReLU activation function are added after each convolution layer to accelerate model convergence. The specific method is:

[0017] (1) Calculate the difference between the center point and the surrounding area, as described below:

[0018]

[0019] (2) Combining the central difference convolution with the vanilla convolution can obtain stronger feature extraction capabilities, as described below:

[0020]

[0021] In step 3 of the present invention, a four-stage Swin Transformer is used to encode these image blocks to extract microscopic features. The specific method is:

[0022] (1) Similar to the traditional Transformer, the processed face image is divided into N non-overlapping small blocks. The specific formula is as follows:

[0023] X={x1,...,x i ,...,x N}

[0024] Where N is the number of image blocks, represents the i-th image block, P is x i These image patches are then expanded two-dimensionally and linearly mapped into a matrix of size N×C, where C is equal to P×P×3.

[0025] (2) After embedding the image block, the Swin Transformer module is used to learn the attention features. Swin Transformer consists of a multi-layer perceptron (MLP), a normalization layer (LP), and a sliding window module. The sliding window module can be divided into a conventional window-based multi-head self-attention layer (W-MSA) and a sliding window-based multi-head self-attention layer (SW-MSA). The specific formula is as follows:

[0026]

[0027] in, and z l They are the output features of the (S)W-MSA module and the MLP module. The MLP consists of two fully connected layers and a GELU nonlinear activation function.

[0028] (3) In addition, in order to generate more hierarchical representations, these image patches are followed by a patch merging layer, which can reduce the number of patches and expand the channel dimension. As shown in the figure, in the first stage layer, each set of features consisting of 2×2 adjacent patches is first connected to a 4×C dimensional vector, and then the feature map is connected through a linear layer to reduce the number of patches from 4×C to 2×C dimensions. The output resolution formula of the four stages is as follows:

[0029]

[0030] Among them, s1, s2, s3 and s4 are the feature map resolutions of the first stage, second stage, third stage and fourth stage respectively.

[0031] In step 4 of the present invention, a dynamic feature fusion module is designed to combine the feature information of different depth layers in the four stages, thereby constructing a texture feature descriptor and a pixel-level depth descriptor. The specific method is:

[0032] (1) The dynamic feature fusion module first performs global average pooling on the output features of different depth layers of the main model, then uses the CA attention mechanism to fuse feature information of four different scales, and then uses the convolutional layer to construct a depth map, as described below:

[0033] D′=a1·d1+a2·d2+a3·d3+a4·d4

[0034]

[0035] in, CA(·) represents the channel attention operator; Conv represents the convolution operation with activation function; is the predicted depth map, the value of m is 48;

[0036]

[0037] Among them, a1, a2, a3, and a4 represent the weights corresponding to the feature information respectively. w i is the initialization weight.

[0038] (2) Use the output feature maps of different stages to calculate texture features.

[0039] In step 5 of the present invention, a depth information feature enhancement module is designed for deep reasoning, and an accurate depth map is used as auxiliary supervision to improve the network feature expression capability. The specific method is:

[0040] (1) Given the input features of the backbone network output Predicting discretized deep category maps through two convolutional layers The probability of D, where D represents the number of depth categories. The probability of each pixel represents the confidence that the depth value of the pixel belongs to a certain depth category.

[0041] (2) Discretize the continuous depth space and classify the depth values ​​using a linear increasing discretization method. The resulting intermediate feature map is regarded as the initial depth-aware feature. At the same time, the feature of each pixel is further enhanced by introducing the central representation of each depth category, namely the depth prototype. Each depth prototype is calculated by aggregating the depth-aware features of pixels belonging to the specified category.

[0042] (3) Use group convolution to process the predicted depth map, merge adjacent depth categories, and reduce the number of categories from D to D′=D / r, where r is the scaling ratio.

[0043] (4) Each deep prototype F d It is generated by collecting the features of all pixels X′ and weighting them according to the probability that these pixels belong to the depth category d, as described below.

[0044]

[0045] Among them, X i ′ is X i The i-th pixel feature; is the set of pixels in the feature map; is the normalized probability to the d-th depth prototype; F d is the global information of each depth category; the reconstructed feature F′ can be expressed by the following formula:

[0046]

[0047] (5) Perform multi-scale feature extraction on feature map X. First, a set of 1×1 convolutions is used to obtain feature map X1. Then, a set of 1×1 convolutions, 5×5 convolutions and 3×3 convolutions are used to obtain feature map X2. A set of 1×1 convolutions, followed by 3×3 convolutions are used to obtain feature map X3. A set of pooling layers is used first, followed by 1×1 convolutions to obtain feature map X4. Multi-scale features are concatenated to obtain X e , and connect it with the reconstructed feature F′ to obtain the final fused enhanced deep feature.

[0048] In step 6 of the present invention, local texture features and pixel-level depth information are combined and input into the fully connected layer, and two loss functions are used for supervision to achieve face liveness detection. The specific method is:

[0049] (1) When updating the network parameters, the binary cross entropy loss function and the mean square error function are used to calculate the distance between the local texture and construction depth predicted by the model and the input label. It can be expressed as: Among them, y i is the one-hot label value of the i-th training sample, which is a manual label record. is the predicted value of the i-th neuron, n is the number of face categories, and its value is 2.

[0050] (2) Use the depth prediction map to estimate the depth map of each image, expressed as

[0051] Among them, D i,j and D i ' ,j is the pixel value at position (i,j) and m is the size of the prediction or depth label.

[0052] (3) During the training phase, the binary cross entropy loss function and the mean square error are used to update the network parameters, expressed as: Loss = α·Loss Binary +β·Loss MSE , where Loss is the final loss function. In this method, the values ​​of α and β are set to 0.5.

[0053] (4) For MSE loss function Loss MSE The depth label D used has a binary mask size of 48×48. If the input image is a real face, all values ​​in D are set to 1, otherwise, all values ​​of D are set to 0. The training environment is configured as follows: the framework is the Pytorch framework. The Adam optimizer is used to optimize the network during training, the learning rate is 0.0003, and the batch size is 32. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is the overall network framework of the present invention.

[0055] Figure 2 It is the difference between the real face and the attack face in local texture and depth map.

[0056] Figure 3 Schematic diagram of the central difference convolution module.

[0057] Figure 4 It is a structure of two consecutive blocks in Swin Transformer.

[0058] Figure 5 Schematic diagram of the depth feature information enhancement module of the method of the present invention.

[0059] Figure 6 A schematic diagram showing the visualization of the results of the method of the present invention. DETAILED DESCRIPTION

[0060] In order to deepen the understanding of the present invention, the present invention will be further described in detail with reference to the accompanying drawings and embodiments. The embodiments are only used to explain the present invention and do not limit the protection scope of the present invention.

[0061] like Figure 1-6 As shown in the figure, a method for face liveness detection network is a face anti-spoofing detection method based on multi-scale texture features and pixel-level deep supervision. The central difference convolution module and the pre-trained Swin-Transformer model are introduced to construct local texture features of the face and construct depth information, thereby improving the network's ability to extract details and global features of live images. At the same time, a dynamic feature fusion module and a deep feature information enhancement module are designed to further improve the network's feature expression ability. Finally, two loss functions are used to supervise the network training.

[0062] Step 1: Preprocess the video stream dataset to obtain face images with the same posture and size.

[0063] Step 2: Design an improved central difference convolution module to extract detailed texture information of the face to enhance the network feature expression capability.

[0064] Step 3: Encode these image patches using a Swin Transformer with four stages to extract micro features.

[0065] Step 4: Design a dynamic feature fusion module to combine the feature information of different depth layers in the four stages to construct texture feature descriptors and pixel-level depth descriptors.

[0066] Step 5: Design a deep feature information enhancement module for deep reasoning, using accurate depth maps as auxiliary supervision to improve the network feature expression capability.

[0067] Step 6: Combine local texture features and pixel-level depth information into the fully connected layer, and use two loss functions for supervision to achieve liveness detection of the face.

[0068] Furthermore, in step 1, the video stream data set is preprocessed. The specific method is as follows:

[0069] The three face liveness datasets of Replay-Attack, CASIA-FASD and OULU-NPU were cropped to eliminate the background influence and obtain face images. Then, the Dlib face detector was used to detect and align the face images in the dataset; finally, the face images were normalized and resized to 224×224×3. The dataset was divided into a training set: test set: validation set = 6:2:2 ratio.

[0070] Furthermore, in step 2, the center difference convolution module is used to extract the detailed texture information of the face. The specific method is as follows:

[0071] (1) Since the Swin Transformer model and the traditional CNN network have different input data dimensions, the data dimensions need to be adjusted accordingly. First, the data dimension of the input module is transformed from (B, H×W, C); then, the data is input into a 1×1 convolution layer to reduce the number of channels to C / 4; then, local features are extracted through a 3×3 central difference convolution; finally, a 1×1 convolution layer is used to restore the dimension, and a residual connection is introduced to prevent model degradation. A batch normalization layer and a ReLU activation function are added after each convolution layer to accelerate model convergence. The difference between the center point and the surrounding is calculated as follows:

[0072]

[0073] (2) Combining the central difference convolution with the vanilla convolution can obtain stronger feature extraction capabilities, as described below:

[0074]

[0075] Furthermore, in step 3, these image blocks are encoded using a four-stage SwinTransformer to extract microscopic features. The specific method is:

[0076] (1) Similar to the traditional Transformer, the processed face image is divided into N non-overlapping small blocks. The specific formula is as follows:

[0077] X={x1,...,x i ,...,x N}

[0078] Where N is the number of image blocks, represents the i-th image block, P is x i These image patches are then expanded two-dimensionally and linearly mapped into a matrix of size N×C, where C is equal to P×P×3.

[0079] (2) After embedding the image block, the Swin Transformer module is used to learn the attention features. Swin Transformer consists of a multi-layer perceptron (MLP), a normalization layer (LP), and a sliding window module. The sliding window module can be divided into a conventional window-based multi-head self-attention layer (W-MSA) and a sliding window-based multi-head self-attention layer (SW-MSA). The specific formula is as follows:

[0080]

[0081] in, and z l They are the output features of the (S)W-MSA module and the MLP module. The MLP consists of two fully connected layers and a GELU nonlinear activation function.

[0082] (3) To generate more hierarchical representations, these image patches are followed by a patch merging layer, which can reduce the number of patches and expand the channel dimension. As shown in the figure, in the first stage layer, each set of features consisting of 2×2 adjacent patches is first concatenated into a 4×C dimensional vector, and then the feature map is concatenated through a linear layer to reduce the number of patches from 4×C to 2×C dimensions. The output resolution formula of the four stages is as follows:

[0083]

[0084] Among them, s1, s2, s3 and s4 are the feature map resolutions of the first stage, second stage, third stage and fourth stage respectively.

[0085] Furthermore, in step 4, a dynamic feature fusion module is designed to combine the feature information of different depth layers in the four stages to construct texture feature descriptors and pixel-level depth descriptors. The specific method is:

[0086] (1) The dynamic feature fusion module first performs global average pooling on the output features of different depth layers of the main model, then uses the CA attention mechanism to fuse feature information of four different scales, and then uses the convolutional layer to construct a depth map, as described below:

[0087] D′=a1·d1+a2·d2+a3·d3+a4·d4

[0088]

[0089] in, CA(·) represents the channel attention operator; Conv represents the convolution operation with activation function; is the predicted depth map, the value of m is 48;

[0090]

[0091] Among them, a1, a2, a3, and a4 represent the weights corresponding to the feature information respectively. w i is the initialization weight.

[0092] (2) Use the output feature maps of different stages to calculate texture features.

[0093] Furthermore, in step 5 of the present invention, a depth information feature enhancement module is designed for deep reasoning, and accurate depth maps are used as auxiliary supervision to improve the network feature expression capability. The specific method is:

[0094] (1) Given the input features of the backbone network output Predicting discretized deep category maps through two convolutional layers The probability of D, where D represents the number of depth categories. The probability of each pixel represents the confidence that the depth value of the pixel belongs to a certain depth category.

[0095] (2) Discretize the continuous depth space and classify the depth values ​​using a linear increasing discretization method. The resulting intermediate feature map is regarded as the initial depth-aware feature. At the same time, the feature of each pixel is further enhanced by introducing the central representation of each depth category. Each depth prototype is calculated by aggregating the depth-aware features of pixels belonging to the specified category.

[0096] (3) Use group convolution to process the predicted depth map, merge adjacent depth categories, and reduce the number of categories from D to D′=D / r, where r is the scaling ratio.

[0097] (4) Each deep prototype F d It is generated by collecting the features of all pixels X′ and weighting them according to the probability that these pixels belong to the depth category d, as described below.

[0098]

[0099] Among them, X i ′ is X i The i-th pixel feature; is the set of pixels in the feature map; is the normalized probability to the d-th depth prototype; F d is the global information of each depth category; the reconstructed feature F′ can be expressed by the following formula:

[0100]

[0101] (5) Perform multi-scale feature extraction on feature map X. First, a set of 1×1 convolutions is used to obtain feature map X1. Then, a set of 1×1 convolutions, 5×5 convolutions and 3×3 convolutions are used to obtain feature map X2. A set of 1×1 convolutions, followed by 3×3 convolutions are used to obtain feature map X3. A set of pooling layers is used first, followed by 1×1 convolutions to obtain feature map X4. Multi-scale features are concatenated to obtain X e , and connect it with the reconstructed feature F′ to obtain the final fused enhanced deep feature.

[0102] Furthermore, in step 6, the local texture features and pixel-level depth information are combined and input into the fully connected layer, and two loss functions are used for supervision. The specific method is:

[0103] (1) When updating the network parameters, the binary cross entropy loss function and the mean square error function are used to calculate the distance between the local texture and construction depth predicted by the model and the input label. It can be expressed as: Among them, y i is the one-hot label value of the i-th training sample, which is a manual label record. is the predicted value of the i-th neuron, n is the number of face categories, and its value is 2.

[0104] (2) Use the depth prediction map to estimate the depth map of each image, expressed as

[0105] Among them, D i,j and D i ' ,j is the pixel value at position (i,j) and m is the size of the prediction or depth label.

[0106] (3) During the training phase, the binary cross entropy loss function and the mean square error are used to update the network parameters, expressed as: Loss = α·Loss Binary +β·Loss MSE , where Loss is the final loss function. In this method, the values ​​of α and β are set to 0.5.

[0107] (4) For MSE loss function Loss MSE The depth label D used has a binary mask size of 48×48. If the input image is a real face, all values ​​in D are set to 1, otherwise, all values ​​of D are set to 0. The training environment is configured as follows: the framework is the Pytorch framework. The Adam optimizer is used to optimize the network during training, the learning rate is 0.0003, and the batch size is 32.

[0108] Example

[0109] The following classification examples on three face liveness datasets, Replay-Attack, CASIA-FASD and OULU-NPU, are used to illustrate the specific implementation of the present invention.

[0110] (1) Image preprocessing: Use the Dlib face detector to detect and align the face images in the dataset; finally, normalize the face images and resize them to 224*224*3. The dataset is divided into a training set: test set: validation set ratio of 6:2:2.

[0111] (2) Model training: The model parameter initialization specifically includes the following parts: weight decay coefficient is 0.0005; Epoch is set to 300; Batch-size is set to 8. The experimental environment is: system: Ubuntu18.0464bit; processor Intel(R)Xeon(R)Gold 6240R CPU@2.4GHz; memory: 16GB; graphics card: NVIDIA Tesla A10040G; deep learning framework is Pytorch version 1.8.0; compilation language is Python version 3.8.0; cuda version 11.1; Cudnn version 8.0.5;

[0112] (3) This method is compared with other face liveness detection methods. The results are shown in Tables 1, 2, and 3.

[0113] Through comparative experiments, it is found that the method proposed in the present invention requires higher recognition accuracy and can accurately distinguish between real faces and attacking faces.

[0114] Table 1 Results of different methods on the Replay-Attack dataset (unit: %)

[0115]

[0116] Table 2 Results of different methods on the CASIA-FASD dataset (unit: %)

[0117]

[0118] Table 3 Results of different methods on the OULU-NPU dataset (unit: %)

[0119]

[0120]

[0121] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0123] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision, characterized by: The following steps are involved: Step 1: Preprocess the video stream dataset to obtain face images with the same posture and size; Step 2: Design an improved central difference convolution module to extract detailed texture information of the face to enhance the network feature expression capability; Step 3: Encode these image patches using a Swin Transformer with four stages to extract micro features; Step 4: Design a dynamic feature fusion module to combine the feature information of different depth layers in the four stages to construct texture feature descriptors and pixel-level depth descriptors; Step 5: Design a deep feature information enhancement module for deep reasoning, using accurate depth maps as auxiliary supervision to improve the network feature expression capability; Step 6: Combine local texture features and pixel-level depth information into the fully connected layer, and use two loss functions for supervision to achieve liveness detection of the face.

2. The face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision according to claim 1 is characterized in that: In step 1, the face live dataset is cropped to eliminate the background effect. The specific method is as follows: The three face liveness datasets of Replay-Attack, CASIA-FASD and OULU-NPU were cropped to eliminate the background influence. After obtaining the face images, the Dlib face detector was used to detect and align the face images of the datasets. Finally, the face images were normalized and resized to 224×224×3. The datasets were divided into a ratio of training set: test set: validation set = 6:2:

2.

3. The face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision according to claim 1, characterized in that: The step 2 uses central difference convolution to extract detailed texture information of the face to enhance the network feature expression ability. Since the Swin Transformer model and the traditional CNN network have differences in input data dimensions, the data dimensions need to be adjusted accordingly. First, the data dimension input to the module is transformed from to, and then the data is input into the convolution layer to reduce the number of channels to; then, the local features are extracted through the central difference convolution; finally, the convolution layer used is used to restore the dimension, and the residual connection is introduced to prevent model degradation. A batch normalization layer and a ReLU activation function are added after each convolution layer to accelerate model convergence. The specific method is: (1) Calculate the difference between the center point and the surrounding area, as described below: (2) Combining the central difference convolution with the vanilla convolution can obtain stronger feature extraction capabilities, as described below:

4. The face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision according to claim 1, characterized in that: In the step 1, a local texture descriptor and a pixel-level depth map are constructed by a four-stage Swin Transformer. The specific method is as follows: (1) Similar to the traditional Transformer, the processed face image is divided into N non-overlapping small blocks. The specific formula is as follows: X={x1,...,x i ,...,x N } Where N is the number of image blocks, represents the i-th image block, P is x i The height or width of the image blocks are then expanded two-dimensionally and linearly mapped into a matrix of size N×C, where C is equal to P×P×3; (2) After embedding the image blocks, the Swin Transformer module is used to learn the attention features. The Swin Transformer consists of a multi-layer perceptron (MLP), a normalization layer (LP), and a sliding window module. The sliding window module can be divided into a conventional window-based multi-head self-attention layer (W-MSA) and a sliding window-based multi-head self-attention layer (SW-MSA). The specific formula is as follows: in, and z l They are the output features of the (S)W-MSA module and the MLP module, respectively. The MLP consists of two fully connected layers and a GELU nonlinear activation function; (3) In addition, in order to generate more hierarchical representations, these image patches are followed by a patch merging layer, which can reduce the number of patches and expand the channel dimension. As shown in Figure 1, in the first stage layer, each set of features consisting of 2×2 adjacent patches is first connected to a 4×C dimensional vector. Then, the feature map is connected through a linear layer to reduce the number of patches from 4×C to 2×C dimensions. The output resolution formula of the four stages is as follows: Among them, s1, s2, s3 and s4 are the feature map resolutions of the first stage, second stage, third stage and fourth stage respectively.

5. The face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision according to claim 1, characterized in that: In step 4, a dynamic feature fusion module is designed to combine the feature information of different depth layers in the four stages, so as to construct a texture feature descriptor and a pixel-level depth descriptor. The specific method is as follows: (1) The dynamic feature fusion module first performs global average pooling on the output features of different depth layers of the main model, then uses the CA attention mechanism to fuse feature information of four different scales, and then uses the convolutional layer to construct a depth map, as described below: D′=a1·d1+a2·d2+a3·d3+a4·d4 in, CA(·) represents the channel attention operator; Conv represents the convolution operation with activation function; is the predicted depth map, the value of m is 48; Among them, a1, a2, a3, and a4 represent the weights corresponding to the feature information respectively. w i To initialize the weights; (2) Use the output feature maps of different stages to calculate texture features.

6. The face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision according to claim 1, characterized in that: In step 5, a deep feature information enhancement module is designed for deep reasoning, and an accurate depth map is used as auxiliary supervision to improve the network feature expression capability. The specific method is as follows: (1) Given the input features of the backbone network output Predicting discretized deep category maps through two convolutional layers The probability of, where D represents the number of depth categories, and the probability of each pixel represents the confidence that the depth value of the pixel belongs to a certain depth category; (2) Discretize the continuous depth space and classify the depth values ​​using a linear increasing discretization method. The resulting intermediate feature map It is regarded as the initial depth-aware features. At the same time, the features of each pixel are further enhanced by introducing the central representation of each depth category, namely the depth prototype. Each depth prototype is calculated by aggregating the depth-aware features of pixels belonging to the specified category. (3) Use group convolution to process the predicted depth map, merge adjacent depth categories, and reduce the number of categories from D to D′=D / r, where r is the scaling ratio; (4) Each deep prototype F d It is generated by collecting the features of all pixels X′ and weighting them according to the probability that these pixels belong to the depth category d, as described below: Among them, X i ′ is X i The i-th pixel feature; is the set of pixels in the feature map; is the normalized probability to the d-th depth prototype; F d is the global information of each depth category; the reconstructed feature F′ can be expressed by the following formula: (5) Perform multi-scale feature extraction on feature map X. First, a set of 1×1 convolutions is used to obtain feature map X1. Then, a set of 1×1 convolutions, 5×5 convolutions and 3×3 convolutions are used to obtain feature map X2. A set of 1×1 convolutions, followed by 3×3 convolutions are used to obtain feature map X3. A set of pooling layers is used first, followed by 1×1 convolutions to obtain feature map X4. Multi-scale features are concatenated to obtain X e , and connect it with the reconstructed feature F′ to obtain the final fused enhanced deep feature.

7. The face anti-fraud detection method based on multi-scale texture features and pixel-level deep supervision according to claim 1, characterized in that: In step 6, the local texture features and pixel-level depth information are combined and input into the fully connected layer, and two loss functions are used for supervision to realize face liveness detection. The specific method is as follows: (1) When updating the network parameters, the binary cross entropy loss function and the mean square error function are used to calculate the distance between the local texture and construction depth predicted by the model and the input label, expressed as: Among them, y i is the one-hot label value of the i-th training sample, which is a manual label record. is the predicted value of the i-th neuron, n is the number of face categories, and its value is 2; (2) Use the depth prediction map to estimate the depth map of each image, expressed as Among them, D i,j and D i ' ,j is the pixel value at position (i, j), and m is the size of the prediction or depth label; (3) During the training phase, the binary cross entropy loss function and the mean square error are used to update the network parameters, expressed as: Loss = α·Loss Binary +β·Loss MSE , where Loss is the final loss function. In this method, the values ​​of α and β are set to 0.5; (4) For MSE loss function Loss MSE The depth label D used has a binary mask size of 48×48. If the input image is a real face, all values ​​in D are set to 1, otherwise, all values ​​of D are set to 0. The training environment is configured as follows: the framework is the Pytorch framework, the Adam optimizer is used to optimize the network during training, the learning rate is 0.0003, and the batch size is 32.

Images