Face detection method based on multi-scale residual texture convolution and attention mechanism

Abstract

The invention relates to the field of face detection, and discloses a face detection method based on multi-scale residual texture convolution and an attention mechanism, and the method comprises the steps: obtaining a face image; extracting face image features based on a convolutional neural network combining texture information and a multi-scale attention aggregation mechanism; in combination with the extracted features, carrying out living body discrimination on the face image; and warning the detected non-living body face image, and terminating execution of a subsequent face recognition process. The method has the beneficial effects that efficient detection of various face presentation attacks such as printing, playback, 3D masks and masks can be realized. The comprehensive and effective defense on the attack of the mask is realized for the first time; the problem that the technology lacks generality due to the fact that fine-grained face information is not fully considered in an existing research scheme is solved, and the problem that application limitation exists in the technology due to the help of external hardware is solved.

Description

Technical Field

[0001] The present invention relates to the field of face detection, and in particular to a face detection method based on multi-scale residual texture convolution and attention mechanism. Background Art

[0002] Face recognition occupies an important position in the biometrics market due to its huge user base and the advantages of the Internet. Compared with other biometric features, face recognition has the characteristics of uniqueness, easy collection, unchangeability and non-copyability. It has been widely used in mobile phone unlocking, access control systems, financial payments and public security pursuit, and is playing an important role in the construction of smart cities. However, in recent years, the application of face recognition systems in real environments is showing some new security issues. For example, an imposter may deceive the face recognition system by presenting the facial image of a legitimate user. This behavior is also called a face presentation attack.

[0003] In response to the face presentation attack problem, the currently commonly used face recognition liveness detection solutions usually only detect a single face presentation attack type, and it is usually difficult to effectively prevent attacks such as high-quality 3D masks and respirator presentations. It can be seen that face presentation attacks have seriously threatened the security and credibility of existing face recognition systems. In addition, some liveness detection solutions require additional equipment assistance and are complicated to operate, increasing deployment and usage costs, which is inconsistent with the design concept of efficient and convenient face recognition systems. Summary of the invention

[0004] The purpose of the present invention is to propose a face detection method based on multi-scale residual texture convolution and attention mechanism to solve the technical problem that traditional face liveness detection relies heavily on hardware equipment.

[0005] The present invention provides a face detection method based on multi-scale residual texture convolution and attention mechanism, comprising the following steps:

[0006] S1: Get face image;

[0007] S2: Extracting facial image features based on a convolutional neural network that combines texture information with a multi-scale attention aggregation mechanism;

[0008] S3: Combine the extracted features to perform liveness identification on the face image;

[0009] S4: Issue a warning for the detected non-living face image and terminate the subsequent face recognition process.

[0010] A storage medium stores instructions and data for implementing a face detection method based on multi-scale residual texture convolution and attention mechanism.

[0011] A face detection device based on multi-scale residual texture convolution and attention mechanism, comprising: a processor and the storage medium; the processor loads and executes instructions and data in the storage medium to implement a face detection method based on multi-scale residual texture convolution and attention mechanism.

[0012] The beneficial effects provided by the present invention are: the present invention provides a face liveness detection solution combining residual texture convolution with multi-scale attention aggregation mechanism. The present invention introduces residual texture convolution into face liveness detection, and by fusing fine-grained face information such as color texture and skin gloss, and using a multi-scale attention fusion module, it can effectively resist a variety of face presentation attacks, such as printing attacks, replay attacks, 3D mask attacks, and mask presentation attacks. At the same time, the present invention does not require additional hardware equipment, is simple and flexible to operate, and has a short detection time, and has the advantages of being fast and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic flow chart of the method of the present invention;

[0014] Figure 2 Schematic diagram of residual texture convolution block of the present invention;

[0015] Figure 3 Schematic diagram of the multi-scale attention fusion module of the present invention;

[0016] Figure 4 This is a structural diagram of the multi-scale feature aggregation convolutional neural network of the present invention;

[0017] Figure 5 A comparative analysis chart of the printing and replay attack detection capabilities of different systems of the present invention;

[0018] Figure 6 A comparative analysis diagram of 3D mask attack detection capabilities of different systems of the present invention;

[0019] Figure 7 A comparative analysis diagram of attack detection results for masks of different systems of the present invention;

[0020] Figure 8 It is a working schematic diagram of the hardware device of the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0022] Before formally describing the present invention, the scheme of the present invention is first generally described for easy understanding.

[0023] Please refer to Figure 1 , Figure 1It is a schematic flow diagram of the method of the present invention;

[0024] The present invention provides a face detection method based on multi-scale residual texture convolution and attention mechanism, comprising the following steps:

[0025] S1: Get face image;

[0026] It should be noted that during the facial image acquisition process, the system detects the face and the wearing of the mask from the image captured by the camera, and uses a 300×300 image as the input image size of the face detection model. If a face is detected, the face is cropped according to the coordinates of the face bounding box, and then the face key points are located, and a radial transformation is performed based on the face key points to obtain the aligned face image.

[0027] S2: Extracting facial image features based on a convolutional neural network that combines texture information with a multi-scale attention aggregation mechanism;

[0028] It should be noted that in step S2, a multi-scale feature aggregation convolutional neural network combined with a spatial attention mechanism is used, and the residual texture convolution block is used to replace the ordinary convolution operator. By integrating fine-grained facial information such as color texture and skin gloss, it is ultimately possible to achieve efficient detection of various facial presentation attacks such as printing, replay, 3D masks, and masks.

[0029] Among them, the convolutional neural network that combines texture information with multi-scale attention aggregation mechanism includes residual texture convolution blocks of different scales, pooling layers, multi-scale feature connection layers and fully connected layers.

[0030] Please refer to Figure 2 , Figure 2 Schematic diagram of residual texture convolution block of the present invention; the residual texture convolution block is used to replace the ordinary convolution operator. In the residual texture convolution block, the input feature map is used to calculate the fine-grained spatial texture feature and the ordinary convolution feature respectively. The two parts of the feature are connected by a fast connection operation to obtain the output convolution feature, which specifically includes the following steps:

[0031] S21: Traditional convolution feature extraction: First, use a 3×3 ordinary convolution to perform convolution calculation on the input feature map x to obtain the traditional convolution feature

[0032] S22: Spatial texture feature extraction:

[0033] Using the input feature map F, calculate the Weibull differential excitation operator features F Weber and the Sobel space gradient operator feature F Sobel , and its calculation formula is:

[0034]

[0035]

[0036] Among them, v s is the sum of the brightness differences between other pixels and the central pixel in the image neighborhood window, x c is the brightness value of the center pixel of each neighborhood window, G x is the horizontal gradient component of the Sobel operator, G y is the vertical gradient component of the Sobel operator;

[0037] Then connect the above two features along the channel to obtain the initial connection feature F orgin , and use a 1×1 ordinary convolution on F orgin Convolution calculation is performed along the channel, and the calculation formula is:

[0038] F texture =Conv 1×1 (F orgin )=Conv 1×1 (F Weber ⊙F Sobel )

[0039] Among them, Conv 1×1 (·) represents a 1×1 ordinary convolution operation, ⊙ represents the feature connection operation along the channel;

[0040] S23: Feature residual connection, its calculation formula is:

[0041]

[0042] in, represents the batch regularization operation, ⊕ represents the factor addition operation, and F conv3_3 represents the traditional convolution feature, F texture represents the spatial fine-grained texture feature, F RTB Represents the output feature map of the residual texture convolution block.

[0043] After obtaining the output feature map, the spatial attention mechanism is introduced to refine the feature map, and the multi-scale attention fusion module is used to process the feature maps of different convolutional layers.

[0044] Please refer to Figure 3 , Figure 3 Schematic diagram of the multi-scale attention fusion module of the present invention;

[0045] In this invention, a spatial attention mechanism is introduced to refine the feature map, fully retain and transmit valuable discriminant information, and a multi-scale attention fusion module is used to process the feature maps of different convolutional layers. Assuming that the input feature map is F, the output feature map F′ processed by the attention mechanism can be expressed as:

[0046] F′=F⊙(σ(C(A(F)⊕M(F))))

[0047] Among them, ⊙ represents the Hadamard product, C represents the convolution operation used to generate the attention weight map, the size of the convolution kernel will decrease as the scale of the input feature map increases, A(·) represents the average value along the channel, M(·) represents the maximum value along the channel, ⊕ represents the feature concatenation along the channel, and σ represents the Sigmoid activation function, which can limit the size of the feature factor value to the [0,1] interval.

[0048] The specific processing process of the multi-scale feature connection layer is as follows:

[0049] Use bilinear interpolation algorithm to adjust the spatial size of medium and low feature maps to the same size as high-scale features. Figure 1 The high, medium and low scale features are connected in the channel dimension to obtain the connection feature F concat :

[0050] F concst =F high ⊕F mid ⊕F low

[0051] Among them, ⊕ represents the connection operation along the channel.

[0052] Preferably, the present invention provides an embodiment of a multi-scale feature aggregation convolutional neural network structure, please refer to Figure 4 , Figure 4 This is a structural diagram of the multi-scale feature aggregation convolutional neural network of the present invention;

[0053] The input image size is (3, 256, 256), where 3 is the number of channels and 256 is the height and width of the image. In order to extract and transfer the input feature map according to different feature scales, this network is divided into three scale convolution operation sets: high, medium, and low. Each convolution operation set consists of three residual texture convolution blocks with a convolution kernel size of 3×3 and one maximum pooling layer. The residual texture convolution block is used to extract convolution features with fine-grained face information, and the maximum pooling layer is used to reduce the size of the feature map and increase the scale of the feature map. The feature map after the pooling operation will be used as the input feature map of the next scale convolution operation set.

[0054] In order to preserve the multi-scale forgery discrimination information well, the feature maps need to be weighted by the spatial attention module before the mid- and low-scale feature maps are compressed. After the high, mid- and low-scale feature maps are connected along the channel, the connected features are fused using two residual texture convolution blocks and a maximum pooling layer operation to obtain an output feature map F of size (1, 32, 32). output, where 1 is the number of channels of the feature map, and 32 are the height and width of the feature map respectively.

[0055] For the output feature map F output Use two layers of full connection operations to reduce the feature dimension and obtain an output vector V of length 2 output Among them, the output feature map F output In the training phase, it is used to calculate the loss function for pixel-level auxiliary supervision, and the output vector V output It is used for cross entropy loss calculation in the training phase and as a confidence representation of true and false faces in the inference phase.

[0056] At the same time, traditional binary loss supervision and pseudo depth map auxiliary supervision are used to extract the forged features of the attack image in the plane presentation attack scenario. The loss function calculation formula is:

[0057] L surface =α·L BCE +(1-α)·(L CDL +L EDL )

[0058] Among them, L BCE Represents the binary cross entropy loss value, L CDL Represents the comparison depth loss value, L EDL represents the Euclidean distance loss, L surface represents the total loss of the supervision scheme for the plane presentation attack scenario, α is a hyperparameter that adjusts the ratio of binary supervision and pixel-assisted supervision losses, and its value is set to 0.7.

[0059] Using binary mask supervision as pixel-level auxiliary supervision in the 3D mask attack scenario, the loss function calculation formula is:

[0060] L 3D =β·L focal +(1-β)·L mask

[0061] Among them, L focal represents the focus loss value, L mask represents the binary mask supervision loss value, L discri represents the identity-assisted supervision loss value, L 3D represents the total loss of the supervised scheme for the 3D mask attack scenario, β is a hyperparameter for adjusting the ratio, and its value is set to 0.6.

[0062] S3: Combine the extracted features to perform liveness identification on the face image;

[0063] In step S3, during the face liveness detection process, the face liveness detection module performs face liveness detection on the cropped and aligned face image.

[0064] S4: Issue a warning for detected non-living face images and terminate the subsequent face recognition process

[0065] In step S104, if a non-living face image is detected, a warning is issued and the subsequent face recognition process is terminated; if a living face image is detected, the subsequent face recognition process continues to be executed.

[0066] The above method can achieve efficient detection of various face presentation attacks such as printing, replay, 3D masks, and masks.

[0067] Figure 5 Print and replay attack detection capability comparison analysis charts for different systems; Figure 6 The following is a comparative analysis of the 3D mask attack detection capabilities of different systems. The 3DMAD dataset used is composed of real faces and 3D mask attack videos of 17 subjects, with a video resolution of 640×480. The HKBU-MARs V1+ dataset is composed of 170 videos from 12 subjects and their corresponding masks, with a video resolution of 1280×720. Figure 7 A comparative analysis chart of the attack detection results for masks of different systems is shown. The OULU-NPU dataset and HKBU-MARs V1+ dataset used are cropped to obtain the OULU-NPU dataset and HKBU-MARs V1+ dataset that only contain images above the bridge of the nose.

[0068] See also Figure 8 , Figure 8 It is a working diagram of the hardware device of an embodiment of the present invention, and the hardware device specifically includes: a face detection device 401 based on multi-scale residual texture convolution and attention mechanism, a processor 402 and a storage medium 403.

[0069] A face detection device 401 based on multi-scale residual texture convolution and attention mechanism: The face detection device 401 based on multi-scale residual texture convolution and attention mechanism implements the face detection method based on multi-scale residual texture convolution and attention mechanism.

[0070] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the face detection method based on multi-scale residual texture convolution and attention mechanism.

[0071] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the face detection method based on multi-scale residual texture convolution and attention mechanism.

[0072] The beneficial effects of the present invention are as follows: the present invention provides a face liveness detection scheme that combines residual texture convolution with a multi-scale attention aggregation mechanism. The present invention introduces residual texture convolution, and analyzes the authenticity of the face from multiple angles by fusing fine-grained face information such as color texture and skin gloss captured by the Weber operator and the Sobel operator; introduces a spatial attention mechanism to refine the feature map to ensure that valuable discriminant information can be fully retained and transmitted after the feature map size is compressed; at the same time, multi-scale feature aggregation is introduced to splice features of different convolutional layers, aggregate convolutional features from rich scales, and provide rich contextual information for the model to perform face forgery detection; pixel-level auxiliary supervision is performed by comparing the depth loss function, making full use of prior knowledge such as depth and facial reflection, so that the features extracted by the model are closer to the essence of forged faces, and the generalization ability of the model is effectively improved. Through the above-mentioned face liveness detection scheme, the present invention can achieve efficient detection of various face presentation attacks such as printing, replay, 3D masks, and masks. For the first time, comprehensive and effective defense against mask presentation attacks was achieved; the problem of lack of generality in existing research schemes due to insufficient consideration of fine-grained facial information was solved, and the problem of application limitations of the technology due to the reliance on external hardware was improved.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A face detection method based on multi-scale residual texture convolution and attention mechanism, characterized by: The method comprises the following steps: S1: Get face image; S2: Extracting facial image features based on a convolutional neural network that combines texture information with a multi-scale attention aggregation mechanism; S3: Combine the extracted features to perform liveness identification on the face image; S4: Issue a warning for the detected non-living face image and terminate the subsequent face recognition process.

2. The face detection method based on multi-scale residual texture convolution and attention mechanism as claimed in claim 1, characterized in that: In step S2, the convolutional neural network that combines texture information with a multi-scale attention aggregation mechanism includes residual texture convolution blocks of different scales, a pooling layer, a multi-scale feature connection layer, and a fully connected layer.

3. The face detection method based on multi-scale residual texture convolution and attention mechanism as claimed in claim 1, characterized in that: The processing process of the residual texture convolution blocks of different scales in step S2 is as follows: S21: Traditional convolution feature extraction: First, use a 3×3 ordinary convolution to perform convolution calculation on the input feature map x to obtain the traditional convolution feature S22: Spatial texture feature extraction: Using the input feature map F, calculate the Weibull differential excitation operator features F Weber and the Sobel space gradient operator feature F Sobel , and its calculation formula is: Among them, v s is the sum of the brightness differences between other pixels and the central pixel in the image neighborhood window, x c is the brightness value of the center pixel of each neighborhood window, G x is the horizontal gradient component of the Sobel operator, G y is the vertical gradient component of the Sobel operator; Then connect the above two features along the channel to obtain the initial connection feature F orgin , and use a 1×1 ordinary convolution on F orgin Convolution calculation is performed along the channel, and the calculation formula is: F texture =Conv 1×1 (F orgin )=Conv 1×1 (F Weber ⊙F Sobel ) Among them, Conv 1×1 (·) represents a 1×1 ordinary convolution operation, ⊙ represents the feature connection operation along the channel; S23: Feature residual connection, its calculation formula is: in, represents the batch regularization operation, represents the factor addition operation, F conv3_3 represents the traditional convolution feature, F texture represents the spatial fine-grained texture feature, F RTB Represents the output feature map of the residual texture convolution block.

4. The face detection method based on multi-scale residual texture convolution and attention mechanism as claimed in claim 3, characterized in that: After obtaining the output feature map, the spatial attention mechanism is introduced to refine the feature map, and the multi-scale attention fusion module is used to process the feature maps of different convolutional layers, as follows: Where F represents the input feature map, F′ represents the output feature map processed by the attention mechanism, ⊙ represents the Hadamard product, c represents the convolution operation used to generate the attention weight map, and the size of the convolution kernel will decrease as the scale of the input feature map increases. A(·) represents the average value along the channel, and M(·) represents the maximum value along the channel. represents the concatenation of features along the channel, and σ represents the Sigmoid activation function.

5. The face detection method based on multi-scale residual texture convolution and attention mechanism as claimed in claim 4, characterized in that: The specific processing process of the multi-scale feature connection layer is as follows: Use the bilinear interpolation algorithm to adjust the spatial size of the medium and low feature maps to be consistent with the high-scale feature map, and connect the high, medium, and low scale features in the channel dimension to obtain the connected feature F concat : in, Represents a join operation along a channel.

6. The face detection method based on multi-scale residual texture convolution and attention mechanism as claimed in claim 1, characterized in that: The loss function of the convolutional neural network that combines texture information with the multi-scale attention aggregation mechanism includes: the loss function L for planar scenes surface And the loss function L for the 3D mask attack scenario 3D .

7. The face detection method based on multi-scale residual texture convolution and attention mechanism as claimed in claim 6, characterized in that: Loss function L for planar scenes surface At the same time, traditional binary loss supervision and pseudo depth map auxiliary supervision are used as follows: L surface =α·L BCE +(1-a)·(L CDL +L EDL ) Among them, L BCE Represents the binary cross entropy loss value, L CDL Represents the comparison depth loss value, L EDL represents the Euclidean distance loss, and α is a hyperparameter that adjusts the ratio of binary supervision and pixel-assisted supervision losses.

8. The face detection method based on multi-scale residual texture convolution and attention mechanism as claimed in claim 1, characterized in that: Loss function L for 3D mask attack scenario 3D Use binary mask supervision as follows: L 3D =β·L focal +(1-β)·L mask Among them, L focal represents the focus loss value, L mask represents the binary mask supervision loss value, L discri represents the identity-assisted supervision loss value, and β is a hyperparameter for adjusting the proportion.

9. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a face detection method based on multi-scale residual texture convolution and attention mechanism as described in any one of claims 1 to 8.

10. A face detection device based on multi-scale residual texture convolution and attention mechanism, characterized by: include: A processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a face detection method based on multi-scale residual texture convolution and attention mechanism as described in any one of claims 1 to 8.

Images