A high-resolution face anti-spoofing method and system for effectively learning face details

Abstract

The present invention relates to the field of deep learning and computer vision technology, and in particular to a high-resolution face anti-spoofing method and system for effectively learning face details, wherein the method comprises the following steps: S1, through an adaptive feature enhancement network, feature weighting is performed on a face image and its corresponding depth map and reflective map; S2, through a diagonal feature fusion network, feature extraction is performed on the feature-weighted face image to obtain a discriminative face local detail feature map; S3, through a deception area detection network, the face local detail feature map is amplified, and face deception feature areas are extracted in a high-dimensional space; S4, through a fully connected layer and a SoftMax classifier, the face deception feature area is processed to determine whether the face is a deception face. The present invention can improve the recognition of face deception areas and further improve the efficiency of face liveness detection.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and computer vision technology, and in particular to a high-resolution face anti-spoofing method and system for effectively learning face details. Background Art

[0002] Due to the convenience and excellent accuracy of face recognition technology, it has been widely used in some interactive intelligent applications, such as attendance check-in, remote identity authentication and mobile payment. Existing face liveness detection methods can be divided into two categories, traditional methods based on hand-crafted features and methods based on deep learning. Methods based on hand-crafted features require rich task-aware prior knowledge for design, and the physiological activity cues produced are usually captured in long-term interactive face videos, which is very inconvenient for actual deployment. Benefiting from advanced convolutional neural networks and the recently released large-scale face anti-spoofing datasets, face liveness detection methods based on deep learning use a large amount of face data as support. Different datasets include multiple protocols to meet the training needs in different scenarios, and also achieve the best performance in face liveness detection tasks. Summary of the invention

[0003] The first purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a high-resolution face anti-spoofing method that effectively learns facial details. The feature information of the input face is enriched by adaptive feature enhancement, overcoming the problem of single RGB face image information. A diagonal feature fusion method is used in the enhanced face image, and feature fusion of non-overlapping face areas is utilized to capture richer face context information. At the same time, a deception area detection network is designed, which repeatedly amplifies the local feature information of the face image through a carefully designed attention mechanism, thereby improving the recognition of the face deception area and further improving the efficiency of face liveness detection.

[0004] A second object of the present invention is to provide a high-resolution face anti-spoofing system that effectively learns facial details.

[0005] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A high-resolution face anti-spoofing method for effectively learning face details comprises the following steps:

[0007] S1, feature weighting of the face image and its corresponding depth image and reflection image through an adaptive feature enhancement network;

[0008] S2. Through the diagonal feature fusion network, feature extraction is performed on the feature-weighted face image to obtain a discriminative face local detail feature map; wherein the diagonal feature fusion network includes three layers, the first layer includes an Embedding layer with a convolution kernel size of 16×16 and a step size of 16; the second layer is a diagonal feature fusion module composed of self-attention submodules with a convolution kernel size of 2×2, and the diagonal feature fusion module fuses face context information; the third layer performs matrix conversion and outputs face feature information;

[0009] S3, through the deception area detection network, the local detail feature map of the face is enlarged, and the face deception feature area is extracted in the high-dimensional space;

[0010] S4. Through the fully connected layer and the SoftMax classifier, the face deception feature area is processed to determine whether the face is a deceptive face.

[0011] Preferably, step S1 comprises the following steps:

[0012] S101, transforming the face image to generate auxiliary information, wherein the auxiliary information includes a depth map and a reflection map corresponding to the face image, wherein the reflection map corresponding to the real face is 0, and the depth map corresponding to the spoofed face is 0:

[0013]

[0014] Among them, F is a face image, Y is the label of the face image F, is the depth map of the face, |D| is the normalization factor, which normalizes the value of the depth map to the range of [0, 1];

[0015]

[0016] Among them, F is a face image, Y is the label of the face image F, is the reflection map of the face, |R| is the normalization factor;

[0017] S102, performing a binarization operation on the face image and the corresponding depth image and reflection image to obtain a binarized image;

[0018] S103, extracting the binarized depth map and the reflectance map through a convolution operation to extract feature information of the depth map, a weight of the depth map, feature information of the reflectance map, and a weight of the reflectance map;

[0019] S104: Fusing feature information of the depth map according to the weight of the depth map and feature information of the reflectance map according to the weight of the reflectance map into the original face input to obtain a feature-weighted face map.

[0020] Preferably, step S2 comprises the following steps:

[0021] S201, for the face image after feature enhancement Where X is the face image after feature weighting, is a real number, H is the height, W is the width, and C is the number of channels, which is divided into multiple windows through the Embedding layer;

[0022] S202. Propose a 2×2 convolution to use all adjacent keys in a 3×3 network And the diagonal keys K are added in pairs to enhance the facial feature information carried by each key. The relationship between the key value and the index value can be expressed as:

[0023]

[0024] Q=XW q

[0025] Among them, α represents the face feature map after cross-region fusion, δ represents the SoftMax operation, K represents the fusion of diagonal keys, K′ represents the fusion of another set of diagonal keys, T represents matrix transposition, d represents the dimension of the index value, Q represents the face feature information, and W q represents a 1×1 convolution matrix, and X represents the face image after feature enhancement;

[0026] S203, converting the cross-region fused face feature map α into a face map feature matrix V through an embedding matrix 1×1 convolution, and fusing the static background information through an attention mechanism to obtain a discriminative face local detail feature map Y, which can be expressed as:

[0027]

[0028] Among them, Y represents the local detail feature map of the face, V represents the feature matrix of the converted face map, and is the parameter required for the operation of the diagonal feature fusion network. Represents a matrix multiplication operation in a 3×3 local grid in space.

[0029] Preferably, step S3 comprises the following steps:

[0030] S301, inputting the local detail feature map of the face into the deception area detection network, and locking the face deception feature area through the attention mechanism;

[0031] S302, inputting the locked face deception feature area into a high-dimensional space for amplification;

[0032] S303. Repeat steps S301-S302 three times to obtain the final face deception feature area.

[0033] Preferably, step S4 comprises the following steps:

[0034] S401, after the final face deception feature area is input into the fully connected layer, it is input into the SoftMax classifier for classification to obtain the probability score of the face image being a real face and the probability score of the deception face;

[0035] S402: Compare the probability score of the real face with the probability score of the deceptive face, and output the category with the larger probability score.

[0036] A high-resolution face anti-spoofing system that effectively learns facial details, including:

[0037] An adaptive feature enhancement network unit is used to perform feature weighting on the face image and its corresponding depth image and reflectance image;

[0038] The diagonal feature fusion network unit is used to extract features from the weighted face image to obtain a discriminative local detail feature map of the face;

[0039] The deception area detection network unit is used to amplify the local detail feature map of the face and extract the face deception feature area in the high-dimensional space;

[0040] The classifier unit is used to process the face deception feature area to determine whether the face is a deceptive face.

[0041] Preferably, the adaptive feature enhancement network unit comprises:

[0042] Transformation module, used to transform the face image and generate auxiliary information;

[0043] A binarization module is used to perform a binarization operation on the face image and the corresponding depth image and reflection image to obtain a binarized image;

[0044] An extraction module is used to extract the binarized depth map and reflectance map through a convolution operation to extract feature information of the depth map, a weight of the depth map, feature information of the reflectance map, and a weight of the reflectance map;

[0045] The fusion module is used to fuse the feature information of the depth map according to the weight of the depth map and the feature information of the reflectance map according to the weight of the reflectance map into the original face input to obtain a feature-weighted face map.

[0046] Preferably, the diagonal feature fusion network unit comprises:

[0047] Division module, for face images after feature enhancement Where X is the face image after feature weighting, is a real number, H is the height, W is the width, and C is the number of channels, which is divided into multiple windows through the Embedding layer;

[0048] Enhancement module, which proposes a 2×2 convolution to use all adjacent keys in a 3×3 network And the diagonal keys K are added in pairs to enhance the facial feature information carried by each key;

[0049] The conversion module is used to convert the cross-region fused face feature map α into a face map feature matrix V through an embedding matrix 1×1 convolution, and fuse the static background information through the attention mechanism to obtain a discriminative face local detail feature map Y.

[0050] Preferably, the deception area detection network unit includes:

[0051] The input module is used to input the local detail feature map of the face into the deception area detection network and lock the face deception feature area through the attention mechanism;

[0052] A magnification module inputs the locked face deception feature area into a high-dimensional space for magnification;

[0053] The repetitive module is used to input the local detail feature map of the face into the deception area detection network, lock the face deception feature area through the attention mechanism, input the locked face deception feature area into the high-dimensional space for amplification and repeat three times to obtain the final face deception feature area.

[0054] Preferably, the classifier unit comprises:

[0055] The scoring module is used to input the final face deception feature area into the fully connected layer and then into the SoftMax classifier for classification to obtain the probability score of the face image being a real face and the probability score of the face image being a deception face;

[0056] The output module is used to compare the probability scores of real faces and spoofed faces, and output the category with the larger probability score.

[0057] One of the above technical solutions has the following beneficial effects:

[0058] (1) Compared with other methods, the adaptive feature enhancement method for auxiliary information of face images proposed in the present invention enriches the original input face features to a certain extent, so that the input face features carry richer face feature information, while strengthening the weights of effective features and weakening useless features. Therefore, better test results are achieved under the training of the same number of data sets.

[0059] (2) The present invention adopts a diagonal feature fusion network including a diagonal feature fusion module composed of self-attention sub-modules, and replaces the feature extraction part therein with a self-designed diagonal feature fusion algorithm. Since the diagonal feature fusion network has a rich structure and a deep number of layers, it can fully extract the facial feature information of the feature-weighted face image.

[0060] (3) Based on the fine-grained pixel deception segmentation of visual attention mechanism information, the present invention proposes a deception region detection network to compare the fine-grained features of the face, and activate the face deception region based on the features, amplify the local detail features of the face, and after repeating three times, input the final face deception feature region into the classifier unit and use it as one of the criteria for the final classification result, further improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a flowchart of the overall steps of the high-resolution face anti-spoofing method for effectively learning face details of the present invention;

[0062] Figure 2 It is a flow chart of step S1 of the high-resolution face anti-spoofing method for effectively learning face details of the present invention;

[0063] Figure 3 It is a flowchart of step S2 of the high-resolution face anti-spoofing method for effectively learning face details of the present invention;

[0064] Figure 4 It is a flowchart of step S3 of the high-resolution face anti-spoofing method for effectively learning face details of the present invention;

[0065] Figure 5 It is a flowchart of step S4 of the high-resolution face anti-spoofing method for effectively learning face details of the present invention;

[0066] Figure 6 It is a schematic diagram of the structure of the high-resolution face anti-spoofing system of the present invention that effectively learns face details; DETAILED DESCRIPTION

[0067] The technical solution of the present invention is further described below with reference to the accompanying drawings and through specific implementation methods.

[0068] like Figure 1 As shown, a high-resolution face anti-spoofing method for effectively learning face details includes the following steps:

[0069] S1. Through the adaptive feature enhancement network, feature weighting is performed on the face image and its corresponding depth image and reflection image; Figure 2 As shown, the specific steps include:

[0070] S101, transforming the face image to generate auxiliary information, wherein the auxiliary information includes a depth map and a reflection map corresponding to the face image, wherein the reflection map corresponding to the real face is 0, and the depth map corresponding to the spoofed face is 0:

[0071]

[0072] Among them, F is a face image, Y is the label of the face image F, is the depth map of the face, |D| is the normalization factor, which normalizes the value of the depth map to the range of [0, 1];

[0073]

[0074] Among them, F is a face image, Y is the label of the face image F, is the reflection map of the face, |R| is the normalization factor;

[0075] S102, performing a binarization operation on the face image and the corresponding depth image and reflection image to obtain a binarized image;

[0076] S103, extracting the binarized depth map and the reflectance map through a convolution operation to extract feature information of the depth map, a weight of the depth map, feature information of the reflectance map, and a weight of the reflectance map;

[0077] S104: Fusing feature information of the depth map according to the weight of the depth map and feature information of the reflectance map according to the weight of the reflectance map into the original face input to obtain a feature-weighted face map.

[0078] Further explanation, S2, through the diagonal feature fusion network, feature extraction is performed on the feature-weighted face image to obtain a discriminative face local detail feature map; wherein, the diagonal feature fusion network includes three layers, the first layer includes an Embedding layer with a convolution kernel size of 16×16 and a step size of 16; the second layer is a diagonal feature fusion module composed of self-attention submodules with a convolution kernel size of 2×2, and the diagonal feature fusion module fuses face context information; the third layer performs matrix conversion and outputs face feature information; Figure 3 As shown, the specific steps include:

[0079] S201, for the face image after feature enhancement Where X is the face image after feature weighting, is a real number, H is the height, W is the width, and C is the number of channels, which is divided into multiple windows through the Embedding layer;

[0080] S202. Propose a 2×2 convolution to use all adjacent keys in a 3×3 network And the diagonal keys K are added in pairs to enhance the facial feature information carried by each key. The relationship between the key value and the index value can be expressed as:

[0081]

[0082] Q=XW q

[0083] Among them, α represents the face feature map after cross-region fusion, δ represents the SoftMax operation, K represents the fusion of diagonal keys, K′ represents the fusion of another set of diagonal keys, T represents matrix transposition, d represents the dimension of the index value, Q represents the face feature information, and W q represents a 1×1 convolution matrix, and X represents the face image after feature enhancement;

[0084] S203, converting the cross-region fused face feature map α into a face map feature matrix V through an embedding matrix 1×1 convolution, and fusing the static background information through an attention mechanism to obtain a discriminative face local detail feature map Y, which can be expressed as:

[0085]

[0086] Among them, Y represents the local detail feature map of the face, V represents the feature matrix of the converted face map, and is the parameter required for the operation of the diagonal feature fusion network. Represents a matrix multiplication operation in a 3×3 local grid in space.

[0087] To further illustrate, S3, through the deception area detection network, the local detail feature map of the face is enlarged, and the face deception feature area is extracted in the high-dimensional space; Figure 4 As shown, the specific steps include:

[0088] S301, inputting the local detail feature map of the face into the deception area detection network, and locking the face deception feature area through the attention mechanism;

[0089] S302, inputting the locked face deception feature area into a high-dimensional space for amplification;

[0090] S303. Repeat steps S301-S302 three times to obtain the final face deception feature area.

[0091] To further illustrate, S4, through the fully connected layer and the SoftMax classifier, the face deception feature area is processed to determine whether the face is a deceptive face; Figure 5 As shown, the specific steps include:

[0092] S401, after the final face deception feature area is input into the fully connected layer, it is input into the SoftMax classifier for classification to obtain the probability score of the face image being a real face and the probability score of the deception face;

[0093] S402: Compare the probability score of the real face with the probability score of the deceptive face, and output the category with the larger probability score.

[0094] like Figure 6 As shown, a high-resolution face anti-spoofing system that effectively learns face details includes:

[0095] An adaptive feature enhancement network unit is used to perform feature weighting on the face image and its corresponding depth image and reflectance image;

[0096] The diagonal feature fusion network unit is used to extract features from the weighted face image to obtain a discriminative local detail feature map of the face;

[0097] The deception area detection network unit is used to amplify the local detail feature map of the face and extract the face deception feature area in the high-dimensional space;

[0098] The classifier unit is used to process the face deception feature area to determine whether the face is a deceptive face.

[0099] To further illustrate, the adaptive feature enhancement network unit includes:

[0100] Transformation module, used to transform the face image and generate auxiliary information;

[0101] A binarization module is used to perform a binarization operation on the face image and the corresponding depth image and reflection image to obtain a binarized image;

[0102] An extraction module is used to extract the binarized depth map and reflectance map through a convolution operation to extract feature information of the depth map, a weight of the depth map, feature information of the reflectance map, and a weight of the reflectance map;

[0103] The fusion module is used to fuse the feature information of the depth map according to the weight of the depth map and the feature information of the reflectance map according to the weight of the reflectance map into the original face input to obtain a feature-weighted face map.

[0104] To further illustrate, the diagonal feature fusion network unit includes:

[0105] Division module, for face images after feature enhancement Where X is the face image after feature weighting, is a real number, H is the height, W is the width, and C is the number of channels, which is divided into multiple windows through the Embedding layer;

[0106] Enhancement module, which proposes a 2×2 convolution to use all adjacent keys in a 3×3 network And the diagonal keys K are added in pairs to enhance the facial feature information carried by each key;

[0107] The conversion module is used to convert the cross-region fused face feature map α into a face map feature matrix V through an embedding matrix 1×1 convolution, and fuse the static background information through the attention mechanism to obtain a discriminative face local detail feature map Y.

[0108] Further explanation, the deception area detection network unit includes:

[0109] The input module is used to input the local detail feature map of the face into the deception area detection network and lock the face deception feature area through the attention mechanism;

[0110] A magnification module inputs the locked face deception feature area into a high-dimensional space for magnification;

[0111] The repetitive module is used to input the local detail feature map of the face into the deception area detection network, lock the face deception feature area through the attention mechanism, input the locked face deception feature area into the high-dimensional space for amplification and repeat three times to obtain the final face deception feature area.

[0112] To further illustrate, the classifier unit includes:

[0113] The scoring module is used to input the final face deception feature area into the fully connected layer and then into the SoftMax classifier for classification to obtain the probability score of the face image being a real face and the probability score of the face image being a deception face;

[0114] The output module is used to compare the probability scores of real faces and spoofed faces, and output the category with the larger probability score.

[0115] The technical principle of the present invention is described above in conjunction with specific embodiments. These descriptions are only for explaining the principle of the present invention and cannot be interpreted as limiting the scope of protection of the present invention in any way. Based on the explanations herein, those skilled in the art can associate other specific embodiments of the present invention without creative work, and these equivalent variations or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A high-resolution face anti-spoofing method for effectively learning facial details, characterized in that: The following steps are involved: S1, feature weighting of the face image and its corresponding depth image and reflection image through an adaptive feature enhancement network; S2. Through the diagonal feature fusion network, feature extraction is performed on the feature-weighted face image to obtain a discriminative face local detail feature map; wherein the diagonal feature fusion network includes three layers, the first layer includes an Embedding layer with a convolution kernel size of 16×16 and a step size of 16; the second layer is a diagonal feature fusion module composed of self-attention submodules with a convolution kernel size of 2×2, and the diagonal feature fusion module fuses face context information; the third layer performs matrix conversion and outputs face feature information; S3, through the deception area detection network, the local detail feature map of the face is enlarged, and the face deception feature area is extracted in the high-dimensional space; S4, through the fully connected layer and the SoftMax classifier, the face deception feature area is processed to determine whether the face is a deceptive face; The step S1 comprises the following steps: S101, transforming the face image to generate auxiliary information, wherein the auxiliary information includes a depth map and a reflection map corresponding to the face image, wherein the reflection map corresponding to the real face is 0, and the depth map corresponding to the spoofed face is 0: Among them, F is a face image, Y is the label of the face image F, is the depth map of the face, is the normalization factor, which normalizes the value of the depth map to the range of [0,1]; Among them, F is a face image, Y is the label of the face image F, It is a reflection of a human face. is the normalization factor; S102, performing a binarization operation on the face image and the corresponding depth image and reflection image to obtain a binarized image; S103, extracting the binarized depth map and the reflectance map through a convolution operation to extract feature information of the depth map, a weight of the depth map, feature information of the reflectance map, and a weight of the reflectance map; S104, fusing feature information of the depth map according to the weight of the depth map, and feature information of the reflectance map according to the weight of the reflectance map into the original face input, to obtain a feature-weighted face map; The step S2 comprises the following steps: S201, for the face image after feature enhancement ,in is the face image after feature weighting, is a real number, H is the height, W is the width, and C is the number of channels, which is divided into multiple windows through the Embedding layer; S202. Propose a 2×2 convolution to use all adjacent keys in a 3×3 network , and the diagonal keys Add two by two to enhance the facial feature information carried by each key. The relationship between the key value and the index value can be expressed as: in represents the face feature map after cross-region fusion, represents the SoftMax operation, K represents the fusion of diagonal keys, represents the fusion of another set of diagonal keys, T represents matrix transpose, d represents the dimension of the index value, and Q represents the facial feature information. express Convolution matrix, X represents the face image after feature enhancement; S203, embedding the face feature map after cross-region fusion through 1×1 convolution of the matrix It is converted into a face feature matrix V, and the static background information is fused through the attention mechanism to obtain a discriminative face local detail feature map Y, which can be expressed as: Among them, Y represents the local detail feature map of the face, V represents the feature matrix of the converted face map, and is the parameter required for the operation of the diagonal feature fusion network. Indicates space Matrix multiplication operation in a local grid.

2. A high-resolution face anti-spoofing method for effectively learning face details according to claim 1, characterized in that: The step S3 comprises the following steps: S301, inputting the local detail feature map of the face into the deception area detection network, and locking the face deception feature area through the attention mechanism; S302, inputting the locked face deception feature area into a high-dimensional space for amplification; S303. Repeat steps S301-S302 three times to obtain the final face deception feature area.

3. A high-resolution face anti-spoofing method for effectively learning face details according to claim 2, characterized in that: The step S4 comprises the following steps: S401, after the final face deception feature area is input into the fully connected layer, it is input into the SoftMax classifier for classification to obtain the probability score of the face image being a real face and the probability score of the deception face; S402: Compare the probability score of the real face with the probability score of the deceptive face, and output the category with the larger probability score.

4. A high-resolution face anti-spoofing system that effectively learns face details, characterized in that: A high-resolution face anti-spoofing method for effectively learning face details as described in any one of claims 1 to 3, comprising: An adaptive feature enhancement network unit is used to perform feature weighting on the face image and its corresponding depth image and reflectance image; The diagonal feature fusion network unit is used to extract features from the weighted face image to obtain a discriminative local detail feature map of the face; The deception area detection network unit is used to amplify the local detail feature map of the face and extract the face deception feature area in the high-dimensional space; A classifier unit, used for processing the face deception feature area to determine whether the face is a deceptive face; The adaptive feature enhancement network unit comprises: Transformation module, used to transform the face image and generate auxiliary information; A binarization module is used to perform a binarization operation on the face image and the corresponding depth image and reflection image to obtain a binarized image; An extraction module is used to extract the binarized depth map and reflectance map through a convolution operation to extract feature information of the depth map, a weight of the depth map, feature information of the reflectance map, and a weight of the reflectance map; A fusion module is used to fuse the feature information of the depth map according to the weight of the depth map, and the feature information of the reflectance map according to the weight of the reflectance map into the original face input to obtain a feature-weighted face image; The diagonal feature fusion network unit comprises: Division module, for face images after feature enhancement ,in is the face image after feature weighting, is a real number, H is the height, W is the width, and C is the number of channels, which is divided into multiple windows through the Embedding layer; Enhancement module, which proposes a 2×2 convolution to use all adjacent keys in a 3×3 network , and the diagonal keys Add two by two to enhance the facial feature information carried by each key; The conversion module is used to embed the cross-region fused face feature map through a 1×1 convolution of the matrix It is converted into a face image feature matrix V, and the static background information is fused through the attention mechanism to obtain a discriminative face local detail feature map Y.

5. A high-resolution face anti-spoofing system for effectively learning face details according to claim 4, characterized in that: The deception area detection network unit comprises: The input module is used to input the local detail feature map of the face into the deception area detection network and lock the face deception feature area through the attention mechanism; A magnification module inputs the locked face deception feature area into a high-dimensional space for magnification; The repetitive module is used to input the local detail feature map of the face into the deception area detection network, lock the face deception feature area through the attention mechanism, input the locked face deception feature area into the high-dimensional space for amplification and repeat three times to obtain the final face deception feature area.

6. A high-resolution face anti-spoofing system for effectively learning face details according to claim 5, characterized in that: The classifier unit comprises: The scoring module is used to input the final face deception feature area into the fully connected layer and then into the SoftMax classifier for classification to obtain the probability score of the face image being a real face and the probability score of the face image being a deception face; The output module is used to compare the probability scores of real faces and spoofed faces, and output the category with the larger probability score.

Images