A face living body detection model generation method and device, and an electronic device

Abstract

The application discloses a face living body detection model generation scheme, which comprises the following steps: inputting training samples into a pre-created twin neural network; generating augmented face images by augmenting the training samples through a first branch and a second branch of the twin neural network; inputting the augmented face images into the two branches respectively to obtain first face features, second face features, a first depth map and a second depth map; calculating an encoding level dense consistency loss according to the first face features and the second face features; calculating a prediction level consistency loss according to the first depth map and the second depth map; determining a total loss of the twin neural network according to the encoding level dense consistency loss and the prediction level consistency loss; and generating a face living body detection model by optimizing the parameters of the twin neural network according to the total loss by using a stochastic gradient descent method. The face living body detection model generated by the scheme has high face recognition result accuracy and strong universality.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and in particular to a method and apparatus for generating a facial liveness detection model, and an electronic device. Background Technology

[0002] With the advancement of computer vision technology, facial recognition has become a primary method of biometric authentication. Facial recognition systems have been successfully applied in various scenarios, such as access control systems and electronic payments. However, the security of these systems remains a hot topic of concern in the industry. Attackers can present deceptive faces to the system, such as through printing attacks, video playback attacks, and 3D mask attacks, to impersonate users and mislead the system to gain legitimate access. Therefore, facial liveness detection systems are an important component of facial recognition systems. Commonly used facial liveness detection methods include:

[0003] Traditional methods utilize artificially created facial features, such as local binary patterns, scale-invariant feature transformations, gradient histograms, speed-up robust features, and Gaussian difference functions, to capture deceptive features in facial images. Furthermore, dynamic cues such as motion features, eye blinks, and subtle movements are used to extract fraudulent features from videos. Support vector machines are then used to perform binary classification on these features to determine whether the input facial image is a genuine face.

[0004] Early deep learning-based face liveness detection methods directly used cross-entropy for binary classification. However, binary classification tasks are prone to losing important information or capturing incorrect classification features. Then, depth map-supervised networks emerged, which can capture more fine-grained information through full-pixel supervision. Based on depth map supervision, researchers have successively developed operators or auxiliary remote photoplethysmography pulse waves suitable for depth map supervision to improve the accuracy of face liveness detection.

[0005] However, face liveness detection models experience severe performance degradation and poor accuracy when applied to data from different domains or to unseen attack methods. This demonstrates the poor universality of existing face liveness detection models. Summary of the Invention

[0006] The purpose of this invention is to provide a method, apparatus, and electronic device for generating a face liveness detection model, which can solve the problem of poor universality of existing face liveness detection models.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] This invention provides a method for generating a face liveness detection model, comprising: inputting training samples into a pre-created Siamese neural network; wherein the Siamese neural network includes a first branch and a second branch, the first branch including a first feature encoder, a feature converter, and a first classifier; and the second branch including a second feature encoder and a second classifier.

[0009] The training samples are augmented using the first branch and the second branch to generate augmented face images.

[0010] The augmented face image is input into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map and the second depth map;

[0011] Based on the first facial feature and the second facial feature, the coding-level dense consistency loss of the Siamese neural network is calculated;

[0012] Based on the first depth map and the second depth map, calculate the prediction level consistency loss of the Siamese neural network;

[0013] The total loss of the Siamese neural network is determined based on the coding-level dense consistency loss and the prediction-level consistency loss.

[0014] The stochastic gradient descent method is used to optimize the parameters of the twin neural network based on the total loss, thereby generating a face liveness detection model.

[0015] Optionally, the step of inputting the augmented face image into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map, and the second depth map includes:

[0016] The augmented face image is input into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature;

[0017] The first facial feature is input into the first classifier to obtain the first depth map, and the second facial feature is input into the second classifier to obtain the second depth map.

[0018] Optionally, the step of augmenting the training samples through the first branch and the second branch to generate an augmented face image includes:

[0019] The training sample is input into the first branch, and the first branch augments the training sample according to the first perspective to obtain the augmented first face image.

[0020] The training sample is input into the second branch, which augments the training sample according to the second perspective to obtain the augmented second face image.

[0021] Optionally, the step of inputting the augmented face image into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature includes:

[0022] The augmented first face image is input into the first feature encoder to obtain three levels of face features: low, medium, and high. The three levels of face features are then concatenated and input into the feature converter to generate the first face feature.

[0023] The augmented second face image is input into the second feature encoder to obtain three levels of face features: low, medium, and high. The three levels of face features are then stitched together to generate the second face feature.

[0024] Optionally, after the step of generating the face liveness detection model, the method further includes:

[0025] The face image to be identified is input into the second feature encoder of the face liveness detection model to obtain three levels of face features: low, medium, and high. The three levels of face features are then concatenated to generate a third face feature.

[0026] The third facial feature is input into the second classifier to obtain the third depth map;

[0027] Based on the third depth map and the preset threshold, it is determined whether the face image to be identified is a real face image.

[0028] Optionally, after the step of generating the face liveness detection model, the method further includes:

[0029] The face image to be identified is input into the first branch and the second branch of the face liveness detection model, respectively;

[0030] Based on the prediction results obtained from the first branch and the prediction results obtained from the second branch, it is determined whether the face image to be identified is a real face image.

[0031] Optionally, the first feature encoder and the second feature encoder have the same structure, both including:

[0032] Input layer, first convolutional module, second convolutional module, third convolutional module, first spoofing attention module, second spoofing attention module, third spoofing attention module, first max pooling layer, second max pooling layer, third max pooling layer;

[0033] The outputs of the input layer are respectively input to the first convolutional module and the first fraudulent attention module. The outputs of the first convolutional module and the first fraudulent attention module are added together and then input to the first max pooling layer to generate low-level features. The low-level features are input to the second convolutional module and the second fraudulent attention module. The outputs of the second convolutional module and the second fraudulent attention module are added together and then input to the second max pooling layer to generate medium-level features. The medium-level features are input to the third convolutional module and the third fraudulent attention module. The outputs of the third convolutional module and the third fraudulent attention module are input to the third max pooling layer to obtain high-level features.

[0034] This invention also provides a face liveness detection model generation device, comprising:

[0035] The first input module is used to input training samples into a pre-created Siamese neural network; wherein the Siamese neural network includes a first branch and a second branch, the first branch includes a first feature encoder, a feature converter, and a first classifier; the second branch includes a second feature encoder and a second classifier;

[0036] An augmentation module is used to augment the training samples through the first branch and the second branch to generate an augmented face image;

[0037] The second input module is used to input the augmented face image into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map and the second depth map;

[0038] The first calculation module is used to calculate the coding-level dense consistency loss of the Siamese neural network based on the first facial feature and the second facial feature.

[0039] The second calculation module is used to calculate the prediction level consistency loss of the Siamese neural network based on the first depth map and the second depth map.

[0040] The total loss determination module is used to determine the total loss of the Siamese neural network based on the encoding-level dense consistency loss and the prediction-level consistency loss.

[0041] The parameter tuning module is used to optimize the parameters of the Siamese neural network based on the total loss using the stochastic gradient descent method to generate a face liveness detection model.

[0042] Optionally, the second input module includes:

[0043] The first submodule is used to input the augmented face image into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature;

[0044] The second submodule is used to input the first facial features into the first classifier to obtain a first depth map, and to input the second facial features into the second classifier to obtain a second depth map.

[0045] Optionally, the augmentation module includes:

[0046] The third submodule is used to input the training samples into the first branch, and the first branch augments the training samples according to the first perspective to obtain the augmented first face image.

[0047] The fourth submodule is used to input the training samples into the second branch, which augments the training samples according to the second perspective to obtain the augmented second face image.

[0048] Optionally, the first submodule is specifically used for:

[0049] The augmented first face image is input into the first feature encoder to obtain three levels of face features: low, medium, and high. The three levels of face features are then concatenated and input into the feature converter to generate the first face feature.

[0050] The augmented second face image is input into the second feature encoder to obtain three levels of face features: low, medium, and high. The three levels of face features are then stitched together to generate the second face feature.

[0051] Optionally, the device further includes:

[0052] The first prediction module is used to input the face image to be identified into the second feature encoder of the face liveness detection model after the parameter tuning module generates the face liveness detection model, to obtain three levels of face features: low, medium and high; and to generate a third face feature by concatenating the three levels of face features.

[0053] The third input module is used to input the third facial feature into the second classifier to obtain the third depth map;

[0054] The first recognition module is used to determine whether the face image to be recognized is a real face image based on the third depth map and a preset threshold.

[0055] Optionally, the device further includes:

[0056] The second prediction module is used to input the face image to be identified into the first branch and the second branch of the face liveness detection model after the parameter tuning module generates the face liveness detection model.

[0057] The second recognition module is used to determine whether the face image to be recognized is a real face image based on the prediction results obtained from the first branch and the prediction results obtained from the second branch.

[0058] Optionally, the first feature encoder and the second feature encoder have the same structure, both including:

[0059] Input layer, first convolutional module, second convolutional module, third convolutional module, first spoofing attention module, second spoofing attention module, third spoofing attention module, first max pooling layer, second max pooling layer, third max pooling layer;

[0060] The outputs of the input layer are respectively input to the first convolutional module and the first fraudulent attention module. The outputs of the first convolutional module and the first fraudulent attention module are added together and then input to the first max pooling layer to generate low-level features. The low-level features are input to the second convolutional module and the second fraudulent attention module. The outputs of the second convolutional module and the second fraudulent attention module are added together and then input to the second max pooling layer to generate medium-level features. The medium-level features are input to the third convolutional module and the third fraudulent attention module. The outputs of the third convolutional module and the third fraudulent attention module are input to the third max pooling layer to obtain high-level features.

[0061] This invention provides an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of any of the above-described methods for generating a face liveness detection model.

[0062] This invention provides a readable storage medium storing a program or instructions, which, when executed by a processor, implement the steps of any of the above-described face liveness detection model generation methods.

[0063] The face liveness detection model generation scheme provided in this embodiment of the invention inputs training samples into a pre-created Siamese neural network; augments the training samples through a first branch and a second branch to generate augmented face images; inputs the augmented face images into the first branch and the second branch respectively to obtain a first face feature, a second face feature, a first depth map, and a second depth map; calculates the coding-level dense consistency loss of the Siamese neural network based on the first face feature and the second face feature; calculates the prediction-level consistency loss of the Siamese neural network based on the first depth map and the second depth map; determines the total loss of the Siamese neural network based on the coding-level dense consistency loss and the prediction-level consistency loss; and optimizes the Siamese neural network parameters using stochastic gradient descent based on the total loss to generate a face liveness detection model. In this embodiment, by constructing a complex two-branch Siamese neural network for sample image feature extraction, coding-level dense consistency loss, and prediction-level consistency loss training, the trained face liveness detection model has high accuracy in predicting faces and good universality, applicable to various unknown domains and unknown attack methods. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the steps of a face liveness detection model generation method according to an embodiment of this application;

[0065] Figure 2 This is a schematic diagram illustrating the overall structure of a face liveness detection model according to an embodiment of this application;

[0066] Figure 3 This is a schematic diagram illustrating the structure of the first feature encoder and the second feature encoder in an embodiment of this application;

[0067] Figure 4 This is a schematic diagram illustrating the structure of the convolution module in the feature encoder of an embodiment of this application;

[0068] Figure 5 This is a schematic diagram illustrating the structure of the fraud attention module in the feature encoder of an embodiment of this application;

[0069] Figure 6 This is a schematic diagram illustrating the structure of the feature converter according to an embodiment of this application;

[0070] Figure 7 This is a schematic diagram illustrating the structure of the first classifier and the second classifier in an embodiment of this application;

[0071] Figure 8 This is a structural block diagram illustrating a face liveness detection model generation device according to an embodiment of this application. Detailed Implementation

[0072] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0073] The face liveness detection model generation scheme provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0074] As attached Figure 1 As shown, the face liveness detection model generation method of this application embodiment includes the following steps:

[0075] Step 101: Input the training samples into the pre-created Siamese neural network.

[0076] In this application, a twin neural network is pre-built and training samples are prepared. After training the twin neural network based on the training samples, a face liveness detection model is generated.

[0077] The training samples are face images. In this application, the face images are first preprocessed, and then the Siamese neural network is trained based on the preprocessed face images.

[0078] Preprocessing of face images may include, but is not limited to: scaling the face image to a preset size, such as 256×256 pixels, and then normalizing the scaled face image.

[0079] Figure 2 This is a schematic diagram illustrating the overall structure of a face liveness detection model according to an embodiment of this application. The face liveness detection model is generated by training a Siamese neural network. Figure 2 As shown, the Siamese neural network includes a first branch and a second branch. The first branch includes a first feature encoder, a feature converter, and a first classifier; the second branch includes a second feature encoder and a second classifier. The first and second feature encoders have the same structure and share parameters; the first and second classifiers have the same structure and share parameters.

[0080] Figure 3 This is a schematic diagram illustrating the structure of the first feature encoder and the second feature encoder according to an embodiment of this application. The first feature encoder and the second feature encoder have the same structure, as shown below. Figure 3As shown, both the first and second feature encoders include: an input layer, a first convolutional module, a second convolutional module, a third convolutional module, a first spoofing attention module, a second spoofing attention module, a third spoofing attention module, a first max pooling layer, a second max pooling layer, and a third max pooling layer. The output of the input layer is fed into the first convolutional module and the first spoofing attention module, respectively. The outputs of the first convolutional module and the first spoofing attention module are summed and then fed into the first max pooling layer to generate low-level features. The low-level features are fed into the second convolutional module and the second spoofing attention module. The outputs of the second convolutional module and the second spoofing attention module are summed and then fed into the second max pooling layer to generate mid-level features. The mid-level features are fed into the third convolutional module and the third spoofing attention module. The outputs of the third convolutional module and the third spoofing attention module are fed into the third max pooling layer to obtain high-level features.

[0081] Figure 4 This is a schematic diagram illustrating the structure of the convolutional module in the feature encoder of this application embodiment. The first convolutional module, the second convolutional module, and the third convolutional module can all be regarded as convolutional modules. The following description of the structure of the first convolutional module is as an example. The first convolutional module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer. The first convolutional layer is connected to the second convolutional layer, and the second convolutional layer is connected to the third convolutional layer.

[0082] Figure 5 This is a schematic diagram illustrating the structure of the fraud attention module in the feature encoder of an embodiment of this application, as shown below. Figure 5 As shown, the fraud attention module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, and a SoftMax activation layer. Assume the input features are... The size is The mapping is performed through the first and second convolutional layers respectively, and then the size is converted to... and Multiplying the two features yields a size of The feature is passed through a SoftMax activation layer to obtain an attention map. The attention map is then multiplied by the original input feature and its size is converted. The feature is linearly mapped through a convolutional layer to obtain the residual of the output feature. This residual is then added to the original input feature to obtain the attention-processed feature. Finally, the feature is passed through the fourth, fifth, and sixth convolutional layers to obtain the final output feature.

[0083] Figure 6 This is a schematic diagram illustrating the structure of the feature converter in an embodiment of this application, as shown below. Figure 6The feature converter shown includes an input layer, a first feature conversion module, a second feature conversion module, a third feature conversion module, and an output layer. The input layer is connected to the first feature conversion module, the first feature conversion module is connected to the second feature conversion module, the second feature conversion module is connected to the third feature conversion module, and the third feature conversion module is connected to the output layer.

[0084] The transformation module in the feature transformer includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first batch normalization layer, a second batch normalization layer, a third batch normalization layer, a first ReLU activation layer, a second ReLU activation layer, a third ReLU activation layer, and a max pooling layer. The first convolutional layer is connected to the first batch normalization layer, the first batch normalization layer is connected to the first ReLU activation layer, the first ReLU activation layer is connected to the second convolutional layer, the second convolutional layer is connected to the second batch normalization layer, the second batch normalization layer is connected to the second ReLU activation layer, the second ReLU activation layer is connected to the third convolutional layer, the third convolutional layer is connected to the third batch normalization layer, the third batch normalization layer is connected to the third ReLU activation layer, and the third ReLU activation layer is connected to the max pooling layer.

[0085] Figure 7 This is a schematic diagram illustrating the structure of the first classifier and the second classifier in an embodiment of this application. For example... Figure 7 As shown, the structures of the first and second classifiers include an input layer, a first convolutional layer, a second convolutional layer, a third convolutional layer, a first ReLU activation layer, a second ReLU activation layer, a third ReLU activation layer, and an output layer. The input layer is connected to the first convolutional layer, the first convolutional layer is connected to the first ReLU activation layer, the first ReLU activation layer is connected to the second convolutional layer, the second convolutional layer is connected to the second ReLU activation layer, the second ReLU activation layer is connected to the third convolutional layer, and the third convolutional layer is connected to the third ReLU activation layer.

[0086] It should be noted that the above is only an example of the structure of the Siamese neural network and its branches and modules. In actual implementation, it is not limited to this. Those skilled in the art can flexibly set the structure of each module according to actual needs. This application does not impose specific restrictions on this.

[0087] Step 102: Augment the training samples using the first and second branches to generate augmented face images.

[0088] An optional method for augmenting training samples using a first branch and a second branch to generate augmented face images is as follows: the training samples are input into the first branch, which augments the training samples according to a first perspective to obtain an augmented first face image; the training samples are input into the second branch, which augments the training samples according to a second perspective to obtain an augmented second face image.

[0089] In practical implementation, training samples can be input into the aforementioned Siamese neural network for training; the training samples in the training set are denoted as... The training samples are first augmented through the upper and lower branches of the Siamese neural network to generate augmented images. The processing method is as follows:

[0090]

[0091]

[0092] Among them and These are the face image augmentation methods for the upper and lower branches of the Siamese neural network, respectively. and Face images augmented by a twin neural network from both the first and second perspectives.

[0093] Step 103: Input the augmented face image into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map and the second depth map.

[0094] When the augmented face image is input into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map and the second depth map, specifically, the augmented face image can be input into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature; the first face feature is input into the first classifier to obtain the first depth map, and the second face feature is input into the second classifier to obtain the second depth map.

[0095] More specifically, the augmented face image is input into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature. The specific methods for generating these features may include:

[0096] The augmented first face image is input into the first feature encoder to obtain low, medium, and high-level face features; the low, medium, and high-level face features are then concatenated and input into the feature converter to generate the first face feature; the augmented second face image is input into the second feature encoder to obtain low, medium, and high-level face features; the low, medium, and high-level face features are then concatenated to generate the second face feature.

[0097] In practical implementation, the face images augmented in different ways are input into the first feature encoder and the second feature encoder respectively to generate coded face features. The processing method is as follows:

[0098]

[0099]

[0100] in, and These are the first and second feature encoders of the Siamese neural network, respectively. , , Extracted by the first feature encoder Low, medium, and high-level characteristics, , Extracted by the second feature encoder Low, medium, and high level characteristics.

[0101] Will Low, medium and high level characteristics , , splicing becomes a feature Similarly, Low, medium and high level characteristics , , splicing becomes a feature That is, the second facial feature:

[0102]

[0103]

[0104] in, This indicates a splicing operation.

[0105] Will Input into the feature converter to generate features That is, the first facial feature:

[0106]

[0107] in, This indicates a feature converter.

[0108] Step 104: Calculate the coding-level dense consistency loss of the Siamese neural network based on the first face feature and the second face feature.

[0109] In the actual implementation, the dense similarity loss of the two facial features mentioned above is calculated to constrain the Siamese neural network to maintain encoding-level consistency in the two branches. Before calculation, the dimensions of the two features are converted to... The loss is calculated as follows:

[0110]

[0111] in, represent The OK, Is The center of all normalized row vectors in the matrix can be considered as .

[0112] The dense consistency loss at the coding level of the network is then:

[0113]

[0114] in, This represents the output of the first viewpoint feature encoder. This represents the output of the feature encoder from the second viewpoint. This represents the output of the feature converter from the first viewpoint; To stop the gradient update operation.

[0115] Step 105: Calculate the prediction level consistency loss of the Siamese neural network based on the first depth map and the second depth map.

[0116] The goal of the first and second classifiers is to predict the depth map of the input features and then classify whether it is a real human face.

[0117]

[0118]

[0119] in, and These are the first classifier and the second classifier, respectively. and These are the depth maps output by the first classifier and the second classifier, respectively.

[0120] The calculation of prediction level consistency loss can be as follows:

[0121]

[0122]

[0123] in, The input depth map labels are 1 for real face images and 0 for fake face images. and This represents the supervised loss of the depth map.

[0124] Step 106: Determine the total loss of the Siamese neural network based on the coding-level dense consistency loss and the prediction-level consistency loss.

[0125] Based on the calculated encoding-level loss and prediction-level consistency loss, the overall loss of the Siamese neural network can be derived:

[0126]

[0127] Step 107: Use stochastic gradient descent to optimize the parameters of the Siamese neural network based on the total loss to generate a face liveness detection model.

[0128] It should be noted that steps 102 to 107 are a single-cycle parameter tuning training process for the Siamese neural network based on a training sample. In actual implementation, after each parameter tuning, it is necessary to determine whether the convergence of the trained Siamese neural network meets the preset conditions. If it does, the trained Siamese neural network is determined as a face liveness detection model. If it does not meet the conditions, the process returns to step 102 to perform parameter tuning training on the Siamese neural network again using the next training sample. This training process is repeated until the trained Siamese neural network meets the preset conditions.

[0129] The face liveness detection model generation method provided in this application involves inputting training samples into a pre-created Siamese neural network; augmenting the training samples through a first branch and a second branch to generate augmented face images; inputting the augmented face images into the first branch and the second branch respectively to obtain first face features, second face features, a first depth map, and a second depth map; calculating the coding-level dense consistency loss of the Siamese neural network based on the first and second face features; calculating the prediction-level consistency loss of the Siamese neural network based on the first and second depth maps; determining the total loss of the Siamese neural network based on the coding-level dense consistency loss and the prediction-level consistency loss; and optimizing the Siamese neural network parameters using stochastic gradient descent based on the total loss to generate a face liveness detection model. In this application embodiment, by constructing a complex two-branch Siamese neural network for sample image feature extraction, coding-level dense consistency loss, and prediction-level consistency loss training, the trained face liveness detection model exhibits high accuracy in face prediction and good universality, applicable to various unknown domains and unknown attack methods.

[0130] In an optional embodiment, after generating the face liveness detection model, the method further includes: inputting the face image to be identified into the second feature encoder of the face liveness detection model to obtain three levels of face features: low, medium, and high; concatenating the three levels of face features to generate a third face feature; inputting the third face feature into a second classifier to obtain a third depth map; and determining whether the face image to be identified is a real face image based on the third depth map and a preset threshold.

[0131] The preset threshold can be flexibly set by those skilled in the art, and no specific restrictions are imposed on it in the embodiments of this application.

[0132] This optional method of recognizing face images can be completed using only the second branch of the face liveness detection model, which requires little computation and has high processing efficiency.

[0133] In an optional embodiment, after generating the face liveness detection model, the method further includes: inputting the face image to be identified into the first branch and the second branch of the face liveness detection model respectively; and determining whether the face image to be identified is a real face image based on the prediction results obtained from the first branch and the prediction results obtained from the second branch.

[0134] For example: if the prediction results of both branches indicate that the face image to be identified is a real face image, then the face image to be identified is determined to be a real face image; or if the prediction results of either of the two branches indicate that the face image to be identified is a real face image.

[0135] This optional method of recognizing face images uses two branches of a face liveness detection model to complete the recognition, resulting in a highly accurate final recognition result.

[0136] The face liveness detection model generation method provided in this application will be described below with reference to a specific embodiment.

[0137] The face liveness detection model generation method in this application mainly includes the following steps:

[0138] S1: Preprocess and normalize the training samples used to train the face liveness detection network model.

[0139] Specifically, the size of the face images used as training samples is scaled up to... To input the face images into the network for training, normalize them:

[0140]

[0141] in, To input a face image, For normalized face images, The minimum value in the image. This represents the maximum value in the image.

[0142] S2: Build a twin neural network, optimize the parameters of the twin neural network model based on training samples, and obtain a face liveness detection model.

[0143] The specific structure of the constructed twin neural network can be referred to the relevant description in the aforementioned embodiments, and will not be repeated here.

[0144] The specific process of optimizing the parameters of the Siamese neural network model based on training samples includes the following sub-steps:

[0145] S21: Denote the training samples of the training sample set as... The training samples are first augmented through the upper and lower branches of the Siamese neural network to generate augmented images. The processing method is as follows:

[0146]

[0147]

[0148] in, and These are the face image augmentation methods for the upper and lower branches of the Siamese neural network, respectively. and Face images augmented by a twin neural network from both the first and second perspectives.

[0149] S22: Will and The features are input into the first feature encoder and the second feature encoder respectively to generate encoded facial features. The processing method is as follows:

[0150]

[0151]

[0152] in, and These are the first and second feature encoders of the Siamese neural network, respectively. , , Extracted by the first feature encoder Low, medium, and high-level characteristics, , Extracted by the second feature encoder Low, medium, and high level characteristics.

[0153] S23: Will Low, medium and high level characteristics , , splicing becomes a feature Similarly, Low, medium and high level characteristics , , splicing becomes a feature :

[0154]

[0155]

[0156] in, This indicates a splicing operation.

[0157] Will Input into the feature converter to generate features :

[0158]

[0159] in, This indicates a feature converter.

[0160] S24: Calculation and The dense similarity loss constrains the Siamese neural network to maintain encoding-level consistency in both branches, and transforms the dimensions of the two features before computation. The loss is calculated as follows:

[0161]

[0162] in, represent The OK, Is The center of all normalized row vectors in the matrix can be considered as .

[0163] The dense consistency loss at the encoding level of the Siamese neural network is:

[0164]

[0165] in, This represents the output of the first viewpoint feature encoder. This represents the output of the feature encoder from the second viewpoint. This represents the output of the feature converter from the first viewpoint; To stop the gradient update operation.

[0166] S25: The goal of the first and second classifiers is to predict the depth map of the input features and then classify whether it is a real human face.

[0167]

[0168]

[0169] in, and These are the first classifier and the second classifier, respectively. and These are the depth maps output by the first classifier and the second classifier, respectively.

[0170] The prediction level consistency loss is calculated as follows:

[0171]

[0172]

[0173] in, The input depth map is labeled. For real face images, the depth map value is 1, while for fake face images, the depth map value is 0. and This represents the supervised loss of the depth map.

[0174] S26: Based on the above calculations of the coding-level dense consistency loss and the prediction-level consistency loss, the overall loss of the Siamese neural network can be derived:

[0175]

[0176] Based on the overall loss of the Siamese neural network, the network parameters are optimized using the SGD stochastic gradient descent algorithm to obtain a face liveness detection network model.

[0177] S3: Input the face image to be identified into the second feature encoder, the second classifier obtains the depth map of the image, and manually set a threshold to determine whether the input face is a real face image.

[0178] The face liveness detection model generation method provided in this specific embodiment improves the network's ability to detect fraud clues by using a designed fraud attention module. By using a Siamese neural network to tune the coding-level dense consistency loss and the prediction-level consistency loss, the generalization ability and recognition accuracy of the face liveness detection model can be improved.

[0179] Figure 8 The structural block diagram of a face liveness detection model generation device according to an embodiment of this application is shown.

[0180] The face liveness detection model generation device provided in this application includes the following functional modules:

[0181] The first input module 801 is used to input training samples into a pre-created Siamese neural network; wherein the Siamese neural network includes a first branch and a second branch, the first branch includes a first feature encoder, a feature converter, and a first classifier; the second branch includes a second feature encoder and a second classifier;

[0182] Augmentation module 802 is used to augment the training samples through the first branch and the second branch to generate an augmented face image;

[0183] The second input module 803 is used to input the augmented face image into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map and the second depth map;

[0184] The first calculation module 804 is used to calculate the coding-level dense consistency loss of the Siamese neural network based on the first face feature and the second face feature.

[0185] The second calculation module 805 is used to calculate the prediction level consistency loss of the Siamese neural network based on the first depth map and the second depth map.

[0186] The total loss determination module 806 is used to determine the total loss of the Siamese neural network based on the coding-level dense consistency loss and the prediction-level consistency loss.

[0187] The parameter tuning module 807 is used to optimize the parameters of the Siamese neural network based on the total loss using the stochastic gradient descent method to generate a face liveness detection model.

[0188] Optionally, the second input module includes:

[0189] The first submodule is used to input the augmented face image into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature;

[0190] The second submodule is used to input the first facial features into the first classifier to obtain a first depth map, and to input the second facial features into the second classifier to obtain a second depth map.

[0191] Optionally, the augmentation module includes:

[0192] The third submodule is used to input the training samples into the first branch, and the first branch augments the training samples according to the first perspective to obtain the augmented first face image.

[0193] The fourth submodule is used to input the training samples into the second branch, which augments the training samples according to the second perspective to obtain the augmented second face image.

[0194] Optionally, the first submodule is specifically used for:

[0195] The augmented first face image is input into the first feature encoder to obtain three levels of face features: low, medium, and high. The three levels of face features are then concatenated and input into the feature converter to generate the first face feature.

[0196] The augmented second face image is input into the second feature encoder to obtain three levels of face features: low, medium, and high. The three levels of face features are then stitched together to generate the second face feature.

[0197] Optionally, the device further includes:

[0198] The first prediction module is used to input the face image to be identified into the second feature encoder of the face liveness detection model after the parameter tuning module generates the face liveness detection model, to obtain three levels of face features: low, medium and high; and to generate a third face feature by concatenating the three levels of face features.

[0199] The third input module is used to input the third facial feature into the second classifier to obtain the third depth map;

[0200] The first recognition module is used to determine whether the face image to be recognized is a real face image based on the third depth map and a preset threshold.

[0201] Optionally, the device further includes:

[0202] The second prediction module is used to input the face image to be identified into the first branch and the second branch of the face liveness detection model after the parameter tuning module generates the face liveness detection model.

[0203] The second recognition module is used to determine whether the face image to be recognized is a real face image based on the prediction results obtained from the first branch and the prediction results obtained from the second branch.

[0204] Optionally, the first feature encoder and the second feature encoder have the same structure, both including:

[0205] Input layer, first convolutional module, second convolutional module, third convolutional module, first spoofing attention module, second spoofing attention module, third spoofing attention module, first max pooling layer, second max pooling layer, third max pooling layer;

[0206] The outputs of the input layer are respectively input to the first convolutional module and the first fraudulent attention module. The outputs of the first convolutional module and the first fraudulent attention module are added together and then input to the first max pooling layer to generate low-level features. The low-level features are input to the second convolutional module and the second fraudulent attention module. The outputs of the second convolutional module and the second fraudulent attention module are added together and then input to the second max pooling layer to generate medium-level features. The medium-level features are input to the third convolutional module and the third fraudulent attention module. The outputs of the third convolutional module and the third fraudulent attention module are input to the third max pooling layer to obtain high-level features.

[0207] The face liveness detection model generation device provided in this application embodiment extracts sample image features by constructing a complex two-branch twin neural network, trains the twin neural network model by encoding-level dense consistency loss and prediction-level consistency loss, and the trained face liveness detection model has high accuracy in predicting faces and good universality, which can be applied to various unknown domains and unknown attack methods.

[0208] In the embodiments of this application Figure 8 The face liveness detection model generation device shown can be a device, or it can be a component, integrated circuit, or chip in a 3D scanner. (This is from an embodiment of the present application.) Figure 8 The device shown can be a device with an operating system.

[0209] The embodiments provided in this application Figure 8 The face liveness detection model generation device shown can achieve Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0210] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When executed by the processor, the program or instructions implement the various processes of the above-described face liveness detection model generation method embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here. It should be noted that the electronic device in the embodiments of this application includes the server described above.

[0211] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described face liveness detection model generation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0212] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0213] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described face liveness detection model generation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0214] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0215] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0216] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for generating a face liveness detection model, characterized in that, include: Training samples are input into a pre-created Siamese neural network; wherein the Siamese neural network includes a first branch and a second branch, the first branch including a first feature encoder, a feature converter, and a first classifier; the second branch including a second feature encoder and a second classifier; The training samples are augmented using the first branch and the second branch to generate augmented face images. The augmented face image is input into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map and the second depth map; Based on the first facial feature and the second facial feature, the coding-level dense consistency loss of the Siamese neural network is calculated; Based on the first depth map and the second depth map, calculate the prediction level consistency loss of the Siamese neural network; The total loss of the Siamese neural network is determined based on the coding-level dense consistency loss and the prediction-level consistency loss. The stochastic gradient descent method is used to optimize the parameters of the twin neural network based on the total loss, thereby generating a face liveness detection model; The coding-level dense consistency loss is as follows: ； in, This represents the output of the first viewpoint feature encoder. This represents the output of the feature encoder from the second viewpoint. This represents the output of the feature converter in the first viewpoint. To stop the gradient update operation; where, ； in, represent The OK, Is The center of all normalized row vectors in the matrix, where, ; The formula for calculating the prediction level consistency loss is as follows: ； in, and These are the depth maps output by the first classifier and the second classifier, respectively. The first feature encoder and the second feature encoder have the same structure, both including: Input layer, first convolutional module, second convolutional module, third convolutional module, first spoofing attention module, second spoofing attention module, third spoofing attention module, first max pooling layer, second max pooling layer, third max pooling layer; The outputs of the input layer are respectively input to the first convolutional module and the first fraudulent attention module. The outputs of the first convolutional module and the first fraudulent attention module are added together and then input to the first max pooling layer to generate low-level features. The low-level features are input to the second convolutional module and the second fraudulent attention module. The outputs of the second convolutional module and the second fraudulent attention module are added together and then input to the second max pooling layer to generate medium-level features. The medium-level features are input to the third convolutional module and the third fraudulent attention module. The outputs of the third convolutional module and the third fraudulent attention module are input to the third max pooling layer to obtain high-level features.

2. The method according to claim 1, characterized in that, The steps of inputting the augmented face image into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map, and the second depth map include: The augmented face image is input into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature; The first facial feature is input into the first classifier to obtain the first depth map, and the second facial feature is input into the second classifier to obtain the second depth map.

3. The method according to claim 2, characterized in that, The step of augmenting the training samples using the first branch and the second branch to generate augmented face images includes: The training sample is input into the first branch, and the first branch augments the training sample according to the first perspective to obtain the augmented first face image. The training sample is input into the second branch, which augments the training sample according to the second perspective to obtain the augmented second face image.

4. The method according to claim 3, characterized in that, The step of inputting the augmented face image into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature includes: The augmented first face image is input into the first feature encoder to obtain three levels of face features: low, medium, and high. The three levels of face features are then concatenated and input into the feature converter to generate the first face feature. The augmented second face image is input into the second feature encoder to obtain three levels of face features: low, medium, and high. The three levels of face features are then stitched together to generate the second face feature.

5. The method according to claim 1, characterized in that, Following the step of generating the face liveness detection model, the method further includes: The face image to be identified is input into the second feature encoder of the face liveness detection model to obtain three levels of face features: low, medium, and high. The three levels of face features are then concatenated to generate a third face feature. The third facial feature is input into the second classifier to obtain the third depth map; Based on the third depth map and the preset threshold, it is determined whether the face image to be identified is a real face image.

6. The method according to claim 1, characterized in that, Following the step of generating the face liveness detection model, the method further includes: The face image to be identified is input into the first branch and the second branch of the face liveness detection model, respectively; Based on the prediction results obtained from the first branch and the prediction results obtained from the second branch, it is determined whether the face image to be identified is a real face image.

7. A face liveness detection model generation device, characterized in that, include: The first input module is used to input training samples into a pre-created Siamese neural network; wherein the Siamese neural network includes a first branch and a second branch, the first branch includes a first feature encoder, a feature converter, and a first classifier; the second branch includes a second feature encoder and a second classifier; An augmentation module is used to augment the training samples through the first branch and the second branch to generate an augmented face image; The second input module is used to input the augmented face image into the first branch and the second branch respectively to obtain the first face feature, the second face feature, the first depth map and the second depth map; The first calculation module is used to calculate the coding-level dense consistency loss of the Siamese neural network based on the first facial feature and the second facial feature. The second calculation module is used to calculate the prediction level consistency loss of the Siamese neural network based on the first depth map and the second depth map. The total loss determination module is used to determine the total loss of the Siamese neural network based on the encoding-level dense consistency loss and the prediction-level consistency loss. The parameter tuning module is used to optimize the parameters of the Siamese neural network based on the total loss using the stochastic gradient descent method to generate a face liveness detection model. The coding-level dense consistency loss is as follows: ； in, This represents the output of the first viewpoint feature encoder. This represents the output of the feature encoder from the second viewpoint. This represents the output of the feature converter from the first viewpoint; To stop the gradient update operation; where, ； in, represent The OK, Is The center of all normalized row vectors in the matrix, where, ; The prediction level consistency loss is: ； in, and These are the depth maps output by the first classifier and the second classifier, respectively. The first feature encoder and the second feature encoder have the same structure, both including: Input layer, first convolutional module, second convolutional module, third convolutional module, first spoofing attention module, second spoofing attention module, third spoofing attention module, first max pooling layer, second max pooling layer, third max pooling layer; The outputs of the input layer are respectively input to the first convolutional module and the first fraudulent attention module. The outputs of the first convolutional module and the first fraudulent attention module are added together and then input to the first max pooling layer to generate low-level features. The low-level features are input to the second convolutional module and the second fraudulent attention module. The outputs of the second convolutional module and the second fraudulent attention module are added together and then input to the second max pooling layer to generate medium-level features. The medium-level features are input to the third convolutional module and the third fraudulent attention module. The outputs of the third convolutional module and the third fraudulent attention module are input to the third max pooling layer to obtain high-level features.

8. The apparatus according to claim 7, characterized in that, The second input module includes: The first submodule is used to input the augmented face image into the first feature encoder and the second feature encoder respectively to generate the first face feature and the second face feature; The second submodule is used to input the first facial features into the first classifier to obtain a first depth map, and to input the second facial features into the second classifier to obtain a second depth map.

9. An electronic device, comprising a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of a face liveness detection model generation method as described in any one of claims 1-6.

Images