Face living body detection method, model training method, device and equipment

Abstract

The invention provides a human face in-vivo detection method, a model training method, devices and equipment, and belongs to the field of human face in-vivo detection. The face living body detection method comprises the following steps: acquiring a plurality of target face images and extracting initial feature information; inputting the initial feature information into an adaptive Transform module to obtain target feature information and an image feature vector corresponding to the target feature information; generating a corresponding text embedding vector according to the target feature information; judging the plurality of target face images according to the similarity between the image feature vector and a preset text feature vector; obtaining a target loss function according to the plurality of target face images, the target feature vector, the text embedding vector and the judgment result; and according to the target loss function, parameters of an adapter inserted in the adaptive Transform module are updated, and a trained human face in-vivo detection model is obtained. The accuracy of face living body detection can be improved, and the generalization ability and robustness of the model are enhanced.

Description

Technical Field

[0001] The present application relates to the field of face liveness detection, and specifically to a face liveness detection method, model training method, device and equipment. Background Art

[0002] Face liveness detection refers to determining whether the face being recognized is a real, living person, rather than a photo, video, or other fake static image or model. This technology is designed to ensure that only authentic faces pass identity verification, preventing fraudulent attempts using photos or videos.

[0003] Traditional face liveness detection methods rely on manually designed feature extraction strategies. For example, texture analysis uses local binary patterns (LBP) or histograms of oriented gradients (HoG) to capture microscopic differences in facial texture. Frequency domain analysis uses Fourier transforms or wavelet transforms to detect abnormal signals in the image frequency domain, such as frequency ripples caused by screen reflections. Liveness behavior detection uses physiological activities such as blinking and slight head movements to distinguish liveness from static attacks (such as photo attacks and video replays). However, this approach suffers from limitations such as limited feature expression capabilities and the difficulty of manually designed features in covering complex and varied attack scenarios (such as 3D masks and high-resolution video replays). Furthermore, it is sensitive to environmental factors, and interference such as lighting changes and occlusion can significantly reduce recognition accuracy. Summary of the Invention

[0004] The present application provides a face liveness detection method, model training method, device and equipment to improve the accuracy of face liveness detection, enhance the generalization ability and robustness of the model, and reduce the computing resources consumed by training.

[0005] This application provides a model training method for a face liveness detection model, including: Obtain multiple target face images based on the training data set; Extracting initial feature information of the plurality of target face images; Inputting the initial feature information into an adaptive Transformer module to obtain target feature information for face liveness detection; Obtaining an image feature vector corresponding to the target feature information; Generating a text embedding vector corresponding to the target face image according to the target feature information; Judging the plurality of target face images based on similarities between the image feature vector and a preset text feature vector to obtain a judgment result, wherein the judgment result is used to indicate whether the target face image is a living body image; Obtaining a target loss function according to the multiple target face images, the target feature vector, the text embedding vector, and the judgment result; The parameters of the adapter inserted in the adaptive Transformer module are updated according to the target loss function to obtain a trained face liveness detection model.

[0006] According to the model training method of the face liveness detection model provided in the present application, the initial feature information is input into the adaptive Transformer module to obtain target feature information for face liveness detection, including: performing layer normalization on the initial feature information and inputting it into the multi-head self-attention layer to obtain weighted feature information; performing layer normalization on the weighted feature information and inputting it into the adapter to obtain adaptation feature information; performing layer normalization on the weighted feature information and inputting it into the multi-layer perceptron layer to obtain task feature information; obtaining the scaling factor of the adaptation feature; and obtaining the target feature information for face liveness detection based on the weighted feature information, the adaptation feature information, the scaling factor and the task feature information.

[0007] According to the model training method of the face liveness detection model provided in the present application, the method of obtaining multiple target face images based on the training data set includes: obtaining multiple initial face images based on the training data set; performing the following operations on each of the multiple initial face images: detecting facial key points in the current initial face image; performing affine transformation based on the facial key points to obtain multiple reference face images; and performing data enhancement on the multiple reference face images to obtain multiple target face images corresponding to the current initial face image.

[0008] According to the model training method of the face liveness detection model provided in the present application, the target loss function is obtained based on the multiple target face images, the target feature vector, the text embedding vector and the judgment result, including: respectively obtaining the first target feature vectors corresponding to multiple first target face images, and the initial face images corresponding to the multiple first target face images are the same; determining the contrast loss function based on the distance between the first target feature vectors of the multiple first target face images; obtaining a multimodal dual-weight focal loss function based on the multiple target face images and the judgment result; obtaining a mean square error loss function based on the target feature vector and the text embedding vector; and obtaining a target loss function based on the contrast loss function, the multimodal dual-weight focal loss function and the mean square error loss function.

[0009] According to the model training method of the face liveness detection model provided in the present application, the multimodal dual-weight focal loss function is obtained based on the multiple target face images and the judgment results, including: obtaining the image category and image domain to which each target face image belongs, the image category including a living image or a fraudulent image, and the image domain being used to indicate the data source of the target face image; obtaining the number of target face images corresponding to the image category included in each image domain based on the image category and image domain to which each target face image belongs; determining whether the judgment result of each target face image is correct, and obtaining the probability of a correct judgment; and obtaining a multimodal dual-weight focal loss function based on the number of target face images and the probability of a correct judgment.

[0010] According to the model training method of the face liveness detection model provided in the present application, the parameters of the adapter inserted in the adaptive Transformer module are updated according to the target loss function, including: determining the gradient of the target loss function in the current parameters of the adapter; determining the maximum disturbance of the current parameters according to the gradient; performing minimization optimization of the target loss function according to the maximum disturbance and the current parameters to obtain the optimized target parameters of the adapter; and updating the current parameters of the adapter according to the target parameters.

[0011] This application also provides a method for detecting live faces, including: Get the face image to be detected; The face image to be detected is input into the face liveness detection model to obtain an output target judgment result, which is used to indicate whether the face image to be detected is alive. The face liveness detection model is obtained based on the model training method of any of the above-mentioned face liveness detection models.

[0012] This application also provides a model training device for a face liveness detection model, comprising: A first acquisition unit is used to acquire multiple target face images according to a training data set; an extraction unit, configured to extract initial feature information of the plurality of target face images; A first input unit is used to input the initial feature information into an adaptive Transformer module to obtain target feature information for performing face liveness detection; A second acquisition unit is used to acquire an image feature vector corresponding to the target feature information; A generating unit, configured to generate a text embedding vector corresponding to the target face image based on the target feature information; a judgment unit, configured to judge the plurality of target face images based on similarities between the image feature vector and a preset text feature vector, and obtain a judgment result, wherein the judgment result is used to indicate whether the target face image is a living body image; A third acquisition unit is used to acquire a target loss function according to the multiple target face images, the target feature vector, the text embedding vector and the judgment result; An updating unit is used to update the parameters of the adapter inserted in the adaptive Transformer module according to the target loss function to obtain a trained face liveness detection model.

[0013] This application also provides a face liveness detection device, comprising: A fourth acquiring unit, configured to acquire a face image to be detected; A second input unit is used to input the face image to be detected into the face liveness detection model to obtain an output target judgment result, wherein the target judgment result is used to indicate whether the face image to be detected is alive. The face liveness detection model is obtained based on the model training method of any of the above-mentioned face liveness detection models.

[0014] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, it implements a model training method or a face liveness detection method such as any of the above-mentioned face liveness detection models.

[0015] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements a model training method or a face liveness detection method such as any of the above-mentioned face liveness detection models.

[0016] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements a model training method or a face liveness detection method such as any of the above-mentioned face liveness detection models.

[0017] The present application provides a method, model training method, device and equipment for face liveness detection. During model training, multiple target face images are first obtained according to a training data set, and then initial feature information of the multiple target face images is extracted. The initial feature information is then input into an adaptive Transformer module to obtain target feature information for face liveness detection. An image feature vector corresponding to the target feature information is then obtained. A text embedding vector corresponding to the target face image is generated based on the target feature information. The multiple target face images are judged based on the similarity between the image feature vector and the preset text feature vector to obtain a judgment result, which is used to indicate whether the target face image is a live image. A target loss function is then obtained based on the multiple target face images, the target feature vector, the text embedding vector and the judgment result. Finally, the parameters of the adapter inserted in the adaptive Transformer module are updated according to the target loss function to obtain a trained face liveness detection model. In this way, the present application can improve the accuracy of face liveness detection and enhance the generalization ability and robustness of the model. At the same time, only the adapter parameters are trained, which can effectively reduce the computing resources consumed by training. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a flowchart of a model training method for a face liveness detection model provided in this application.

[0020] Figure 2 This is a schematic diagram of the structure of the adaptive Transformer module provided by this application.

[0021] Figure 3 This is a schematic diagram of the model framework of the face liveness detection model provided in this application.

[0022] Figure 4 This is a schematic diagram of experimental indicators corresponding to different data sets provided in this application.

[0023] Figure 5 This is a graph showing the effect of different parameter values ​​on the Half Total Error Rate (HTER) provided in this application.

[0024] Figure 6This is a schematic diagram of the attention heat map comparison provided by this application.

[0025] Figure 7 It is a characteristic distribution diagram provided by this application.

[0026] Figure 8 This is a flowchart of a face liveness detection method provided by this application.

[0027] Figure 9 This is a block diagram of the functional units of a model training device for a face liveness detection model provided by this application.

[0028] Figure 10 This is a block diagram of the functional units of a face liveness detection device provided by this application.

[0029] Figure 11 It is a structural diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0031] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0032] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0033] Current face detection methods are sensitive to the environment and have difficulty covering complex and changing attack scenarios, resulting in low detection accuracy.

[0034] To address the above issues, this application provides a method for detecting liveness of a face, a method for training a model, an apparatus and a device. Figure 1 The model training method of the face liveness detection model includes the following steps.

[0035] S101, obtaining multiple target face images according to a training data set.

[0036] The training datasets may include multiple datasets, i.e., multiple target face images may come from different training datasets. For example, the training datasets may include MSU-MFSD (M), CASIA-FAD (C), Replay-Attack (I), OULU-NPU (O) ​​and other face liveness detection datasets.

[0037] S102: Extracting initial feature information of the plurality of target face images.

[0038] The face liveness detection model can be a Contrastive Language–Image Pretraining (CLIP) model. Specifically, the Vision Transformer / Base - 16x16 patches (ViT / B-16) encoder in the CLIP model can be used to extract image features.

[0039] S103: Input the initial feature information into an adaptive Transformer module to obtain target feature information for performing live face detection.

[0040] The CLIP model includes an adaptive Transformer module, which embeds adjustable parameters or layers within the model structure to facilitate fine-tuning or adjustment based on different task requirements. This solution, which requires performing face liveness detection, uses the adaptive Transformer module to adjust the initial features obtained so that the target feature information can be used for face liveness detection.

[0041] S104: Obtain an image feature vector corresponding to the target feature information.

[0042] The image encoder may also generate an image feature vector corresponding to the target feature information.

[0043] S105: Generate a text embedding vector corresponding to the target face image according to the target feature information.

[0044] Based on the target image's feature information, a description sentence matching the target image can be selected from a preset text library. For example, the description sentences selected for target image 1 and target image 2 are "a real face photo" and "a printed photo used for attacking," respectively. The selected description sentences are then converted into text embedding vectors using the text encoder included in the CLIP model. The dimension of the text embedding vector can be 512.

[0045] S106 , judging the plurality of target face images according to the similarity between the image feature vector and the preset text feature vector to obtain a judgment result.

[0046] The judgment result is used to indicate whether the target facial image is a live image. The preset text feature vector can be a first text feature vector indicating that the image is a live image and a second text feature vector indicating that the image is at least a fraudulent image. The cosine similarity between the image feature vector and the first text feature vector is then calculated. If the similarity is greater than a dynamic threshold, the judgment result is that the target image is a live image. If the similarity is less than the dynamic threshold, the cosine similarity between the image feature vector and the second text feature vector is calculated. Similarly, if the similarity is greater than the dynamic threshold, the judgment result is that the target image is a fraudulent image. In particular, the dynamic threshold can be set to 0.5.

[0047] S107: Obtain a target loss function according to the multiple target face images, the target feature vector, the text embedding vector, and the judgment result.

[0048] Among them, the target loss function can be obtained based on the multimodal dual-weight focal loss function, the contrast loss function and the mean square error loss function.

[0049] S108: Update the parameters of the adapter inserted into the adaptive Transformer module according to the target loss function to obtain a trained face liveness detection model.

[0050] A lightweight adapter can be inserted into the CLIP Transformer module. The adapter learns forgery cues, enabling the model to identify forgeries or anomalies in the input data. During model training, the CLIP backbone network can be frozen, and only the adapter parameters can be trained.

[0051] It can be seen that in this embodiment, first, multiple target face images are obtained according to the training data set, and then the initial feature information of the multiple target face images is extracted, and then the initial feature information is input into the adaptive Transformer module to obtain the target feature information for face liveness detection, and then the image feature vector corresponding to the target feature information is obtained, and then the text embedding vector corresponding to the target face image is generated according to the target feature information, and then the multiple target face images are judged according to the similarity between the image feature vector and the preset text feature vector to obtain a judgment result, which is used to indicate whether the target face image is a live image, and then the target loss function is obtained according to the multiple target face images, the target feature vector, the text embedding vector and the judgment result, and finally the parameters of the adapter inserted in the adaptive Transformer module are updated according to the target loss function to obtain a trained face liveness detection model. In this way, the present application can improve the accuracy of face liveness detection and enhance the generalization ability and robustness of the model. At the same time, only the adapter parameters are trained, which can effectively reduce the computing resources consumed by training.

[0052] In a possible embodiment, the initial feature information is input into an adaptive Transformer module to obtain target feature information for face liveness detection, including: performing layer normalization on the initial feature information and inputting it into a multi-head self-attention layer to obtain weighted feature information; performing layer normalization on the weighted feature information and inputting it into the adapter to obtain adaptation feature information; performing layer normalization on the weighted feature information and inputting it into a multi-layer perceptron layer to obtain task feature information; obtaining a scaling factor of the adaptation feature; and obtaining target feature information for face liveness detection based on the weighted feature information, the adaptation feature information, the scaling factor, and the task feature information.

[0053] Among them, Figure 2 As shown, the Transformer module consists of a multi-head self-attention layer and a multi-layer perceptron (MLP) layer. An adapter is inserted into the MLP layer. This adapter includes a dimensionality reduction layer, a ReLU activation layer, and a dimensionality increase layer. Based on the adapter, a bottleneck structure that first reduces and then increases the dimensionality of weighted features and a nonlinear activation function is used to learn some forgery clues. Finally, the original features are fused using a scaling factor S and residual connections. The weighted feature information x can be obtained using the following formula: The query (Q), key (K), and value (V) are all obtained by linearly transforming the input features. d is the bottleneck dimension, which can be 64.

[0054] The adaptation feature x' can be obtained by the following formula: Among them, LN(x) is the weighted feature after layer normalization, W down is the dimension reduction parameter, W up is the dimension-raising parameter.

[0055] The final target feature xl can be calculated by the following formula: Where MLP(LN(x)) is the task feature information output after the MLP layer, and s is the scaling factor of the adapter. In particular, s = 0.1.

[0056] It can be seen that in this embodiment, a lightweight adapter is inserted into the Transformer module, and then the target features of user face liveness detection are obtained based on the adapter. This allows only the adapter parameters to be trained during model training, freezing the CLIP backbone network and reducing the computing resources consumed by training.

[0057] In a possible embodiment, obtaining multiple target facial images based on a training data set includes: obtaining multiple initial facial images based on the training data set; performing the following operations on each of the multiple initial facial images: detecting facial key points in the current initial facial image; performing affine transformation based on the facial key points to obtain multiple reference facial images; and performing data enhancement on the multiple reference facial images to obtain multiple target facial images corresponding to the current initial facial image.

[0058] Among them, after obtaining the initial face image, you can use (Multi-task Cascaded Convolutional Networks, MTCNN) to detect the five key points of the face (eyes, nose tip, mouth corners) and perform affine transformation alignment. The MTCNN is a deep learning model for face detection. Affine transformation can perform operations including translation, rotation, scaling and shearing based on the five detected key points, and obtain multiple reference face images based on the same initial face image. When performing data enhancement, random data enhancement can be applied to the obtained reference face image, such as random cropping (range ±10%), rotation (±15°), brightness adjustment (±20%), etc., to obtain multiple target face images. In particular, two target face images can be generated for the same initial face image.

[0059] It can be seen that in this embodiment, affine transformation and data enhancement are performed based on the initial image to obtain multiple target face images. Training based on multiple target face images can improve the accuracy and generalization ability of the model.

[0060] In a possible embodiment, the target loss function is obtained based on the multiple target facial images, the target feature vector, the text embedding vector and the judgment result, including: respectively obtaining the first target feature vectors corresponding to multiple first target facial images, and the initial facial images corresponding to the multiple first target facial images are the same; determining the contrast loss function based on the distance between the first target feature vectors of the multiple first target facial images; obtaining a multimodal dual-weight focal loss function based on the multiple target facial images and the judgment result; obtaining a mean square error loss function based on the target feature vector and the text embedding vector; and obtaining a target loss function based on the contrast loss function, the multimodal dual-weight focal loss function and the mean square error loss function.

[0061] By comparing the feature vectors of multiple target face images corresponding to the same initial face image, the feature distance between multiple views generated from the same image can be measured to obtain a contrast loss function. A multimodal dual-weighted focal loss function, determined based on face images and judgment results, can be used to address imbalances between image categories and data sources. A mean squared error loss function, determined based on the target feature vector and text embedding vector, can constrain image-text feature alignment.

[0062] In the specific implementation, the target loss function L total It can be calculated by the following formula: Among them, L mdf1 is the multimodal dual-weight focal loss function, L con is the contrast loss function, L mse is the mean square error loss function, and λ is the weighting value. In the specific implementation, λ=0.1.

[0063] It can be seen that in this embodiment, the target loss function is obtained based on the multimodal dual-weight focal loss function, the contrast loss function and the mean square error loss function, which can improve the accuracy and generalization ability of the model.

[0064] In a possible embodiment, the obtaining of a multimodal dual-weight focal loss function based on the multiple target facial images and the judgment results includes: obtaining the image category and image domain to which each target facial image belongs, the image category including a living image or a fraudulent image, and the image domain being used to indicate the data source of the target facial image; obtaining the number of target facial images corresponding to the image category included in each image domain based on the image category and image domain to which each target facial image belongs; determining whether the judgment result of each target facial image is correct, and obtaining the probability of a correct judgment; obtaining a multimodal dual-weight focal loss function based on the number of target facial images and the probability of a correct judgment.

[0065] After obtaining the image category and image domain to which each target face image belongs, the number of target face images included in each image category corresponding to each image domain can be determined. Then, based on the number and the accuracy of the model's judgment of the target face image, a multimodal dual-weighted focal loss function is determined. The multimodal dual-weighted focal loss function can be specifically calculated using the following formula: in, Among them, ni is used to indicate the number of target face images included in the image category corresponding to an image domain, label is used to indicate the label corresponding to the target face image, for example, label=1 indicates that the target face image is a living image, label=0 indicates that the target face image is a fraudulent image, and pt is used to indicate the probability of correct judgment. γ and λ are hyperparameters, where γ can be equal to 2. The weight used to control the loss.

[0066] It can be seen that in the embodiment of the present application, the multimodal dual-weight focal loss function can emphasize the small sample domain through inverse weights, and enhance the attention to difficult samples and the difference in the number of true and false samples through the focal factor, which fully solves the long-standing problems of domain imbalance and category imbalance in face liveness detection datasets.

[0067] In a possible embodiment, updating the parameters of the adapter inserted in the adaptive Transformer module according to the target loss function includes: determining the gradient of the target loss function in the current parameters of the adapter; determining the maximum disturbance of the current parameters according to the gradient; performing minimization optimization of the target loss function according to the maximum disturbance and the current parameters to obtain the optimized target parameters of the adapter; and updating the current parameters of the adapter according to the target parameters.

[0068] When performing generalization optimization, you can first configure the sharpness-aware minimization (SAM) optimizer, setting the perturbation radius γ = 0.05, the learning rate η = 5 × 10^(-6), and the weight decay to 0.05. Then, determine the training strategy and conduct 200 rounds of training, with 1000 batches per round and a batch size of 32, using mixed-precision training acceleration. When updating parameters, you can first calculate the perturbation value based on the gradient and perform a maximum search on it. Then, combine the perturbation value with the model parameters to optimize for overall loss minimization.

[0069] The specific formula can be expressed as follows: in, are the parameters of the original model, is the maximum perturbation imposed on the model parameters.

[0070] As can be seen, in this embodiment, SAM is used to update parameters. After accumulating gradients in each batch, a one-step optimization is performed, ultimately completing the parameter update of the entire model through batch iteration. In each batch of optimization, a minimum-maximum dual-level optimization is performed based on the SAM algorithm. This can improve the robustness, stability, and generalization ability of the model.

[0071] See also Figure 3 , the overall process of the model training method of the face liveness detection model of this application is introduced.

[0072] First, data is obtained from datasets such as MSU-MFSD (M), CASIA-FAD (C), Replay-Attack (I), and OULU-NPU (O). Two views of the same initial image are then obtained using an affine transformation. The dataset is then divided into training and test sets with a 7:3 ratio to ensure balanced sample size across domains. During training, image encoders are used to extract image features from target face images 1 and 2, obtaining feature vectors of these features. Textual hints (i.e., descriptions of the target face images) are also obtained for target face images 1 and 2. For example, hint 1 is "a fake face image" and hint 2 is "this is not a real face." The text encoder generates embedded text features based on the textual hints. A target loss function is then derived using a multimodal dual-weighted focal loss, contrastive loss, and mean squared error loss. The adapter parameters are then updated based on this target loss function. Furthermore, a minimum-maximum two-layer optimization based on SAM is performed to optimize the adapter parameters, resulting in a trained face liveness detection model. At the same time, when testing data, a face image is input, and a multimodal dual-weight focal loss is obtained based on the feature vector corresponding to the image encoder and the label of the face image (for example, "this is a real / fake face") based on the text encoder.

[0073] To validate the effectiveness of this solution, we used the benchmark datasets MSU-MFSD (M), CASIA-FAD (C), Replay-Attack (I), and OULU-NPU (O), totaling four domains. We used leave-one-domain cross-validation (e.g., training on M / I / O and testing on C) to evaluate cross-domain generalization performance. The evaluation metric for this experiment was determined to be Half Total Error Rate (HTER) = (FRR + FAR) / 2, where FRR is the false rejection rate and FAR is the false acceptance rate. Lower HTER values ​​are preferred. The area under the curve (AUC) is calculated, with AUC values ​​closer to 100% indicating better performance.

[0074] like Figure 4As shown, in the leave-one-domain test, the average HTER = 4.83%, and the average AUC = 98.80%. The average HTER decreased by 2.03% compared to the optimal baseline, and the average AUC increased by 0.57%. After the ablation experiment, it can be seen that after removing the adapter, the HTER increased by 3.36% (8.19% vs 4.83%), indicating that the adapter is crucial for task adaptation. After removing the multimodal dual-weight focal loss, the AUC decreased by 2.06% (96.74% vs 98.80%), verifying its effect on alleviating data imbalance. After removing the SAM optimization, the cross-domain performance decreased by 2.01% (HTER = 6.84%), indicating that the flattening loss significantly improves generalization. In terms of parameter analysis, the adapter dimension, when d = 64, the HTER is the lowest (3.02%), balancing the number of parameters and performance. Scaling factor, when s = 0.1, the liveness detection accuracy is the highest (99.33%). As Figure 5 As shown in Figure 2, the HTER is lowest (4.83%) when the loss function weight value λ=1.0. Figure 6 As shown in the figure, our face liveness detection model (BG-FAS) focuses on fraudulent features (such as paper texture), while the baseline model misclassifies background noise. This model is also highly robust, with a low recognition error rate in scenes with occlusion and changing lighting. It also offers a degree of interpretability. Using attention heatmap visualization, the model can accurately locate fraudulent features (such as reflections on photo edges and screen moiré). Figure 7 (a) is the t-SNE visualization of the face liveness detection model (BG-FAS) of this scheme. Figure 7 (b) is the t-SNE visualization display without the adapter. Figure 7 (c) in the figure shows the t-SNE visualization of removing the multimodal dual-weight focal loss (w / o MDFL). Figure 7 (d) in the figure is the t-SNE visualization without the global mean (w / o GM). Figure 7 The t-SNE visualization shows that the live and fraudulent samples of this scheme are significantly separated in the feature space, and the clustering density is significantly improved.

[0075] This application also provides a method for detecting human liveness in the face. Figure 8 , the face liveness detection method includes the following steps.

[0076] S801: Obtain a face image to be detected.

[0077] After obtaining the face image to be detected, data enhancement processing can be performed on the image, and the face image after data enhancement is used as the model input.

[0078] S802: Input the face image to be detected into a face liveness detection model to obtain an output target judgment result.

[0079] The target judgment result is used to indicate whether the facial image to be detected is alive, and the facial liveness detection model is obtained based on the model training method of any of the above-mentioned facial liveness detection models. The trained facial liveness detection model can be the facial liveness detection model in the above-mentioned embodiment. That is, the facial liveness detection model can be trained using the model training method of the above-mentioned facial liveness detection model.

[0080] It can be seen that in this embodiment, performing face liveness detection based on the trained face liveness detection model can improve the accuracy of detection.

[0081] The following describes a model training device for a face liveness detection model provided in this application. The model training device for the face liveness detection model described below corresponds to the model training method for the face liveness detection model described above.

[0082] See also Figure 9 The model training device 900 of the face liveness detection model includes: a first acquisition unit 901, used to acquire multiple target face images according to a training data set; an extraction unit 902, used to extract initial feature information of the multiple target face images; a first input unit 903, used to input the initial feature information into an adaptive Transformer module to obtain target feature information for face liveness detection; a second acquisition unit 904, used to acquire an image feature vector corresponding to the target feature information; a generation unit 905, used to generate a text embedding corresponding to the target face image according to the target feature information. input vector; a judgment unit 906, used to judge the multiple target face images according to the similarity between the image feature vector and the preset text feature vector, and obtain a judgment result, wherein the judgment result is used to indicate whether the target face image is a living image; a third acquisition unit 907, used to obtain a target loss function according to the multiple target face images, the target feature vector, the text embedding vector and the judgment result; an updating unit 908, used to update the parameters of the adapter inserted in the adaptive Transformer module according to the target loss function, and obtain a trained face liveness detection model.

[0083] In one possible embodiment, in terms of inputting the initial feature information into the adaptive Transformer module to obtain target feature information for face liveness detection, the first input unit 903 is specifically used to: perform layer normalization on the initial feature information and input it into the multi-head self-attention layer to obtain weighted feature information; perform layer normalization on the weighted feature information and input it into the adapter to obtain adaptation feature information; perform layer normalization on the weighted feature information and input it into the multi-layer perceptron layer to obtain task feature information; obtain a scaling factor of the adaptation feature; and obtain target feature information for face liveness detection based on the weighted feature information, the adaptation feature information, the scaling factor, and the task feature information.

[0084] In one possible embodiment, in terms of obtaining multiple target facial images based on a training data set, the first acquisition unit 901 is specifically used to: obtain multiple initial facial images based on the training data set; perform the following operations on each of the multiple initial facial images: detect facial key points in the current initial facial image; perform affine transformation based on the facial key points to obtain multiple reference facial images; and perform data enhancement on the multiple reference facial images to obtain multiple target facial images corresponding to the current initial facial image.

[0085] In a possible embodiment, in terms of obtaining the target loss function based on the multiple target facial images, the target feature vector, the text embedding vector and the judgment result, the third acquisition unit 907 is specifically used to: respectively obtain the first target feature vectors corresponding to the multiple first target facial images, and the initial facial images corresponding to the multiple first target facial images are the same; determine the contrast loss function based on the distance between the first target feature vectors of the multiple first target facial images; obtain a multimodal dual-weight focal loss function based on the multiple target facial images and the judgment result; obtain a mean square error loss function based on the target feature vector and the text embedding vector; obtain a target loss function based on the contrast loss function, the multimodal dual-weight focal loss function and the mean square error loss function.

[0086] In a possible embodiment, in terms of obtaining a multimodal dual-weight focal loss function based on the multiple target facial images and the judgment results, the third acquisition unit 907 is specifically used to: obtain the image category and image domain to which each target facial image belongs, the image category including a living image or a fraudulent image, and the image domain is used to indicate the data source of the target facial image; according to the image category and image domain to which each target facial image belongs, obtain the number of target facial images corresponding to the image category included in each image domain; determine whether the judgment result of each target facial image is correct, and obtain the probability of correct judgment; obtain a multimodal dual-weight focal loss function based on the number of target facial images and the probability of correct judgment.

[0087] In one possible embodiment, in terms of updating the parameters of the adapter inserted in the adaptive Transformer module according to the target loss function, the updating unit 908 is specifically used to: determine the gradient of the target loss function in the current parameters of the adapter; determine the maximum disturbance of the current parameters according to the gradient; perform minimization optimization of the target loss function according to the maximum disturbance and the current parameters to obtain the optimized target parameters of the adapter; and update the current parameters of the adapter according to the target parameters.

[0088] The following describes a face liveness detection device provided by the present application. The face liveness detection device described below corresponds to the face liveness detection method described above.

[0089] See also Figure 10 The face liveness detection device 1000 includes: a fourth acquisition unit 1001, used to obtain a face image to be detected; a second input unit 1002, used to input the face image to be detected into the trained face liveness detection model described in any of the above embodiments, and obtain an output target judgment result, which is used to indicate whether the face image to be detected is alive.

[0090] See also Figure 11 , Figure 11 This is a schematic diagram of the structure of the electronic device provided by this application. Figure 11As shown, the electronic device may include: a processor (processor) 1110 , a communication interface (Communications Interface) 1120 , a memory (memory) 1130 and a communication bus 1140 , wherein the processor 1110 , the communication interface 1120 , and the memory 1130 communicate with each other via the communication bus 1140 . The processor 1110 can call the logic instructions in the memory 1130 to execute the model training method of the face liveness detection model, which includes: obtaining multiple target face images according to the training data set; extracting initial feature information of the multiple target face images; inputting the initial feature information into the adaptive Transformer module to obtain target feature information for face liveness detection; obtaining the image feature vector corresponding to the target feature information; generating a text embedding vector corresponding to the target face image according to the target feature information; judging the multiple target face images according to the similarity between the image feature vector and the preset text feature vector to obtain a judgment result, and the judgment result is used to indicate whether the target face image is a live image; obtaining a target loss function according to the multiple target face images, the target feature vector, the text embedding vector and the judgment result; updating the parameters of the adapter inserted in the adaptive Transformer module according to the target loss function to obtain a trained face liveness detection model.

[0091] Alternatively, the processor 1110 can call the logic instructions in the memory 1130 to execute a face liveness detection method, which includes: obtaining a face image to be detected; inputting the face image to be detected into a face liveness detection model to obtain an output target judgment result, wherein the target judgment result is used to indicate whether the face image to be detected is alive, and the face liveness detection model is obtained based on a model training method of any of the above-mentioned face liveness detection models.

[0092] Furthermore, the logic instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a model training method for executing the face liveness detection model provided by the above-mentioned methods, the method comprising: obtaining multiple target face images based on a training data set; extracting initial feature information of the multiple target face images; inputting the initial feature information into an adaptive Transformer module to obtain target feature information for face liveness detection; obtaining an image feature vector corresponding to the target feature information; generating a text embedding vector corresponding to the target face image based on the target feature information; judging the multiple target face images based on the similarity between the image feature vector and a preset text feature vector to obtain a judgment result, the judgment result being used to indicate whether the target face image is a live image; obtaining a target loss function based on the multiple target face images, the target feature vector, the text embedding vector and the judgment result; updating the parameters of the adapter inserted in the adaptive Transformer module based on the target loss function to obtain a trained face liveness detection model.

[0094] Alternatively, when the computer program is executed by a processor, it is implemented to execute the face liveness detection method provided by the above methods, the method including: obtaining a face image to be detected; obtaining a face image to be detected; inputting the face image to be detected into a face liveness detection model to obtain an output target judgment result, the target judgment result is used to indicate whether the face image to be detected is alive, and the face liveness detection model is obtained based on the model training method of any of the above face liveness detection models.

[0095] On the other hand, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements a model training method for any of the above-mentioned face liveness detection models, the method including: obtaining multiple target face images based on a training data set; extracting initial feature information of the multiple target face images; inputting the initial feature information into an adaptive Transformer module to obtain target feature information for face liveness detection; obtaining an image feature vector corresponding to the target feature information; generating a text embedding vector corresponding to the target face image based on the target feature information; judging the multiple target face images based on the similarity between the image feature vector and a preset text feature vector to obtain a judgment result, wherein the judgment result is used to indicate whether the target face image is a live image; obtaining a target loss function based on the multiple target face images, the target feature vector, the text embedding vector and the judgment result; updating the parameters of the adapter inserted in the adaptive Transformer module based on the target loss function to obtain a trained face liveness detection model.

[0096] Alternatively, when the computer program is executed by the processor, it implements any of the above-mentioned face liveness detection methods, the method comprising: obtaining a face image to be detected; obtaining a face image to be detected; inputting the face image to be detected into a face liveness detection model to obtain an output target judgment result, the target judgment result being used to indicate whether the face image to be detected is alive, and the face liveness detection model is obtained based on a model training method of any of the above-mentioned face liveness detection models.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0098] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A model training method for a face liveness detection model, characterized in that: include: Obtain multiple target face images based on the training data set; Extracting initial feature information of the plurality of target face images; Inputting the initial feature information into an adaptive Transformer module to obtain target feature information for face liveness detection; Obtaining an image feature vector corresponding to the target feature information; Generating a text embedding vector corresponding to the target face image according to the target feature information; Judging the plurality of target face images based on similarities between the image feature vector and a preset text feature vector to obtain a judgment result, wherein the judgment result is used to indicate whether the target face image is a living body image; Obtaining a target loss function according to the multiple target face images, the target feature vector, the text embedding vector, and the judgment result; The parameters of the adapter inserted in the adaptive Transformer module are updated according to the target loss function to obtain a trained face liveness detection model.

2. The method according to claim 1, characterized in that Inputting the initial feature information into the adaptive Transformer module to obtain target feature information for face liveness detection includes: The initial feature information is layer-normalized and then input into the multi-head self-attention layer to obtain weighted feature information; Normalizing the weighted feature information by layers and inputting it into the adapter to obtain adapted feature information; Normalizing the weighted feature information layer by layer and then inputting it into the multi-layer perceptron layer to obtain task feature information; Obtaining a scaling factor of the adaptation feature; Target feature information for performing face liveness detection is obtained according to the weighted feature information, the adaptation feature information, the scaling factor, and the task feature information.

3. The method according to claim 1, characterized in that The step of obtaining a plurality of target face images based on a training data set includes: Acquire multiple initial face images according to the training data set; Perform the following operations on each of the multiple initial face images: Detect facial key points in the current initial face image; Performing affine transformation on the facial key points to obtain multiple reference facial images; Data enhancement is performed on the multiple reference face images to obtain multiple target face images corresponding to the current initial face image.

4. The method according to claim 3, characterized in that The obtaining of a target loss function according to the plurality of target face images, the target feature vector, the text embedding vector, and the judgment result includes: respectively obtaining first target feature vectors corresponding to a plurality of first target facial images, wherein the plurality of first target facial images correspond to the same initial facial image; Determining a contrast loss function based on the distances between the first target feature vectors of the plurality of first target face images; Obtaining a multimodal dual-weight focal loss function according to the multiple target face images and the judgment result; Obtaining a mean square error loss function based on the target feature vector and the text embedding vector; A target loss function is obtained according to the contrast loss function, the multimodal dual-weight focal loss function and the mean square error loss function.

5. The method according to claim 4, characterized in that The obtaining of a multimodal dual-weight focal loss function according to the plurality of target face images and the judgment result includes: Obtaining an image category and an image domain to which each target facial image belongs, wherein the image category includes a living image or a fraudulent image, and the image domain is used to indicate a data source of the target facial image; According to the image category and image domain to which each target face image belongs, obtaining the number of target face images corresponding to the image category included in each image domain; Determining whether the judgment result of each target face image is correct, and obtaining a probability of correct judgment; A multimodal dual-weight focal loss function is obtained according to the number of the target face images and the probability of the correct judgment.

6. The method according to any one of claims 1 to 5, characterized in that The updating of the parameters of the adapter inserted into the adaptive Transformer module according to the objective loss function includes: determining the gradient of the objective loss function with respect to the current parameters of the adapter; determining a maximum disturbance of the current parameter according to the gradient; Performing minimization optimization of the target loss function according to the maximum disturbance and the current parameters to obtain optimized target parameters of the adapter; The current parameters of the adapter are updated according to the target parameters.

7. A method for detecting liveness of a face, characterized in that: include: Get the face image to be detected; The face image to be detected is input into the face liveness detection model to obtain an output target judgment result, wherein the target judgment result is used to indicate whether the face image to be detected is alive. The face liveness detection model is obtained based on the model training method of the face liveness detection model described in any one of claims 1-6.

8. A model training device for a face liveness detection model, characterized in that: include: A first acquisition unit is used to acquire multiple target face images according to a training data set; an extraction unit, configured to extract initial feature information of the plurality of target face images; A first input unit is used to input the initial feature information into an adaptive Transformer module to obtain target feature information for performing face liveness detection; A second acquisition unit is used to acquire an image feature vector corresponding to the target feature information; A generating unit, configured to generate a text embedding vector corresponding to the target face image based on the target feature information; a judgment unit, configured to judge the plurality of target face images based on similarities between the image feature vector and a preset text feature vector, and obtain a judgment result, wherein the judgment result is used to indicate whether the target face image is a living body image; A third acquisition unit is used to acquire a target loss function according to the multiple target face images, the target feature vector, the text embedding vector and the judgment result; An updating unit is used to update the parameters of the adapter inserted in the adaptive Transformer module according to the target loss function to obtain a trained face liveness detection model.

9. A face liveness detection device, characterized in that: include: A fourth acquiring unit, configured to acquire a face image to be detected; A second input unit is used to input the face image to be detected into the face liveness detection model to obtain an output target judgment result, wherein the target judgment result is used to indicate whether the face image to be detected is alive, and the face liveness detection model is obtained based on the model training method of the face liveness detection model described in any one of claims 1-6.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it implements the training method of the face liveness detection model as described in any one of claims 1 to 6 or the face liveness detection method as described in claim 7.

Images