Living body detection model training method, living body detection method and device
By introducing feature vectors from prior data and interference data into the training process of the liveness detection model, and combining the total loss function with adversarial generated noise, the model training is optimized, which solves the problem of insufficient detection performance of silent face liveness detection technology under interference data, and achieves higher detection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ARCSOFT CORP LTD
- Filing Date
- 2023-06-20
- Publication Date
- 2026-07-07
Smart Images
Figure CN116959124B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing, and more particularly to a liveness detection model training method, a liveness detection method, and an apparatus. Background Technology
[0002] In recent years, facial recognition has become increasingly popular due to its convenience and contactless nature, and is widely used in fields such as finance and security. However, precisely because it is easily accessible, it is also easily exploited by others. People can create fake faces by printing photos or recording videos to attack facial recognition systems and impersonate others. Therefore, liveness detection based on facial images is extremely important and is a prerequisite for ensuring the success of facial recognition.
[0003] Currently, there are two common types of facial liveness detection technologies. One type is user-cooperative liveness detection, where the liveness detection system randomly sends multiple facial action commands, requiring the user to respond in sequence within a specified time. The system then judges the accuracy of the user's responses to determine whether the face is live. The other type is silent liveness detection, where the user completes the liveness detection task without being aware of it.
[0004] As can be seen from the two definitions of liveness detection, silent face liveness detection breaks the formula of cooperative liveness detection, which is "completing a specified action = I am alive". It does not require user cooperation, is more humane, has a fast detection speed, and provides a better user experience.
[0005] For silent face liveness detection methods, the detection performance of existing detection schemes still needs further improvement. Summary of the Invention
[0006] To address any of the aforementioned technical problems, embodiments of this application provide a method for training a human liveness detection model, a liveness detection method, and an apparatus.
[0007] To achieve the objectives of the embodiments of this application, the embodiments of this application provide a method for training a liveness detection model, including:
[0008] Create a training dataset;
[0009] The representation vectors corresponding to each image sample in the training dataset are extracted by a feature extraction network, wherein the representation vectors include the feature vectors of the prior data contained in the image sample and the feature vectors of the interference data corresponding to the image sample.
[0010] The representation vector is input into the initial detection model, and the initial detection model is iteratively trained based on the total loss function until the model converges to obtain a liveness detection model. The total loss function is at least a weighted sum of the loss of the feature extraction network and the loss of the initial detection model.
[0011] A liveness detection method, comprising:
[0012] Acquire a target image, wherein the target image contains at least facial data;
[0013] The target image is input into the liveness detection model to obtain the liveness detection result. The liveness detection model is a model trained using the method described above.
[0014] A liveness detection model training device, comprising:
[0015] Create a module to build the training dataset;
[0016] An extraction module is used to extract the representation vector corresponding to each image sample in the training dataset through a feature extraction network, wherein the representation vector includes the feature vector of the prior data contained in the image sample and the feature vector of the interference data corresponding to the image sample;
[0017] The training module is used to input the representation vector into the initial detection model, and iteratively train the initial detection model based on the total loss function until the model converges to obtain a liveness detection model. The total loss function is composed of at least the weighted sum of the loss of the feature extraction network and the loss of the initial detection model.
[0018] A liveness detection device, characterized in that it comprises:
[0019] The acquisition module is used to acquire a target image, wherein the target image contains at least facial data;
[0020] The liveness detection module is used to input the target acquisition image into the liveness detection model to obtain the liveness detection result. The liveness detection model is a model trained using the method described above.
[0021] A storage medium storing a computer program, wherein the computer program is configured to execute the method described above when run.
[0022] An electronic device includes a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the methods described above.
[0023] One of the above technical solutions has the following advantages or beneficial effects:
[0024] The feature vectors of the prior data contained in the image samples and the feature vectors of the interference data corresponding to the image samples are used as the representation vectors of the initial detection model. The weighted sum of the loss of the feature extraction network and the loss of the initial detection model is used as the total loss function. This fully utilizes the prior knowledge of the image samples and improves the anti-interference performance of the model, so that the trained liveness detection model can simultaneously improve the detection accuracy and reduce the impact of interference data on the liveness detection results, thus optimizing the training results of the model.
[0025] In addition, using the trained liveness detection model to perform face detection on target acquisition images can simultaneously improve detection accuracy and reduce the impact of interference data on liveness detection results, thereby improving the effectiveness of liveness detection.
[0026] Other features and advantages of the embodiments of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the embodiments of this application. The objects and other advantages of the embodiments of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description
[0027] The accompanying drawings are used to provide a further understanding of the technical solutions of the embodiments of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions of the embodiments of this application.
[0028] Figure 1 A schematic flowchart illustrating the liveness detection model training method provided in this application embodiment;
[0029] Figure 2 A schematic flowchart of the liveness detection method provided in the embodiments of this application;
[0030] Figure 3 This is a schematic diagram of the structure of the liveness detection model training device provided in the embodiments of this application;
[0031] Figure 4 This is a schematic diagram of the structure of the liveness detection device provided in the embodiments of this application. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be arbitrarily combined with each other. Method
[0033] Silent face liveness detection solutions can be broadly categorized into traditional liveness detection schemes that use manually designed classification features and deep learning-based liveness detection schemes. Among them:
[0034] Liveness detection schemes based on traditional artificial design features
[0035] Early silent face detection had a clear objective: to identify the differences between live and non-live attack images, design highly targeted features, and train live and non-live face classifiers. These differences typically manifest as texture differences (e.g., moiré patterns), color differences (different color distributions in different color spaces), and non-rigid changes in facial movements analyzed from consecutive frames. To address these differences, Local Binary Pattern (LBP) features and Gabor features with different parameters are generally designed. Finally, a live and non-live feature classifier is trained using machine learning methods; the feature classifier is generally a classifier itself.
[0036] Artificially designed features are constrained by the designer's prior knowledge and require repeated experimentation and adjustments to find effective artificial features, which takes a long time. In addition, with the development of technology, the imaging of electronic products is extremely delicate, and the rise of beauty cameras has diluted the differences in image texture, making it difficult to classify artificially designed features.
[0037] Deep learning-based liveness detection scheme
[0038] When using deep learning to handle face liveness detection, it is treated as a binary or multi-class classification task. Driven by a large amount of live and non-live face data, it automatically learns features that can effectively distinguish between real people and attackers. Binary classification groups all attack images into one category, while multi-class classification categorizes attack images according to attack type, such as printing attacks, screen attacks, mask attacks, etc.
[0039] Generally, more complex neural networks (i.e., deeper and wider network structures) extract better knowledge from the training dataset. Similarly, designing wide and deep networks can directly and simply improve the accuracy of silent face liveness detection. Due to the increasing number of attack types, improving the accuracy and robustness of liveness detection results in deep learning-based liveness detection schemes is an urgent problem to be solved.
[0040] To address the aforementioned technical problems, embodiments of this application provide the following solutions, including:
[0041] Figure 1 This is a schematic flowchart illustrating the liveness detection model training method provided in an embodiment of this application. Figure 1 As shown, the method includes:
[0042] Step 101: Create a training dataset;
[0043] In the above steps, the training dataset includes at least two sets of training data, each set of training data including image samples of faces and the corresponding liveness detection results of the image samples.
[0044] The image sample can be a real human face image or a face image corresponding to different attack types; the liveness detection result is either a real human or an attack. Attack sample types include, but are not limited to, paper, cropped images, masks, screens, etc. Furthermore, this application does not limit the ratio between real human image samples and attack image samples.
[0045] For example, one set of training data includes image sample 1 and the corresponding liveness detection result of image sample 1, which is a real person; another set of training data includes image sample 2 and the corresponding liveness detection result of image sample 2, which is an attack.
[0046] Step 102: Extract the representation vector corresponding to each image sample in the training dataset through a feature extraction network, wherein the representation vector includes the feature vector of the prior data contained in the image sample and the feature vector of the interference data corresponding to the image sample;
[0047] Specifically, while models can automatically learn some highly discriminative features, they often fit to unimportant features, causing the model to locally collapse onto features that introduce noise. This application addresses this by incorporating prior information into the model, helping it learn some key features.
[0048] In the above steps, the feature vectors of the prior data embody a variety of potential information about the image and are knowledge that has been verified in advance. Therefore, introducing feature vectors with prior knowledge into the representation vector can further improve the quality of knowledge / representation during model training, so as to guide the training to iterate in the optimal direction and improve the detection accuracy of the model.
[0049] Furthermore, since liveness detection technology is often used in scenarios such as phone unlocking and financial identity verification, the ability to reliably identify fraudulent activities such as photo manipulation, face swapping, and masking is crucial for protecting user interests. A stable and reliable system must be able to withstand various disturbances under different conditions and exhibit consistent high performance. Therefore, the model's robustness is particularly critical. Thus, by introducing feature vectors representing interfering data into the representation vector, the model can uncover the interfering data contained within the main dataset, thereby improving the model's robustness.
[0050] Using the aforementioned input vectors from the training data to build the model can effectively improve the model's accuracy and robustness, optimize the model building process, and enhance its detection performance. Furthermore, the representation vectors contain effective feature information from the input data; by gradually improving the quality of the representation vectors through training, they gain the ability to identify key features, thereby achieving good classification and detection performance.
[0051] Step 103: Input the representation vector into the initial detection model, and iteratively train the initial detection model based on the total loss function until the model converges to obtain a liveness detection model. The total loss function is at least a weighted sum of the loss of the feature extraction network and the loss of the initial detection model.
[0052] During the training of the initial detection model, it is necessary to consider not only the classification loss of the initial detection model during training, but also the loss corresponding to the feature extraction of the input data, that is, the loss of the feature extraction network in the process of extracting the representation vector, so as to ensure the accuracy of the liveness detection model.
[0053] When there are at least two types of losses, the total loss function can be obtained by calculating a weighted sum, where the weights of each loss function can be preset.
[0054] The method provided in this application uses the feature vectors of prior data contained in the image sample and the feature vectors of the interference data corresponding to the image sample as the representation vectors of the initial detection model, and uses the weighted sum of the loss of the feature extraction network and the loss of the initial detection model as the total loss function. This fully utilizes the prior knowledge of the image sample and improves the anti-interference performance of the model, so that the trained liveness detection model can simultaneously improve the detection accuracy and reduce the impact of interference data on the liveness detection results, thereby optimizing the training results of the model.
[0055] The method described above in this embodiment will be further illustrated with examples below:
[0056] In one exemplary embodiment, the interference data is maximized interference noise generated by an adversarial generative network from the training dataset, and / or the training dataset is based on adversarial samples generated by the fast gradient sign method.
[0057] Because liveness detection technology requires high security in its applications, reliably identifying fraudulent activities such as photo manipulation, face swapping, and masking is crucial for protecting user interests. A stable and reliable liveness detection model must be able to withstand various disturbances under different conditions and exhibit consistently high performance. Therefore, the model's robustness is particularly critical. This application improves the model's robustness through adversarial noise generation and FGSM gradient training. The methods for improving model robustness are further described below.
[0058] The first method involves using the aforementioned interference data as maximum confusing noise generated by a Generative Adversarial Network (GAN).
[0059] The aforementioned liveness detection model is a deep learning model. Since deep learning models are susceptible to noise perturbations, their performance can become unstable. Therefore, noise needs to be input during training to help the model adapt to its influence. Noise can be of various types, such as Gaussian noise and uniform noise. However, in liveness detection models, it is difficult to determine which type or combination of noise is affecting the model. Therefore, this embodiment uses an adversarial generative network to generate the noise that the model is most difficult to adapt to, i.e., maximizing the perturbation noise.
[0060] This application maximizes the noise resistance and robustness of the liveness detection model by using the feature vector that maximizes the interference noise as part of the representation vector during model training.
[0061] In the second scenario, the aforementioned interference data is an adversarial example generated based on the Fast Gradient Sign Method (FGSM).
[0062] The aforementioned liveness detection model is a binary classification model that distinguishes between real people and attacks. Therefore, during the training of the classification model, gradient updates are achieved by subtracting the calculated gradient from the parameters, thereby reducing the loss value and increasing the probability of correct prediction. In contrast, in untargeted attacks, any classification other than the correct one is considered a successful attack. Therefore, increasing the loss value—meaning minimizing the probability that the predicted probability corresponds to the true label—achieves the desired attack effect. Thus, the gradient can be used as a perturbation to aid model training. In this embodiment, perturbation data corresponding to the gradient direction is added to the image to improve the robustness of the liveness detection model.
[0063] In the third scenario, the aforementioned interference data is a combination of two types of data.
[0064] By using the feature vector that maximizes the interference noise as part of the representation vector for model training, and by adding interference data corresponding to the gradient direction in the image, the robustness of the liveness detection model is improved in multiple dimensions.
[0065] Furthermore, this application also greatly improves model performance by introducing comprehensive prior knowledge to guide model training in the right direction.
[0066] In one exemplary embodiment, the training dataset includes pre-acquired original images and / or new images obtained by removing background data from the original images, wherein the original images contain facial data and background data.
[0067] In the above exemplary embodiments, the original image may include not only face data, but also background data behind the face data. Therefore, this application may directly use the original image as an image sample, or use the new image obtained after removing non-face data as an image sample.
[0068] Specifically, facial region data contains most of the information about a person's face. Therefore, training with images containing only facial data can reduce the amount of image data processed by the liveness detection model. However, in the liveness detection process, background information such as the edges of the attack image is helpful in determining whether a person is alive. For example, when the image sample is paper, the transition between the paper edge and the background is abrupt. Therefore, this embodiment also adds the original image containing background data to the training dataset. Images containing only facial data and images containing both facial and background data are input into the model simultaneously to achieve the effect of paying attention to both local and global information. This allows the model to not only learn facial feature information and background information separately, but also to obtain the differences between real images and attack images in the background region. This strengthens the recognition of the attack background, reduces the impact of non-facial data on the liveness detection results, and provides support for improving the accuracy of the detection model.
[0069] Furthermore, this application comprehensively considers the types of image attributes and designs dedicated tasks to help the model learn potential and important image attributes as an effective supplement to prior knowledge.
[0070] In one exemplary embodiment, the feature vector of the prior data included in the training dataset comprises the following:
[0071] The training dataset contains feature vectors corresponding to at least one image attribute, wherein the image attribute includes color, high-resolution detail, context, image angle, and face position.
[0072] Specifically, when the image attribute is color, the grayscale image of the input image sample is used to restore the color of the grayscale image through an encoder-decoder network, resulting in a restored image. A discriminator then determines whether the restoration was successful, thus completing the training process. The similarity between the restored image and the real image of the image sample can be measured, and the loss corresponding to the color can be determined based on the similarity measure, serving as part of the loss of the feature extraction network.
[0073] When the image attribute is high-resolution detail, a low-resolution image of the input image sample is used. A generator in the adversarial generative network (GCN) then generates a reconstructed resolution image. A discriminator determines whether the reconstruction was successful, thus completing the training process. The similarity loss between the reconstructed resolution image and the real image of the image sample can be determined and used as part of the loss of the feature extraction network.
[0074] When image attributes serve as context, processing techniques such as image matting and slicing / shattering are used to obtain contextual information of image samples. Image matting can be implemented using generative adversarial networks, while slicing / shattering can be achieved using Siamese neural networks to predict the arrangement of slices. The loss between the contextual information obtained through these processing techniques and the actual image samples can be used as part of the loss of the feature extraction network.
[0075] When the image attribute is image angle, each image can be used to derive four images from different angles. The training process is completed by predicting the angle of each derived image. The similarity between the predicted image angle and the real image angle can be measured and used as part of the loss of the feature extraction network.
[0076] When the image attribute is face location, an auxiliary head can be used to locate the face. The loss between the predicted face location and the actual face location in the image can be used as part of the loss of the feature extraction network.
[0077] This application fully considers comprehensive image attributes, uses multi-dimensional image attributes as prior information, effectively mines image information in real-world scenarios, and further improves the ability of representation vectors to express and describe image information.
[0078] Furthermore, in order to reduce manual annotation, this application embodiment also employs a self-supervised learning training feature extraction network to fully mine the low-level texture features of the image.
[0079] In one exemplary embodiment, the feature extraction network is a cascaded neural network, wherein the cascaded neural network includes multiple cascaded neural networks, each of which is used to extract an image attribute, and the feature vector corresponding to each image attribute is obtained based on the self-supervised learning model corresponding to each image attribute.
[0080] Self-supervised learning models utilize pretext tasks to extract supervisory information from large-scale unsupervised data. By training the network with the constructed supervisory information, they can learn representations that are valuable for downstream tasks without the need for manual annotation.
[0081] Correspondingly, the overall loss function of the feature extraction network during training can also be the loss function of the self-supervised learning model corresponding to each image attribute. By training a self-supervised learning model for each image attribute, the efficiency of obtaining the feature vectors of image attributes can be improved.
[0082] After the image samples pass through the feature extraction network to obtain the representation vector, the corresponding liveness detection prediction result can be obtained through the classification network. The liveness detection prediction result is compared with the sample label and the model parameters are then optimized.
[0083] Furthermore, in this embodiment, the aforementioned prior knowledge is learned using an assisted learning method.
[0084] In an exemplary embodiment, the initial detection model includes a first model corresponding to the main task and a second model corresponding to the auxiliary task. The loss of the initial detection model includes the loss function of the first model and the loss function of the second model. The first model is used to perform liveness binary classification detection on the representation vector, and the second model is used to determine the attack category.
[0085] Since liveness detection is a binary classification problem between real people and attacks, but attacks are also divided into various categories, such as paper attacks, cropping attacks, mask attacks, and screen attacks, etc., different attack categories require different features to be learned. For example, screen attacks require learning reflections and moiré patterns, paper attacks require learning material-related features, and mask attacks require learning mask texture and mask boundaries, etc. If different attack categories are trained as a single class, the model often collapses locally onto some features with weak discriminative power, resulting in an inability to learn features that can distinguish between different attack categories. However, the aforementioned features are also crucial for distinguishing between real people and various types of attacks. Therefore, it is necessary to incorporate prior knowledge that attacks exist in different categories. Thus, a model for determining the attack category needs to be introduced during the training of the liveness detection model.
[0086] Furthermore, in the context of liveness detection, if the determination of the attack category is difficult to solve in the binary classification detection of real people or attacks, and different attack categories have unified learnable features, then the determination of the attack category can be used as another model for learning.
[0087] Furthermore, for the liveness detection task, the ultimate goal is to distinguish between real people and attackers. The features of different attack categories serve to assist the model in performing binary classification of real people / attacks. Therefore, the first model that performs liveness binary classification detection on the representation vector is taken as the main task, and the second model that distinguishes attack categories is taken as the auxiliary task. Prior knowledge of different attack categories is used to improve the performance of liveness binary classification detection.
[0088] This application proposes to utilize an auxiliary learning mechanism to introduce training for judging attack categories, thereby improving the accuracy and robustness of the liveness detection model. Auxiliary learning refers to dividing the model into a main task and an auxiliary task, and using the auxiliary information learned from the auxiliary task to assist the model in executing the main task.
[0089] In one exemplary embodiment, the sample labels of the first model are provided by the training dataset, and the sample labels of the second model are generated by a label generator to generate auxiliary labels, wherein the optimization objective of the label generator is to reduce the loss function of the first model, and the optimization objective of the initial detection model is to reduce the loss functions of both the first model and the second model simultaneously.
[0090] In an exemplary embodiment, when the initial detection model is iteratively trained based on the total loss function until the model converges, the process includes: updating the first model and the feature extraction network based on the prediction result of the attack category of the representation vector using the second model, including: determining the attack category of the representation vector through the second model to obtain the predicted attack category; and training and iteratively updating the model parameters of the first model and the feature extraction network based on the comparison result of the auxiliary label and the predicted attack category, so as to reduce the loss function of the first model.
[0091] Correspondingly, since the initial detection model includes two models, during the iterative training of the initial detection model based on the total loss function until the model converges, the optimization objective of the initial detection model is to reduce the loss functions of both the first model and the second model simultaneously, so that the total loss function can be reduced, thereby achieving the goal of model convergence.
[0092] Furthermore, the sample labels for the first and second models are generated in different ways.
[0093] Specifically, the sample labels for the first model are provided by the training dataset, while the sample labels for the second model are generated by an auxiliary label generator.
[0094] Since the training dataset does not contain attack category labels, a label generator is used to label different attack types in order to reduce manual annotation, which can effectively avoid the problem of low efficiency of manual operation.
[0095] Furthermore, since the second model serves as an auxiliary task to the first model to assist in the judgment of the first model's main task, the optimization objective of the label generator in the second model is to reduce the loss function of the first model.
[0096] Specifically, when the initial detection model is iteratively trained based on the total loss function until the model converges, the second model updates the model parameters of the first model and the feature extraction network according to the prediction results of the attack category of the representation vector, so as to reduce the loss function of the first model.
[0097] In one implementation, the second model is used to determine the attack category of the representation vector to obtain the predicted attack category, and the model parameters of the first model and the feature extraction network are trained and iteratively updated based on the comparison result of the auxiliary label and the predicted attack category, so as to reduce the loss function of the first model.
[0098] By using the second model's prediction of the attack category of the representation vector, the model parameters of the first model and the feature extraction network are updated, thereby reducing the loss function of the first model, which can accelerate the convergence speed of the model and improve the training efficiency of the model.
[0099] Optionally, the total loss function of the initial detection model further includes an activation graph loss function, including:
[0100] During the initial detection model's processing of the representation vector, the obtained feature maps are linearly weighted to generate activation maps, wherein the activation maps are only images of facial regions in the training dataset;
[0101] The activation values in the activation map that are outside the facial region and are less than the first threshold are weighted and summed to form the activation map loss function.
[0102] The activation map is generated by linearly weighting the feature maps output from the penultimate convolutional layer of the classification network, where the weights are the weights of the last classification layer. Since the weights of the classification layer encode class information, the weighted response map has regional responses based on different classes. Activation map constraint is achieved by designing a corresponding loss function based on the activation map element values to restrict the activation regions to the desired areas.
[0103] In liveness detection schemes, the face region can contain both human-like activations and attack-like activations, while areas outside the face region (e.g., the human body region and background region) cannot contain human-like activations, only attack-like activations. Therefore, it is necessary to remove inappropriate human-like activations generated in the background and human body locations. Based on the working principle of activation map constraints, by using a loss function that suppresses human-like activations outside the face region, it is possible to limit or even remove human-like activations outside the face region.
[0104] Specifically, the output feature map of the penultimate layer of the classification network in the liveness detection model is linearly weighted to obtain the activation map, where the weights are the weights of the last classification layer in the liveness detection model. Taking a first threshold of 0 for real-person activation and a second threshold of 1 for attack activation as an example, activation values less than 0 can be considered as real-person activation, while regions with activation values greater than 1 can be considered as attack activation. Based on the above threshold setting method, a loss function is designed using activation values less than 0 to limit real-person activation. The expression for the activation map loss function is loss = -sum(where(face_outside<0)).
[0105] As can be seen from the activation map loss function above, if the activation value is less than 0, it means that the corresponding region is a real person's activation. The more positive the activation, the more aggressive the activation. Therefore, an activation value less than 0 can be regarded as a real person's activation. However, there cannot be real person's activation in areas outside the face. Therefore, the activation values less than 0 outside the face (face_outside<0) are summed to form the activation map constraint loss function, which is part of the total loss function during model training.
[0106] As can be seen from the above, the total loss function in this embodiment includes a weighted sum of loss terms such as the loss of the feature extraction network, the loss of the initial detection model, the loss of the activation graph constraint, and the loss of self-supervised learning. The specific composition of the total loss function is determined by the specific model composition of the model to be trained.
[0107] In an exemplary embodiment, prior to step S102, initial model parameters are assigned to the initial detection model through pre-training. Specifically, the pre-trained model often possesses a general ability to recognize basic information about images (e.g., pre-trained via ImageNet). The pre-trained initial detection model is typically sensitive to edge, texture, and color information of the input data. Furthermore, assigning the pre-trained model parameters to the initial detection model as initial model parameters enables the initial detection model to quickly acquire the basic ability to recognize information, improving training efficiency while laying the foundation for subsequent accurate classification learning.
[0108] Figure 2 This is a schematic flowchart of the liveness detection method provided in an embodiment of this application. Figure 2 As shown, the method includes:
[0109] Step 201: Acquire a target image, wherein the target image contains at least facial data;
[0110] Step 202: Input the target acquisition image into the liveness detection model to obtain the liveness detection result. The liveness detection model is a model trained using the method described above.
[0111] Since liveness detection models are often used in scenarios with high requirements for security, real-time performance, and robustness, the liveness detection method proposed in this implementation can solve this problem.
[0112] For example, identity recognition scenarios in the financial sector, such as facial recognition in banking services, or facial recognition payment services in transactions; or facial recognition scenarios in portable devices with cameras, such as facial recognition in bank clients; or facial unlocking functions; or identity recognition in security scenarios, such as access control systems based on facial recognition.
[0113] The portable terminal mentioned above can be a laptop, tablet, or mobile phone, etc.
[0114] The method provided in this application uses a liveness detection model to detect facial data in a target image, which can simultaneously improve detection accuracy and reduce the impact of interference data on liveness detection results, thereby improving the effectiveness of liveness detection.
[0115] Specifically, during the training process of the liveness detection model, the feature vectors of the prior data contained in the image samples and the feature vectors of the interference data corresponding to the image samples are used as the representation vectors of the initial detection model. The weighted sum of the loss of the feature extraction network and the loss of the initial detection model is used as the total loss function. This fully utilizes the prior knowledge of the image samples and improves the model's anti-interference performance. Therefore, when using the trained liveness detection model to perform face detection on the target acquisition image, it can simultaneously improve the detection accuracy of the target acquisition image and reduce the impact of interference data in the target acquisition image on the liveness detection results.
[0116] Furthermore, during the training process of the liveness detection model, the interference data used is the maximum interference noise generated by the adversarial generative network through the training dataset, and / or the training dataset is adversarial examples generated based on the fast gradient sign method. This makes the trained liveness detection model have strong noise resistance, so when using the trained liveness detection model to perform face detection on the target acquisition image, it can further reduce the influence of interference data in the target acquisition image on the liveness detection result and improve the robustness of the liveness detection model.
[0117] Furthermore, during the training process of the liveness detection model, the original image is directly used as the image sample, or a new image obtained after removing non-facial data is used as the image sample. This enables the liveness detection model to not only learn facial feature information and background information separately, but also to obtain the differences between real images and attack images in the background region, thus strengthening the recognition of the attack background. Therefore, when using the trained liveness detection model to perform face detection on the target image, if there is non-facial data in the target image, the influence of non-facial data on the liveness detection results can be reduced, improving the accuracy of the liveness detection model.
[0118] Furthermore, during the training process of the liveness detection model, comprehensive image attributes are fully considered, and multi-dimensional image attributes are used as prior information to effectively mine image information in real-world scenarios. Therefore, when using the trained liveness detection model to perform face detection on target acquisition images, it can improve the ability of the target acquisition image's representation vector to express and describe image information, thus providing support for improving the detection accuracy of target acquisition images.
[0119] Figure 3 This is a schematic diagram of the structure of the liveness detection model training device provided in an embodiment of this application. Figure 3 As shown, the device includes:
[0120] Module 301 is established to create the training dataset;
[0121] The extraction module 302, connected to the establishment module 301, is used to extract the representation vector corresponding to each image sample in the training dataset through the feature extraction network. The representation vector includes the feature vector of the prior data contained in the image sample and the feature vector of the interference data corresponding to the image sample.
[0122] The training module 303, connected to the extraction module 302, is used to input the representation vector into the initial detection model, and iteratively train the initial detection model based on the total loss function until the model converges to obtain a liveness detection model. The total loss function is at least a weighted sum of the loss of the feature extraction network and the loss of the initial detection model.
[0123] The apparatus provided in this application uses the feature vectors of prior data contained in the image sample and the feature vectors of the interference data corresponding to the image sample as the representation vectors of the initial detection model, and uses the weighted sum of the loss of the feature extraction network and the loss of the initial detection model as the total loss function. This fully utilizes the prior knowledge of the image sample and improves the anti-interference performance of the model, so that the trained liveness detection model can simultaneously improve the detection accuracy and reduce the impact of interference data on the liveness detection results, thereby optimizing the training results of the model.
[0124] In this implementation embodiment, the process by which each module performs its corresponding function can be found in the aforementioned content related to the training method of the liveness detection model, and will not be repeated here.
[0125] In this embodiment, the liveness detection model training device can also be applied to computing devices such as computers and servers, or to devices including at least one.
[0126] Figure 4 This is a schematic diagram of the structure of the liveness detection device provided in an embodiment of this application. Figure 4 As shown, the device includes:
[0127] Acquisition module 401 is used to acquire a target acquisition image, wherein the target acquisition image contains at least facial data;
[0128] The liveness detection module 402 is connected to the acquisition module 401 and is used to input the target acquisition image into the liveness detection model to obtain the liveness detection result. The liveness detection model is a model trained using the method described above.
[0129] In this implementation embodiment, the process by which each module performs its corresponding function can be found in the aforementioned content related to the liveness detection method, and will not be repeated here.
[0130] In this embodiment, the liveness detection device can also be applied to computing devices such as computers and servers, or to a cluster of computing devices including at least one computing device, to achieve the liveness detection function.
[0131] The apparatus provided in this application uses a liveness detection model to detect facial data in a target image, which can simultaneously improve detection accuracy and reduce the impact of interference data on liveness detection results, thereby improving the effectiveness of liveness detection.
[0132] This application provides a storage medium storing a computer program, wherein the computer program is configured to execute the method described in any of the preceding descriptions when it runs.
[0133] This application provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the method described in any of the preceding descriptions.
[0134] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.
Claims
1. A method for training a liveness detection model, comprising: Create a training dataset; The representation vectors corresponding to each image sample in the training dataset are extracted by a feature extraction network, wherein the representation vectors include the feature vectors of the prior data contained in the image sample and the feature vectors of the interference data corresponding to the image sample. The representation vector is input into the initial detection model, and the initial detection model is iteratively trained based on the total loss function until the model converges to obtain a liveness detection model. The total loss function is at least a weighted sum of the loss of the feature extraction network and the loss of the initial detection model. The initial detection model includes a first model corresponding to the main task and a second model corresponding to the auxiliary task. The loss of the initial detection model includes the loss function of the first model and the loss function of the second model. The first model is used to perform liveness binary classification detection on the representation vector; the second model is used to determine the attack category. The process of iteratively training the initial detection model based on the total loss function until the model converges includes: The sample labels of the first model are provided by the training dataset, and the sample labels of the second model are generated by a label generator. The optimization objective of the label generator is to reduce the loss function of the first model, and the optimization objective of the initial detection model is to reduce the loss functions of both the first model and the second model simultaneously. The second model updates the first model and the feature extraction network based on the prediction result of the attack category of the representation vector, including: The attack category is determined by the second model, and the predicted attack category is obtained. Based on the comparison results of the auxiliary labels and the predicted attack categories, the model parameters of the first model and the feature extraction network are trained and iteratively updated to reduce the loss function of the first model.
2. The method according to claim 1, characterized in that, The interference data is the maximum interference noise generated by the adversarial generative network from the training dataset, and / or the training dataset is based on adversarial samples generated by the fast gradient sign method.
3. The method according to claim 1, characterized in that, The training dataset contains pre-acquired original images and / or new images obtained by removing background data from the original images, wherein the original images contain facial data and background data.
4. The method according to claim 1, characterized in that, The feature vectors of the prior data included in the training dataset contain at least one of the following: feature vectors corresponding to at least two image attributes of the training dataset, wherein the image attributes include color, high-resolution details, context, image angle, and face position.
5. The method according to claim 4, characterized in that, The feature extraction network is a cascaded neural network, which includes multiple cascaded neural networks. Each neural network is used to extract an image attribute, and the feature vector corresponding to each image attribute is obtained based on the self-supervised learning model corresponding to each image attribute.
6. The method according to claim 1, characterized in that, The total loss function of the initial detection model also includes an activation graph loss function, including: During the initial detection model's processing of the representation vector, the obtained feature maps are linearly weighted to generate activation maps, wherein the activation maps are only images of facial regions in the training dataset; The activation values in the activation map that are outside the facial region and are less than the first threshold are weighted and summed to form the activation map loss function.
7. A method for detecting liveness, comprising: Acquire a target image, wherein the target image contains at least facial data; The target image is input into the liveness detection model to obtain the liveness detection result. The liveness detection model is a model trained using any one of the methods described in claims 1 to 6.
8. A liveness detection model training device, comprising: Create a module to build the training dataset; An extraction module is used to extract the representation vector corresponding to each image sample in the training dataset through a feature extraction network, wherein the representation vector includes the feature vector of the prior data contained in the image sample and the feature vector of the interference data corresponding to the image sample; The training module is used to input the representation vector into the initial detection model, and iteratively train the initial detection model based on the total loss function until the model converges to obtain a liveness detection model. The total loss function is at least a weighted sum of the loss of the feature extraction network and the loss of the initial detection model. The initial detection model includes a first model corresponding to the main task and a second model corresponding to the auxiliary task. The loss of the initial detection model includes the loss function of the first model and the loss function of the second model. The first model is used to perform liveness binary classification detection on the representation vector; the second model is used to determine the attack category. The process of iteratively training the initial detection model based on the total loss function until the model converges includes: The sample labels of the first model are provided by the training dataset, and the sample labels of the second model are generated by a label generator. The optimization objective of the label generator is to reduce the loss function of the first model, and the optimization objective of the initial detection model is to reduce the loss functions of both the first model and the second model simultaneously. The process of iteratively training the initial detection model based on the total loss function until the model converges includes: The second model updates the first model and the feature extraction network based on the prediction result of the attack category of the representation vector, including: The attack category is determined by the second model, and the predicted attack category is obtained. Based on the comparison results of the auxiliary labels and the predicted attack categories, the model parameters of the first model and the feature extraction network are trained and iteratively updated to reduce the loss function of the first model.
9. A liveness detection device, characterized in that, include: The acquisition module is used to acquire a target image, wherein the target image contains at least facial data; A liveness detection module is used to input the target acquisition image into a liveness detection model to obtain a liveness detection result. The liveness detection model is a model trained using any one of the methods described in claims 1 to 6.
10. A storage medium, characterized in that a computer program is stored in the storage medium, wherein, The computer program is configured to execute the method described in any one of claims 1 to 6 when it is run.
11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method as described in any one of claims 1 to 6.