Conditional adversarial domain generalization method and network model architecture for face liveness detection

Abstract

The application discloses a face living body detection method and a network model architecture based on conditional adversarial domain generalization. The network model architecture comprises a feature generation module, a conditional domain adversarial training module and an auxiliary supervised classification module. The face living body detection method based on conditional adversarial domain generalization is used for face living body detection. In the training stage, the feature generation network and the classification network are trained through a face depth map loss, an asymmetric ternary loss, a classification loss and a domain adversarial loss. In the testing stage, the test sample is sent into the feature generation network and the classification network, so that the face living body detection is performed. The application can effectively improve the generalization performance of unknown fraud detection in cross-scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the fields of data processing, computer vision, etc., and particularly relates to a conditional adversarial domain generalization face liveness detection method and network model architecture. BACKGROUND

[0002] Face recognition, as a problem that attracts much attention, is widely used in the field of biometric recognition (for example, smart phone unlocking, access control, face payment), and face fraud attacks (for example, print attacks, video attacks, 3D mask attacks, etc.) pose a great threat to face recognition systems. Researchers have used photos published by legitimate users on social networks to easily pass the recognition and authentication of mature commercial face recognition systems. Face liveness detection technology can distinguish whether the current acquired face image is from a live sample or an attack sample, and can verify the authenticity of the user, effectively improving the security performance of the face recognition system.

[0003] With the popularity of high-definition cameras and the application of high-quality 3D masks, face liveness detection technology is facing new challenges. Specifically, the following points are manifested: the distinguishing features between prosthetic faces and live faces are increasingly subtle, and the feature overlap between classes is increasingly serious, reducing the precision and generalization performance of face liveness detection; due to the diversity of image texture features, color differences, and attack types, the face data distribution of different databases is very different, and the generalization performance of cross-scene, unknown fraud form face liveness detection is poor.

[0004] Many existing adversarial domain generalization methods only align the feature distributions of multiple source domains, ignoring the class-level alignment, and cannot guarantee that the distribution differences of multiple source domains are sufficiently fused, affecting the generalization performance of the system; in addition, the domain discriminator gives the same importance to all samples, and those inaccurate prediction difficult-to-transfer samples may affect the performance of the model. SUMMARY

[0005] The present application provides a conditional adversarial domain generalization face liveness detection method, comprising:

[0006] extracting a plurality of source domain image features from a plurality of source domain images, inputting the extracted features into a feature generator to generate feature encodings, and inputting the extracted features into a depth estimation network to output face depth loss;

[0007] inputting the feature encodings into a classifier to predict classification results, fusing the feature encodings output by the feature encoder and the classification results output by the classifier, inputting the fusion results into a domain discriminator, performing domain adversarial network training of the domain discriminator and the feature generator, and obtaining a domain adversarial loss;

[0008] The classifier is trained using a cross-entropy loss function to obtain a classification loss, and the feature encoding is trained using an asymmetric triplet training model to obtain an asymmetric triplet loss;

[0009] A comprehensive optimization target is constructed according to the face depth map loss, the asymmetric triplet loss, the classification loss and the domain adversarial loss, and the feature generator and the classifier are trained using the comprehensive optimization target;

[0010] In the test phase, the test sample is input into the feature generator and the classifier for face liveness detection.

[0011] The conditional adversarial domain generalization face liveness detection method as described above, wherein the multi-source domain image features are extracted from the multi-source domain images, specifically: inputting the multi-source domain images into an attention residual network to extract the features of the input images, and using the attention mechanism to suppress irrelevant regions in the input images while highlighting the significant features of specific local regions.

[0012] The conditional adversarial domain generalization face liveness detection method as described above, wherein the domain adversarial network training of the domain discriminator and the feature generator is performed, specifically including: multi-linear mapping-based data fusion domain adversarial training, which performs data fusion through multi-linear mapping, specifically using the tensor product of the feature encoding and the classification result vectors for data fusion.

[0013] The conditional adversarial domain generalization face liveness detection method as described above, wherein the domain adversarial network training of the domain discriminator and the feature generator is also included, which is based on entropy-weighted conditional domain adversarial training, and different weights are given to different samples according to their classification prediction results, reducing the influence of difficult-to-transfer samples with inaccurate predictions on the performance of the model.

[0014] The conditional adversarial domain generalization face liveness detection method as described above, wherein the entropy-weighted conditional domain adversarial training specifically includes:

[0015] The information entropy is used to measure the uncertainty of the classifier classification, where C is the number of classes, g c is the classification prediction result of the sample;

[0016] The entropy-aware weight is calculated according to the information entropy and is used to reweight the samples, and the weight value is smaller when the uncertainty of the classifier prediction is greater.

[0017] The conditional adversarial domain generalization face liveness detection method as described above, wherein a gradient reversal layer is added between the feature generator and the domain discriminator, that is, when backpropagating, the gradient of the feature generation part is multiplied by -λ to train the feature generator and the domain discriminator, wherein current_iters is the current iteration number of model training, and total_iters is the total iteration number.

[0018] The face liveness detection method of conditional adversarial domain generalization as described above, wherein the constructed comprehensive optimization objective is represented as follows:

[0019] L DG =λ1L Cls +λ2L Ada +λ3L Dep +λ4L trip

[0020] In the formula, L Cls , L Ada , L Dep , L trip respectively represent the cross-entropy classification loss, the domain adversarial loss, the face depth loss and the asymmetric triplet loss, and λ1-λ4 are hyperparameters.

[0021] The face liveness detection method of conditional adversarial domain generalization as described above, wherein the face depth loss is as follows:

[0022] L Dep (x,Dep)=||Dep(E(x))-I||

[0023] Wherein E(x) is the feature output by the attention residual network, Dep(E(x)) is the depth map estimated by the depth estimator in the attention residual network module, and I is the depth map estimated by the face alignment network.

[0024] The face liveness detection method of conditional adversarial domain generalization as described above, wherein the asymmetric triplet loss loss function is represented as follows:

[0025]

[0026] In the formula, are the positive sample and the negative sample, and α is the boundary threshold.

[0027] The face liveness detection method of conditional adversarial domain generalization as described above, assuming that there are N source domains, which are defined as X=X1,X2,...,X N , the domain label is defined as Y=Y1,Y2,...,Y N Each domain contains two types of samples: live face and fake face, and then the final optimization objective of adversarial training is as follows:

[0028]

[0029] In the formula, L Adais the domain adversarial loss, D is the domain discriminator, G is the feature generator, y is the domain label of sample x, x, y are subject to X, Y distribution respectively, 1 [n＝y] is the indicator function, when n=y, that is, the discriminator judges that the sample domain is correct, the indicator function is 1, otherwise 0; h(x) is the fused data, G(x) is the feature encoder output of sample x, and g(x) is the classifier output thereof.

[0030] The application also provides a conditional adversarial domain generalization face liveness detection network model architecture, comprising: a feature generation module, a conditional domain adversarial training module and an auxiliary supervised classification module; the conditional adversarial domain generalization face liveness detection method in any one of claims 1-9 is used for face liveness detection, in the training stage, the feature generation network and the classification network are trained through the face depth map loss, the asymmetric ternary loss, the classification loss and the domain adversarial loss; in the test stage, the test sample is sent into the feature generation network and the classification network, so as to perform face liveness detection.

[0031] The application realizes the beneficial effects as follows:

[0032] (1) The application uses attention residual network, metric learning and other technologies to extract more distinctive micro-features in living face and fake face, improve the tightness within the class and the distinguishability between the classes, and improve the precision of face liveness detection. In addition, the algorithm fully utilizes the difference between the living face and the fake face in the depth image, and uses the characteristics that the face depth map does not change with the scene to improve the precision and generalization performance of face liveness detection.

[0033] (2) The conditional domain adversarial network is used to improve the detection performance of unknown fraud under cross-scenarios, the domain adversarial network not only aligns the feature distribution of multiple source domains, but also considers the classification prediction information; by adding the entropy weight of pre-classification to the sample, the influence of inaccurate prediction samples on the model is reduced, and the generalization performance of unknown fraud detection under cross-scenarios is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0035] Figure 1 is the network model architecture schematic diagram of the conditional adversarial domain generalization face liveness detection method provided by the embodiments of the application;

[0036] Figure 2This is a flowchart of a face liveness detection method that generalizes to the conditional adversarial domain;

[0037] Figure 3 This is the network structure diagram of the Attention-UNet sub-model;

[0038] Figure 4 This is a schematic diagram of an asymmetric triplet sample mining strategy. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Example 1

[0041] like Figure 1 As shown, this embodiment of the invention provides a network model architecture, which is divided into a feature generation module, a conditional adversarial training module, and an auxiliary supervised classification module. A face liveness detection method using conditional adversarial domain generalization is employed. During the training phase, the feature generation network and classification network are trained using face depth map loss, asymmetric triplet loss, classification loss, and domain adversarial loss. During the testing phase, test samples are fed into the feature generation network and classification network to perform face liveness detection.

[0042] like Figure 2 As shown, the face liveness detection method with conditional adversarial domain generalization includes:

[0043] Step 210: Extract multi-source domain image features from the multi-source domain image, input the extracted features into the feature generator to generate feature codes, and input the extracted features into the depth estimation network to output the face depth loss.

[0044] First, Attention-UNet (Attention Residual Network) is used to extract features from the input image. The attention mechanism is used to suppress irrelevant regions in the input image while highlighting the salient features of specific local regions. Then, the extracted features are fed into the feature generator. The feature encoder includes ResNet-18 layers 2-4, an adaptive average pooling layer, and a fully connected layer with 512 nodes to further encode the features as input data for subsequent networks.

[0045] Figure 3The network structure of the Attention-UNet submodel is shown, the submodel uses SE-ResNet-50 as a pre-trained model, and layer0-layer4 layers thereof are used as an encoder part of the Attention-UNet, the input image is down-sampled, in a decoder part, a spatial attention mechanism is added in the up-sampling process to extract more domain-shared features and face region features that are helpful for classification, then a bilinear interpolation method is used to transform the feature map after each up-sampling into the same size, and the feature maps are spliced.

[0046] In addition, the features extracted by the Attention-UNet module are also sent to a depth estimation network. The face depth map as a kind of scene-invariant information can be used to assist the training of the model to extract more general difference clues. There is a certain depth between the mouth, nose and forehead of a live face, and the gray value of the depth map thereof should be 0. The depth information estimated by the face alignment network PRNet is used as the depth information of the assumed live face; and the depth map of the prosthesis face is set to 0 because the prosthesis face is a plane.

[0047] Specifically, the depth estimation network is trained by supervised learning, and the corresponding face depth loss is as follows:

[0048] L Dep (x,Dep)=||Dep(E(x))-I||

[0049] Wherein, E(x) is the feature output by the Attention-UNet network, Dep(E(x)) is the depth map estimated by the depth estimator in the Attention-UNet module, and I is the depth map estimated by the PRNet.

[0050] Step 220, input the feature code into the classifier to estimate the classification result, fuse the feature code output by the feature encoder and the classification result output by the classifier, input the fusion result into the domain discriminator, and perform domain discriminator and feature generator domain adversarial network training to obtain a domain adversarial loss;

[0051] The domain generalization technology assumes that there is a general feature space under visible multiple source domains and unknown target domains. The adversarial training of the domain discriminator and the feature generator is used to realize the adversarial domain generalization, and the conditional mechanism is added for the generator and the discriminator to further constrain the performance of the adversarial network, realize the conditional adversarial domain generalization, map the features of multiple source domains to the unknown target domain, and thus improve the generalization performance of the model under cross scenes.

[0052] In the embodiment of the application, the domain discriminator and the feature generator are trained by the domain adversarial network, and specifically include:

[0053] (1) Data fusion domain adversarial training based on multilinear mapping, data fusion is performed by multilinear mapping, and the specific operation is to perform data fusion by using the tensor product of two vectors of feature encoding and classification result. Assuming that x represents an input sample, G(x) is the feature encoder output of the sample x, and g(x) is the classifier output thereof, fusion of G(x) and g(x) is expressed in the following formula:

[0054]

[0055] h(x) is the fused data, which is sent to the feature generator to participate in adversarial training. Assuming that there are N source domains, which are defined as X=X1,X2,...,X N , the domain label is defined as Y=Y1,Y2,...,Y N , and each domain contains two kinds of samples: living human faces and fake human faces. The adversarial network training of the feature generator G and the domain discriminator D is a maximin optimization problem:

[0056]

[0057] In the formula, L Ada represents the adversarial training loss, y is the domain label of the sample x, x and y are subject to the distributions X and Y respectively, 1 [n＝y] is an indicator function, which is 1 when n=y, that is, when the domain of the sample is correctly judged by the discriminator, and is 0 otherwise. In the optimization process, G and D will be iteratively trained, and the optimization goal of G is to maximize the adversarial training loss, that is, to make the domain discriminator D unable to distinguish the domain of the sample, so as to extract more domain-independent features; and D hopes to minimize the adversarial training loss, so as to distinguish the source type of the sample.

[0058] Preferably, when training the model, considering that the training purposes of G and D are opposite, the G network and the D network are usually trained separately, and the training efficiency is low, therefore, a gradient reversal layer (Gradient Reversal Layer, GRL) is added between the feature generator and the domain discriminator, that is, when back propagation, the gradient of the feature generation part is multiplied by-λ, in this way, G and D can be trained at the same time, which can improve the training efficiency of GANs. Wherein, current_iters is the current iteration number of model training, and total_iters is the total iteration number.

[0059] (2) Entropy-weighted conditional domain adversarial training, different weights are given to different samples according to the classification prediction results of the samples, so as to reduce the influence of inaccurate prediction of difficult transfer samples on the model performance.

[0060] Specifically, the distribution matching of multiple domains is better by adding conditions to associate the feature generator and the domain discriminator. Therefore, the module adds conditions to the domain discriminator according to the classification prediction result, and gives a smaller weight to the classification uncertain sample, so as to reduce the influence of the inaccurate prediction sample on the model. The information entropy is used to measure the uncertainty of the classifier classification, wherein C is the number of categories, g c is the classification prediction result of the sample. The entropy perception weight is calculated according to the information entropy, and is used to reweight the sample. When the uncertainty of the classifier prediction is greater, the weight value is smaller, so that the degree of confusion of the domain discriminator is reduced during the adversarial training. Assuming that there are N source domains, which are defined as X=X1, X2,..., X N , the domain label is defined as Y=Y1, Y2,..., Y N , each domain contains two categories of samples: living face and fake face. After adding the entropy weighting adjustment, the final optimization objective of the model adversarial training is updated as follows:

[0061]

[0062] , wherein L Ada represents the domain adversarial loss, D is the domain discriminator, G is the feature generator, y is the domain label of the sample x, x and y are subject to X and Y distribution respectively, 1 [n＝y] is an indicator function, which is 1 when n=y, that is, the discriminator judges that the sample domain is correct, otherwise it is 0; h(x) is the fused data, G(x) is the feature encoder output of the sample x, and g(x) is the classifier output thereof.

[0063] Step 230, using the cross-entropy loss function to train the classifier to obtain the classification loss, and using the asymmetric triplets to train the feature encoding to obtain the asymmetric triplets loss;

[0064] In order to ensure that the feature generator generates the distinguishing features required by the system, such as the distinguishing features of living and fake, instead of other features, such as the distinguishing features of different individuals, a supervised classifier is added to the model, and a cross-entropy loss function is used as the objective function to improve the accuracy of face living body detection.

[0065] In addition, in order to reduce the overlap degree of the feature space of the living face and the false face, the algorithm introduces a metric learning technology to improve the tightness within the class and the distinguishability between the classes. Due to the diversity of the types of fraudulent faces, the feature space distribution is quite different, so it is difficult to find a common compact feature space for them, therefore, the application assumes that different types of fraudulent faces have different feature distribution spaces, and the living faces share the same feature distribution space. The algorithm introduces an asymmetric triplet loss, and adopts a batch-all negative sample mining strategy, such as the asymmetric triplet sample mining strategy shown in Figure 4 The sample with a black border is a drawing sample. If the drawing sample is a living sample, the positive sample can be a living sample in the same domain or other domains, and the negative sample is usually a false sample in the same domain. If the drawing sample is a false sample, the positive sample is a false sample in the same domain and the negative sample is a living or false sample in other domains. In this way, the living samples in different domains are compact, and the living samples and the false samples are discrete, and the false samples in different domains are discrete;

[0066] The asymmetric triplet loss loss function can be expressed as follows:

[0067]

[0068] In the formula, are the drawing sample, the positive sample and the negative sample respectively, and a is a boundary threshold.

[0069] Step 240, constructing a comprehensive optimization target according to the face depth map loss, the asymmetric triplet loss, the classification loss and the domain adversarial loss, and training the feature generator and the classifier using the comprehensive optimization target;

[0070] The comprehensive optimization target constructed by the application is expressed as follows:

[0071] L DG =λ1L Cls +λ2L Ada +λ3L Dep +λ4L trip

[0072] In the formula, L Cls , L Ada , L Dep , L trip respectively represent the cross-entropy classification loss, the domain adversarial loss, the face depth loss and the asymmetric triplet loss, and λ1-λ4 are hyperparameters. The application adopts an end-to-end method to train the model, so that a more generalized domain-shared feature space is generated, therefore, the model can be generalized to an unknown target domain.

[0073] Step 250, in the test stage, the test sample is sent into the feature generator and the classifier for face living body detection.

[0074] The detection effect of the face liveness detection method of the present application on anti-domain generalization is verified through the following experimental process:

[0075] (1) Build an experimental environment:

[0076] ① Set performance evaluation indicators: the algorithm uses AUC (Area Under the Curve) and half total error rate (Half Total Error Rate, HTER) as evaluation indicators. AUC is the area under the ROC (Receiver Operating Characteristic Curve) curve, ROC is a two-dimensional curve drawn by FAR (False Acceptance Rate) and FRR (False Reject Rate) at different thresholds, FAR is the false acceptance rate, which is the ratio of false body face to living body face; FRR is the false rejection rate, which is the ratio of living body face to false body face. To calculate the half error rate HTER, first draw the ROC curve on the verification set to find the threshold when FRR is equal to FAR; then calculate the FRR and FAR at a given threshold on the test set, and the average of FRR and FAR is the HTER indicator.

[0077] ② Select the data set: the algorithm is tested across data sets on four public data sets, Casia (abbreviated as C), Oulu-NPU (abbreviated as O), MSU-MFSD (abbreviated as M) and Replay-Attack (abbreviated as R). In the experiment, three of the four public data sets are randomly selected as the source domain (training set), and the remaining one is the target domain (test set), so there are four cross-data set experimental schemes, R&C&MtoO, O&C&RtoM, O&M&RtoC and O&C&MtoR. In the training set, a frame of image is randomly selected from each video, and in the test set, two frames of image are randomly selected from each video.

[0078] ③ Soft and hardware parameter setting: the experimental hardware environment is built as NVDIA GeForce RTX 2080Ti graphics card, the programming language is Python3.7, and the framework is PyTorch. The input of the model is preprocessed by the face detection and alignment algorithm MTCNN, and the input size of the RGB image is 256x256x3, which is randomly cropped to 128x128x3. During training, the batch size is set to 60, the model optimizer is Stochastic Gradient Descent (SGD), the momentum parameter is set to 0.9, the weight decay is set to 5e-4, the initial learning rate is 0.001, and after every 100 rounds of training, it becomes 0.1 times of the original.

[0079] (2) Algorithm comparison experiment:

[0080] ①Comparison with commonly used face liveness detection algorithms: In the four experimental schemes (R&C&MtoO, O&C&RtoM, O&M&RtoC and O&C&MtoR) across the data sets, the algorithm is compared with some existing commonly used face liveness detection algorithms: MS_LBP, BinaryCNN, IDA, ColorTexture, LBP-TOP, Auxiliary, and the experimental results are shown in Table 1.

[0081] Table 1 Comparison of the algorithm and existing commonly used face liveness detection algorithms on four public data sets

[0082]

[0083]

[0084] Table 1 shows that compared with commonly used face liveness detection algorithms, the detection accuracy of the algorithm is the highest and the error rate is the lowest. Although the commonly used face liveness detection algorithms have high testing accuracy in the same database, the accuracy decreases when testing across databases. The main reason is that these methods do not align the feature distribution from different domains, do not extract common patterns between different domains, weaken the model's extraction of features from the source domain to the specific domain, and cannot reduce the differences between different database features.

[0085] ②Comparison with other domain generalization face liveness detection algorithms

[0086] Under the four experimental schemes across the data sets, the algorithm is compared with some excellent domain generalization algorithms such as MMD-AAE, MADDG, SSDG-M, and the results are shown in Table 2.

[0087] Table 2 Comparison of the algorithm and other domain generalization face liveness detection algorithms

[0088]

[0089] From Table 2, compared with MADDG, the HTER of the algorithm is reduced by 13.6% on average, and the AUC is increased by 11.4%. The main reason is that although MADDG algorithm extracts the common features of multiple source domains by adopting the idea of multiple adversarial and double triplet minimization constraints, it only aligns the data distribution of multiple source domains in the feature, ignoring the alignment of the class distribution. Compared with SSDG-M algorithm, the HTER of the algorithm is reduced by 11.6% on average, and the AUC is increased by 7.7%. The main reason is that the SSDG algorithm does not consider the adverse effects of inaccurate observation samples. The algorithm embeds the attention mechanism into the U-Net network to extract more generalized difference clues when upsampling, adds class information as a condition, aligns the distribution of multiple source domains in feature and class distribution at the same time, and uses information entropy to control the priority of samples, which improves the generalization performance of the cross-scene live detection model.

[0090] (3) To verify the influence of each module of the algorithm on the whole model, an ablation experiment was carried out

[0091] In order to verify the influence of each module of the algorithm on the whole model, some sub-modules were deleted in the experiment, including attention mechanism (w / o attention), asymmetric triplet (w / o triplet), domain discriminator (w / o ad), multilinear mapping and entropy adjustment (w / o multilinear & entropy), depth loss (w / o depth) and so on. Then the performance of the system after deleting these sub-modules was observed to understand the role of these sub-modules to the system. The results are shown in Table 3.

[0092] Table 3 Ablation experiment of the algorithm on four public data sets

[0093]

[0094] From Table 3, when any module is removed, the system performance will decrease, which shows that all modules are beneficial to the improvement of the model generalization performance. Compared with other modules, the asymmetric triplet and domain discriminator modules have greater influence on the system performance. For example, when Casia and Replay-attack are used as the test set, removing the domain discriminator module will cause the HTER to increase by 9.9% and 10.3% respectively, and removing the triplet module will cause the HTER to increase by 14.0% and 11.5% respectively. Removing the multilinear mapping will cause the HTER of the model to increase by 6.7% and 6.0% respectively, and the AUC to decrease by 5.8% and 4.4% respectively. The main reason is that the algorithm not only aligns the feature distribution of multiple source domains, but also adds class-based semantic information through the method of multilinear mapping to align the distribution of multiple source domains at the class level, making the model have better generalization performance. In addition, adding asymmetric triplet loss as auxiliary supervision can obtain better classification boundary.

[0095] The above detailed description of the specific embodiments of the present application is provided for the purpose of further explaining the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A method for condition-adversarial domain generalization face liveness detection, characterized in that, Comprise: extracting multi-source domain image features from the multi-source domain image, inputting the extracted features into a feature generator to generate feature encodings, and inputting the extracted features into a depth estimation network to output a face depth loss; inputting the feature encodings into a classifier to predict a classification result, performing tensor product fusion on the feature encodings output by the feature encoder and the classification result output by the classifier, inputting the fusion result into a domain discriminator, performing domain adversarial network training of the domain discriminator and the feature generator, and obtaining a domain adversarial loss; Wherein, the domain adversarial network training of the domain discriminator and the feature generator comprises: multi-linear mapping-based data fusion domain adversarial training, data fusion is performed through multi-linear mapping, and the specific operation is to perform data fusion using the tensor product of the two vectors of the feature encodings and the classification result; entropy-weighted conditional domain adversarial training, based on the classification prediction result of the sample, adding a condition to the domain discriminator according to the classification prediction result, associating the feature generator and the domain discriminator by adding the condition, and giving different weights to different samples; Wherein, the information entropy is calculated according to the classification prediction result, and the entropy-aware weight is generated based on the information entropy, which is used to weight the samples and suppress the influence of prediction uncertain samples on the domain adversarial training; The classifier is trained using a cross-entropy loss function to obtain a classification loss, and the feature encodings are trained using an asymmetric triplet training model to make the live samples of different domains compact and promote the dispersion between the live samples and the fake samples and between the fake samples of different domains, thereby obtaining an asymmetric triplet loss; A comprehensive optimization target is constructed according to the face depth map loss, the asymmetric triplet loss, the classification loss and the domain adversarial loss, and the feature generator and the classifier are trained using the comprehensive optimization target; In the test phase, the test sample is sent into the feature generator and the classifier for face live detection.

2. The conditional adversarial domain generalization method for face liveness detection of claim 1, wherein, The entropy-weighted conditional domain adversarial training comprises: Using information entropy to measure the uncertainty of the classifier classification, in which C is the number of categories, g c is the classification prediction result of the sample; Entropy-aware weights are computed from the information entropy are used to re-weight the samples, with the weights becoming smaller when the classifier's prediction is more uncertain. 3.The method of claim 2, wherein, A gradient reversal layer is added between the feature generator and the domain discriminator, that is, during back propagation, the gradient of the feature generation part is multiplied by -λ to train the feature generator and the domain discriminator, wherein, current_iters is the current iteration number of model training, and total_iters is the total iteration number. 4.The method of claim 1, wherein The constructed comprehensive optimization target is represented as follows: L DG = λ1L Cls + λ2L Ada + λ3L Dep + λ4L trip In the formula, L Cls , L Ada , L Dep , L trip respectively represent cross-entropy classification loss, domain adversarial loss, face depth loss and asymmetric triplet loss, and λ1~λ4 are hyperparameters.

5. The conditional adversarial domain generalization method for face liveness detection of claim 4, wherein, The face depth loss is as follows: L Dep (x, Dep) = ||Dep(E(x)) - I|| Wherein, E(x) is the feature output by the attention residual network, Dep(E(x)) is the depth map estimated by the depth estimator in the attention residual network module, and I is the depth map estimated by the face alignment network.

6. The conditional adversarial domain generalization method for face liveness detection of claim 4, wherein, The asymmetric triplet loss loss function is represented as follows: In the formula, are the positive and negative samples, respectively, and a is the boundary threshold value.

7. The conditional adversarial domain generalization face live detection method of claim 4, wherein, Suppose there are N source domains, defined as X = X1, X2,..., X N , and domain labels defined as Y = Y1, Y2,..., Y N . Each domain contains two classes of samples: live human faces and fake human faces. The final optimization goal of the adversarial training is as follows: wherein L Ada represents the domain adversarial loss, D is the domain discriminator, G is the feature generator, y is the domain label of sample x, x, y are subject to X, Y distribution respectively, 1 [n＝y] is an indicator function, which is 1 when n=y, i.e. the discriminator judges the sample domain correctly, otherwise 0; h(x) is the fused data, G(x) is the feature encoder output of sample x, and g(x) is the classifier output thereof.

8. A conditional adversarial domain generalization face liveness detection network model architecture, characterized in that, Comprise: a feature generation module, a conditional domain adversarial training module and an auxiliary supervised classification module; The conditional adversarial domain generalization face live detection method of any one of claims 1-7 is used for face live detection, in the training phase, the feature generation network and the classification network are trained through the face depth map loss, the asymmetric triplet loss, the classification loss and the domain adversarial loss; in the test phase, the test sample is sent into the feature generation network and the classification network, thereby performing face live detection.

Images