Unsupervised domain generalization face anti-counterfeiting method, system, medium and electronic equipment
By employing an unsupervised domain generalization-based face anti-spoofing method, and utilizing unsupervised twin networks and data augmentation techniques, the problem of insufficient generalization ability of face recognition systems in complex environments is solved, achieving low-cost and highly robust face anti-spoofing recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2023-09-06
- Publication Date
- 2026-07-21
AI Technical Summary
Existing face recognition systems have poor generalization ability in real-world complex environments, rely on expensive manually labeled data, struggle to maintain high performance in cross-dataset tests, and lack robustness, making them vulnerable to spoofing attacks.
An unsupervised domain generalization face anti-spoofing method is adopted. By obtaining a training set of face images with labeled domain indices, an unsupervised Siamese network is used for model training. Combined with data augmentation and domain normalization, the bias of different domains is eliminated, and generalized face features are learned.
It achieves efficient and low-cost face anti-spoofing recognition under unlabeled data, improves the robustness and generalization ability of the model, reduces the labeling cost, and improves the application performance in complex scenarios.
Smart Images
Figure CN117935325B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer vision technology, and more specifically, to an unsupervised domain generalization method, system, medium, and electronic device for face anti-spoofing. Background Technology
[0002] With the rapid development of artificial intelligence technology, computer vision technology is increasingly being applied to human production and daily life. Facial recognition, as a mature biometric identification technology, is widely used in interactive intelligent applications in daily life, such as mobile phone unlocking, access control verification, and financial payments. However, existing facial recognition systems have many security vulnerabilities and are susceptible to deliberate attacks, namely, deceiving the facial recognition system by spoofing the user's face. Common methods of facial forgery include printed photos, video playback, and 3D-printed masks. Therefore, to ensure the security of facial recognition systems, facial anti-spoofing technology has been proposed to determine the authenticity of faces. Early facial anti-spoofing technologies used manually designed feature extraction operators to model the texture information, facial motion information, three-dimensional shape information, and image quality information of the face, and then determine the authenticity of the face. However, these traditional methods are based on human prior knowledge design, have poor robustness, and are difficult to use for facial anti-spoofing applications in real-world, complex environments.
[0003] With the rise of deep learning, convolutional neural networks (CNNs) with stronger feature extraction capabilities have been proposed for end-to-end learning of face anti-spoofing tasks. This involves modeling the authenticity problem as a supervised binary classification problem and training the neural network with labels to extract features. Further research has constructed finer-grained supervisory signals, such as pseudo-depth maps and reflection maps, to better train the model to extract robust face anti-spoofing features. However, these methods rely on expensive manually labeled data for model training. Face anti-spoofing data is scarce and costly, making it unsuitable for real-world applications. Furthermore, with limited labeled data, model training convergence is difficult, resulting in poor performance. Moreover, in cross-dataset testing (training data from the source domain dataset, test data from the target domain dataset, with differences in distribution between the source and target domains), the model performance drops sharply due to differences in illumination, background, and resolution distributions, indicating poor generalization ability of existing face anti-spoofing technologies and hindering their application in complex real-world scenarios.
[0004] Domain-invariant methods and causal learning methods from computer vision have been introduced into face anti-spoofing to improve generalization capabilities. Domain-invariant methods obtain a domain-invariant feature space through adversarial learning; causal learning methods mine robust face anti-spoofing features by modeling real causal relationships. However, these methods still rely on large amounts of expensive labeled data, hindering the development of face anti-spoofing in real-world scenarios. Research on the application of face anti-spoofing in complex real-world scenarios is of great significance. Summary of the Invention
[0005] In view of the deficiencies in the prior art, the purpose of this disclosure is to provide an unsupervised domain generalization method, system, medium and electronic device for face anti-spoofing.
[0006] To achieve the above objectives, according to a first aspect of this disclosure, an unsupervised domain generalization method for face anti-spoofing is provided, comprising:
[0007] Obtain a face image training set with labeled domain indices. The face image training set includes face image samples collected by different acquisition devices or environments, and each domain index corresponds to a certain acquisition device or environment.
[0008] The first positive sample is determined based on the feature similarity between the enhanced features of the current face image sample and the enhanced features of face image samples under the same domain index.
[0009] The second positive sample is determined based on the feature similarity between the domain normalized features of the current face image and the domain normalized features of face image samples under different domain indices.
[0010] The pre-defined unsupervised twin network is trained using the face image training set, and the pre-defined unsupervised twin network is optimized according to the overall loss function to determine the unsupervised twin network model.
[0011] The target face image is input into the unsupervised twin network model to determine the anti-spoofing features of the target face image.
[0012] Optionally, the method further includes:
[0013] The face image samples in the face image training set are subjected to data augmentation processing. The face image samples after data augmentation processing are then input into the preset unsupervised Siamese network to determine the augmentation features of the face image samples.
[0014] Optionally, the step of performing data augmentation processing on the face image samples in the face image training set, and inputting the face image samples after data augmentation processing into the preset unsupervised Siamese network to determine the augmentation features of the face image samples includes:
[0015] The face image sample is divided into image blocks according to a preset size;
[0016] Each image block is rotated by a preset angle to determine the rotated image block;
[0017] A preset number of rotated image blocks are sampled from the face image sample and input into the preset unsupervised Siamese network. The average feature value of the preset number of rotated image blocks is determined as the enhanced feature of the face image sample.
[0018] Optionally, the method further includes:
[0019] The enhanced features of the face image samples are subjected to domain normalization processing to determine the domain normalized features of the face image samples.
[0020] Optionally, the method further includes:
[0021] The first contrast loss function is determined based on the cosine similarity between the enhanced feature prediction value of the current face image sample and the original image features of the current face image sample.
[0022] The second contrast loss function is determined based on the cosine similarity between the original image features of the first positive sample and the enhanced feature prediction value of the current face image sample.
[0023] The third contrast loss function is determined based on the cosine similarity between the original image features of the second positive sample and the enhanced feature prediction value of the current face image sample.
[0024] The first contrast loss function, the second contrast loss function, and the third contrast loss function are weighted to determine the overall loss function.
[0025] Optionally, the step of weighting the first contrastive loss function, the second contrastive loss function, and the third contrastive loss function to determine the overall loss function includes:
[0026] L = L SRM +λ1·L IDNN +λ2·L CDNN
[0027] Where L represents the overall loss function, L SRM L represents the first contrastive loss function. IDNN Let L represent the second contrastive loss function, λ1 represent the weighting weights of the second contrastive loss function, and L represent the weighting weights of the second contrastive loss function. CDNN Let λ represent the third contrastive loss function, and λ2 represent the weighting weight of the third contrastive loss function.
[0028] Optionally, the step of using the face image training set to train a preset unsupervised twin network, and optimizing the preset unsupervised twin network according to the overall loss function to determine the unsupervised twin network model, includes:
[0029] The face image sample is input into the preset unsupervised twin network, and the face image features encoded by the preset unsupervised twin network are output.
[0030] The overall loss function is used to constrain the feature space of the current face image sample to be consistent with that of the first positive sample and the second positive sample, and the network parameters of the preset unsupervised twin network are optimized to determine the unsupervised twin network model.
[0031] According to a second aspect of this disclosure, an unsupervised domain generalization face anti-spoofing system is provided, comprising:
[0032] The acquisition module is used to acquire a face image training set with labeled domain indices. The face image training set includes face image samples acquired by different acquisition devices or environments, and each domain index corresponds to a certain acquisition device or environment.
[0033] The first determining module is used to determine the first positive sample based on the feature similarity between the enhanced features of the current face image sample and the enhanced features of face image samples under the same domain index.
[0034] The second determining module is used to determine the second positive sample based on the feature similarity between the domain normalized features of the current face image and the domain normalized features of face image samples under different domain indices.
[0035] The model training module is used to train a preset unsupervised twin network using the face image training set, and to optimize the preset unsupervised twin network according to the overall loss function to determine the unsupervised twin network model.
[0036] The third determination module is used to input the target face image into the unsupervised twin network model to determine the anti-spoofing features of the target face image.
[0037] According to a third aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the unsupervised domain generalization face anti-spoofing method provided in the first aspect of this disclosure.
[0038] According to a fourth aspect of this disclosure, an electronic device is provided, comprising:
[0039] A memory on which computer programs are stored;
[0040] A processor is configured to execute the computer program in the memory to implement the steps of the unsupervised domain generalization face anti-spoofing method provided in the first aspect of this disclosure.
[0041] Compared with the prior art, the embodiments disclosed herein have at least one of the following beneficial effects:
[0042] The above technical solution involves data augmentation processing of face image samples in the face image training set, extensively acquiring unlabeled data without the need for labeling costs, and learning generalized face features, enabling efficient and low-cost application in practical deployments. It employs first positive samples (i.e., nearest neighbor samples in the same domain) for comparative learning to eliminate face features that affect face generalization. Domain normalization processing is performed on face image samples in the face image training set to eliminate biases between different domains and remove domain features that affect face generalization. Furthermore, an unsupervised network model is introduced, trained with a large amount of unlabeled data, exhibiting strong generalization. It can also effectively improve the anti-spoofing performance of the unsupervised network model with fine-tuning using a small amount of labeled data, achieving low-cost and highly robust face anti-spoofing recognition. Attached Figure Description
[0043] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0044] Figure 1 This is a flowchart illustrating an unsupervised domain generalization face anti-spoofing method according to an exemplary embodiment.
[0045] Figure 2 This is a schematic diagram illustrating the structure of an unsupervised twin network according to an exemplary embodiment.
[0046] Figure 3 This is a flowchart illustrating a method for training a pre-defined unsupervised Siamese network according to an exemplary embodiment.
[0047] Figure 4 This is a flowchart illustrating an unsupervised domain generalization face anti-spoofing method according to another exemplary embodiment.
[0048] Figure 5 This is a flowchart illustrating a method for data augmentation processing of face image samples according to an exemplary embodiment.
[0049] Figure 6 This is a visualization of a face anti-counterfeiting recognition result according to an exemplary embodiment.
[0050] Figure 7 This is a block diagram illustrating an unsupervised domain generalization face anti-spoofing system according to an exemplary embodiment.
[0051] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation
[0052] The present disclosure will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present disclosure, but do not limit the present disclosure in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present disclosure. These all fall within the protection scope of the present disclosure.
[0053] Figure 1 This is a flowchart illustrating an unsupervised domain generalization face anti-spoofing method according to an exemplary embodiment. Figure 1 As shown, an unsupervised domain generalization face anti-spoofing method includes S11 to S15.
[0054] S11, Obtain the face image training set with the labeled domain index.
[0055] The face image training set includes face image samples collected by different acquisition devices or environments, with each field index corresponding to a certain acquisition device or environment.
[0056] As an example, a data acquisition device or environment corresponds to a domain index d. Based on the acquired face image samples x and the domain index d corresponding to each face image sample, the face image training set D = {x, d} is determined.
[0057] The face image training set can also include face image samples corresponding to multiple domain indices.
[0058] S12, determine the first positive sample based on the feature similarity between the enhanced features of the current face image sample and the enhanced features of face image samples under the same domain index.
[0059] Among them, the current face image samples are used for face anti-spoofing recognition.
[0060] In one possible embodiment, the enhanced features of the current face image sample are used as the query vector. The face image sample with the highest feature similarity to the enhanced features of the current face image sample is searched among the samples corresponding to the same domain index. This search result is then used as the first positive sample of the current face image sample. The first positive sample is a positive sample obtained through contrastive learning, used to eliminate face features that affect the generalization of face anti-spoofing.
[0061] Among them, face image samples with the same domain index d are considered as face image samples under the same domain index.
[0062] For example, for the current face image sample x, the enhancement feature corresponding to the current face image sample x is z, and the domain index is d. Construct a set of samples in the face image training set that belong to the same domain as the current face image sample x:
[0063]
[0064]
[0065] in, This represents the set of samples in the face image training set that belong to the same domain as the current face image sample x. This represents the enhanced feature of the k-th sample in the feature set. This represents the domain index of the k-th sample in the feature set of this sample.
[0066] Using the enhanced feature z of the current face image sample as the query vector, the sample with the highest feature similarity in the same domain sample set is searched as the positive sample of the current face image sample, i.e., the first positive sample:
[0067]
[0068] in, This indicates that the first positive sample corresponds to a specific sample in the sample set. Domain index in This indicates that the first positive sample represents the facial image features encoded by a pre-defined unsupervised Siamese network. Indicates the first in the sample set Enhanced features corresponding to individual face image samples.
[0069] S13. Determine the second positive sample based on the feature similarity between the domain normalized features of the current face image sample and the domain normalized features of face image samples under different domain indices.
[0070] In one possible embodiment, the domain-normalized features of the current face image sample are used as the query vector. Among samples corresponding to different domain indices of the current face image sample, the face image sample with the highest feature similarity to the domain-normalized features of the current face image sample is searched and selected as the second positive sample of the target face image. This second positive sample is also a positive sample for contrastive learning, used to eliminate domain features that affect the generalization of face anti-spoofing.
[0071] For example, for the current face image sample x, the enhancement feature corresponding to the current face image sample x is z, and the domain index is d.
[0072] Based on domain-normalized features, a sample set that does not belong to the same domain as the current face image sample is constructed:
[0073]
[0074]
[0075] in, This represents the set of samples that do not belong to the same domain as the current face image sample x. This represents the domain-normalized feature of the k-th sample in a set of samples that do not belong to the same domain as the current face image sample. This represents the domain index of the k-th sample in the feature set of this sample.
[0076] The domain-normalized features of the current face image sample x As a query vector, in a sample set that does not belong to the same domain The sample with the highest similarity to the search feature is taken as the positive sample of that sample, i.e., the second positive sample:
[0077]
[0078] in, This represents the sample set corresponding to the second positive sample. Domain index in This indicates that the second positive sample represents the facial image features encoded by a pre-defined unsupervised Siamese network. In the sample set The Middle Individual domain normalized features.
[0079] S14. The pre-defined unsupervised twin network is trained using a face image training set, and the pre-defined unsupervised twin network is optimized according to the overall loss function to determine the unsupervised twin network model.
[0080] The face image training set includes a first positive sample, a second positive sample, and samples that are neither the first positive sample nor the second positive sample.
[0081] Figure 2 This is a schematic diagram illustrating the structure of an unsupervised twin network model according to an exemplary embodiment.
[0082] like Figure 2 As shown, the preset unsupervised twin network includes a first subnetwork and a second subnetwork. The network parameters between the first subnetwork and the second subnetwork are shared, and the first subnetwork and the second subnetwork have the same structure.
[0083] The first and second sub-networks are concatenated to form a pre-defined unsupervised Siamese network. Each sub-network contains a feature extractor f and a mapper g. The initial parameters of each sub-network are randomly generated and optimized according to a first contrastive loss function, a second contrastive loss function, and a third contrastive loss function. The pre-defined unsupervised Siamese network also includes a predictor q, which is used to align the contrastively learned features.
[0084] Figure 3 This is a flowchart illustrating a method for training a pre-defined unsupervised Siamese network according to an exemplary embodiment.
[0085] like Figure 3 As shown, in one possible embodiment, face image samples are used to train a preset unsupervised twin network, and the preset unsupervised twin network is optimized according to the overall loss function to determine the unsupervised twin network model, including S31 to S32.
[0086] S31, Input the face image sample into the preset unsupervised twin network, and output the face image features encoded by the preset unsupervised twin network;
[0087] S32, using the overall loss function to constrain the feature space of the current face image sample to be consistent with that of the first positive sample and the second positive sample, optimize the network parameters of the preset unsupervised twin network, and determine the unsupervised twin network model.
[0088] The overall loss function includes a first contrastive loss function, a second contrastive loss function, and a third contrastive loss function.
[0089] The first contrastive loss function constrains the consistency between the enhanced features of the current face image sample and the predicted value of the enhanced features output by the predictor, and the original image features of the current face image sample.
[0090] The second contrastive loss function constrains the consistency between the enhanced features of the current face image sample and the predicted value of the enhanced features output by the predictor and the original image features of the first positive sample.
[0091] The enhanced features of the current face image sample are constrained by the third contrast loss function to ensure consistency between the enhanced feature prediction value output by the predictor and the original image features of the second positive sample.
[0092] As an example, the enhanced features of the current face image sample are input into the predictor of a pre-defined unsupervised Siamese network to obtain the predicted value corresponding to the current face image sample:
[0093] p i′ =q(v i )
[0094] Where, p i’ Let q represent the predicted augmented features of the current face image sample, and v represent the predictor. i This represents the enhanced features of the current face image sample.
[0095] The predicted values of the enhanced features corresponding to the current face image sample are normalized:
[0096] p i =p i / ||p i ||2
[0097] Where, pi This represents the predicted value of the enhanced features corresponding to the current face image sample after normalization.
[0098] As another example, the current face image sample x is input into the feature extractor and mapper of a pre-defined unsupervised Siamese network to obtain the original image features of the current face image sample:
[0099] z ori′ =g(f(x))
[0100] Normalize the original image features of the current face image sample:
[0101] z ori =z ori′ / ||z ori′ ||2
[0102] Among them, z ori z represents the original image features of the current face image sample after normalization. ori‘ This represents the original image features of the current face image sample.
[0103] Based on the overall loss function, gradient descent is used to optimize the parameters of the unsupervised Siamese network until the parameters converge, forming an unsupervised Siamese network model. This model can aggregate image features from positive samples for real-face recognition. The optimization process can employ parameter updates.
[0104] S15, input the target face image into the unsupervised Siamese network model to determine the anti-spoofing features of the target face image.
[0105] The target face image is processed by data augmentation and then input into an unsupervised Siamese network model trained by the model. The unsupervised Siamese network model outputs the face image features encoded by the face image, and the encoded face image features are used as the anti-spoofing features of the target face image to determine the authenticity of the current face recognition.
[0106] The above technical solution involves data augmentation processing of face image samples in the face image training set, extensively acquiring unlabeled data without the need for labeling costs, and learning generalized face features, enabling efficient and low-cost application in practical deployments. Comparative learning is performed using the first positive sample (i.e., the nearest neighbor sample in the same domain) to eliminate face features that affect face generalization. Domain normalization processing is applied to the face image samples in the face image training set to eliminate biases between different domains and remove domain features that affect face generalization. Furthermore, an unsupervised network model is introduced, trained with a large amount of unlabeled data, exhibiting strong generalization. It can also effectively improve the anti-spoofing performance of the unsupervised network model with fine-tuning using a small amount of labeled data, achieving low-cost and highly robust face anti-spoofing recognition.
[0107] Figure 4 This is a flowchart illustrating an unsupervised domain generalization face anti-spoofing method according to another exemplary embodiment.
[0108] like Figure 4 As shown, in some possible embodiments, an unsupervised domain generalization face anti-spoofing method further includes S16.
[0109] S16, perform data augmentation processing on the face image samples in the face image training set, and input the face image samples after data augmentation processing into a preset unsupervised Siamese network to determine the augmentation features of the face image samples.
[0110] Among them, data augmentation processing can employ strategies such as splitting, rotating, and fusing to accurately model the semantic features of face anti-spoofing.
[0111] Figure 5 This is a flowchart illustrating a method for data augmentation processing of face image samples according to an exemplary embodiment.
[0112] like Figure 5 As shown, in one possible embodiment, S16 includes S41 to S43.
[0113] S41, split the face image sample into image blocks according to the preset size.
[0114] like Figure 2 As shown, for example, the current face image sample x is divided into m×m image blocks of the same size according to a preset size {x}. p |p∈{1,…,m 2}}.
[0115] S42, rotate each image block by a preset angle to determine the rotated image block.
[0116] Following the example above, the preset angle can be achieved by randomly sampling angles. The image block is acquired and rotated according to a preset angle to determine the x-axis of the rotated image block. p =r(x p This method disrupts the identity structure information of a face. Here, xp represents a rotated image block, and r represents a preset angle, where each preset angle is an integer multiple of 90°.
[0117] S43, sample a preset number of rotated image blocks of the face image samples, input them into a preset unsupervised Siamese network, and determine the average feature value of the preset number of rotated image blocks as the enhanced feature of the face image samples.
[0118] Following the example above, the m×m rotated image blocks are input into the feature extractor of a pre-defined unsupervised Siamese network to obtain the image features e corresponding to each rotated image block. p =f(x) p From m×m rotated image patches, sample n rotated image patches and obtain n image features {e} corresponding to the n rotated image patches. s |s∈{1,…,n}}.
[0119] The image features of n rotated image patches are fused by averaging to obtain the average feature value of the n rotated image patches:
[0120]
[0121] Where v represents the average feature value of a preset number of rotated image patches, i.e., the enhanced features of the face image samples, n represents the preset number, and e s This represents the image features corresponding to the rotated image block.
[0122] After fusion processing, obtain Enhanced features of personal face image samples
[0123] in, This represents the number of all possible combinations of sampling n rotated image blocks from m×m rotated image blocks.
[0124] By employing the above technical solutions and data augmentation processing methods of splitting, rotating, and fusing, unlabeled data can be widely obtained without the need for labeling costs, and generalized facial features can be learned, enabling efficient and low-cost deployment in complex real-world scenarios.
[0125] like Figure 4 As shown, in some possible embodiments, an unsupervised domain generalization face anti-spoofing method further includes S17.
[0126] S17, perform domain normalization processing on the enhanced features of the face image samples to determine the domain normalized features of the face image samples.
[0127] Specifically, based on the domain index of each face image sample, the feature mean and feature variance of the enhanced features of the face image sample in each domain are determined. The difference between the enhanced features of the face image sample in each domain and the feature mean of the enhanced features of the face image sample in its domain is processed, and then divided by the feature variance of the domain to obtain the domain normalized features of each face image sample.
[0128] This explanation will be based on an example of performing domain normalization on face image samples within the domain of the current face image sample x:
[0129] Determine the feature mean and feature variance of the enhanced features of the face image samples in the domain where the current face image sample x is located. Then, based on the feature mean and feature variance, the face image samples within this domain are normalized.
[0130]
[0131] in, Let μ represent the domain-normalized features of the current face image sample x, and z represent the enhanced features of the current face image sample x. d This represents the average feature value of the enhanced features of the face image samples in the domain where the current face image sample x is located. denoted as the feature variance of the enhanced features of the face image samples in the domain where the current face image sample x is located, and ∈ represents a constant 1e-6 to prevent computational instability.
[0132] The above technical solution performs domain normalization processing on the predicted values of face image samples and the original image features to eliminate the bias between different domains and eliminate the domain features that affect the generalization of face anti-counterfeiting, thereby improving the generalization of face anti-counterfeiting recognition.
[0133] like Figure 4 As shown, in some possible embodiments, an unsupervised domain generalization face anti-spoofing method further includes S18 to S21.
[0134] S18. Determine the first contrast loss function based on the cosine similarity between the enhanced feature prediction value of the current face image sample and the original image features of the current face image sample.
[0135]
[0136] in, This represents the first contrastive loss function. p represents the number of all possible combinations of sampling n rotated image patches from m×m rotated image patches.i z represents the predicted value of the enhanced features of the current face image sample. ori This represents the original image features of the current face image.
[0137] The value of the first contrast loss function decreases as the cosine similarity between the enhanced feature prediction value of the current face image sample and the original image feature of the current face image sample increases. When the first contrast loss function is 0, it means that the enhanced feature prediction value of the face image sample is exactly the same as the original image feature of the face image sample, indicating that the unsupervised twin network has extracted the anti-spoofing features of the face.
[0138] S19, determine the second contrast loss function based on the cosine similarity between the original image features of the first positive sample and the predicted enhanced features of the current face image sample.
[0139]
[0140] in, This represents the second contrastive loss function. This represents the facial image features of the first positive sample encoded by a pre-defined unsupervised twin network.
[0141] The second contrast loss function decreases as the cosine similarity between the original image features of the first positive sample and the enhanced feature prediction value of the current face image sample increases. When the second contrast loss function is 0, it means that the original image features of the first positive sample and the enhanced feature prediction value of the current face image sample are exactly the same, indicating that the unsupervised twin network has extracted the anti-spoofing features of the face.
[0142] S20, determine the third contrast loss function based on the cosine similarity between the original image features of the second positive sample and the predicted enhanced features of the current face image sample.
[0143]
[0144] in, This represents the third contrastive loss function. This represents the facial image features of the second positive sample encoded by a pre-defined unsupervised twin network.
[0145] The third contrast loss function decreases as the cosine similarity between the original image features of the third positive sample and the enhanced feature prediction value of the current face image sample increases. When the third contrast loss function is 0, it means that the original image features of the third positive sample and the enhanced feature prediction value of the current face image sample are exactly the same, indicating that the unsupervised Siamese network has extracted the anti-spoofing features of the face.
[0146] S21, the first contrastive loss function, the second contrastive loss function, and the third contrastive loss function are weighted to determine the overall loss function.
[0147] L = L SRM +λ1·L IDNN +λ2·L CDNN
[0148] Where L represents the overall loss function, L SRM Let L represent the first contrastive loss function. IDNN Let L represent the second contrastive loss function, λ1 represent the weighting weights of the second contrastive loss function, and L represent the weighting weights of the second contrastive loss function. CDNN Let λ represent the third contrastive loss function, and λ2 represent the weighting of the third contrastive loss function.
[0149] In this disclosure, λ1 = 1 and λ2 = 1.
[0150] This disclosure is validated experimentally on seven datasets: OULU(O), CASIA(C), Replay(I), MSU(M), CelebA-Spoof(CA), 3DMAD(D), and HKBU-MARs(H). The testing protocol involves training an unsupervised model on several unlabeled source domain datasets to obtain an unsupervised Siamese network model. The model is then fine-tuned using a small amount of labeled source domain data, and tested on target domain datasets with different distributions.
[0151] For example, experiments were conducted on four datasets: OULU(O), CASIA(C), Replay(I), and MSU(M). O&C&I-to-M means that unsupervised model training was performed on the O, C, and I datasets using only unlabeled data. After the model training was completed, an unsupervised Siamese network model was obtained. Then, the model was fine-tuned using a small amount of labeled O, C, and I data. Finally, the model performance was tested and verified on the M dataset.
[0152] The evaluation metrics for the model include HTER and AUC. HTER represents the average overall error rate, which is the average of the false acceptance rate and the false rejection rate. AUC represents the area under the ROC curve (receiver operating characteristic curve). A smaller HTER and a larger AUC indicate better model performance.
[0153] The specific comparison results are shown in Tables 1, 2, 3, and 4 below. Table 1 presents a comparison of the unsupervised domain generalization accuracy results of an embodiment of this disclosure on the O, C, I, and M datasets. Table 1 shows that the unsupervised domain generalization face anti-spoofing method (UDG-FAS) of this disclosure significantly improves the generalization performance for unknown target domains, and its average AUC is better than that of ImageNet pre-training, indicating that the method of this disclosure can provide better model initialization for the FAS model. In addition, compared with recent unsupervised methods, the unsupervised domain generalization face anti-spoofing method of this disclosure greatly improves performance by eliminating the bias of face identity and domain distribution. For example, the average HTER is 6.19% lower than SimSiam.
[0154] Table 2 shows a comparison of the unsupervised domain generalization accuracy results of an embodiment of this disclosure on the I, C dataset and the M, O dataset. Table 3 shows a comparison of the unsupervised domain generalization accuracy results of an embodiment of this disclosure on the O, C, I, M, D, H dataset. Table 4 shows a comparison of the unsupervised domain generalization accuracy results of an embodiment of this disclosure on the M, C, O, CA dataset. In the tables, a lower HTER and a higher AUC indicate better generalization performance.
[0155]
[0156] Table 1
[0157]
[0158] Table 2
[0159]
[0160] Table 3
[0161]
[0162] Table 4
[0163] Figure 6 This is a visualization of a face anti-counterfeiting recognition result according to an exemplary embodiment.
[0164] like Figure 6 The visualization of the face anti-counterfeiting recognition results shown in the diagram illustrates that the unsupervised domain generalization face anti-counterfeiting method of this disclosure focuses on the facial region to predict real faces and emphasizes the use of printed paper, palms, and device edges to distinguish counterfeit faces.
[0165] Through the above technical solutions, the unsupervised domain generalization face anti-spoofing method proposed in this disclosure can achieve a lower average error rate and a higher area under the curve in cross-dataset tests with different distributions of test set data and training set data, while saving labeling costs. This shows that the unsupervised Siamese network model can learn features with better out-of-distribution generalization by utilizing unlabeled data, thus having a wider range of application scenarios.
[0166] Figure 7 This is a block diagram illustrating an unsupervised domain generalization face anti-spoofing system according to an exemplary embodiment.
[0167] Based on the same concept, this disclosure also provides an unsupervised domain generalization face anti-spoofing system, referring to... Figure 7 The unsupervised domain generalization face anti-spoofing system 100 includes an acquisition module 110, a first determination module 120, a second determination module 130, a model training module 140, and a third determination module 150.
[0168] The acquisition module 110 is used to acquire a face image training set with labeled domain indexes. The face image training set includes face image samples acquired by different acquisition devices or environments, and each domain index corresponds to an acquisition device or environment.
[0169] The first determining module 120 is used to determine the first positive sample based on the feature similarity between the enhanced features of the current face image sample and the enhanced features of face image samples under the same domain index.
[0170] The second determining module 130 is used to determine a second positive sample based on the feature similarity between the domain normalized features of the current face image and the domain normalized features of face image samples under different domain indices.
[0171] The model training module 140 is used to train a preset unsupervised twin network using the face image training set, and to optimize the preset unsupervised twin network according to the overall loss function to determine the unsupervised twin network model.
[0172] The third determining module 150 is used to input the target face image into the unsupervised twin network model to determine the anti-spoofing features of the target face image.
[0173] Optionally, the unsupervised domain generalization face spoofing system 100 also includes:
[0174] The data augmentation processing module is used to perform data augmentation processing on the face image samples in the face image training set, and input the face image samples after data augmentation processing into the preset unsupervised Siamese network to determine the augmentation features of the face image samples.
[0175] Optionally, the data augmentation processing module includes:
[0176] The splitting submodule is used to rotate each of the image blocks by a preset angle to determine the rotated image blocks;
[0177] The first determining submodule is used to rotate each of the image blocks by a preset angle to determine the rotated image block;
[0178] The second determining submodule is used to sample a preset number of rotated image blocks of the face image sample, input them into the preset unsupervised Siamese network, and determine the feature average value of the preset number of rotated image blocks as the enhanced feature of the face image sample.
[0179] Optionally, the unsupervised domain generalization face spoofing system 100 also includes:
[0180] The domain normalization processing module is used to perform domain normalization processing on the enhanced features of the face image sample to determine the domain normalized features of the face image sample.
[0181] Optionally, the unsupervised domain generalization face spoofing system 100 also includes:
[0182] The fourth determining module is used to determine the first contrast loss function based on the cosine similarity between the enhanced feature prediction value of the current face image sample and the original image features of the current face image sample;
[0183] The fifth determining module is used to determine the second contrast loss function based on the cosine similarity between the original image features of the first positive sample and the predicted value of the enhanced features of the current face image sample.
[0184] The sixth determining module is used to determine the third contrast loss function based on the cosine similarity between the original image features of the second positive sample and the predicted value of the enhanced features of the current face image sample.
[0185] The seventh determining module is used to perform weighted processing on the first contrast loss function, the second contrast loss function, and the third contrast loss function to determine the overall loss function.
[0186] Optionally, the seventh determining module includes:
[0187] L = L SRM +λ1·L IDNN +λ2·L CDNN
[0188] Where L represents the overall loss function, L SRM L represents the first contrastive loss function. IDNNLet L represent the second contrastive loss function, λ1 represent the weighting weights of the second contrastive loss function, and L represent the weighting weights of the second contrastive loss function. CDNN Let λ represent the third contrastive loss function, and λ2 represent the weighting weight of the third contrastive loss function.
[0189] Optionally, the model training module 140 includes:
[0190] The first training submodule is used to input the face image samples into the preset unsupervised twin network and output the face image features encoded by the preset unsupervised twin network.
[0191] The second training submodule is used to constrain the feature space of the current face image sample to be consistent with that of the first positive sample and the second positive sample by using the overall loss function, optimize the network parameters of the preset unsupervised twin network, and determine the unsupervised twin network model.
[0192] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.
[0193] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment.
[0194] like Figure 8 As shown, in some possible embodiments, this disclosure may also provide an electronic device 800, which may include: a processor 801, a memory 802. The electronic device 800 may also include one or more of a multimedia component 803, an input / output interface 804, and a communication component 805.
[0195] The processor 801 controls the overall operation of the electronic device 800 to complete all or part of the steps in the unsupervised domain generalization face anti-spoofing method of the first aspect described above. The memory 802 stores various types of data to support the operation of the electronic device 800. This data may include, for example, instructions for any application or method operating on the electronic device 800, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 802 or transmitted via communication component 805. The audio component also includes at least one speaker for outputting audio signals. Input / output interface 804 provides an interface between processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 805 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.
[0196] In another exemplary embodiment, a non-transitory computer-readable storage medium including program instructions is also provided. When executed by a processor, these program instructions implement the steps of the unsupervised domain generalization face anti-spoofing method of the first aspect described above. For example, the computer-readable storage medium may be the memory including the program instructions described above, which may be executed by a processor of an electronic device to complete the unsupervised domain generalization face anti-spoofing method.
[0197] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described unsupervised domain generalization face anti-spoofing method when executed by the programmable device.
[0198] The specific embodiments of this disclosure have been described above. It should be understood that this disclosure is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this disclosure. The above-described preferred features can be used in any combination without conflict.
Claims
1. A face anti-spoofing method with unsupervised domain generalization, characterized in that, include: Obtain a face image training set with labeled domain indices. The face image training set includes face image samples collected by different acquisition devices or environments, and each domain index corresponds to a certain acquisition device or environment. The first positive sample is determined based on the feature similarity between the enhanced features of the current face image sample and the enhanced features of face image samples under the same domain index. The second positive sample is determined based on the feature similarity between the domain normalized features of the current face image and the domain normalized features of face image samples under different domain indices. The face image training set is used to train a preset unsupervised twin network. The face image samples in the face image training set are subjected to data augmentation. The face image samples after data augmentation are input into the preset unsupervised twin network to determine the augmentation features of the face image samples. The preset unsupervised twin network is then optimized according to the overall loss function to determine the unsupervised twin network model. The target face image is input into the unsupervised twin network model to determine the anti-spoofing features of the target face image; The step of performing data augmentation processing on the face image samples in the face image training set, and then inputting the augmented face image samples into the preset unsupervised Siamese network to determine the augmentation features of the face image samples includes: The face image sample is divided into image blocks according to a preset size; Each image block is rotated by a preset angle to determine the rotated image block; A preset number of rotated image blocks are sampled from the face image sample and input into the preset unsupervised Siamese network. The average feature value of the preset number of rotated image blocks is determined as the enhanced feature of the face image sample.
2. The method according to claim 1, characterized in that, The method further includes: The enhanced features of the face image samples are subjected to domain normalization processing to determine the domain normalized features of the face image samples.
3. The method according to claim 1, characterized in that, The method further includes: The first contrast loss function is determined based on the cosine similarity between the enhanced feature prediction value of the current face image sample and the original image features of the current face image sample. The second contrast loss function is determined based on the cosine similarity between the original image features of the first positive sample and the enhanced feature prediction value of the current face image sample. The third contrast loss function is determined based on the cosine similarity between the original image features of the second positive sample and the enhanced feature prediction value of the current face image sample. The first contrast loss function, the second contrast loss function, and the third contrast loss function are weighted to determine the overall loss function.
4. The method according to claim 3, characterized in that, The step of weighting the first contrastive loss function, the second contrastive loss function, and the third contrastive loss function to determine the overall loss function includes: ； in, This represents the overall loss function. This represents the first contrastive loss function. This represents the second contrastive loss function. This represents the weighting of the second contrastive loss function. This represents the third contrastive loss function. This represents the weighting of the third contrastive loss function.
5. The method according to claim 1, characterized in that, The process of training a preset unsupervised twin network using the face image training set and optimizing the preset unsupervised twin network based on the overall loss function to determine the unsupervised twin network model includes: The face image sample is input into the preset unsupervised twin network, and the face image features encoded by the preset unsupervised twin network are output. The overall loss function is used to constrain the feature space of the current face image sample to be consistent with that of the first positive sample and the second positive sample, and the network parameters of the preset unsupervised twin network are optimized to determine the unsupervised twin network model.
6. An unsupervised domain generalization face anti-spoofing system, characterized in that, include: The acquisition module is used to acquire a face image training set with labeled domain indices. The face image training set includes face image samples acquired by different acquisition devices or environments, and each domain index corresponds to a certain acquisition device or environment. The first determining module is used to determine the first positive sample based on the feature similarity between the enhanced features of the current face image sample and the enhanced features of face image samples under the same domain index. The second determining module is used to determine the second positive sample based on the feature similarity between the domain normalized features of the current face image and the domain normalized features of face image samples under different domain indices. The model training module is used to train a preset unsupervised twin network using the face image training set, and to optimize the preset unsupervised twin network according to the overall loss function to determine the unsupervised twin network model. The model training module also includes a data augmentation module, which is used to perform data augmentation processing on the face image samples of the face image training set, and input the face image samples after data augmentation processing into the preset unsupervised twin network to determine the augmentation features of the face image samples. The third determination module is used to input the target face image into the unsupervised twin network model to determine the anti-spoofing features of the target face image; The data augmentation processing module includes: The splitting submodule is used to rotate each of the image blocks by a preset angle to determine the rotated image blocks; The first determining submodule is used to rotate each of the image blocks by a preset angle to determine the rotated image block; The second determining submodule is used to sample a preset number of rotated image blocks of the face image sample, input them into the preset unsupervised Siamese network, and determine the feature average value of the preset number of rotated image blocks as the enhanced feature of the face image sample.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-5.
8. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-5.