A living body detection method based on self-supervised domain clustering and domain generalization

Abstract

The application discloses a kind of based on self-supervision domain clustering and domain generalization live body detection method, including domain information migration sample generation, construct self-supervision domain clustering network and train and using domain generalization live body detection model to train with pseudo domain label sample;First, through the data enhancement method of domain information migration generation positive and negative samples, then using self-supervision domain clustering module assigns pseudo domain label for sample, obtains the source domain data set of domain generalization live body detection;Using the way of training of antithesis, domain generalization live body detection module is trained, using memory triplet loss auxiliary optimization model, using the unbalanced memory cache update strategy to save the characteristics of previous training.The application can independently allocate domain label for face sample, and directly on the face data without domain label carries out domain generalization live body detection training, with more flexible generalization ability, while saving labeling cost, with greater application value.

Description

Technical Field

[0001] This invention relates to a liveness detection method, and more particularly to a liveness detection method based on self-supervised domain clustering and domain generalization. Background Technology

[0002] Facial recognition technology is widely used in scenarios such as identity verification, security control, social networks, and photo management. However, due to technological limitations, facial recognition technology may malfunction when faced with various spoofing attacks. One of the most common attack methods is presentational attacks, which use methods such as photographing a face, playing back a video of a face, and using fake face masks to attack the facial recognition system. Therefore, to improve the security and accuracy of facial recognition technology, liveness detection technology is receiving increasing attention from researchers.

[0003] Liveness detection technology is used to distinguish between real human faces and artificially synthesized facial information. It typically analyzes biometric features in facial images or videos to detect traces of forged faces and differentiate between still images, videos, 3D masks, and other artificially synthesized facial information. Commonly used biometric features include eyes, mouth and nose, facial expressions, surface texture, and facial albedo.

[0004] Domain-generalized liveness detection has become a popular research area in recent years. While previous deep learning-based liveness detection methods performed well on single training sets, they may fail during testing due to new and unknown forgery attacks. Domain-generalized liveness detection methods were proposed to improve the robustness and accuracy of the detection model by enabling it to better adapt to new and unknown forgery attacks. The core idea of ​​this method is to map facial information from real and forged faces, belonging to different distributions, into different feature spaces through a training algorithm, ensuring some overlap between these feature spaces. This allows the model to learn a general representation of face attacks across multiple domains. Thus, when the algorithm encounters a new attack sample, it can determine whether it is a real face based on its distribution in the feature space.

[0005] Domain generalization-based liveness detection techniques can adapt to various environments, exhibiting good generalization performance and high accuracy. During the training phase, these methods typically use the dataset to which the sample belongs as an explicit domain label to guide the model's learning. However, real-world attack samples often originate from various shooting devices and scenes, containing a mixture of multiple different domains. The specific domain label of each sample is unknown, making it impossible to directly apply traditional domain generalization models. Summary of the Invention

[0006] Purpose of the invention: The technical problem to be solved by the present invention is to provide a liveness detection method based on self-supervised domain clustering and domain generalization, which addresses the shortcomings of the existing technology.

[0007] To address the aforementioned technical problems, this invention discloses a liveness detection method based on self-supervised domain clustering and domain generalization. The method generates pseudo-domain labels for face samples through self-supervised domain clustering, and then trains the method on the sample data with the pseudo-domain labels using domain generalization to obtain the classification results for liveness detection. The specific method includes the following steps:

[0008] Step 1, self-supervised domain clustering sample generation, involves randomly selecting data belonging to different domains from images in a mixed source domain dataset as samples. Domain information transfer between two images is performed based on low-frequency signals in the image frequency domain to generate positive and negative samples. Specific methods include:

[0009] Step 1-1: Randomly sample two different face images from the mixed source domain dataset as the source image and the target image for migration. Use Fourier transform to obtain the frequency domain signals of the two images, and decompose the amplitude components from the frequency domain signals. and phase components The specific method is as follows:

[0010]

[0011]

[0012] Where x is the input source image or the target image for migration. It is the Fourier transform function, and arg(·) represents taking the phase value over the complex number;

[0013] Steps 1-2: Use a Gaussian low-pass filter to filter the amplitude component of the frequency domain signal of the source image to obtain the low-frequency signal of the source image; use a Gaussian high-pass filter to filter the amplitude component of the frequency domain signal of the target image to obtain the high-frequency signal of the target image.

[0014] Steps 1-3 involve fusing the low-frequency signal from the source image and the high-frequency signal from the target image in a proportional manner, and then using inverse Fourier transform to transform the mixed frequency domain signal back into a two-dimensional image, resulting in the image x after domain information transfer. DT The specific method is as follows:

[0015]

[0016]

[0017] in, It is the amplitude component after domain information migration, x s ,x tFor the source image and the target image to be migrated, H L H H It is a Gaussian low-pass filter and a Gaussian high-pass filter. It is the inverse Fourier transform function, where λ1 and λ2 are hyperparameters, and j is the imaginary unit;

[0018] Steps 1-4: Use contrastive learning data augmentation methods to enhance the source image and the domain information transfer image to obtain positive and negative samples. The positive sample is an image that has undergone domain information transfer and contains domain information similar to the source image. The negative sample is other target images from different domain distributions.

[0019] Step 2: Construct a self-supervised domain clustering network and train it. The training uses a domain clustering method based on contrastive learning. Construct a learning task to extract domain-related features from the positive and negative samples containing domain information obtained in Step 1, and perform domain clustering based on the distribution of samples in the feature space. The self-supervised domain clustering network consists of a dual-branch backbone network, a feature mapping head, a feature dictionary, and a feature clusterer.

[0020] The dual-branch backbone network adopts the ResNet-18 network model, with the two branches being a query encoding network and a momentum encoding network, respectively. The positive samples obtained in step 1 are fed into these two branches to extract domain-related features. The extracted domain-related features are then mapped using a feature mapping head to obtain the contrastive features q and k. + Both branches are initialized with the same weight parameters but updated using different methods. The query encoding network updates its weight parameters using gradient backpropagation during training, while the momentum encoding network updates its weight parameters using momentum updates. The momentum update method for the weight parameters is as follows:

[0021] w k =mw k +(1-m)w q

[0022] Among them, w k To query the weights of the encoding network, m is the momentum coefficient, which takes a value between 0 and 1, and w q These are the weights of the momentum coding network;

[0023] The feature mapping head adopts the structure of a multilayer perceptron and performs feature mapping on the features of two similar samples extracted by the dual-branch encoder.

[0024] The feature dictionary adopts a queue structure to store the feature representations generated by the momentum network branches of previous batches, which serve as negative sample features for comparison with the features of the current batch of samples.

[0025] The feature clusterer uses the K-means clustering algorithm to cluster the domain-related features extracted by the momentum encoder, and obtains the cluster center of each cluster.

[0026] The self-supervised domain clustering network is optimized using the sample contrast learning loss function and the cluster center contrast learning loss function.

[0027] The training of the self-supervised domain clustering network includes the following steps:

[0028] Step 2-1, calculate the cluster to which the sample belongs and the cluster center, specifically including:

[0029] Before each round of training begins, all training samples X = {x1, x2, ..., x...} are... N Input the momentum encoding branch to obtain the domain-related feature representation Z = {z1, z2, ..., z} for each sample. N}, where N represents the number of samples; then, the K-means clustering method is used to cluster the domain-related feature representation Z of each sample, obtaining the cluster to which each sample belongs and K cluster centers {c1,c2,…,c K}

[0030] Step 2-2, calculate the similarity between the contrasting features, specifically including:

[0031] Calculate the similarity sim(q,k) between features of two positive samples. + The feature dictionary stores the feature representations generated by the momentum network branches of previous batches, and the feature k stored in the feature dictionary is used to store the feature representations of previous batches. i As a negative sample feature, calculate its similarity sim(q,k) with the query sample feature q. i Similarity is calculated using the inner product of the two features.

[0032] Steps 2-3 involve constructing the comprehensive loss function for the self-supervised clustering network and training the network using this function. Specifically, this includes:

[0033] Constructing sample contrast loss L s Comparative loss with clustering L c The specific method is as follows:

[0034]

[0035]

[0036] Where r is the number of negative samples; τ and It is a temperature coefficient used to control the shape of the characteristic distribution;

[0037] The comprehensive loss function for the self-supervised domain clustering network is constructed as follows:

[0038] L CL =L s +λL c

[0039] Where λ is the balance coefficient;

[0040] The method of training a self-supervised clustering network by calculating a comprehensive loss function involves assigning the results of self-supervised domain clustering as pseudo-domain labels to the samples. Specifically, the clustering results are divided into K parts according to the clusters, with each part representing a domain; a domain number is assigned to each domain starting from 1 as a pseudo-domain label; and the corresponding pseudo-domain label is assigned to each sample of that cluster.

[0041] Step 3: Train the domain generalization liveness detection model on the samples with pseudo-domain labels. That is, use the samples that have been assigned pseudo-domain labels according to the clustering results as the source domain dataset, and use the domain generalization method based on adversarial training to extract the liveness domain invariant features of the samples. Use the final output of the trained domain generalization liveness detection model as the result of liveness detection classification, and complete the liveness detection based on self-supervised domain clustering and domain generalization.

[0042] The network structure of the domain generalized liveness detection model includes: a feature generator, a feature classifier, a domain discriminator, and a feature memory cache;

[0043] The feature generator obtains the feature representation of the sample, and feeds the generated features into the feature classifier to calculate the classification cross-entropy loss. The feature representation is then input into the domain discriminator to obtain the domain discrimination result. The feature generator and the domain discriminator are trained using adversarial training, and the domain adversarial loss is calculated to reduce the domain bias contained in the features. The generated features are concatenated with the features in the memory cache, and the memory triplet loss is calculated. The feature memory cache is then updated based on the calculation result of the loss function. Finally, the classification cross-entropy loss, adversarial loss, and memory triplet loss are used to optimize the domain generalization network model.

[0044] The specific calculation methods for the adversarial loss and memory triplet loss are as follows:

[0045] The network consisting of a feature generator and a domain discriminator is optimized using adversarial training. During gradient backpropagation of the adversarial loss, the gradient from the domain discriminator to the feature generator is reversed, making the training objectives of the domain discriminator and the feature generator opposite. The entire adversarial training takes the following form:

[0046]

[0047] Where X and Y D Let G be the sample set and D be the pseudo-domain label set, respectively; G be the feature generator; D be the domain discriminator; and K be the number of source domains. The indicator function is 1 when i = y and 0 otherwise, where y represents the domain label of sample x; the adversarial loss is calculated using the binary cross-entropy function.

[0048] The feature memory cache is used to store the feature representations from the previous iterations of the generalized liveness detection model; the features in the memory cache are concatenated with the features generated in the current iteration, and the memory triplet loss is calculated, specifically the loss function L. MemTriplet It is expressed as follows:

[0049]

[0050] Among them, F n Let M be the feature set for this iteration, and f be the feature set in the memory cache. a ,f p ,f n These represent anchor sample features, positive sample features, and negative sample features, respectively, with α being the marginal coefficient.

[0051] An unbalanced feature memory cache update strategy is used, which means that in each training iteration, the top h hard sample features of real faces and all fake face sample features from the memory triplet loss function calculation results are added to the feature memory cache.

[0052] The method described above uses classification cross-entropy loss, adversarial loss, and memory triplet loss to optimize the model, i.e., constructs the comprehensive loss function of the domain generalization liveness detection model as follows:

[0053] L dg =L cls +λ1L ada +λ3L MemTriplet

[0054] Among them, L cls λ1 and λ2 are the binary cross-entropy loss for facial liveness features; λ1 and λ2 are hyperparameters.

[0055] Beneficial effects:

[0056] (1) This invention proposes a data augmentation method for domain information transfer. By using image frequency domain transformation and frequency domain filtering and mixing, the transfer of image domain information is realized, while maintaining the original structural information of the transferred image, thereby helping the model learn better domain-related features.

[0057] (2) Pseudo-domain labels for face images are generated by using unsupervised learning methods such as contrastive learning and clustering, which solves the problem of missing face domain labels in mixed datasets and allows for domain generalization liveness detection training without relying on pre-labeled domain labels.

[0058] (3) Domain adversarial training and memory triplet loss function are adopted to learn the commonalities of different facial features, reducing the impact of domain bias. Furthermore, feature memory caching is used to enrich the features participating in domain generalization training, further improving the model's generalization ability. At the same time, this method also proposes an unbalanced memory cache update strategy to redistribute the model's attention to inter-class differences and intra-class differences between real and fake faces, thereby accelerating the model's convergence speed and improving its stability. Attached Figure Description

[0059] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0060] Figure 1 This is a schematic diagram of the network architecture according to an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram of the overall process of an embodiment of the present invention.

[0062] Figure 3 This is a diagram illustrating the enhanced data migration effect of a domain information migration according to an embodiment of the present invention.

[0063] Figure 4 This is a schematic diagram of a self-supervised domain clustering network structure according to an embodiment of the present invention. Detailed Implementation

[0064] This invention provides a liveness detection method based on self-supervised domain clustering and domain generalization, the method comprising the following:

[0065] (1) Self-supervised domain clustering sample generation

[0066] Data samples belonging to different domains are randomly selected from mixed source domain images, and positive and negative samples required by the self-supervised domain clustering algorithm are generated using the data augmentation method of domain information transfer.

[0067] The specific steps for transferring domain information between two images based on low-frequency signals in the image frequency domain are as follows:

[0068] 1.1 Two different face images are randomly sampled from a mixed source domain dataset as the source image and the target image for migration. Fourier transform is used to obtain the frequency domain signals of the two images, and amplitude and phase components are decomposed from the frequency domain signals. The specific representation of this transform is as follows:

[0069]

[0070]

[0071] Where x is the input source image or the target image for migration. It is the Fourier transform function, and arg(·) represents taking the phase value over the complex number.

[0072] 1.2 A Gaussian low-pass filter is used to filter the amplitude component of the frequency domain signal of the source image to obtain the low-frequency signal of the source image; a Gaussian high-pass filter is used to filter the amplitude component of the frequency domain signal of the migrating target image to obtain the high-frequency signal of the migrating target image.

[0073] 1.3 The low-frequency signal of the source image and the high-frequency signal of the target image are fused proportionally, and the mixed frequency domain signal is transformed back into a two-dimensional image using inverse Fourier transform to obtain the image x after domain information transfer. DT The specific migration process can be represented as follows:

[0074]

[0075]

[0076] Where, x s ,x t For the source image and the target image to be migrated, H L H H It is a Gaussian low-pass filter and a Gaussian high-pass filter. It is the inverse Fourier transform function, and λ1 and λ2 are hyperparameters.

[0077] 1.4 Common contrastive learning data augmentation methods are used to augment the source image and the domain information transfer image, and the final result is used as the input sample for the self-supervised clustering model.

[0078] (2) Construct and train a self-supervised domain clustering network.

[0079] The network is trained using a domain clustering method based on contrastive learning. Its structure consists of a two-branch backbone network, a feature mapping head, a feature dictionary, and a feature clusterer.

[0080] The dual-branch backbone network employs the ResNet-18 model, with two branches: a query encoding network and a momentum encoding network. Positive samples generated in step 1 through domain information transfer are fed into these two branches to extract domain-related features. The dual-branch backbone network is initialized with the same weight parameters but updated using different methods. Specifically, the query encoding network's weight parameters are updated during training using gradient backpropagation, while the momentum encoding network updates its weight parameters using momentum updates, thus ensuring consistency in feature extraction.

[0081] The feature mapping head employs a multilayer perceptron (MLP) structure to perform feature mapping on the features of two similar samples extracted by the dual-branch encoder. Using feature mapping helps the network identify domain-related features of each input image and improves the network's generalization ability across different faces.

[0082] The feature dictionary adopts a queue structure to store the feature representations generated by the momentum network branch of previous batches, which serve as negative sample features for comparison with the features of the current batch samples.

[0083] The feature clusterer uses the K-means clustering algorithm to cluster the domain-related features extracted by the momentum encoder, and obtains the cluster center of each cluster.

[0084] The network was optimized using a sample comparison learning loss function and a cluster center comparison learning loss function.

[0085] Step 2 describes the training steps for the self-supervised domain clustering network, which include:

[0086] 2.1 Before each round of training, all training samples X = {x1, x2, ..., x...} are... N Input the momentum encoding branch to obtain the domain-related feature representation Z = {z1, z2, ..., z} for each sample. N Then, the K-means clustering method is used to cluster Z, obtaining the cluster to which each sample belongs and K cluster centers {c1, c2, ..., c3}. K}

[0087] 2.2 For each training iteration, the samples generated in step 1 after domain information transfer are... and Input the query code network f respectively q (·) and momentum coding network f k (·), thus obtaining the feature representation h of two domain-similar samples. i ,f + :

[0088]

[0089] 2.3 The extracted domain-related features are mapped using a feature mapping head to obtain the final two contrastive features q and k. + :

[0090] q = g q (h),k + =g k (h + )

[0091] Where g(·) is a multilayer perceptron (MLP) containing one hidden layer.

[0092] 2.4 Calculate the similarity sim(q,k) between contrasting features + And the query sample feature q and the negative sample feature k stored in the feature dictionary. i The similarity sim(q,k) i Here, the similarity is calculated using the inner product of two features:

[0093] sim(v i ,v j ) = v i ·v j

[0094] 2.5 Calculate the query sample feature q and its cluster center c. q The similarity sim(q,c) q ) and query sample features q and other cluster centers c j The similarity sim(q,c) j ).

[0095] 2.6 The final comprehensive loss function is constructed as follows:

[0096] L CL =L s +λL c

[0097] L s The sample contrast loss is used to shorten the distance between positive samples and widen the distance between negative samples, serving as the primary optimization objective of the self-supervised clustering network; L c The contrastive loss is used to bring samples belonging to the same domain closer together and push samples from different domains further apart, serving as an auxiliary optimization objective for the network; λ is the balance coefficient. The contrastive loss here is constructed using the InfoNCE loss function commonly used in contrastive learning.

[0098]

[0099]

[0100] Where r,k are the number of negative samples and the number of cluster centers, respectively; τ and It is a temperature coefficient used to control the shape of the characteristic distribution.

[0101] 2.7 A self-supervised domain clustering network is trained by calculating a comprehensive loss function, and the final clustering result is used as a pseudo-domain label to be assigned to the corresponding sample. Specifically, the clustering result is divided into K parts according to the clusters, and each part represents a domain; a domain number is assigned to each domain starting from 1, which serves as a pseudo-domain label; and the corresponding pseudo-domain label is assigned to each sample in that cluster.

[0102] (3) Use the domain generalization liveness detection model to train on samples with pseudo-domain labels.

[0103] The network structure of the domain generalized liveness detection model includes a feature generator, a feature classifier, a domain discriminator, and a feature memory cache.

[0104] The feature generator obtains feature representations of the samples, which are then fed into a feature classifier to calculate the classification cross-entropy loss. The feature representations are then input into a domain discriminator to obtain the domain discrimination results. Adversarial training is used to train the feature generator and domain discriminator, calculating the domain adversarial loss to reduce the domain bias contained in the features. The generated features are concatenated with features in the memory cache, and the memory triplet loss is calculated. The feature memory cache is then updated based on the calculated loss function. Finally, the domain generalization network model is optimized using the classification cross-entropy loss, adversarial loss, and memory triplet loss.

[0105] The feature generator uses a ResNet-18 model as its backbone network and initializes the network with weight parameters pre-trained on ImageNet. Upsampled RGB face images from the source domain are input into the feature generator, and L2 regularization is used to normalize the output features to obtain the feature representation of the sample.

[0106]

[0107] Where G is the feature generator. Let be the input sample, d be the domain to which the sample belongs, and Norm be the L2 regularization function.

[0108] The domain discriminator is structured as a multilayer perceptron with one hidden layer. The domain discriminator G is used to distinguish domain-related information in the generated features and predict the domain label of the input sample. The pseudo-domain label generated in step 2 is used as supervision information to calculate the adversarial loss L. ada .

[0109] The network consisting of a feature generator and a domain discriminator is optimized using adversarial training. Specifically, during gradient backpropagation of the adversarial loss, the gradient from the domain discriminator to the feature generator is reversed, making the training objectives of the domain discriminator and the feature generator opposite. This optimizes the domain discriminator while weakening the feature generator's ability to distinguish domain-related information from samples, thus reducing domain bias in the generated features. The overall optimization form of adversarial training is as follows:

[0110]

[0111] Where X,Y D These are the sample set and the pseudo-domain label set, respectively; the adversarial loss is calculated using the binary cross-entropy function.

[0112] The feature representations from the previous iterations of the network are stored in a feature memory cache to enrich the diversity of features used in training. The features in the memory cache are concatenated with the features generated in the current iteration, and the memory triplet loss is calculated. The specific loss function is expressed as follows:

[0113]

[0114] Among them, F n Let M be the feature set for this iteration, and f be the feature set in the memory cache. a ,f p ,f n These are the anchor sample features, positive sample features, and negative sample features, respectively. α is the marginal coefficient, used to control the degree of difference in distance between samples of the same type and samples of different types.

[0115] To make the domain generalization network focus more on the inter-class differences between real and fake faces rather than their intra-class differences, an unbalanced feature memory cache update strategy is used. In each training iteration, the top h hard-sampling features of real faces and all fake face sample features calculated from the memory triplet loss function are added to the feature memory cache.

[0116] The overall loss function of the domain-generalized liveness detection model is:

[0117] L dg =L cls +λ1L ada +λ2L MemTriplet

[0118] Where λ1 and λ2 are hyperparameters.

[0119] Example:

[0120] like Figure 1 As shown, in a preferred embodiment of the present invention, a liveness detection model based on self-supervised domain clustering and domain generalization is provided. This model can generate pseudo-domain labels on hybrid liveness detection datasets with missing domain labels and perform domain generalization liveness detection training. Figure 2 As shown, the method specifically includes the following steps:

[0121] (1) Use the data augmentation method of domain information transfer to generate the positive and negative samples required by the self-supervised domain clustering algorithm.

[0122] The specific steps for transferring domain information between two images based on low-frequency signals in the image frequency domain are as follows:

[0123] 1.1 Two different face images are randomly sampled from a mixed source domain dataset as the source image and the target image for migration. In this embodiment of the invention, the mixed source domain dataset consists of multiple sample sets from different domains, each sample being a real or fake RGB face image. Fourier frequency domain transforms are performed on both images to obtain their frequency domain signals, and amplitude and phase components are decomposed from them. The specific representation of this transform is as follows:

[0124]

[0125]

[0126] Where x is the input source image or the target image for migration. It is the Fourier transform function, and arg(·) represents taking the phase value over the complex number. In this embodiment of the invention, in order to accelerate the calculation, the Fast Fourier Transform (FFT) is specifically used to perform frequency domain transformation on the image.

[0127] 1.2 A Gaussian low-pass filter is used to filter the amplitude component of the frequency domain signal of the source image to obtain the low-frequency signal of the source image; a Gaussian high-pass filter is used to filter the amplitude component of the frequency domain signal of the target image to obtain the high-frequency signal of the target image. In this embodiment of the invention, the cutoff frequency parameter D0 of both the Gaussian low-pass filter and the Gaussian high-pass filter is between [1, 5] to ensure that the domain information of the source image and the structural information of the target image can be effectively extracted.

[0128] 1.3 The low-frequency signal of the source image and the high-frequency signal of the target image are fused proportionally, and the mixed frequency domain signal is transformed back into a two-dimensional image using inverse Fourier transform to obtain the image x after domain information transfer. DT The specific migration process can be represented as follows:

[0129]

[0130]

[0131] Where, x s ,x t For the source image and the target image to be migrated, H L H H It is a Gaussian low-pass filter and a Gaussian high-pass filter. It is the inverse Fourier transform function, and λ1 and λ2 are hyperparameters.

[0132] Figure 3This demonstrates the effectiveness of the domain information transfer data augmentation method. The left side shows the source face image, the first row on the right shows the target face image, and the second row shows the face image after domain information transfer. The face images in the figure are selected from four public datasets: MSU-MFSD, CASIA-SURF, OULU-NPU, and Idiap Replay-Attack.

[0133] 1.4 Auxiliary data augmentation is performed on the source image and the domain information transfer image. In this invention, random image cropping, random horizontal flipping, and random blurring are used to augment the image data, thereby improving the robustness of subsequent model training. The final result is used as the input sample for the self-supervised domain clustering model.

[0134] (2) Construct and train a self-supervised domain clustering network.

[0135] Training was performed using a domain clustering method based on contrastive learning. Figure 4 This paper demonstrates the specific structure of the Self-Supervised Domain Clustering Network (SDCN). SDCN consists of a query encoding network, a momentum encoding network, a feature mapping head, a feature dictionary, and a clustering module. The network is optimized using sample contrastive loss and cluster contrastive loss functions. Its training steps include:

[0136] 2.1 Before each round of training, all training samples X = {x1, x2, ..., x...} are... N Input the momentum encoding branch to obtain the domain-related feature representation Z = {z1, z2, ..., z} for each sample. N Then, the K-means clustering method is used to cluster Z, obtaining the cluster to which each sample belongs and K cluster centers {c1, c2, ..., c3}. K In this embodiment of the invention, outliers in the clustering results were discarded to reduce the impact of noisy data on training.

[0137] 2.2 For each training iteration, the samples generated in step 1 after domain information transfer are... and Enter the query code network F respectively q (·) and momentum coding network F k (·). The encoding network uses the ResNet-18 model. Both branches are initialized with the same weight parameters but updated using different methods. Specifically, the query encoding network updates its weight parameters using gradient backpropagation during training; the momentum encoding network updates its weight parameters using momentum updates, thus ensuring consistency in feature extraction. In this example, the momentum update method for the weight parameters is: w k=mw k +(1-m)w q m = 0.999. Feature representations h of similar samples from two domains are extracted using a dual-branch coding network. i ,h + :

[0138]

[0139] 2.3 The extracted domain-related features are mapped using a feature mapping head to maximize SDCN's ability to identify domain-similar samples. This ultimately yields two contrasting features q and k. + :

[0140] q = g q (h),k + =g k (h + )

[0141] Where g(·) is a multilayer perceptron (MLP) with one hidden layer and an output dimension of 128.

[0142] 2.4 Calculate the similarity sim(q,k) between features of two positive samples + The feature dictionary stores the feature representations generated by the momentum network branches of previous batches, and the feature k stored in the feature dictionary is used to store the feature representations of previous batches. i As a negative sample feature, calculate its similarity sim(q,k) with the query sample feature q. i Here, the similarity is calculated using the inner product of two features:

[0143] sim(v i ,v j ) = v i ·v j

[0144] 2.5 Calculate the query sample feature q and its cluster center c. q The similarity sim(q,c) q ) and query sample features q and other cluster centers c j The similarity sim(q,c) j ).

[0145] 2.6 The final comprehensive loss function is constructed as follows:

[0146] L CL =L s +λL c

[0147] L sThe sample contrast loss is used to shorten the distance between positive samples and widen the distance between negative samples, serving as the primary optimization objective of the self-supervised clustering network; L c The contrastive loss is used to bring samples belonging to the same domain closer together and push samples from different domains further apart, serving as an auxiliary optimization objective for the network; λ is the balance coefficient. The contrastive loss here is constructed using the InfoNCE loss function commonly used in contrastive learning.

[0148]

[0149]

[0150] Where r and K are the number of negative samples and the number of cluster centers, respectively; τ and This is a temperature coefficient used to control the shape of the feature distribution. By optimizing the contrastive loss function, the distribution differences of samples belonging to different latent domains in the high-dimensional feature space can be gradually increased.

[0151] 2.7 A self-supervised domain clustering network is trained by calculating a comprehensive loss function, and the final clustering result is used as a pseudo-domain label to be assigned to the corresponding sample. Specifically, the clustering result is divided into K parts according to the clusters, and each part represents a domain; a domain number is assigned to each domain starting from 1, which serves as a pseudo-domain label; and the corresponding pseudo-domain label is assigned to each sample in that cluster.

[0152] (3) Use the domain generalization liveness detection model to train on samples with pseudo-domain labels.

[0153] The face samples assigned pseudo-domain labels in step 2 are used as the source domain training dataset and fed into the domain-reverting liveness detection model for training. By modeling and learning the intra-domain and inter-domain differences in data from different domains, the model can better adapt to data from different domains, reduce the impact of sample domain shift, generate more effective face liveness features, and thus improve the model's generalization ability.

[0154] First, a feature generator is used to extract facial liveness feature representations from the samples. The feature generator uses a ResNet-18 model as the backbone network and initializes the network with weight parameters pre-trained on ImageNet. L2 regularization is then used to normalize the output features to obtain the feature representations of the samples.

[0155]

[0156] Where G is the feature generator. Let be the input sample, d be the domain to which the sample belongs, and Norm be the L2 regularization function.

[0157] For generated facial liveness features, their distribution in the feature space varies significantly due to the samples originating from different latent domains. In this invention, domain adversarial training is used to match the distributions across multiple source domains, thereby guiding the feature generator to generate domain-invariant features of the samples. A domain discriminator is added after the feature generator to construct the domain adversarial training module. The domain discriminator is a multilayer perceptron with one hidden layer. The discriminator G distinguishes the domain-related information of the generated features and predicts the domain label of the input sample. Subsequently, the pseudo-domain label generated in step 2 is used as supervision information to calculate the adversarial loss L. ad Adversarial training is performed using gradient backpropagation, specifically as follows:

[0158] The ability of the feature discriminator to identify the domain of a feature is optimized through gradient backpropagation. A gradient inversion layer is inserted between the domain discriminator and the feature generator to invert the gradients propagated to the feature generator. This reverses the training objectives of the domain discriminator and the feature generator, thereby weakening the feature generator's ability to distinguish domain-related information from samples and reducing domain bias in the generated features. The entire adversarial training process can be represented as the following optimization problem:

[0159]

[0160] Where X,Y D The dataset consists of a sample set and a pseudo-domain label set; the adversarial loss is calculated using the cross-entropy function. Adversarial training is performed by minimizing the training error of the domain discriminator and maximizing the generation error of the feature generator.

[0161] For liveness detection tasks, we aim to maximize the difference in the distribution of real and fake faces in the feature space. Therefore, triplet loss functions are typically used to assist model training. Previous methods only used features from the same batch to calculate triplet loss, failing to leverage previous features for optimization. To enrich the diversity of features used in training and improve the model's training convergence speed and stability, this invention employs triplet loss with a feature memory cache. The feature representations from the network's previous iterations are stored in the feature memory cache. The features in the memory cache are concatenated with the features generated in the current iteration to calculate the memory triplet loss. The specific loss function is expressed as follows:

[0162]

[0163] Among them, F n Let M be the feature set for this iteration, and f be the feature set in the memory cache. a ,f p ,f nThese are the anchor sample features, positive sample features, and negative sample features, respectively. α is the marginal coefficient, used to control the degree of difference in distance between samples of the same type and samples of different types.

[0164] Triple loss helps encode similar samples into similar feature spaces while encoding dissimilar samples into distant feature spaces. However, due to the diversity of real faces caused by factors such as skin color, environment, and makeup, focusing too much on narrowing the distance between encoded features can easily lead to model overfitting. To make the model focus more on the inter-class differences between real and fake faces rather than their intra-class differences, this invention uses an unbalanced feature memory cache update strategy. Specifically, in each training iteration, the top h hard-sampling features of real faces and all fake face sample features from the triple loss function calculation are added to the feature memory cache. In this embodiment, h = 10.

[0165] Finally, the generated features are fed into a liveness feature classifier to predict the liveness detection result, and the binary cross-entropy loss L is calculated. cls .

[0166] The comprehensive loss function of the domain generalization adversarial network described in this invention example is as follows:

[0167] L dg =L cls +λ1L ada +λ2L MemTriplet

[0168] Where λ1 and λ2 are hyperparameters. By optimizing the above loss function, domain generalization training is performed, enabling the model to generate domain-invariant liveness features of faces, thereby generalizing to unknown domains.

[0169] The liveness detection network trained and converged as described above can be applied to liveness detection tasks of real-world RGB face images and exhibits good generalization ability. During the testing phase, the feature generator of the domain-generalized liveness detection network is used to obtain the feature representations of the samples, which are then fed into a feature classifier to obtain the final liveness detection results. The following specific experiments verify the liveness detection performance of this invention.

[0170] Experimental Data: The effectiveness of the proposed method was evaluated using four publicly available liveness detection datasets: Oulu-NPU (O), CASIA-MFSD (C), IDIAP ReplayAttack (I), and MSU-MFSD (M). The MTCNN model was used to preprocess the face videos provided in the four datasets, sampling valid frames as experimental images and cropping face regions from them, resulting in 32,800 data points. In the experiments, three datasets were randomly selected and merged into a large mixed source domain dataset. The data in the mixed source domain dataset did not contain any domain labels, and the remaining dataset was used as the target domain for cross-dataset testing. Therefore, this experiment included a total of four test tasks: O&C&I to M, O&M&I to C, O&C&M to I, and I&C&M to O.

[0171] Experimental parameters: In the experiment, the model of this invention was trained and tested on a hardware environment with four RTX 2080Ti GPUs. The batch size of the self-supervised domain clustering network was 256, the cluster size K was 4, and the learning rate was set to 0.03; the batch size of the domain generalization network was set to 60, and the learning rate was set to 0.01; the entire model used stochastic gradient descent (SDG) as the optimizer, and the momentum value was set to 0.9.

[0172] Evaluation metrics: Liveness detection tasks need to take into account the classification errors of both attack samples and real samples. Therefore, this invention uses the Half Total Error Rate (HTER) and the Area Under the ROC Curve (AUC) as experimental evaluation metrics.

[0173] Experimental results:

[0174] (1) Comparison with common liveness detection models

[0175] The method of this invention is compared with several common non-domain generalized liveness detection methods: MS_LBP, BinaryCNN, Color Texture LBPTOP, and Auxiliary. The experimental results are shown in Table 1.

[0176] Table 1 Comparison of experimental results of our method and common liveness detection methods on four test tasks.

[0177]

[0178] As can be seen from the experimental results in Table 1, compared with these methods that are trained on only a single domain, the method in this invention has significant advantages in both evaluation metrics. This shows that our model can extract domain-invariant features related to live faces from multi-source domain data, thereby generalizing to unknown domains.

[0179] (2) Comparison with similar domain-generalized liveness detection methods

[0180] We selected several existing domain-generalized liveness detection methods: MMD-AAE, MADDG, and D. 2 A comparative experiment was conducted with AM, DRDG, ANRL, SSDG and the method in the example of this invention, and the results are shown in Table 2.

[0181] Table 2 compares the experimental results of our method with existing domain-generalized liveness detection methods on four test tasks.

[0182]

[0183] As can be seen, our model achieved near-best performance across all four test tasks. Therefore, this model can efficiently extract liveness domain-invariant features, has high generalization ability, and unlike most existing domain-generalized liveness detection models, this model did not use any pre-labeled domain labels during the training phase.

[0184] Furthermore, unlike existing methods that use the dataset as domain labels, our method generates pseudo-domain labels using self-supervised domain clustering. In test tasks ranging from O&C&I to M, our method outperforms other existing domain generalization models on both evaluation metrics, indicating that this module can perform a more detailed and accurate classification of the domains to which different face samples belong.

[0185] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a liveness detection method based on self-supervised domain clustering and domain generalization, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0186] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0187] This invention provides a concept and method for liveness detection based on self-supervised domain clustering and domain generalization. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A live body detection method based on self-supervised domain clustering and domain generalization, characterized in that, The pseudo domain label of the face sample is generated by a self-supervised domain clustering method, and a domain generalization method is used to train the sample data with the pseudo domain label to obtain a classification result of the living body detection, and the specific method comprises the following steps: Step 1, self-supervised domain clustering sample generation, that is, randomly selecting data belonging to different domains from the images of the mixed source domain data set as samples, performing domain information migration between two images based on the low frequency signals on the image frequency domain to generate positive and negative samples; Step 2, constructing a self-supervised domain clustering network and training the self-supervised domain clustering network, that is, training using a domain clustering method based on contrast learning, constructing a learning task, extracting domain related features from the positive and negative samples containing domain information obtained in step 1, and performing domain clustering according to the distribution of the samples in the feature space; Step 3, using a domain generalization living body detection model to train the sample with the pseudo domain label, that is, using the sample assigned with the pseudo domain label according to the clustering result as the source domain data set, using a domain generalization method based on adversarial training to extract the living body domain invariant features of the sample, and using the classification cross entropy loss, the adversarial loss and the memory triplet loss to optimize the domain generalization living body detection model; the final output of the trained domain generalization living body detection model is used as the classification result of the living body detection, and the living body detection based on self-supervised domain clustering and domain generalization is completed; The network structure of the domain generalization living body detection model in step 3 comprises: a feature generator, a feature classifier, a domain discriminator and a feature memory cache; The feature generator is used to obtain the feature representation of the sample, the generated feature is sent to the feature classifier, and the classification cross entropy loss is calculated; the feature representation is input into the domain discriminator to obtain the domain discrimination result, the feature generator and the domain discriminator are trained in the manner of adversarial training, the domain adversarial loss is calculated to reduce the domain bias contained in the feature; the generated feature and the feature in the memory cache are spliced to calculate the memory triplet loss, and the feature memory cache is updated according to the calculation result of the loss function; finally, the domain generalization living body detection model is optimized using the classification cross entropy loss, the adversarial loss and the memory triplet loss; The adversarial loss and the memory triplet loss in step 3 are calculated as follows: The network composed of the feature generator and the domain discriminator is optimized in the manner of adversarial training, and when the gradient of the adversarial loss is back propagated, the gradient between the domain discriminator and the feature generator is reversed, so that the training targets of the domain discriminator and the feature generator are opposite, and the whole adversarial training form is as follows: wherein X and Y D are the sample set and pseudo domain label set respectively, G is the feature generator, D is the domain discriminator, K is the number of source domains, is an indicator function, which takes the value 1 when i = y and 0 otherwise, y denotes the domain label of sample x; the adversarial loss is calculated using the binary cross-entropy function; The feature memory cache stores feature representations of previous rounds of iterations of the domain generalization liveness detection model; features in the memory cache are spliced with features generated in the current iteration, a memory triplet loss is calculated, and a specific loss function L MemTriplet is represented as follows: where F n is the feature set of the current iteration, M is the feature set in the memory cache, f a , f p , and f n are the anchor sample feature, positive sample feature, and negative sample feature, respectively, and a is the margin coefficient. An unbalanced feature memory cache update strategy is used, that is, the first h difficult sample features of the real human face and all the fake human face sample features in the memory triplet loss function calculation result are added to the feature memory cache in each training iteration; The optimization of the domain generalization living body detection model using the classification cross entropy loss, the adversarial loss and the memory triplet loss in step 3, that is, the comprehensive loss function of the domain generalization living body detection model is constructed as: L dg = L cls + λ1L ada + λ2L MemTriplet wherein L cls is the binary cross-entropy loss of the face liveness feature; λ1, λ2 are hyperparameters.

2. The method of claim 1, wherein the method is based on self-supervised domain clustering and domain generalization. The specific method of the domain information migration between two images based on the low frequency signals on the image frequency domain in step 1 comprises: Step 1-1, randomly sample two different human face images as source image and transfer target image from mixed source domain dataset, use Fourier transform to get the frequency domain signals of the two images, and decompose the amplitude component from the frequency domain signals and phase component The specific method is as follows: wherein x is an input source image or a migration target image, is a Fourier transform function, and srg( ) represents a phase value for a complex number; Step 1-2, filtering the amplitude component of the frequency domain signal of the source image using a Gaussian low-pass filter to obtain a low-frequency signal of the source image; filtering the amplitude component of the frequency domain signal of the migration target image using a Gaussian high-pass filter to obtain a high-frequency signal of the migration target image; Step 1-3, the low frequency signal of the source image and the high frequency signal of the migrated target image are fused in proportion, and the mixed frequency domain signal is transformed back to a two-dimensional image using inverse Fourier transform to obtain the image x after the domain information migration DT The specific method is as follows: wherein, is the amplitude component migrated through the domain information, x s ,x t is the source image and the migration target image, H L ,H H is a Gaussian low-pass filter and a Gaussian high-pass filter, is an inverse Fourier transform function, λ1, λ2 are hyperparameters, and j is an imaginary unit; Step 1-4, using a contrastive learning data augmentation method to assist in enhancing the source image and the domain information migration image to obtain positive and negative samples; wherein the positive sample is an image migrated by domain information, containing similar domain information as the source image; the negative sample is another target image from a different domain distribution.

3. The method of claim 2, wherein the method is based on self-supervised domain clustering and domain generalization. The self-supervised domain clustering network in step 2 is composed of a double-branch backbone network, a feature mapping head, a feature dictionary, and a feature clusterer. Wherein, the double-branch backbone network adopts a ResNet-18 network model, two branches are query encoding network and momentum encoding network respectively, and the positive samples obtained in step 1 are respectively sent into the two branches to extract domain-related features; the domain-related features extracted are mapped using a feature mapping head to obtain comparison features q and k + ; the two branches are initialized using the same weight parameters and are updated using different methods, wherein the weight parameters of the query encoding network are updated using gradient back propagation during training; the momentum encoding network updates the weight parameters using a momentum update method, and the momentum update method of the weight parameters is: w k = mw k + (1 - m)w q where w k is the weight of the query encoding network, m is a momentum coefficient, which is in the range of 0 to 1, and w q is the weight of the momentum encoding network. The feature mapping head adopts the structure of a multi-layer perceptron to perform feature mapping on the features of the two similar samples extracted by the double-branch encoder. The feature dictionary adopts the structure of a queue to store the feature representations generated by the momentum network branch in the previous batch as negative sample features for comparison with the current batch of sample features. The feature clusterer adopts a K-means clustering algorithm to cluster the domain-related features extracted by the momentum encoder to obtain the cluster center of each cluster. The self-supervised domain clustering network is optimized using a sample contrastive learning loss function and a clustering contrastive learning loss function.

4. The method of claim 3, wherein the method is characterized by, Training the self-supervised domain clustering network in step 2 includes the following steps: Step 2-1, calculating the cluster to which the sample belongs and the cluster center; Step 2-2, calculating the similarity between the contrastive features; Step 2-3, constructing a comprehensive loss function for the self-supervised domain clustering network, and training the self-supervised domain clustering network by calculating the comprehensive loss function.

5. The method of claim 4, wherein, The calculation of the cluster to which the sample belongs and the cluster center in step 2-1 specifically includes: Before each round of training, all training samples X = {x1, x2, …, x N} are input into the momentum encoding branch to obtain the domain-related feature representation Z = {z1, z2, …, z N} of each sample, where N represents the number of samples; then the K-means clustering method is used to cluster the domain-related feature representation Z of each sample to obtain the cluster to which each sample belongs and the K cluster centers {c1, c2, …, c K}.

6. The method of claim 5, wherein the method is based on self-supervised domain clustering and domain generalization. The calculation of the similarity between the contrastive features in step 2-2 specifically includes: Calculate the features q,k of two positive samples + The similarity between sim(q,k) + The feature dictionary stores the feature representations generated by the momentum network branches of previous batches, and the feature k stored in the feature dictionary is used to store the feature representations of previous batches. i As a negative sample feature, calculate its similarity sim(q,k) with the query sample feature q. i Similarity is calculated using the inner product of the two features.

7. The method of claim 6, wherein the method is based on self-supervised domain clustering and domain generalization. The construction of the comprehensive loss function for the self-supervised domain clustering network in step 2-3 specifically includes: Constructing the sample contrastive loss L s With the clustering contrastive loss L c The specific method is as follows: where r is the number of negative samples; τ and is the temperature coefficient, which controls the shape of the characteristic distribution; The comprehensive loss function for the self-supervised domain clustering network is constructed as follows: L CL = L s + λL c Where λ is a balance coefficient; The self-supervised clustering network is trained by calculating the comprehensive loss function, i.e., the results of self-supervised domain clustering are assigned as pseudo-domain labels to the samples belonging to them, specifically: the clustering results are divided into K parts according to the cluster, each part represents a domain; starting from 1, each domain is assigned a domain number as a pseudo-domain label; the corresponding pseudo-domain label is assigned to each sample in the cluster.

Images