A method and system for face liveness detection with increased style diversity

Abstract

The present invention discloses a method and system for face liveness detection that increases style diversity, comprising the following steps: extracting a face image and setting a true / false label and a domain label; extracting content features and style features of the face image; inputting the content features into a domain discriminator with a gradient reversal layer and calculating the domain discriminator loss; performing a class-guided style feature blending operation to obtain a mixed style feature; constructing self-assembly features and recombination features; constraining the self-assembly features and recombination features using an agent-based contrastive learning loss; calculating a probability value of a face image being genuine using a classifier, supervising the classifier output based on the true / false label to obtain a classification loss; constructing a total loss function; and training a prediction model based on the total loss function to obtain a predicted probability of the face image being genuine. The present invention increases style diversity, can access a large number of styles from unknown domains, improves representation capabilities, and has good generalization performance and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of human face liveness detection, and in particular to a human face liveness detection method and system that increases style diversity. Background Art

[0002] Currently, facial recognition systems are widely used in our daily lives, including mobile payments, clocking in at work, storing and retrieving packages, and access control. However, with the increasing popularity of social networks, people can easily obtain facial images posted on social networks without the consent of the person and use them for forgery attacks. In addition, current facial forgery and spoofing attack technologies are constantly maturing, and new attack methods are constantly emerging, posing a significant security threat to facial recognition systems. Therefore, the research on facial liveness detection algorithms is of great practical significance.

[0003] Current face liveness detection algorithms can be divided into two categories: algorithms based on traditional handcrafted features and algorithms based on deep neural networks. Traditional handcrafted feature-based algorithms use manually designed descriptors to extract handcrafted features to distinguish real faces from fake ones, such as local binary patterns (LBP) and difference of Gaussians (DoG). These algorithms are highly interpretable and require low computational resources, but they require manually designed feature descriptors. Due to their over-reliance on feature selection and function design, traditional methods based on handcrafted features often perform poorly when faced with test set data. In comparison, algorithms based on deep neural networks can learn adaptive features and have stronger feature representation capabilities. Therefore, they are widely used in liveness detection tasks and are gradually becoming mainstream.

[0004] However, current mainstream liveness detection algorithms still have significant shortcomings. While most algorithms achieve high accuracy in on-database testing, their performance often degrades significantly when encountering test data from an unknown domain. This is because different liveness detection databases vary in various aspects, such as the capture device, presentation device, attack medium, lighting conditions, and shooting environment. In other words, the data distribution between the training and test domains differs significantly. As a result, models trained on the training set often experience performance degradation when faced with test data from a significantly different data distribution. Summary of the Invention

[0005] In order to overcome the defects and shortcomings of the existing technology, the present invention provides a face liveness detection method and system with increased style diversity. The present invention utilizes a class-guided style feature mixing strategy to greatly increase style diversity by controlling the randomness of the mixing coefficient, thereby improving the representation ability of the model. The present invention utilizes an agent-based contrastive learning module to perform asymmetric optimization of real face features and fake face features, making the real face features more compact in the feature space and providing better class boundaries for unknown domains. In addition, the agent-sample relationship is introduced in contrastive learning to solve the positive alignment problem caused by positive sample pairs that are more difficult to optimize, making the model more robust to noisy samples or abnormal samples.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a method for detecting live faces with increased style diversity, comprising the following steps:

[0008] Divide the dataset video into frames, extract the face area of ​​each frame image to obtain the face image, and set the corresponding authenticity label and domain label for the face image;

[0009] Extract content features and style features of facial images;

[0010] The content features are input into the domain discriminator with a gradient reversal layer. The domain labels are used to supervise the output features of the domain discriminator. The adversarial learning is performed with the gradient reversal layer to obtain the domain discriminator loss.

[0011] Perform a class-guided style feature mixing operation on the feature statistical information of the face image instance to obtain a mixed style feature;

[0012] The content features and style features are assembled through the adaptive style conversion method to obtain self-assembled features, and the content features and mixed style features are assembled through the adaptive style conversion method to obtain recombined features;

[0013] Calculate the correlation coefficient between the self-assembly features and the recombinant features, divide the facial image dataset into categories, construct positive and negative pairs based on the category information, obtain the correlation coefficient vector based on the correlation coefficient of the positive and negative pairs, set the correlation coefficient between the positive pairs to zero, calculate the score of the self-assembly features combined with the class proxy weights of the corresponding target category, concatenate the correlation coefficient vector with the score, and optimize it using the agent-based contrastive learning loss to reduce the correlation coefficient between the negative pairs and increase the score of the self-assembly features combined with the class proxy weights of the corresponding target category;

[0014] The self-assembled features are used to obtain the probability value of the face image being real through the classifier, and the output of the classifier is supervised based on the true and false labels to obtain the classification loss;

[0015] The total loss function is obtained by weighted summing the domain discriminator loss, classification loss, and contrastive learning loss;

[0016] The prediction model is obtained based on the total loss function training;

[0017] The face image test set is input into the trained prediction model to obtain the predicted probability that the face image is real.

[0018] As a preferred technical solution, the content features are input into a domain discriminator with a gradient reversal layer, and the output features of the domain discriminator are supervised by using domain labels, specifically including:

[0019] The content features are passed through the dimensionality reduction network to obtain the reduced content features. The reduced content features are passed through the domain discriminator to obtain the output features. The domain labels are used to supervise the output features of the domain discriminator to determine the source domain of the input content features. The gradient reversal layer is used to control the gradient of the back propagation, reverse the gradient, and multiply the gradient in the back propagation process by a dynamically changing coefficient λ. Specifically expressed as:

[0020]

[0021] Among them, i c Indicates the current number of iterations, i s Indicates the total number of iterations.

[0022] As a preferred technical solution, the domain discriminator loss function is expressed as:

[0023]

[0024] Among them, x represents the input image, y represents the domain label of the input image, X represents the set of input images, and Y D Represents the set of domain labels, M represents the number of different source domains, G and D represent the content feature generator and domain discriminator respectively, E represents the mathematical expectation, and the domain discriminator loss of all images in the set is averaged as the domain discriminator loss of the entire set. is an indicator function that is used to divide the input images into different sets according to the domain labels.

[0025] As a preferred technical solution, the feature statistical information of the face image instance is subjected to a class-guided style feature mixing operation, specifically including:

[0026] When the style feature mixing operation is performed between face image samples of the same category, the content feature F of the face image sample F is obtained. c and style characteristics F s , calculate the content feature F c Instance-normalized content features Style Features Fs Obtain affine parameters γ and β through a single fully connected layer, randomly select another face image sample F1, face image sample F1 and face image sample F belong to the same category, and obtain its style feature F 1s The corresponding affine parameters γ1 and β1, the style feature mixing operation is specifically expressed as:

[0027] γ mix =λγ1+(1-λ)γ

[0028] β mix =λβ1+(1-λ)β

[0029]

[0030] Where λ represents the mixing coefficient randomly sampled from the uniform distribution U(0, 1), γ mix , β mix Represents the affine parameters corresponding to the mixed style features;

[0031] When the style feature mixing operation is performed between face image samples of different categories, the face image sample F is obtained and the content feature F is calculated. c Instance-normalized content features Get style feature F s Corresponding affine parameters γ and β, randomly select another face image sample F1, and obtain its style feature F 1s Corresponding affine parameters γ1 and β1, randomly select another face image sample F2, and obtain its style feature F 2s Corresponding affine parameters γ2 and β2, face image sample F and face image sample F1 belong to different categories, face image sample F1 and face image sample F2 belong to the same category, and the style feature mixing operation is specifically expressed as:

[0032] γ mix =λγ1+(1-λ)γ2

[0033] β mix =λβ1+(1-λ)β2

[0034]

[0035] Where λ represents the mixing coefficient randomly sampled from the uniform distribution U(0, 1), γ mix , β mix Represents the affine parameters corresponding to the mixed style features.

[0036] As a preferred technical solution, the content features and style features are assembled by an adaptive style conversion method to obtain self-assembled features, which are specifically expressed as:

[0037]

[0038] Among them, F ii Indicates the self-assembly characteristics, F ci Represents a face image sample F i content characteristics, Represents a face image sample F i Channel mean of content features, Represents a face image sample F i Channel standard deviation of content features, γ i , β i Represents a face image sample F i Affine parameters corresponding to style features.

[0039] and / or;

[0040] The content features and the mixed style features are assembled by the adaptive style conversion method to obtain the recombined features, which are specifically expressed as:

[0041]

[0042] Among them, F ij Indicates the recombination characteristics, F ci Represents a face image sample F i content characteristics, Represents a face image sample F i Channel mean of content features, Represents a face image sample F i Channel standard deviation of content features, γ mix , β mix Represents the affine parameters corresponding to the mixed style features.

[0043] As a preferred technical solution, the correlation coefficient vector and the score are concatenated and then optimized through the agent-based contrastive learning loss, which is specifically expressed as:

[0044] Select one of the face image samples as the anchor sample, perform dimensionality reduction on its self-assembly features and recombinant features, and map the reduced dimensionality self-assembly features and recombinant features through a mapping head to obtain the mapped self-assembly features and recombinant features;

[0045] Calculate the autocorrelation coefficient of the mapped self-assembly feature, calculate the mutual correlation coefficient of the mapped self-assembly feature and the recombinant feature, and concatenate the autocorrelation coefficient and the mutual correlation coefficient along the Y dimension to obtain the correlation coefficient vector R corresponding to the anchor sample. i ;

[0046] According to the category information, the positive and negative pair relationship is constructed. The correlation coefficient corresponding to the negative pair remains unchanged, and the correlation coefficient corresponding to the positive pair is set to zero to obtain the correlation coefficient vector R corresponding to the anchor sample. i′ ;

[0047] The proxy weights of multiple categories are concatenated to obtain the proxy weight W, and the mapped self-assembly features are jointly calculated with the proxy weight W to obtain the score vector S i , the score s of the target category corresponding to the self-assembly feature after screening and mapping i ;

[0048] The correlation coefficient vector R i′ With score s i Splicing to get vector U i , using agent-based contrastive learning loss to train the vector U i To constrain.

[0049] As a preferred technical solution, the agent-based contrastive learning loss is specifically expressed as:

[0050]

[0051] Among them, L PCL represents the agent-based contrastive learning loss, F ii′ , represents the self-assembly characteristics after mapping, w c Represents the self-assembly feature F after mapping ii′ , the corresponding target class proxy weight, K represents the face image sample F i The number of negative pairs in the batch when used as anchor samples, α represents the scaling factor.

[0052] As a preferred technical solution, the classification loss is specifically expressed as:

[0053] L cls =-(ylog(y′)+(1-y)log(1-y′))

[0054] Among them, y represents the face image sample label, and y′ represents the predicted label.

[0055] The present invention also provides a face liveness detection system for increasing style diversity, comprising: a data preprocessing module, a content feature extractor, a style feature extractor, a domain discriminator with a gradient reversal layer, a domain discriminator loss calculation module, a class-guided style feature mixing module, a feature self-assembly module, a feature recombination module, an agent-based contrastive learning module, a classifier, a classification loss calculation module, a total loss function construction module, a model training module, and a prediction module;

[0056] The data preprocessing module is used to divide the dataset video into frames, extract the face area of ​​each frame image to obtain a face image, and set corresponding authenticity labels and domain labels for the face image;

[0057] The content feature extractor is used to extract content features of the face image;

[0058] The style feature extractor is used to extract style features of a face image;

[0059] The domain discriminator with the gradient reversal layer is used to process content features of multiple domains;

[0060] The domain discriminator loss calculation module is used to supervise the output features of the domain discriminator using the domain label, and cooperate with the gradient reversal layer to perform adversarial learning to obtain the domain discriminator loss;

[0061] The class-guided style feature mixing module is used to perform a class-guided style feature mixing operation on the feature statistical information of the face image instance to obtain a mixed style feature;

[0062] The feature self-assembly module is used to assemble content features and style features through an adaptive style conversion method to obtain self-assembled features;

[0063] The feature recombination module is used to assemble content features and mixed style features through an adaptive style conversion method to obtain recombined features;

[0064] The agent-based contrastive learning module is used to perform contrastive learning, classify the facial image dataset into categories, construct positive and negative pair relationships based on category information, obtain a correlation coefficient vector based on the correlation coefficients of the positive and negative pairs, set the correlation coefficients between the positive pairs to zero, calculate the score of the self-assembled features combined with the class proxy weights of the corresponding target category, and then optimize the concatenation of the correlation coefficient vector and the score using the agent-based contrastive learning loss to reduce the correlation coefficients between the negative pairs and increase the score of the self-assembled features combined with the class proxy weights of the corresponding target category.

[0065] The classifier is used to take the self-assembled features as input and calculate the probability value of the face image being real;

[0066] The classification loss calculation module is used to supervise the output of the classifier based on the true and false labels to obtain the classification loss;

[0067] The total loss function construction module is used to obtain a total loss function by weighted summing of the domain discriminator loss, the classification loss, and the contrastive learning loss;

[0068] The model training module is used to obtain a prediction model based on total loss function training;

[0069] The prediction module is used to input the face image test set into the trained prediction model to obtain the predicted probability that the face image is real.

[0070] As a preferred technical solution, the network structure of the content feature extractor includes a first convolutional layer, a first maximum pooling layer, multiple groups of residual blocks, a second convolutional layer and a BN layer;

[0071] The facial image is subjected to a first convolutional layer and a first maximum pooling layer to obtain initial features, the initial features are subjected to multiple groups of residual blocks to obtain feature maps of different sizes, and the feature maps are subjected to a second convolutional layer and a BN layer to obtain content features of the facial image;

[0072] The network structure of the style feature extractor includes a third convolutional layer, a second maximum pooling layer, multiple groups of residual blocks, a feature pyramid structure, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first IN layer, a second IN layer, a third IN layer, a third maximum pooling layer and a fully connected layer;

[0073] The face image is passed through the third convolution layer and the second maximum pooling layer to obtain initial features, and the initial features are passed through multiple groups of residual blocks to obtain feature maps F of different sizes. Block_1 , feature map F Block_2 and feature map F Block_3 , feature map F Block_1 , feature map F Block_2 and feature map F Block_3 After the multi-scale features are aggregated through the feature pyramid structure, the feature map F Block_1 The output feature F is obtained through the fourth convolution layer and the first IN layer out_1 , output feature F out_1 With the feature map F Block_2 The concatenation is performed in the form of addition, and after concatenation, the output feature F is obtained through the fifth convolutional layer and the second IN layer. out_2 , output feature F out_2 With the feature map F Block_3 Splice in the form of addition to obtain the intermediate feature F m , intermediate feature F m The style features of the face image are obtained through the sixth convolutional layer, the third IN layer, the third maximum pooling layer and the fully connected layer.

[0074] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0075] (1) The class-guided style feature mixing strategy proposed in this invention increases style diversity by controlling the randomness of the mixing coefficient to improve the representation ability of the model. Class guidance is to avoid the problem of label definition while simulating the style of unknown domains. When two samples from different source domains are randomly selected, the style features between the two domains can be simulated through this mixing strategy. When two samples from the same source domain are randomly selected, since the samples in the same data set also have differences in shooting equipment, presentation equipment, shooting environment, lighting conditions, forgery methods, etc., this mixing strategy also helps to mine implicit subdomains in the source domain. This allows the model to be exposed to a large number of styles of unknown domains during the training phase, thereby improving the representation ability of the model. In addition, this strategy only needs to operate at the feature statistical information level of the two instances, which is very simple to implement and almost does not increase the computational cost.

[0076] (2) The agent-based contrastive learning proposed in the present invention performs asymmetric optimization on real faces and fake faces, regards the real face images of the three training sets as the same category, and regards the fake face images of the three training sets as one category respectively, and performs asymmetric optimization on the real face features and fake face features, so that the real face features are more compact in the feature space, providing better class boundaries for the unknown domain. In addition, the agent-based contrastive learning considers both the sample-sample relationship and the agent-sample relationship, combining the advantages of traditional contrastive learning and agent-based metric learning. On the one hand, it utilizes the rich data-to-data relationship, and on the other hand, it solves the positive alignment problem caused by the difficult positive sample pairs in traditional contrastive learning, so that the network can converge quickly and safely, and is more robust to noisy samples or abnormal samples.

[0077] (3) The network proposed in the present invention has good generalization performance and is a lightweight network. It effectively solves the problem that neural networks occupy high resources and are not suitable for deployment on mobile terminals. While ensuring performance, it reduces the requirements for memory and computing resources. It is suitable for deployment on mobile phones, embedded terminals, etc. and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A schematic diagram of the flow chart of the face liveness detection method for increasing style diversity of the present invention;

[0079] Figure 2 Schematic diagram of the operation of the class-guided style feature mixing module of the present invention in which two samples belong to the same category;

[0080] Figure 3 Schematic diagram of the operation of the class-guided style feature mixing module of the present invention in which two samples belong to different categories;

[0081] Figure 4 A schematic diagram of the architecture of a face liveness detection system for increasing style diversity in the present invention;

[0082] Figure 5 Schematic diagram of the network structure of the content feature extractor of the present invention;

[0083] Figure 6 Schematic diagram of the network structure of the style feature extractor of the present invention. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0085] Example 1

[0086] This embodiment uses the Replay-Attack, CASIA-MFSD, MSU_MFSD and Oulu_NPU liveness detection datasets for training and testing as examples to introduce its implementation process in detail. Among them, the Replay-Attack dataset contains 1200 videos, using a MacBook camera with a resolution of 320×240 pixels to collect real faces from 50 testers and deceptive faces generated based on them, and is divided into training set, verification set and test set according to 3:3:4; the CASIA-MFSD dataset contains 600 videos, using three cameras with resolutions of 640×480 pixels, 480×640 pixels and 1920×1080 pixels to collect real faces from 50 testers and deceptive faces generated based on them. The generated deceptive faces are divided into training set and test set according to the ratio of 2:3; the MSU_MFSD dataset includes 280 videos, which collect real faces from 35 testers and deceptive faces generated based on them, of which 15 people are used for training set and 20 people are used for test set; the Oulu_NPU dataset collects real faces from 55 testers and deceptive faces generated based on them, with 90 videos for each person, of which 20 people are used for training set, 15 people are used for verification set, and 20 people are used for test set. The attack methods include two printing attacks and two video replay attacks.

[0087] This example runs on a Linux system and is implemented based on the deep learning framework Pytorch. The graphics card used in the experiment is GTX1080Ti, the CUDA version is 10.1.105, and the cudnn version is 7.6.4.

[0088] like Figure 1 As shown, this embodiment provides a method for face liveness detection that increases style diversity, including the following steps:

[0089] S1: Divide the dataset video into frames, extract the face area of ​​each frame image to obtain the face image, and set the corresponding authenticity label and domain label for the face image;

[0090] In this embodiment, the OpenCV tool library is used to frame all videos in the training set and the test set, and the MTCNN face recognition algorithm is used to detect the face area to obtain the face position frame. The face area is then cropped and the Lanczos interpolation algorithm is used to adjust the resolution to H×W×C, where H, W, and C represent the height, width, and number of channels of the face image, respectively. In this embodiment, H=256, W=256, and C=3;

[0091] S2: Input the face image into the content feature extractor to obtain content features, and input the face image into the style feature extractor to obtain style features;

[0092] S3: Construct a domain discriminator with a gradient reversal layer, input content features into the domain discriminator with a gradient reversal layer, use domain labels to supervise the output features of the domain discriminator, and perform adversarial learning with the gradient reversal layer to obtain the domain discriminator loss. Through adversarial learning, the content features of multiple domains are made indistinguishable.

[0093] In this embodiment, the domain discriminator with a gradient reversal layer makes the content features of multiple domains indistinguishable through adversarial learning. It is mainly constructed based on the domain discriminator and the gradient reversal layer, where the domain discriminator includes two fully connected layers, a ReLU activation function layer, and a Dropout layer.

[0094] The specific process is as follows: the input size is The content features of the sample are first obtained through a simple dimensionality reduction network to obtain the reduced content features with a size of 4N. The dimensionality reduction network mainly includes an adaptive average pooling layer and a Reshape layer. The reduced content features are then passed through the domain discriminator to obtain output features with a size of M. The domain discriminator mainly includes a fully connected layer with an input dimension of 4N and an output dimension of 4N, a ReLU activation function layer, a Dropout layer, and a fully connected layer with an input dimension of 4N and an output dimension of M. The domain label of the sample is then used to supervise the output features of the domain discriminator to determine which source domain the input content features come from. The gradient reversal layer is used to control the gradient of back propagation. On the one hand, the gradient is reversed to cooperate with the domain discriminator to achieve the effect of adversarial learning, making the content features of multiple domains indistinguishable. On the other hand, the gradient in the back propagation process is multiplied by a dynamically changing coefficient λ to suppress the influence of noise signals in the early stage of training. The λ formula is as follows:

[0095]

[0096] Among them, i c Indicates the current iteration number of the neural network, i s Represents the total number of iterations of the neural network;

[0097] This process involves the domain discriminator loss function, which is formulated as follows:

[0098]

[0099] Among them, x represents the input image, y represents the domain label of the input image, X represents the set of input images, and Y D Represents the set of domain labels, M represents the number of different source domains, G and D represent the content feature generator and domain discriminator respectively, E represents the mathematical expectation, and the domain discriminator loss of all images in the set is averaged as the domain discriminator loss of the entire set. is an indicator function used to divide the input image into different sets according to the domain label. In this embodiment, the input image is divided into four subsets according to the domain label.

[0100] S4: Perform a class-guided style feature blending operation on the feature statistics of the face image instance to obtain a mixed style feature to simulate the style of the unknown domain, thereby improving the model's representation ability;

[0101] In this embodiment, the feature statistical information of two face image instances is mixed to obtain a mixed style feature to simulate the style of the unknown domain, thereby improving the representation ability of the model. The method mainly includes two parts: class guidance and mixing strategy. The former is to solve the problem of sample label definition. The liveness detection task is different from the image generation task. Each sample has a corresponding category label. If a style mixing strategy without class guidance is adopted, when the mixing strategy is performed between samples of two different categories, it will make the category label of the mixed feature difficult to define, resulting in the style mixing strategy being unable to be applied to the liveness detection task. The latter increases style diversity by controlling the randomness of the mixing coefficient. When samples from two different source domains are randomly selected, the mixing strategy can simulate the style features between the two domains. When two samples from the same source domain are randomly selected, since samples in the same dataset may also have differences in shooting equipment, presentation equipment, shooting environment, lighting conditions, forgery methods, etc., the mixing strategy also helps to mine implicit subdomains in the source domain. The above operations enable the model to be exposed to a large number of styles of unknown domains during the training phase, thereby improving the representation ability of the model.

[0102] There are two specific cases. In case 1, the style feature mixing operation is performed between samples of the same category, such as Figure 2 As shown, the input is the content feature F of sample F c and style characteristics F s , content feature F c Size Style Features F s The size is 4N, the content feature F c Compute instance normalized content features The sizes are Style Features F s The affine parameters γ and β are obtained through a single fully connected layer, with sizes of 4N and 4N respectively. Another sample F1 is randomly selected in the Batch, and its style feature F 1s The corresponding affine parameters are denoted as γ1 and β1. If F and F1 belong to the same category, γ and γ1 are assigned random weights, and the sum of the two weights is 1. The weighted sum of γ and γ1 is obtained to obtain a γ between the two. mix , assign random weights to β and β1, the sum of the two weights is 1, and the weighted sum of β and β1 is obtained to obtain β between the two mix , using γ mix and β mix Normalize content features for instances Style transfer can be achieved by replacing feature statistics.

[0103] In this embodiment, the formula corresponding to the style feature mixing operation is as follows:

[0104] γ mix =λγ1+(1-λ)γ

[0105] β mix =λβ1+(1-λ)β

[0106]

[0107] Where λ represents the mixing coefficient randomly sampled from the uniform distribution U(0, 1), γ and β represent the affine parameters corresponding to the style features of sample F, γ1 and β1 represent the affine parameters corresponding to the style features of sample F1, and γ mix , β mix Represents the affine parameters corresponding to the mixed style features;

[0108] In case 2, the style feature mixing operation is performed between samples of different categories, such as Figure 3 As shown, the specific process is as follows: the content feature F of sample F c Instance-normalized content features Size Style Features F s The corresponding affine parameters γ and β have sizes of 4N and 4N respectively. Randomly select another sample F1 in the batch, and its style feature F 1sThe corresponding affine parameters are denoted as γ1 and β1. If F and F1 belong to different categories, another sample F2 is introduced into the Batch, and F2 and F1 belong to the same category, the style feature F 2s The corresponding affine parameters are denoted as γ2 and β2. Random weights are assigned to γ1 and γ2, and the sum of the two weights is 1. The weighted sum of γ1 and γ2 is obtained to obtain a γ between the two. mix , assign random weights to β1 and β2, the sum of the two weights is 1, and the weighted sum of β1 and β2 is obtained to obtain a β between the two. mix , using γ mix and β mix Normalize content features for instances Style transfer can be achieved by replacing feature statistics.

[0109] In this embodiment, the formula corresponding to the style feature mixing operation is as follows:

[0110] γ mix =λγ1+(1-λ)γ2

[0111] β mix =λβ1+(1-λ)β2

[0112]

[0113] Where λ represents the mixing coefficient randomly sampled from the uniform distribution U(0, 1), γ1 and β1 represent the affine parameters corresponding to the style features of sample F1, γ2 and β2 represent the affine parameters corresponding to the style features of sample F2, and γ mix , β mix Represents the affine parameters corresponding to the mixed style features;

[0114] S5: Assemble the content features and style features through the adaptive style conversion method to obtain self-assembled features, and assemble the content features and mixed style features through the adaptive style conversion method to obtain recombined features;

[0115] In this embodiment, the adaptive style transfer method AdaIN is mainly used for implementation. The style information corresponds to the instance-level feature statistics. AdaIN can achieve style transfer by replacing the instance-level feature statistics while preserving the content information. This process does not require explicit image synthesis, but only requires replacement operations at the instance-level feature statistics level. It is very simple to implement and hardly increases the computational cost.

[0116] The specific process is as follows: for the sample F in the Batch i (i∈{1, 2, ..., N}), whose content feature is denoted as F ci , the style feature is recorded as F si, the content feature channel mean is recorded as The standard deviation of the content feature channel is denoted as Style Features F si The corresponding affine parameter is denoted as γ i and β i . Set the content feature F ci and style characteristics F si Assembled by the adaptive style conversion method AdaIN to obtain sample F i Self-assembly characteristics F ii , size is F i As anchor samples, take other samples F in the Batch j (j∈{1, 2, ..., N}, j≠i) performs a class-guided style feature mixing operation to obtain the mixed style feature F mix The corresponding affine parameter γ mix , β mix , in terms of specific implementation, according to sample F i and sample F j There are two cases of whether they belong to the same category. For details, please refer to step S4. ci and mixed style features F mix The recombinant feature F is assembled by the adaptive style conversion method AdaIN ij , size is

[0117] In order to reduce the amount of calculation in subsequent contrastive learning, the self-assembly feature F ii and recombination characteristics F ij Through a simple dimensionality reduction network, the specific implementation process is as follows: ii Through a convolution layer with a convolution kernel size of 3×3, and then through an adaptive average pooling layer to obtain the self-assembly feature F after dimensionality reduction ii′ , with a size of 8N, will reorganize the feature F ij Through a convolution layer with a convolution kernel size of 3×3, and then through an adaptive average pooling layer to obtain the reconstructed feature F after dimensionality reduction ij′ , the size is 8N. In order to facilitate the subsequent comparative learning, it is necessary to splice the self-assembly features and the recombinant features. The specific implementation process is as follows: i Self-assembly characteristics F ii′ and recombination feature F ij′ Splice along the Y dimension to get a 1×N feature vector V i , here we call it sample F i The corresponding eigenvector;

[0118] The self-assembly process formula is as follows:

[0119]

[0120] Among them, F ci Represents sample F i content characteristics, Represents sample F i Channel mean of content features, Represents sample F i Channel standard deviation of content features, γ i , β i Represents sample F i Affine parameters corresponding to style features;

[0121] The formula for the recombination process is as follows:

[0122]

[0123] Among them, F ci Represents sample F i content characteristics, Represents sample F i Channel mean of content features, Sample F i Channel standard deviation of content features, γ mix , β mix Represents the affine parameters corresponding to the mixed style features;

[0124] S6: Calculate the correlation coefficient between the self-assembly features and the recombinant features, divide the face image dataset into four categories: Real, Fakel, Fake2, and Fake3, construct the positive and negative pair relationship based on the category information, obtain the correlation coefficient vector based on the correlation coefficient of the positive and negative pairs, set the correlation coefficient between the positive pairs to zero, calculate the score of the self-assembly features combined with the class proxy weight of the corresponding target category, concatenate the correlation coefficient vector with the score, and optimize it using the agent-based contrastive learning loss to reduce the correlation coefficient between the negative pairs and increase the score of the self-assembly features combined with the class proxy weight of the corresponding target category;

[0125] In this embodiment, we propose agent-based contrastive learning to perform asymmetric optimization on real and fake face features, making real face features more compact in the feature space and providing better class boundaries for unknown domains. We also consider both sample-sample relationships and agent-sample relationships, solving the positive alignment problem caused by difficult positive sample pairs. This allows the network to converge quickly and securely, and is more robust to noisy or abnormal samples.

[0126] In this embodiment, the images of the three training sets are divided into four categories. All real faces are divided into one category, denoted as Real. The fake faces in each training set are separately regarded as a category, denoted as Fake1, Fake2, and Fake3 respectively. The real face features and fake face features are asymmetrically optimized to make the real face features more compact in the feature space, providing better class boundaries for the unknown domain. In addition, traditional contrastive learning only considers the sample-sample relationship, and there are positive sample pairs that are more difficult to optimize. Directly aligning these difficult positive sample pairs will hinder the training of the neural network, which is called the positive alignment problem. Agent-based contrastive learning considers both the sample-sample relationship and the agent-sample relationship, increases the distance between negative sample pairs in the feature space, and decreases the distance between the anchor sample and the corresponding target class agent. It combines the advantages of traditional contrastive learning and agent-based metric learning. On the one hand, it utilizes the rich data-to-data relationship, and on the other hand, it solves the positive alignment problem caused by the difficult positive sample pairs in traditional contrastive learning, so that the network can converge quickly and safely, and makes the network more robust to noisy samples or abnormal samples.

[0127] The specific process is as follows: Select the sample F in the Batch i As the anchor sample, its corresponding feature vector V i Contains N features, self-assembled features F related to the anchor sample ii′ (i∈{1, 2, ..., N}) and recombinant features F ij′ (j∈{1, 2, ..., N}, j≠i), first, F ii′ and F ij′ Mapping is performed through the mapping head to obtain the mapped self-assembly feature F ii′ , and recombination feature F ij″ , then calculate the correlation coefficient of the mapped features and calculate F ii′ , the autocorrelation coefficient R ii , calculate F ii′ , and F ij″ The mutual correlation coefficient R ij , R ii and R ij After splicing along the Y dimension, we get sample F i The corresponding correlation coefficient vector R i , scale is 1×N, F i When used as an anchor sample, it corresponds to a total of N sample pairs. Then, the positive and negative pair relationships are constructed based on the category information. If the two samples in a sample pair belong to different categories, they are considered negative pairs. If the two samples in a sample pair belong to the same category, they are considered positive pairs. For negative pairs, their corresponding correlation coefficients remain unchanged. For positive pairs, their corresponding correlation coefficients are set to zero. After completing the above operations, the sample F iThe corresponding correlation coefficient vector is denoted as R i′ , the scale is 1×N;

[0128] In this embodiment, the mapping head is a small network used to map feature embeddings to a new feature space, which is widely used in contrastive learning tasks. In this embodiment, a three-layer fully connected layer is used in conjunction with a ReLU activation function layer to form the mapping head.

[0129] The learnable proxy weight W is introduced, which is composed of w1, w2, w3, and w4. w1, w2, w3, and w4 represent the class proxy weights of the four categories of Real, Fake1, Fake2, and Fake3 respectively. The reason for introducing the class proxy weight is to measure the distance between the anchor sample features and the target class proxy, thereby introducing the proxy-sample relationship to solve the positive alignment problem of traditional contrastive learning. ii′ , after joint calculation with the proxy weight W, a score vector S with a scale of 4×1 is obtained i , use the score to measure the distance between the anchor sample features and the four class agents, and filter F ii′ , the score s corresponding to the target category i , the scale is 1×1, and s i and R i′ Splice and get a 1×(N+1) vector U i , using the agent-based contrastive learning loss L PCL U i To constrain, on the one hand, increase F ii′ , the score s corresponding to the target category i , thereby shortening the distance between the anchor sample and the corresponding target class proxy. On the other hand, it reduces the correlation coefficient between negative pairs within the batch, thereby increasing the distance between negative sample pairs in the feature space. Through this operation, the neural network learns both the relationship between samples and the relationship between samples and class proxies, solving the positive alignment problem caused by difficult positive sample pairs in traditional contrastive learning.

[0130] The process involves an agent-based contrastive learning loss function, which is formulated as follows:

[0131]

[0132] Where N represents the number of samples in the batch, F ii′ , represents the self-assembly characteristics after mapping, w c Represents the self-assembly feature F after mapping ii′ , the corresponding target class proxy weight, K represents the sample F i As the number of negative pairs in the batch when it is an anchor sample, α represents the scaling factor. It is worth noting that the self-assembly feature F ii′,,Recombination feature F ij″ and class proxy weight w c All need to be normalized;

[0133] S7: Construct a classifier and pass the self-assembly features through the classifier to output the probability value of the sample being true;

[0134] In this embodiment, the classifier is used to output the probability value of the sample being true, including two fully connected layers, a ReLU activation function layer, and a Dropout layer;

[0135] The specific process is as follows: input the self-assembly feature F with a size of 8N ii′ , the self-assembly feature F ii′ The classifier obtains an output feature of size 2. In terms of implementation, the classifier includes a fully connected layer with an input dimension of 8N and an output dimension of 4N, a ReLU activation function layer, a Dropout layer, and a fully connected layer with an input dimension of 4N and an output dimension of 2. Calculations are performed based on the output features to obtain predicted labels and predicted probability values. Then, the true and false labels of the samples are used for supervision to obtain the classification loss function.

[0136] This process involves a classification loss function, which is formulated as follows:

[0137] L cls =-(ylog(y′)+(1-y)log(1-y′))

[0138] Where y represents the sample label and y′ represents the predicted label;

[0139] S8: Use the true or false labels of the samples and the domain labels of the samples to supervise the output of the classifier and the output of the domain discriminator, respectively, to obtain the classification loss and the domain discriminator loss. Combined with the loss of contrastive learning, the total loss function is obtained. The total loss function is used to train the model and save the network model and the optimal weights.

[0140] The specific process is as follows: RGB face images are input into the neural network for end-to-end training, and the Adam optimizer is used as the training optimizer, and the learning rate is set to 1×10 -4 , the exponential decay rate β1 of the first-order moment estimate is 0.9, and the exponential decay rate β2 of the second-order moment estimate is 0.999. The network weight coefficients are updated with the purpose of minimizing the total loss function. After the training is completed, the network model and the optimal weight are saved;

[0141] This process involves a total loss function, which is formulated as follows:

[0142] L all =λ1L adv +λ2L cls +λ3LPCL

[0143] Among them L adv is the domain discriminator loss, L cls is the binary classification loss, L PCL is the agent-based contrastive learning loss, λ1, λ2, and λ3 are the weights of the three losses. In this embodiment, their values ​​are 1, 1, and 0.5 respectively;

[0144] S9: Use the trained model to test. Input the facial image test set samples into the trained neural network for feature extraction to obtain self-assembled features. The self-assembled features are passed through the classifier to obtain output features. The output features are calculated to obtain the predicted probability of the facial image being real. Various benchmark indicators are calculated based on the predicted probability and the true and false labels. The specific process is as follows:

[0145] The test set samples are input into the trained neural network for feature extraction and prediction. The prediction results of the test set samples are output, including the predicted probability of the face image being real and the predicted probability of the face image being fake. Various benchmark indicators are calculated based on the prediction results. The benchmark indicators used include false positive rate (FPR), false negative rate (FNR), half total error rate (HTER), and area under the receiver operating characteristic curve (AUC).

[0146] The relevant calculations of the above benchmark indicators require the use of a confusion matrix, which is shown below:

[0147]

[0148] The ROC curve is plotted by calculating the false positive rate and true positive rate (TPR) at different thresholds, with FPR as the horizontal axis and TPR as the vertical axis. The AUC is the area under the ROC curve. The larger the AUC value, the better the detection effect of the model.

[0149] The false positive rate (FPR) refers to the ratio of the number of non-live faces judged as live faces to the number of faces labeled as non-live faces. The formula is as follows:

[0150]

[0151] The false negative rate (FNR) refers to the ratio of the number of live faces judged as non-live faces to the number of faces labeled as live faces. The formula is as follows:

[0152]

[0153] The half-error rate (HTER) refers to the average of the false positive rate and the false negative rate. The smaller the HTER value, the better the detection effect of the model. The formula is as follows:

[0154]

[0155] This embodiment conducts a three-database cross-one-database experiment on four databases, Replay-Attack, CASIA-MFSD, MSU-MFSD, and Oulu_NPU, to verify the generalization performance of the algorithm. The three-database cross-one-database experiments are respectively OCI database cross-M database, OMI database cross-C database, OCM database cross-I database, and ICM database cross-O database. The experimental results are shown in Tables 1 and 2 below.

[0156] Table 1 Cross-library test HTER

[0157]

[0158] Table 2 Cross-library test AUC

[0159]

[0160]

[0161] This embodiment uses SSDG-M and GDA as the comparison algorithms of the present invention. SSDG-M was published in CVPR in 2020 and improves cross-library performance through unilateral domain generalization. GDA was published in ECCV in 2022 and improves cross-library performance through domain adaptation. Both are representative algorithms in the field of liveness detection, of which GDA is one of the algorithms with relatively advanced performance. As can be seen from Tables 1 and 2, the half-error rate (HTER) of the present invention in the four experiments of OCI library across M library, OMI library across C library, OCM library across I library, and ICM library across O library are 5.42%, 8.61%, 9.75%, and 11.63%, respectively. The area under the ROC curve (AUC) is 97.5%, 96.06%, 93.93%, and 94.14%, respectively. Overall, it is better than the two comparison algorithms, showing good cross-library performance, which verifies the effectiveness of the method of this embodiment.

[0162] Example 2

[0163] like Figure 4As shown, this embodiment provides a face liveness detection system with increased style diversity, which is used to implement the face liveness detection method with increased style diversity in the above-mentioned embodiment 1. The system includes: a data preprocessing module, a content feature extractor, a style feature extractor, a domain discriminator with a gradient reversal layer, a domain discriminator loss calculation module, a class-guided style feature mixing module, a feature self-assembly module, a feature recombination module, an agent-based contrastive learning module, a classifier, a classification loss calculation module, a total loss function construction module, a model training module, and a prediction module;

[0164] In this embodiment, the data preprocessing module is used to divide the dataset video into frames, extract the face area of ​​each frame image to obtain a face image, and set corresponding authenticity labels and domain labels for the face image;

[0165] In this embodiment, the content feature extractor is used to extract content features of the face image;

[0166] like Figure 5 As shown in Figure 1, the content feature extractor is used to extract content features. Its network structure includes a 7×7 convolution layer, a maximum pooling layer, three groups of residual blocks, a 3×3 convolution layer, and a BN layer.

[0167] The specific process is as follows: the resolution of the input image is H×W×C. First, the input image is passed through an initial feature extraction network to obtain a size of The initial feature extraction network mainly includes a convolution layer with a convolution kernel size of 7×7, a step size of 2, and an output channel number of N, and a maximum pooling layer; then the initial features are passed through three groups of residual blocks to obtain the size of The feature map F Block_1 、F Block_2 and F Block_3 ,It is worth noting that the residual block consists of stacked convolutional layers and skip connections; finally, F Block_3 Through a convolution layer with a convolution kernel size of 3×3 and a BN layer, the size is obtained. Content characteristics;

[0168] Among them, the batch normalization BN (Batch Normalization) in the BN layer is specifically expressed as follows:

[0169]

[0170]

[0171]

[0172] Among them, μ c(x) represents the channel mean of the features of N face image samples in the Batch, σ c (x) represents the channel standard deviation of the features of N face image samples in the batch, and γ and β represent the affine parameters to be learned;

[0173] In this embodiment, the style feature extractor is used to extract style features of the face image;

[0174] like Figure 6 As shown in the figure, the style feature extractor is used to extract style features. The network structure is mainly constructed based on residual blocks and feature pyramids. The feature pyramid is used to aggregate the shallow features output by three groups of residual blocks, and then the aggregated features are passed through the IN layer to obtain style features. Specifically, it includes a 7×7 convolutional layer, a maximum pooling layer, three groups of residual blocks, a feature pyramid structure, multiple 3×3 convolutional layers, multiple IN layers, a maximum pooling layer and a fully connected layer.

[0175] The specific process is as follows: the input image resolution is H×W×C. First, the input image is passed through an initial feature extraction network to obtain a size of The initial feature extraction network mainly includes a convolution layer with a convolution kernel size of 7×7, a step size of 2, and an output channel number of N, and a maximum pooling layer; then the initial features are passed through three groups of residual blocks to obtain the size of The feature map F Block_1 、F Block_2 and F Block_3 , then in F Block_1 、F Block_2 and F Block_3 The feature pyramid structure is used to aggregate multi-scale features. The specific operation process is as follows: Block_1 Through a convolution layer with a convolution kernel size of 3×3 and an IN layer, the size is obtained. The output feature F out_1 , F out_1 and F Block_2 The concatenation is performed in the form of addition, and then a convolution layer with a convolution kernel size of 3×3 and an IN layer is obtained to obtain a size of The output feature F out_2 , F out_2 and F Block_3 The intermediate feature F is obtained by splicing in the form of addition m Finally, the intermediate feature F m Through a convolution layer with a convolution kernel size of 3×3, an IN layer, a maximum pooling layer, and a fully connected layer, a style feature of size 4N is obtained;

[0176] Among them, the instance normalization IN (Instance Normalization) in the IN layer is specifically expressed as follows:

[0177]

[0178]

[0179]

[0180] Among them, μ nc (x) represents the channel mean of the sample feature, σ nc (x) represents the channel standard deviation of the sample feature, γ and β represent the affine parameters to be learned;

[0181] In this embodiment, a domain discriminator with a gradient reversal layer is used to process content features of multiple domains;

[0182] In this embodiment, the domain discriminator loss calculation module is used to supervise the output features of the domain discriminator using the domain label, and cooperate with the gradient reversal layer to perform adversarial learning to obtain the domain discriminator loss;

[0183] In this embodiment, the class-guided style feature mixing module is used to perform a class-guided style feature mixing operation on the feature statistical information of the face image instance to obtain a mixed style feature;

[0184] In this embodiment, the feature self-assembly module is used to assemble content features and style features through an adaptive style conversion method to obtain self-assembled features;

[0185] In this embodiment, the feature recombination module is used to assemble the content features and the mixed style features through an adaptive style conversion method to obtain a recombined feature;

[0186] In this embodiment, an agent-based contrastive learning module is used to perform contrastive learning. The facial image dataset is divided into four categories: Real, Fake1, Fake2, and Fake3. A positive-negative pair relationship is constructed based on the category information. A correlation coefficient vector is obtained based on the correlation coefficients of the positive and negative pairs. The correlation coefficients between the positive pairs are set to zero. A score is calculated by combining the self-assembled features with the class proxy weights of the corresponding target category. The correlation coefficient vector is concatenated with the score and optimized using the agent-based contrastive learning loss to reduce the correlation coefficients between the negative pairs and increase the score after combining the self-assembled features with the class proxy weights of the corresponding target category.

[0187] In this embodiment, the classifier is used to take the self-assembled features as input and calculate the probability value of the face image being real;

[0188] In this embodiment, the classification loss calculation module is used to supervise the output of the classifier based on the true and false labels to obtain the classification loss;

[0189] In this embodiment, the total loss function construction module is used to obtain the total loss function by weighted summing the domain discriminator loss, classification loss, and contrastive learning loss;

[0190] In this embodiment, the model training module is used to obtain a prediction model based on the total loss function training;

[0191] In this embodiment, the prediction module is used to input the facial image test set into the trained prediction model to obtain the predicted probability that the facial image is real.

[0192] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A method for face liveness detection with increased style diversity, characterized in that: The steps include: Divide the dataset video into frames, extract the face area of ​​each frame image to obtain the face image, and set the corresponding authenticity label and domain label for the face image; Extract content features and style features of facial images; The content features are input into the domain discriminator with a gradient reversal layer. The domain labels are used to supervise the output features of the domain discriminator. The adversarial learning is performed with the gradient reversal layer to obtain the domain discriminator loss. Perform a class-guided style feature mixing operation on the feature statistical information of the face image instance to obtain a mixed style feature; The content features and style features are assembled through the adaptive style conversion method to obtain self-assembled features, and the content features and mixed style features are assembled through the adaptive style conversion method to obtain recombined features; Calculate the correlation coefficient between the self-assembly features and the recombinant features, divide the facial image dataset into categories, construct positive and negative pairs based on the category information, obtain the correlation coefficient vector based on the correlation coefficient of the positive and negative pairs, set the correlation coefficient between the positive pairs to zero, calculate the score of the self-assembly features combined with the class proxy weights of the corresponding target category, concatenate the correlation coefficient vector with the score, and optimize it using the agent-based contrastive learning loss to reduce the correlation coefficient between the negative pairs and increase the score of the self-assembly features combined with the class proxy weights of the corresponding target category; The self-assembled features are used to obtain the probability value of the face image being real through the classifier, and the output of the classifier is supervised based on the true and false labels to obtain the classification loss; The total loss function is obtained by weighted summing the domain discriminator loss, classification loss, and contrastive learning loss; The prediction model is obtained based on the total loss function training; The face image test set is input into the trained prediction model to obtain the predicted probability that the face image is real.

2. The method for face liveness detection with increased style diversity according to claim 1, characterized in that: The content features are input into a domain discriminator with a gradient reversal layer, and the output features of the domain discriminator are supervised by using domain labels, specifically including: The content features are passed through the dimensionality reduction network to obtain the reduced content features. The reduced content features are passed through the domain discriminator to obtain the output features. The domain labels are used to supervise the output features of the domain discriminator to determine the source domain of the input content features. The gradient reversal layer is used to control the gradient of the back propagation, reverse the gradient, and multiply the gradient in the back propagation process by a dynamically changing coefficient λ. Specifically expressed as: Among them, i c Indicates the current number of iterations, i s Indicates the total number of iterations.

3. The method for face liveness detection with increased style diversity according to claim 1, characterized in that: The domain discriminator loss function is expressed as: Among them, x represents the input image, y represents the domain label of the input image, X represents the set of input images, and Y D Represents the set of domain labels, M represents the number of different source domains, G and D represent the content feature generator and domain discriminator respectively, E represents the mathematical expectation, and the domain discriminator loss of all images in the set is averaged as the domain discriminator loss of the entire set. is an indicator function that is used to divide the input images into different sets according to the domain labels.

4. The method for face liveness detection with increased style diversity according to claim 1, characterized in that: The feature statistics of the face image instance are subjected to a class-guided style feature mixing operation, specifically including: When the style feature mixing operation is performed between face image samples of the same category, the content feature F of the face image sample F is obtained. c and style characteristics F s , calculate the content feature F c Instance-normalized content features Style Features F s Obtain affine parameters γ and β through a single fully connected layer, randomly select another face image sample F1, face image sample F1 and face image sample F belong to the same category, and obtain its style feature F 1s The corresponding affine parameters γ1 and β1, the style feature mixing operation is specifically expressed as: c mix =λγ1+(1-λ)γ b mix =λβ1+(1-λ)β Where λ represents the mixing coefficient randomly sampled from the uniform distribution U(0,1), γ mix , β mix Represents the affine parameters corresponding to the mixed style features; When the style feature mixing operation is performed between face image samples of different categories, the face image sample F is obtained and the content feature F is calculated. c Instance-normalized content features Get style feature F s Corresponding affine parameters γ and β, randomly select another face image sample F1, and obtain its style feature F 1s Corresponding affine parameters γ1 and β1, randomly select another face image sample F2, and obtain its style feature F 2s Corresponding affine parameters γ2 and β2, face image sample F and face image sample F1 belong to different categories, face image sample F1 and face image sample F2 belong to the same category, and the style feature mixing operation is specifically expressed as: c mix =λγ1+(1-λ)γ2 b mix =λβ1+(1-λ)β2 Where λ represents the mixing coefficient randomly sampled from the uniform distribution U(0,1), γ mix , β mix Represents the affine parameters corresponding to the mixed style features.

5. The method for face liveness detection with increased style diversity according to claim 1, characterized in that: The content features and style features are assembled through the adaptive style conversion method to obtain self-assembled features, which are specifically expressed as: Among them, F ii Indicates the self-assembly characteristics, F ci Represents a face image sample F i content characteristics, Represents a face image sample F i Channel mean of content features, Represents a face image sample F i Channel standard deviation of content features, γ i , β i Represents a face image sample F i Affine parameters corresponding to style features; and / or; The content features and the mixed style features are assembled by the adaptive style conversion method to obtain the recombined features, which are specifically expressed as: Among them, F ij Indicates the recombination characteristics, F ci Represents a face image sample F i content characteristics, Represents a face image sample F i Channel mean of content features, Represents a face image sample F i Channel standard deviation of content features, γ mix , β mix Represents the affine parameters corresponding to the mixed style features.

6. The method for face liveness detection with increased style diversity according to claim 1, characterized in that: The correlation coefficient vector is concatenated with the score and then optimized using the agent-based contrastive learning loss, which is specifically expressed as: Select one of the face image samples as the anchor sample, perform dimensionality reduction on its self-assembly features and recombinant features, and map the reduced dimensionality self-assembly features and recombinant features through a mapping head to obtain the mapped self-assembly features and recombinant features; Calculate the autocorrelation coefficient of the mapped self-assembly feature, calculate the mutual correlation coefficient of the mapped self-assembly feature and the recombinant feature, and concatenate the autocorrelation coefficient and the mutual correlation coefficient along the Y dimension to obtain the correlation coefficient vector R corresponding to the anchor sample. i ; According to the category information, the positive and negative pair relationship is constructed. The correlation coefficient corresponding to the negative pair remains unchanged, and the correlation coefficient corresponding to the positive pair is set to zero to obtain the correlation coefficient vector R corresponding to the anchor sample. i′ ; The proxy weights of multiple categories are concatenated to obtain the proxy weight W, and the mapped self-assembly features are jointly calculated with the proxy weight W to obtain the score vector S i , the score s of the target category corresponding to the self-assembly feature after screening and mapping i ; The correlation coefficient vector R i′ With score s i Splicing to get vector U i , using agent-based contrastive learning loss to train the vector U i To constrain.

7. The method for face liveness detection with increased style diversity according to claim 6, characterized in that: The agent-based contrastive learning loss is specifically expressed as: Among them, L PCL represents the agent-based contrastive learning loss, F ii′ ' represents the self-assembly characteristics after mapping, w c Represents the self-assembly feature F after mapping ii′ 'The corresponding target class proxy weight, K represents the face image sample F i The number of negative pairs in the batch when used as anchor samples, α represents the scaling factor.

8. The method for face liveness detection with increased style diversity according to claim 1, characterized in that: The classification loss is specifically expressed as: L cls =-(ylog(y′)+(1-y)log(1-y′)) Among them, y represents the face image sample label, and y′ represents the predicted label.

9. A face liveness detection system with increased style diversity, characterized in that: include: Data preprocessing module, content feature extractor, style feature extractor, domain discriminator with gradient reversal layer, domain discriminator loss calculation module, class-guided style feature mixing module, feature self-assembly module, feature recombination module, agent-based contrastive learning module, classifier, classification loss calculation module, total loss function construction module, model training module, prediction module; The data preprocessing module is used to divide the dataset video into frames, extract the face area of ​​each frame image to obtain a face image, and set corresponding authenticity labels and domain labels for the face image; The content feature extractor is used to extract content features of the face image; The style feature extractor is used to extract style features of a face image; The domain discriminator with the gradient reversal layer is used to process content features of multiple domains; The domain discriminator loss calculation module is used to supervise the output features of the domain discriminator using the domain label, and cooperate with the gradient reversal layer to perform adversarial learning to obtain the domain discriminator loss; The class-guided style feature mixing module is used to perform a class-guided style feature mixing operation on the feature statistical information of the face image instance to obtain a mixed style feature; The feature self-assembly module is used to assemble content features and style features through an adaptive style conversion method to obtain self-assembled features; The feature recombination module is used to assemble content features and mixed style features through an adaptive style conversion method to obtain recombined features; The agent-based contrastive learning module is used to perform contrastive learning, classify the facial image dataset into categories, construct positive and negative pair relationships based on category information, obtain a correlation coefficient vector based on the correlation coefficients of the positive and negative pairs, set the correlation coefficients between the positive pairs to zero, calculate the score of the self-assembled features combined with the class proxy weights of the corresponding target category, and then optimize the concatenation of the correlation coefficient vector and the score using the agent-based contrastive learning loss to reduce the correlation coefficients between the negative pairs and increase the score of the self-assembled features combined with the class proxy weights of the corresponding target category. The classifier is used to take the self-assembled features as input and calculate the probability value of the face image being real; The classification loss calculation module is used to supervise the output of the classifier based on the true and false labels to obtain the classification loss; The total loss function construction module is used to obtain a total loss function by weighted summing of the domain discriminator loss, the classification loss, and the contrastive learning loss; The model training module is used to obtain a prediction model based on total loss function training; The prediction module is used to input the face image test set into the trained prediction model to obtain the predicted probability that the face image is real.

10. The face liveness detection system for increasing style diversity according to claim 9, characterized in that: The network structure of the content feature extractor includes a first convolutional layer, a first maximum pooling layer, multiple groups of residual blocks, a second convolutional layer and a BN layer; The facial image is subjected to a first convolutional layer and a first maximum pooling layer to obtain initial features, the initial features are subjected to multiple groups of residual blocks to obtain feature maps of different sizes, and the feature maps are subjected to a second convolutional layer and a BN layer to obtain content features of the facial image; The network structure of the style feature extractor includes a third convolutional layer, a second maximum pooling layer, multiple groups of residual blocks, a feature pyramid structure, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first IN layer, a second IN layer, a third IN layer, a third maximum pooling layer and a fully connected layer; The face image is passed through the third convolution layer and the second maximum pooling layer to obtain initial features, and the initial features are passed through multiple groups of residual blocks to obtain feature maps F of different sizes. Block_1 , feature map F Block_2 and feature map F Block_3 , feature map F Block_1 , feature map F BlOck_2 and feature map F Block_3 After the multi-scale features are aggregated through the feature pyramid structure, the feature map F Block_1 The output feature F is obtained through the fourth convolution layer and the first IN layer out_1 , output feature F out_1 With the feature map F Block_2 The concatenation is performed in the form of addition, and after concatenation, the output feature F is obtained through the fifth convolutional layer and the second IN layer. out_2 , output feature F out_2 With the feature map F Bloxk_3 Splice in the form of addition to obtain the intermediate feature F m , intermediate feature F m The style features of the face image are obtained through the sixth convolutional layer, the third IN layer, the third maximum pooling layer and the fully connected layer.

Images