A generalized face forgery detection method based on domain robust representation learning

Abstract

This invention discloses a generalized face forgery detection method based on domain robust representation learning, comprising: sampling a rich set of face forgery videos and real face videos to construct a training set of face forgery video frames; training a designed domain robust representation learning-based face forgery detection network using the face forgery video frame training set to obtain network weight parameters with optimal performance; applying the trained face forgery detection network and its weight parameters to detect a cropped and scaled face image to be tested to a specified size, and determining whether the input face image has been forged based on the classification prediction label output by the network. This invention proposes a data style enhancement mechanism and a robust feature learning mechanism, which respectively enhance the training data variation in the visible domain and remove domain representation features, thereby learning feature representations robust to complex domain variations. This significantly improves generalization performance, and the generated feature representations are more stable and robust to data variations.

Description

Technical Field

[0001] This invention relates to the field of deepfake detection, and in particular to a generalized face forgery detection method based on domain robust representation learning. Background Technology

[0002] Currently, deepfake technology allows individuals to easily manipulate facial attributes or identities in videos, thereby deceiving the human visual system. In recent years, many methods have treated face forgery detection as a binary classification task. Initially, these methods relied on extracting manual features, but subsequently shifted to utilizing various deep models to identify potential forgery cues. These models have continuously improved in detection accuracy and demonstrated satisfactory performance on multiple datasets with similar forgery patterns and image scenes. However, in real-world scenarios, forged face data often originates from diverse content contexts and employs various forgery techniques, leading to different statistical distributions between training and unknown test data, thus introducing domain bias. Therefore, although forgery detection models need to demonstrate stable detection performance in real-world scenarios, unfortunately, these detection methods often overfit to visible domain data. When faced with unknown domain test data with different statistical distributions, the model's detection performance deteriorates significantly.

[0003] While many current methods have focused on exploring more generalized forgery feature representations to improve the generalization of detection models, most still lack stable generalization performance. This is partly because current methods inevitably rely on limited data variations or specific forgery cues in the existing training data, lacking representativeness for data in unknown domains. Furthermore, they fail to effectively learn discriminative features robust to domain variations, making it difficult to address the domain bias problem caused by different domain characteristics in complex data scenarios. These issues indicate that current face forgery detection models still have room for improvement in stable generalization. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings and deficiencies of existing technologies and propose a generalized face forgery detection method based on domain-robust representation learning. This method learns discriminative feature representations robust to unknown domains by weakening the representation of domain features in rich data variations. The method consists of two key processes: data style enhancement and robust feature learning. Data style enhancement broadens the data with rich and representative new data based on content style changes and forgery style changes to alleviate the overfitting of the detection model to the visible data. Robust feature learning accurately removes domain features from the discriminative features, including content relevance irrelevant to forgery and domain-sensitive forgery features, and enhances the consistency of discriminative feature expression across different domain variations, ultimately obtaining accurate and robust forgery discriminative feature representations.

[0005] To achieve the above objectives, the technical solution provided by this invention is: a generalized face forgery detection method based on domain robust representation learning, comprising the following steps:

[0006] 1) Sample a rich variety of fake face videos and real face videos, involving different fake face methods and content scenarios. Crop the facial areas in the fake face videos and scale them to a uniform image size. Then perform random data augmentation to obtain a training set of fake face video frames with rich variations.

[0007] 2) The designed domain-robust representation learning face forgery detection network is trained using a training set of face forgery video frames. This learns forgery representations that are robust to domain changes and obtains network weight parameters with optimal performance. The face forgery detection network includes a data style enhancement part and a robust feature learning part. During training, the data style enhancement part stylizes the content information and feature forgery information of the image in the image space and feature space, respectively. Then, it samples representative new style vectors from the corresponding hard style space and reconstructs new samples with the input samples to augment the representative content information and forgery information changes, thus alleviating the overfitting of the network to the visible domain samples. For rich forgery features, the robust feature learning part first removes the content correlation in the forgery features through a content decorrelation module based on feature adversarial fusion reconstruction. Then, a sensitive channel discarding module based on channel correlation changes removes channels in the forgery features that are sensitive to domain changes. Finally, it enhances the expression consistency of the forgery features under different forgery differences through inter-class asymmetric robust consistency loss, thereby obtaining a discriminative feature representation that is robust to domain changes.

[0008] 3) Using the face forgery detection network constructed in step 2) and the network weight parameters trained based on domain robust representation learning, the face image to be detected is cropped and scaled to a specified size and then input into the trained face forgery detection network. The network outputs the classification prediction label to determine whether the input face image has been forged.

[0009] Furthermore, in step 1), for face-forged videos and real face videos sampled from different face-forging methods and content scenarios, video frames of face-forged videos are extracted according to a preset frame interval. The publicly available face detection network MTCNN is used to detect and crop the facial regions in the video frames. The size of the cropped facial images is uniformly scaled to 256×256. The images are enhanced using data augmentation methods such as cropping, noise reduction, contrast enhancement, and compression. The labels of the images are consistent with the labels of the source videos, resulting in a training set of face-forged video frames with rich variations.

[0010] Furthermore, in step 2), the designed domain-robust representation learning face forgery detection network is trained using the face forgery video frame training set constructed in step 1). The training process includes the following steps:

[0011] 2.1) Through the content style enhancement module in the data style enhancement section, the global content information changes of the input image are enriched, involving the image's color, texture, and contrast. The specific process is as follows: In each update cycle, select the data-enhanced N... * A sample set with high classification cross-entropy is difficult to form. The mean and variance of hard samples are extracted as their content style, whereby... Indicating difficult samples The true label of the category for the i-th hard sample This represents the set of all C×H×W real matrices, where C, H, and W represent the number of channels, length, and width of the image, respectively, and the mean value of its channel c. and variance Defined as:

[0012]

[0013]

[0014] In the formula, j represents the element index in channel c. The overall content style vector is The mean vector representing the channel. The variance vector representing the channel, based on the difficult sample set D w Constructing style vectors for all samples in the content style space is difficult. Let x represent the multivariate spaces of the mean vector and variance vector, respectively; for the input sample x i ,from Randomly sampled content styles and compared them with x i The original style Combined into a new content style Let these represent the new content mean vector and content variance vector, respectively:

[0015]

[0016] In the formula, Represents the standard normal distribution, ∈ μ and ∈ σ All from A random vector sampled from the middle, k augHyperparameters representing the degree of enhancement are used to embed the new content style vector into the input sample x. i :

[0017]

[0018] In the formula, the newly generated sample x' i This is a sample with enhanced content style, compared to the original sample x. i Together they serve as input samples for subsequent feature extraction networks;

[0019] 2.2) Based on the input samples with enhanced content style, a feature extraction network is used to obtain their deep features. The content-relevance module in robust feature learning removes content relevance unrelated to the forged features. Specifically, the feature extraction network obtains the forged features z of the input sample. i The AdaIN method is used to embed the sample into the content features of the input sample for constraint, and is represented as follows: q i This represents the input sample x extracted by the content encoder. i The content features are defined by μ(·) and σ(·), which represent the mean and variance of the feature channel dimensions. Then, the embedded and fused features are reconstructed into the image space, and the content decorrelation constraint L is applied. CTD The loss function is defined as:

[0020]

[0021] In the formula, G represents the content encoder, R represents the content decoder containing the AdaIN layer, ||·||1 represents the L1 reconstruction constraint, and n is the number of samples. This is achieved by using the forged features z of the original samples. i The forgery features z' of random content style enhancement samples i Reconstruct the same input sample and suppress content-related expressions within it;

[0022] 2.3) Based on the obtained forged features after removing content relevance, the forged information variation in the features is enriched through the forged style enhancement module in the data style enhancement section. The specific process is as follows: Similar to content style enhancement, firstly, the hard sample set D is extracted in each style space update cycle. w Medium sample characteristics The style of forgery, Represents all C z ×H z ×W z The set of real matrices, C z H z W z These represent the number of channels, length, and width of the feature, respectively, and are calculated as follows:

[0023]

[0024] In the formula, Representing the features of difficult samples respectively The mean and variance of channel c, and the mean and variance vectors in its overall forgery style represent respectively... Construct a hard forgery style space for each category based on the forgery style of hard samples from different categories. Let r represent the multivariate spaces of the class mean vector and variance vector, respectively, and let f represent the real class and f represent the fake class. In each training batch, for the content-style-enhanced input sample x' i Forgery feature z' i Its forgery style The corresponding mean vector and variance vector are respectively from New forgery style variations are randomly sampled and combined to form a new forgery style. The corresponding mean vector and variance vector are respectively

[0025]

[0026] In the formula, k aug To be consistent with the hyperparameters used in content style enhancement, this is used to control the degree of enhancement and to generate a new fake style vector. Embedded forgery feature z' i The embedding method is defined as follows:

[0027]

[0028] In the formula, the obtained feature z” i To forge features for samples after style augmentation, i.e., style-enhanced features;

[0029] 2.4) Based on the sensitive channel discarding module in robust feature learning, feature channels sensitive to domain changes are discarded, and the expression consistency of forged features under different forgery differences is enhanced through asymmetric robust consistency loss, resulting in a feature representation robust to changes in the forgery domain. The specific process is as follows: Calculate the sensitivity score of the feature channel: For feature... First, calculate the covariance matrix ∑, which represents the correlation between its characteristic channels. i :

[0030]

[0031] In the formula, Represents all C z ×C z The set of real matrices, Cz Let T represent the number of channels for the feature, and T represent the transpose of the matrix. Then, obtain ∑. i The row mean vector is used as the channel correlation vector. Each element represents the average correlation between that channel and other channels within the feature, calculating the sample's correlation with feature z. i and z” i Channel correlation vector and The variance between them is used as the sensitivity score vector of the feature channels. The larger the element value in the vector, the more sensitive the channel is to changes in forged information. Different samples will obtain different sensitivity score vectors.

[0032] To obtain robust output for forged features: for feature z i and its channel's sensitivity score vector Given a defined channel drop ratio k drop Determine the threshold of this sensitivity score vector Based on this, the mask M of the feature channel is calculated. ii” Define the mask for channel j′ as M. ii” (j′):

[0033]

[0034] The same method is used to obtain the sensitivity score vector. Corresponding channel mask M il” , mask M ii” and M il” respectively with feature z i Multiplication:

[0035]

[0036] In the formula, ⊙ represents the dot product operation of matrices. and Representing feature z i Robust output of discarding sensitive channels under different sample pair selections;

[0037] Align the fake feature z using robust consistency constraints. i Different robust outputs and To enhance the consistency of forgery features in the expression of forged information, robust consistency loss L... RC Defined as:

[0038]

[0039] In the formula, H′ represents the classification network, KL represents the Kullback-Leibler divergence metric, S represents the softmax function, n represents the number of samples, and τ r / f The temperature parameter used for different classes of samples in the loss function is represented by τ, corresponding to the real class and the fake class, respectively. r and τ f Considering the asymmetric distribution between classes in the face spoofing data distribution, τ r and τ f The settings are also asymmetrical, τ r Greater than τ f For style enhancement feature z” i Using the same features as the original sample z i The same method is used to discard sensitive channels and apply robust consistency loss; cross-entropy loss is used to supervise the network learning feature representations related to the face forgery detection task, with cross-entropy loss L... CE Defined as:

[0040]

[0041] In the formula, y i x represents i The category of real labels, and d represents the output of the sensitive channel discarding module, which is the input of the original sample features and the enhanced sample features, respectively. Here, d represents the discarding operation of the sensitive channel, and d1 and d2 represent the discarding operations based on different channel masks, respectively.

[0042] 2.5) The overall optimization loss L of a face forgery detection network that learns domain robustness representation total for:

[0043] L total =L CE +λ1L CTD +λ2L RC

[0044] In the formula, λ1 and λ2 represent the scaling factors of the loss function, based on the overall optimization loss L. total The face forgery detection network is trained for E epochs with a learning rate α, and the weight parameters of the network with the best domain robust representation are saved.

[0045] Further, in step 3), for the face image to be detected, it is cropped and scaled to a size of 256×256. The face forgery detection network trained in step 2) with the domain robust representation learning and the network weight parameters with the best performance are applied. The cropped and scaled image is input into the face forgery detection network. The network outputs the category prediction label to determine whether the input face image has been forged, that is, whether the input image is a face forgery image. If the category prediction label is 0, it is a real face image. If the category prediction label is 1, it is a face forgery image.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] 1. This invention proposes a representative data style enhancement mechanism, which includes a content style enhancement module and a fake style enhancement module. These modules randomly sample in a constructed challenging style space to explore representative nonlinear feature style changes. Through difficult style recombination, the unknown content features and fake features are enriched, thereby improving the representativeness of the visible data to the unknown domain.

[0048] 2. This invention proposes a robust feature learning mechanism for different types of domain differences, including a content decorrelation module and a sensitive channel discarding module, which respectively remove content-related features unrelated to forgery and domain-sensitive forgery features from the feature representation. While achieving effective domain-robust representation learning, it ensures the class discriminativeness of features, and strengthens the consistency of forgery feature representation in domain changes through asymmetric robust consistency constraints, and improves its adaptability to the asymmetry of data distribution between classes.

[0049] 3. This invention combines a data style enhancement mechanism and a robust feature learning mechanism. Compared with other face forgery detection methods, it significantly improves generalization performance, generates more stable feature representations to data changes, and has better overall robustness. Attached Figure Description

[0050] Figure 1 This is a flowchart of the method of the present invention.

[0051] Figure 2 This is an architecture diagram of a face spoofing detection network. Detailed Implementation

[0052] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0053] like Figure 1 and Figure 2 As shown, this embodiment discloses a generalized face forgery detection method based on domain robust representation learning, including the following steps:

[0054] 1) For face-spoofing videos and real face videos sampled from different face-spoofing methods and content scenarios, a face-spoofing video frame training set with rich variations was created. Video frames of the face-spoofing videos were extracted at certain frame intervals. The main facial regions in the video frames were detected and cropped using the publicly available face detection network MTCNN. The cropped facial images were then uniformly scaled to a size of 256×256. Initial enhancement of the images was performed using common data augmentation methods such as cropping, noise reduction, contrast adjustment, and compression. The labels of the images were kept consistent with the labels of the source videos.

[0055] 2) Based on the training set of face forgery video frames constructed in the previous step, train the face forgery detection network that learns the domain-robust representation. The training process includes the following steps:

[0056] 2.1) Through the content style enhancement module in the data style enhancement section, the global content information changes of the input image are enriched, involving the image's color, texture, and contrast. The specific process is as follows: In each update cycle, select the data-enhanced N... * A sample set with high classification cross-entropy is difficult to form. The mean and variance of hard samples are extracted as their content style, whereby... Indicating difficult samples The true label of the category for the i-th hard sample This represents the set of all C×H×W real matrices, where C, H, and W represent the number of channels, length, and width of the image, respectively, and the mean value of its channel c. and variance Defined as:

[0057]

[0058] In the formula, j represents the element index in channel c. The overall content style vector is The mean vector representing the channel. The variance vector representing the channel, based on the difficult sample set D w Constructing style vectors for all samples in the content style space is difficult. Let x represent the multivariate spaces of the mean vector and variance vector, respectively; for the input sample x i ,from Randomly sampled content styles and compared them with x i The original style Combined into a new content style Let these represent the new content mean vector and content variance vector, respectively:

[0059]

[0060] In the formula, Represents the standard normal distribution, ∈ μ and ∈ σ All from A random vector sampled from the middle, k aug Hyperparameters representing the degree of enhancement are used to embed the new content style vector into the input sample x. i :

[0061]

[0062] In the formula, the newly generated sample x' i This is a sample with enhanced content style, compared to the original sample x. i Together they serve as input samples for subsequent feature extraction networks;

[0063] 2.2) Based on the input samples with enhanced content style, a feature extraction network is used to obtain their deep features. The content-relevance module in robust feature learning removes content relevance unrelated to the forged features. Specifically, the feature extraction network obtains the forged features z of the input sample. i The AdaIN method is used to embed the sample into the content features of the input sample for constraint, and is represented as follows: q i This represents the input sample x extracted by the content encoder. i The content features are defined by μ(·) and σ(·), which represent the mean and variance of the feature channel dimensions. Then, the embedded and fused features are reconstructed into the image space, and the content decorrelation constraint L is applied. CTD The loss function is defined as:

[0064]

[0065] In the formula, G represents the content encoder, R represents the content decoder containing the AdaIN layer, ||·||1 represents the L1 reconstruction constraint, and n is the number of samples. This is achieved by using the forged features z of the original samples. i The forgery features z' of random content style enhancement samples i Reconstruct the same input sample and suppress content-related expressions within it;

[0066] 2.3) Based on the obtained forged features after removing content relevance, the forged information variation in the features is enriched through the forged style enhancement module in the data style enhancement section. The specific process is as follows: Similar to content style enhancement, firstly, the hard sample set D is extracted in each style space update cycle.w Medium sample characteristics The style of forgery, Represents all C z ×H z ×W z The set of real matrices, C z H z W z These represent the number of channels, length, and width of the feature, respectively, and are calculated as follows:

[0067]

[0068] In the formula, Representing the features of difficult samples respectively The mean and variance of channel c, and the mean and variance vectors in its overall forgery style represent respectively... Construct a hard forgery style space for each category based on the forgery style of hard samples from different categories. Let r represent the multivariate spaces of the class mean vector and variance vector, respectively, and let f represent the real class and f represent the fake class. In each training batch, for the content-style-enhanced input sample x' i Forgery feature z' i Its forgery style The corresponding mean vector and variance vector are respectively from New forgery style variations are randomly sampled and combined to form a new forgery style. The corresponding mean vector and variance vector are respectively

[0069]

[0070] In the formula, k aug To be consistent with the hyperparameters used in content style enhancement, this is used to control the degree of enhancement and to generate a new fake style vector. Embedded forgery feature z' i The embedding method is defined as follows:

[0071]

[0072] In the formula, the obtained feature z” i To forge features for samples after style augmentation, i.e., style-enhanced features;

[0073] 2.4) Based on the sensitive channel discarding module in robust feature learning, feature channels sensitive to domain changes are discarded, and the expression consistency of forged features under different forgery differences is enhanced through asymmetric robust consistency loss, resulting in a feature representation robust to changes in the forgery domain. The specific process is as follows: Calculate the sensitivity score of the feature channel: For feature... First, calculate the covariance matrix ∑, which represents the correlation between its characteristic channels. i :

[0074]

[0075] In the formula, Represents all C z ×C z The set of real matrices, C z Let T represent the number of channels for the feature, and T represent the transpose of the matrix. Then, obtain ∑. i The row mean vector is used as the channel correlation vector. Each element represents the average correlation between that channel and other channels within the feature, calculating the sample's correlation with feature z. i and z” i Channel correlation vector and The variance between them is used as the sensitivity score vector of the feature channels. The larger the element value in the vector, the more sensitive the channel is to changes in forged information. Different samples will obtain different sensitivity score vectors.

[0076] To obtain robust output for forged features: for feature z i and its channel's sensitivity score vector Given a defined channel drop ratio k drop Determine the threshold of this sensitivity score vector Based on this, the mask M of the feature channel is calculated. ii” Define the mask for channel j′ as M. ii” (j′):

[0077]

[0078] The same method is used to obtain the sensitivity score vector. Corresponding channel mask M il” , mask M ii” and M il” respectively with feature z i Multiplication:

[0079]

[0080] In the formula, ⊙ represents the dot product operation of matrices. and Representing feature z i Robust output of discarding sensitive channels under different sample pair selections;

[0081] Align the fake feature z using robust consistency constraints. i Different robust outputs and To enhance the consistency of forgery features in the expression of forged information, robust consistency loss L... RC Defined as:

[0082]

[0083] In the formula, H′ represents the classification network, KL represents the Kullback-Leibler divergence metric, S represents the softmax function, n represents the number of samples, and τ r / f The temperature parameter used for different classes of samples in the loss function is represented by τ, corresponding to the real class and the fake class, respectively. r and τ f Considering the asymmetric distribution between classes in the face spoofing data distribution, τ r and τ f The settings are also asymmetrical, τ r Greater than τ f For style enhancement feature z” i Using the same features as the original sample z i The same method is used to discard sensitive channels and apply robust consistency loss; cross-entropy loss is used to supervise the network learning feature representations related to the face forgery detection task, with cross-entropy loss L... CE Defined as:

[0084]

[0085] In the formula, y i x represents i The category of real labels, and d represents the output of the sensitive channel discarding module, which is the input of the original sample features and the enhanced sample features, respectively. Here, d represents the discarding operation of the sensitive channel, and d1 and d2 represent the discarding operations based on different channel masks, respectively.

[0086] 2.5) The overall optimization loss L of a face forgery detection network that learns domain robustness representation total for:

[0087] L total =L CE +λ1L CTD +λ2L RC

[0088] In the formula, λ1 and λ2 represent the scaling factors of the loss function, based on the overall optimization loss L. total The face forgery detection network is trained for E epochs with a learning rate α, and the weight parameters of the network with the best domain robust representation are saved.

[0089] 3) For the face image to be detected, crop and scale it to a size of 256×256. Apply the face forgery detection network trained in step 2) with the network weight parameters that have the best performance, and input the cropped and scaled image into the face forgery detection network. Determine whether the input face image has been forged based on the category prediction label output by the network. That is, whether the input image is a face forgery image. If the category prediction label is 0, it is a real face image. If the category prediction label is 1, it is a face forgery image.

[0090] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A generalization face spoofing detection method based on domain robust representation learning, characterized in that, The method comprises the following steps: 1) sampling face-spoofing videos and real face videos, involving different face-spoofing methods and content scenes, cropping the face region in the face-spoofing video, and scaling to a uniform image size, and then performing random data augmentation to obtain a face-spoofing video frame training set; 2) training the face-spoofing detection network designed for domain-robust representation learning using the face-spoofing video frame training set, learning a spoofing representation that is robust to domain changes, and obtaining the network weight parameters with the best performance; wherein the face-spoofing detection network comprises a data style enhancement part and a robust feature learning part, during training, the data style enhancement part respectively styles the content information and the spoofing information of the image space and the feature space, then samples a representative new style vector from the corresponding difficult style space, and recombines the new sample with representative content information and spoofing information changes to alleviate the overfitting of the network to the visible domain sample, for the spoofing feature, the robust feature learning part first removes the content correlation in the spoofing feature through a content decorrelation module based on feature adversarial fusion reconstruction, then removes the channels in the spoofing feature that are sensitive to domain changes by a sensitive channel discarding module based on channel correlation changes, and enhances the expression consistency of the spoofing feature under different spoofing differences through an inter-class asymmetric robust consistency loss, thereby obtaining a discriminative feature representation that is robust to domain changes; 3) applying the face-spoofing detection network constructed in step 2) and the network weight parameters trained based on domain-robust representation learning, cropping and scaling the face image to be detected to a specified size, and inputting the face image to the trained face-spoofing detection network, and determining whether the input face image is spoofed according to the classification prediction label output by the network.

2. The domain-robust representation learning based generalized face spoof detection method according to claim 1, characterized in that, In step 1), for the face-spoofing videos and real face videos sampled from different face-spoofing methods and content scenes, video frames of the face-spoofing videos are extracted according to a preset frame interval, a public face detection network MTCNN is used to detect and crop the face region in the video frame, and the size of the cropped face image is uniformly scaled to The image is enhanced using the cropping, noise, contrast and compression in the data enhancement method, and the label of the image is consistent with the label of the source video, to obtain a face-spoofing video frame training set.

3. The domain-robust representation learning based generalized face spoof detection method of claim 1, wherein, In step 2), the face-spoofing detection network designed for domain-robust representation learning is trained using the face-spoofing video frame training set constructed in step 1), and the training process comprises the following steps: 2.1) Enhance the global content information of the input image by the content style enhancement module in the data style enhancement part, which involves the color, texture and contrast of the image, the specific process is: in each update cycle, select the data enhanced samples with high classification cross-entropy to form a difficult sample set , extract the mean and variance of the difficult samples as their content style, where represents the class real label of the difficult sample , for the th difficult sample , represents the set of real matrices of all , respectively represents the channel number, length and width of the image, the mean and variance of the channel of the image are defined as: ； ； In the formula, denotes the element sequence number in the channel , The overall content style vector of , represents the mean vector of the channel, represents the variance vector of the channel, and the style vector of all samples in the difficult sample set is used to construct the difficult content style space , respectively represent the multivariate space of the mean vector and the variance vector; for the input sample , the content style is randomly sampled from , and combined with the original style of to form a new content style , respectively represent the new content mean vector and the content variance vector:​ ； ； ； wherein denotes the standard normal distribution, and are random vectors sampled from , and denotes a hyperparameter controlling the degree of augmentation, embedding the new content style vector into the input sample : ； In the formula, the generated new sample is a sample after content style enhancement, and the original sample is taken as an input sample of a subsequent feature extraction network together. 2.2) Based on the input sample enhanced based on the obtained content style, the deep features thereof are obtained using a feature extraction network, and the content correlation irrelevant to the fake features is removed through a content decorrelation module in robust feature learning. The specific process is as follows: the feature extraction network obtains the fake features of the input sample , and the sample adaptive normalization is used to constrain the embedding of the fake features into the content features of the input sample, which is expressed as , represents the content features of the input sample extracted by the content encoder, and represent the mean and variance of the feature channel dimension; then the fused features are reconstructed to the image space, and the content decorrelation constraint is applied, and the loss function is defined as: ； where G represents a content encoder, R represents a content decoder including a layer, represents a reconstruction constraint, n is the number of samples, by faking features of the original samples and faking features of the random content style augmented samples to reconstruct the same input samples, suppressing the content-related expression therein; 2.3) Based on the obtained forgery features after removing content correlation, enhance the forgery information in the features through the forgery style enhancement module in the data style enhancement part, the specific process is: first extract the difficult sample set in each style space update period The forgery style of the medium sample feature , represents the set of real matrices of all , respectively represent the channel number, length and width of the feature, and the calculation method is as follows: ； ； In the formula, Representing the features of difficult samples respectively aisle The mean and variance of the overall forgery style are represented by the mean vector and variance vector, respectively. Based on the forgery styles of different categories of difficult samples, a corresponding difficult forgery style space is constructed. , Let represent the multivariate spaces of the category mean vector and variance vector, respectively. Represents the real class, This represents the forgery class; in each training batch, for the input samples after content style enhancement... Forgery characteristics Its forgery style The corresponding mean vector and variance vector are respectively ,from New forgery style variations are randomly sampled and combined to form a new forgery style. The corresponding mean vector and variance vector are respectively : ； In the formula, To be consistent with the hyperparameters used in content style enhancement, this is used to control the degree of enhancement and to generate a new fake style vector. Embedded forgery features The embedding method is defined as follows: ； In the formula, the obtained feature is a fake style augmented sample fake feature, that is, a style enhanced feature; 2.4) Based on the sensitive channel discarding module in robust feature learning, the feature channels sensitive to domain changes are discarded, and the expression consistency of the fake features under different fake differences is enhanced through an asymmetric robust consistency loss, obtaining a feature representation robust to fake domain changes. The specific process is as follows: calculate the sensitive score of the feature channel: for the feature , first calculate the covariance matrix representing the correlation between its feature channels: ； In the formula, Indicates all The set of real matrices, The number of channels representing the feature. This represents the transpose of a matrix, and then obtains... The row mean vector is used as the channel correlation vector. Each element represents the average correlation between that channel and other channels within the feature, calculating the sample's correlation with the feature. and Channel correlation vector and The variance between them is used as the sensitivity score vector of the feature channels. , The larger the element value in the vector, the more sensitive the channel is to changes in forged information. Different samples will obtain different sensitivity score vectors. robust output of counterfeit features: for features and their channel's sensitivity score vector with a determined channel dropout ratio determining a threshold for the sensitivity score vector based on which a mask for the feature channel is computed defining the mask for the channel as : ； In the same way, a sensitivity score vector is obtained The corresponding channel mask The mask And Is multiplied respectively with the features ​ ； ； wherein denotes a dot product operation of matrices, and denotes a feature Robust output discarding sensitive channels under different sample pair selection Aligning fake features using robust consistency constraints of different robustness and , enhancing the expression consistency of fake features in fake information changes, robust consistency loss is defined as: ； In the formula, Representing a classification network, express Divergence measure express function, Indicates the number of samples. The temperature parameter used for different classes of samples in the loss function is represented as follows: [insert temperature parameter here]. and Considering the asymmetric distribution between classes in the face spoofing data distribution, and The settings are also asymmetrical. Greater than For style enhancement features Using features similar to those of the original sample The same method is used to discard sensitive channels and apply robust consistency loss; cross-entropy loss is used to supervise the network learning feature representations related to the face forgery detection task. Defined as: ； In the formula, represents the category true label of and respectively represent the output of the original sample feature and the enhanced sample feature input sensitive channel drop module, wherein represents the drop operation of the sensitive channel, and respectively represent the drop operation based on different channel masks; 2.5) Overall optimization loss for face spoofing detection network with domain-robust representation learning is: ； wherein and denotes a scale factor of the loss function, based on the overall optimization loss and learning rate training cycles, the weight parameters of the face-spoofing detection network with the best domain-robust representation learning are saved.

4. The domain-robust representation learning based generalized face spoof detection method of claim 1, wherein, In step 3), for the face image to be detected, the face image is cropped and scaled to a size of 256x256, the face-spoofing detection network trained in step 2) and the network weight parameters with the best performance are applied, the cropped and scaled image is input into the face-spoofing detection network, and it is determined whether the input face image is spoofed according to the class prediction label output by the network, i.e., whether the input image is a face-spoofing image, if the class prediction label is 0, it is a real face image, and if the class prediction label is 1, it is a face-spoofing image.

Images