A face forgery detection method based on cross-domain consistency learning

Abstract

The present invention claims protection for a face forgery detection method (CDCL) based on cross-domain consistency learning, which aims to explore the common forgery features of images generated by different forgery methods and achieve highly generalized face forgery detection, belonging to the field of computer vision technology. The method comprises the following steps: Step 1. The present invention proposes a cross-domain learning module, which helps to extract cross-domain consistency features of images generated by different forgery methods. Step 2. The present invention designs a diversion center differential attention, which can generate key and value pairs by aggregating the pixel-level intensity and gradient information of the query, and inject heterogeneous receptive field sizes into the label to capture deep-level fine-grained features. Step 3. The present invention uses data generated by a set of specific forgery techniques (source domain) and different forgery techniques (target domain) to induce the model to learn cross-domain consistency features. Step 4. In order to reduce the intra-class distance while increasing the inter-class distance, the present invention designs a cross-domain consistency center loss, and obtains the final network model through iterative adversarial training and parameter updating.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a face forgery detection method based on cross-domain consistency learning. Background Art

[0002] With the popularity of deep generative models, facial forgery technology has rapidly advanced, producing a large number of images that are difficult for the human eye to distinguish. These forged media can be abused for unethical and malicious purposes, posing a significant threat to security. Consequently, researchers around the world have joined the research on "face anti-forgery."

[0003] Face forgery detection is a binary classification task. Existing face forgery detection methods primarily detect suspected forgery artifacts left behind during the manipulation process. Early works primarily used hand-crafted features to identify differences between real and forged images, such as the lack of blinking or inconsistent head poses. While effective in some cases, these methods often lack sufficient generalization capabilities because hand-crafted features often require in-depth expert knowledge and experience to design and can be significantly less applicable to different forgery techniques.

[0004] With the continuous development of deep learning, a large number of deep learning-based methods have emerged that learn texture-related or spectrum-related deep features in the spatial and frequency domains. Representative methods include FDFL (Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection) and SFDG (Dynamic Graph Learning with Content-guided Spatial-Frequency Relation Reasoning for Deepfake Detection). These methods primarily improve the accuracy of identifying forged content by extracting deep features. Subsequently, researchers have also focused on improving the generalization of face forgery detection methods by focusing on data and learning strategies. Representative methods include EFNB4+SBIs (Detecting Deepfakes with Self-Blended Images) and SLAE (Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection). The core idea of ​​this type of method is to enrich the training face image set and use unsupervised learning methods to force the network model not to overfit to a certain forgery method, so that it can perform well in the face of different forgery methods.

[0005] Although these methods have made significant progress in face forgery detection, they still cause the accuracy of model detection to decrease when faced with completely unknown forgery methods. That is, when new types of operations appear, their performance will be adversely affected even if they are semantically similar. This is because these methods all start from the perspective of binary classification, analyzing the feature differences between images generated by the same forgery method and the original images, but ignoring the commonalities between different forgery methods. Therefore, it is not robust to solve the generalization problem of face forgery detection models in real scenarios by training only on a dataset of forged faces. Therefore, how to make the model learn the commonalities between different forged data and achieve highly generalized face forgery detection remains a challenging problem. In order to solve the above problems, the present invention proposes a face forgery detection method based on cross-domain consistency learning.

[0006] After searching, the application publication number CN117292442B is a cross-modal and cross-domain universal face forgery positioning network. The steps of the present invention are as follows: 1. Preprocessing of experimental data; 2. Construction of face forgery positioning network ForgeryNet; 3. Training of ForgeryNet network model; 4. Application of the trained ForgeryNet network model on multiple forgery positioning data sets, and evaluation of the positioning results. However, the patent still only trains the model to learn the interactive features of the spatial domain and frequency domain under the data set of the same forgery method, and cannot well explore the cross-domain consistency features between multiple forgery methods, and the generalization problem of face forgery detection has not been well solved. Therefore, this patent uses a set of data generated by a specific forgery technology (source domain) and different forgery technologies (target domain) to jointly train the model. At the same time, a cross-domain learning module is designed to learn the cross-domain consistency features of images generated by multiple forgery methods. In addition, a cross-domain consistency center loss is further designed to expand the inter-class distance and reduce the intra-class distance, thereby optimizing the accuracy of face forgery detection. Summary of the Invention

[0007] The present invention aims to solve the above problems in the prior art. It proposes a face forgery detection method based on cross-domain consistency learning. The technical solution of the present invention is as follows:

[0008] A face forgery detection method based on cross-domain consistency learning includes the following steps:

[0009] Step 1: Input a set of labeled source data and labeled target data into the detection network. The basic network extracts the training sample features of the source domain and target domain, and feeds them into the differential attention of the diversion center of each branch respectively.

[0010] Step 2: The split center differential attention obtains the local fine-grained features of the forged image by capturing pixel intensity information and pixel gradient information. At the same time, the heterogeneous receptive field size is injected into the markup, and the deep fine-grained features are extracted by using different sizes of keys and values ​​for different attention heads.

[0011] Step 3: The cross-domain learning module captures cross-domain consistency features between the source and target domains by sharing the query in the source domain’s split center differential attention and the key and value in the target domain data.

[0012] Step 4: Input the final features of the source domain and target domain into the classifier to judge the authenticity of the image, and design a cross-domain consistency center loss for supervision; use distillation loss on the final features of the target domain and the cross-domain consistency features of the cross-domain learning module to narrow the difference between the two, so that the target domain branch can obtain the knowledge of the source domain branch as much as possible; at the same time, input the final features of the source domain and target domain pseudo-images into the discriminator for domain judgment, and further learn deep cross-domain consistency representation by minimizing the discrimination loss.

[0013] Furthermore, the step 1 specifically includes the following steps:

[0014] A1. Detect facial key points of the forged face images of the source and target domains through the face detection and alignment network MTCNN, align the forged face images, and crop them into an input image of size 224×224. s , I t ;

[0015] A2、I s , I t Input into the basic network to extract the primary features of the image, respectively using x s 、x t express; x s 、x t Feed to the diversion center differential attention module; C, H, W represent the number of image channels, height and width respectively.

[0016] Furthermore, the step 2 specifically includes:

[0017] B1. Input low-level features x s 、x t Use 3×3 convolution to generate query features X q , the formula is:

[0018] Xq(p0)=∑ pn∈R W(pn)·X(p0+pn) (1)

[0019] Where W(pn) represents the weight, p0 represents the current position on the input and output feature maps, and pn enumerates the position in the receptive field area R; X q Flattened into a sequence of tokens Treat each pixel as a token;

[0020] B2, in X q Local center difference convolution is used to generate fine-grained invariant features It has the shared weight W(pn) of the above 3×3 convolution, and its formula is:

[0021] X kv(p0)=Xq(p0)-∑ pn∈R W(pn)·X(p0) (2)

[0022] B3. In order to obtain deeper and fine-grained features, a diversion system is adopted to divide multiple attention heads into several groups. Each group uses different sizes of key tokens and value tokens. Specifically, for different attention heads, K and V are downsampled to different sizes. The specific formula is as follows:

[0023]

[0024] Among them, MAT() is the multi-scale token aggregation layer, i is the head index, ri is the downsampling rate, W i K , W i V are the parameters of the linear projection; Sum Value Token

[0025] B4. Finally, to obtain the final fine-grained features, the attention score between the inputs is calculated by the dot product between the query and the key, and the softmax function is applied to the score to obtain the probability distribution of positive values; the value is weighted and summed using the attention weight to obtain the final output; the calculation formula is as follows:

[0026]

[0027] Among them, d h Indicates the dimension.

[0028] Furthermore, the step 3 specifically includes:

[0029] C1, low-level features of input x s 、x t By splitting the center differential attention, we can get the Q corresponding to the source domain. s , K s 、V s and Q corresponding to the target domain t , K t 、V t ; Use the source domain’s Q in the cross-domain learning module s and K in the target domain t 、V t To learn forged features that make the target domain image similar to the source domain;

[0030] C2, through Q s and K t The attention score between the inputs is calculated by the dot product between them, and the softmax function is applied to the score to obtain a probability distribution of positive values.t Perform weighted summation to obtain the final output. The calculation formula is as follows:

[0031]

[0032] Among them, d h Indicates the dimension.

[0033] Furthermore, step 4 inputs the final features of the source domain and the target domain into the classifier to judge the authenticity of the image, and designs a cross-domain consistency center loss for supervision, which specifically includes the following steps:

[0034] D1. Center loss maintains a center vector matrix during training, with one center vector for each category. In each training iteration, the loss calculates the Euclidean distance between the feature vector of the current sample and its corresponding center vector, and updates the center vector of the category to which the sample belongs through backpropagation to better cluster similar samples. The formula is as follows:

[0035]

[0036] Where N is the total number of training samples and c is the category center;

[0037] D2. Design cross-domain consistency center loss function L ce , this function adds a maximization category center constraint to expand the distance between categories;

[0038]

[0039] Where N is the number of samples in the mini-batch, x i is the feature of the i-th sample, c is the category center, c s 、c t are the true and false category centers respectively, and m is the interval boundary between category centers;

[0040] D3. Introduce the softmax loss with global information to guide the update of the center point. The formula of the softmax loss function is as follows:

[0041]

[0042] where x i represents the characteristics of the i-th sample belonging to the j-th class, represents the weight matrix of the i-th sample, represents the weight matrix of the jth class, b j represents the bias, and N represents the number of categories.

[0043] Furthermore, step 4 uses distillation loss on the final features of the target domain and the cross-domain consistency features of the cross-domain learning module, specifically including the following steps:

[0044] E1. Feed the features obtained by the cross-domain learning module into the classifier to obtain a probability distribution. The probability distribution is regarded as a soft label to further supervise the target domain branch through distillation loss. The formula is as follows:

[0045]

[0046] Among them, q k and p k are the probabilities of category k obtained from the cross-domain learning branch and the target domain branch, respectively.

[0047] Furthermore, step 4 inputs the final features of the source domain and target domain pseudo images into the discriminator for domain judgment, and further learns the deep cross-domain consistency representation by minimizing the discrimination loss, which specifically includes the following steps:

[0048] F1. Consider the entire feature extraction process as a feature extractor G, and train the entire feature extractor G to deceive fake faces so that the domain labels of fake faces cannot be distinguished. The entire feature extractor G and the discriminator D are trained in an adversarial manner to reduce domain differences and learn deep domain-invariant features of fake faces across different related domains.

[0049] F2. During training, the discriminator is optimized by minimizing the loss of domain classification and the feature extractor is optimized by maximizing the same loss. The specific formula is as follows:

[0050]

[0051] Where M represents the total number of pseudo training samples, x f is a forged image sample, z i is the domain label, E(.) represents the feature extractor, and D(.) is the discriminator.

[0052] Furthermore, the four losses are added together to obtain the final image forgery detection loss L total , specifically including:

[0053] By integrating the cross-domain learning module and the diversion center differential attention into a unified network framework, a face forgery detection framework based on cross-domain consistency learning is basically formed, and its parameters can be obtained by L total To optimize:

[0054] L total =λ1L dtl +λ2L adv +λ3L ce +LSoftmax (11)

[0055] Among them, λ1, λ2 and λ3 represent the hyperparameters in the overall loss function.

[0056] An electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting face forgery based on cross-domain consistency learning as described in any one of the preceding claims is implemented.

[0057] A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any one of the methods for detecting face forgery based on cross-domain consistency learning.

[0058] The advantages and beneficial effects of the present invention are as follows:

[0059] The present invention mainly addresses the problem of poor accuracy of existing face forgery detection methods when faced with an endless stream of forgery methods and volatile real-world factors. A face forgery detection method based on cross-domain consistency learning is designed. The core of the method is to learn the commonalities of multiple forged images by learning the consistent forgery features of images generated by different forgery methods, thereby improving the generalization performance of existing methods in the face of various forged images. At the same time, the diversion center differential attention designed by the present invention prompts the model to pay more attention to the deep and fine-grained features of the forged images, enabling the cross-domain learning module to learn deeper cross-domain consistency features. Finally, the present invention designs a cross-domain consistency center loss to further improve recognition accuracy.

[0060] The main advantages and beneficial effects are as follows:

[0061] 1. This paper designs a split-center differential attention, which extracts features by capturing pixel intensity information and pixel gradient information while injecting heterogeneous receptive field sizes into the label, effectively capturing the deep and fine-grained features of the source and target domains, and improving the detection accuracy of the model on known forged data methods.

[0062] 2. This paper designs a cross-domain learning module, which learns the cross-domain consistency features of the source and target domains by sharing the query (Query) in the source domain diversion center differential attention and the key (Key) and value (Value) in the target domain data, thereby improving the detection accuracy of the model on unknown forged data methods.

[0063] 3. The present invention designs a cross-domain consistency center loss to effectively reduce the intra-class distance between images, increase the inter-class distance, and further improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a schematic diagram of the overall network model structure of a preferred embodiment of the present invention;

[0065] Figure 2 Schematic diagram of the diversion center differential attention structure of an embodiment of the present invention;

[0066] Figure 3 Schematic diagram of the cross-domain learning module structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0068] The technical solution of the present invention to solve the above technical problems is:

[0069] This embodiment of the present invention uses a vision transformer feature extraction network as its basic framework. For details, see the literature Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale [J]. arXiv preprint arXiv:2010.11929, 2020.

[0070] First, the primary features of the source domain and target domain are extracted through the basic network, and are respectively sent to the diversion center differential attention network proposed in the present invention to learn their respective fine-grained forgery features. At the same time, the features of the source domain and target domain are respectively sent to the cross-domain learning module, and the key-value pairs K and V generated from the query Q generated from the image and the target domain features are used to learn the cross-domain consistency features. The source domain features, cross-domain consistency features and target domain features are sent to the label classifier, and the standard cross entropy loss is replaced by the Softmax loss, distillation loss, cross-domain consistency center loss and discrimination loss. The above four losses are used to constitute the total loss to supervise the learning process of the entire face forgery detection model.

[0071] Figure 1 This is a schematic diagram of the overall network model structure of an embodiment of the present invention, which includes the following steps:

[0072] 1. As attached Figure 1 As shown in the figure, a set of labeled source data and labeled target data are input into the detection network. The basic network extracts the training sample features of the source domain and target domain, and feeds them into the differential attention of the diversion center of each branch respectively. The specific steps include:

[0073] A1. Detect facial key points of the forged face images of the source and target domains through the face detection and alignment network MTCNN, align the forged face images, and crop them into an input image of size 224×224. s , I t ;

[0074] A2、I s , I t Input into the basic network to extract the primary features of the image, respectively using x s 、x t express. x s 、x t Feed to the split center differential attention module. C, H, W represent the number of image channels, height, and width respectively.

[0075] 2. As attached Figure 2 This is a schematic diagram of the split-center differential attention structure of an embodiment of the present invention. The split-center differential attention obtains local fine-grained features of the forged image by capturing pixel intensity information and pixel gradient information, while injecting heterogeneous receptive field sizes into the markup. By using different sizes of keys and values ​​for different attention heads, deep fine-grained features are extracted. Specifically, the following steps are included:

[0076] B1. Input low-level features x s 、x t Use 3×3 convolution to generate query features X q , the formula is:

[0077] Xq(p0)=∑ pn∈R W(pn)·X(p0+pn)(1)

[0078] Where W(pn) represents the weight, p0 represents the current position on the input and output feature maps, and pn enumerates the position in the receptive field area R. q Flattened into a sequence of tokens Treat each pixel as a token.

[0079] B2, in X q Local center difference convolution is used to generate fine-grained invariant features It has the shared weight W(pn) of the above 3×3 convolution, and its formula is:

[0080] X kv (p0)=Xq(p0)-∑ pn∈R W(pn)·X(p0) (2)

[0081] B3. To obtain deeper, fine-grained features, a splitting system is used to divide multiple attention heads into several groups, with each group using different sizes of key tokens and value tokens. Specifically, for different attention heads, K and V are downsampled to different sizes. The specific formula is as follows:

[0082] k i 、v i =MAT(X k , r i )W i K ,MAT(X v , r i )W i V (3)

[0083] Among them, MAT() is the multi-scale token aggregation layer, i is the head index, ri is the downsampling rate, W i K , W i V are the parameters of the linear projection. Sum Value Token

[0084] B4. Finally, to obtain the final fine-grained features, the attention score between the inputs is calculated by the dot product between the query and the key, and the softmax function is applied to the score to obtain a probability distribution of positive values. The value is weighted and summed using the attention weight to obtain the final output. The calculation formula is as follows:

[0085]

[0086] Among them, d h Indicates the dimension.

[0087] 3. If Figure 3 This is a schematic diagram of the cross-domain learning module structure of an embodiment of the present invention. The cross-domain learning module captures cross-domain consistency features of the source and target domains by sharing the query (Query) in the source domain diversion center differential attention and the key (Key) and value (Value) in the target domain data; specifically, it includes the following steps:

[0088] C1, low-level features of input x s 、x t By splitting the center differential attention, we can get the Q corresponding to the source domain. s , K s 、V s and Q corresponding to the target domain t , K t 、V t . Use the source domain’s Q in the cross-domain learning modules and K in the target domain t 、V t To learn forged features that make the target domain image similar to the source domain.

[0089] C2, through Q s and K t The attention score between the inputs is calculated by the dot product between them, and the softmax function is applied to the score to obtain a probability distribution of positive values. t Perform weighted summation to obtain the final output. The calculation formula is as follows:

[0090]

[0091] Among them, d h Representation dimension

[0092] 4. Further, Figure 1 As shown in the figure, the final features of the source domain and the target domain are input into the classifier to judge the authenticity of the image, and a cross-domain consistency center loss is designed for supervision. The specific steps include:

[0093] D1. Center loss maintains a center vector matrix during training, with one center vector for each category. In each training iteration, this loss calculates the Euclidean distance between the feature vector of the current sample and its corresponding center vector, and updates the center vector of the category to which the sample belongs through backpropagation to better cluster similar samples. The formula is as follows:

[0094]

[0095] Where N is the total number of training samples and c is the category center.

[0096] D2. Designed a cross-domain consistency center loss function L ce , this function adds a maximization category center constraint to expand the distance between categories;

[0097]

[0098] Where N is the number of samples in the mini-batch, x i is the feature of the i-th sample, c is the category center, c s 、c t are the true and false category centers respectively, and m is the interval boundary between category centers.

[0099] D3. Introduce the softmax loss with global information to guide the update of the center point. The formula of the softmax loss function is as follows:

[0100]

[0101] where x i represents the characteristics of the i-th sample belonging to the j-th class, Represents the weight matrix of the i-th sample, W j T represents the weight matrix of the jth class, b j represents bias, N represents the number of classifications;

[0102] 5. Further, if Figure 1 As shown in Figure 1, distillation loss is used for the final features of the target domain and the cross-domain consistency features of the cross-domain learning module. The specific steps include:

[0103] E1. Feed the features obtained by the cross-domain learning module into the classifier to obtain a probability distribution. The probability distribution is regarded as a soft label to further supervise the target domain branch through distillation loss. The formula is as follows:

[0104]

[0105] Among them, q k and p k are the probabilities of category k obtained from the cross-domain learning branch and the target domain branch, respectively.

[0106] 6. Further, if Figure 1 As shown in the figure, although the use of distillation loss can enable the target domain to learn the forged features of the source domain image, it is easy for the target domain to over-focus on the deep fine-grained information of the source domain rather than the deep domain consistency information of the source and target domains. Therefore, a discriminator is used to further learn cross-domain consistency features. The final features of the source and target domain pseudo images are input to the discriminator for domain judgment, and the deep cross-domain consistency representation is further learned by minimizing the discrimination loss. Specifically, the following steps are included:

[0107] F1. The entire feature extraction process is considered as a feature extractor G. The entire feature extractor G is trained to fool fake faces, making their domain labels indistinguishable. In other words, the entire feature extractor G and the discriminator D are trained in an adversarial manner to reduce domain differences and learn deep domain-invariant features for fake faces across different related domains.

[0108] F2. During training, the discriminator is optimized by minimizing the loss of domain classification and the feature extractor is optimized by maximizing the same loss. The specific formula is as follows:

[0109]

[0110] Where M represents the total number of pseudo training samples, x f is a forged image sample, z iis the domain label, E(.) represents the feature extractor, and D(.) is the discriminator.

[0111] 7. Further, the above four losses are added together to obtain the final image forgery detection loss L total , specifically including:

[0112] By integrating the cross-domain learning module and the diversion center differential attention into a unified network framework, a face forgery detection framework based on cross-domain consistency learning is basically formed, and its parameters can be obtained by L total To optimize:

[0113] L total =λ1L dtl +λ2L adv +λ3L ce +L Softmax (11)

[0114] Among them, λ1, λ2 and λ3 represent the hyperparameters in the overall loss function.

[0115] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.

[0116] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0117] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0118] The above embodiments should be understood as merely illustrating the present invention and not as limiting the scope of protection of the present invention. After reading the contents of the present invention, technicians may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A face forgery detection method based on cross-domain consistency learning, characterized by: The following steps are involved: Step 1: Input a set of labeled source data and labeled target data into the detection network. The basic network extracts the training sample features of the source domain and target domain, and feeds them into the differential attention of the diversion center of each branch respectively. Step 2: The split center differential attention obtains the local fine-grained features of the forged image by capturing pixel intensity information and pixel gradient information. At the same time, the heterogeneous receptive field size is injected into the markup, and the deep fine-grained features are extracted by using different sizes of keys and values ​​for different attention heads. Step 3: The cross-domain learning module captures cross-domain consistency features between the source and target domains by sharing the query in the source domain’s split center differential attention and the key and value in the target domain data. Step 4: Input the final features of the source domain and target domain into the classifier to judge the authenticity of the image, and design a cross-domain consistency center loss for supervision; use distillation loss on the final features of the target domain and the cross-domain consistency features of the cross-domain learning module to narrow the difference between the two, so that the target domain branch can obtain the knowledge of the source domain branch as much as possible; at the same time, input the final features of the source domain and target domain pseudo-images into the discriminator for domain judgment, and further learn deep cross-domain consistency representation by minimizing the discrimination loss.

2. The face forgery detection method based on cross-domain consistency learning according to claim 1 is characterized in that: The step 1 specifically includes the following steps: A1. Detect facial key points of the forged face images of the source and target domains through the face detection and alignment network MTCNN, align the forged face images, and crop them into an input image of size 224×224. s , I t ; A2、I s , I t Input into the basic network to extract the primary features of the image, respectively using x s 、x t Indicates; x s 、 x s 、x t Feed to the diversion center differential attention module; C, H, W represent the number of image channels, height and width respectively.

3. The face forgery detection method based on cross-domain consistency learning according to claim 2 is characterized in that: The step 2 specifically includes: B1. Input low-level features x s 、x t Use 3×3 convolution to generate query features X q , the formula is: Xq(p0)=∑ pn∈R W(pn)·X(p0+pn) (1) Where W(pn) represents the weight, p0 represents the current position on the input and output feature maps, and pn represents the position in the receptive field area R; q Flattened into a sequence of tokens Treat each pixel as a token; B2, in X q Local center difference convolution is used to generate fine-grained invariant features It has the shared weight W(pn) of the above 3×3 convolution, and its formula is: X kv (p0)=Xq(p0)-∑ pn∈R W(pn)·X(p0) (2) B3. In order to obtain deeper and fine-grained features, a diversion system is adopted to divide multiple attention heads into several groups. Each group uses different sizes of key tokens and value tokens. Specifically, for different attention heads, K and V are downsampled to different sizes. The specific formula is as follows: Among them, MAT() is the multi-scale token aggregation layer, i is the head index, r i is the downsampling rate, are the parameters of the linear projection; Sum Value Token B4. Finally, to obtain the final fine-grained features, the attention score between the inputs is calculated by the dot product between the query and the key, and the softmax function is applied to the score to obtain the probability distribution of positive values; the value is weighted and summed using the attention weight to obtain the final output; the calculation formula is as follows: Among them, d h Indicates the dimension.

4. The face forgery detection method based on cross-domain consistency learning according to claim 3 is characterized in that: The step 3 specifically includes: C1, low-level features of input x s 、x t By splitting the center differential attention, we can get the Q corresponding to the source domain. s , K S 、V S and Q corresponding to the target domain t , K t 、V t ; Use the source domain’s Q in the cross-domain learning module s and K in the target domain t 、V t To learn forged features that make the target domain image similar to the source domain; C2, through Q s and K t Calculate the attention score between the inputs by the dot product between them, apply the softmax function to the score to get the probability distribution of positive values; use the attention weight to V t Perform weighted summation to obtain the final output. The calculation formula is as follows: Among them, d h Indicates the dimension.

5. The face forgery detection method based on cross-domain consistency learning according to claim 4 is characterized in that: Step 4 inputs the final features of the source domain and the target domain into the classifier to judge the authenticity of the image, and designs a cross-domain consistency center loss for supervision, which specifically includes the following steps: D1. Center loss maintains a center vector matrix during training, with one center vector for each category. In each training iteration, the loss calculates the Euclidean distance between the feature vector of the current sample and its corresponding center vector, and updates the center vector of the category to which the sample belongs through backpropagation to better cluster similar samples. The formula is as follows: Where N is the total number of training samples and c is the category center; D2. Design cross-domain consistency center loss function L ce , this function adds a maximization category center constraint to expand the distance between categories; Where N is the number of samples in the mini-batch, x i is the feature of the i-th sample, c is the category center, c s 、c t are the true and false category centers respectively, and m is the interval boundary between category centers; D3. Introduce the softmax loss with global information to guide the update of the center point. The formula of the softmax loss function is as follows: where x i represents the characteristics of the i-th sample belonging to the j-th class, represents the weight matrix of the i-th sample, represents the weight matrix of the jth class, b j represents the bias, and N represents the number of categories.

6. The face forgery detection method based on cross-domain consistency learning according to claim 5 is characterized in that: Step 4 uses distillation loss on the final features of the target domain and the cross-domain consistency features of the cross-domain learning module, specifically including the following steps: E1. Feed the features obtained by the cross-domain learning module into the classifier to obtain a probability distribution. The probability distribution is regarded as a soft label to further supervise the target domain branch through distillation loss. The formula is as follows: Among them, q k and p k are the probabilities of category k obtained from the cross-domain learning branch and the target domain branch, respectively.

7. The face forgery detection method based on cross-domain consistency learning according to claim 6 is characterized in that: Step 4 inputs the final features of the source domain and target domain pseudo images into the discriminator for domain judgment, and further learns the deep cross-domain consistency representation by minimizing the discrimination loss. Specifically, it includes the following steps: F1. Consider the entire feature extraction process as a feature extractor G, and train the entire feature extractor G to deceive fake faces so that the domain labels of fake faces cannot be distinguished. The entire feature extractor G and the discriminator D are trained in an adversarial manner to reduce domain differences and learn deep domain-invariant features of fake faces across different related domains. F2. During training, the discriminator is optimized by minimizing the loss of domain classification and the feature extractor is optimized by maximizing the same loss. The specific formula is as follows: Where M represents the total number of pseudo training samples, x f is a forged image sample, z i is the domain label, G represents the feature extractor, and D(.) is the discriminator.

8. The face forgery detection method based on cross-domain consistency learning according to claim 7 is characterized in that: The four losses are added together to obtain the final image forgery detection loss L total , specifically including: By integrating the cross-domain learning module and the diversion center differential attention into a unified network framework, a face forgery detection framework based on cross-domain consistency learning is basically formed, and its parameters can be obtained by L total To optimize: L total =λ1L dtl +λ2L adv + λ3L ce + L Softmax (11) Among them, λ1, λ2 and λ3 represent the hyperparameters in the overall loss function.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting face forgery based on cross-domain consistency learning according to any one of claims 1 to 8 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting face forgery based on cross-domain consistency learning according to any one of claims 1 to 8 is implemented.

Images