Face forgery detection method based on feature decoupling

Abstract

The present invention provides a face forgery detection method based on feature decoupling, and proposes a progressive way to decouple the representation of forged features. First, at the image level, the input original image is converted into two complementary views, one uses an adaptive learnable filter to mine subtle frequency perception clues, and the other uses a data enhancement method based on scene conversion to highlight the forged face area and weaken the influence of interference factors. The intermediate layer output features from these two complementary branches are further fused through a trainable hybrid attention module, which contains two branches working in parallel: channel-level self-attention and spatial-level self-attention. Secondly, at the feature level, in order to automatically decompose the features related to forgery and the features related to the source, and reduce the interference of irrelevant factors in the decision-making, two complementary feature decoupling schemes are proposed. Finally, only the decomposed feature space related to forgery is used for prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of face forgery detection, and in particular relates to a face forgery detection method based on feature decoupling. Background Art

[0002] With the continuous development of deep learning technology, more and more deep generative models have been developed to synthesize realistic fake facial images and videos that are difficult for the human eye to detect. In addition to being used for benign entertainment purposes, these synthetic faces can also be abused maliciously.

[0003] The core of face forgery detection is to uncover the essential differences between forged and real face images. When forging a face (whether through face-swapping or expression reshaping), the forgery operation only occurs on part or all of the face (face-swapping modifies the entire face, while expression reshaping modifies a localized area). This means that two types of information are always present in the input face image: source-related information and forgery-related information. Source-related information refers to information retained from the source image, such as background information in face-swapping images and identity information in expression reshaping images; forgery-related information refers to traces or artifacts generated during the forgery process. Unsurprisingly, when using features containing both types of information to classify an image, the presence of a large amount of source-related information can significantly confuse the detector, leading to incorrect decisions. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a face forgery detection method based on feature decoupling to solve at least one of the above problems existing in the prior art.

[0005] Based on the above objectives, one or more embodiments of the present application provide a face forgery detection method based on feature decoupling, which includes the following steps:

[0006] S1. Data preprocessing: convert each video sample into a frame-by-frame image set. Then, use the neural network model for face detection to detect and align the face region of each frame. The detected face frame is then expanded and cropped. The cropped face image is then resized to a uniform size and used as the input image X.

[0007] S2. Convert the input image X into two complementary views, which include an adaptive frequency-domain perceptual feature map and a scene conversion-based view;

[0008] S3, input the two generated complementary views into the dual-branch network to extract features;

[0009] S4, input the intermediate layer features output by the two branch networks into the HAM module for feature fusion to obtain fused features;

[0010] S5. Concatenate the intermediate layer features to obtain concatenated features, and perform channel-level feature decoupling on them.

[0011] S6. Concatenate the fused features to obtain features containing multi-scale information and perform spatial-level feature decoupling.

[0012] S7. Input the decoupled features obtained in step S5 and step S6 into classifiers respectively. Each classifier contains two fully connected layers, and then outputs the corresponding binary decision. Each feature is labeled with a true or false label and supervised training is performed.

[0013] S8. Take the mean of the labels corresponding to the features related to forgery in the output decision as the final prediction result for the input image X.

[0014] Based on the above technical solution of the present invention, the following improvements can also be made:

[0015] Optionally, step S2 includes:

[0016] S21, when extracting the adaptive frequency domain perceptual feature map of the input image, first convert the input image X from the original RGB domain to the frequency domain using discrete cosine transform, and then combine the binary basis filter and learnable frequency filters Adaptively divide it into multiple frequency domain components;

[0017] Among them, the four binary basic filters roughly divide the spectrum into four sub-bands, corresponding to the low frequency of the first 1 / 16 of the entire spectrum, the intermediate frequency between 1 / 16 and 1 / 8 of the spectrum, the high frequency of the last 7 / 8 of the spectrum, and the full-frequency components of the entire spectrum, and the four learnable frequency filters are used to adaptively adjust and select the frequency response of interest outside the fixed filter; finally, the divided frequency components are inversely transformed to the original spatial domain by applying the inverse discrete cosine transform; the process is expressed as:

[0018]

[0019] In the above formula, D(·) and D -1 (·) represents discrete cosine transform and inverse discrete cosine transform operations, ⊙ represents the multiplication of corresponding elements, and the function Used to normalize t to between -1 and 1; then, the four frequency domain components Stacking along the channel dimension, we get a frequency-aware feature map X of size 380×380×12 f ;

[0020] S22, when the input image X is enhanced by scene conversion-based data, for an input image X with Ω pixels and a region Where 0 represents the non-face area and 1 represents the face area. Then, in the same training batch, a random image Y with the same label as X is selected. Then, the scene-converted samples are generated in the following two ways:

[0021] X r =(1-r)⊙X+r⊙Y (2)

[0022] X r =r⊙X+(1-r)⊙Y (3)

[0023] In the above formula, ⊙ represents the multiplication of corresponding elements. Formula (2) means replacing the face area of X with image Y, and formula (3) means replacing the background area of X except the face area with image Y. The above two generation methods are randomly executed with the same probability during the training process, and the labels of the generated images are consistent with the input image X.

[0024] Optionally, the dual-branch network adopts EfficientNet-b4, which includes 7 modules, namely B1, B2, B3, B4, B5, B6 and B7. Starting from B1, it can capture low-level features of the image. As the number of network layers increases, the receptive field gradually becomes larger. By the time it reaches the B7 layer, the image semantic information will be captured. The dual-branch network includes a frequency domain branch and a scene conversion branch. The features extracted by the tth (t=1, 2, ..., 7) module in the frequency domain branch and the scene conversion branch are represented as F t (X f ) and F t (X r ).

[0025] Optionally, when the intermediate layer features output by the two branches are input into the HAM module for feature fusion, the output features of the shallow layer B3, the middle layer B5 and the deep layer B7 of the two branches are respectively selected for HAM operation to cover a more comprehensive feature representation from low-level texture features to high-level semantic information, and finally the features are obtained respectively. and

[0026] Optional, for features and First, the element addition operation is used to fuse the two, and the fused features are expressed as Next, it is input into two parallel attention branches, namely channel-level self-attention and spatial-level self-attention. Finally, the output feature maps of the two are combined again by element-wise multiplication.

[0027] Optionally, when the fused features are input to the channel-level self-attention branch, the feature f t Divide the space into s×s non-overlapping patches Then flatten them into one-dimensional feature vectors, and use three embedding functions θ, γ and η to generate three matrices:

[0028]

[0029] Next, combine K and V to generate the weight matrix

[0030]

[0031] In the above formula, Ct represents the embedding dimension, and the obtained That is, it describes the correlation between the current local features and other local features, and the matrix Q and To perform the fusion:

[0032]

[0033] Finally, all generated They are spatially concatenated to form an enhanced feature map of the channel-level self-attention branch.

[0034] Optionally, when the fused features are input to the spatial self-attention branch, the feature f t Divide into k groups of sub-feature maps by channel It is then flattened into a one-dimensional feature vector and input into the attention layer with the same channel-level self-attention branch structure to output the attention map All generated They are concatenated channel by channel to form an enhanced feature map of the spatial-level self-attention branch.

[0035] Optionally, step S5 includes: f ) and F7(X r ) are connected in series to obtain the feature And f c Adopt channel-level feature decoupling scheme; first, f c It is fed into a global average pooling layer to compress its spatial information, and then input into a multi-layer perceptron network and a Sigmoid activation function σ. The calculated channel-level attention vector is expressed as:

[0036] v c =σ(MLP(GAP(f c )))

[0037] Then, the source-related feature f1 and the forgery-related feature f2 are decoupled as:

[0038] f1=v c ⊙ c f c

[0039] f2=(1-v c )⊙ c f c

[0040] In the above formula, ⊙ c Represents channel-wise multiplication.

[0041] Optionally, step S6 includes: and Connect them in series to obtain features containing multi-scale information And f s Perform spatial level feature decoupling; for feature f s , firstly, the average pooling layer is used to compress its channel information, and then the spatial attention map is calculated through the MLP network and Sigmoid function, which is expressed as:

[0042] v s =σ(MLP(Avg(f s )))

[0043] The source-related feature f3 and the forgery-related feature f4 are finally decomposed into:

[0044] f3=v s ⊙ s f s

[0045] f4=(1-v s )⊙ s f s

[0046] In the above formula, ⊙ s Represents element-wise multiplication.

[0047] Optionally, step S7 includes: after obtaining the four decoupled features f1, f2, f3 and f4, inputting them into four classifiers C1, C2, C3 and C4 respectively, each classifier comprising two fully connected layers, and finally outputting four binary decisions, defined as y1, y2, y3 and y4 respectively; supervised training is performed using four binary cross entropy (BCE) losses, which are expressed as:

[0048]

[0049]

[0050]

[0051]

[0052] Where N represents the number of training samples, and the overall loss function is a combination of these four loss terms:

[0053] L cls =αL1+(1-α)L2+βL3+(1-β)L4

[0054] In the above formula, α and β represent the trade-off parameters, which are set here to α = β = 0.5, that is, the weight of each loss term is the same.

[0055] The beneficial effect of the present invention is that it provides a face forgery detection method based on feature decoupling, and proposes a progressive approach, that is, decoupling the representation of forgery features from the image level to the feature level. First, at the image level, the input original image is converted into two complementary views, one uses an adaptive learnable filter to mine subtle frequency perception clues, and the other uses a scene conversion-based data enhancement method to highlight the forged face area and weaken the influence of interference factors. Among them, the intermediate layer output features from the two complementary branches are further fused through a trainable hybrid attention module, which contains two branches working in parallel: a channel-level self-attention branch and a spatial-level self-attention branch. Secondly, at the feature level, in order to automatically decompose the features related to forgery and the features related to the source, and reduce the interference of irrelevant factors in the decision, two complementary feature decoupling schemes are proposed, and finally only the decomposed feature space related to forgery is used for prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The figure is a flowchart of a face forgery detection method based on feature decoupling according to an embodiment of the present invention.

[0057] Figure 2 Schematic diagram of the network structure of EfficientNet-b4, a face forgery detection method based on feature decoupling according to an embodiment of the present invention.

[0058] Figure 3 The figure is a flow chart of the HAM module of a face forgery detection method based on feature decoupling according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0060] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in one or more embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0061] refer to Figure 1-Figure 3 , one or more embodiments of the present application provide a face forgery detection method based on feature decoupling, such as Figure 1 As shown, it includes the following steps:

[0062] S1. Data preprocessing: convert each video sample into a frame-by-frame image set. Then, use the neural network model for face detection to detect and align the face region of each frame. The detected face frame is then expanded and cropped. The cropped face image is then resized to a uniform size and used as the input image X.

[0063] Specifically, most of the currently available face forgery datasets are composed of videos, so the videos need to be preprocessed first. For each video sample, it is converted into a set of images frame by frame, and then the MTCNN network (a neural network model for face detection tasks) is used to detect and align the face area of the single frame image. Then, the detected face frame is expanded outward by 1.3 times (in order to include a small amount of background) and cropped. Finally, the cropped face image is uniformly resized to 380×380×3 as the input image.

[0064] S2. Convert the input image X into two complementary views, which include an adaptive frequency-domain perceptual feature map and a scene conversion-based view;

[0065] Specifically, step S2 includes:

[0066] S21. For the face forgery detection task, many key discriminative clues cannot be learned directly from the original RGB data, but are hidden in the frequency domain. Therefore, here we consider extracting the frequency domain features of image X. For the extraction of frequency-aware features, it is difficult to adaptively expose subtle forgery traces using fixed or hand-crafted filters. Therefore, an adaptive frequency-aware feature extraction method is adopted here. That is, when extracting the adaptive frequency-domain-aware feature map of the input image, the input image X is first converted from the original RGB domain to the frequency domain using discrete cosine transform, and then the binary basis filter is combined to extract the frequency domain features. and learnable frequency filters Adaptively divide it into multiple frequency domain components;

[0067] Among them, the four binary basic filters roughly divide the spectrum into four sub-bands, corresponding to low frequency (the first 1 / 16 of the entire spectrum), medium frequency (between 1 / 16 and 1 / 8 of the spectrum), high frequency (the last 7 / 8 of the spectrum) and full frequency (the entire spectrum) components, and the four learnable frequency filters are used to adaptively adjust and select the frequency response of interest outside the fixed filter; finally, the divided frequency components are inversely transformed to the original spatial domain by applying the inverse discrete cosine transform; the process is expressed as:

[0068]

[0069] In the above formula, D(·) and D -1 (·) represents discrete cosine transform and inverse discrete cosine transform operations, ⊙ represents the multiplication of corresponding elements, and the function Used to normalize t to between -1 and 1; then, the four frequency domain components Stacking along the channel dimension, we get a frequency-aware feature map X of size 380×380×12 f ;

[0070] S22. When the input image X is augmented with scene conversion-based data, for forged images of face swapping and expression reshaping, the features related to forgery are concentrated in the face area, while other areas outside the face are irrelevant to forgery, that is, areas related to the source. Based on this, a specific face enhancement technology based on scene conversion is adopted here, which has two purposes: first, to guide the model to learn discriminative features related to forgery from the face area, and second, to destroy the correlation between the face identity and the background in the original image, thereby weakening the interference of irrelevant false factors. For an input image X with Ω pixels and a region Where 0 represents the non-face area and 1 represents the face area. Then, in the same training batch, a random image Y with the same label as X is selected. Then, the scene-converted samples are generated in the following two ways:

[0071] X r =(1-r)⊙X+r⊙Y (2)

[0072] X r =r⊙X+(1-r)⊙Y (3)

[0073] In the above formula, ⊙ represents the multiplication of corresponding elements. Formula (2) means replacing the face area of X with image Y, and formula (3) means replacing the background area of X except the face area with image Y. The above two generation methods are randomly executed with the same probability during the training process, and the labels of the generated images are consistent with the input image X.

[0074] S3, input the two generated complementary views into the dual-branch network to extract features;

[0075] Specifically, the two views X generated f and X r Input to the dual-branch network (denoted by F) to extract features. Here, EfficientNet-b4 is selected as the backbone network of the dual-branch framework. Figure 2 The network structure of EfficientNet-b4 is shown. It mainly consists of 7 modules. The EfficientNet-b4 network contains 7 blocks. These blocks have different numbers of sub-blocks, and each of them has no clear function. Starting from B1, it can capture low-level features of the image. As the number of network layers deepens, the receptive field gradually becomes larger. By the time it reaches the B7 layer, the image semantic information will be captured. It is represented by B1 to B7. The features extracted from the tth (t=1,2,…,7) module of the frequency domain branch and the scene conversion branch are represented as F t (X f ) and F t (X r ).

[0076] S4, input the intermediate layer features output by the two branch networks into the HAM module for feature fusion to obtain fused features;

[0077] Specifically, the intermediate layer features output by the two branches are input into the HAM module for feature fusion. The output features of the shallow layer B3, the middle layer B5 and the deep layer B7 of the two branches are respectively selected for HAM operation to cover a more comprehensive feature representation from low-level texture features to high-level semantic information, and finally the features are obtained respectively. and The flowchart of the HAM module is as follows Figure 3 Specifically, for the feature and First, the element addition operation is used to fuse the two, and the fused features are expressed as Next, it is input into two parallel attention mechanisms: channel-level self-attention (CSA) and spatial-level self-attention (SSA), and finally the output feature maps of the two are combined again by element-wise multiplication.

[0078] CSA branch; Generally speaking, the feature expression and discriminative power of local feature vectors in spatial dimensions are very limited, because it represents a local area with a limited receptive field, but it is actually associated with and affected by information from a larger area. In order to improve the discriminative power of these local feature vectors, long-range dependencies are captured by assigning different weights to them. First, f t The space is divided into s×s (in this embodiment, s=1) non-overlapping patches Then flatten them into one-dimensional feature vectors. Next, use the three embedding functions θ, γ and η to generate three matrices:

[0079]

[0080] Next, combine K and V to generate the weight matrix

[0081]

[0082] In the above formula, Ct represents the embedding dimension. That is, it describes the correlation between the current local features and other local features. In theory, the weights between local features from the forged area are larger, while the weights between local features from the forged area and local features from the non-forged area are smaller. Then, the matrix Q and To perform the fusion:

[0083]

[0084] Finally, all generated They are spatially concatenated to form the enhanced feature map of the CSA branch.

[0085] SSA branch: The CSA branch described above focuses on capturing the correlation between local features in the spatial dimension. Different from it, the SSA branch tends to reflect the correlation between each sub-feature map in the channel dimension, and its purpose is to assign different weights to each channel-level feature map. Specifically, the feature f t Divide into k groups by channel (in this embodiment, k=C, that is, the feature map of each channel is divided into a group) It is then flattened into a one-dimensional feature vector and input into the attention layer with the same structure as CSA to output the attention map Finally, all generated They are connected in series by channel to form the enhanced feature map of the SSA branch.

[0086] S5. Concatenate the intermediate layer features to obtain concatenated features, and perform channel-level feature decoupling on them.

[0087] Specifically, feature F7(X f ) and F7(X r ) are connected in series to obtain the feature And f c A channel-level feature decoupling scheme is adopted. Specifically, considering that different spatial pixels of the feature map have different importance in the forgery representation, the purpose of channel-level feature decoupling is to generate a channel-level attention vector to separate the features related to forgery and the features related to the source. First, f c It is fed into a global average pooling (GAP) layer to compress its spatial information, and then input into a multi-layer perceptron (MLP) network and a Sigmoid activation function σ. The calculated channel-level attention vector is expressed as:

[0088] v c =σ(MLP(GAP(f c )))

[0089] Then, the source-related feature f1 and the forgery-related feature f2 are decoupled as:

[0090] f1=v c ⊙ c f c

[0091] f2=(1-v c )⊙ c f c

[0092] In the above formula, ⊙ c Represents channel-wise multiplication.

[0093] S6. Concatenate the fused features to obtain features containing multi-scale information and perform spatial-level feature decoupling.

[0094] Specifically, the characteristics and Connect them in series to obtain features containing multi-scale information And f sA spatial-level feature decoupling scheme is adopted. Since different channels of the fused features contribute to different degrees of forgery representation and generalization, spatial-level feature decoupling aims to assign a certain weight to each channel of the feature map, thereby separating features related to forgery and features related to the source. s , firstly, the average pooling layer is used to compress its channel information, and then the spatial attention map is calculated through the MLP network and Sigmoid function, which is expressed as:

[0095] v s =σ(MLP(Avg(f s )))

[0096] The source-related feature f3 and the forgery-related feature f4 are finally decomposed into:

[0097] f3=v s ⊙ s f s

[0098] f4=(1-v s )⊙ s f s

[0099] In the above formula, ⊙ s Represents element-wise multiplication.

[0100] S7. Input the decoupled features obtained in step S5 and step S6 into classifiers respectively. Each classifier contains two fully connected layers, and then outputs the corresponding binary decision. Each feature is labeled with a true or false label and supervised training is performed.

[0101] Specifically, after obtaining four decoupled features f1, f2, f3 and f4, they are input into four classifiers C1, C2, C3 and C4 respectively. Each classifier contains two fully connected layers and finally outputs four binary decisions, defined as y1, y2, y3 and y4 respectively. Note that f1 and f3 are source-related features, which means that no matter whether the input image is real or fake, the labels of these two features should belong to the real category, that is, label y t =Real(R). As for features f2 and f4, they are features related to forgery, and their labels should be consistent with the input image, that is, label y gt = Ground Truth (GT), that is, if the input image is real, then the two do not contain any traces of forgery, that is, the label y gt =Real(R), on the contrary, if the image is forged, then both contain forgery traces, that is, label y gt=Fake(F). The proposed framework is supervised by four binary cross entropy (BCE) losses, which are expressed as:

[0102]

[0103]

[0104]

[0105]

[0106] Where N represents the number of training samples. Ultimately, the overall loss function is a combination of these four loss terms:

[0107] L cls =αL1+(1-α)L2+βL3+(1-β)L4

[0108] In the above formula, α and β represent trade-off parameters. In this embodiment, α=β=0.5 is set, that is, the weight of each loss item is the same.

[0109] S8. Take the mean of the labels corresponding to the features related to forgery in the output decision as the final prediction result for the input image X.

[0110] Specifically, since features f2 and f4 contribute most to predicting the authenticity of the input image, we only use these two features to make the final decision. Specifically, to improve detection accuracy, the average of the two decisions y2 and y4 is taken as the final prediction result for the input image X.

[0111] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0112] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A face forgery detection method based on feature decoupling, characterized by: The following steps are involved: S1. Data preprocessing: convert each video sample into a frame-by-frame image set. Then, use the neural network model for face detection to detect and align the face region of each frame. The detected face frame is then expanded and cropped. The cropped face image is then resized to a uniform size and used as the input image X. S2, convert the input image X into two complementary views, the complementary views include an adaptive frequency domain perception feature map and a view based on scene conversion; when extracting the adaptive frequency domain perception feature map from the input image, first convert the input image X from the original RGB domain to the frequency domain using discrete cosine transform, and then combine the binary basis filter and learnable frequency filters Adaptively divide it into multiple frequency domain components; when the input image X is enhanced based on scene conversion, for an input image X with Ω pixels and a region r, Where 0 represents the non-face area and 1 represents the face area. Then, in the same training batch, a random image Y with the same label as X is selected. Then, the scene-converted samples are generated in the following two ways: X r =(1-r)⊙X+r⊙Y (2) X r =r⊙X+(1-r)⊙Y (3) In the above formula, ⊙ represents the multiplication of corresponding elements. Formula (2) represents replacing the face area of X with image Y. Formula (3) represents replacing the background area of X except the face area with image Y. The above two generation methods are randomly executed with the same probability during the training process, and the labels of the generated images are consistent with the input image X. S3, input the two generated complementary views into the dual-branch network to extract features; S4, input the intermediate layer features output by the two branch networks into the HAM module for feature fusion to obtain fused features; S5. Concatenate the intermediate layer features to obtain concatenated features, and perform channel-level feature decoupling on them. S6. Concatenate the fused features to obtain features containing multi-scale information and perform spatial-level feature decoupling. S7. Input the decoupled features obtained in step S5 and step S6 into classifiers respectively. Each classifier contains two fully connected layers, and then outputs the corresponding binary decision. Each feature is labeled with a true or false label and supervised training is performed. S8. Take the mean of the labels corresponding to the features related to forgery in the output decision as the final prediction result for the input image X.

2. The face forgery detection method based on feature decoupling according to claim 1, wherein in step S2: These four binary basic filters roughly divide the spectrum into four sub-bands, corresponding to the low frequency of the first 1 / 16 of the entire spectrum, the intermediate frequency between 1 / 16 and 1 / 8 of the spectrum, the high frequency of the last 7 / 8 of the spectrum, and the full-frequency components of the entire spectrum, and the four learnable frequency filters are used to adaptively adjust and select the frequency response of interest outside the fixed filter; finally, the divided frequency components are inversely transformed to the original spatial domain by applying the inverse discrete cosine transform; the process is expressed as: In the above formula, D(·) and D -1 (·) represents discrete cosine transform and inverse discrete cosine transform operations, ⊙ represents the multiplication of corresponding elements, and the function Used to normalize t to between -1 and 1; then, the four frequency domain components Stacking along the channel dimension, we get a frequency-aware feature map X of size 380×380×12 f .

3. The face forgery detection method based on feature decoupling as claimed in claim 2, characterized in that: The dual-branch network adopts EfficientNet-b4, which includes 7 modules, namely B1, B2, B3, B4, B5, B6 and B7. Starting from B1, it can capture low-level features of the image. As the number of network layers increases, the receptive field gradually becomes larger. By the time it reaches the B7 layer, the semantic information of the image will be captured. The dual-branch network includes a frequency domain branch and a scene conversion branch. The features extracted by the tth module in the frequency domain branch and the scene conversion branch are represented as F t (X f ) and F t (X r ), where t=1,2,…,7.

4. The face forgery detection method based on feature decoupling as claimed in claim 3, wherein: When the intermediate layer features output by the two branches are input into the HAM module for feature fusion, the output features of the shallow layer B3, the middle layer B5 and the deep layer B7 of the two branches are respectively selected for HAM operation to cover a more comprehensive feature representation from low-level texture features to high-level semantic information, and finally the features are obtained respectively. and 5. The face forgery detection method based on feature decoupling as claimed in claim 4 is characterized in that feature and Among them, t = 3, 5, 7; first, the element addition operation is used to fuse the two, and the fused features are expressed as Next, it is input into two parallel attention branches, namely channel-level self-attention and spatial-level self-attention. Finally, the output feature maps of the two are combined again by element-wise multiplication.

6. The face forgery detection method based on feature decoupling according to claim 5, characterized in that: When the fused features are input to the channel-level self-attention branch, the feature f t Divide the space into s×s non-overlapping patches Where n = 1, 2, ..., s 2 ; Then flatten them into one-dimensional feature vectors, and use three embedding functions θ, γ and η to generate three matrices: Next, combine K and V to generate the weight matrix In the above formula, Ct represents the embedding dimension, and the obtained That is, it describes the correlation between the current local features and other local features, and the matrix Q and To perform the fusion: Finally, all generated They are spatially concatenated to form an enhanced feature map of the channel-level self-attention branch.

7. The face forgery detection method based on feature decoupling according to claim 6, characterized in that: When the fused features are input to the spatial self-attention branch, the feature f t Divide into k groups of sub-feature maps by channel Where n = 1, 2, ..., k; it is then flattened into a one-dimensional feature vector and input into the attention layer with the same structure as the channel-level self-attention branch, outputting the attention map All generated Where n = 1, 2, ..., K; they are connected in series by channel to form an enhanced feature map of the spatial level self-attention branch.

8. The face forgery detection method based on feature decoupling according to claim 7, characterized in that: Step S5 includes: f ) and F7(X r ) are connected in series to obtain the features And f c Adopt channel-level feature decoupling scheme; first, f c It is fed into a global average pooling layer to compress its spatial information, and then input into a multi-layer perceptron network and a Sigmoid activation function σ. The calculated channel-level attention vector is expressed as: v c σ(MLP(GAP(f c ))) Then, the source-related feature f1 and the forgery-related feature f2 are decoupled as: f1=v c ⊙ c f c f2=(1-v c )⊙ c f c In the above formula, ⊙ c Represents channel-wise multiplication.

9. The face forgery detection method based on feature decoupling as claimed in claim 8, characterized in that: Step S6 includes: and Connect them in series to obtain features containing multi-scale information And f s Perform spatial level feature decoupling; for feature f s , firstly, the average pooling layer is used to compress its channel information, and then the spatial attention map is calculated through the MLP network and Sigmoid function, which is expressed as: in s =σ(MLP(Avg(f s ))) The source-related feature f3 and the forgery-related feature f4 are finally decomposed into: f3=v s ⊙ s f s f4=(1-v s )⊙ s f s In the above formula, ⊙ s Represents element-wise multiplication.

10. The face forgery detection method based on feature decoupling according to claim 9, wherein: Step S7 includes: after obtaining the four decoupled features f1, f2, f3 and f4, inputting them into four classifiers C1, C2, C3 and C4 respectively, each classifier contains two fully connected layers, and finally outputs four binary decisions, defined as y1, y2, y3 and y4 respectively; supervised training is performed using four binary cross entropy losses, which are expressed as: Where N represents the number of training samples, and the overall loss function is a combination of these four loss terms: L cls =αL1+(1-α)L2+βL3+(1-β)L4 In the above formula, α and β represent the trade-off parameters, which are set here to α = β = 0.5, that is, the weight of each loss term is the same.

Images