Deep fake detection method based on irrelevant feature elimination and related feature enhancement

Abstract

The present invention discloses a deepfake detection method based on irrelevant feature elimination and associated feature enhancement. The method comprises: firstly, extracting features using a dual-channel detection backbone network based on the combination of shallow texture features and deep semantic features; at the same time, the method modifies conventional data enhancement and proposes an innovative method for eliminating irrelevant features to enhance the generalization ability of the detector; finally, the method performs landmark-based block preprocessing on the input image and adds an attention mechanism module to the processed image to further enhance the detection accuracy. The method improves the generalization ability of the detection network from multiple perspectives, and the processing module can be easily combined with different networks, ultimately achieving an effective solution to the problem of weak generalization ability of the detection network.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a deep fake detection method based on irrelevant feature elimination and correlated feature enhancement. Background Art

[0002] With the advancement of artificial intelligence (AI), AI-generated audio and video content is gradually entering the public eye. The emergence of large-scale models, such as Sora, has enabled the generation of large quantities of realistic fake videos and images through AI. While this technology has enriched public entertainment, it can also pose significant social risks if used for malicious forgeries. In response to the increasing prevalence of AI-based multimedia forgeries, academia and industry have proposed numerous forgery detection techniques. However, most deepfake detection techniques rely on forgery artifacts generated by specific forgery algorithms, which often exhibit significant variability between different algorithms, resulting in poor generalization. Furthermore, due to the inherent characteristics of neural networks, most current detectors offer strong detection performance for images in the training domain, but their accuracy declines significantly when faced with data with feature distributions that differ from those in the training set. Therefore, developing a high-performance, highly generalizable, and robust deepfake detection network is crucial for defending against and combating malicious AI forgeries. Based on this, this technical solution takes the generalization ability of detectors in deep fake technology as the main entry point, exploring a universal deep fake detection technology that can be trained on a limited data set while having stronger generalization detection capabilities, and can provide better and more reliable detection results when faced with actual data.

[0003] Research has shown that using large models, rich training sets, and general-purpose networks can improve network detection performance on different datasets. However, in actual training, the training set size is often limited. To improve the generalization of detection models in this scenario, the following common strategies are currently available:

[0004] 1. General detection models. Convolutional neural networks are well-suited for capturing regional features in images. Common convolutional image detection neural network model architectures, such as Xception and EfficientNet, often achieve good generalization capabilities.

[0005] 2. Spatial Attention Mechanism. Spatial attention is a method in deep learning that increases the neural network's focus on specific regions of input data. This method generates an attention map on the input feature map, enabling the network to focus more on spatial locations that are useful for the task, improving the network's detection performance in most scenarios.

[0006] 3. Data augmentation: Before the network acquires data, data augmentation such as brightness and contrast adjustment, blurring, rotation, translation, and noise addition can effectively improve the robustness and generalization of model detection.

[0007] Multi-dimensional feature extraction. Shallow features captured by neural networks often represent texture details in an image, while deep features often represent high-dimensional semantic features. The detection network can extract features from different image dimensions to aid detection, such as extracting texture and frequency domain features separately, improving the model's detection performance.

[0008] The existing technology has the following problems:

[0009] 1. Poor cross-domain detection performance. Despite numerous methods to improve model generalization, detection networks often perform poorly when testing cross-domain data. In practical applications, detectors can achieve an AUC of over 99% on training sets, but even the best detectors currently struggle to achieve an AUC of 85% on cross-domain datasets. This is a major problem with existing detectors, and the technical solution of the present invention can address this issue.

[0010] 2. Poor robustness. In actual detection scenarios, data may undergo compression, noise addition, blurring, cropping, and resizing, all of which introduce some interference. When faced with noisy data, the detection network often suffers from poor detection performance, indicating poor network robustness.

[0011] 3. Poor anti-escape and countermeasure capabilities. To evade detection, many algorithms generate malicious samples to interfere with detection. Common methods include reconstruction-based evasion algorithms and noise-based countermeasure algorithms. Detectors often have poor detection capabilities against these maliciously processed forged samples.

[0012] Detector selection is difficult. Different detection algorithms often target different forgery features, resulting in significant discrepancies in performance across different test data. Choosing a good detection algorithm for a specific detection problem often requires actual comparative testing, which is costly. Summary of the Invention

[0013] The purpose of the present invention is to address the shortcomings of the existing technology and provide a deep fake detection method based on irrelevant feature elimination and related feature enhancement.

[0014] The object of the present invention is achieved through the following technical solutions:

[0015] A deep fake detection method based on irrelevant feature elimination and relevant feature enhancement includes the following steps:

[0016] S1: Input image A passes through the irrelevant feature elimination module to obtain the image pairs A1 and A2 before and after enhancement, which are input into the Xception detection network as a whole;

[0017] S2: The image pairs A1 and A2 are converted into shallow features F1 and F2 through the shallow network of Xception;

[0018] S3: Calculate the difference between the shallow features F1 and F2 as the difference loss L1;

[0019] S4: Calculate the landmark of image A1 and divide image A1 into blocks according to the landmark;

[0020] S5: Input the divided image into the shallow network of Xception to obtain multiple feature maps M i ;

[0021] S6: Combine the shallow feature F1 with the feature map M i The associated feature enhancement module is input together to enhance the shallow texture features through the feature map to obtain the enhanced texture features F3;

[0022] S7: Input the shallow feature F1 into the deep network of Xception to obtain the deep feature D;

[0023] S8: Concatenate the texture feature F3 and the deep feature D through a fully connected layer, and then obtain the classification prediction result through the fully connected layer;

[0024] S9: Calculate the cross entropy between the prediction result and the label to obtain the loss L2;

[0025] S10: Weight the losses L1 and L2, calculate the final loss L, and use L to optimize the network parameters to achieve network training.

[0026] Furthermore, in step S1, the irrelevant feature elimination module is used to adjust the brightness, contrast and saturation of the image.

[0027] Furthermore, in step S2, the shallow network is the Entry Flow part of Xception, which is used to convert the input image into shallow features.

[0028] Furthermore, in step S3, for features F1 and F2, MSE is used to calculate the loss; the loss function L1 is:

[0029]

[0030] Among them, yi and Represent the elements in F1 and F2 respectively, and n represents the total number of elements in F1 and F2.

[0031] Furthermore, the step S4 is specifically as follows: extracting the landmark of the image A1, and then dividing the image into blocks according to the landmark, and dividing the input face into outer face, upper inner face, and lower inner face images.

[0032] Furthermore, in step S5, the obtained outer face, upper inner face, and lower inner face images are input into the shallow network respectively to obtain the regional shallow feature maps M1, M2, and M3 for subsequent correlation feature enhancement; the data is converted into:

[0033] [M1, M2, M3]=NET2(PART(I0))=NET2[S a , S b , S c ]

[0034] Among them S a , S b , S c They are the outer face, the upper inner face, and the lower inner face images respectively. PART represents the landmark-based block operation. i (i∈[1,2,3]) is the output regional feature map, and NET2 is a shallow network.

[0035] Furthermore, in step S6, for the feature map M i Compression and convolution operations are performed to generate attention weights, which are then multiplied with the original feature matrix to enhance the extracted features. The calculation formula is as follows:

[0036] F att =σ(W2δ(W1(M i ))), where F att ∈R c

[0037] Among them, M i To obtain the feature map, σ represents the sigmoid activation function, δ represents the ReLU activation function, and are the weight matrices of the two fully connected layers, and r is the scaling factor used to reduce the number of parameters and calculations;

[0038] Get F att Finally, the original features are weighted to achieve the effect of attention enhancement:

[0039] F′=F1⊙F att

[0040] F′ is the feature after regional attention processing, through F1 and F att Get by element-by-element multiplication; F attis the regional attention matrix.

[0041] Furthermore, in step S7, the deep network is the Middle flow part of Xception, which is used to extract deep semantic information.

[0042] Furthermore, in step S9, the cross entropy loss L2 is calculated by the prediction result P and the label label; the loss function is as follows:

[0043]

[0044] Where N is the number of samples, y i ∈[0, 1] is the true or false label of the image, p i is the predicted result; through y i With p i Calculate the difference between the predicted distribution and the true distribution as the loss to optimize network training.

[0045] The beneficial effects of the present invention are:

[0046] 1. Improved generalization capabilities. Deepfake detectors typically have weak generalization capabilities. This invention, by adding an irrelevant feature removal module and a correlated feature enhancement module, can effectively improve the generalization capabilities of the detector and enhance detection performance when testing on cross-domain data.

[0047] 2. Improved detection accuracy. By modifying the traditional spatial attention mechanism to a regional attention mechanism based on landmark blocks, this invention can better extract specific forgery traces in different areas, enhance the network's attention to image details, and improve detection performance.

[0048] 3. Transferable Modules. Most detection networks focus on detecting specific forgery traces and are not well transferred. The irrelevant feature removal module and the associated feature enhancement module in this invention detect generalized forgery traces. Their structures are relatively separate from the original network structure, making them transferable to different detection networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a model framework diagram of the present invention;

[0050] Figure 2 This is a schematic diagram of traditional spatial attention. DETAILED DESCRIPTION

[0051] The present invention will be described in detail below with reference to the accompanying drawings. Unless there is any conflict, the features of the following embodiments and implementations may be combined with each other.

[0052] Based on the problems of weak generalization and poor cross-domain capabilities in conventional detection technologies, the present invention proposes a generalized detection network based on regional attention mechanism and irrelevant feature elimination. The technical innovations are mainly the following three points. 1: A new method for eliminating irrelevant features is proposed. 2: Combining the attention mechanism with landmark-based image segmentation preprocessing can generate attention maps for different areas of the image. 3: As a general processing flow of the detection algorithm, it can be easily added to different detection networks and has strong portability.

[0053] Explanation of terms:

[0054] 1. Attention mechanism: It is divided into self-attention mechanism, spatial attention mechanism, channel attention mechanism, etc. Based on the spatial attention mechanism, this paper replaces the original features with features extracted by the network according to the regional block image, generates an attention map (i.e., weight), and then weights it to the original features to achieve the effect of enhancing the associated features.

[0055] 2. Irrelevant Feature Elimination: Image features contain attributes that are irrelevant to the forgery trace and the semantics of the image itself, such as brightness, saturation, and contrast. These attributes become irrelevant features (of the forgery trace). By eliminating irrelevant features, the relevance of the extracted features to the forgery trace is enhanced.

[0056] 3. Enhancement of Correlated Features: In a forged face image, different regions may correspond to different forgery traces. The extracted forgery traces are divided into regions and enhanced to enhance the correlation between different forgery traces and corresponding regions.

[0057] 4. Landmark: refers to a set of key points marked on a face image to describe the location and shape of facial features.

[0058] 5. Xception: A general image feature extraction neural network used as the framework of the detection network. e flo w Represents the shallow and middle layers of the network.

[0059] 6. Shallow and deep features: As the number of neural network layers increases, the features extracted will gradually progress from shallow to deep. Shallow features are the features output by the first few layers of the network, often representing image texture information. Deep features are the features output by the data after passing through more layers of the network, often representing image semantic information.

[0060] 7. Feature map: A tensor used to represent image features in the attention mechanism. It is often extracted through the network and can enhance the original features.

[0061] The core of this method lies in using irrelevant feature elimination and correlation feature enhancement modules to extract generalizable forgery features for detection, improving the model's cross-domain detection capabilities. The entire detection network's backbone network uses Xception, extracting features through a dual-channel approach of shallow texture features and deep semantic features, respectively, to obtain image texture features and semantic features. These features are then enhanced through the irrelevant feature elimination and correlation feature enhancement modules to ultimately obtain detection results. It is important to note that the irrelevant feature elimination and correlation feature enhancement modules are highly portable, and different backbone networks can be selected for authenticity detection in practice.

[0062] The present invention provides a deep fake detection method based on irrelevant feature elimination and related feature enhancement. The overall data flow is as follows:

[0063] S1: Input image A passes through the irrelevant feature elimination module to obtain the enhanced image pairs A1 and A2. These are fed into the detection network (Xception network) as a whole. The shallow network and deep network are the first and middle layers of the detection network, namely the entry flow and middle flow, respectively. The last few layers of the Xception network are used to generate the output.

[0064] S2: The image pairs A1 and A2 are converted into shallow features F1 and F2 through the shallow network of Xception.

[0065] S3: Calculate the difference between the shallow features F1 and F2 as the difference loss L1.

[0066] S4: Calculate the landmark of image A1 and divide image A1 into blocks according to the landmark.

[0067] S5: Input the divided image into the shallow network of Xception to obtain multiple feature maps M i .

[0068] S6: Combine the shallow features F1 obtained above with the feature map M i The associated feature enhancement module is input together to enhance the shallow texture features through the feature map to obtain the enhanced texture features F3.

[0069] S7: Continue to input the shallow feature F1 into the deep network of Xception to obtain the deep feature D.

[0070] S8: Concatenate the texture feature F3 and the deep feature D through a fully connected layer, and finally obtain the classification prediction result through another fully connected layer.

[0071] S9: Calculate the cross entropy between the classification result and the label to obtain the loss L2.

[0072] S10: Weight the losses L1 and L2, calculate the final loss L, and use L to optimize the network parameters to achieve network training.

[0073] The corresponding model framework is as follows Figure 1 As shown. The following is a detailed analysis of the functions of each part:

[0074] Input: image to be detected (already resized and preprocessed).

[0075] S1: irrelevant feature elimination module. A forged image contains the semantic features of the image itself, trace features introduced by the forgery algorithm, and some irrelevant features of the image such as brightness and contrast. These irrelevant features will interfere with the detection performance. In extreme cases, when the brightness difference between the real and fake images in the training data set is large, the detector is very likely to mistakenly use brightness as the classification criterion. The function of this module is to eliminate the interference of brightness, contrast, and saturation on the extraction of forged features during image detection. The specific steps are as follows. Adjust the brightness, contrast, and saturation of the image, and combine the images before and after processing into an image pair to input the shallow network in S2. The data is converted into:

[0076] I1=G(I0,i,t), Input1=[I0,I1]

[0077] Where I0 is the input image, I1 is the enhanced image, G is the random enhancement process, i∈[1, 3], t∈[0.7, 1.3] are random parameters that determine the type of enhancement and the degree of change before and after the process. Input1 is the processed image pair, which is fed into the shallow network in S2.

[0078] S2: Shallow network. The Entry Flow portion of Xception is used as the shallow network, which converts input tensors into shallow features. Xception is a general image classification detection network. This technology serves as the backbone of the detection network, with modules (such as irrelevant feature elimination and correlated feature enhancement) added to the network for shared use. Entry Flow is the input flow, a general term for the first few layers of the Xception network.

[0079] For images, shallow features generally correspond to features of image texture details. This component processes two inputs: the image pair Input1 is converted into shallow features F1 and F2. The image, after landmark segmentation, is processed into regional shallow features and input into S5 to calculate the regional feature map of the image. Landmarks are a set of key feature points marked on a face image, describing the position and shape of features such as facial features.

[0080] The data is converted to:

[0081] [F1, F2] = NET2 (Input1) = NET2 [I0, I1]

[0082] NET2 is the shallow network in S2.

[0083] S3: Calculate the difference loss L1. Since the difference between the images F1 and F2 is only in features unrelated to forgery, such as brightness, contrast, and saturation, F1 and F2 should be as similar as possible. This ensures that the extracted features are unrelated to the difference (i.e., features unrelated to forgery, such as brightness, contrast, and saturation), thereby achieving the effect of eliminating irrelevant features. Specifically, the loss is calculated using MSE (Mean Square Error) on the input feature tensors F1 and F2. The loss function is:

[0084]

[0085] Among them, n represents the total number of elements in F1 and F2, and yi is Represent the elements in tensors F1 and F2 respectively.

[0086] S4: Acquisition of regional block images. Since the forged traces in different regions of a forged face may be different, using the traditional spatial attention mechanism, that is, directly using all facial features to generate the attention mechanism weight matrix, will result in the features of different regions being averaged due to the pooling operation, and will not be able to effectively enhance the specific forged traces of the region. Based on this problem, the image can be pre-blocked using landmarks, and features can be extracted from the blocked image to obtain regional feature maps for subsequent correlation feature enhancement of the features directly extracted from the image. The specific approach is as follows: First, extract the landmark of image A1. Here, the toolkit dlib (a library that can be used to calculate image landmarks) is used for feature point extraction. After obtaining the image landmark, the image is blocked according to the landmark, and the input face is divided into three parts: the outer face, the upper part of the inner face, and the lower part of the inner face.

[0087] S5: Calculation of regional feature maps. The network structure is the same as S2. The three parts of the image are input into the shallow network of S2 respectively to obtain the regional shallow feature maps M1, M2, and M3 for subsequent correlation feature enhancement. The data is converted into:

[0088] [M1, M2, M3]=NET2(PART(I0))=NET2[S a , S b , S c ]

[0089] Among them S a , S b , S cis the block image after S4, PART represents the landmark-based block operation in S4, M i (i∈[1,2,3]) is the output regional feature map.

[0090] S6: Correlation feature enhancement module. The spatial attention mechanism is used on the shallow feature F1 of the input to enhance the artifact traces in the feature. The spatial attention mechanism is a technique used in deep learning models that improves the performance of the model by focusing on the important spatial locations in the input feature map. Figure 2 As shown:

[0091] In this architecture, input features are compressed and then subjected to a convolution or full connection, allowing for autonomous learning of weights to be used to weight different channel dimensions, thereby highlighting features that have a greater impact on the results. Feature weights are calculated through convolution and then multiplied with the original features.

[0092] As mentioned above, the traditional spatial attention mechanism uses global pooling, which causes the features of different regions to be averaged due to the pooling operation, and cannot effectively enhance the specific forgery traces of the region. The corresponding improvement is to use regional feature maps for enhancement. Since different regions are explicitly separated according to landmarks, the forgery traces of different regions can be better preserved. The specific approach is as follows: For the shallow features extracted after block division (i.e., the feature map M obtained by S5), i ) performs compression and convolution operations to generate attention weights, which are then multiplied with the original feature matrix to enhance the extracted features. The corresponding calculation formula is as follows:

[0093] F att =σ(W2δ(W1(M i ))), where F att ∈R C

[0094] Among them, M i is the feature map obtained by S5, σ represents the sigmoid activation function, 6 represents the ReLU activation function, and are the weight matrices of the two fully connected layers, and r is the scaling factor used to reduce the number of parameters and calculations. As a whole, represents the dimension of the feature tensor; represents a matrix with rows and C columns; this matrix is ​​used as a parameter of the fully connected layer (the fully connected layer is a common network layer in the neural network), which means that the tensor with feature dimension C is transformed into Dimension; here r and C represent parameters, which are specific numbers in actual use, such as C is 512 and r is 8; Same thing.

[0095] It should be noted that the feature map M i The image is extracted through landmark blocks and shallow networks, which are used to calculate the attention weights. The shallow features F1 in S3 are feature matrices. The former is used to calculate the attention weights, and the latter, as a feature, is multiplied with the weights calculated by the former to obtain the texture features F3 enhanced by the attention mechanism. Both are extracted through shallow networks, but the feature map M i It needs to be segmented by landmark, but F1 does not.

[0096] Get F att Finally, the original features are weighted to achieve the effect of attention enhancement.

[0097] F′=F1⊙F att

[0098] F′ is the feature after regional attention processing (F3 in the framework diagram), which is obtained by combining F1 and F att Obtained by element-by-element multiplication. F1 is the original feature (i.e. shallow feature), F att is the regional attention matrix.

[0099] S7: Deep network. The middle flow of Xception is used to extract deep semantic information. This layer consists of eight layers of depthwise separable convolutions, each of which includes a depthwise convolution and a pointwise convolution. Depthwise convolutions perform convolution operations on each channel separately, while pointwise convolutions combine information between channels using 1x1 convolutions. This architecture effectively reduces network parameters. Repeated depthwise convolutions extract deep semantic features, converting shallow features F1 into deep features D.

[0100] S8: After obtaining the deep features D and shallow features F3, these features need to be fused as a basis for true / false classification. Due to scale differences, the features need to be converted to the same size before performing feature concatenation and fully connected classification. S8 consists of three layers of convolution, pooling, and fully connected layers. The purpose of convolution and pooling is to further extract features while achieving dimensionality reduction and dimensionality normalization. After pooling, the dimensions of the deep features D and texture features F3 become [B, N]. Concatenating D and F3 reduces the dimensions to [B, 2*N]. Finally, after passing through a fully connected layer, the binary classification prediction vector, i.e., the prediction result P, is output.

[0101] S9: Cross entropy loss. The cross entropy loss L2 is calculated by combining the predicted result P with the label label. This loss is used to constrain the predicted result to be close to the true result. Since the prediction is true or false, the binary cross entropy loss is used as the classification result. The loss function is as follows:

[0102]

[0103] N is the number of samples, y i ∈[0, 1] is the true or false label of the image, p i is the predicted result. i With p i The difference between the predicted distribution and the true distribution is calculated as the loss to optimize network training. N is the total number of samples used for training. In practice, the training is divided into batches. Each training run sends a corresponding number of samples to the model for training, so N is the batch size.

[0104] S10: Calculate the mixed loss. By weighting the difference loss L1 and the cross entropy loss L2, we get the mixed loss L, which is used as the final loss for network training. The loss function is as follows:

[0105] L=αL1+(1-α)L2

[0106] Where α is the weighting coefficient. After obtaining the mixed loss L, the entire network can be trained using L.

[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0108] The above embodiments are intended only to illustrate the design concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. The scope of protection of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design concepts disclosed in the present invention are within the scope of protection of the present invention.

Claims

1. A deep fake detection method based on irrelevant feature elimination and related feature enhancement, characterized in that: The following steps are involved: S1: Input image A passes through the irrelevant feature elimination module to obtain the image pairs A1 and A2 before and after enhancement, which are input into the Xception detection network as a whole; S2: The image pairs A1 and A2 are converted into shallow features F1 and F2 through the shallow network of Xception; S3: Calculate the difference between the shallow features F1 and F2 as the difference loss L1; S4: Calculate the landmark of image A1 and divide image A1 into blocks according to the landmark; S5: Input the divided image into the shallow network of Xception to obtain multiple feature maps M i ; S6: Combine the shallow feature F1 with the feature map M i The associated feature enhancement module is input together to enhance the shallow texture features through the feature map to obtain the enhanced texture features F3; S7: Input the shallow feature F1 into the deep network of Xception to obtain the deep feature D; S8: Concatenate the texture feature F3 and the deep feature D through a fully connected layer, and then obtain the classification prediction result through the fully connected layer; S9: Calculate the cross entropy between the prediction result and the label to obtain the loss L2; S10: Weight the losses L1 and L2, calculate the final loss L, and use L to optimize the network parameters to achieve network training.

2. The deep fake detection method based on irrelevant feature elimination and related feature enhancement according to claim 1 is characterized in that: In step S1, the irrelevant feature elimination module is used to adjust the brightness, contrast and saturation of the image.

3. The deep fake detection method based on irrelevant feature elimination and related feature enhancement according to claim 1 is characterized in that: In step S2, the shallow network is the Entry Flow part of Xception, which is used to convert the input image into shallow features.

4. The deep fake detection method based on irrelevant feature elimination and related feature enhancement according to claim 1 is characterized in that: In step S3, for features F1 and F2, MSE is used to calculate the loss; the loss function L1 is: L1=MSE(F1,F2), Among them, y i and Represent the elements in F1 and F2 respectively, and n represents the total number of elements in F1 and F2.

5. The deep fake detection method based on irrelevant feature elimination and related feature enhancement according to claim 1 is characterized in that: The step S4 specifically comprises: extracting the landmark of the image A1, and then dividing the image into blocks according to the landmark, and dividing the input face into outer face, upper inner face, and lower inner face images.

6. The deep fake detection method based on irrelevant feature elimination and related feature enhancement according to claim 5 is characterized in that: In step S5, the obtained outer face, upper inner face, and lower inner face images are input into the shallow network respectively to obtain regional shallow feature maps M1, M2, and M3 for subsequent correlation feature enhancement; The data is converted into: [M1,M2,M3]=NET2(PART(I0))=NET2[S a ,S b ,S c ] Among them S a ,S b ,S c They are the outer face, the upper inner face, and the lower inner face images respectively. PART represents the landmark-based block operation. i (i∈[1,2,3]) is the output regional feature map, and NET2 is a shallow network.

7. The deep fake detection method based on irrelevant feature elimination and related feature enhancement according to claim 6 is characterized in that: In step S6, for the feature map M i Perform compression and convolution operations to generate attention weights, which are then multiplied with the original feature matrix to enhance the extracted features. The calculation formula is as follows: F att =σ(W2δ(W1(M i ))), where F att ∈R C Among them, M i To obtain the feature map, σ represents the sigmoid activation function, δ represents the ReLU activation function, and are the weight matrices of the two fully connected layers, and r is the scaling factor used to reduce the number of parameters and calculations; Get F att Finally, the original features are weighted to achieve the effect of attention enhancement: F ′ =F1⊙F att F ′ It is the feature after regional attention processing, through F1 and F att Get by element-by-element multiplication; F att is the regional attention matrix.

8. The deep fake detection method based on irrelevant feature elimination and related feature enhancement according to claim 1 is characterized in that: In step S7, the deep network is the middle flow part of Xception, which is used to extract deep semantic information.

9. The deep fake detection method based on irrelevant feature elimination and related feature enhancement according to claim 1 is characterized in that: In step S9, the cross entropy loss L2 is calculated by the prediction result P and the label label; the loss function is as follows: Where N is the number of samples, y i ∈[0,1] is the true or false label of the image, p i is the predicted result; through y i With p i Calculate the difference between the predicted distribution and the true distribution as the loss to optimize network training.

Images