Depth forgery detection method based on irrelevant feature elimination and associated feature enhancement

Abstract

The invention discloses a deep forgery detection method based on irrelevant feature elimination and associated feature enhancement. Comprising the following steps: firstly, based on combination of shallow texture features and deep semantic features, extracting features by using a dual-channel detection backbone network; meanwhile, the technology is modified on the basis of conventional data enhancement, and an innovative irrelevant feature elimination means is provided to enhance the generalization ability of the detector; and finally, the technology carries out landmark-based block preprocessing on the input image, and an attention mechanism module is added to the processed image, so that the detection precision is further enhanced. According to the method, the generalization ability of the detection network is improved from multiple angles, meanwhile, the processing module can be conveniently combined with different networks, and finally an effective solution for the problem that the generalization ability of the detection network is weak is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a deep forgery detection method based on irrelevant feature elimination and associated feature enhancement. Background Art

[0002] With the development of artificial intelligence technology, AI-generated audio and video content has gradually come into the public eye. The emergence of large models represented by Sora has enabled people to generate a large number of realistic forged videos and images through AI. On the one hand, this technology has enriched the entertainment life of the public. On the other hand, if this technology is used for malicious forgery, it may cause great harm to society. In response to the ever-increasing proliferation of artificial intelligence multimedia forgery technology, academia and industry have proposed many forgery detection technologies. However, since most deep forgery detection technologies are based on forgery traces generated by specific forgery algorithms, and there are often large differences between forgery traces generated by different forgery algorithms, they perform poorly in terms of generalization. At the same time, due to the characteristics of the neural network itself, most detectors currently have strong image detection performance for the training domain, but when faced with data with different feature distributions from the training set, the detection accuracy will be greatly reduced. Therefore, studying a high-performance, high-generalization and robust deep forgery detection network is of great significance for defending and combating malicious AI forgery technology. Based on this, this technical solution takes the generalization ability of detectors in deep fake technology as the main entry point, and explores a universal deep fake detection technology, which can be trained on a limited data set and has stronger generalization detection capabilities, and can provide better and more reliable detection results when faced with actual data.

[0003] Research shows that using large models, rich training sets, and general networks can improve the network's detection performance on different data sets. However, in actual training, the size of the training set is often limited. To improve the generalization of the detection model in this scenario, there are several common strategies:

[0004] 1. General detection model. Convolutional neural networks can well obtain regional features of images. General convolutional image detection neural network model architectures such as Xception and Efficientnet can often achieve good generalization capabilities.

[0005] 2. Spatial attention mechanism. Spatial attention mechanism is a method in deep learning that is used to increase the attention of the neural network to a specific input data area. This method generates an attention map on the input feature map, allowing the network to pay more attention to the spatial locations that are useful for the task, which can improve the network's detection performance in most scenarios.

[0006] 3. Data enhancement. Before the network obtains data, the input data is enhanced by adjusting brightness, contrast, blurring, rotating, translating, adding noise, etc., which can effectively improve the robustness and generalization of model detection.

[0007] Multi-dimensional feature extraction. The shallow features obtained by the neural network often represent the texture detail features of the image, while the deep features often represent the high-dimensional semantic features of the image. The detection network can obtain features of different dimensions of the image to assist in detection, such as extracting texture / frequency domain features separately to improve the detection performance of the model.

[0008] The prior art has the following problems:

[0009] 1. Poor cross-domain detection performance. Even though there are many methods to improve the generalization of the model, the performance of the detection network is often very poor when facing cross-domain data detection. In practical applications, the AUC of the detector on the training set can reach more than 99%, but on the cross-domain data set, the AUC of the best detector currently can hardly reach 85%. This is the main problem of existing detectors, and the technical solution of the present invention can solve this technical problem.

[0010] 2. Poor robustness. In actual detection scenarios, data may have been compressed, denoised, blurred, cropped, resized, and other processes, which will cause some interference to the data. When the detection network faces interfering data, the detection performance is often poor, that is, the network has poor robustness.

[0011] 3. Poor anti-escape and anti-counterfeiting capabilities. In order to evade detection, many algorithms will generate some malicious samples to interfere with detection. Common methods include reconstruction-based escape algorithms and noise-based adversarial algorithms. The detector's detection capability for these maliciously processed forged samples is often poor.

[0012] Difficulty in selecting detectors. Different detection algorithms often detect different counterfeit features, and their performance on different test data varies greatly. For specific detection problems, how to select a good detection algorithm for detection often requires actual comparative testing, which is costly. Summary of the invention

[0013] The purpose of the present invention is to provide a deep fake detection method based on irrelevant feature elimination and associated feature enhancement in view of the shortcomings of the prior art.

[0014] The objective of the present invention is achieved through the following technical solutions:

[0015] A deep fake detection method based on irrelevant feature elimination and related feature enhancement includes the following steps:

[0016] S1: Input image A passes through the irrelevant feature elimination module to obtain the image pairs A1 and A2 before and after enhancement, which are input into the Xception detection network as a whole;

[0017] S2: The image pairs A1 and A2 are converted into shallow features F1 and F2 through the shallow network of Xception;

[0018] S3: Calculate the difference between the shallow features F1 and F2 as the difference loss L1;

[0019] S4: Calculate the landmark of image A1 and divide image A1 into blocks according to the landmark;

[0020] S5: Input the divided image into the shallow network of Xception to obtain multiple feature maps M i ;

[0021] S6: Combine the shallow feature F1 with the feature map M i The associated feature enhancement module is input together to enhance the shallow texture feature through the feature map to obtain the enhanced texture feature F3;

[0022] S7: Input the shallow feature F1 into the deep network of Xception to obtain the deep feature D;

[0023] S8: Concatenate the texture feature F3 and the deep feature D through a fully connected layer, and then obtain the classification prediction result through the fully connected layer;

[0024] S9: Calculate the cross entropy between the prediction result and the label to obtain the loss L2;

[0025] S10: Weight the losses L1 and L2, calculate the final loss L, and use L to optimize the network parameters to achieve network training.

[0026] Furthermore, in the step S1, the irrelevant feature elimination module is used to adjust the brightness, contrast and saturation of the image.

[0027] Furthermore, in step S2, the shallow network is the Entry Flow part of Xception, which is used to convert the input image into shallow features.

[0028] Furthermore, in step S3, for features F1 and F2, MSE is used to calculate the loss; its loss function L1 is:

[0029]

[0030] Among them, yi and Represent the elements in F1 and F2 respectively, and n represents the total number of elements in F1 and F2.

[0031] Furthermore, the step S4 is specifically as follows: extracting the landmark of the image A1, and then dividing the image into blocks according to the landmark, and dividing the input face into outer face, upper inner face, and lower inner face images.

[0032] Furthermore, in step S5, the obtained outer face, upper inner face, and lower inner face images are respectively input into the shallow network to obtain the regional shallow feature maps M1, M2, and M3 for subsequent associated feature enhancement; the data is converted into:

[0033] [M1, M2, M3]=NET2(PART(I0))=NET2[S a , S b , S c ]

[0034] Where S a , S b , S c They are the outer face, the upper inner face, and the lower inner face images respectively. PART represents the landmark-based block operation. i (i∈[1,2,3]) is the output regional feature map, and NET2 is a shallow network.

[0035] Furthermore, in step S6, for the feature map M i Compression and convolution operations are performed to generate attention weights, which are then multiplied with the original feature matrix to enhance the extracted features. The calculation formula is as follows:

[0036] F att =σ(W2δ(W1(M i ))), where F att ∈R c

[0037] Among them, M i is the feature map obtained, σ represents the sigmoid activation function, δ represents the ReLU activation function, and are the weight matrices of the two fully connected layers, and r is the scaling factor, which is used to reduce the number of parameters and calculations;

[0038] Get F att Finally, the original features are weighted to achieve the effect of attention enhancement:

[0039] F′=F1⊙F att

[0040] F′ is the feature after regional attention processing, through F1 and F att Get by element-by-element multiplication; F attis the regional attention matrix.

[0041] Furthermore, in step S7, the deep network is the Middle flow part of Xception, which is used to extract deep semantic information.

[0042] Furthermore, in step S9, the cross entropy loss L2 is calculated by the prediction result P and the label label; the loss function is as follows:

[0043]

[0044] Where N is the number of samples, y i ∈[0, 1] is the true or false label of the image, p i is the predicted result; through y i With p i The difference between the predicted distribution and the true distribution is calculated as the loss to optimize network training.

[0045] The beneficial effects of the present invention are:

[0046] 1. Improved generalization ability. Generally, the generalization ability of deep fake detectors is weak. By adding an irrelevant feature removal module and a related feature enhancement module, the present invention can effectively improve the generalization detection ability of the detector and improve the detection performance during cross-domain data testing.

[0047] 2. Improved detection accuracy. By modifying the traditional spatial attention mechanism into a regional attention mechanism based on landmark blocks, the present invention can better extract specific forgery traces in different regions, enhance the network's attention to image details, and improve detection performance.

[0048] 3. Migrateable modules. Most detection networks detect specific forgery traces and cannot be migrated well. The irrelevant feature removal module and the associated feature enhancement module in the present invention detect forgery traces based on generalization. At the same time, their own structure is relatively separated from the original network structure and can be migrated to different detection networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a model framework diagram of the present invention;

[0050] Figure 2 This is a schematic diagram of traditional spatial attention. DETAILED DESCRIPTION

[0051] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0052] Based on the problems of weak generalization and poor cross-domain capabilities of conventional detection technologies, the present invention proposes a generalized detection network based on regional attention mechanism and irrelevant feature elimination. The technical innovations are mainly the following three points. 1: A new method for eliminating irrelevant features is proposed. 2: Combining the attention mechanism with landmark-based image segmentation preprocessing can generate attention maps for different areas of the image. 3: As a general processing flow of the detection algorithm, it can be easily added to different detection networks and has strong portability.

[0053] Terminology explanation:

[0054] 1. Attention mechanism: divided into self-attention mechanism, spatial attention mechanism, channel attention mechanism, etc. Based on the spatial attention mechanism, the present invention replaces the original features with the features extracted by the network according to the regional block image, generates an attention map (i.e., weight), and then weights it to the original features to achieve the effect of enhancing the associated features.

[0055] 2. Elimination of irrelevant features: Image features contain attribute features that are irrelevant to the forgery traces and the semantics of the image itself, including brightness, saturation, contrast, etc., which become irrelevant features (of forgery traces). By eliminating irrelevant features, the correlation between the extracted features and the forgery traces is enhanced.

[0056] 3. Enhancement of related features: On a forged face image, different regions may have different corresponding forged traces. The extracted forged traces are divided into regions for enhancement to enhance the correlation between different forged traces and corresponding regions.

[0057] 4. Landmark: refers to a set of key points marked on a face image to describe the location and shape of facial features.

[0058] 5. Xception: A general image feature extraction neural network used as the framework basis for the detection network. e flo w Represents the shallow and middle layers of the network.

[0059] 6. Shallow features and deep features: As the number of neural network layers increases, the extracted features will gradually go from shallow to deep. Shallow features are the features of the data output after passing through the first few layers of the network, which often represent image texture information, while deep features are the features of the data output after passing through more network layers, which often represent image semantic information.

[0060] 7. Feature map: A tensor used to express image features in the attention mechanism. It is often extracted through the network and can enhance the original features.

[0061] The core of this method is to use irrelevant feature elimination and associated feature enhancement modules to extract generalized forgery features for detection and improve the cross-domain detection capability of the model. The skeleton network of the entire detection network uses Xception, and extracts features through dual channels of shallow texture features and deep semantic features to obtain image texture features and semantic features respectively. After the irrelevant feature elimination and associated feature enhancement modules, the features are enhanced to finally obtain the detection results. It should be noted that the irrelevant feature elimination and associated feature enhancement modules are highly portable, and different skeleton networks can be selected for authenticity detection in actual use.

[0062] The present invention provides a deep fake detection method based on irrelevant feature elimination and related feature enhancement. The overall data flow is as follows:

[0063] S1: Input image A passes through the irrelevant feature elimination module to obtain the image pairs A1 and A2 before and after enhancement, which are input into the detection network (Xception network) as a whole. Among them, the shallow network and the deep network are the first and middle layers of the detection network, namely Entry Flow and Middle Flow, respectively, and the last few layers of the Xception network are used to generate output.

[0064] S2: The image pairs A1 and A2 are converted into shallow features F1 and F2 through the shallow network of Xception.

[0065] S3: Calculate the difference between the shallow features F1 and F2 as the difference loss L1.

[0066] S4: Calculate a landmark for the image A1 and divide the image A1 into blocks according to the landmark.

[0067] S5: Input the divided image into the shallow network of Xception to obtain multiple feature maps M i .

[0068] S6: Combine the shallow features F1 obtained above with the feature map M i The associated feature enhancement module is input together to enhance the shallow texture features through the feature map to obtain the enhanced texture features F3.

[0069] S7: Continue to input the shallow feature F1 into the deep network of Xception to obtain the deep feature D.

[0070] S8: Concatenate the texture feature F3 and the deep feature D through a fully connected layer, and finally obtain the classification prediction result through another fully connected layer.

[0071] S9: Calculate the cross entropy between the classification result and the label to obtain the loss L2.

[0072] S10: Weight the losses L1 and L2, calculate the final loss L, and use L to optimize the network parameters to achieve network training.

[0073] The corresponding model framework is as follows Figure 1 As shown. The following is a detailed analysis of the functions of each part:

[0074] Input: image to be detected (preprocessed by resize).

[0075] S1: irrelevant feature elimination module. A forged image contains the semantic features of the image itself, the trace features introduced by the forgery algorithm, and some forgery irrelevant features such as image brightness and contrast. These irrelevant features will interfere with the detection performance. In extreme cases, when the brightness difference between the real and fake images in the training data set is very large, the detector is very likely to mistakenly use brightness as the classification criterion. The function of this module is to eliminate the interference of brightness, contrast, and saturation on the extraction of forged features during image detection. The specific steps are as follows. Adjust the brightness, contrast, and saturation of the image, and combine the images before and after processing into image pairs and input them into the shallow network in S2. The data is converted into:

[0076] I1=G(I0,i,t), Input1=[I0,I1]

[0077] Where I0 is the input image, I1 is the enhanced image, G is the random enhancement process, i∈[1,3], t∈[0.7,1.3] are random parameters that determine the type of enhancement and the degree of change before and after the process. Input1 is the processed image pair, which is input into the shallow network in S2.

[0078] S2: Shallow network. The Entry Flow part of Xception is actually taken as the shallow network, which is used to convert the input tensor into shallow features. Among them, Xception is a general image classification detection network. This technology is the backbone of the detection network. Each module (irrelevant feature elimination and related feature enhancement module) is added to the network for common use; EntryFlow is the input flow, which is a general term for the first few layers of the Xception network.

[0079] For images, shallow features generally correspond to the features of the image texture details. This part will process two inputs. The image pair input Input1 is converted into shallow features F1 and F2. The image after landmark segmentation will be processed into regional shallow features and input into S5 to calculate the regional feature map of the image. Among them, landmark refers to a set of key feature points marked on the face image, which is used to describe the position and shape of facial features and other features.

[0080] The data is converted to:

[0081] [F1, F2] = NET2 (Input1) = NET2 [I0, I1]

[0082] NET2 is the shallow network in S2.

[0083] S3: Calculate the difference loss L1. Since the image difference between F1 and F2 is only in the features irrelevant to forgery such as brightness, contrast, saturation, etc., F1 and F2 should be as similar as possible, so as to ensure that the extracted features are irrelevant to the difference (i.e., irrelevant features such as brightness, contrast, saturation, etc.), thereby achieving the effect of eliminating irrelevant features. The specific method is to use MSE (Mean square error) to calculate the loss of the input feature tensors F1 and F2. The loss function is:

[0084]

[0085] Where n represents the total number of elements in F1 and F2, and yi is Represent the elements in tensors F1 and F2 respectively.

[0086] S4: Obtaining regional block images. Since the forged traces in different regions of the forged face may be different, the traditional spatial attention mechanism, that is, the method of directly using all facial features to generate the attention mechanism weight matrix, will result in the features of different regions being averaged due to the pooling operation, and the regional specific forged traces cannot be well enhanced. Based on this problem, the image can be pre-blocked using landmarks, and features can be extracted from the blocked images to obtain regional feature maps for subsequent correlation feature enhancement of the features directly extracted from the image. The specific approach is as follows: First, extract the landmark of image A1. Here, the toolkit dlib (a library that can be used to calculate image landmarks) is used to extract feature points. After obtaining the image landmark, the image is blocked according to the landmark, and the input face is divided into three parts: the outer face, the upper part of the inner face, and the lower part of the inner face.

[0087] S5: Calculation of regional feature maps. The network structure is the same as S2. The three parts of the image are input into the shallow network of S2 respectively to obtain the regional shallow feature maps M1, M2, and M3 for subsequent correlation feature enhancement. The data is converted into:

[0088] [M1, M2, M3]=NET2(PART(I0))=NET2[S a , S b , S c ]

[0089] Where S a , S b , S cis the block image after S4, PART represents the landmark-based block operation in S4, M i (i∈[1,2,3]) is the output regional feature map.

[0090] S6: Correlation feature enhancement module. The spatial attention mechanism is used on the shallow feature F1 of the input to enhance the traces of forgery in the feature. The spatial attention mechanism is a technique used in deep learning models that improves the performance of the model by focusing on important spatial locations in the input feature map. Traditional spatial attention Figure 2 As shown:

[0091] In this structure, the input features are compressed and a convolution or full connection is performed, and the weights can be learned autonomously to weight different channel dimensions, thereby highlighting the features that have a greater impact on the results. The feature weights are calculated through convolution and finally multiplied with the original features.

[0092] As mentioned above, the traditional spatial attention mechanism uses global pooling, which causes the features of different regions to be averaged due to the pooling operation, and cannot enhance the specific forgery traces of the region well. The corresponding improvement is to use regional feature maps for enhancement. Since different regions are explicitly separated according to landmarks, the forgery traces of different regions can be better preserved. The specific approach is as follows: For the shallow features extracted after block division (i.e., the feature map M obtained by S5), i ) is compressed and convolved to generate attention weights, and finally multiplied with the original feature matrix to enhance the extracted features. The corresponding calculation formula is as follows:

[0093] F att =σ(W2δ(W1(M i ))), where F att ∈R C

[0094] Among them, M i is the feature map obtained by S5, σ represents the sigmoid activation function, 6 represents the ReLU activation function, and are the weight matrices of the two fully connected layers, and r is the scaling factor, which is used to reduce the number of parameters and calculations. As a whole, represents the dimension of the feature tensor; represents a matrix with rows and C columns; this matrix is ​​used as a parameter of the fully connected layer (a fully connected layer is a common network layer in a neural network), which means that a tensor with a feature dimension of C is transformed into Dimension; here r and C represent parameters, which are specific numbers in actual use, such as C is 512 and r is 8; Same reason.

[0095] It should be noted that the feature map M i The image is obtained by passing through the landmark extraction block and the shallow network, which is used to calculate the attention weight; while the shallow feature F1 in S3 is a feature matrix. The former is used to calculate the attention weight, and the latter, as a feature, will be multiplied with the weight calculated by the former to obtain the texture feature F3 enhanced by the attention mechanism. Both are extracted through the shallow network, but the feature map M i It needs to be segmented by landmark, but F1 does not.

[0096] Get F att Finally, the original features are weighted to achieve the effect of attention enhancement.

[0097] F′=F1⊙F att

[0098] F′ is the feature after regional attention processing (F3 in the framework diagram), which is obtained by combining F1 and F att F1 is the original feature (i.e. shallow feature), F att is the regional attention matrix.

[0099] S7: Deep network. The middle flow part of Xception is used to extract deep semantic information. This layer consists of 8 layers of depthwise separable convolutions, each of which contains a depthwise convolution and a pointwise convolution. The depthwise convolution performs convolution operations on each channel separately, and the pointwise convolution combines information between channels through 1x1 convolution. Through this architecture, the network parameters can be effectively reduced. Through repeated depthwise convolution operations, deep semantic features can be extracted, that is, the shallow features F1 are converted into deep features D.

[0100] S8: After obtaining the deep feature D and the shallow feature F3, the above features need to be fused as the basis for true and false classification. Due to the scale difference, the features need to be converted to the same size before feature splicing and full connection classification. S8 consists of 3 layers of convolution, pooling and fully connected layers. The purpose of convolution and pooling is to further extract features while achieving dimensionality reduction and dimensionality normalization. After pooling, the dimensions of the deep feature D and the texture feature F3 become [B, N]. D and F3 are spliced, and the dimension becomes [B, 2*N]. Finally, after a fully connected layer, the binary classification prediction vector is output, that is, the prediction result P.

[0101] S9: Cross entropy loss. The cross entropy loss L2 is calculated by the predicted result P and the label label. This loss is used to constrain the predicted result to be close to the true result. Since the true or false prediction is performed, the true or false is used as the binary cross loss of the classification result. The loss function is as follows:

[0102]

[0103] N is the number of samples, y i ∈[0, 1] is the true or false label of the image, p i is the predicted result. i With p i The difference between the predicted distribution and the true distribution is calculated as the loss to optimize the network training. N is the number of samples used for training. In actual use, batches are divided. Each training will send the corresponding number of samples to the model training according to the batch size. In this case, N is the batch size.

[0104] S10: Calculation of mixed loss. By weighting the difference loss L1 and the cross entropy loss L2, the mixed loss L is obtained, which is used as the final loss of network training. The loss function is as follows:

[0105] L=αL1+(1-α)L2

[0106] Where α is the weighting coefficient. After obtaining the mixed loss L, the entire network can be trained using L.

[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0108] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A deep fake detection method based on irrelevant feature elimination and related feature enhancement, It is characterized in that The following steps are involved: S1: Input image A passes through the irrelevant feature elimination module to obtain the image pair A before and after enhancement 1 , A 2 , as a whole, input into the Xception detection network; S2: The image pair A is transformed through the shallow network of Xception 1 , A 2 Transformed into shallow features F 1 、F 2 ; S3: For shallow features F 1 、F 2 Calculate the difference as the difference loss L 1 ; S4: for image A 1 Calculate the landmark and map image A based on the landmark 1 Divide into blocks; S5: Input the divided image into the shallow network of Xception to obtain multiple feature maps M i ; S6: The shallow feature F 1 With feature map M i The associated feature enhancement module is input together to enhance the shallow texture features through the feature map to obtain the enhanced texture features F 3 ; S7: The shallow feature F 1 Input the deep network of Xception to obtain the deep feature D; S8: The texture feature F 3 It is concatenated with the deep feature D through a fully connected layer, and then the classification prediction result is obtained through the fully connected layer; S9: Calculate the cross entropy between the prediction result and the label to obtain the loss L 2 ; S10: will lose L 1 With L 2 Weighted, calculate the final loss L, and use L to optimize the network parameters to achieve network training.

2. The deep fake detection method based on irrelevant feature elimination and associated feature enhancement according to claim 1, It is characterized in that In the step S1, the irrelevant feature elimination module is used to adjust the brightness, contrast and saturation of the image.

3. The deep fake detection method based on irrelevant feature elimination and associated feature enhancement according to claim 1, It is characterized in that In step S2, the shallow network is the Entry Flow part of Xception, which is used to convert the input image into shallow features.

4. The deep fake detection method based on irrelevant feature elimination and associated feature enhancement according to claim 1, It is characterized in that In step S3, for feature F 1 、F 2 , MSE is used to calculate the loss; its loss function L 1 for: L1=MSE(F 1 ,F 2 ), Among them, y i and Respectively represent F 1 、F 2 The elements in F 1 、F 2 The total number of elements in .

5. The deep fake detection method based on irrelevant feature elimination and associated feature enhancement according to claim 1, It is characterized in that The step S4 specifically includes: extracting image A 1 The landmark is then divided into blocks according to the landmark, and the input face is divided into outer face, upper inner face, and lower inner face images.

6. The deep fake detection method based on irrelevant feature elimination and associated feature enhancement according to claim 5, It is characterized in that In step S5, the obtained outer face, upper inner face, and lower inner face images are respectively input into the shallow network to obtain the regional shallow feature map M 1 ,M 2 ,M 3 , used for subsequent correlation feature enhancement; The data is transformed into: [M 1 ,M 2 ,M 3 ]=NET 2 (PART(I0))=NET 2 [S a ,S b ,S c ] Where S a ,S b ,S c They are the outer face, the upper inner face, and the lower inner face images respectively. PART represents the landmark-based block operation. i (i∈[1,2,3]) is the output regional feature map, NET 2 It is a shallow network.

7. The deep fake detection method based on irrelevant feature elimination and associated feature enhancement according to claim 6, It is characterized in that In step S6, for the feature map M i Perform compression and convolution operations to generate attention weights, and finally multiply them with the original feature matrix to enhance the extracted features; The calculation formula is as follows: F att =σ(W 2 δ(W 1 (M i ))), among themF att ∈R C Among them, M i is the feature map obtained, σ represents the sigmoid activation function, δ represents the ReLU activation function, and are the weight matrices of the two fully connected layers, and r is the scaling factor, which is used to reduce the number of parameters and calculations; Get F att Finally, the original features are weighted to achieve the effect of attention enhancement: F ′ =F 1 ⊙F att F ′ is the feature after regional attention processing, through F 1 With F att Get by element-by-element multiplication; F att is the regional attention matrix.

8. The deep fake detection method based on irrelevant feature elimination and associated feature enhancement according to claim 1, It is characterized in that In step S7, the deep network is the Middle flow part of Xception, which is used to extract deep semantic information.

9. The deep fake detection method based on irrelevant feature elimination and associated feature enhancement according to claim 1, It is characterized in that In step S9, the cross entropy loss L2 is calculated by the prediction result P and the label label; the loss function is as follows: Where N is the number of samples, y i ∈[0,1] is the true or false label of the image, p i is the predicted result; through y i With p i The difference between the predicted distribution and the true distribution is calculated as the loss to optimize network training.

Images