Deep forgery detection method based on space-frequency feature integration and dynamic edge optimization

Abstract

The invention discloses a depth forgery detection method based on space-frequency feature integration and dynamic edge optimization. The detection accuracy and robustness of forgery content are remarkably improved through a space-frequency feature integration module and an authenticity perception boundary loss function. According to the method, the spatial domain and frequency domain characteristics of the image are effectively combined, so that the model can more comprehensively identify the nuance in the forged image. Besides, the authenticity perception boundary loss function successfully solves the problem of class imbalance by dynamically adjusting the boundary, and the classification performance is remarkably improved especially under the condition that real image samples are scarce. The effectiveness of the method is verified by experiments performed on a plurality of challenging data sets.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep fake detection, and specifically relates to a design of a deep fake detection method based on space-frequency feature integration and dynamic edge optimization. Background Art

[0002] Breakthroughs in deep learning technology have significantly expanded the boundaries of digital content creation, and deep generative models have become the core of this field. However, the double-edged sword effect of technology has also led to the proliferation of fake facial images and videos. These highly realistic contents spread rapidly in cyberspace, which may not only mislead the public, but also threaten the social trust system in terms of synthesizing false news. With the advancement of deep fake technology, its application in invading privacy and tampering with speech has posed a severe challenge to the authenticity of information. In response to this problem, deep fake detection technology has emerged.

[0003] Deep fake detection technology is constantly innovating with the rapid development of generative models. The significant breakthroughs in the field of image generation by generative adversarial networks (GANs) have provided new ideas for deep learning detection algorithms, enabling them to capture the unique features of GAN-generated images, thereby improving the accuracy of detection. However, new generation techniques such as diffusion models pose new challenges to existing detection techniques with their higher fidelity. These advances have made fake images more delicate, further weakening the effectiveness of traditional detection algorithms.

[0004] In the field of deep fake detection, early methods mainly rely on convolutional neural networks (CNNs) to perform binary classification of facial images through existing network architectures. These methods can effectively identify local features, but they only focus on the local information of the image and fail to deeply capture the complexity of the forged image, resulting in limited detection results. To address this problem, recent research methods have begun to introduce new detection ideas, focusing on identifying specific patterns in forged images, such as noise features, local textures, and frequency information. Although these improvements have improved detection accuracy in some cases, there are still limitations, especially when dealing with unknown forgery techniques that are not included in the training set, the effect is significantly reduced.

[0005] At present, there are endless deep fake detection methods. LSDA adopts a teacher-student model architecture. The teacher module learns the features of a specific field and is trained through domain loss; the student module extracts features from the teacher to enhance generalization ability. However, LSDA has high requirements for the diversity of training data, and the generalization effect will be limited when the data is insufficient. UCF decouples image features through a multi-task learning strategy and a conditional decoder to extract common forgery features for detection. However, UCF increases the model complexity in the feature decoupling process, and the decision process lacks transparency, making it difficult to intuitively interpret the detection results. Methods such as SRM and F3Net distinguish between forged and real images by analyzing the frequency components of the image. Although they perform well when dealing with known forgery patterns, their detection capabilities for unknown forgery patterns are still limited. In addition, CORE and RECCE improve the detection algorithm by designing specific loss functions and reconstruction learning. However, these methods are often easily interfered by irrelevant information (such as race, gender, or identity characteristics), and it is difficult to completely exclude the influence of these factors. On the other hand, models such as Meso4 and Xception, as pure convolutional network structures, fail to fully utilize data augmentation, feature decoupling or frequency information, so their performance drops significantly when processing unknown deep fake videos.

[0006] In summary, existing deepfake detection methods have the following shortcomings: (1) Although existing deepfake detection methods have made some progress, the generalization ability and detection effect of the model still need to be improved when dealing with forged patterns that have not been seen in the training set. (2) The local details of the forged image are not captured enough, and the spectral information is not fully utilized, resulting in limited generalization ability and detection accuracy when dealing with complex or unknown forged patterns. Summary of the invention

[0007] The purpose of the present invention is to solve the problem of limited generalization ability and detection accuracy of existing deep fake detection methods when dealing with complex or unknown fake patterns, and proposes a deep fake detection method based on space-frequency feature integration and dynamic edge optimization.

[0008] The technical solution of the present invention is: a deep fake detection method based on space-frequency feature integration and dynamic edge optimization, comprising the following steps:

[0009] S1. Obtain the dataset required for deep fake detection and divide the dataset into training set and test set.

[0010] S2. Preprocess and enhance the data on the training set and the test set respectively to obtain the processed training set and the test set.

[0011] S3. Build and initialize a deep fake detection network based on spatial frequency feature integration and dynamic edge optimization.

[0012] S4. Input the processed training set into the deep fake detection network, train the deep fake detection network, and obtain the output probability of the deep fake detection network.

[0013] S5. Construct an authenticity-aware boundary loss function based on the output probability of the deep fake detection network, and optimize the deep fake detection network according to the authenticity-aware boundary loss function.

[0014] S6. Repeat steps S4 to S5 until a preset number of iterations is reached. At the end of each iteration, the processed test set is input into the trained deep fake detection network for testing, and the test indicators of the current deep fake detection network are calculated.

[0015] S7. Input the processed test set into the deep fake detection network with the highest test index value, and output the deep fake detection result.

[0016] Furthermore, the data set in step S1 includes FaceForensics++, CelebDF-v1, CelebDF-v2 and DFDCP, where FaceForensics++ is used as a training set, and CelebDF-v1, CelebDF-v2 and DFDCP are used as test sets.

[0017] Furthermore, the preprocessing in step S2 is specifically as follows: extracting frames for the training set and the test set respectively, detecting faces in each video frame using the Dlib face detection algorithm, aligning and cropping faces according to the detected facial landmarks using the Dlib shape predictor model, and saving the processed face images in a separate folder.

[0018] Furthermore, the deep fake detection network based on spatial-frequency feature integration and dynamic edge optimization in step S3 includes a spatial-frequency feature integration module SFFI, a pre-trained Xception network and a fully connected layer connected in sequence.

[0019] Further, step S4 includes the following sub-steps:

[0020] S41, input the processed training set into the space-frequency feature integration module SFFI, fuse the space domain and frequency domain features of the image, and output a space-frequency fusion feature map.

[0021] S42, input the space-frequency fusion feature map into the pre-trained Xception network for in-depth feature extraction, and output a fine feature map .

[0022] S43, fine feature map The input is processed in the fully connected layer to obtain the output probability of the deep fake detection network.

[0023] Further, step S41 includes the following sub-steps:

[0024] S411, input the processed training set into the space-frequency feature integration module SFFI, calculate the pixel-by-pixel mean of the three channels of the input image, and obtain a single-channel grayscale image.

[0025] S412, performing fast Fourier transform on the grayscale image to obtain a spectrum image.

[0026] S413, performing frequency domain offset and amplitude calculation on the spectrum graph, and using logarithmic transformation to compress the dynamic range of the spectrum graph to obtain a compressed spectrum graph.

[0027] S414, normalize the compressed spectrum to obtain a normalized spectrum .

[0028] S415, normalize the spectrum It is concatenated with the input image according to the channel dimension to obtain a space-frequency fusion feature map.

[0029] Furthermore, the authenticity-aware boundary loss function constructed in step S5 is:

[0030]

[0031] in represents the authenticity-aware boundary loss function, represents the batch size, represents the number of categories, represents the output probability of the i-th sample in the j-th category, represents the true category of the i-th sample, Indicates that the i-th sample is in the true category The output probability on Represents the true category The borders of Represents the scaling factor.

[0032] Furthermore, the test indicators in step S6 include accuracy ACC, area under the curve AUC and average precision AP.

[0033] The beneficial effects of the present invention are:

[0034] (1) The present invention provides a deep fake detection method based on spatial frequency features and dynamic edge optimization to achieve deep fake detection in unknown environments with high generalization ability and detection accuracy.

[0035] (2) The present invention proposes a space-frequency feature integration module SFFI, which combines the original color image with the frequency domain information, enabling the model to capture richer and more useful feature information in the feature extraction stage.

[0036] (3) The present invention proposes a real-world-aware boundary loss function, which successfully addresses the problem of class imbalance by dynamically adjusting the boundary, and significantly improves the classification performance, especially when real image samples are scarce. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Shown is a flow chart of a deep counterfeit detection method based on space-frequency feature integration and dynamic edge optimization provided by an embodiment of the present invention.

[0038] Figure 2 Shown is a schematic diagram of the deep fake detection network structure based on spatial frequency feature integration and dynamic edge optimization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] Now, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the accompanying drawings are only exemplary and are intended to explain the principles and spirit of the present invention, rather than to limit the scope of the present invention.

[0040] The embodiment of the present invention provides a deep fake detection method based on space-frequency feature integration and dynamic edge optimization, such as Figure 1 As shown, the following steps S1 to S7 are included:

[0041] S1. Obtain the dataset required for deep fake detection and divide the dataset into training set and test set.

[0042] In an embodiment of the present invention, the data sets required for deep fake detection include FaceForensics++, CelebDF-v1, CelebDF-v2 and DFDCP, among which FaceForensics++ is used as a training set, and CelebDF-v1, CelebDF-v2 and DFDCP are used as test sets.

[0043] In the embodiment of the present invention, the divided data sets are downloaded from the paper of FaceForensics++ and the official website of Celeb-DF, and the source data sets are all videos ranging from 10 seconds to 30 seconds.

[0044] S2. Preprocess and enhance the data on the training set and the test set respectively to obtain the processed training set and the test set.

[0045] In an embodiment of the present invention, the specific method of preprocessing is: extract frames for the training set and the test set respectively, and use the Dlib face detection algorithm to detect the face in each video frame, use the Dlib shape predictor model to align and crop the face according to the detected facial landmarks, and save the processed face images in a separate folder.

[0046] In the embodiment of the present invention, the data enhancement processing includes methods such as flip_prob, rotate_prob, rotate_limit, blur_prob, blur_limit, brightness_prob, brightness_limit, contrast_limit, quality_lower and quality_upper.

[0047] S3. Build and initialize a deep fake detection network based on spatial frequency feature integration and dynamic edge optimization.

[0048] In the embodiment of the present invention, Figure 2 As shown in the figure, the deep fake detection network based on spatial frequency feature integration and dynamic edge optimization includes a spatial-frequency feature integration module SFFI, a pre-trained Xception network and a fully connected layer connected in sequence. The spatial-frequency feature integration module SFFI is to fuse the spatial domain and frequency domain features of the image so that the model can more comprehensively identify the subtle differences in the fake image. The pre-trained Xception network is used to perform feature extraction tasks. The network can capture rich feature representations from the input image, which are then passed to the fully connected layer. In the fully connected layer, through a series of complex calculations, a logits (probability) is finally output to predict the authenticity of the input image.

[0049] S4. Input the processed training set into the deep fake detection network, train the deep fake detection network, and obtain the output probability of the deep fake detection network.

[0050] like Figure 2 As shown, step S4 includes the following sub-steps S41 to S43:

[0051] S41, input the processed training set into the space-frequency feature integration module SFFI, fuse the space domain and frequency domain features of the image, and output a space-frequency fusion feature map.

[0052] like Figure 2 As shown, step S41 includes the following sub-steps S411 to S415:

[0053] S411, input the processed training set into the space-frequency feature integration module SFFI, calculate the pixel-by-pixel mean of the three channels of the input image, and obtain a single-channel grayscale image.

[0054] S412, performing fast Fourier transform on the grayscale image to obtain a spectrum image, which can extract texture information and forgery traces under different frequency components.

[0055] S413, performing frequency domain shift (through centering) and amplitude calculation on the spectrum graph, and using logarithmic transformation to compress the dynamic range of the spectrum graph to obtain a compressed spectrum graph, thereby enhancing the visibility of tiny features.

[0056] S414, normalize the compressed spectrum to obtain a normalized spectrum , ensuring that the numerical range of the spectrum data meets the needs of subsequent calculations.

[0057] S415, normalize the spectrum It is concatenated with the input image according to the channel dimension to obtain a space-frequency fusion feature map.

[0058] S42, input the space-frequency fusion feature map into the pre-trained Xception network for in-depth feature extraction, and output a fine feature map .

[0059] like Figure 2 As shown, the fusion feature map First, it is sent to the encoder part of Xception. In the encoder, the fused feature map undergoes a series of convolution and pooling operations, gradually abstracting and compressing to extract the deep features of the image. Subsequently, these features are sent to the middle layer of the network (composed of several RELU layers and depth-separable convolution layers), and the expressive power of the features is further refined and enhanced through an iterative process. Finally, after the reverse operation of the decoder, these deep features are mapped back to the spatial dimensions of the original feature map, thereby obtaining a refined feature map. .

[0060] S43, fine feature map The input is processed in the fully connected layer to obtain the output probability of the deep fake detection network.

[0061] The output probability logits represents the possibility that the input image belongs to the real category or the forged category. Ultimately, this probability value is used as the basis for judging the authenticity of the image, achieving efficient identification of the authenticity of the image.

[0062] S5. Construct an authenticity-aware boundary loss function based on the output probability of the deep fake detection network, and optimize the deep fake detection network according to the authenticity-aware boundary loss function.

[0063] Class imbalance is a common challenge in image authenticity detection tasks, especially when fake images tend to constitute a large number of samples while real images are relatively scarce. By dynamically adjusting the margin to enhance the model's ability to discriminate real images, this significantly improves the classification performance, especially in severe data imbalance situations.

[0064] Based on this, the authenticity perception boundary loss function constructed in the embodiment of the present invention is:

[0065]

[0066] in represents the authenticity-aware boundary loss function, represents the batch size, represents the number of categories, represents the output probability of the i-th sample in the j-th category, represents the true category of the i-th sample, Indicates that the i-th sample is in the true category The output probability on Represents the true category The boundary of ,in represents the boundary of the jth category, Represents the true category The number of samples, represents the maximum boundary value, Represents the scaling factor.

[0067] In the embodiment of the present invention, the dynamic boundary of the category is calculated according to the number of samples in each category, and the weight is adjusted by the inverse square root of the category frequency to generate the corresponding dynamic boundary to solve the problem of category imbalance. At the same time, the scaling factor s is introduced to amplify the output of the loss function so that the model pays more attention to those samples that are difficult to classify during the training process. The output after dynamic weight adjustment is combined with the cross entropy loss function to improve the classification performance of the model for unbalanced data sets.

[0068] S6. Repeat steps S4 to S5 until a preset number of iterations (10 times in the embodiment of the present invention) is reached. At the end of each iteration, the processed test set is input into the trained deep fake detection network for testing, and the test indicators of the current deep fake detection network are calculated.

[0069] In the embodiment of the present invention, the test indicators include accuracy ACC, area under the curve AUC and average precision AP.

[0070] S7. Input the processed test set into the deep fake detection network with the highest test index value, and output the deep fake detection result.

[0071] To further illustrate the effectiveness of the proposed method, the proposed method is compared with other existing methods. In order to make a fair comparison, the officially released codes of other methods are used and their experimental settings are followed, where all methods are implemented in the same computing environment and quantitative and qualitative analysis is performed simultaneously. The eight methods are as follows: Method 1 is the Meso4 method, which is an image analysis technology based on deep learning; Method 2 is Xception, which, like Meso4, is also a pure convolutional network structure that focuses on the extraction of image features; Method 3 is F3Net, which uses the frequency components of the image to distinguish between forged images and real images; Method 4 is SRM, which also uses image frequency analysis technology to enhance the accuracy of forgery detection; Method 5 is CORE, which focuses on designing specific loss functions and reconstruction learning strategies to improve detection performance; Method 6 is Recce, which is an innovative detection algorithm that improves the accuracy of forgery detection through different strategies; Method 7 is UCF, which decouples image features through multi-task learning strategies and conditional decoders to extract universal forgery features for deep forgery detection; Method 8 is LSDA, which adopts a teacher-student model architecture, in which the teacher module learns features in a specific field and is trained through domain loss; the student module extracts features from the teacher module to improve generalization ability.

[0072] Since the method of the present invention is mainly aimed at deep fake detection in unknown environments, the forgery techniques of the forged videos in the training set and the test set come from different sources. Table 1, Table 2, and Table 3 give the quantitative comparison results of the accuracy ACC, area under the curve AUC, and average precision AP of 9 different network structures on 3 data sets.

[0073] Table 1 Comparison results of different methods on CelebDF-v1

[0074]

[0075] Table 2 Comparison results of different methods on CelebDF-v2

[0076]

[0077] Table 3 Comparison results of different methods in DFDCP

[0078]

[0079] In Table 1, Table 2 and Table 3, the percentage counting method is used to expand the measurement index by 100 times. From the comparison of the results in Table 1, Table 2 and Table 3, it can be seen that the method of the present invention has higher detection accuracy and better detection coverage of the target to be detected than all other methods.

[0080] The effectiveness of each proposed module is verified through ablation experiments. The training set of the ablation experiment is FaceForensics++, and the test set is CelebDF-v1, CelebDF-v2 and DFDCP datasets.

[0081] Table 4. Comparison results of ablation experiments on CelebDF-v1

[0082]

[0083] Table 5. Comparison results of ablation experiments on CelebDF-v2

[0084]

[0085] Table 6 Comparison results of ablation experiments in DFDCP

[0086]

[0087] The quantitative results are shown in Tables 4 to 6. The proposed method selects Xception as the baseline model. When modules are added one by one on the baseline, it is verified that the proposed method has obvious improvements. In addition, the effect of adding a single module on the baseline is tested. The experimental results show that adding any module improves the detection performance, which proves the effectiveness of the proposed module.

[0088] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A deep fake detection method based on space-frequency feature integration and dynamic edge optimization, characterized in that: The following steps are involved: S1. Obtain the data set required for deep fake detection and divide the data set into a training set and a test set; S2, preprocessing and data enhancement processing are performed on the training set and the test set respectively to obtain the processed training set and the test set; S3. Build and initialize a deep fake detection network based on spatial frequency feature integration and dynamic edge optimization; S4. Input the processed training set into the deep fake detection network, train the deep fake detection network, and obtain the output probability of the deep fake detection network; S5. Constructing an authenticity-aware boundary loss function based on the output probability of the deep fake detection network, and optimizing the deep fake detection network according to the authenticity-aware boundary loss function; S6. Repeat steps S4 to S5 until a preset number of iterations is reached. At the end of each iteration, the processed test set is input into the trained deep fake detection network for testing, and the test index of the current deep fake detection network is calculated. S7. Input the processed test set into the deep fake detection network with the highest test index value, and output the deep fake detection result.

2. The deep fake detection method based on space-frequency feature integration and dynamic edge optimization according to claim 1 is characterized in that: The data sets in step S1 include FaceForensics++, CelebDF-v1, CelebDF-v2 and DFDCP, wherein FaceForensics++ is used as a training set, and CelebDF-v1, CelebDF-v2 and DFDCP are used as test sets.

3. The deep fake detection method based on space-frequency feature integration and dynamic edge optimization according to claim 1 is characterized in that: The preprocessing in step S2 is specifically as follows: extracting frames for the training set and the test set respectively, detecting faces in each video frame using the Dlib face detection algorithm, aligning and cropping faces according to detected facial landmarks using the Dlib shape predictor model, and saving the processed face images in a separate folder.

4. The deep fake detection method based on space-frequency feature integration and dynamic edge optimization according to claim 1 is characterized in that: The deep fake detection network based on spatial-frequency feature integration and dynamic edge optimization in step S3 includes a spatial-frequency feature integration module SFFI, a pre-trained Xception network and a fully connected layer connected in sequence.

5. The deep fake detection method based on space-frequency feature integration and dynamic edge optimization according to claim 4 is characterized in that: The step S4 comprises the following sub-steps: S41, inputting the processed training set into the space-frequency feature integration module SFFI, fusing the space domain and frequency domain features of the image, and outputting a space-frequency fusion feature map; S42, input the space-frequency fusion feature map into the pre-trained Xception network for in-depth feature extraction, and output a fine feature map ; S43, fine feature map The input is processed in the fully connected layer to obtain the output probability of the deep fake detection network.

6. The deep fake detection method based on space-frequency feature integration and dynamic edge optimization according to claim 5 is characterized in that: The step S41 includes the following sub-steps: S411, input the processed training set into the space-frequency feature integration module SFFI, calculate the pixel-by-pixel mean of the three channels of the input image, and obtain a single-channel grayscale image; S412, performing fast Fourier transform on the grayscale image to obtain a spectrum image; S413, performing frequency domain offset and amplitude calculation on the spectrum graph, and compressing the dynamic range of the spectrum graph by logarithmic transformation to obtain a compressed spectrum graph; S414, normalize the compressed spectrum to obtain a normalized spectrum ; S415, normalize the spectrum It is concatenated with the input image according to the channel dimension to obtain a space-frequency fusion feature map.

7. The deep fake detection method based on space-frequency feature integration and dynamic edge optimization according to claim 1 is characterized in that: The authenticity perception boundary loss function constructed in step S5 is: ； in represents the authenticity-aware boundary loss function, represents the batch size, represents the number of categories, represents the output probability of the i-th sample in the j-th category, represents the true category of the i-th sample, Indicates that the i-th sample is in the true category The output probability on Represents the true category The borders of Represents the scaling factor.

8. The deep fake detection method based on space-frequency feature integration and dynamic edge optimization according to claim 1 is characterized in that: The test indicators in step S6 include accuracy ACC, area under the curve AUC and average precision AP.

Images