Deep fake detection method and system based on knowledge distillation of space-frequency feature fusion

Abstract

The present invention discloses a method and system for deep fake detection based on knowledge distillation of space-frequency feature fusion, comprising: constructing a teacher model and a student model under a knowledge distillation framework, wherein both the teacher model and the student model use a space-frequency fusion branch as a backbone network; training the teacher model using original images; training the student model by inputting paired original images and compressed images into a knowledge distillation framework, wherein during the training of the student model, the space-frequency features and soft labels of the teacher model and the hard labels of the student model serve as supervision of the student model; and performing deep fake detection of images using the trained student model. The present invention integrates spatial and frequency domain information within the knowledge distillation framework to achieve efficient deep fake detection, and the method has superior performance in compression scenarios.

Description

Technical Field

[0001] The present invention relates to the field of deep fake detection, and specifically to a deep fake detection method and system based on knowledge distillation of space-frequency feature fusion. Background Art

[0002] Deepfake technology is a potentially malicious facial manipulation tool. With the advancement of deep learning, the generated fake faces are becoming increasingly realistic. Once manipulated faces are maliciously disseminated, they pose a significant threat to personal information security and social stability.

[0003] Early deepfake detection methods were mostly based on handcrafted features, such as heartbeats, blinks, and lip shapes, and were classified using machine learning classifiers such as support vector machines (SVMs) and random forests (RFs). However, methods based on handcrafted features primarily focus on facial information, which is sensitive to the human eye, and may not be applicable to all datasets. This makes these detection methods susceptible to failure if the data source changes.

[0004] In recent years, deep learning methods have become the tool of choice for a variety of tasks, including image recognition, natural language processing, and speech recognition. Compared to traditional machine learning algorithms, deep learning models can automatically learn features from large amounts of data and achieve superior performance on many tasks. Current research on deepfake detection technology also focuses on deep learning methods.

[0005] However, deep feature-based methods are not perfect. Most existing detection methods perform well on high-quality images, but their performance degrades significantly when tested on compressed images. This is because during the compression process, some artifacts that could serve as detection clues are weakened or even removed. Furthermore, these models typically extract a single spatial or frequency domain feature. Single feature extraction can lead to model overfitting, resulting in poor generalization. In real-world scenarios, facial images on social platforms are mostly processed through image post-processing operations such as JPEG compression, making effective detection of these compressed images urgent. Summary of the Invention

[0006] In view of this, the present invention proposes a deep fake detection method and system based on knowledge distillation of space-frequency feature fusion, integrating spatial and frequency domain information within the knowledge distillation framework to achieve efficient deep fake detection. This method has superior performance in compression scenarios.

[0007] To this end, the present invention proposes the following technical solutions:

[0008] In one aspect, the present invention provides a deep fake detection method based on knowledge distillation and space-frequency feature fusion, comprising:

[0009] Construct a teacher model and a student model under the knowledge distillation framework. Both the teacher model and the student model use the space-frequency fusion branch as the backbone network.

[0010] Use the original images to train the teacher model;

[0011] The student model is trained by inputting paired original and compressed images into the knowledge distillation framework. During the student model training process, the spatial-frequency features and soft labels of the teacher model and the hard labels of the student model serve as supervision for the student model.

[0012] Use the trained student model to detect deepfakes in images.

[0013] Furthermore, the space-frequency fusion branch includes:

[0014] Data preprocessing module, preprocesses the input image;

[0015] Feature extraction module, which realizes spatial and frequency domain feature extraction of facial images;

[0016] Feature fusion module, which fuses spatial and frequency domain features under the guidance of attention;

[0017] The classification module inputs the fused features into the classifier to output the authenticity probability of the face image.

[0018] Furthermore, image preprocessing includes image enhancement and DCT frequency domain transformation.

[0019] Furthermore, the feature extraction module uses the entry stream and intermediate stream of Xception as the backbone network.

[0020] Furthermore, under the guidance of attention, the spatial and frequency domain features are fused, including:

[0021] Apply element-wise summation to the two feature maps to obtain a mixed feature map;

[0022] Perform channel reduction on the mixed feature map, including global average pooling and full connection operations;

[0023] Then, two fully connected layers are used to process the output of the channel-reduced fully connected operation, and the obtained features are multiplied with the corresponding spatial domain feature map and frequency domain feature map one by one, and the element-by-element summation operation of the feature map after element-by-element multiplication is performed with the original corresponding feature map;

[0024] Concatenate features from two domains along the channel direction and apply Convolution is used to fuse features.

[0025] Furthermore, the classification module uses the output stream of Xception as the backbone network.

[0026] Furthermore, both response-based knowledge and feature-based knowledge are adopted in the knowledge distillation framework.

[0027] Furthermore, during the student model training process, the loss function is:

[0028]

[0029] in, , and is a hyperparameter that controls the weights between different losses;

[0030] 、 The spatial and frequency domain feature maps before the fusion module are used as the loss of the transferred features; is the loss function used to supervise the response knowledge; is the cross entropy loss, which is used for supervision of the true value label.

[0031] Furthermore,

[0032] in When the temperature parameter is When the teacher model is in the class The softmax output value on , It means that the student model is in the class The softmax output value on ;

[0033]

[0034]

[0035] in is the logit of the teacher model, is the logit of the student model, Refers to the total number of tags.

[0036] In another aspect, the present invention further provides a deep fake detection system based on knowledge distillation and space-frequency feature fusion, comprising:

[0037] The model construction module is used to build the teacher model and student model under the knowledge distillation framework. Both the teacher model and the student model use the space-frequency fusion branch as the backbone network;

[0038] The first training module uses the original images to train the teacher model;

[0039] The second training module uses paired original and compressed images as input into the knowledge distillation framework to train the student model. During the student model training process, the spatial-frequency features and soft labels of the teacher model and the hard labels of the student model serve as supervision for the student model.

[0040] The forgery detection module uses the student model trained in the second training module to detect deep forgeries in images.

[0041] Advantages and positive effects of the present invention:

[0042] 1. The present invention extracts frequency domain and spatial domain features, then fuses them using a fusion module and inputs them into a classifier to distinguish the authenticity of the face, thus solving the problem of incomplete feature extraction.

[0043] 2. The present invention adds an attention mechanism to the fusion module to guide the student model to learn the effective information extracted by the teacher model.

[0044] 3. The present invention adds knowledge distillation to transfer the spatial and frequency domain features of the teacher model to the student model. At the same time, logits are added as soft labels to supervise the student model to improve the classification results of the student model and solve the problem that the student model is not focused on the forged area. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0046] Figure 1 It is the overall framework of the embodiment of the present invention;

[0047] Figure 2 This is the space-frequency fusion branch structure in the embodiment of the present invention;

[0048] Figure 3 is the attention fusion module in the embodiment of the present invention;

[0049] Figure 4 This is the visualization result of evaluating the embodiment of the present invention on the FaceForensics+ dataset. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0051] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products or apparatuses. (Spatial-Frequency Fusion Branch)

[0052] The present invention proposes a deep fake detection method based on knowledge distillation of spatial-frequency feature fusion, which optimizes detection performance in compression scenarios based on knowledge transfer and feature fusion. Knowledge distillation is a model compression method and a training method based on the "teacher-student network concept". Unlike pruning and quantization in model compression, knowledge distillation is to build a lightweight small model (student model) and use the supervision information of a larger model (teacher model) with better performance to train this small model in order to achieve better performance and accuracy. The supervision information output from the teacher model is called knowledge, and the process by which the student model learns to transfer the supervision information from the teacher model is called distillation. In the present invention, the teacher model is first trained using the original image, and then the student model is trained by inputting paired original images and compressed images into the knowledge distillation framework. In this process, the spatial-frequency features and soft labels of the teacher model and the hard labels of the student model serve as supervision for the student model.

[0053] like Figure 1 As shown, the knowledge distillation deep fake detection method based on space-frequency feature fusion provided by the embodiment of the present invention is divided into three steps:

[0054] S1. Training of the teacher model: The original uncompressed public dataset is used to train the teacher model. Both the teacher model and the student model use the Spatial-Frequency Fusion Branch (SFFB) as the backbone network, such as Figure 2 As shown, SFFB consists of four modules:

[0055] (1) Data preprocessing module, which preprocesses the input image, mainly including image enhancement and DCT frequency domain transformation.

[0056] Overfitting is one of the reasons for poor generalization performance of the model. In order to avoid overfitting, the RGB image is first enhanced, including image flipping, grayscale conversion, color jittering, etc. The image after data enhancement is directly used as the input of the spatial domain. In the embodiment of the present invention, the RGB image is represented as (Superscript: Indicates students, To extract frequency domain information, we first apply a DCT transform to the RGB image. The frequency domain image is then filtered through three filters at different frequency bands (low, mid, and high). Finally, the image is reconstructed using an inverse DCT transform. The frequency domain conversion process can be described as follows:

[0057]

[0058]

[0059] in, Represents filters of different frequency bands, represents an image after a single filter, Indicates splicing along the channel direction, is the final frequency domain input.

[0060] (2) Feature extraction module, which realizes the spatial domain and frequency domain feature extraction of facial images.

[0061] Will and After inputting the feature extraction module, the frequency domain feature map is obtained and spatial feature maps The entry stream and intermediate stream of Xception are selected as the backbone network for feature extraction. It is worth noting that the spatial domain stream and the frequency domain stream have the same structure but the weights are not shared.

[0062] (3) Feature fusion module, which fuses spatial and frequency domain features under the guidance of attention.

[0063] Next, the obtained spatial and frequency domain feature maps are input into the fusion module. The fusion module of the present invention is inspired by SKAttention. Figure 3 As shown, unlike SKAttention, the fusion module of the present invention consists of three parts: splitting, fusion and selection. The original “splitting” part of SKAttention is replaced by the feature extraction module. and Specifically, we first apply element-wise summation to the two feature maps to obtain a mixed feature map :

[0064]

[0065] Its purpose is to mix information from different domains. Furthermore, since the spatial and frequency domain streams have the same structure except for the data preprocessing module, the features at the same position in each feature map from different domains are correlated, which means that element-by-element summation does not interfere with the arrangement of the original features. Channel reduction is then performed, which specifically includes two steps: the first step is global average pooling (GAP), which can be expressed as:

[0066]

[0067] in express No. Feature maps The elements at express The number of feature maps in , represents the size of the feature map, Represents the output after global average pooling.

[0068] Next, we pass through the fully connected layer:

[0069]

[0070] in Indicates the fully connected operation, which enables information between different channels to interact. In order to match the size of the input feature map, the other two fully connected layers ( and ) is applied to , and multiply the obtained features with the corresponding spatial domain feature maps and frequency domain feature maps one by one. Unlike SKAttention, this invention adds the operation of summing the feature maps after element-by-element multiplication with the original corresponding feature maps element-by-element to enhance the fusion effect, and obtains and :

[0071]

[0072]

[0073] Finally, features from the two domains are concatenated along the channel direction and applied Convolution is used to fuse features. This process can be expressed as:

[0074]

[0075] in is the spatial fusion feature used for classification, express convolution.

[0076] (4) Classification module: inputs the fused features into the classifier to output the authenticity probability of the face image.

[0077] Finally, the output stream of Xception is selected as the backbone network of the classification module.

[0078] It is worth mentioning that the SFFB branch can be trained independently for deep fake detection and can achieve competitive results.

[0079] S2. Distillation training of the student model: Fix the weights of the teacher model, use paired input of original images and compressed images for training, transfer the spatial-frequency features and soft labels of the teacher model to the student model, and calculate the distillation loss and the cross-entropy loss of the student model.

[0080] S3. Model convergence and testing process: using the total loss function Supervise student model training until convergence.

[0081] Compared to traditional knowledge distillation, the knowledge distillation of this invention differs in three aspects: 1) input, 2) model size, and 3) knowledge. First, the teacher model is pre-trained on the original image. During the student model training phase, the teacher model input is the original image with fixed weights, and the student model input is the corresponding compressed image. However, in traditional knowledge distillation, the inputs of the teacher and student models are the same. Furthermore, unlike traditional knowledge distillation aimed at model compression, in this invention, both the teacher and student models use the same SFFB structure because the invention focuses on the knowledge transfer between the original and compressed images. The knowledge used for knowledge distillation is generally divided into three types: response-based knowledge, relationship-based knowledge, and feature-based knowledge. In order for the student model to simultaneously imitate the feature representation and final prediction probability of the teacher model, the present invention uses both response-based knowledge and feature-based knowledge, while traditional knowledge distillation only selects one type of knowledge for transfer.

[0082] In this paper, the spatial and frequency domain feature maps before the fusion module are used as the features of the transfer and supervised by the mean square error (MSE). The loss here can be expressed as:

[0083]

[0084]

[0085] The loss function used to supervise the response-based knowledge can be expressed as:

[0086]

[0087] in When the temperature parameter is When the teacher model is in the class The softmax output value on , It means that the student model is in the class The softmax output value on . Get and The formula can be expressed as:

[0088]

[0089]

[0090] in is the logit of the teacher model, is the logit of the student model, Refers to the total number of labels; logit represents the data of the last layer of the model in deep learning, that is, raw data, which can then be scaled by softmax or sigmoid.

[0091] In addition, the present invention uses cross entropy loss to supervise the true value label:

[0092]

[0093] in Representation Class The true value label on , Represents the prediction of the student model. Because the teacher model also has a certain error rate, The addition of can reduce the possibility of errors propagating to the student model. The total loss of the model can be expressed as:

[0094]

[0095] in , and is a hyperparameter that controls the weights between different losses.

[0096] In order to verify the effectiveness of the above-mentioned knowledge distillation deep fake detection method based on space-frequency feature fusion, experimental verification was carried out in the present invention.

[0097] Experimental dataset settings:

[0098] Experiments are evaluated on two widely used public datasets: FaceForensics++ (FF++) and Celeb-DeepFake (Celeb-DF). FaceForensics++ is a large dataset containing over 100,000 synthetic videos, comprised of four types of facial manipulation: Deepfake, Face2Face, FaceSwap, and NeuralTexture. The videos in the FF++ dataset are compressed into two versions: lightly compressed (c23) and heavily compressed (c40), using the H.264 codec with constant rate quantization parameters of 23 and 40, respectively. Celeb-DF consists of 408 original videos and 795 fake videos downloaded from YouTube. For the FF++ dataset, 720 videos were randomly selected for training, 140 for validation, and 140 for testing. The Celeb-DF dataset is split into training, validation, and test sets in a 6:1:1 ratio. Accuracy (ACC) and area under the ROC curve (AUC) were used as evaluation indicators.

[0099] The proposed method is implemented using Pytorch. 50 frames are sampled for each video in the dataset and the frame size is adjusted to The Xception network pre-trained on ImageNet is used as the backbone network of SFFB. Specifically, the entry flow and intermediate flow of Xception are used for feature extraction, and the exit flow is used for classification after feature fusion. The present invention adopts Adam optimizer with a learning rate of , the batch size is 32. The hyperparameters in are set to , , Temperature parameters Set to 1.

[0100] We first tested the FF++ dataset with varying video quality. The test results (ACC) (%) are shown in Table 1. The teacher model was pre-trained on the original image (c0), and the student model was trained and tested on compressed images (c23 and c40). The compared methods included deep learning-based methods, machine learning-based methods, and ADD, which also utilizes knowledge distillation.

[0101] Table 1

[0102]

[0103] As shown in Table 1, in most cases, the accuracy of our method outperforms other methods. The average accuracy of the model on versions c23 and c40 of FF++ was 98.22% and 91.30%, respectively. Notably, on version c23 of FF++, our method achieved a 2.77% improvement in average accuracy compared to the ADD method, which also utilizes knowledge distillation, demonstrating its superiority.

[0104] NeuralTexture is a powerful deep fake generation technology that extracts textures from one image and applies them to another image using a neural network. This process ensures high resolution and high detail textures of the output forged faces. Because it retains the original colors and details, effectively detecting forged faces generated by NeuralTexture remains a major challenge. However, the present invention can achieve an accuracy of up to 98.36% even on the c40 version of the most challenging NT dataset, an improvement of nearly 30% compared to ADD. In addition, it can be observed from Table 1 that even SFFB used alone can achieve an accuracy of 98.13% on the c40 version of the NT dataset, which proves that the features extracted by SFFB are comprehensive and effective.

[0105] To demonstrate the flexibility of our invention, SFFB was tested on the FF++ dataset (c0 version) and the Celeb-DF dataset. Table 2 shows the test results (ACC%). Even without the knowledge distillation framework, our proposed SFFB still achieves competitive results, achieving 100% AUC on the Celeb dataset, achieving near-perfect performance and reaching the state-of-the-art. Tables 1 and 2 show that SFFB achieves average accuracy of 99.15%, 96.19%, and 88.30% on the FF++ dataset at compression levels c0, c23, and c40, respectively. While weaker than SFFB and KD, it remains competitive with other similar works.

[0106] Table 2

[0107]

[0108] The present invention carried out four groups of experiments:

[0109] (1) Verifying the effectiveness of the knowledge distillation framework: To verify the effectiveness of the knowledge distillation framework, the SFFB branch and SFFB&KD were tested on the compressed version of FF++ respectively. The test results (ACC) (%) of SFFB and SFFB&KD on the compressed FF++ dataset are shown in Table 3.

[0110] Table 3

[0111]

[0112] Although SFFB alone can achieve competitive results, there is still room for improvement. As shown in Table 3, after applying knowledge distillation, the model's accuracy metrics achieved an improvement of up to 6.49%. This improvement in model performance after applying knowledge distillation is due to the fact that the information transferred from the teacher model during training provides additional guidance to the student model, enabling it to more effectively learn effective features of the original image and better generalize to compressed data.

[0113] (2) Experiment to verify the effectiveness of the fusion module: SFFB is used alone to evaluate the effectiveness of different fusion modules. As shown in Table 4, the summation refers to the direct and Perform element-by-element summation; concatenation means directly adding and Splice along the channel direction and use Convolution is used for dimensionality reduction; block attention consists of three steps: 1) divide the frequency domain and spatial domain feature maps into equal number of blocks, 2) traverse each block and press Figure 3 The method first fuses the frequency and spatial blocks at their corresponding locations to obtain a fused block. 3) All fused blocks are concatenated according to their positions in the original feature map to obtain a fused feature map. Table 4 shows the test results (%) of different fusion modules on the F2Fc40 dataset of FF++. SFFB achieves the best performance due to the inclusion of attention guidance in its fusion module, enabling the student model to effectively learn useful information from the teacher model. The poor performance of block-based attention may be due to the loss of global information caused by block-based attention.

[0114] Table 4

[0115]

[0116] (3) Experiment on the effectiveness of multi-knowledge distillation: Most knowledge distillations only use one of the response-based knowledge, relationship-based knowledge, and feature-based knowledge. This invention uses both response-based knowledge and feature-based knowledge. To verify the effectiveness of multi-knowledge, experiments were conducted on the F2Fc40 dataset in FF++. The results are shown in Table 5. and The sum of and The fusion of )、 and Distillation and 、 Both and logits participate in distillation using several different distillation methods. The test results (%) of different knowledge types on the FF++ F2Fc40 dataset are shown in Table 5. As can be seen from Table 5, the best performance is achieved when all three knowledge types participate in distillation.

[0117] Table 5

[0118]

[0119] (4) Visualization experiment of the results. Grad-CAM is used to visualize the sample features of the model on the FF++ dataset. Figure 4 As shown in the figure, when the input is the original image, SFFB can focus on the tampered facial area, which is why SFFB can achieve excellent test performance on the original image (c0). However, once the image quality drops to heavy compression (c40), SFFB's attention is distracted to areas such as the person's hair and background, or the model's attention is overly focused on a small area, resulting in a significant performance degradation. When SFFB and multi-knowledge distillation are used simultaneously, under the guidance of the teacher model, the student model's attention is restored to the tampered facial area, and the problem of too small an attention area is also alleviated, ensuring the model's generalization ability.

[0120] Corresponding to the method for detecting deep fakes using knowledge distillation based on space-frequency feature fusion in the above embodiment, the present invention further provides a system for detecting deep fakes using knowledge distillation based on space-frequency feature fusion, comprising:

[0121] The model construction module is used to build the teacher model and student model under the knowledge distillation framework. Both the teacher model and the student model use the space-frequency fusion branch as the backbone network;

[0122] The first training module uses the original images to train the teacher model;

[0123] The second training module uses paired original and compressed images as input into the knowledge distillation framework to train the student model. During the student model training process, the spatial-frequency features and soft labels of the teacher model and the hard labels of the student model serve as supervision for the student model.

[0124] The forgery detection module uses the student model trained in the second training module to detect deep forgeries in images.

[0125] For a knowledge distillation deep fake detection system based on space-frequency feature fusion in an embodiment of the present invention, since it corresponds to a knowledge distillation deep fake detection method based on space-frequency feature fusion in the above embodiment, the description is relatively simple. For relevant similarities, please refer to the description of a knowledge distillation deep fake detection method based on space-frequency feature fusion in the above embodiment, which will not be described in detail here.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0127] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0128] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep fake detection method based on knowledge distillation of space-frequency feature fusion, characterized by: include: Constructing a teacher model and a student model under a knowledge distillation framework, wherein both the teacher model and the student model use a space-frequency fusion branch as a backbone network; Use the original images to train the teacher model; A student model is trained by inputting paired original images and compressed images into a knowledge distillation framework. During the training of the student model, the spatial-frequency features and soft labels of the teacher model and the hard labels of the student model serve as supervision of the student model. Use the trained student model to detect deepfakes in images; During the student model training process, the loss function is: in, , and is a hyperparameter that controls the weights between different losses; 、 The spatial and frequency domain feature maps before the fusion module are used as the loss of the transferred features; is the loss function used to supervise the response knowledge; is the cross entropy loss, which is used for supervision of the true value label; ； ； in, Indicates the size of the feature map; Represents the frequency domain feature map of the student model, represents the frequency domain feature map of the teacher model, represents the time domain feature map of the student model, Represents the time domain feature map of the teacher model; ； in When the temperature parameter is When the teacher model is in the class The softmax output value on , It means that the student model is in the class The softmax output value on ; in is the logit of the teacher model, is the logit of the student model, Refers to the total number of tags.

2. The knowledge distillation deep fake detection method based on space-frequency feature fusion according to claim 1 is characterized in that: The space-frequency fusion branch includes: Data preprocessing module, preprocesses the input image; Feature extraction module, which realizes spatial and frequency domain feature extraction of facial images; Feature fusion module, which fuses spatial and frequency domain features under the guidance of attention; The classification module inputs the fused features into the classifier to output the authenticity probability of the face image.

3. The knowledge distillation deep fake detection method based on space-frequency feature fusion according to claim 2 is characterized in that: The image preprocessing includes: image enhancement and DCT frequency domain transformation.

4. The deep fake detection method based on knowledge distillation of space-frequency feature fusion according to claim 2 is characterized in that: The feature extraction module uses the entry stream and intermediate stream of Xception as the backbone network.

5. The deep fake detection method based on knowledge distillation of space-frequency feature fusion according to claim 2 is characterized in that: Under the guidance of attention, the spatial and frequency domain features are fused, including: Apply element-wise summation to the two feature maps to obtain a mixed feature map; Performing channel reduction on the mixed feature map, including global average pooling and full connection operations; Then, two fully connected layers are used to process the output of the channel-reduced fully connected operation, and the obtained features are multiplied with the corresponding spatial domain feature map and frequency domain feature map one by one, and the element-by-element summation operation of the feature map after element-by-element multiplication is performed with the original corresponding feature map; Concatenate features from two domains along the channel direction and apply Convolution is used to fuse features.

6. The deep fake detection method based on knowledge distillation of space-frequency feature fusion according to claim 2 is characterized in that: The classification module uses the export flow of Xception as the backbone network.

7. The method for detecting deep fakes based on knowledge distillation and space-frequency feature fusion according to claim 1 is characterized in that: The knowledge distillation framework adopts both response-based knowledge and feature-based knowledge.

8. A knowledge distillation deep fake detection system based on space-frequency feature fusion, characterized by: include: A model construction module is used to construct a teacher model and a student model under the knowledge distillation framework, wherein both the teacher model and the student model use a space-frequency fusion branch as a backbone network; The first training module uses the original images to train the teacher model; The second training module trains the student model by inputting paired original images and compressed images into the knowledge distillation framework. During the training of the student model, the spatial-frequency features and soft labels of the teacher model and the hard labels of the student model serve as supervision of the student model. During the training of the student model, the loss function is: in, , and is a hyperparameter that controls the weights between different losses; 、 The spatial and frequency domain feature maps before the fusion module are used as the loss of the transferred features; is the loss function used to supervise the response knowledge; is the cross entropy loss, which is used for supervision of the true value label; ； ； in, Indicates the size of the feature map; Represents the frequency domain feature map of the student model, represents the frequency domain feature map of the teacher model, represents the time domain feature map of the student model, Represents the time domain feature map of the teacher model; ； in When the temperature parameter is When the teacher model is in the class The softmax output value on , It means that the student model is in the class The softmax output value on ; in is the logit of the teacher model, is the logit of the student model, refers to the total number of tags; The forgery detection module uses the student model trained by the second training module to perform deep forgery detection on images.

Images