A method, apparatus and equipment for detecting face forgery based on knowledge distillation

Abstract

This invention discloses a face forgery detection method, apparatus, and device based on knowledge distillation, relating to the fields of artificial intelligence and visual content detection technology. This invention uses the pure supervisory signal under active defense as transferable knowledge, actively transferring it to the student network model under passive detection. During the inference stage, the student network does not need to rely on any pre-embedded watermarks or defense signals; it can achieve preliminary high-precision detection solely through the transferred pure forgery clues. Simultaneously, it achieves a balance between smoothly absorbing new knowledge and consolidating existing knowledge to achieve final high-precision detection. In other words, this invention reshapes the learning paradigm of the student network model from "learning traces" to "learning the essence of tampering," breaking the "identifying forgery by traces" learning paradigm in passive detection methods and balancing new and old knowledge. Ultimately, it improves the generalization performance of the student network model in cross-domain scenarios while achieving both active defense and passive detection, thus realizing accurate and efficient face forgery detection.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and visual content detection technology, and in particular to a method, apparatus and equipment for detecting face forgery based on knowledge distillation. Background Technology

[0002] With the rapid development of deep learning generation technology, highly realistic face forgery content (such as DeepFake, FaceSwap, etc.) has appeared on the Internet; therefore, developing efficient and robust face forgery detection technology has become a key problem that urgently needs to be solved in the field of computer vision.

[0003] Currently, methods for detecting face forgery are mainly divided into two categories: active defense and passive detection. Active defense methods typically embed defense signals (such as digital watermarks or adversarial perturbations) into the original image beforehand. When the image is attacked, these signals are used to destroy the visual quality of the forged image or extract verification information to determine its authenticity. Although active defense methods have strong forgery detection performance, they rely heavily on pre-embedded prior knowledge and cannot directly detect unprotected public images, which severely limits their practical application scenarios. In contrast, passive detection methods (i.e., blind detection) do not require any prior signals and directly determine authenticity by analyzing the features of the image to be detected, thus having a wider range of applicability.

[0004] However, the learning paradigm of passive detection methods is "identifying fakes by traces," which attempts to achieve detection by fitting "shallow forgery traces" (such as texture artifacts, fusion boundaries, color inconsistencies, etc.) left by different forgery algorithms that have significant differences in data distribution. This paradigm reduces the essential behavior of "being tampered with" to overfitting the "traces" produced by specific tampering tools. Therefore, when faced with a brand-new forgery algorithm that has never appeared in the training data, the model will experience a sharp drop in performance because it cannot match its unique "new traces." This results in poor generalization performance in cross-domain scenarios (i.e., when facing unknown new forgery techniques), making it difficult to achieve accurate and efficient face forgery detection. Summary of the Invention

[0005] This invention provides a method, apparatus, and device for detecting face forgery based on knowledge distillation, which can solve the problems existing in the prior art.

[0006] This invention provides a method, apparatus, and device for detecting face forgery based on knowledge distillation, comprising: Obtain a forged image containing a corrupted watermark signal; A pre-trained teacher network model is used to perceive fake regions in a fake image and generate an attention distribution indicating the fake regions. The pre-trained teacher network model is then used to perform knowledge distillation on a student network model so that the student network model learns to imitate the attention distribution generated by the pre-trained teacher network model, resulting in a trained student network model. The image to be detected is input into the trained student network model, which outputs the true / false classification result of the image to be detected.

[0007] Preferably, obtaining the trained student network model includes: A student network model for passive detection and a teacher network model for active defense detection are constructed. The teacher network model includes a feature-selective destruction and reconstruction module and a classifier based on the ViT architecture. The student network model is a classifier based on the ViT architecture, which includes a cascaded multi-head self-attention layer, a construction layer, and a classification head. The feature selective destruction and reconstruction module of the teacher network model is used to extract multi-scale features of the forged image and perceive the watermark integrity status of the multi-scale features to generate a feature suppression mask. The mask is used to suppress and reconstruct the features corresponding to the forged region in the multi-scale features to obtain the selectively destroyed image and pre-train the teacher network model. The forged image is processed using multi-head self-attention layers in both the teacher and student network models, generating a first attention map and a second attention map respectively. The difference in attention distribution between the first and second attention maps is obtained to construct a distillation loss function for iterative knowledge distillation. A differentiated learning rate is configured according to the parameter update ratio decreasing with each distillation round. The parameters of the student network model are updated using the distillation loss function of the current round with the differentiated learning rate. Training stops when the distribution difference between the feature response heatmaps constructed by the teacher and student network models for the same input image in the construction layer is lower than a preset dynamic threshold that increases with each distillation round, resulting in the trained student network model.

[0008] Preferably, the acquisition of the selectively destroyed image includes: The feature selective destruction and reconstruction module of the teacher network model includes a cascaded feature selective destruction module and a feature reconstruction module. The feature selective destruction module includes a cascaded encoder and a watermark analyzer, and the feature reconstruction module is an encoder-decoder structure. The forged image is input into the encoder of the teacher network model and downsampled step by step to obtain the first... Layer to the first Multi-scale features ; Using watermark analyzer Perceive multi-scale features The system assesses the integrity of the watermark, identifies areas of forgery where the watermark is damaged, and generates a feature suppression mask. , represented as: ; in: This represents the Sigmoid activation function; This indicates the mapping operation performed by the watermark analyzer; Indicates the encoder's first... The output features of the layer; This represents the generated feature suppression mask; Feature suppression mask With encoder number The output features of the layer are multiplied element-wise to obtain the suppressed encoder features. , represented as: ; in: This represents the generated feature suppression mask; Indicates the encoder's first... The output features of the layer; This represents the features after suppression by the feature suppression mask; This indicates element-wise multiplication; Suppressed encoder features Input features corresponding to the decoder layer Performing element-wise addition and merging is represented as: ; in: Indicates the encoder's first... The layer output features are suppressed by a feature suppression mask; The decoder representing the feature reconstructor Layer input features; Indicates the decoder's first The final input features of the layer; This indicates element-wise addition; Using the decoder to fuse features Perform layer-by-layer reconstruction and output selectively destroyed images. .

[0009] Preferably, the pre-training of the teacher network model includes: The total loss function of the teacher network model is constructed by weighting and summing the watermark perception loss, image destruction loss, and the classification loss of the teacher network model itself. And using the total loss function Update the parameters of the teacher network model to achieve pre-training of the teacher network model; the total loss function Represented as: ; in: This indicates the perceived loss of the watermark; Indicates image corruption and loss; This represents the classification loss of the teacher network model itself; The watermark sensing loss Represented as: ; in: Represents the binary cross-entropy function; This represents the total number of layers in the encoder of the teacher network model; Indicates the first in the encoder The predicted watermark signal is reconstructed by a multilayer perceptron from the feature suppression mask corresponding to the layer output feature. This indicates a reference watermark extracted from a forged image; The image destruction loss Represented as: ; in: This indicates a forged image; This indicates selective image destruction; This indicates a forged region binary mask; This represents the threshold for mean squared error loss. Represents the mean square error function; This represents the function for calculating the maximum value. This represents element-wise multiplication. The classification loss of the teacher network model itself Represented as: ; in: Represents the binary cross-entropy function; This represents the prediction result output by the teacher network model classifier; This indicates the authenticity label corresponding to the forged image.

[0010] Preferably, the initial knowledge distillation stage of the knowledge distillation includes: Obtain the first attention map generated by the multi-head self-attention layer of the pre-trained teacher network model for the forged image. And the second attention map generated by the multi-head self-attention layer of the student network model for forged images. The first attention map is calculated using a pre-trained teacher network model. The JS divergence between the matrix and the uniform distribution matrix is ​​used to determine the importance weights. Among them, the dimension of the uniform distribution matrix is ​​the same as that of the first attention map. The dimensions are consistent; Using importance weights For the first attention map With the second attention map The distillation loss is obtained by performing a weighted summation operation on the mean square errors between them. , represented as: ; in: Represents the mean square error function; and They represent the first Attention maps corresponding to the student network model and the pre-trained teacher network model classifier; express JS divergence between the matrix and the uniform distribution matrix; Represents the set of network layers. , This represents the total number of layers in the student network model; Obtain the classification loss of the student network model itself. , represented as: ; in: This represents the classification prediction results of the student network model; Indicates the authenticity of a forged image. Represents the binary cross-entropy function; Utilizing distillation losses The classification loss of the student network model itself The resulting total loss function The parameters of the student network model are updated to obtain a pre-trained student network model, wherein the total loss function is... Represented as: ; in: and This represents the weighting coefficient.

[0011] Preferably, in the iterative knowledge distillation stage of the knowledge distillation, the parameters of the pre-trained teacher network model are first fine-tuned to obtain the fine-tuned teacher network model, including: The pre-defined novelty loss function is weighted and summed with the classification loss of the pre-trained teacher network model to construct the total loss function of the teacher network model. And using the total loss function The parameters of the pre-trained teacher network model are fine-tuned to obtain the fine-tuned teacher network model, and the total loss function is... Represented as: ; in: , Indicates the weighting coefficient; Represents the novelty loss function; This represents the classification loss of the pre-trained teacher network model itself; The classification loss of the pre-trained teacher network model itself. Represented as: ; in: The classification prediction results of the pre-trained teacher network model; To forge genuine and fake labels corresponding to images, It is a binary cross-entropy function; The novelty loss function Represented as: ; in: Indicates the current iteration round. Positive integers greater than 1; Indicates the index of the historical iteration round; Indicates the first In each round, the features output by the penultimate layer of the pre-trained teacher network model's classifier; Indicates the first At each round, the historical mean of the features output by the penultimate layer of the pre-trained teacher network model's classifier; Indicates the novelty loss threshold; express Norm calculation function; This represents the function for calculating the maximum value.

[0012] Preferably, in the iterative knowledge distillation stage of the knowledge distillation, after the teacher network model is fine-tuned, the parameters of the initially trained student network model are updated with a differentiated learning rate, including: Determine the iteration round The proportion of key parameters that increases and gradually decreases , represented as: ; in: This is the current iteration round; For key parameter proportions; For each trainable parameter in the initially trained student network model, sort them in descending order based on the absolute value of the gradient during backpropagation, and then rank the top-ranked parameters. The parameters were identified as critical parameters, and the remaining parameters were identified as non-critical parameters. When updating the parameters, the critical parameters were assigned a first learning rate. Perform gradient descent; apply a second learning rate to non-critical parameters. Perform gradient descent; where the second learning rate... Below the first learning rate .

[0013] Preferably, the acquisition of the distribution differences and dynamic thresholds between the response heatmaps, and the obtaining of the trained student network model, includes: Grad-CAM heatmaps generated by the constructive layers of the fine-tuned teacher and student network models for the same input image are obtained as feature response heatmaps; and the distribution differences between the two feature response heatmaps are obtained. , represented as: ; in: Represents spatial coordinates Calculate the expected value of all pixels on the graph; and The coordinates in the normalized heatmaps obtained from the student network model and the fine-tuned teacher network model classifiers are respectively: Pixel values; This indicates an indicator function based on a preset activation threshold, and its calculation logic is as follows: ; Pixel-level filtering of the feature response heatmap is performed using an indicator function to identify high-response regions; simultaneously, the characteristics of each iteration are determined. The dynamic threshold that increases gradually with each increment , represented as: ; in: , and Both represent constants; Indicates the current iteration round; When distribution differences Below the dynamic threshold When the heatmap of the feature response generated by the student network model is determined to be aligned with the fine-tuned teacher network model in the key discrimination region, the distillation training at the current stage is stopped, and the trained student network model is output.

[0014] This invention also provides a face forgery detection device based on knowledge distillation, comprising: The data module is used to acquire forged images containing signals from damaged watermarks; The detection module is used to perceive the forged regions in the forged image using the teacher network model and generate an attention distribution indicating the forged regions. The teacher network model is then pre-trained using the forged regions to obtain a pre-trained teacher network model. The pre-trained teacher network model is used to perform knowledge distillation on the student network model so that the student network model learns and imitates the attention distribution generated by the pre-trained teacher network model to obtain a trained student network model. The image to be detected is input into the trained student network model, which outputs the true / false classification result of the image to be detected.

[0015] This invention also provides an electronic device, including a memory and a processor; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the knowledge distillation-based face forgery detection method described above.

[0016] This invention provides a method, apparatus, and device for detecting face forgery based on knowledge distillation. Compared with the prior art, its advantages are as follows: This invention utilizes a knowledge distillation mechanism to construct a knowledge transfer pathway from active defense represented by the teacher network model to passive detection represented by the student network model. Throughout the process, the pure supervisory signal initially provided by the teacher network model under active defense is treated as transferable knowledge and actively transferred to the student network model under passive detection. During the inference phase, the student network does not need to rely on any pre-embedded watermarks or defense signals; it can achieve preliminary high-precision detection solely through the pure forgery clues transferred from the teacher network. Simultaneously, a novelty loss function and a parameter selective update mechanism are combined to achieve an excellent balance between smoothly absorbing new knowledge and consolidating existing knowledge, thus achieving final high-precision detection. In essence, this invention reshapes the learning paradigm of the student network model from "learning traces" to "learning the essence of tampering," breaking the "trace-based forgery detection" learning paradigm in passive detection methods and balancing new and old knowledge. Ultimately, it improves the generalization performance of the student network model in cross-domain scenarios while achieving both active defense and passive detection, thereby realizing accurate and efficient face forgery detection. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall process of a face forgery detection method based on knowledge distillation provided in an embodiment of the present invention; Figure 2 A schematic diagram of a detection network architecture for a face forgery detection method based on knowledge distillation, provided for an embodiment of the present invention; Figure 3 A schematic diagram of the framework of a selective destruction module for a face forgery detection method based on knowledge distillation provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the effect of a dynamic progressive distillation strategy for a knowledge-based face forgery detection method provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0019] With the rapid development of deep learning technology, face spoofing technology has become increasingly sophisticated, posing a serious challenge to social information security due to the spread of misinformation. Currently, face spoofing detection methods are mainly divided into two categories: active defense and passive detection. Active defense methods typically embed defense signals (such as digital watermarks or adversarial perturbations) into the original image beforehand. When the image is attacked, these signals are used to destroy the visual quality of the spoofed image or extract verification information to determine its authenticity. Although this type of method has strong spoofing detection performance, it relies heavily on pre-embedded prior knowledge and cannot directly detect unprotected public images, severely limiting its practical application scenarios. In contrast, passive detection methods (i.e., blind detection) do not require any prior signals and directly determine authenticity by analyzing the features of the image to be detected, making them more widely applicable. However, due to the significant differences in the data distribution of spoofing traces generated by various spoofing algorithms, the generalization performance of existing passive detection methods is generally insufficient in cross-domain scenarios (i.e., when facing unknown new spoofing technologies).

[0020] Therefore, a new technical solution is urgently needed to establish a cross-paradigm knowledge transfer pathway between active defense and passive detection, guiding the passive detection model to accurately capture the essential features of forgery, thereby breaking through the bottleneck of cross-domain generalization performance. Based on this, this invention provides a passive face forgery detection method based on active defense knowledge distillation, such as... Figure 1 As shown, it specifically includes: Step S101: Obtain the forged image containing the damaged watermark signal and the corresponding binary mask of the forged region; construct a student network for passive detection and an active defense teacher network containing a selective destruction module. This step involves system initialization and data preparation operations, as detailed below: First, a forged image containing the damaged watermark signal and the corresponding binary mask of the forged region are obtained. Specifically, the data generation process is as follows: An original real face image (e.g., from the CelebA-HQ dataset) is acquired, and a binary watermark signal is covertly embedded into it using a watermark embedding module to generate a visually lossless protected image. Then, various existing face forgery algorithms (such as DeepFake) are used to locally tamper with the protected image. The watermark signal in the tampered area will be destroyed, thus obtaining a forged image containing the damaged watermark signal. Simultaneously, the labeled facial region segmentation mask from the training dataset is obtained as the binary mask of the forged region (typically, a pixel value of 1 represents the forged region, and 0 represents the real region). Figure 2 This paper presents the architecture of the passive face forgery detection method based on active defense knowledge distillation proposed in this invention. The specific architectures of the student network and the teacher network are as follows: (1) The student network corresponds to the conventional passive detection method. It adopts the ViT architecture model based on CLIP pre-trained weights. The self-attention mechanism of this architecture can effectively capture the global context and detail forgery traces of the image.

[0021] (2) The teacher network is an active defense method based on the selective destruction module, which includes a watermark embedding module, a selective destruction module and a downstream classifier. In order to ensure perfect alignment of subsequent feature dimensions, the downstream classifier adopts the same ViT architecture as the student network. The selective destruction module includes a watermark analyzer composed of multiple convolutional attention modules and a feature reconstructor based on a visual state space model (such as the VM-UNet architecture). It utilizes the advantage of its linear complexity global receptive field to reveal discriminative forgery clues to the student network by accurately destroying the forgery region.

[0022] Step S102: Input the forged image into the teacher network for feature extraction to obtain multi-scale features; use the watermark analyzer in the selective destruction module to perceive the watermark integrity status of the multi-scale features to generate a feature suppression mask, and combine the feature reconstructor to use the feature suppression mask to suppress and reconstruct the multi-scale features, outputting a selective destruction image in which the real area remains intact and the forged area is precisely destroyed; and complete the pre-training of the teacher network based on the selective destruction image.

[0023] The architecture of the selective destruction module is as follows: Figure 3 As shown, when the selective destruction module is executed, it first inputs the forged image into the encoder of the teacher network for step-by-step downsampling, obtaining the first... Layer to the first layer( Multi-scale features (total number of layers), denoted as Subsequently, a watermark analyzer was used. The system perceives the integrity of watermarks at multiple scales, identifies forged areas where the watermark is damaged, and generates feature suppression masks. The calculation formula is: .

[0024] in: This represents the Sigmoid activation function; This represents the mapping operation performed by the watermark analyzer to determine multi-scale features. The distribution weights of damaged watermark signals; Indicates the encoder's first... The output features of the layer; This represents the generated feature suppression mask. Next, the feature suppression mask... Corresponding multi-scale features Perform element-wise multiplication to obtain the suppressed encoder features. The calculation formula is: .

[0025] in: This represents the generated feature suppression mask; Indicates the encoder's first... The output features of the layer; This represents the features after suppression by the feature suppression mask; This represents element-wise multiplication. The physical meaning of this operation is to "erase" the features of the forged region; further, it is then multiplied by the corresponding layer input features of the decoder. The element-wise addition and merging process is performed using the following formula: .

[0026] in: Indicates the encoder's first... The layer output features are suppressed by a feature suppression mask; The decoder representing the feature reconstructor Layer input features; Indicates the decoder's first The final input features of the layer; This represents element-wise addition; finally, the decoder is used to fuse the features. Perform layer-by-layer reconstruction and output selectively destroyed images. Because the forged features in the multi-scale features are continuously suppressed by the feature suppression mask during the reconstruction process, the image is selectively corrupted. The destructive effect is only displayed in the area corresponding to the damaged watermark signal, while the actual area of ​​both remains consistent.

[0027] The teacher network was pre-trained based on this selectively corrupted image, and the total loss function was... Including the following sub-losses: (1) Watermark perception loss In order to drive the network to accurately locate the watermark-damaged area, this invention will Input a multilayer perceptron to obtain the predicted watermark signal. And use the watermark extraction module to extract watermarks from selectively corrupted images. Extract the reference watermark Using a teacher network to calculate watermark perceived loss To minimize the difference between the two, the calculation formula is as follows: .

[0028] in: It is a binary cross-entropy function; This indicates the total number of layers in the teacher's network encoder; For the encoder The predicted watermark signal is reconstructed by a multilayer perceptron from the feature suppression mask corresponding to the layer output feature. The reference watermark extracted from the forged image.

[0029] (2) Image destruction loss To constrain the network from causing severe image signal corruption in forged regions while maintaining lossless image signal corruption in real regions, a teacher network is used to calculate the image corruption loss. The calculation formula is: .

[0030] in: To forge images; To selectively destroy the image; To forge a region binary mask; The mean squared error loss threshold; Let the mean square error function be used. This is a function for calculating the maximum value. This indicates element-wise multiplication.

[0031] (3) Classification loss of teacher networks To ensure the network's basic ability to classify true and false information, the classification loss... The calculation formula is: .

[0032] in: It is a binary cross-entropy function; The prediction results output by the classifier of the teacher network; To identify the genuine and fake labels corresponding to the forged image.

[0033] During the pre-training phase, the overall objective function is used. The optimization formula for teacher network parameters is as follows: .

[0034] in: This indicates the perceived loss of the watermark; Indicates image corruption and loss; This represents the classification loss of the teacher network.

[0035] Step S103: In the first round of dynamic progressive distillation, obtain the first attention map generated by the pre-trained teacher network and the second attention map generated by the student network for the forged image; combine the divergence weights that reflect the attention concentration, and calculate the distillation loss function based on the difference in attention distribution between the first attention map and the second attention map; then, update the parameters of the student network using the distillation loss function and the classification loss of the student network, so that the student network can obtain a preliminary forgery discrimination ability.

[0036] In the first round of dynamic progressive distillation, the specific implementation method is as follows: First, obtain the first attention map generated by the teacher network for the forged image. and the second attention map generated by the student network Calculate the first attention map using a teacher network. The JS divergence between the matrix and the uniform distribution matrix is ​​used to determine the importance weights. Among them, the dimension of the uniform distribution matrix is ​​the same as that of the first attention map. The dimensions are consistent; then, importance weights are used. For the first attention map With the second attention map The distillation loss is obtained by performing a weighted summation operation on the mean square errors between them. The calculation formula is: .

[0037] in: Let the mean square error function be used. and They represent the first Attention graphs corresponding to the classifiers of the student network and teacher network; express The JS (Jensen-Shannon) divergence between the matrix and the uniform distribution matrix; It is a set of network layers, where , The total number of layers in the student network is represented; then, the classification loss of the student network is calculated. The calculation formula is: .

[0038] in: For the classification prediction results of the student network, To forge genuine and fake labels corresponding to images, The binary cross-entropy function; ultimately, the total loss function of the student network. Includes distillation loss Classification loss with student networks The calculation formula is: .

[0039] in: and As the weighting coefficient, preferred , .

[0040] Step S104: Enter the iteration phase. Using the novelty loss function and the classification loss of the teacher network, fine-tune the parameters of the teacher network to push the current round's features away from the feature centers of previous rounds, thus uncovering new forgery clues. Based on the difference in attention distribution between the fine-tuned teacher network and student network, calculate the distillation loss function for the current round. Based on the parameter selective update mechanism, update the parameters of the student network with a differentiated learning rate, according to the key parameter update ratio decreasing with each distillation round, using the distillation loss function of the current round and the classification loss of the student network. During the loop, obtain the feature response heatmaps generated by the teacher network and student network for the same input, and calculate the distribution difference between the feature response heatmaps. When the distribution difference is lower than a dynamic threshold that increases with each distillation round, stop the current stage of training, obtaining the trained student network. The teacher network and the student network will be updated collaboratively during the iteration phase. Round iteration ( The specific implementation method is as follows: (1) Teacher network fine-tuning: The novelty loss function and the classification loss of the teacher network are weighted and summed to construct the total loss function of the teacher network. And using the total loss function For fine-tuning, the calculation formula is: .

[0041] in: , These are the weighting coefficients; Represents the novelty loss function; Represents the classification loss of the teacher network; classification loss of the teacher network. The calculation formula is: .

[0042] in: The classification prediction results for the teacher network; To forge genuine and fake labels corresponding to images, The binary cross-entropy function; the novelty loss function. The calculation formula is: .

[0043] in: For the current iteration round, It is a positive integer greater than 1; This is the index for the historical iteration rounds, and its value range is... ; In the first In each round, the features output by the penultimate layer of the classifier in the teacher network; In the first At each round, the historical mean of the features output by the penultimate layer of the classifier in the teacher network; This is the novelty loss threshold; for Norm calculation function; This is a function for calculating the maximum value.

[0044] (2) Student network fine-tuning: The parameters of the student network are selectively updated using differentiated learning rates. The specific process is as follows: First, determine the parameters of the student network as the iteration rounds progress. The proportion of key parameters that increases and gradually decreases Its calculation formula is expressed as: .

[0045] in: This is the current iteration round; For key parameter proportions; Next, for each trainable parameter in the student network, they are sorted in descending order according to the absolute value of their gradients during backpropagation, and the top-ranked parameters are... The parameters were identified as critical parameters, and the remaining parameters were identified as non-critical parameters. When updating the parameters, the critical parameters were assigned a first learning rate. Perform gradient descent; apply a second learning rate to non-critical parameters. Perform gradient descent; where the second learning rate... Below the first learning rate In this invention Set as of .

[0046] Through the above differentiated learning rate configuration, during the fine-tuning phase, standard-strength updates can be performed on key parameters with high contribution, while weak updates are performed on non-key parameters. This mechanism enables the student network to effectively retain the discriminative knowledge acquired in previous distillation rounds when learning novel forged cues output by the teacher network; for example... Figure 4 As shown, the first and second rows respectively illustrate the teacher network and the student network in different distillation rounds. The generated Grad-CAM heatmaps are as follows: the heatmap of the teacher network comes from its reasoning results on the fake faces in the current round; the heatmap of the student network is taken from its reasoning results on the same fully fake face image after each round of distillation. It can be observed from the figure that the teacher network can pay attention to different fake regions in each round, while the student network gradually focuses its attention on the entire fake region as the distillation progresses.

[0047] (3) Distillation termination condition: During the iteration process, based on the distribution differences of the characteristic response heatmap The specific process for determining the current training state is as follows: First, obtain the Grad-CAM (Gradient-weighted Class Activation Mapping) heatmaps generated by the teacher network and the student network for the same input image, respectively, as feature response heatmaps; then, calculate the distribution differences between the feature response heatmaps. The calculation formula is: .

[0048] in: Represents spatial coordinates Calculate the expected value of all pixels in the array, i.e., calculate the spatial average value; and The coordinates in the normalized heatmaps obtained from the student network and teacher network classifiers are respectively: Pixel values; The indicator function is based on a preset activation threshold, and its calculation logic is as follows: .

[0049] Pixel-level filtering of the feature response heatmap is performed using an indicator function to identify high-response regions; simultaneously, the characteristics of each iteration are determined. The dynamic threshold that increases gradually with each increment The calculation formula is: .

[0050] in: , and All are constants, preferably , as well as ; For the current iteration round; when the distribution difference Below the dynamic threshold When the heatmap of the feature response generated by the student network is determined to be aligned with the teacher network in the key discrimination region, the distillation training at the current stage is stopped, and the trained student network is output.

[0051] Step S105: Obtain the image to be detected, input the image to be detected into the trained student network for feature extraction, and output the true / false classification result; the specific implementation method is as follows: First, acquire the image to be detected; in practical applications, the image to be detected is a face image that does not require watermark embedding or selective destruction processing; then, as... Figure 2 As shown in the data stream of the middle school student network, the image to be detected is directly input into the student network. The student network uses the discriminative knowledge extracted in the previous distillation process to extract the deep semantic features of the image to be detected and generate the corresponding student network attention map. Subsequently, the student network performs true / false classification on the deep semantic features and outputs the classification result containing the category label and confidence level.

[0052] As shown in Table 1, by adopting a training mechanism based on active defense knowledge distillation, the average AUC of the trained student network on cross-domain datasets such as CDF and DFDC is significantly improved compared with other passive detection methods. Experimental results show that the method of the present invention can effectively improve the generalization discrimination of the model when facing unknown forgery attack algorithms.

[0053] Table 1. Results of different passive detection methods on CDF, DFDC, DFDCP, and DFD datasets.

[0054] This invention innovatively introduces a knowledge distillation mechanism into the face forgery detection task, constructing a knowledge transfer path from active defense to passive detection. This mechanism effectively transfers the strong discriminative ability of active defense methods, which rely on defense signals, to the passive detection model, breaking the extreme dependence of traditional active defense on pre-embedded signals (such as digital watermarks). This enables the passive detection model to achieve high-precision blind detection without any prior information during the actual inference stage, significantly solving the technical bottleneck of insufficient generalization performance of existing passive detection methods when facing unknown forgery types (cross-domain scenarios).

[0055] This invention addresses the shortcomings of traditional active defense methods, which suffer from the mixing of real and fake features due to "global perturbation." By using watermark integrity as a guide, it precisely locates and destroys fake regions within the feature space while preserving the features of real regions. This asymmetric destruction effect explicitly reveals fake clues with extremely high discriminative power and effectively filters out irrelevant background noise. This provides high-purity, high-quality teacher knowledge for the training of passive detection models, significantly improving the accuracy of model feature extraction.

[0056] The dynamic progressive distillation strategy proposed in this invention abandons the one-time full knowledge transfer mode that easily leads to model performance degradation in traditional knowledge distillation. By dynamically fine-tuning the teacher network during the training process to continuously explore new clues, and by using a parameter selective update mechanism to perform differentiated learning rate updates on key parameters of the student network, an excellent balance is achieved between smoothly absorbing new knowledge and consolidating existing knowledge. This effectively alleviates the catastrophic forgetting problem of the model in long-term learning and significantly enhances the convergence stability and robustness of the overall distillation process.

[0057] The selective destruction module proposed in this invention uses watermark integrity as a guide to precisely locate and directionally destroy features of forged regions within the feature space, while perfectly preserving features of the real regions. This operation transforms the "prior advantage of proactive defense" into "visualized, high signal-to-noise ratio forgery clues," outputting a selectively destroyed image where "the real region is intact and the forged region is destroyed." This image, as teacher knowledge, explicitly reveals the spatial distribution of the forged regions, providing the student network with a pure supervisory signal far exceeding traditional methods. The student network, by imitating the attention distribution learned by the teacher network on the selectively destroyed image, elevates its learning objective from "fitting specific forgery traces" to "understanding the essential spatial structure of forged regions." This "meta-knowledge" learning approach enables the model to actively locate "suspicious regions inconsistent with the real regions" from the image when facing unknown forgery algorithms, rather than relying on matching specific traces, thus significantly improving cross-domain generalization ability.

[0058] In the iterative distillation stage, this invention introduces a novelty loss function to fine-tune the teacher network. This loss function drives the teacher network to discover novel forgery cues that are significantly different from the feature centers of previous rounds in the current round, thereby avoiding the teacher network from overfitting to specific forgery traces and continuously providing diverse and novel supervision signals to the student network. This "dynamic teacher" mechanism ensures that the student network can be exposed to rich forgery features at different stages, rather than repeatedly learning the same content. Furthermore, a parameter selective update mechanism is introduced. Based on the gradient importance of the parameters, the parameters of the student network are dynamically divided into key parameters and non-key parameters, and updated with differentiated learning rates. Key parameters use a standard learning rate to quickly absorb novel cues provided by the teacher network, while non-key parameters use a lower learning rate to consolidate the discriminative knowledge learned in previous rounds. This differentiated update strategy enables the student network to achieve a fine balance between absorbing new knowledge and retaining old knowledge.

[0059] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A face forgery detection method based on knowledge distillation, characterized in that, Includes the following steps: Obtain a forged image containing a corrupted watermark signal; A pre-trained teacher network model is used to perceive fake regions in a fake image and generate an attention distribution indicating the fake regions. The pre-trained teacher network model is then used to perform knowledge distillation on a student network model so that the student network model learns to imitate the attention distribution generated by the pre-trained teacher network model, resulting in a trained student network model. Input the image to be detected into the trained student network model, and output the true or false classification result of the image to be detected; The acquisition of the trained student network model includes: A student network model for passive detection and a teacher network model for active defense detection are constructed. The teacher network model includes a feature-selective destruction and reconstruction module and a classifier based on the ViT architecture. The student network model is a classifier based on the ViT architecture, which includes a cascaded multi-head self-attention layer, a construction layer, and a classification head. The feature selective destruction and reconstruction module of the teacher network model is used to extract multi-scale features of the forged image and perceive the watermark integrity status of the multi-scale features to generate a feature suppression mask. The mask is used to suppress and reconstruct the features corresponding to the forged region in the multi-scale features to obtain the selectively destroyed image and pre-train the teacher network model. The forged image is processed by multi-head self-attention layers in the teacher network model and the student network model respectively, generating a first attention map and a second attention map. The difference in attention distribution between the first attention map and the second attention map is obtained to construct a distillation loss function for iterative knowledge distillation. The differential learning rate is configured according to the parameter update ratio decreasing with each distillation round. The parameters of the student network model are updated using the distillation loss function of the current round with the differential learning rate. Training stops when the distribution difference between the feature response heatmaps constructed by the teacher network model and the student network model for the same input image in the construction layer is lower than the preset dynamic threshold that increases with each distillation round, thus obtaining the trained student network model. The acquisition of the selectively destroyed image includes: The feature selective destruction and reconstruction module of the teacher network model includes a cascaded feature selective destruction module and a feature reconstruction module. The feature selective destruction module includes a cascaded encoder and a watermark analyzer, and the feature reconstruction module is an encoder-decoder structure. The forged image is input into an encoder of the teacher network model for step-by-step down-sampling to obtain multi-scale features of the first layer to the 25th layer , the 50th layer , and the 101st layer . Utilizing a watermark analyzer Perceptual multi-scale features of the watermark integrity condition, determine a counterfeit region in which the watermark is damaged, and generate a feature suppression mask , represented as: ； wherein: represents a Sigmoid activation function; represents a mapping operation performed by the watermark analyzer; represents the output features of the encoder’s layer; represents the generated feature suppression mask; Feature suppression mask With encoder number The output features of the layer are multiplied element-wise to obtain the suppressed encoder features. , is represented as: ； in: This represents the generated feature suppression mask; Indicates the encoder's first... The output features of the layer; This represents the features after suppression by the feature suppression mask; This indicates element-wise multiplication; Suppressed encoder features Input features corresponding to the decoder layer Performing element-wise addition and merging is represented as: ； in: Indicates the encoder's first... The layer output features are suppressed by a feature suppression mask; The decoder representing the feature reconstructor Layer input features; Indicates the decoder's first The final input features of the layer; This indicates element-wise addition; Using the decoder to fuse features Perform layer-by-layer reconstruction and output selectively destroyed images. .

2. The face forgery detection method based on knowledge distillation according to claim 1, characterized in that, The pre-training of the teacher network model includes: The total loss function of the teacher network model is constructed by weighting and summing the watermark perception loss, image destruction loss, and the classification loss of the teacher network model itself. And using the total loss function Update the parameters of the teacher network model to achieve pre-training of the teacher network model; the total loss function Represented as: ； in: This indicates the perceived loss of the watermark; Indicates image corruption or loss; This represents the classification loss of the teacher network model itself; The watermark perception loss Represented as: ； in: Represents the binary cross-entropy function; This represents the total number of layers in the encoder of the teacher network model; Indicates the first in the encoder The predicted watermark signal is reconstructed by a multilayer perceptron from the feature suppression mask corresponding to the layer output feature. This indicates a reference watermark extracted from a forged image; The image destruction loss Represented as: ； in: This indicates a forged image; This indicates selective image destruction; This indicates a forged region binary mask; This represents the threshold for mean squared error loss. Represents the mean square error function; This represents the function for calculating the maximum value. This represents element-wise multiplication. The classification loss of the teacher network model itself Represented as: ； in: Represents the binary cross-entropy function; This represents the prediction result output by the teacher network model classifier; This indicates the authenticity label corresponding to the forged image.

3. The face forgery detection method based on knowledge distillation according to claim 2, characterized in that, The initial knowledge distillation stage of the aforementioned knowledge distillation includes: Obtain the first attention map generated by the multi-head self-attention layer of the pre-trained teacher network model for the forged image. And the second attention map generated by the multi-head self-attention layer of the student network model for forged images. The first attention map is calculated using a pre-trained teacher network model. The JS divergence between the matrix and the uniform distribution matrix is ​​used to determine the importance weights. Among them, the dimension of the uniform distribution matrix is ​​the same as that of the first attention map. The dimensions are consistent; Using importance weights For the first attention map With the second attention map The distillation loss is obtained by performing a weighted summation operation on the mean square errors between them. , is represented as: ； in: Represents the mean square error function; and They represent the first Attention maps corresponding to the student network model and the pre-trained teacher network model classifier; express JS divergence between the matrix and the uniform distribution matrix; Represents the set of network layers. , This represents the total number of layers in the student network model; Obtain the classification loss of the student network model itself. , is represented as: ； in: This represents the classification prediction results of the student network model; Indicates the authenticity of a forged image. Represents the binary cross-entropy function; Utilizing distillation losses The classification loss of the student network model itself The resulting total loss function The parameters of the student network model are updated to obtain a pre-trained student network model, wherein the total loss function is... Represented as: ； in: and This represents the weighting coefficient.

4. The face forgery detection method based on knowledge distillation according to claim 3, characterized in that, The iterative knowledge distillation stage of the knowledge distillation process first fine-tunes the parameters of the pre-trained teacher network model to obtain the fine-tuned teacher network model, including: The pre-defined novelty loss function is weighted and summed with the classification loss of the pre-trained teacher network model to construct the total loss function of the teacher network model. And using the total loss function The parameters of the pre-trained teacher network model are fine-tuned to obtain the fine-tuned teacher network model, and the total loss function is... Represented as: ； in: , Indicates the weighting coefficient; Represents the novelty loss function; This represents the classification loss of the pre-trained teacher network model itself; The classification loss of the pre-trained teacher network model itself. Represented as: ； in: The classification prediction results of the pre-trained teacher network model; To forge genuine and fake labels corresponding to images, It is a binary cross-entropy function; The novelty loss function Represented as: ； in: Indicates the current iteration round. Positive integers greater than 1; Indicates the index of the historical iteration round; Indicates the first In each round, the features output by the penultimate layer of the pre-trained teacher network model's classifier; Indicates the first At each round, the historical mean of the features output by the penultimate layer of the pre-trained teacher network model's classifier; Indicates the novelty loss threshold; express Norm calculation function; This represents the function for calculating the maximum value.

5. The face forgery detection method based on knowledge distillation according to claim 4, characterized in that, In the iterative knowledge distillation stage of the knowledge distillation, after the teacher network model is fine-tuned, the parameters of the initially trained student network model are updated using a differentiated learning rate, including: Determine the iteration round The proportion of key parameters that increases and gradually decreases , is represented as: ； in: This is the current iteration round; For key parameter proportions; For each trainable parameter in the initially trained student network model, sort them in descending order based on the absolute value of the gradient during backpropagation, and then rank the top-ranked parameters. The parameters were identified as critical parameters, and the remaining parameters were identified as non-critical parameters. When updating the parameters, the critical parameters were assigned a first learning rate. Perform gradient descent; apply a second learning rate to non-critical parameters. Perform gradient descent; where the second learning rate... Below the first learning rate .

6. The face forgery detection method based on knowledge distillation according to claim 5, characterized in that, The acquisition of the distribution differences and dynamic thresholds between the response heatmaps, and the obtaining of the trained student network model, includes: Grad-CAM heatmaps generated by the constructive layers of the fine-tuned teacher and student network models for the same input image are obtained as feature response heatmaps; and the distribution differences between the two feature response heatmaps are obtained. , is represented as: ； in: Represents spatial coordinates Calculate the expected value of all pixels on the graph; and The coordinates in the normalized heatmaps obtained from the student network model and the fine-tuned teacher network model classifiers are respectively: Pixel values; This indicates an indicator function based on a preset activation threshold, and its calculation logic is as follows: ； Pixel-level filtering of the feature response heatmap is performed using an indicator function to identify high-response regions; simultaneously, the characteristics of each iteration are determined. The dynamic threshold that increases gradually with each increment , is represented as: ； in: , and Both represent constants; Indicates the current iteration round; When distribution differences Below the dynamic threshold When the heatmap of the feature response generated by the student network model is determined to be aligned with the fine-tuned teacher network model in the key discrimination region, the distillation training at the current stage is stopped, and the trained student network model is output.

7. A face forgery detection device based on knowledge distillation, used to implement the steps of the face forgery detection method based on knowledge distillation as described in any one of claims 1 to 6, characterized in that, include: The data module is used to acquire forged images containing signals from damaged watermarks; The detection module is used to perceive the forged regions in the forged image using the teacher network model and generate an attention distribution indicating the forged regions. The teacher network model is then pre-trained using the forged regions to obtain a pre-trained teacher network model. The pre-trained teacher network model is used to perform knowledge distillation on the student network model so that the student network model learns and imitates the attention distribution generated by the pre-trained teacher network model to obtain a trained student network model. The image to be detected is input into the trained student network model, which outputs the true / false classification result of the image to be detected.

8. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the face forgery detection method based on knowledge distillation as described in any one of claims 1 to 6.

Images