Generalized deep forgery detection method based on multi-classification guidance

Abstract

The invention discloses a generalizable deep forgery detection method based on multi-classification guidance, and belongs to the technical field of image processing. According to the technical scheme, data enhancement is carried out on an image, and then the image is input into a binary classification network and a multi-classification network of a double-flow neural network; combining the features of the two branches through a fusion module; the fused features are input into a binary classifier to determine whether the input is authentic or counterfeited. The method has the beneficial effects that the generalizable deep counterfeiting detection method based on multi-classification guidance shows remarkable beneficial effects in the aspects of improving the model generalization ability, enhancing the feature expression ability, optimizing the loss function design, preventing over-fitting, improving the calculation efficiency, enhancing the label consistency and the like; the method is beneficial to suppressing the propagation of false information, maintaining the right to know and security of the public, enhancing the vigilance of people to the authenticity of information, laying a solid foundation for the applications of identity authentication, digital forensics and the like in multiple fields, and making an important contribution to the construction of a safer and more credible information society.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and specifically relates to a generalizable deep fake detection method based on multi-classification guidance. Background Art

[0002] With the rapid development of deep learning technology, deepfake technology has become increasingly mature. How to effectively detect these high-quality forged contents has become an important issue that needs to be solved urgently. At present, a variety of deepfake detection methods have been proposed in academia and industry. Most of the existing face image forgery detection methods use advanced binary classification models, focusing on identifying and classifying specific forgery patterns, such as noise patterns, local textures, frequency information, and implicit identities. Although these methods have shown good results, their performance depends heavily on prior knowledge of specific forgery patterns. This dependence brings great challenges when dealing with unknown or novel forgery patterns. The generalization ability of the model is poor, which limits their effectiveness in real-world scenarios where forgeries are constantly evolving.

[0003] With the rapid development of deep fake technology, the detection of high-quality fake content has become a pressing issue. Existing technologies have been further improved and perfected in terms of improving model generalization ability, optimizing feature selection, reducing computational complexity, solving data imbalance problems, and protecting privacy and security. Summary of the invention

[0004] In order to solve the technical problems existing in the above-mentioned prior art, the present invention provides a generalizable deep fake detection method based on multi-classification guidance. The detection method starts from a novel perspective, fully explores the differences between different types of fakes, increases the distinction between subcategories, and thus improves the generalization range.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A generalizable deepfake detection method based on multi-classification guidance, the steps are as follows:

[0007] S1, the image is data enhanced and then input into the binary classification network and multi-classification network of the two-stream neural network respectively;

[0008] S2, combining the features of the two branches through the fusion module;

[0009] S3. The fused features are input into a binary classifier to determine whether the input is real or fake.

[0010] Furthermore, in step S1, in the first stage, a multivariate classifier is used to distinguish between authenticity and different types of forgeries; in the second stage, the classification results of the multivariate classifier are used as a guide to train a binary classifier for detecting "authentic" or "forgery".

[0011] Furthermore, the two-stream neural network is detected by a binary classification loss function and a multi-classification loss function; the consistency between the binary classification and multi-classification predictions at the true and false label levels is ensured by a label loss function; the binary classification loss function, the multi-classification loss function, and the label loss function are synthesized by a weighted sum to create an overall loss function for the training framework.

[0012] Furthermore, the freezing mechanism is used to freeze the multi-classification model in an adjustable manner to prevent the model from overfitting.

[0013] Furthermore, in step S1, the features extracted by the SeparableConv 2048 layer of the binary classification and multi-classification networks are input into the fusion module; the fused features are passed through the ReLU activation layer and the global average pooling layer, and then through the fully connected layer to obtain the classification result.

[0014] Furthermore, in step S2, after the input image passes through the binary classification network and the multi-classification network, the binary classification network features F and binary and multi-classification network features F multi ; Both dimensions are H×W×C, where H×W represents the spatial dimension and C represents the number of channels; perform the following steps:

[0015] S2.1. Connect the feature maps of the two branches together, with the spatial dimension unchanged and the number of channels doubled to obtain a feature map with a dimension of H×W×2C.

[0016] S2.2, divided into two branches for operation, one branch passes through two 3×3 convolutional layers in succession to learn the nonlinear mapping of input features; the other branch passes through a 1×1 convolutional layer;

[0017] S2.3, add the input features to the output of the convolutional layer;

[0018] S2.4. Add the features of the two branches by simple addition operation to obtain the fusion feature F out ; The dimension of the fused feature is still H×W×C, which is consistent with the dimension of the two input features. The fused feature is then fed into the backbone network for end-to-end training.

[0019] Furthermore, the binary classification loss function is as follows:

[0020] The binary classification network uses AM-Softmax Loss as the objective function, as shown in Formula 1:

[0021]

[0022] Where N represents the number of training samples, y i represents the true category of the i-th sample, s is the scale parameter, Indicates that it belongs to the true category y i The angle between the feature vector of the i-th sample and the weight vector, θ j represents the angle between the feature vector of the i-th sample and the weight vector of the non-true category j, m is an auxiliary parameter that controls the angle interval between categories; cos(·) represents the cosine similarity function, e (·) Represents an exponential function.

[0023] Furthermore, the multi-classification loss function is as follows:

[0024] Focal Loss is used as the loss function in multi-classification tasks, as shown in Formula 2 and Formula 3:

[0025]

[0026] L multi =-a t (1-p t ) γ log(p t ) (3)

[0027] Among them, y = 1 represents a false label, p t is the prediction corresponding to the label, a t is the balance weight and γ is the focus adjustment factor.

[0028] Furthermore, the label loss function is as follows:

[0029] assumed is a training set generated by k (k>1) forgeries, which contains images and the corresponding label Y i ∈{0,1,2,...,k}, where 0 represents the real image and the others represent the forged images with different forgery methods;

[0030] When performing binary classification, the labels are transformed.

[0031]

[0032] Get the binary label Y j ∈{0,1};

[0033] In the training phase, it is assumed that the predicted value of the multi-classification is P i∈{0,1,2,...,k}, the predicted value of the two-class classification is P j ∈{0,1}; transform the multi-classification prediction value to the true and false dimension,

[0034]

[0035] Then, take the absolute value of the multi-classification prediction value and the binary classification prediction value as the label loss:

[0036] L label =|P j -P j '|(6).

[0037] Furthermore, the freezing mechanism is: updating the parameters of the multi-classification network in the first half of the training, and freezing the parameters of the multi-classification network in the second half of the training.

[0038] Beneficial effects of the present invention:

[0039] Compared with the prior art, the generalizable deep fake detection method based on multi-classification guidance described in the present invention exhibits excellent generalization ability by utilizing the guidance provided by multi-classification features; it can effectively adapt to previously unseen types of fakes and can accurately detect them even when the operation mode is unfamiliar, with a cross-database detection accuracy of up to 96.34% on the DFD dataset; this ability to detect unknown patterns is a significant improvement over traditional methods that focus mainly on known fake patterns; at the social level, it helps to curb the spread of false information, safeguard the public's right to know and safety, enhance people's vigilance about the authenticity of information, and protect personal privacy and rights; at the economic level, it can reduce the economic losses caused by false information, provide technical support for digital content copyright protection, and improve the transparency and efficiency of economic operations; at the technical level, it will promote the innovative development of cutting-edge technologies such as deep learning and computer vision, lay a solid foundation for applications such as identity authentication and digital forensics in multiple fields, and make important contributions to building a more secure and trustworthy information society.

[0040] In addition, the present invention also has the following significant advantages and technical innovations:

[0041] (1) Improving generalization ability: By introducing a multi-classification network as a guide, it can not only identify the real and the forged, but also distinguish different types of forgeries. This method can learn richer forgery features, thereby improving the generalization ability of the model when facing unknown or novel forgery techniques.

[0042] (2) Enhanced feature expression capability: The combination of two-stream neural network structures (binary classification network and multi-classification network) can extract features at different levels and effectively combine these features through the fusion module, making the features ultimately used for classification more expressive and discriminative.

[0043] (3) Optimize the loss function design: The weighted sum of the binary classification loss function, the multi-classification loss function, and the label loss function is used as the overall loss function, which not only ensures the accuracy of the classification, but also ensures the consistency between the true and false label level predictions, which helps to improve the stability and robustness of the model.

[0044] (4) Preventing overfitting: The freezing mechanism can be used to adjustably freeze the multi-classification model at different stages of training, which can effectively control the complexity of the model and prevent overfitting while maintaining the model's sensitivity and accuracy to spurious features.

[0045] (5) Improve computational efficiency: In the fusion module, effective fusion of features is achieved through specific convolutional layers and addition operations while keeping the feature dimensions unchanged, which helps reduce the amount of computation and improve the real-time performance of the model.

[0046] (6) Flexible loss function selection: According to the characteristics of binary classification and multi-classification tasks, AM-Softmax Loss and Focal Loss are used as loss functions respectively. These two loss functions are suitable for different classification scenarios and help improve the classification performance of the model.

[0047] (7) Enhanced label consistency: Through the design of the label loss function, the consistency of the multi-classification prediction value with the binary classification prediction value in the true and false dimension is ensured, which helps to improve the accuracy of the model in the real and fake classification tasks.

[0048] In summary, the generalizable deep fake detection method based on multi-classification guidance of the present invention shows significant beneficial effects in improving model generalization ability, enhancing feature expression ability, optimizing loss function design, preventing overfitting, improving computational efficiency, and enhancing label consistency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0050] Figure 1 It is the overall framework diagram of the M2B method of the present invention;

[0051] Figure 2 It is a framework diagram of the backbone network (i.e., the improved Xception network) in the present invention;

[0052] Figure 3It is a structural block diagram of the fusion module in the present invention;

[0053] Figure 4 Example images of three perturbations with five different severities.

[0054] Figure 5 The figure shows the results of the robustness experiment in the face of unknown disturbances. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Figure 1-5 The generalizable deep fake detection method based on multi-classification guidance is further explained.

[0056] Unlike existing methods that rely on specific forgery patterns, the present invention starts from a novel perspective, fully explores the differences between different forgery types, increases the distinction between subcategories, and thus improves the generalization range. Specifically, the present invention is a two-stream neural network, which includes two branches, multi-classification and binary classification, and adopts a two-stage approach. In the first stage, a multi-class classifier is used to distinguish between real and different forgery types. For example, in the FaceForensics++ dataset, there are four forgery methods, DF, F2F, FS, and NT, so the multi-class classifier performs a five-element classification of "real", "DF", "F2F", "FS", and "NT". In the second stage, the classification results of these multi-classifiers are used as guidance to train a binary classifier that detects "real" or "forgery". This guidance is achieved through a complex fusion mechanism composed of multiple convolutional layers, which can effectively capture the features of multi-classification. In addition, the present invention proposes a freezing mechanism that enables the multi-classification model to be frozen in an adjustable manner to prevent the model from overfitting, and to use the features learned from the multi-classification stage to improve the performance of the binary classification model. In order to better synchronize the multi-classification and binary classification models, the present invention introduces a label loss specifically customized for the network structure. This loss function enables the multi-classification model to better guide the binary classification model.

[0057] Specifically, in order to solve the problem that the previous methods have poor performance when encountering unknown forgery patterns, this paper proposes a deep forgery detection method based on multi-classification guidance, called the Multi-To-Binary (M2B) method. The complete M2B method framework is shown in the figure below. Figure 1As shown. M2B is a two-stream network that includes binary classification and multi-classification. In the training phase, the image is first data augmented and input into the binary classification network and the multi-classification network respectively. Both networks use the same improved Xception architecture to extract features. The features of the two branches are then combined through the fusion module. In the inference phase, the classifier head of the multi-classification branch is discarded, and the fused features are only input into the binary classifier to determine whether the input is real or forged. This process ensures that the binary classifier benefits from the rich feature extraction capabilities of the multi-classification model, thereby enhancing its ability to generalize to unknown forged patterns.

[0058] Backbone network:

[0059] In order to obtain comprehensive features and perform effective fusion to improve the generalization ability of the network, the present invention proposes an improved Xception network as the backbone network, such as Figure 2 As shown. The present invention uses the same entry layer and intermediate layer as Xception, and only changes the exit layer. The present invention inputs the features extracted by the SeparableConv 2048 layer of the binary classification and multi-classification networks into the fusion module. Then, the fused features pass through the ReLU activation layer and the global average pooling layer, and then pass through the fully connected layer to obtain the classification result. During the training phase, the differences between the binary classification network and the multi-classification network are eliminated, thereby improving the generalization ability of the network.

[0060] Fusion Module:

[0061] After the input image passes through the binary classification and multi-classification networks, the binary classification network features F binary and multi-classification network features F multi Next, the two features need to be fused. In the fusion stage, in order to make the two features more fully fused and to prevent one feature from suppressing or interfering with another feature, the present invention does not simply add, multiply or connect the features of the two branches, but designs a fusion network to achieve full fusion. Figure 3 As shown, the fusion module has two inputs: the feature map F from the binary classification network binary and the feature map F from the multi-classification network multi, their dimensions are both H×W×C, where H×W represents the spatial dimension and C represents the number of channels. First, the feature maps of the two branches are connected together so that the spatial dimension remains unchanged and the number of channels becomes twice as large as before, resulting in a feature map with a dimension of H×W×2C. Then, it is divided into two branches for operation, which is inspired by the ResNet network. One of the branches passes through two 3×3 convolutional layers in succession to learn the nonlinear mapping of the input features. The other branch passes through a 1×1 convolutional layer to match the output dimension of the other branch. The idea of ​​skip connection is adopted here to add the input features directly to the output of the convolutional layer. This connection method allows information to skip some layers in the network and pass it directly to subsequent layers, thereby avoiding information loss and degradation. Finally, the features of the two branches are added together through a simple addition operation to obtain the fused feature F out The dimension of the fused features is still H×W×C, which is consistent with the dimension of the two input features. The fused features are then fed into the backbone network for end-to-end training.

[0062] Loss function:

[0063] In order to implement the M2B framework for detection, the present invention designs three different loss functions: binary classification loss, multi-classification loss and label loss, where the label loss ensures the consistency between binary and multi-classification predictions at the true and false label levels. These losses are combined into a weighted sum to create an overall loss function for the training framework.

[0064] 1) Binary Classification Loss

[0065] The binary classification network uses AM-Softmax Loss as the objective function, as shown in Formula 1.

[0066] Compared with regular cross entropy, it can make the intra-class variation smaller and the inter-class variation larger.

[0067]

[0068] Where N represents the number of training samples, y i represents the true category of the i-th sample, s is the scale parameter, Indicates that it belongs to the true category y i The angle between the feature vector of the i-th sample and the weight vector, θ j represents the angle between the feature vector of the i-th sample and the weight vector of the non-true category j, and m is an auxiliary parameter that controls the angle interval between categories. cos(·) represents the cosine similarity function, e (·) Represents an exponential function.

[0069] 2) Multi-classification loss

[0070] Focal Loss is used as the loss function in multi-classification tasks to reduce the weight of well-learned samples, thereby further improving the performance of the model, as shown in Formula 2 and Formula 3.

[0071]

[0072] L multi =-a t (1-p t ) γ log(p t ) (3)

[0073] Among them, y = 1 represents a false label, p t is the prediction corresponding to the label, a t is the balance weight and γ is the focus adjustment factor.

[0074] 3) Label loss

[0075] assumed is a training set generated by k (k>1) forgery methods, which contains images and the corresponding label Y i ∈{0,1,2,...,k}, where 0 represents a real image and the others represent forged images of different forging methods. For example, the training set of the method of the present invention selects the FaceForensics++ dataset, in which the forged images are generated by four forging methods: Deepfake, Face2Face, FaceSwap, and NeuralTexture. Therefore, k=4.

[0076] When performing binary classification, the labels are transformed.

[0077]

[0078] In this way, the present invention obtains the binary classification label Y j ∈{0,1}.

[0079] In the training phase, it is assumed that the predicted value of the multi-classification is P i ∈{0,1,2,...,k}, the predicted value of the two-class classification is P j ∈{0,1}. In order to ensure the consistency of the prediction values ​​of multi-classification and binary classification in the true and false dimensions, the present invention first transforms the multi-classification prediction values ​​into the true and false dimensions.

[0080]

[0081] Then, take the absolute value of the multi-classification prediction value and the binary classification prediction value as the label loss

[0082] L label=|P j -P j '| (6)

[0083] 4) Overall loss

[0084] The final loss function of the training process is the weighted sum of the above loss functions.

[0085] L total =L binary +λ1L multi +λ2L label (7)

[0086] Wherein, λ1 and λ2 are hyperparameters for balancing the overall loss. In this method, the present invention sets the value of λ1 to 0.1 and the value of λ2 to 1.

[0087] Freezing mechanism:

[0088] In addition, in order to prevent overfitting of the learning of the multi-classification network, the present invention introduces a freezing mechanism. The parameters of the multi-classification network are updated in the first half of the training, and the parameters of the multi-classification network are frozen in the second half of the training. In this way, the binary classification network can fully learn the knowledge provided to it by the multi-classification network in the first half of the training, and focus on improving the performance of the binary classification network in the second half.

[0089] Experimental setup and results:

[0090] To evaluate the generalization ability, the present invention conducts experiments on four large-scale benchmark datasets, namely FaceForensics++ (FF++), Celeb-DF, DeepFakeDetection (DFD), and DeeperForensics-1.0 (DF-1.0). Their detailed specifications are shown in Table 1. Due to the diversity of forgeries, the present invention uses FF++ as a training dataset. This is a large dataset containing 5,000 videos, including four types of face processing: Deepfake (DF), Face2Face (F2F), FaceSwap (FS), and NeuralTexture (NT). The method performance is evaluated on Celeb-DF, DFD, and DF-1.0.

[0091] Table 1 Specifications of the benchmark datasets

[0092] Dataset Video scale Fake Algorithm FF++ 1,000 real videos, 4,000 fake videos DF,F2F,FS,NT Celeb-DF 408 real videos, 795 fake videos Improved DF DFD 363 real videos, 3068 fake videos Improved DF DF-1.0 11,000 fake videos DF-VAE

[0093] Given the rapid development of forgery generation techniques, it is crucial to develop a detector that can accurately classify samples from novel and previously unseen forgery methods. To simulate this situation, the goal of this paper is to train the detector using a set of known forgery samples and evaluate its performance on unfamiliar and unknown forgery techniques. Table 2 shows the accuracy (ACC) and average precision (AP) values ​​when trained on the FF++ dataset and tested on the Celeb-DF, DFD, and DF-1.0 datasets. The "Average" column represents the average across all datasets. The best results are highlighted in bold, while the second-best results are underlined. M2B outperforms other methods on most datasets. On the Celeb-DF dataset, the present invention achieves a best AP of 87.65%. On the DF-1.0 dataset, the present invention achieves an ACC of 79.91% and a best AP of 88.83%. On average, M2B achieves the best performance with an average ACC of 79.91%, 1.27% higher than the second-place result. In addition, the present invention achieves an average AP of 90.94%, 3.73% higher than the second-place result. These results show that our approach consistently outperforms other methods, performing well in both accuracy and precision, which provides hope for future improvements in its performance on forgery. While our approach falls slightly short of the best method on the DFD dataset, it remains the average best choice and achieves the best overall generalization performance. These findings provide additional strong evidence that exploring the distribution of different forgery techniques is critical for generalization.

[0094] Given the ubiquity of image manipulation operations on social media, deployed deepfake detectors must be robust to common perturbations. We evaluate the robustness of our detectors by training them on the FF++ dataset and testing them on a variety of unseen corrupted DFD samples. We consider five severity levels of the following operations: saturation change, contrast change, and blockiness. Figure 4 Examples of each type of corruption at five different severities are shown. In Table 3, the mean ACC score changes on the DFD dataset (compared to the clean data) when the images are subjected to various corruptions are provided, averaged over all severity levels. “Average” means the average of all corruptions (and all severity levels). The ACC of M2B varies within 1.2% under each perturbation, with an average change of 0.37%. Some methods show significant fluctuations for certain perturbations, for example, contrast changes reduce SBI by 33.73%, and blocky distortion reduces MADD and RECCE by 20.89% and 11.78%, respectively. It is obvious that the method of the present invention shows higher robustness to most perturbations compared to other methods. Figure 5In the example, the present invention shows the effect of increasing the severity of each disturbance. Since different methods have different initial detection accuracy for clean data, the present invention focuses on the smoothness of the curve rather than the value. According to the smoothness of the curve, the method of the present invention remains relatively stable under various disturbances, indicating that it has excellent robustness.

[0095] Table 2 Generalization test across datasets

[0096]

[0097] Table 3 Robustness test against unknown disturbances

[0098]

[0099] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A generalizable deep fake detection method based on multi-classification guidance, characterized in that: Here are the steps: S1, the image is data enhanced and then input into the binary classification network and multi-classification network of the two-stream neural network respectively; S2, combining the features of the two branches through the fusion module; S3. The fused features are input into a binary classifier to determine whether the input is real or fake.

2. The generalizable deep fake detection method based on multi-classification guidance as claimed in claim 1, characterized in that: In the step S1, in the first stage, a multivariate classifier is used to distinguish between the real and different types of forgeries; In the second stage, the classification results of the multivariate classifier are used as guidance to train a binary classifier for detecting "real" or "fake".

3. The generalizable deep fake detection method based on multi-classification guidance as claimed in claim 1, characterized in that: The two-stream neural network is detected by a binary classification loss function and a multi-classification loss function; the consistency between the binary classification and multi-classification predictions at the true and false label levels is ensured by a label loss function; the binary classification loss function, the multi-classification loss function, and the label loss function are synthesized by weighted sum to create an overall loss function for the training framework.

4. The generalizable deep fake detection method based on multi-classification guidance according to any one of claims 1 to 3, characterized in that: The freezing mechanism freezes the multi-classification model in an adjustable way to prevent the model from overfitting.

5. The generalizable deep fake detection method based on multi-classification guidance as claimed in claim 1, characterized in that: In the step S1, the features extracted by the SeparableConv2048 layer of the binary classification and multi-classification networks are input into the fusion module; the fused features are passed through the ReLU activation layer and the global average pooling layer, and then through the fully connected layer to obtain the classification result.

6. The generalizable deep fake detection method based on multi-classification guidance as claimed in claim 1, characterized in that: In step S2, after the input image passes through the binary classification network and the multi-classification network, the binary classification network features F binary and multi-classification network features F multi ; Both dimensions are H×W×C, where H×W represents the spatial dimension and C represents the number of channels; perform the following steps: S2.

1. Connect the feature maps of the two branches together, with the spatial dimension unchanged and the number of channels doubled to obtain a feature map with a dimension of H×W×2C. S2.2, divided into two branches for operation, one of which passes through two 3×3 convolutional layers continuously to learn the nonlinear mapping of input features; The other branch passes through a 1×1 convolutional layer; S2.3, add the input features to the output of the convolutional layer; S2.

4. Add the features of the two branches by simple addition operation to obtain the fusion feature F out ; The dimension of the fused features is still H×W×C, which is consistent with the dimension of the two input features. The fused features are then fed into the backbone network for end-to-end training.

7. The generalizable deep fake detection method based on multi-classification guidance as claimed in claim 3, characterized in that: The binary classification loss function is as follows: The binary classification network uses AM-Softmax Loss as the objective function, as shown in Formula 1: Where N represents the number of training samples, y i represents the true category of the i-th sample, s is the scale parameter, Indicates that it belongs to the true category y i The angle between the feature vector of the i-th sample and the weight vector, θ j represents the angle between the feature vector of the i-th sample and the weight vector of the non-true category j, m is an auxiliary parameter that controls the angle interval between categories; cos(·) represents the cosine similarity function, e (·) Represents an exponential function.

8. The generalizable deep fake detection method based on multi-classification guidance as claimed in claim 3, characterized in that: The multi-classification loss function is as follows: Focal Loss is used as the loss function in multi-classification tasks, as shown in Formula 2 and Formula 3: L multi =-a t (1-p t ) γ log(p t ) (3) Among them, y = 1 represents a false label, p t is the prediction corresponding to the label, a t is the balance weight and γ is the focus adjustment factor.

9. The generalizable deep fake detection method based on multi-classification guidance as claimed in claim 3, characterized in that: The label loss function is as follows: assumed is a training set generated by k (k>1) forgeries, which contains images and the corresponding label Y i ∈{0,1,2,...,k}, where 0 represents the real image and the others represent the forged images with different forgery methods; When performing binary classification, the labels are transformed. Get the binary label Y j ∈{0,1}; In the training phase, it is assumed that the predicted value of the multi-classification is P i ∈{0,1,2,...,k}, the predicted value of the two-class classification is P j ∈{0,1}; transform the multi-classification prediction value to the true and false dimension, Then, take the absolute value of the multi-classification prediction value and the binary classification prediction value as the label loss: L label =|P j -P j '|(6)。 10. The generalizable deep fake detection method based on multi-classification guidance as claimed in claim 4, characterized in that: The freezing mechanism is: updating the parameters of the multi-classification network in the first half of the training, and freezing the parameters of the multi-classification network in the second half of the training.

Images