Deep counterfeit image detection method based on knowledge distillation and domain adversarial training

Abstract

The invention relates to the field of image detection, and provides a deep counterfeit image detection method based on knowledge distillation and domain adversarial training, which comprises the following steps: constructing a source domain data set and a target domain data set; performing model source domain training by using the source domain data set to obtain a trained basic model; performing model migration training based on knowledge distillation and domain adversarial training by using the source domain data set, the target domain data set and the trained basic model to obtain a deep counterfeit image detection model; and when a new type of deep counterfeit image appears and the data volume of the labeled data of the new type of deep counterfeit image cannot be subjected to domain adversarial training, performing fine adjustment on the deep counterfeit image detection model by adopting the labeled data of the new type of deep counterfeit image. According to the method, different transfer learning methods are combined, on one hand, the detection accuracy of a deep counterfeit image detection model on a counterfeit image of a known type is improved, and on the other hand, the condition that a very small amount of new-type label data appears is quickly adapted.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image detection, and in particular, to a deep fake image detection method based on knowledge distillation and domain adversarial training. BACKGROUND

[0002] With the development of generative artificial intelligence technology, deep fake technology has become one of the most popular multimedia tampering technologies. For example, FaceSwap replaces the face in an image or video with a face from a different image or video set; Face2Face transfers the expression of a source image or video to a target image or video while preserving the identity of the target person. These technologies can be easily used for false propaganda images or videos, and are also easily exploited by fraudsters, causing irreparable losses. Researching deep fake image detection can effectively prevent the technology from being misused, and is of great significance.

[0003] Deep fake image detection is usually defined as a binary classification task that predicts whether a given image is real or fake. Existing detection methods mainly use traditional handcrafted features, utilize unique biometric features of faces, and use hierarchical feature representations learned automatically by deep convolutional neural networks. Although these methods have achieved certain detection performance, the following two problems are prevalent: First, there are many methods of deep fake, such as the four types of deep fake data in the commonly used academic dataset FaceForensics++, including DeepFake, Face2Face, Faceswap, and NeuralTextures. Typical deep neural network models are trained on datasets that combine multiple types of deep fake images, often failing to achieve good detection results, and the detection results for various types of fake are uneven. The possible reason is that the excessive diversity within the dataset makes it difficult for the model to distinguish the key differences between fake images and real images; Second, the generalization ability of the detection model is poor, that is, when a new type of deep fake image is presented, even if it is semantically similar, the detection performance of the model will be greatly affected. This is mainly because the deep neural network may overfit to the useful features for a specific task, so it cannot be transferred to detect deep fake images generated using different techniques.

[0004] To solve the above two problems, the academic community has done a lot of beneficial research, one of the more typical ways is to solve through transfer learning. Research found that when there are multiple deepfake image data in the training data set, compared with merging training, by first training the detection model on a deepfake image data set, the detection model obtains a detailed understanding of the differences between the true and false images of a single deepfake type, and then uses transfer learning to expand its knowledge about different types of deepfake images, better detection results can be achieved. In order to improve the detection ability of new types of deepfake images through transfer learning, the more typical way is to use model (local) fine-tuning migration, model fine-tuning migration based on feature supervision (parameter regularization, feature constraint, knowledge distillation), and domain adaptation methods. Although these methods can transfer knowledge to the target domain to some extent based on relatively less target domain data, current research usually involves single transfer learning from the source domain to the target domain, which still faces the problem of catastrophic forgetting, that is, when the model is transferred to the target domain, it may forget the knowledge of the source domain, resulting in a significant decline in detection performance. SUMMARY

[0005] The present application aims to provide a deepfake image detection method based on knowledge distillation and domain adversarial training to solve the above problems existing in current deepfake image detection.

[0006] The present application provides a deepfake image detection method based on knowledge distillation and domain adversarial training, comprising: Constructing a data set, including a source domain data set and a target domain data set; Using the source domain data set to perform model source domain training to obtain a trained basic model; Using the source domain data set and the target domain data set and the trained basic model, performing model transfer training based on knowledge distillation and domain adversarial training to obtain a deepfake image detection model; the deepfake image detection model is used for deepfake image detection; When a new type of deepfake image appears and the amount of labeled data of the new type of deepfake image cannot perform domain adversarial training, fine-tuning the deepfake image detection model using the labeled data of the new type of deepfake image.

[0007] In a preferred embodiment, the model source domain training using the source domain data set comprises: Selecting a basic model; Using the basic model to perform true and false classification training on the source domain data set to obtain a trained basic model; the true and false classification loss function for true and false classification training is a classification cross-entropy loss.

[0008] In a preferred embodiment, the method of performing model transfer training based on knowledge distillation and domain adversarial training using the source domain dataset, the target domain dataset, and the trained basic model includes: Build teacher model and student model based on the trained basic model; The teacher model and the student model are trained on the source domain dataset and the target domain dataset through model transfer based on knowledge distillation and domain adversarial training to obtain a deep fake image detection model.

[0009] In a preferred embodiment, the construction of the teacher model and the student model based on the trained basic model includes: Divide the trained base model into connected feature extractors F and true / false classifier C ; Copy the trained base model and freeze its parameters as the teacher model; Feature extractor of the trained base model F Add domain classifier later D Then it is used as the student model; among them, the domain classifier D Do nothing during forward propagation and update the feature extractor during backward propagation F The gradient needs to be reversed when the parameters are

[0010] In a preferred embodiment, the loss function for model transfer training based on knowledge distillation and domain adversarial training includes classification cross entropy loss, domain adversarial training loss, feature representation loss, and knowledge distillation loss.

[0011] In a preferred embodiment, the loss function for fine-tuning the deep fake image detection model includes classification cross entropy loss, feature representation loss, and knowledge distillation loss.

[0012] In a preferred embodiment, the categorical cross entropy loss Expressed as:

[0013] in, N is the number of samples input to the model, It is i The authenticity labels of samples, It is i The predicted probability of a sample.

[0014] In a preferred embodiment, the domain adversarial training loss Expressed as:

[0015] in, xi, i = 1, 2,..., N, N is the number of samples of the input model, is the domain label of the i-th sample, i F is a feature extractor, is the feature extracted from the i-th sample, is a gradient reversal function, whose input is the feature extracted from the i-th sample, is a domain classifier, whose input is the output of the gradient reversal function.

[0016] In a preferred embodiment, the feature representation loss is defined as:

[0017] where, is the maximum value in the vector of the softmax normalized image feature on the student model corresponding image feature; is the maximum value in the vector of the softmax normalized image feature on the teacher model corresponding image feature; the value range of the vector of the softmax normalized image feature on the student model and the teacher model is , and the value interval is v.

[0018] In a preferred embodiment, the knowledge distillation loss is:

[0019]

[0020]

[0021] where, is the temperatureized softmax function in the student model, is the temperatureized softmax function in the teacher model; is the temperature, N is the number of samples of the input model, is the unnormalized score of the i-th sample by the student model, i is the unnormalized score of the i-th sample by the student model, is the unnormalized score of the i-th sample by the teacher model, j is the unnormalized score of the i-th sample by the teacher model, is the unnormalized score of the i-th sample by the teacher model, i is the unnormalized score of the i-th sample by the teacher model, is the unnormalized score of the i-th sample by the teacher model, j ​​​​un-normalized scores of the samples; is an exponential function with the natural constant e as the base.

[0022] In summary, due to the adoption of the technical solutions described above, the present application has the following beneficial effects: By skillfully combining the advantages of different transfer learning methods, the present application improves the detection accuracy of the deep fake image detection model for known types of fake images through a unified model framework, and quickly adapts to the situation where a small amount of new type labeled data appears. Specifically, during the model transfer training process, the domain adversarial training is fully utilized to extract general features, and the stability of the domain adversarial training is improved through the supervision of knowledge distillation, thereby improving the detection accuracy of the model for known types of fake images. In the model usage, if a new type appears and only a small amount of labeled data is available, the model can also be fine-tuned under the supervision of knowledge distillation to quickly adapt to the new type of data and avoid overfitting. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A flowchart of a deep fake image detection method based on knowledge distillation and domain adversarial training is provided for the embodiments of the present application.

[0024] Figure 2 A principle diagram of model transfer training based on knowledge distillation and domain adversarial training is provided for the embodiments of the present application.

[0025] Figure 3 A structural schematic diagram of an electronic device is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts fall within the scope of protection of the present application.

[0028] There are two problems in current deepfake image detection: one is that the single model training based on the combination of multiple types of fake data often does not achieve good detection effect; the second is that the generalization ability of the detection model is poor when new types of deepfake images are presented. But there are differences between the data of the two, the model training stage is known deepfake type, the data has labels and the data is more; When dealing with new types of deepfake images, there may be few labeled data, or there may be more, and there may also be unlabeled data. Research shows that transfer learning may be a way to solve these two problems. But the current deepfake image detection research based on transfer learning only covers single transfer learning from source domain to target domain, still facing the problem of catastrophic forgetting, and the training of detection model for multiple types of deepfake images through transfer learning, or to adapt to the constantly emerging new types of deepfake images, all need to do multiple target domain transfer learning on the source domain model, which puts forward higher challenges to realize knowledge transfer while avoiding catastrophic forgetting.

[0029] Therefore, the present application widely combines the advantages of existing transfer learning methods, proposes a deepfake image detection method based on knowledge distillation and domain adversarial training, and expects to solve the above problems in current deepfake image detection with a unified model framework. The design principle of the method of the present application is as follows: (1) Model fine-tuning transfer is a commonly used technique in transfer learning, which is to adjust part or all parameters of the source domain model to adapt the model to the target domain data. Model fine-tuning is suitable for the case of labeled data, and the required training time is less, which can be quickly adapted to new type data through incremental learning. But for the case of only a small amount of labeled data of new type, the fine-tuning method needs to be carefully designed, otherwise it is easy to fall into a local suboptimal solution, overfitting to the target domain specific features, while destroying the general features learned from the source domain, so model fine-tuning is not suitable for multiple target domain transfer learning. Model fine-tuning usually needs some feature supervision (parameter regularization, feature constraint, knowledge distillation) to reduce the forgetting of source domain features as much as possible.

[0030] (2) Domain adversarial training distinguishes features from source domain or target domain by using domain classifier, and the backbone network tries to learn features that are useful for the task and domain-invariant. Adversarial training makes the backbone network reduce the distribution difference between the source domain and the target domain while learning feature representation, so that the domain classifier is difficult to distinguish the two domains. Domain adversarial training can simultaneously utilize a small amount of labeled data and a large amount of unlabeled data. Domain adversarial training eliminates the inter-domain difference features by confusing the features of the domain discriminator, thereby improving the transfer effect. At the same time, because the source domain data is involved in training during training, it is not easy to destroy the general features learned by the source domain. By extending the domain adversarial training, it may be more suitable for multiple target domain transfer learning. However, domain adversarial training is unstable, the training time is long, and it cannot quickly adapt to new types of fake data. The deep fake image detection method based on knowledge distillation and domain adversarial training expects: during model training, make full use of the ability of domain adversarial training to extract general features, and at the same time, through the supervision of knowledge distillation, improve the stability of domain adversarial training, so as to improve the detection accuracy of the model on known fake images; in the use of the model, if a new image fake type appears, and only a small amount of labeled data is available, the model can also be fine-tuned under the supervision of knowledge distillation to quickly adapt to new types of data.

[0031] Based on the above design principles, as shown in Figure 1 The embodiment of the application provides a deep fake image detection method based on knowledge distillation and domain adversarial training, which comprises the following steps: S100, constructing a data set, comprising a source domain data set and a target domain data set; S200, using the source domain data set to perform model source domain training to obtain a trained basic model; S300, using the source domain data set and the target domain data set and the trained basic model, performing model transfer training based on knowledge distillation and domain adversarial training to obtain a deep fake image detection model; the deep fake image detection model is used for deep fake image detection; S400, when a new type of deep fake image appears and the amount of labeled data of the new type of deep fake image cannot be used for domain adversarial training, the labeled data of the new type of deep fake image is used to fine-tune the deep fake image detection model.

[0032] Therefore, the present invention cleverly combines the advantages of different transfer learning methods and uses a unified model framework to improve the detection accuracy of the deep fake image detection model for known types of fake images on the one hand, and quickly adapt to the situation where a very small amount of new types of labeled data appear on the other hand. Specifically, during the model transfer training process, the ability of domain adversarial training to extract common features is fully utilized, and at the same time, the stability of domain adversarial training is improved through the supervision of knowledge distillation, thereby improving the model's detection accuracy for known types of fake images; during model use, if a new type appears and there is only a very small amount of labeled data, the model can also be fine-tuned under the supervision of knowledge distillation to quickly adapt to the new type of data to avoid overfitting.

[0033] The following describes in detail the specific implementation of the above-mentioned deep fake image detection method based on knowledge distillation and domain adversarial training.

[0034] S100, build a dataset, including the source domain dataset and the target domain dataset: In this embodiment, the real and fake videos in the FaceForensics++ dataset are processed to form source and destination domain datasets. Specifically, the four deepfake videos (DeepFake, Face2Face, Faceswap, and NeuralTextures) and real videos in the FaceForensics++ dataset are extracted using the FFmpeg library. The faces in each extracted frame are then cropped using the MTCNN (Multi-task Cascaded Convolutional Networks) algorithm, forming four deepfake image datasets containing both real and fake images. From these four deepfake image datasets, one deepfake image dataset is selected as the source domain dataset, such as the DeepFake dataset, and another deepfake image dataset is selected as the destination domain dataset, such as the Face2Face dataset.

[0035] S200: Use the source domain dataset to perform source domain training on the model to obtain a trained basic model: S201, select a basic model. In this embodiment, the basic model selected is the Xception network as an example.

[0036] S202, using the basic model ( Figure 2 The purple part in the middle) is used to perform true-false classification training on the source domain dataset to obtain a trained basic model. The true-false classification loss function for true-false classification training is the classification cross entropy loss , defined as follows:

[0037] wherein, N is the number of samples inputted into the model, is the authenticity label of the i-th sample, i is the authenticity label of the i-th sample, is the predicted probability of the i-th sample. i is the predicted probability of the i-th sample.

[0038] S300, using the source domain dataset and the target domain dataset and the trained base model, performing model migration training based on knowledge distillation and domain adversarial training to obtain a deep fake image detection model: When the detection capability of the model on the target domain is to be expanded, model migration training based on knowledge distillation and domain adversarial training is performed. The dataset used in the model migration training is the set of source domain data and target domain data. Specifically as follows: S301, constructing a teacher model (T) and a student model (S) based on the trained base model First, the trained base model (the purple part in FIG. 3) is divided into a connected feature extractor (the blue part in FIG. 3) and an authenticity classifier (the green part in FIG. 3). Figure 2 The present embodiment takes the Xception network as an example, takes the fully connected layer FC of the Xception network as the authenticity classifier (the green part in FIG. 3), and takes the network before the fully connected layer FC of the Xception network as the feature extractor (the blue part in FIG. 3). The image feature representation vector f is the output before the fully connected layer FC of the Xception network, and the vector dimension is 2048. F C Next, a copy of the trained base model is made and its parameters are frozen to serve as the teacher model (the gray part in FIG. 3). The supervision of the teacher model on the student model is mainly reflected in the feature representation loss (the red part in FIG. 3) and the knowledge distillation loss (the yellow part in FIG. 3). C F Then, a domain classifier (the yellow part in FIG. 3) is added after the feature extractor (the blue part in FIG. 3) of the trained base model to serve as the student model. That is to say, the student model includes the domain classifier (the yellow part in FIG. 3) in addition to the trained base model (the purple part in FIG. 3). The domain classifier refers to predicting the domain label of a given input sample.

[0039] Figure 2

[0040] F D Figure 2 Figure 2 D Figure 2 D D ​​​​​​​​​​​​​​​​Here, the labels are source or target domain. The domain adversarial training refers to the feature extractor F will be updated in an adversarial way, including two competing objectives: For the classification of real or fake, the feature extractor F and the real or fake classifier C will be updated to correctly classify the data (labeled source and target domain data) real or fake; For the classification of domain, the domain classifier D will be updated to correctly classify the given sample from which domain.

[0041] But the feature extractor F will be updated to make the domain classifier D cannot correctly classify the given sample from which domain. This will ensure that the feature extractor F learns domain invariant features. To achieve domain adversarial training, the domain classifier D does not need to do anything during forward propagation, and the gradient needs to be reversed, that is, take the negative gradient, when updating the parameters of the feature extractor F during backpropagation. Here, the gradient reversal function is defined as , where the input is the feature extractor F extracted from the sample features , and the forward propagation and backpropagation behavior are represented as:

[0042]

[0043] where, is the derivative of the gradient reversal function , is the identity matrix, is used to control the strength of adversarial training.

[0044] For deep fake image detection, there is little difference between real samples in the source or target domain. If you want to focus on eliminating low-level features (such as texture, lighting) domain differences, aligning mid-level semantic features (such as facial components), and preserving high-level task-related features, you can achieve this by applying different strengths of gradient reversal at different levels of the network, for example α 浅层 > α 中层 > α 高层 , α 浅层 is the strength of the shallow layer, α 中层 is the strength of the middle layer, α高层 is high. Thus, domain adversarial training loss is:

[0045] where, denotes the input sample of the model, N is the number of input samples of the model, is the domain label of the i th sample, F denotes the feature extractor, denotes the feature extracted from the sample , denotes the gradient inversion function, and the input is the feature extracted from the sample , denotes the domain classifier, and the input is the output result of the gradient inversion function.

[0046] Feature representation loss is designed based on the assumption that there are similar features between different types of deepfake images. The teacher model trained on the source domain dataset can help the student model learn the target domain with fewer data samples. The image features on the teacher model are represented as , and the image features on the student model are represented as . After softmax normalization of these image features, the maximum value in the vector is . The image features whose values in the vector after softmax normalization of the image features on the student model and the teacher model are within the range are stored at a set interval. By dividing the features, it helps to minimize the domain shift in the learning process. Thus, the feature representation loss is defined as:

[0047] where, is the maximum value in the vector after softmax normalization of the image features on the student model corresponding image feature; is the maximum value in the vector after softmax normalization of the image features on the teacher model corresponding image feature.

[0048] Knowledge distillation loss is to measure the difference between the output of the student model and the soft label of the teacher model. Assuming is the unnormalized score (logit) of the student model for the i th sample, is the soft label of the teacher model for the​j the unnormalized score of the i-th sample, is the unnormalized score (logit) of the i-th sample by the teacher model, i is the unnormalized score (logit) of the i-th sample by the teacher model, is the unnormalized score (logit) of the i-th sample by the teacher model, j The temperatureized softmax function is defined as: The temperatureized softmax function in the student model is defined as:

[0049] The temperatureized softmax function in the teacher model is defined as:

[0050] wherein, is the temperature, a larger temperature parameter can help the student model learn the knowledge of the teacher model better by making the probability distribution smoother; is the exponential function with the natural constant e as the base. Thus, the knowledge distillation loss is expressed as:

[0051] In summary, the loss function of the model migration training based on knowledge distillation and domain adversarial training includes the classification cross-entropy loss, the domain adversarial training loss, the feature representation loss and the knowledge distillation loss, and is expressed as:

[0052] wherein, , and are hyperparameters.

[0053] It should be noted that at this time, the model has only completed training in the source domain and the target domain 2 fields. If further training is required, the domain classifier needs to be expanded to 3 classes for further training, and so on.

[0054] S400, when a new type of deepfake image appears and the amount of labeled data of the new type of deepfake image cannot perform domain adversarial training, the labeled data of the new type of deepfake image (that is, a very small amount of labeled data) is used to fine-tune the deepfake image detection model. Since domain adversarial training cannot be performed, the loss function for fine-tuning the deepfake image detection model includes the classification cross-entropy loss, the feature representation loss and the knowledge distillation loss, and is expressed as:

[0055] At this time, overfitting in a small amount of labeled data can be well avoided due to the supervision of the feature representation loss and the knowledge distillation loss. In the embodiment, the random gradient descent method is used to optimize the model during the model training and fine-tuning.

[0056] Based on the same technical concept, the embodiment of the present application also provides an electronic device which can implement the deep fake image detection method based on knowledge distillation and domain adversarial training provided by the above-mentioned embodiments of the present application. In an embodiment, the electronic device can be a server, a terminal device or other electronic device. As shown in Figure 3 The electronic device can include: at least one processor, and a memory connected with the at least one processor, and the specific connection medium between the processor and the memory in the embodiment of the present application is not limited, Figure 3 In the embodiment, the connection between the processor and the memory is taken by way of example through a bus. The bus is represented by a thick line in Figure 3 the embodiment, and the connection mode between other components is only schematically illustrated and is not limited. The bus can be divided into an address bus, a data bus, a control bus, etc., for the convenience of representation, Figure 3 only one thick line is used in the embodiment, but it does not mean that there is only one bus or only one type of bus. Alternatively, the processor can also be referred to as a controller, and the name is not limited.

[0057] In the embodiment of the present application, the memory stores instructions executable by the at least one processor, and the at least one processor can execute the foregoing deep fake image detection method based on knowledge distillation and domain adversarial training by executing the instructions stored in the memory.

[0058] The processor is the control center of the device, can connect various parts of the entire control device through various interfaces and lines, and can monitor the entire device by running or executing the instructions stored in the memory and calling the data stored in the memory. Various functions and processing data of the device, thereby overall monitoring the device.

[0059] In an alternative design, the processor can include one or more processing units, and the processor can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface and the application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor. In some embodiments, the processor and the memory can be implemented on the same chip, and in some embodiments, they can also be implemented on separate chips respectively.

[0060] The processor can be a general purpose processor, such as a CPU, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general purpose processor can be a microprocessor or any conventional processor. The steps of a deepfake image detection method based on knowledge distillation and domain adversarial training disclosed in conjunction with the embodiments of the present application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0061] The memory is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read only memory (PROM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory in the embodiments of the present application can also be a circuit or any other device capable of realizing the storage function, used for storing program instructions and / or data.

[0062] By designing and programming the processor, the code corresponding to the deepfake image detection method based on knowledge distillation and domain adversarial training introduced in the foregoing embodiments can be fixed into the chip, so that the chip can execute the steps of the method of the above embodiments when running. How to design and program the processor is a technology known to those skilled in the art, which will not be described here.

[0063] Based on the same inventive concept, the embodiments of the present application also provide a storage medium storing computer instructions, when the computer instructions run on a computer, the computer executes the foregoing deepfake image detection method based on knowledge distillation and domain adversarial training.

[0064] In some alternative embodiments, the various aspects of the method for deep fake image detection based on knowledge distillation and domain adversarial training can also be implemented in the form of a program product, which includes program code, when the program product is run on an apparatus, the program code is used to make the control device execute the steps of the method for deep fake image detection based on knowledge distillation and domain adversarial training according to various exemplary embodiments of the present application described above in the specification.

[0065] It should be noted that although several units or sub-units of the apparatus are mentioned in the foregoing detailed description, such division is merely exemplary and not mandatory. Indeed, features and functions of two or more such units described above can be embodied in one unit, according to the embodiments of the present application. Conversely, a unit described above can be further split into several sub-units, each sub-unit embodying some of the features and functions of the unit. Moreover, although the operations of the method of the present application are described in a particular, sequential order, this order is not mandatory and is not intended to imply that there is an order in which the operations must be performed, or that one

[0066] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0067] The present application is described in reference to the flowchart illustrations and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 The flowchart illustrations and / or block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the methods, apparatus (systems) and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart illustrations and / or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that, in some alternative implementations, the functions noted in the flowchart illustrations and / or block diagrams can occur out of the order noted in the figures. For example, two Figure 1 The flowchart illustrations and / or block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the methods, apparatus (systems) and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart illustrations and / or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that, in some alternative implementations, the functions noted in the flowchart illustrations and / or block diagrams can occur out of the order noted in the figures. For example, two

[0068] Program code to implement the application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. The embodiments are not limited by the

[0069] In the case of using a remote computing device, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by connecting through the Internet using an Internet service provider).

[0070] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a Figure 1 one or more functions specified in the flow or flows and / or blocks Figure 1 one or more functions specified in the flow or flows and / or blocks

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow or flows and / or blocks Figure 1 one or more functions specified in the flow or flows and / or blocks Figure 1 one or more functions specified in the flow or flows and / or blocks

[0072] The above description is merely illustrative of the application, and not restrictive. Since the application can be modified in various ways and we adhere to the principles of the application, any modification, equivalent replacement, improvement, etc. made within the scope of the application should be included in the protection scope of the application.

Claims

1. A deep fake image detection method based on knowledge distillation and domain adversarial training, characterized by: include: Construct a dataset, including a source domain dataset and a target domain dataset; Use the source domain dataset to train the model in the source domain to obtain a trained basic model; Using the source domain dataset and the target domain dataset and the trained basic model, model transfer training based on knowledge distillation and domain adversarial training is performed to obtain a deep fake image detection model; the deep fake image detection model is used for deep fake image detection; When a new type of deep fake image appears and the amount of labeled data for the new type of deep fake image is insufficient for domain adversarial training, the deep fake image detection model is fine-tuned using the labeled data for the new type of deep fake image.

2. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 1 is characterized in that The method of using the source domain dataset to perform model source domain training includes: Choose a base model; The basic model is used to perform true-false classification training on the source domain dataset to obtain a trained basic model; the true-false classification loss function for the true-false classification training is classification cross entropy loss.

3. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 2 is characterized in that The method uses the source domain dataset, the target domain dataset, and the trained basic model to perform model transfer training based on knowledge distillation and domain adversarial training, including: Build teacher model and student model based on the trained basic model; The teacher model and the student model are trained on the source domain dataset and the target domain dataset through model transfer based on knowledge distillation and domain adversarial training to obtain a deep fake image detection model.

4. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 3 is characterized in that The teacher model and the student model are constructed based on the trained basic model, including: Divide the trained base model into connected feature extractors F and true / false classifier C ; Copy the trained base model and freeze its parameters as the teacher model; Feature extractor of the trained base model F Add domain classifier later D Then it is used as the student model; among them, the domain classifier D Do nothing during forward propagation and update the feature extractor during backward propagation F The gradient needs to be reversed when the parameters are .

5. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 4 is characterized in that The loss functions for model transfer training based on knowledge distillation and domain adversarial training include classification cross entropy loss, domain adversarial training loss, feature representation loss, and knowledge distillation loss.

6. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 5 is characterized in that The loss functions used to fine-tune the deepfake image detection model include categorical cross entropy loss, feature representation loss, and knowledge distillation loss.

7. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 6 is characterized in that Categorical cross entropy loss Expressed as: in, N is the number of samples input to the model, It is i The authenticity labels of samples, It is i The predicted probability of a sample.

8. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 6 is characterized in that Domain Adversarial Training Loss Expressed as: in, represents the sample input to the model, N is the number of samples input to the model, It is i The domain labels of samples, F represents the feature extractor, Indicates the sample Extracted features, Represents the gradient reversal function, whose input is the sample Extracted features , represents the domain classifier, whose input is the output of the gradient reversal function.

9. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 6, characterized in that Feature representation loss Defined as: in, The maximum value of the vector after softmax normalization of the image features on the student model Corresponding image features; The maximum value of the vector after softmax normalization of the image features on the teacher model The corresponding image features; the value range of the vector after softmax normalization of the image features on the student model and the teacher model is , the value interval is v.

10. The deep fake image detection method based on knowledge distillation and domain adversarial training according to claim 6, characterized in that Knowledge Distillation Loss for: in, is the temperature-normalized softmax function in the student model, is the temperature-normalized softmax function in the teacher model; is the temperature, N is the number of samples input to the model, Is the student model for the i The unnormalized score of samples, Is the student model for the j The unnormalized score of samples, Is the teacher model for the i The unnormalized score of samples, Is the teacher model for the j Unnormalized score of samples; It is an exponential function with the natural constant e as its base.

Images