Student model training method, image forgery detection method and computer device

Abstract

The application discloses a student model training method, an image forgery detection method and computer equipment, and belongs to the image detection field. The image forgery detection method comprises the following steps: determining a target image to be detected; calling a trained student model based on the target image to generate a detection result; wherein the student model is trained by fusing a synthetic forgery feature, training images and intermediate features; the training images are provided by more than two domains; the intermediate features are generated by calling a corresponding teacher model based on the training images; the synthetic forgery feature is generated by calling a teaching assistant model based on all the intermediate features; and the synthetic forgery feature refers to a feature corresponding to common information in all the intermediate features. The method can improve the accuracy of identifying whether an image is a forged image.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image detection, and particularly relates to a student model training method, an image forgery detection method and a computer device. BACKGROUND

[0002] With the rapid development of deep learning technology in the field of image synthesis, deep forged images containing human faces are highly realistic in vision, which has brought serious security risks to social platforms, identity verification, digital media content and other scenarios.

[0003] At present, the deep learning-based forged image detection method has made certain progress. For example, a distillation framework is adopted, that is, two or more teacher models are first constructed and trained through a training data set. Among them, different teacher models correspond to different targets, which refers to the forgery type, that is, the means used when forging images; the training data set can also be called a domain, such as a training data set corresponding to forged images is a forged domain, and a training data set corresponding to real images is a real domain. After the teacher model is trained, the student model learns the knowledge of the teacher model, so that the student model has the ability to be comparable or even surpass the teacher model. Among them, the distillation framework can be divided into output distillation and intermediate feature distillation; the knowledge refers to the way the teacher model extracts features from the image, and the learning refers to the features extracted from the image by the student model approaching the features extracted from the image by the teacher model.

[0004] However, for intermediate feature distillation, since the teacher model is usually a deep network with more parameters, and the student model is a light network. In the case where the student model needs to learn the knowledge of two or more teacher models, it is difficult for it to completely reconstruct the complex feature expression of the teacher model, so that the image detection performance of the finally obtained student model is not good, that is, the accuracy when identifying whether an image is a forged image is not high. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a student model training method, an image forgery detection method and a computer device, which can solve the problem of how to improve the image detection performance of the student model, that is, improve the accuracy when identifying whether an image is a forged image.

[0006] In order to solve the above technical problems, the present application is implemented as follows:

[0007] In a first aspect, the embodiments of the present application provide a student model training method, which comprises:

[0008] determining training images from two or more domains;

[0009] based on the training images, calling a corresponding teacher model to generate intermediate features;

[0010] Based on all the intermediate features, a teaching model is called to generate a fusion forgery feature; the fusion forgery feature refers to a feature corresponding to common information in all the intermediate features;

[0011] The student model is trained based on the fusion forgery feature, the training image, and the intermediate feature.

[0012] Optionally, the process of generating the fusion forgery feature based on all the intermediate features, calling the teaching model, is represented by the following formula:

[0013]

[0014] wherein, is the batch size; is the number of domains; is the number of channels of the fusion forgery feature, is the spatial size of the feature map corresponding to the fusion forgery feature; wherein, is 1 1 convolutional projection layer, ,

[0015]

[0016] wherein, is a reshape operation; is a tensor obtained by stacking the intermediate features in the dimension of the domain; is the i-th intermediate feature; represents the number of channels of the intermediate feature output by the teacher model; is the spatial size of the feature map corresponding to the intermediate feature output by the teacher model. Optionally, the student model is trained based on the fusion forgery feature, the training image, and the intermediate feature, and is realized based on the following loss:

[0017] a forgery detection classification loss, a teacher distillation loss, a teaching distillation loss, and a domain classification loss.

[0018] Optionally, the student model is trained based on the fusion forgery feature, the training image, and the intermediate feature, and is realized based on the following loss:

[0019] a forgery detection classification loss, a teacher distillation loss, a teaching distillation loss, a domain classification loss, and a real feature consistency loss.

[0020]

[0021] ​​Optionally, each loss is provided with a corresponding weight, and a weighted sum of the corresponding losses is taken as the total loss in the training process of training the student model.

[0022] In a second aspect, the embodiments of the present application also provide an image forgery detection method, which comprises:

[0023] determining a target image to be detected;

[0024] based on the target image, calling the trained student model to generate a detection result;

[0025] wherein the student model is trained by fusing the synthetic forgery feature, the training image and the intermediate feature; the training image is provided by two or more domains; the intermediate feature is generated by calling the corresponding teacher model based on the training image; the synthetic forgery feature is generated by calling the assistant model based on all intermediate features; the synthetic forgery feature refers to the feature corresponding to the common information in all intermediate features.

[0026] Optionally, the process of generating the synthetic forgery feature by calling the assistant model based on all intermediate features is represented by the following formula:

[0027]

[0028] wherein, is the batch size; is the number of domains; is the number of channels of the synthetic forgery feature, is the spatial size of the feature map corresponding to the synthetic forgery feature; wherein, is 1 1 convolutional projection layer, ,

[0029]

[0030] wherein, is the reshape operation; is a tensor obtained by stacking the intermediate features in the domain dimension; is the i-th intermediate feature; represents the number of channels of the intermediate feature output by the teacher model; is the spatial size of the feature map corresponding to the intermediate feature output by the teacher model. Optionally, the student model is trained based on the synthetic forgery feature, the training image and the intermediate feature, and is realized based on the following losses:

[0031] ​​

[0032] The forgery detection classification loss, the teacher distillation loss, the assistant distillation loss, and the domain classification loss.

[0033] Optionally, a student model is trained based on the fusion forgery feature, the training image, and the intermediate feature, and the following losses are realized:

[0034] The forgery detection classification loss, the teacher distillation loss, the assistant distillation loss, the domain classification loss, and the real feature consistency loss.

[0035] In a third aspect, an embodiment of the present application provides a computer device, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the method according to the first aspect and the second aspect.

[0036] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a program or instruction, and the program or instruction is executed by a processor to implement the steps of the method according to the first aspect and the second aspect.

[0037] In a fifth aspect, an embodiment of the present application further provides a computer program product comprising a computer program, which is executed by a processor to implement the steps of the method according to the first aspect and the second aspect.

[0038] In the embodiment of the present application, a student model training method is provided, which provides simplified and semantically consistent feature guidance for the student model by setting an assistant model to perform intermediate fusion and representation guidance on the intermediate feature output by the teacher model, so as to alleviate the distribution difference, and then facilitate the student model to effectively learn the fusion forgery feature, thereby improving the image detection performance of the student model, i.e., improving the accuracy of identifying whether an image is a fake image. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a flowchart of a student model training method provided by some embodiments of the present application;

[0040] Figure 2 is a flowchart of the assistant model generating a fusion forgery feature provided by some embodiments of the present application;

[0041] Figure 3 is a flowchart of an image forgery detection method provided by some embodiments of the present application;

[0042] Figure 4 is an internal structure diagram of a computer device provided by some embodiments of the present application. DETAILED DESCRIPTION

[0043] Clearly, the described embodiments are only some, but not all of embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts should belong to the protection scope of the present application.

[0044] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0045] The student model training method provided by the embodiments of the present application will be described in detail below in combination with the drawings, through specific embodiments and application scenarios.

[0046] As mentioned in the background: for intermediate feature distillation, the teacher model is usually a deep network with more parameters, and the student model is a lightweight network. In the case where the student model needs to learn the knowledge of two or more teacher models at the same time, it is difficult for the student model to completely reconstruct the complex feature expression of the teacher model, so that the image detection performance of the final student model is not good, that is, the accuracy of identifying whether an image is a fake image is not high. The core lies in: in the traditional technology, the input of the student model is the simple stacking of the outputs of all teacher models, which makes it difficult for the student model to fully absorb the knowledge of the teacher model.

[0047] The main reason why it is difficult to fully absorb the knowledge of the teacher model is:

[0048] 1. Different teacher models of different forgery types focus on different semantic regions and forgery patterns, resulting in differences in the distribution of their output features in spatial position, semantic context and texture dimension, etc. In the absence of an intermediate coordination mechanism, the student model is difficult to align multiple targets at the same time, and is prone to "semantic mismatch".

[0049] 2. The teacher model is usually a deep network with more parameters, and the student model is a lightweight network, which means limited ability, and it is difficult to completely reconstruct the complex feature expression of the teacher model.

[0050] Therefore, the student model is difficult to effectively learn and fuse forgery features, which limits the final detection performance. Specifically:

[0051] Because the operation principles are different among different forgery types, the intermediate features learned by each teacher model have obvious representation differences. The reason is that the spatial focus areas of different forgery types are different. For example, the Deepfakes dataset focuses on the face fusion edge; the Face2Face dataset focuses on the mouth and eye micro-expression synchronization; the FaceSwap dataset focuses on the face mask and edge occlusion area; and the NeuralTextures dataset captures more skin texture and frequency domain high-frequency artifacts.

[0052] This also leads to the fact that the feature maps generated by each teacher model at the intermediate layer have spatial distribution offsets, that is, the intermediate features of different teachers have distribution differences in spatial distribution. This distribution difference makes it difficult for the student model to effectively learn the fusion forgery features.

[0053] In an exemplary embodiment, as shown in Figure 1 a student model training method is provided. The method performs intermediate fusion and representation guidance on the intermediate features output by the teacher model by setting a tutor model, thereby providing the student model with simplified and semantically consistent feature guidance to alleviate the distribution difference, and thus facilitating the student model to effectively learn the fusion forgery features, so as to improve the image detection performance of the student model, that is, to improve the accuracy of identifying whether an image is a forged image.

[0054] The method includes the following steps 102 to 108. Among them:

[0055] Step 102, determine training images from two or more domains.

[0056] Step 104, based on the training images, call the corresponding teacher model to generate intermediate features.

[0057] Step 106, based on all intermediate features, call the tutor model to generate fusion forgery features; the fusion forgery features refer to the features corresponding to the common information in all intermediate features.

[0058] Step 108, train the student model based on the fusion forgery features, the training images and the intermediate features.

[0059] Among them, the above domain can include a forgery domain and a real domain, or both are forgery domains. The real domain is used to improve the robustness of the student model.

[0060] Taking four forgery domains and one real domain as an example, the embodiments of the steps of the method are described.

[0061] Among them, the process of generating intermediate features is as follows:

[0062] Let the input batch size be B, and the number of domains be N = 5, that is, the domains correspond to four types of forgeries (Deepfakes, Face2Face, FaceSwap, NeuralTextures) and real images (Real) respectively.

[0063] Based on this, the network of the first teacher model receives input images from the corresponding fake domain or real domain , extracts intermediate features through the encoder in the network:

[0064]

[0065] wherein, represents the number of channels of the intermediate features output by the teacher model (such as 512); is the spatial size of the feature map corresponding to the intermediate feature; in particular, the intermediate feature extracted by the teacher model corresponding to the real domain (real feature) is denoted as .

[0066] On this basis, in order to unify the subsequent distillation calculation, all intermediate features are subjected to a global average pooling (GAP) operation before entering the alignment module:

[0067]

[0068] This operation can ensure that the intermediate features are comparable vector representations in the semantic space, thereby facilitating cross-domain alignment.

[0069] For the generation of fusion forgery features, refer to Figure 2 , and the process is as follows:

[0070] To alleviate the differences in spatial position, semantic context, and texture distribution of the intermediate features generated by different teacher models, an assistant model is introduced as an intermediate coordinator. The assistant model is a single network, and its weights are shared by all forgery types.

[0071] Therefore, during training, this single network repeatedly receives feature input from different forgery types, and it must find common information in these features (such as spatial anomalies of forgery traces, edge distortions, high-frequency noise in the frequency domain, etc.), that is, in the semantic space, find a general intermediate representation from the intermediate features with distribution differences, rather than a dedicated feature for a specific forgery type, in order to achieve a lower loss on all forgery types. That is, find the fusion forgery features from all intermediate features.

[0072] ​The input of the teaching assistant model is the intermediate feature, that is, the multi-domain feature set formed after stacking and rearranging the feature maps (intermediate features) output by all teacher models. The process is as follows:

[0073] Stack intermediate features in the domain dimension to obtain a tensor .

[0074] Expand the domain dimension to the batch dimension through reshape operation:

[0075]

[0076] To ensure the consistency of the channel dimension and the alignment of the features, a 1x1 convolution projection layer is introduced to obtain:

[0077]

[0078] Then, the teaching assistant model extracts a unified fake guiding feature, that is, a fusion fake feature:

[0079]

[0080] Finally, the GAP operation is performed to obtain a vector representation:

[0081]

[0082] wherein, represents the unified feature map obtained by the teaching assistant model for the sample, and is the vector obtained after global average pooling (GAP).

[0083] It can be understood that the teaching assistant model extracts a unified fake guiding feature from the intermediate features of different fake types. In this process, the reshape operation ensures the uniformity of the input format of the teaching assistant model, thereby ensuring that the input is consistent in tensor structure and channel semantics, avoiding the interference of structural differences in the input stage on semantic alignment, and further ensuring that the fusion fake feature has consistency in semantics.

[0084] Based on this, the student model has excellent cross-dataset (cross-domain) generalization ability and compression robustness, and can work stably under various fake means, different compression rates, and complex video environments. This method enhances the consistency and discriminability of feature representation through collaborative distillation, enabling the student model to have stronger fake image discrimination generalization ability, which cannot be achieved by traditional Convolutional Neural Networks (CNN) methods or single-teacher distillation structures.

[0085] It should be noted that the distillation loss of the teaching assistant model is defined as:

[0086]

[0087] wherein, is the feature vector of the student model of the i-th image sample (which can come from the training images of any domain); is a dimension alignment mapping.

[0088] It can be understood that through the distillation loss, the student model can be aligned with the teaching assistant model in the public subspace first, thereby reducing the training instability caused by the above distribution difference and reducing the difficulty of the student model learning the knowledge of the teacher model.

[0089] During the training of the student model, it can be understood that the teaching assistant model is used to assist the student model in learning the knowledge of all teacher models. Specifically, the student model and the teaching assistant model are jointly trained.

[0090] It should be noted that during the joint training process, the student model and the teaching assistant model can update the model parameters through backpropagation combined with the corresponding loss, so as to finally obtain a well-trained student model through iterative training. Among them, the corresponding loss includes the fake detection classification loss, the teacher distillation loss and the teaching assistant distillation loss.

[0091] The student model is constructed using a lightweight architecture, including but not limited to the EfficientNe series, the MobileNet series. The following takes EfficientNet-B0 as an example to explain the definition of the loss function:

[0092] Among them, the parameter amount of EfficientNet-B0 is about 5.3M, which is significantly reduced compared with the parameter amount of the image fake detection related model with similar performance in the prior art, so as to improve the inference speed and adaptability. Its goal is to achieve high performance in real / fake binary classification tasks.

[0093] It can be understood that by introducing a student model with compact structure, combined with the semantic guidance provided by two or more teacher models and teaching assistant models, the detection accuracy of large models is maintained or even surpassed under the premise of significantly reducing the parameter amount and computational complexity, solving the problem that the deep fake detection method is difficult to be deployed on resource-limited devices (such as edge and mobile terminals).

[0094] Among them, the input sample image is input to the student model , and the following features can be obtained: ​

[0095]

[0096] The feature After GAP operation, the feature vector , can be obtained After inputting the classification head, the true and false probabilities can be predicted.

[0097] Among them, for the forgery detection classification loss, the label needs to be defined first, where 0 represents a real image and 1 represents a fake image. The loss function is as follows:

[0098]

[0099] Among them, is the probability predicted by the student model; is the true value of the sample label of the sample image.

[0100] Among them, for the teacher distillation loss (used to realize that the student model learns the knowledge of the teacher model), the student model needs to learn semantic representation from the teacher model, and in order to align the features output by the student model with the features extracted by the teacher (including fake features and real features), the teacher distillation loss can be expressed as:

[0101]

[0102] Among them, is the number of the th sample image in the fake domain; the vector representation of the real feature of the th sample image is denoted as ; represents the feature vector extracted by the th sample in the th fake type teacher model; represents the feature vector extracted by the th sample in the real teacher model.

[0103] Among them, for the assistant distillation loss, please refer to the relevant records of the distillation loss of the assistant model in the above.

[0104] In an embodiment, in order to enhance the student model's perception of the differences between different fake types and improve the clarity of the classification boundary, the method also proposes a domain classification loss.

[0105] Specifically, for the four fake domains, a domain classifier is introduced , which can predict the fake type to which the feature belongs after inputting the feature of the student model into the domain classifier:

[0106] The corresponding domain classification output probability is: Therefore, the domain classification loss can be expressed as:

[0107]

[0108] wherein, is the Kronecker indicator function, if , then ; represents the prediction probability of the student model that the th sample belongs to the th domain.

[0109] It can be understood that the domain classifier is a label selector, which is used to ensure that the cross-entropy (i.e., the fake detection classification loss) only takes the prediction probability of the real domain label corresponding to the th sample image to participate in the calculation, and the prediction probabilities of the remaining domains are directly ignored in the domain classification loss of the sample image.

[0110] In an embodiment, in order to narrow the vector representation of the sample image from the real domain by the student model and the vector representation of the sample image from the real domain by the teacher model, so as to reduce the risk of misjudging the real image as a fake image, that is, to improve the recall and reduce the false positive. This method also proposes a real feature consistency loss, which can be expressed as:

[0111]

[0112] wherein, is the number of real samples in the current batch; represents traversing all real samples in the current batch; represents the feature representation obtained by the student model for the th sample; represents the feature representation obtained by the real teacher model for the th real sample.

[0113] In an embodiment, in order to further improve the image detection performance of the student model, that is, to improve the accuracy of identifying whether an image is a fake image. On the basis of the above-mentioned losses, this method also proposes a total loss function obtained by weighted combination in the joint training process of the student model and the teaching assistant model, that is, the final optimization target of the joint training process can be expressed as:

[0114]

[0115] wherein, is a hyperparameter, which can be adjusted according to the importance of the detection task.

[0116] For example , , , , It can be understood that by setting this weight, during the joint training process, the steady-state alignment of the teaching assistant model is emphasized first. (With higher weights), after the student model converges in the common feature space, the weight of domain refinement for each teacher model will be increased. The weights are gradually increased, thus achieving a progressive distillation strategy of "first steady-state alignment, then refined learning". This strategy reduces gradient oscillations and improves the student model's ability to generalize to various forgery types.

[0117] It should be noted that even if the calculation of the total loss function does not include the true feature consistency loss, the total loss function can still be obtained by the above weighted combination method. That is, the total loss function is obtained by weighted combination of the other four losses that do not include the true feature consistency loss.

[0118] Specifically, for the supervision signals of these loss functions, the supervision of the teaching assistant model's distillation loss acts as a cross-domain soft objective, applied to all samples (real + fake), to first pull the student model into a common subspace. The supervision of the teacher distillation loss acts as a domain-specific hard constraint; real samples use the real teacher model, and fake samples use the fake teacher model of their corresponding domain, to refine the feature patterns of each fake domain. The supervision of the domain classification loss prevents the student model from compressing the features of all fake images into a single cluster. The supervision of the truth consistency loss (a combination of cosine similarity "close to true, far from false") fixes the geometric anchor point of "true," improving generalization stability.

[0119] Based on this, the method can improve the alignment speed between teaching assistants and students in the early stages of training, and the features first fall into a unified semantic space. In the later stages, teacher supervision (supervision of teacher distillation loss) is used to make detailed adjustments on this basis, thereby completing the alignment of domain features. This combination of training method of "soft target foundation + hard constraint refinement" makes even small models both stable and accurate.

[0120] In other words, this multi-constraint joint training enables the teaching assistant model to align not only the features between different fake types, but also the semantic boundaries between fake and real. This makes the features output by the teaching assistant model a low-discrepancy, highly generalizable intermediate semantic representation, allowing the student model to learn the knowledge of the teacher model more stably and converge more easily.

[0121] In one exemplary embodiment, such as Figure 3 As shown, an image forgery detection method is provided, which includes steps 302 and 304. Wherein:

[0122] Step 302: Determine the target image to be detected.

[0123] Step 304: Based on the target image, call the trained student model to generate detection results.

[0124] The student model is trained using fusion-based forgery features, training images, and intermediate features; the training images are provided by two or more domains; the intermediate features are generated by calling the corresponding teacher model based on the training images; the fusion-based forgery features are generated by calling the teaching assistant model based on all intermediate features; the fusion-based forgery features refer to the features corresponding to the common information among all intermediate features.

[0125] In one embodiment, the process of generating fused forged features by calling teaching assistant model A based on all intermediate features is represented by the following formula:

[0126]

[0127] in, Batch size; The number of fields; The number of channels for fusion-based forgery features. The spatial dimensions of the feature map corresponding to the fusion forgery feature; ,in, 1 1 convolutional projection layer, ,

[0128]

[0129] in, For reshape operation; To pass A tensor obtained by stacking intermediate features along the dimensions of the domain; For the first One intermediate feature; This represents the number of channels for the intermediate features output by the teacher model; The spatial dimensions of the feature map corresponding to the intermediate features output by the teacher model.

[0130] In one embodiment, a student model is trained based on the fused forgery features, the training image, and the intermediate features, using the following loss:

[0131] The losses include falsified detection classification loss, teacher distillation loss, teaching assistant distillation loss, and domain classification loss.

[0132] In an embodiment, a student model is trained based on the fusion forgery feature, the training image and the intermediate feature, and the following loss is implemented:

[0133] The forgery detection classification loss, the teacher distillation loss, the assistant distillation loss, the domain classification loss and the real feature consistency loss.

[0134] Wherein, the target image to be detected can be a forgery image of any forgery type or a real image.

[0135] Wherein, when the target image is a forgery image, the detection result is "forgery"; when the target image is a real image, the detection result is "real".

[0136] Wherein, the specific limitation of the training process of the student model in the image forgery detection method can refer to the limitation of the student model training method in the above, which will not be repeated here.

[0137] The following is the process and results of the experimental verification of the image detection performance of the student model provided by the present application:

[0138] Wherein, the experimental setup is:

[0139] In order to comprehensively evaluate the accuracy and cross-dataset generalization ability of the image forgery detection method (based on the multi-teacher-assistant collaborative distillation framework) proposed by the present application in the deep forgery detection task, multiple mainstream and authoritative deep forgery image datasets are used for experimental testing.

[0140] The deep forgery image dataset specifically includes: FaceForensics++(FF++), DeepfakeDetection(DFD), Deepfake Detection Challenge(DFDC), DFDC preview version(DFDCP), Celeb-DF(v1) and Celeb-DF(v2).

[0141] Wherein, FF++ is used as the training set, which is a large dataset containing more than 180 million forgery images from 1000 original videos. The forgery images are generated by four face processing algorithms (DeepFakes(DF), Face2Face(F2F), FaceSwap(FS) and NeuralTexture(NT)) using the same set of original videos.

[0142] It is worth noting that FF++ provides three versions at the image compression level: original uncompressed, light compression (c23), and heavy compression (c40). In line with existing research, in order to enhance the referenceability of the actual deployment environment while ensuring image quality, the experiment uniformly adopts the light compression version (c23) for training and evaluation.

[0143] Evaluation index of experimental results: In order to evaluate the discrimination performance of the image forgery detection method (multi-teacher-assistant collaborative distillation lightweight deep forgery image detection method) proposed in the present application in various forgery image detection tasks, the area under the receiver operating characteristic curve (Area Under Curve, AUC) is selected as the main evaluation index.

[0144] It can be understood that AUC can comprehensively reflect the classification ability of the model for real images and forged images at different discrimination thresholds, and its numerical range is between [0, 1], and the larger the value, the stronger the overall discrimination ability of the model, especially suitable for the scene of complex data distribution and blurred class boundary.

[0145] In order to verify the effectiveness and generalization ability of the image forgery detection method (MTA-B0) proposed in the present application in the forgery image detection task, the inventors have carried out a series of experiments, and the experimental results are shown in Table 1. During the experiment, all detectors are trained on FF++ (c23), and the cross-dataset evaluation of frame-level AUC index is carried out on other datasets.

[0146] Table 1

[0147]

[0148] As can be seen from Table 1, the image forgery detection method proposed in the present application has achieved excellent detection performance on the five test datasets, with an AUC of 0.889 on the CDF-v1 dataset, which is significantly higher than Xception (0.779) and SPSL (0.815), indicating good recognition ability for low-quality video forgery; on CDF-v2, it reaches 0.822, which is significantly improved compared with the frequency domain method SRM (0.755), showing excellent discrimination ability for high-realistic forgery. The method proposed in the present application has an average AUC of 0.827 on the five deep forgery datasets, leading the current mainstream detection methods.

[0149] It should be emphasized that although the present application adopts lightweight EfficientNet-B0 as the student model, its parameter quantity is much lower than that of Xception or EfficientNetB4, but under the joint action of multi-teacher distillation and assistant guide mechanism, it still maintains high detection accuracy and cross-scene generalization ability.

[0150] To further verify the applicability of the image forgery detection method proposed in the application in a resource-limited environment, the inventors designed a model complexity comparison experiment to compare the differences in parameter size, computational amount (FLOPs) and inference delay of a plurality of mainstream forgery image detection methods. The experimental results are shown in Table 2. Similarly, all detectors are trained on FF++ (c23), and cross-dataset evaluation of frame-level AUC indicators is performed on other datasets.

[0151] Table 2

[0152]

[0153] As can be seen from Table 2, a series of representative methods are listed, from lightweight networks (such as MesoNet, MobileNetV2, ShuffleNetV2) to high-performance detectors (such as Xception, SPSL, F3Net). The experimental results show that the MTA-B0 method proposed in the application significantly reduces the model complexity while maintaining a high detection performance (the average AUC is 0.827), only 5.3M parameters and 0.49GFLOPs are required, and the inference time is 19.3ms, which is much lower than the complexity of other high-precision models. This experimental result shows that MTA-B0 greatly reduces the computational overhead while ensuring accuracy, and has good engineering deployability and real-time processing capability.

[0154] To verify the contribution of each key structural component in the image forgery detection method proposed in the application to the overall detection performance, the inventors designed an ablation experiment of the distillation structure, and evaluated the influence of the multi-teacher mechanism, the teaching assistant module and the lightweight student model design on the detection effect. The experimental results are shown in Table 3.

[0155] Table 3

[0156]

[0157] As can be seen from Table 3, the average AUC comparison results under four different distillation configurations are shown, covering from the basic structure without using distillation (Baseline) to the complete MTA-B0 distillation system.

[0158] In the Baseline setting, the traditional EfficientNet-B4 structure is directly used for forgery image classification training without introducing any teacher or distillation mechanism, and finally an average AUC of 0.756 is obtained.

[0159] Subsequently, the multi-teacher network is introduced to conduct traditional knowledge distillation (KD), and the student model is replaced by a lightweight EfficientNet-B0. Despite the significant compression of the model, the performance is improved to 0.781 with the guidance of the fake features provided by the multi-teacher, indicating the effectiveness of the multi-teacher mechanism.

[0160] Further, an assistant module is added on the basis of multi-teacher distillation as a bridge for intermediate feature alignment and information fusion, helping the student model to better receive multi-source fake features. At this time, the average AUC is improved to 0.811, verifying the significant role of the assistant in alleviating feature distribution differences and improving student learnability.

[0161] Finally, under the complete structure of the multi-teacher mechanism, the assistant module, and the lightweight student network (i.e., MTA-B0), the method achieves the highest average AUC (0.827), which is superior to other comparative schemes and improves by 7.1% compared with the base model, while keeping the model parameter amount extremely low.

[0162] It should be understood that although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless explicitly stated herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0163] In an exemplary embodiment, a computer device, which can be a server, is provided, and its internal structure diagram can be as shown in Figure 4As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to implement a student model training method and / or an image forgery detection method.

[0164] Those skilled in the art can understand, Figure 4 The skilled in the art can understand,

[0165] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. In the embodiments of the present application, any reference to memory, database or other medium can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments of the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments of the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0166] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0167] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A student model training method characterized by comprising: The student model training method comprises: determining training images from two or more domains; based on the training images, calling a corresponding teacher model to generate intermediate features; based on all intermediate features, calling a tutor model to generate a fusion forgery feature; the fusion forgery feature refers to a feature corresponding to common information in all intermediate features; the process of generating a fusion forgery feature based on all intermediate features and calling a tutor model is represented by the following formula: ； wherein, is a teacher model, is a batch size; is a number of domains; is a number of channels of the fusion fake feature, is a spatial size of a feature map corresponding to the fusion fake feature; wherein, is 1 1 convolutional projection layer, ; ； wherein, is a reshape operation; is by stacking the tensors obtained in the dimension of the domain; is the intermediate feature; denotes the number of channels of the intermediate feature output by the teacher model; is the spatial size of the feature map corresponding to the intermediate feature output by the teacher model; training a student model based on the fusion forgery feature, the training images and the intermediate features.

2. The student model training method according to claim 1, characterized by, The training of the student model based on the fusion forgery feature, the training images and the intermediate features is realized based on the following losses: forgery detection classification loss, teacher distillation loss, tutor distillation loss and domain classification loss; or forgery detection classification loss, teacher distillation loss, tutor distillation loss, domain classification loss and real feature consistency loss.

3. The student model training method according to claim 2, characterized by, Each loss is provided with a corresponding weight, and the weighted sum of the corresponding losses is the total loss in the training process of training the student model.

4. An image forgery detection method characterized by, The image forgery detection method comprises: determining a target image to be detected; based on the target image, calling a trained student model to generate a detection result; The student model is trained by a fusion forgery feature, training images and intermediate features; the training images are provided by two or more domains; the intermediate features are generated by calling a corresponding teacher model based on the training images; the fusion forgery feature is generated by calling a tutor model based on all intermediate features; the fusion forgery feature refers to a feature corresponding to common information in all intermediate features; The process of generating a fusion forgery feature based on all intermediate features and calling a tutor model is represented by the following formula: ； wherein, is a teaching model, is a batch size; is a number of domains; is a number of channels of the fusion forgery feature, is a spatial size of a feature map corresponding to the fusion forgery feature; wherein, is 1 1 convolutional projection layer, ; ； wherein, is a reshape operation; is obtained by stacking the tensors of the intermediate features in the dimension of the domain; is the i-th intermediate feature; denotes the number of channels of the intermediate feature output by the teacher model; is the spatial size of the feature map corresponding to the intermediate feature output by the teacher model.

5. The image forgery detection method of claim 4, wherein, The student model is trained based on the fusion forgery feature, the training images and the intermediate features, and realized based on the following losses: forgery detection classification loss, teacher distillation loss, tutor distillation loss and domain classification loss; or forgery detection classification loss, teacher distillation loss, tutor distillation loss, domain classification loss and real feature consistency loss.

6. A computer device, comprising: A processor, a memory and a program or instructions stored on the memory and executable on the processor are included, and the program or instructions are executed by the processor to realize the steps of the method of any one of claims 1-5.

Images