A method and apparatus for detecting deepfake videos based on multi-scale feature decoupling

Abstract

This invention discloses a method and apparatus for deepfake video detection based on multi-scale feature decoupling. First, color and frequency domain features of the fake face are obtained through pre-trained CNN and SRM convolutions. Then, a self-attention module decouples the features into two parts: realistic / natural features and features containing forgery traces. In the realistic / natural feature branch, a discriminator is established to determine the authenticity of the features, while a domain discriminator determines the forgery technique from which the features originate. The adversarial relationship between the discriminator and the feature decoupling network promotes more accurate decoupling of features containing forgery traces. In the forgery trace feature branch, a classification loss is used to allow the network to learn the differences between real and fake faces. Finally, the two are combined and the model is trained end-to-end, achieving accurate and highly generalized deepfake video detection, thus improving the generality and accuracy of the deepfake video detection method.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of deepfake video detection, and particularly relates to a deepfake video detection method and device based on multi-scale feature decoupling. BACKGROUND

[0002] With the development of neural network technology, deepfake methods have also improved, and can generate realistic human faces in various ways and replace characters in videos. Malicious attackers widely spread false videos generated by deepfake human faces on social media, which will cause immeasurable harm to public opinion, media credibility and celebrity image. In order to counter such attack methods, the opposite deepfake video detection technology has become a research trend.

[0003] The mainstream deepfake video detection technology usually uses mature face recognition and tracking technology to obtain a series of faces to be detected after splitting the video at the frame level. The face to be detected is input into a feature extraction network to explore the fake traces, and finally the face features are classified as true or false. For example, patent document CN114693607A discloses a tampered video detection method and system based on multi-domain block feature landmark registration. For another example, CN114743148A discloses a multi-scale feature fusion tampered video detection method, system, medium and equipment. The current method can achieve excellent accuracy in a single data set, but due to the diversity of fake technology and the limitation of fake technology in the training set, the generalization of the fake traces extracted by this method is poor, and it is easy to produce a tendency to a specific fake trace. When facing fake faces generated by unknown fake methods in cross-dataset testing, the detection performance is obviously insufficient.

[0004] It can be predicted that deepfake technology will continue to progress in diversity and realism. Real deepfake videos will mix different types of fake methods, and the degree of realism will make it more difficult for the naked eye or machines to distinguish between true and false. Existing deepfake video detection technology is under such pressure and challenge, and it is difficult to provide satisfactory accurate detection results, and there is an urgent need to improve the universality and generalization of the technology. SUMMARY

[0005] In view of the above, the purpose of the present application is to provide a deepfake video detection method and device based on multi-scale feature decoupling, which decouples the fake trace features in multiple scales and realizes deepfake detection based on the fake trace features, thereby improving the detection effect.

[0006] To achieve the above-mentioned purpose of the application, the deepfake video detection method based on multi-scale feature decoupling provided by the embodiment comprises the following steps:

[0007] The multi-scale feature decoupling network is constructed, including a feature extraction module containing a plurality of scale feature extraction units, color domain features and frequency domain features of different scales are extracted from the color domain and the frequency domain of the face image respectively by the feature extraction units, and the color domain features and the frequency domain features are fused to obtain image features, each feature extraction unit is connected with a decoupling module, the decoupling module includes a self-attention unit, a first mapping unit, a second mapping unit, a classifier, a class discriminator and a domain discriminator, the image features are calculated by the self-attention unit to obtain two complementary self-attention maps, the two self-attention maps are multiplied with the image features to obtain decoupled realistic natural features and forgery trace features, the forgery trace features are input into the classifier after being mapped by the first mapping unit to perform forgery detection, and the realistic natural features are input into the class discriminator and the domain discriminator respectively after being mapped by the second mapping unit to perform feature authenticity discrimination and feature domain attribution judgment;

[0008] A first adversarial loss is constructed for the feature extraction module and the class discriminator to promote correct decoupling of the forgery trace features, a second adversarial loss is constructed for the feature extraction module and the domain discriminator to promote the decoupled features not to be biased to a specific forgery method, and a classification loss is constructed for the feature extraction module and the classifier to promote learning of correct classification, the multi-scale feature decoupling network is parameter optimized by using face image samples and the constructed losses, and after the parameter optimization is completed, the self-attention unit, the first mapping unit and the classifier in the feature extraction module and each decoupling module are extracted as a forgery detection model;

[0009] The forgery detection model is used for deep fake video detection.

[0010] In one embodiment, each feature extraction unit includes two feature extraction branches of the same scale and a fusion operation, color domain features and frequency domain features are extracted by the two feature extraction branches, and image features are obtained by fusing the color domain features and the frequency domain features by the fusion operation, wherein the feature extraction branch adopts a convolutional neural network, and the fusion operation adopts a convolution operation.

[0011] In one embodiment, the feature extraction branch adopts an InceptionResNet network.

[0012] In one embodiment, in the self-attention unit, a self-attention mechanism is used to calculate the image features to obtain a self-attention map A r , according to the complementary relationship, a self-attention map A f of the forgery trace features is calculated r , the image features are multiplied with A r and A f respectively to obtain decoupled realistic natural features and forgery trace features.

[0013] In one embodiment, both the first mapping unit and the second mapping unit employ a convolutional neural network, which includes a downsampling layer, a convolutional layer, and a linear layer. The input features are converted into feature vectors through the downsampling layer, the convolutional layer, and the linear layer.

[0014] In one embodiment, the first adversarial loss includes a class discrimination loss intended to enable the class discriminator to distinguish between real facial features and realistic natural features decoupled from fake faces. The generation loss aims to extract realistic, natural features from fake face images that closely resemble real facial features, making them indistinguishable to classifiers. in:

[0015]

[0016]

[0017] Where i represents the index of the image feature, The vector represents the realistic and natural features obtained after mapping by the second mapping unit, and y represents the real facial feature label. This represents the predicted value of the class discriminator. This indicates averaging within a batch;

[0018] During training, the feature extraction module and the class discriminator are trained alternately using the first adversarial loss.

[0019] In one embodiment, the second adversarial loss includes a domain discrimination loss intended to distinguish realistic natural features decoupled from faces generated by different forgery methods. and generation loss

[0020]

[0021] Where i represents the index of the image feature, The vector y represents the realistic and natural features obtained by mapping through the second mapping unit. k This represents the domain label, and k represents the domain category. The predicted value of the domain discriminator. This indicates averaging within a batch;

[0022] During training, a gradient reversal layer is added before the domain discriminator to improve the domain discrimination loss. The gradient values ​​are reversed during backpropagation to the feature extraction module, thus calculating the generation loss that prevents the domain discriminator from distinguishing the forgery category. And based on the generation loss To optimize the domain discriminator and feature extraction module.

[0023] In one embodiment, the classification loss is expressed as:

[0024]

[0025] Where i represents the index of the image feature, y' represents the true / false label of the sample, and p i The predicted value of the classifier. This indicates averaging within a batch;

[0026] During training, the adversarial loss is balanced using hyperparameters. The final loss function of the entire multi-scale feature decoupling network includes the generation loss. and loss assessment Where α and β are hyperparameters, according to and and The entire multi-scale feature decoupling network is trained using an alternating method.

[0027] In one embodiment, the detection of deepfake videos using a forgery detection model includes:

[0028] The face image to be tested is input into the forgery detection model, and image features at different scales are extracted through each feature extraction unit of the feature extraction module.

[0029] In the decoupling module corresponding to each feature extraction unit, the self-attention unit extracts the self-attention map corresponding to the forgery trace features based on the image, and calculates the forgery trace features with the image features. The forgery trace features are mapped by the first mapping unit and then input to the classifier. After calculation, the true and false detection results corresponding to image features of different sizes are output.

[0030] The final detection result is obtained by weighting and combining the true and false detection results corresponding to the features of all sizes.

[0031] To achieve the above-mentioned objectives, the embodiments also provide a deepfake video detection device based on multi-scale feature decoupling, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-mentioned deepfake video detection method.

[0032] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0033] By decoupling the forgery traces and realistic natural parts in the feature space of different scales, a discriminator is set up to compete with the feature extraction module. In the competition, the two types of partial features are separated, and more general forgery features are obtained to achieve accurate judgment. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the deepfake video detection method based on multi-scale feature decoupling provided in the embodiment;

[0036] Figure 2 This is a schematic diagram of the structure of the multi-scale feature decoupling network provided in the embodiment;

[0037] Figure 3 This is a schematic diagram of the decoupling module provided in the embodiment;

[0038] Figure 4 This is a schematic diagram of the structure of the self-attention unit provided in the embodiment;

[0039] Figure 5 This is a flowchart of a deepfake video detection method using a forgery detection model, provided in the embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0041] To address the problem of inaccurate detection due to poor generalization of existing deepfake video detection technologies, this invention proposes a deepfake video detection method and apparatus based on multi-scale feature decoupling.

[0042] like Figure 1 As shown in the embodiment, the deepfake video detection method based on multi-scale feature decoupling includes the following steps:

[0043] Step 1: Construct a multi-scale feature decoupling network.

[0044] like Figure 2As shown, the multi-scale feature decoupling network provided in this embodiment includes a feature extraction module and a decoupling module. The feature extraction module includes feature extraction units at multiple scales; for example, it includes low, medium, and high-scale feature extraction units. Each feature extraction unit contains two feature extraction branches of the same scale and a fusion operation. Color domain features and frequency domain features are extracted through the two feature extraction branches, and the fusion operation fuses the color domain features and frequency domain features to obtain image features. The feature extraction branches employ convolutional neural networks, preferably the InceptionResNet network, and the fusion operation employs convolutional operations.

[0045] Specifically, when extracting image features using the feature extraction module, firstly, for a given video V to be detected, it is converted into a series of image frames. Then, the dlib method is used to perform face detection bounding boxes and cropping on each frame to obtain a series of face images. A single face image is taken as the input face I of the feature extraction module. Then, given the input face image I, the pre-trained InceptionResNet network is used to extract color domain features. The features output by different scale layers of the network are used as the multi-scale features F of the face image. i , where i represents the level of the feature extraction unit. For example, a value of 3 can be used to indicate the use of low, medium, and high-level features.

[0046] To avoid bias towards specific forgery methods in the color domain, this invention additionally introduces frequency domain information at input, which is more robust and provides additional forgery trace information. For a given input face image I, an initial frequency domain noise feature X is extracted using an SRM filter. This is further extracted into different layers of frequency domain features X using an InceptionResNet network with the same structure as the color domain but different parameters. i After obtaining the color domain and frequency domain features of face images at various scales, to facilitate further decoupling of the two types of information, it is necessary to first fuse them. This involves connecting the color domain and frequency domain features at corresponding scales and obtaining image features M containing both through a 1×1 convolutional layer. i .

[0047] like Figure 2 As shown, a decoupling module is connected to the output of each feature extraction unit. This decoupling module is used to decouple the image features output by the feature extraction unit to obtain realistic natural features and forgery trace features, and to perform forgery detection based on the forgery trace features and adversarial training based on the realistic natural features.

[0048] like Figure 3As shown, the decoupling module includes a self-attention unit, a first mapping unit, a second mapping unit, a classifier, a class discriminator, and a domain discriminator. The image features are processed by the self-attention unit to calculate two complementary self-attention maps. These two self-attention maps are multiplied by the image features to obtain decoupled realistic natural features and forgery trace features. The forgery trace features are mapped by the first mapping unit and then input into the classifier for forgery detection. The realistic natural features are mapped by the second mapping unit and then input into the class discriminator and the domain discriminator for feature authenticity determination and feature domain assignment.

[0049] like Figure 4 As shown, in the self-attention unit SA, a self-attention mechanism is used to process image features M. i Calculations are performed to obtain a self-attention map, which serves as a self-attention map for realistic natural features. The specific process includes: mapping the initial image features to the latent spaces of query, key, and value using three 1×1 convolutional layers to obtain Q. i K i V i , i represents the feature level, reshaping Q i K i The size is n i ×c i ,n i =h i ×w i h i w i c i This represents the height, width, and number of channels of the i-th layer feature map. Afterwards, matrix multiplication and softmax activation are performed to obtain the self-attention map. The formula is as follows:

[0050]

[0051] Forgery traces and realistic / natural features are complementary. The face image to be detected can be decoupled into these two types of features. Therefore, the self-attention map of the forgery trace portion can be calculated from the self-attention map of the realistic / natural features, using the following formula: Self-attention map With M i Multiplication yields realistic and natural features, and self-attention maps are used to achieve this. With M i Multiplication yields forgery trace features. The decoupling module generates a discriminative branch and a classification branch here; realistic and natural features enter the discriminative branch, while forgery trace features enter the classification branch.

[0052] like Figure 4As shown, both the first mapping unit and the second mapping unit adopt a convolutional neural network (CNN). The CNN includes a downsampling layer, a convolutional layer, and a linear layer. The input realistic natural features and forgery trace features are converted into feature vectors through the downsampling layer, the convolutional layer, and the linear layer, respectively.

[0053] Step 2: Construct a loss function and train a multi-scale feature decoupling network to obtain a forgery detection model.

[0054] In this embodiment, realistic natural features are input into a class discriminator, which then determines the authenticity of these features. Adversarial learning is used to ensure correct decoupling of forgery trace features. Specifically, the realistic natural features are transformed by a second mapping unit and then input into the class discriminator to obtain a binary classification result. The class discrimination loss is then calculated.

[0055]

[0056] in, The vector represents the realistic and natural features obtained after mapping by the second mapping unit, and y represents the real facial feature label. This represents the predicted value of the class discriminator. This indicates that the average value is calculated within the batch. The goal is for the class discriminator to be able to distinguish between real facial features and realistic, natural features decoupled from fake faces.

[0057] Conversely to the training objective of the class discriminator, the feature extraction module aims to make the features decoupled from the forged face image approximate those of a real face, thus rendering the class discriminator indistinguishable. The formula for calculating the generation loss is as follows: and The first adversarial loss is used to train the feature extraction module and the class discriminator alternately. In the competition with opposite training directions, the network can more correctly decouple the two types of features, namely forgery traces and realistic and natural features, thereby improving the ability of the discriminative branch to learn more discriminative features.

[0058] In this embodiment, realistic natural features are input into a domain discriminator to determine the category of the feature domain. Adversarial learning is then used to ensure that the decoupled features are not biased towards any particular forgery method. The decoupled realistic natural features originate from forged face images generated by different forgery methods. Without constraints, these features can easily become biased towards the forgery methods in the training set, similar to methods that directly extract forgery traces. Therefore, a domain discriminator is needed to perform adversarial learning, aiming to distinguish the realistic natural features decoupled from faces generated by different forgery methods. The forgery methods in the training set originate from known information and are used as factual domain labels for faces. Real faces have an additional domain to distinguish them from forgery methods.

[0059] Specifically, the realistic natural features are transformed by the second mapping unit and then input into the domain discriminator to obtain the domain category discrimination result, and the domain discrimination loss is calculated. Where E represents the average within the batch, y k Indicates a field label, Representation domain discriminator pair Predicted value for domain category k, This indicates averaging within a batch.

[0060] Contrary to the goal of the domain discriminator, the feature extraction module aims to extract realistic and natural features from forged faces that have high commonality and lack obvious domain category information, thus enabling it to handle face images generated by unknown forgery methods. This is achieved by adding a gradient reversal layer before the domain discriminator to improve the domain discrimination loss. The gradient values ​​are reversed during backpropagation to the feature extraction module, thus calculating the generation loss that prevents the domain discriminator from distinguishing the forgery category. and The first adversarial loss is constituted, and based on the generated loss... This allows for optimization of the domain discriminator and feature extraction module. Adversarial approaches to domain categories can prompt the network to eliminate more generalized true features, thus enabling the method to have good generalization ability.

[0061] In this embodiment, forgery trace features are input into a classifier, which then makes a judgment on the facial features of the test subject. A classification loss is used to help the network learn the correct classification. Specifically, the forgery trace features are converted into feature vectors by a first mapping unit. Then, the feature vector The input is fed into a classifier to obtain a binary classification result, and the classification loss is calculated. y' represents the true / false label of the sample, p i This represents the predicted value from the classifier.

[0062] During training, the adversarial loss is balanced using hyperparameters. The final loss function of the entire multi-scale feature decoupling network includes the generation loss. and loss assessment Where α and β are hyperparameters, according to The entire multi-scale feature decoupling network is trained in an alternating manner, that is, first using... Optimize network parameters and reuse Optimize network parameters and extract and correctly detect forgery traces through decoupling.

[0063] After training, the self-attention unit, first mapping unit, and classifier in the feature extraction module and each decoupling module are extracted as a forgery detection model.

[0064] Step 3: Use a forgery detection model to detect deepfake videos.

[0065] like Figure 5 As shown, the implementation of deepfake video detection using a forgery detection model includes:

[0066] The face image to be tested is input into the forgery detection model. Each feature extraction unit of the feature extraction module extracts color domain features and frequency domain features at different scales, and then the color domain features and frequency domain features are fused to obtain the image features.

[0067] In the decoupling module corresponding to each feature extraction unit, the self-attention unit extracts the self-attention map corresponding to the forgery trace features based on the image, and calculates the forgery trace features with the image features. If there are three scales, the low-scale forgery trace features, medium-scale forgery trace features, and high-scale forgery trace features are obtained simultaneously. These forgery trace features are mapped by the first mapping unit and then input to the classifier. After calculation, the true and false detection results corresponding to the image features of different sizes are output.

[0068] The final detection result is obtained by weighting and combining the true and false detection results corresponding to image features of all sizes. Image features at different scales contain different information and can form complementary relationships. The final judgment of the face to be tested is obtained by multiplying the predefined weights with the true and false detection results of each scale and adding them together.

[0069] Based on the same inventive concept, the embodiment also provides a deepfake video detection device based on multi-scale feature decoupling, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the deepfake video detection method according to any one of claims 1-9, including the following steps:

[0070] Step 1: Construct a multi-scale feature decoupling network;

[0071] Step 2: Construct a loss function and train a multi-scale feature decoupling network to obtain a forgery detection model;

[0072] Step 3: Use a forgery detection model to detect deepfake videos.

[0073] In practical applications, the memory can be volatile memory located at the local end, such as RAM; it can also be non-volatile memory, such as ROM, FLASH, floppy disks, hard disks, etc.; or it can be a remote storage cloud. The processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field-programmable gate array (FPGA), meaning that the steps of a deepfake video detection method can be implemented using these processors.

[0074] The deepfake video detection method and apparatus provided in the above embodiments first obtain the color domain and frequency domain features of the fake face through pre-trained CNN and SRM convolutions. Then, a self-attention module decouples the features into two parts: realistic natural features and forgery trace features. In the realistic natural feature branch, a class discriminator is established to determine the authenticity category of the features, while a domain discriminator is established to determine which forgery technique the features originate from. The adversarial relationship between the discriminator and the feature extraction module is used to help the network more accurately decouple the forgery trace features. In the forgery trace feature branch, a classification loss is used to allow the network to learn the differences between real and fake faces. Finally, the two are combined and the model is trained end-to-end, thereby achieving accurate and highly generalized deepfake video detection, improving the accuracy and versatility of the deepfake video detection method.

[0075] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deepfake video detection method based on multi-scale feature decoupling, characterized in that, The method comprises the following steps: A multi-scale feature decoupling network is constructed, which comprises a feature extraction module comprising a plurality of scale feature extraction units, color domain features and frequency domain features of different scales are extracted from a face image respectively from a color domain and a frequency domain through the feature extraction units, and image features are obtained by fusing the color domain features and the frequency domain features, each feature extraction unit is connected with a decoupling module, the decoupling module comprises a self-attention unit, a first mapping unit, a second mapping unit, a classifier, a class discriminator and a domain discriminator, the image features pass through the self-attention unit to calculate two complementary self-attention maps, the two self-attention maps are multiplied with the image features to obtain a realistic natural feature and a fake trace feature, the fake trace feature is input into the classifier after being mapped by the first mapping unit for fake detection, and the realistic natural feature is input into the class discriminator and the domain discriminator respectively after being mapped by the second mapping unit for feature authenticity discrimination and feature domain attribution judgment; A first adversarial loss is constructed for the self-attention unit and the class discriminator to promote correct decoupling of the fake trace feature, a second adversarial loss is constructed for the self-attention unit and the domain discriminator to promote the decoupled feature not to be biased to a specific fake method, and a classification loss is constructed for the self-attention unit and the classifier to promote learning of correct classification, the multi-scale feature decoupling network is parameter-optimized by using face image samples and the constructed losses, and after the parameter optimization is completed, the self-attention unit, the first mapping unit and the classifier in each decoupling module and the feature extraction module are extracted as a fake detection model; Deep fake video detection is performed by using the fake detection model.

2. The method for deepfake video detection based on multi-scale feature decoupling according to claim 1, characterized in that, Each feature extraction unit comprises two feature extraction branches of the same scale and a fusion operation, color domain features and frequency domain features are extracted through the two feature extraction branches, and image features are obtained by fusing the color domain features and the frequency domain features through the fusion operation, wherein the feature extraction branch adopts a convolutional neural network, and the fusion operation adopts a convolution operation.

3. The method for deepfake video detection based on multi-scale feature decoupling according to claim 2, characterized in that, The feature extraction branch adopts an InceptionResNet network.

4. The method for deepfake video detection based on multi-scale feature decoupling according to claim 1, characterized in that, In the self-attention unit, a self-attention mechanism is adopted to calculate image features to obtain a self-attention map as a realistic natural feature self-attention map , according to the complementary relationship, the self-attention map of the counterfeit trace feature is calculated =1- , the image features are multiplied by , , respectively, to obtain decoupled realistic natural features and counterfeit trace features.

5. The method for deepfake video detection based on multi-scale feature decoupling according to claim 1, characterized in that, The first mapping unit and the second mapping unit both adopt a convolutional neural network, and the convolutional neural network comprises a down-sampling layer, a convolution layer and a linear layer, and the convolutional neural network converts input features into feature vectors through the down-sampling layer, the convolution layer and the linear layer.

6. The method for deepfake video detection based on multi-scale feature decoupling according to claim 1, characterized in that, The first adversarial loss includes a class discrimination loss that expects the class discriminator to be able to distinguish between real human face features and realistic natural features decoupled from the fake human face and a generation loss that expects the realistic natural features decoupled from the fake human face image to be close to the real human face features so that the class discriminator cannot distinguish wherein: wherein i represents an index of the image feature, represents a vector obtained by mapping the realistic natural feature through the second mapping unit, represents a real feature label of the face, represents a predicted value of the class discriminator, represents an average within a batch; During training, the self-attention unit and the class discriminator are alternately trained through the first adversarial loss.

7. The method for deepfake video detection based on multi-scale feature decoupling according to claim 1, characterized in that, The second adversarial loss includes a domain discrimination loss that hopes to be able to distinguish the realistic and natural features decoupled from the human faces generated by different forgery methods and a generation loss , wherein i represents an index of the image feature, represents a vector obtained by mapping a realistic natural feature through a second mapping unit, represents a domain label, k represents a domain category, represents a predicted value of the domain discriminator, represents an average within a batch; In training, the domain discrimination loss is made by adding a gradient reversal layer in front of the domain discriminator The gradient value is reversed when back-propagating to the self-attention unit, so as to calculate the generation loss that makes the domain discriminator unable to distinguish the fake category , and the domain discriminator and the self-attention unit are optimized according to the generation loss .

8. The method for deepfake video detection based on multi-scale feature decoupling according to claim 1, characterized in that, The classification loss is represented as: where i denotes an index of an image feature, denotes a true or false label of a sample, is a predicted value of a classifier, denotes an average within a batch; In training, the final loss function of the whole multi-scale feature decoupling network includes generation loss and discrimination loss , wherein, and are hyperparameters, according to and The sum of and, The whole multi-scale feature decoupling network is trained in an alternating manner.

9. The method for deepfake video detection based on multi-scale feature decoupling according to claim 1, characterized in that, The deep fake video detection by using the fake detection model comprises: A face image to be detected is input into the fake detection model, and different scale image features are extracted through each feature extraction unit of the feature extraction module; In the decoupling module corresponding to each feature extraction unit, a self-attention map corresponding to a fake trace feature is calculated according to the image through the self-attention unit, and the fake trace feature is calculated from the self-attention map and the image features, the fake trace feature is input into the classifier after being mapped by the first mapping unit, and a true-false detection result corresponding to an image feature of different sizes is output after calculation; The true-false detection results corresponding to all size image features are combined to obtain a final detection result.

10. A deepfake video detection device based on multi-scale feature decoupling, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the steps of the deepfake video detection method of any one of claims 1-9 when executing the computer program.

Images