A face forgery detection method and device based on domain adaptation

Abstract

The application provides a face forgery detection method and device based on domain adaptation, and the method comprises the following steps: training a Teacher model from a source domain; transferring the prior knowledge learned by the Teacher model to a target domain through domain adaptation to obtain a first Student model; adjusting the first Student model by using part of the samples of the target domain to obtain a Student model through training; and detecting the face image by using the trained Student model. The application can predict the authenticity of the face image of the target domain type. The fine-tuned Student model can achieve high detection accuracy with a simpler model structure and a small number of training samples.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and specifically to a method and apparatus for detecting face forgery based on domain adaptation. Background Technology

[0002] With the rapid development of deepfake technology, new forgery methods are emerging one after another, and the quality of forgeries is becoming increasingly realistic. Consequently, various forgery detection techniques (mostly data-driven methods) have been developed to detect various types of forged images. A common problem in detection tasks is that when a detection model is fully trained on a certain type of forged dataset, it achieves excellent performance when detecting data with the same distribution. However, when using the same model to detect other types of forged images, its performance drops significantly; that is, the model lacks generalization ability. However, in many cases, collecting and generating a large number of new forged samples for each type of dataset is impractical; that is, the amount of training samples is insufficient to retrain a new detection model for that type. Therefore, how to utilize the knowledge from existing large amounts of samples to meet the detection needs of other types of samples has profound research significance and application value.

[0003] In summary, the existing technology has the following problems: how to utilize the knowledge of a large number of existing samples to meet the detection needs of other types of samples and improve the adaptability and generalization of the face forgery detection model. Summary of the Invention

[0004] The purpose of this invention is to solve the problem of how to utilize the knowledge of a large number of existing samples to meet the detection needs of other types of samples and improve the adaptability and generalization of the face forgery detection model.

[0005] This application utilizes source domain knowledge to solve the target domain detection problem under the condition of scarce target domain data, thereby improving the model's adaptability and generalization.

[0006] To this end, in one aspect, embodiments of the present invention provide a face forgery detection method based on domain adaptation, the method comprising the following steps:

[0007] Train a Teacher model from the source domain;

[0008] Through domain adaptation, the prior knowledge learned by the Teacher model is transferred to the target domain to obtain the first Student model;

[0009] The first Student model is adjusted using samples from a portion of the target domain to train and obtain the Student model.

[0010] The trained Student model is used to detect faces in images.

[0011] On the other hand, embodiments of the present invention also provide a domain-adaptive face forgery detection device, comprising:

[0012] Training unit, used to train a Teacher model from the source domain;

[0013] The transfer unit is used to transfer the prior knowledge learned by the Teacher model to the target domain through domain adaptation, so as to obtain the first Student model.

[0014] The adjustment unit is used to adjust the first Student model using samples from a portion of the target domain, and train the Student model to obtain the Student model.

[0015] The detection unit is used to detect faces in images using a trained Student model.

[0016] The above technical solution has the following beneficial effects: This invention predicts the authenticity of face images of the target domain type. Utilizing a fine-tuned Student model, higher detection accuracy can be achieved with a simpler model structure and a smaller number of training samples. Attached Figure Description

[0017] Figure 1 This is a flowchart of a face forgery detection method based on domain adaptation provided in an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of a face forgery detection device based on domain adaptation provided in an embodiment of the present invention;

[0019] Figure 3 This is a flowchart of the Teacher model training process provided in an embodiment of the present invention;

[0020] Figure 4 This is a flowchart of the fine-tuning process for the Student model provided in an embodiment of the present invention;

[0021] Figure 5 This is a flowchart for predicting the authenticity of a face image provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Suppose we want to perform domain adaptation learning between Faceswap (source domain) and Face2Face (target domain), i.e., we have a large number of Faceswap type samples and a small number of Face2Face samples. Since we cannot train a target domain model that meets the performance requirements using only Face2Face data, the method proposed below aims to train an effective detection model for Face2Face by using a large number of Faceswap samples and a small number of Face2Face samples.

[0024] Since source domain data is often unavailable or retaining it may raise privacy concerns, we do not use source domain data during the fine-tuning of the Student model; we only utilize the parameters of the pre-trained Teacher model.

[0025] In embodiments of the present invention, such as Figure 1 This paper provides a face forgery detection method based on domain adaptation, the method comprising the following steps:

[0026] S101: Train a Teacher model from the source domain;

[0027] First, single-frame images are extracted from all real and fake videos. The MTCNN algorithm is then used to detect and align the face regions in the images. Next, the detected face regions are expanded outward by a factor of 1.3 in both width and height and cropped. The size of the cropped face image is then adjusted to 224×224×3. The processed face image is represented by the symbol I. (All subsequent image operations refer to operations on I).

[0028] Train Teacher model M in the source domain T For the source domain dataset χ s (Faceswap dataset), first train a Teacher model, the training process is as follows: Figure 3 As shown, a multi-view dual-stream prediction framework is adopted. First, the input image is converted into two complementary domains, namely the residual domain and the frequency domain. Then, the images from the two perspectives are respectively input into a backbone network with shared weights to learn features. After feature fusion and a classifier, the prediction result is output.

[0029] S102: Through domain adaptation, the prior knowledge learned by the Teacher model is transferred to the target domain to obtain the first Student model;

[0030] S103: Adjust the first Student model using samples from a portion of the target domain to train and obtain the Student model;

[0031] S104: Detect face images using a trained Student model.

[0032] Training a Teacher model from the source domain includes:

[0033] Convert the sample image in the source domain into feature maps of two complementary domains;

[0034] The feature maps of the two complementary domains are respectively input into a backbone network with shared weights to learn features;

[0035] Output the judgment result and build the Teacher model.

[0036] The original RGB image is converted into a residual image. Since image content information is irrelevant and its presence affects detection accuracy, the residual image aims to suppress image content while preserving very subtle forgery traces. Here, the residual image is obtained using a learning-based predictor; that is, an additional convolutional module Conv(·) is introduced to process the original RGB image to obtain predicted pixel values, and then the residual image I is obtained by subtracting the predicted pixel values ​​from the original pixel values. r ,Right now:

[0037] I r =I-Conv(I)

[0038] The original RGB image is converted into a frequency domain image. Previous research has shown that forged images exhibit abnormal frequency domain statistical properties compared to genuine images. Therefore, this method uses the frequency domain image as a complementary feature to the residual image; the combination of the two helps to uncover the essential differences between genuine and forged images. The frequency domain image is obtained as follows: For an input image I of size 224×224×3, it is first subjected to a discrete cosine transform D:

[0039] I d =D(I)

[0040] In the above formula, the obtained I d The size is 224×224×1. Since image details are high-frequency information, and the texture structure of the forged image changes, resulting in detail loss, a high-pass filter H is used to filter out low-frequency information in the image to amplify the forgery traces generated during the forgery process.

[0041] I h =H(I d ,α)

[0042] In the above formula, α controls the low-frequency components to be filtered out. Finally, the inverse Fourier transform D... -1 Will I h Then convert back to the original RGB color space to obtain the frequency domain representation of the input image:

[0043] I f =D -1(I h )

[0044] In the above formula, I f The size is 224×224×3.

[0045] The images from the two perspectives obtained above are input into a shared-weight backbone network to learn features. ResNet50 is chosen as the backbone network, and its network layer structure is shown in Table 1, consisting of 5 convolutional modules. Residual Image I r Frequency Domain Diagram I f After passing through the backbone network, a feature map of size 7×7×2048 will be output, denoted as... and

[0046] Table 1 Network Layer Structure

[0047]

[0048]

[0049] Feature fusion. The two feature maps output above are combined. and The fusion is performed using the `torch.where()` function in the PyTorch framework. This is because the feature maps... It only includes subtle forgery traces in the image; some key features gradually weaken or even disappear in deep networks, therefore... Some pixels in the map have a value of 0. To address this issue, the feature fusion method used here is to combine the feature maps... Zero-point reservation in China and Africa, using pixels with zero points. The values ​​of the corresponding points in the matrix are replaced, and the purpose of this is to use frequency domain features to compensate for the information loss of the residual features. The fused features are then used... This means that a 1×1 convolution kernel is used at the end. Perform cross-channel feature learning.

[0050] Classification and Prediction: The classifier used here is a fully connected layer with 2 nodes. After Softmax (T=1) mapping, the output value corresponds to the probability value of the respective class, that is, the probability of predicting whether the input image belongs to the real class (label y=1) or the fake class (label y=0). The Teacher model uses binary cross-entropy (BCE) as the loss function during training, defined as:

[0051]

[0052] In the above formula, y j and p j Representing the source domain χ respectively sThe true class label of image j and the probability predicted by the model to be the true class are given, where N is the total number of samples in the source domain. The Teacher model is trained on the Faceswap dataset using the above method. The trained Teacher model is fixed and will not be updated further; it will be used as the feature extractor for subsequent training of the Student model.

[0053] The step of adjusting the first Student model using samples from a portion of the target domain to train a new Student model includes:

[0054] Copy the weights of the Teacher model to the first Student model;

[0055] Obtain the soft labels for the Teacher model, and obtain the soft and hard predicted values ​​for the Student model;

[0056] Calculate the feature loss, KD loss, and BCE loss;

[0057] The Student model is obtained through training.

[0058] The objective function used to train the Student model is:

[0059] L S =αL fea +βL KD +γL BCE ;

[0060] Among them, L S Let L be the target loss function. fea For feature loss, L KD For KD loss, L BCE For BCE loss;

[0061] α, β and γ are weighting factors.

[0062] In the target domain χ t Fine-tuning the first Student model M S Knowledge distillation (KD) was employed when fine-tuning the first Student model. This technique utilizes prior knowledge learned on the large dataset (i.e., the source domain data) (i.e., the Teacher model) to transfer to the target domain, thereby improving the model's performance on the target dataset. The flowchart for fine-tuning the Student model is as follows: Figure 4 As shown.

[0063] The Teacher model (M) T The weights of ) are copied to the Student model (M). S ).like Figure 4As shown, the Student model is structurally simpler than the Teacher model. Its backbone network is identical to the pre-trained backbone network of the Teacher model, and the initialized weights are directly copied from the Teacher model. The training objective of the Student model is to learn the generalization ability of the Teacher model by leveraging the relationship between input and output established by the Teacher model on a large source domain dataset, in order to solve the prediction problem of target domain data.

[0064] Obtain soft labels for the Teacher model. Feed the target domain image into the pre-trained Teacher model to generate soft labels for training the Student model. The Softmax function with a temperature variable T is used to generate the soft labels; its expression is:

[0065]

[0066] In the above formula, z i and z j Let q represent the logits values ​​of the i-th and j-th classes output by the classifier, respectively. i This represents the probability of the i-th class after processing by the Softmax (T = t) function. J represents the total number of classes, here J = 2, i.e., two classes: true and false. T represents the temperature of the Student model, which is helped to mimic the Teacher model by softening the probability distribution of the classes. The higher T is, the smoother the probability distribution of the softmax output becomes, and the information carried by the negative labels will be relatively amplified, so the model training will pay more attention to the negative labels. In this method, the temperature parameter t = 20 is used.

[0067] The soft and hard predicted values ​​of the Student model are obtained. The target domain image is fed into the Student model, passing through a backbone network and a classifier, and then passed through a Softmax function with T=t and T=1 respectively to obtain the soft and hard predicted values. In this method, the temperature parameter t=20 is used.

[0068] Calculate the feature loss. To mitigate the effects of catastrophic forgetting and domain shifting, a feature loss is introduced when training the Student model on the target domain. The target domain χ... t The image is fed into the Teacher model, and after feature fusion, the features are obtained. target domain χ t The images are fed into the Student model, and after passing through the backbone network, features are obtained. The formula for calculating feature loss is:

[0069]

[0070] In the above formula, j represents the target domain χ.t The image in the image.

[0071] Calculate the KD loss. Similarly, to reduce the impact of catastrophic forgetting and domain shifting, the KD loss is introduced when training the Student model on the target domain. The formula for calculating the KD loss is:

[0072]

[0073] In the above formula, j represents the target domain χ. t In the graph, σ represents the Softmax function under temperature T = t, and y j and y j represents the logits values ​​output by the classifier for the Teacher model and the Student model, respectively, and N is the total number of training samples in the target domain.

[0074] Calculate the BCE loss. Since the Teacher model has a certain error rate when predicting target domain data, to reduce the transmission of these errors to the Student model, the BCE loss function is also used when fine-tuning the Student model, similar to the Teacher model. This means using the true class labels to constrain the Student model to accurately classify the target domain data. The BCE loss function is expressed as:

[0075]

[0076] In the above formula, y j and p j Representing the target domain χ t The true class label of image j and the probability that the Student model predicts it as the true class after applying the Softmax (T=1) function are given, where N is the total number of training samples in the target domain.

[0077] Fine-tuning the Student model M S In the target domain χ t When fine-tuning the Student model, the objective loss function L S From feature loss L fea KD loss L KD and BCE loss L BCE It consists of three parts, namely:

[0078] L S =αL fea +βL KD +γL BCE

[0079] In the above formula, α, β, and γ are weighting factors to control the proportions of the three loss terms (in this invention, α = β = γ = 1).

[0080] The trained Student model is used to detect faces in images, including:

[0081] Input one or more face images to be detected into the Student model;

[0082] After analysis by the backbone network, judgment by the classifier, and processing by Softmax, the final prediction result is output.

[0083] The system predicts the authenticity of face images of the target domain type. The Student model, fine-tuned using the above method, achieves high detection accuracy with a simpler model structure and fewer training samples. In practical prediction of face images of the target domain type, the following approach is adopted: Figure 5 The process is illustrated below. Specifically, one or more face images to be detected are input into the fine-tuned Student model, which is then processed by the backbone network, classifier, and Softmax (T=1) to finally output the prediction result.

[0084] This invention also provides a domain-adaptive face forgery detection device, such as... Figure 2 As shown, it includes:

[0085] Training unit 21 is used to train a Teacher model from the source domain;

[0086] The transfer unit 22 is used to transfer the prior knowledge learned by the Teacher model to the target domain through domain adaptation to obtain the first Student model.

[0087] The adjustment unit 23 is used to adjust the first Student model using samples from a portion of the target domain, and train the Student model.

[0088] The detection unit 24 is used to detect face images using the trained Student model.

[0089] The training unit includes:

[0090] The conversion module is used to convert sample images in the source domain into feature maps of two complementary domains;

[0091] The learning module is used to learn features by inputting the feature maps of two complementary domains into a backbone network with shared weights.

[0092] The module is used to output the judgment results and build the Teacher model.

[0093] The adjustment unit includes:

[0094] The copy module is used to copy the weights of the Teacher model to the first Student model;

[0095] The acquisition module is used to acquire the soft labels of the Teacher model and the soft and hard predicted values ​​of the Student model.

[0096] The calculation module is used to calculate the feature loss, KD loss, and BCE loss.

[0097] The training module is used to train the Student model.

[0098] The objective function used to train the Student model is:

[0099] L S =αL fea +βL KD +γL BCE ;

[0100] Among them, L S Let L be the target loss function. fea For feature loss, L KD For KD loss, L BCE For BCE loss;

[0101] α, β and γ are weighting factors.

[0102] The detection unit includes:

[0103] The input module is used to input one or more face images to be detected into the Student model;

[0104] The output module is used to analyze the data through the backbone network, then classify it, process it with Softmax, and finally output the prediction result.

[0105] The working method and principle of the face forgery detection device have been described in detail in the embodiments of the face forgery detection method, so they will not be repeated here.

[0106] This invention predicts the authenticity of face images of the target domain type. A fine-tuned Student model achieves high detection accuracy with a simpler model structure and fewer training samples.

[0107] The technical solutions of the present invention will be described in detail below with reference to specific application examples. For technical details not described in the implementation process, please refer to the relevant descriptions above.

[0108] Example 1:

[0109] This invention provides a face forgery detection method based on domain adaptation, the method comprising the following steps:

[0110] Step 1: Data Preprocessing. First, single-frame images are extracted from all real and fake videos. The MTCNN algorithm is used to detect and align face regions in the images. Then, the detected face regions are expanded outward by a factor of 1.3 in both width and height and cropped. The size of the cropped face image is then adjusted to 224×224×3. The processed face image is represented by the symbol I. (All subsequent image operations refer to operations on I).

[0111] Step 2: Train Teacher model M in the source domain T For the source domain dataset χ s (Faceswap dataset), first train a Teacher model, the training process is as follows: Figure 3 As shown, a multi-view dual-stream prediction framework is adopted. First, the input image is converted into two complementary domains, namely the residual domain and the frequency domain. Then, the images from the two perspectives are respectively input into a backbone network with shared weights to learn features. After feature fusion and a classifier, the prediction result is output.

[0112] Step 2.1: Convert the original RGB image into a residual image. Since image content information is irrelevant and its presence affects detection accuracy, the residual image aims to suppress image content while preserving very subtle forgery traces. Here, the residual image is obtained using a learning-based predictor. Specifically, an additional convolutional module, Conv(·), is introduced to process the original RGB image to obtain predicted pixel values. The residual image I is then obtained by subtracting the predicted pixel values ​​from the original pixel values. r ,Right now:

[0113] I r =I-Conv(I)

[0114] Step 2.2: Convert the original RGB image to a frequency domain image. Previous research has shown that forged images exhibit abnormal frequency domain statistical characteristics compared to genuine images. Therefore, this method uses the frequency domain image as a complementary feature to the residual image; the combination of the two helps to uncover the essential differences between genuine and forged images. The frequency domain image is obtained as follows: For an input image I of size 224×224×3, it is first subjected to a discrete cosine transform D:

[0115] I d =D(I)

[0116] In the above formula, the obtained I d The size is 224×224×1. Since image details are high-frequency information, and the texture structure of the forged image changes, resulting in detail loss, a high-pass filter H is used to filter out low-frequency information in the image to amplify the forgery traces generated during the forgery process.

[0117] Ih =H(I d ,α)

[0118] In the above formula, α controls the low-frequency components to be filtered out. Finally, the inverse Fourier transform D... -1 Will I h Then convert back to the original RGB color space to obtain the frequency domain representation of the input image:

[0119] I f =D -1 (I h )

[0120] In the above formula, I f The size is 224×224×3.

[0121] Step 2.3: Input the images from the two perspectives obtained in Steps 2.1 and 2.2 into a shared-weight backbone network to learn features. Here, ResNet50 is chosen as the backbone network. The network layer structure is shown in Table 1, consisting of 5 convolutional modules. Residual Image I r Frequency Domain Diagram I f After passing through the backbone network, a feature map of size 7×7×2048 will be output, denoted as... and

[0122] Step 2.4: Feature Fusion. Combine the two feature maps output from Step 2.3. and The fusion is performed using the `torch.where()` function in the PyTorch framework. This is because the feature maps... It only includes subtle forgery traces in the image; some key features gradually weaken or even disappear in deep networks, therefore... Some pixels in the map have a value of 0. To address this issue, the feature fusion method used here is to combine the feature maps... Zero-point reservation in China and Africa, using pixels with zero points. The values ​​of the corresponding points in the matrix are replaced, and the purpose of this is to use frequency domain features to compensate for the information loss of the residual features. The fused features are then used... This means that a 1×1 convolution kernel is used at the end. Perform cross-channel feature learning.

[0123] Step 2.5: Classification and Prediction. The classifier used here is a fully connected layer with 2 nodes. After Softmax (T=1) mapping, the output value corresponds to the probability value of the respective class, i.e., predicting the probability that the input image belongs to the real class (label y=1) or the fake class (label y=0). The Teacher model uses binary cross-entropy (BCE) as the loss function during training, defined as:

[0124]

[0125] In the above formula, y j and p j Representing the source domain χ respectively s The true class label of image j and the probability predicted by the model to be the true class are given, where N is the total number of samples in the source domain. The Teacher model is trained on the Faceswap dataset using the above method. The trained Teacher model is fixed and will not be updated further; it will be used as the feature extractor for subsequent training of the Student model.

[0126] Step 3: In the target domain χ t Fine-tuning Student model M S Knowledge distillation (KD) was employed when fine-tuning the Student model. This technique utilizes prior knowledge learned on large datasets (i.e., the source domain data) (i.e., the Teacher model) to transfer to the target domain, thereby improving the model's performance on the target dataset. The flowchart for fine-tuning the Student model is shown below. Figure 4 As shown.

[0127] Step 3.1: Convert the Teacher model (M) T The weights of ) are copied to the Student model (M). S ).like Figure 4 As shown, the Student model is structurally simpler than the Teacher model. Its backbone network is identical to the pre-trained backbone network of the Teacher model, and the initialized weights are directly copied from the Teacher model. The training objective of the Student model is to learn the generalization ability of the Teacher model by leveraging the relationship between input and output established by the Teacher model on a large source domain dataset, in order to solve the prediction problem of target domain data.

[0128] Step 3.2: Obtain soft labels for the Teacher model. The target domain image is fed into the pre-trained Teacher model to generate soft labels for training the Student model. The Softmax function with a temperature variable T is used to generate the soft labels; its expression is:

[0129]

[0130] In the above formula, z i and z j Let q represent the logits values ​​of the i-th and j-th classes output by the classifier, respectively. iThis represents the probability of the i-th class after processing by the Softmax (T = t) function. J represents the total number of classes, here J = 2, i.e., two classes: true and false. T represents the temperature of the Student model, which is helped to mimic the Teacher model by softening the probability distribution of the classes. The higher T is, the smoother the probability distribution of the softmax output becomes, and the information carried by the negative labels will be relatively amplified, so the model training will pay more attention to the negative labels. In this method, the temperature parameter t = 20 is used.

[0131] Step 3.3: Obtain the soft and hard predicted values ​​of the Student model. The target domain image is fed into the Student model, passing through the backbone network and classifier, and then passed through the Softmax function at T=t and T=1 respectively to obtain the soft and hard predicted values ​​of the Student model. In this method, the temperature parameter t=20 is used.

[0132] Step 3.4: Calculate the feature loss. To reduce the impact of catastrophic forgetting and domain shifting, a feature loss is introduced when training the Student model on the target domain. The target domain χ... t The image is fed into the Teacher model, and after feature fusion, the features are obtained. target domain χ t The images are fed into the Student model, and after passing through the backbone network, features are obtained. The formula for calculating feature loss is:

[0133]

[0134] In the above formula, j represents the target domain χ. t The image in the image.

[0135] Step 3.5: Calculate the KD loss. Similarly, to reduce the impact of catastrophic forgetting and domain shifting, the KD loss is introduced when training the Student model on the target domain. The formula for calculating the KD loss is:

[0136]

[0137] In the above formula, j represents the target domain χ. t In the graph, σ represents the Softmax function under temperature T = t, and y j and y j represents the logits values ​​output by the classifier for the Teacher model and the Student model, respectively, and N is the total number of training samples in the target domain.

[0138] Step 3.6: Calculate the BCE loss. Since the Teacher model has a certain error rate when predicting target domain data, to reduce the transmission of these errors to the Student model, the BCE loss function is also used when fine-tuning the Student model, similar to the Teacher model. This means using the true class labels to constrain the Student model to accurately classify the target domain data. The BCE loss function is expressed as:

[0139]

[0140] In the above formula, y j and p j Representing the target domain χ t The true class label of image j and the probability that the Student model predicts it as the true class after applying the Softmax (T=1) function are given, where N is the total number of training samples in the target domain.

[0141] Step 3.7: Fine-tune the Student model M S In the target domain χ t When fine-tuning the Student model, the objective loss function L S From feature loss L fea KD loss L KD and BCE loss L BCE It consists of three parts, namely:

[0142] L S =αL fea +βL KD +γL BCE

[0143] In the above formula, α, β and γ (in this paper, α = β = γ = 1) are weighting factors to control the proportion of the three loss terms.

[0144] Step 4: Predict the authenticity of face images of the target domain type. The Student model, fine-tuned in the above manner, achieves high detection accuracy with a simpler model structure and fewer training samples. In actual prediction of face images of the target domain type, the following method is used: Figure 5 The process is illustrated below. Specifically, one or more face images to be detected are input into the fine-tuned Student model, which is then processed by the backbone network, classifier, and Softmax (T=1) to finally output the prediction result.

[0145] This invention predicts the authenticity of face images of the target domain type. A fine-tuned Student model achieves high detection accuracy with a simpler model structure and fewer training samples.

[0146] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0147] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0148] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0149] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0150] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0151] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0152] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0153] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while disks typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0154] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A face forgery detection method based on domain adaptation, characterized in that, The method includes the following steps: Train a Teacher model from the source domain; Through domain adaptation, the prior knowledge learned by the Teacher model is transferred to the target domain to obtain the first Student model; The first Student model is adjusted using samples from a portion of the target domain to train and obtain the Student model. The trained Student model is used to detect faces in images. Training a Teacher model from the source domain includes: Convert the sample image in the source domain into feature maps of two complementary domains; The feature maps of the two complementary domains are respectively input into a backbone network with shared weights to learn features; Output the judgment result and build the Teacher model.

2. The face forgery detection method based on domain adaptation according to claim 1, characterized in that, The step of adjusting the first Student model using samples from a portion of the target domain to train a new Student model includes: Copy the weights of the Teacher model to the first Student model; Obtain the soft labels for the Teacher model, and obtain the soft and hard predicted values ​​for the Student model; Calculate the feature loss, KD loss, and BCE loss; The Student model is obtained through training.

3. The face forgery detection method based on domain adaptation according to claim 2, characterized in that, The objective function used to train the Student model is: L S =αL fea +βL KD +γL BCE ; Among them, L S Let L be the target loss function. fea For feature loss, L KD For KD loss, L BCE For BCE loss; α, β and γ are weighting factors.

4. The face forgery detection method based on domain adaptation according to claim 3, characterized in that, Face detection is performed on images using the Student model, including: Input one or more face images to be detected into the Student model; After analysis by the backbone network, judgment by the classifier, and processing by Softmax, the final prediction result is output.

5. A face forgery detection device based on domain adaptation, characterized in that, include: Training unit, used to train a Teacher model from the source domain; The transfer unit is used to transfer the prior knowledge learned by the Teacher model to the target domain through domain adaptation, so as to obtain the first Student model. The adjustment unit is used to adjust the first Student model using samples from a portion of the target domain, and train the Student model to obtain the Student model. The detection unit is used to detect faces in images using a trained Student model. The training unit includes: The conversion module is used to convert sample images in the source domain into feature maps of two complementary domains; The learning module is used to learn features by inputting the feature maps of two complementary domains into a backbone network with shared weights. The module is used to output the judgment results and build the Teacher model.

6. The face forgery detection device based on domain adaptation according to claim 5, characterized in that, The adjustment unit includes: The copy module is used to copy the weights of the Teacher model to the first Student model; The acquisition module is used to acquire the soft labels of the Teacher model and the soft and hard predicted values ​​of the Student model. The calculation module is used to calculate the feature loss, KD loss, and BCE loss. The training module is used to train the Student model.

7. The face forgery detection device based on domain adaptation according to claim 6, characterized in that, The objective function used to train the Student model is: L S =αL fea +βL KD +γL BCE ; Among them, L S Let L be the target loss function. fea For feature loss, L KD For KD loss, L BCE For BCE loss; α, β and γ are weighting factors.

8. A face forgery detection device based on domain adaptation according to claim 5, characterized in that, The detection unit includes: The input module is used to input one or more face images to be detected into the Student model; The output module is used to analyze the data through the backbone network, then classify it, process it with Softmax, and finally output the prediction result.

Images