A false face picture detection method based on misleading learning
By combining misleading learning methods with multi-scale latent features and a single-channel attention fusion network, the generalization and fairness issues of fake face image detection technology are solved, achieving higher detection accuracy and sensitivity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANCHANG UNIV
- Filing Date
- 2024-10-24
- Publication Date
- 2026-07-03
AI Technical Summary
Existing fake face image detection technologies suffer from poor model generalization ability and unfair detection for different statistical labels.
A misleading learning approach is adopted, which adaptively combines multi-scale latent features of fake and real images, uses a single-channel attention fusion network and a high-pass filter, designs a misleading learning loss function, and trains and fine-tunes the fake feature extractor.
It improves the model's generalization ability and fairness, increases sensitivity to forgery features, reduces reliance on specific forgery methods, and enhances detection accuracy.
Smart Images
Figure CN119445342B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of face image detection technology, and in particular to a method for detecting fake face images based on misleading learning. Background Technology
[0002] With the development of deep learning technology, fake facial images generated using deep neural networks are emerging in large numbers. This misinformation can mislead the public through news, advertisements, and other social media channels, posing a serious security risk to society. Especially given the rapid iteration and widespread use of facial recognition technology, the rapid and accurate identification of fake facial images has become particularly important. Existing techniques for detecting fake facial images can be broadly categorized into two types: data augmentation methods based on the original image and methods targeting convolutional neural networks or multi-head attention networks. However, these methods still face two significant challenges: poor generalization ability of the models and significant unfairness in detecting different statistical labels. Summary of the Invention
[0003] To address the shortcomings of existing technologies, the present invention aims to provide a fake face image detection method based on misleading learning, proposing a novel approach called misleading learning. During the training phase, multi-scale latent features of fake images are adaptively combined with multi-scale latent features of real images. By combining specific fake images with different statistical labels, the detector is misjudged, thereby improving the sensitivity of the fake feature extractor to subtle fake textures and enhancing the model's generalization and fair generalization capabilities.
[0004] A method for detecting fake face images based on misleading learning includes the following steps:
[0005] S1: Construct a prior knowledge acquisition module to build basic forgery artifact detection capabilities for forgery feature extractors and detectors;
[0006] S2: Building a knowledge flow for misleading learning and a knowledge flow for biased data;
[0007] S3: Introduce a single-channel attention fusion network to enable the model to adaptively select the required multi-scale fake latent features and multi-scale real image latent features;
[0008] S4: Construct a high-pass filter specifically for misleading learning to preprocess the forged images;
[0009] S5: Introduce misleading learning loss to constrain the training of the fake feature extractor;
[0010] S6: Fine-tuning with an adaptive added to the fake feature extractor.
[0011] Furthermore, S1 specifically includes the following steps:
[0012] S11: For the prior knowledge acquisition module construction, select two backbone network architectures with the same structural design, and use them as face forgery feature extractors D. e and real face detector D a ;
[0013] S12: Constructing the training dataset:
[0014]
[0015] Where X i Represents an image, Y i The label represents the identification tag: 0: real image, 1: fake image, and n represents the number of images.
[0016] S13: The prior knowledge acquisition module is constructed as follows:
[0017]
[0018] in These represent the latent features extracted from the face spoofing feature extractor and the real face detector, respectively.
[0019] Furthermore, S2 specifically includes the following steps:
[0020] S21: In misleading the flow of knowledge, a pair of images (Fake image, real image) are used as input, where As input to the forgery feature extractor As input to the real image detector; the specific process is as follows:
[0021]
[0022] Where D hy This represents a hybrid model with dual knowledge flows. This represents the mixed features obtained through a mixture model to represent misleading learning knowledge flow;
[0023] S22: The paranoid data knowledge flow is as follows:
[0024]
[0025] in This represents a pair of images (real image, real image) input into the dual-knowledge-flow hybrid model. This represents the hybrid features obtained by the biased data knowledge flow through a hybrid model.
[0026] Furthermore, S3 specifically includes the following steps:
[0027] S31: The multi-scale features captured by the fake feature extractor are represented as V e The multi-scale features captured by the real image detector are represented as V a Through a single-channel attention fusion network F sc The obtained fusion feature is represented as V h The details are as follows:
[0028] V h =F hy (V a ,F sc (V e ))
[0029] Where F hy For feature mixing modules;
[0030] S32: The single-channel attention fusion network first determines V... e Calculate the attention graph sc∈R 1*1*c Through the function θ e and θ a Calculate V e The maximum and average values for each channel are as follows:
[0031] sc = sigmoid(conv(θ) e (V e ))+conv(θ a (V e )))
[0032] Where sigmoid represents the activation function and conv represents the convolutional block;
[0033] S33: Use the obtained sc attention map to assign adaptive weights to the channels of the multi-scale fake features, as follows:
[0034] V sc =V e +sc*V e
[0035] Where V sc This represents multi-scale forged latent features enhanced by attention maps;
[0036] S34: Through the feature blending module F hy Mixed V sc and V a The details are as follows:
[0037] V h =F hy (V a ,F sc (V e ))=conv(concat(Vsc V a ))
[0038] Where conv is a convolutional block and concat is a feature concatenation operation.
[0039] Furthermore, S4 specifically includes the following steps:
[0040] S41: Preprocess the image that has passed the forgery feature extractor using the SRM high-pass filter;
[0041] S42: In the misleading learning phase, the SRM convolution kernel is set to gradient learnable mode, and adaptive updates are performed through the misleading process.
[0042] Furthermore, S5 designs two loss functions for the misleading learning phase: misleading classification loss and misleading contrastive loss.
[0043] S51: Misclassification loss, as detailed below:
[0044]
[0045] Where L c This refers to the classification loss caused by misleading the flow of knowledge in learning, L c_n This refers to the classification loss of paranoid data knowledge flow, L ce H represents the cross-entropy loss function. c Indicates the category header, Y f =fake, Y k ∈(real,fake);
[0046] S52: The total misclassification loss is expressed as follows:
[0047] L c_m =α1*L c +α2*L c_n
[0048] Where α1 and α2 represent hyperparameters;
[0049] S52: The specific losses due to misleading comparisons are as follows:
[0050] L con =max([m+||f anchor -f + ||2-||f anchor -f - ||2],0)
[0051] Where f anchor f represents the anchor feature of the image. + f represents a positive sample. -denoted as negative sample, and m represents the hyperparameter that measures the distance between anchor feature, positive sample feature, and negative sample feature;
[0052] S53: The overall loss function is expressed as follows:
[0053] L = L c_m +βL con
[0054] Where β is a hyperparameter.
[0055] Furthermore, S6 specifically includes the following steps:
[0056] S61: Guide a feature adapter by using a face forgery feature extractor D trained with a misleading learning loss L. e The extracted forgery features are fine-tuned across the entire dataset; the adapter employs the Xception model, reducing misleading learning of the high-pass filter β. mis_srm Information loss that occurs during the filtering process;
[0057] S62: The final detector structure used for inference includes a face forgery feature extractor D trained with a misleading learning loss L. e And the fine-tuned adaptive.
[0058] The beneficial effects of this invention are:
[0059] 1. In response to the challenges of poor generalization and lack of fair generalization ability faced by existing deepfake image detectors, this invention proposes for the first time a misleading learning training strategy.
[0060] 2. In response to the misleading learning training strategy, this invention proposes a multi-scale latent feature attention fusion module, which enables latent features to adaptively allocate importance parameters for each channel during the training process.
[0061] 3. In response to the misleading learning training strategy, this invention designs two knowledge data streams to prevent the model from relying too much on specific forgery methods. Attached Figure Description
[0062] Figure 1 This is a general block diagram of the present invention;
[0063] Figure 2 It is a single-scale multi-head self-attention processing module;
[0064] Figure 3 It is a hybrid model of dual knowledge flows;
[0065] Figure 4 It is a single-channel attention fusion network;
[0066] Figure 5This is a diagram of the adaptive fine-tuning process. Detailed Implementation
[0067] The present invention will be further described below with reference to specific embodiments, but the scope of protection of the present invention is not limited thereto. A method for detecting fake face images based on misleading learning, the overall block diagram of which is shown below. Figure 1 As shown, the main steps are as follows:
[0068] S1: Construct a prior knowledge acquisition module to build basic forgery artifact detection capabilities for forgery feature extractors and detectors.
[0069] The purpose of constructing a prior knowledge acquisition module is to establish a learning goal orientation for the misleading learning process and prevent overfitting caused by the model's excessive reliance on specific misleading features.
[0070] S11: For the construction of the prior knowledge acquisition module, this invention selects two backbone network architectures with identical structural designs, which are respectively used as face forgery feature extractors D. e and real face detector D a .
[0071] S12: Constructing the training dataset:
[0072]
[0073] Where X i Represents an image (real image, fake image), Y i This represents the identification label (0: real image, 1: fake image), and n represents the number of images.
[0074] S13: The prior knowledge acquisition module can be constructed as follows:
[0075]
[0076] in These represent the latent features extracted from the face spoofing feature extractor and the real face detector, respectively.
[0077] S2: Establish two types of knowledge flows: such as Figure 2 As shown, these represent misleading learning knowledge flow and paranoid data knowledge flow, respectively.
[0078] S21: In misleading the flow of knowledge, a pair of images (Fake image, real image) are used as input, where As input to the forgery feature extractor As input to the real image detector. The specific process is as follows:
[0079]
[0080] Where D hy A hybrid model representing dual knowledge flows, such as Figure 3 As shown. Its main structure includes: a prior knowledge acquisition module and a trained real face detector D. a And keep the parameters frozen, the second prior knowledge acquisition module is trained on the face forgery feature extractor D e It should be noted that its parameters are in a trainable mode. This indicates that the knowledge flow from misleading learning is obtained through a hybrid model, resulting in hybrid features.
[0081] S22: If only misleading learning knowledge flow is used, the extractor often relies on only a single forgery trace to mislead the detector. Therefore, this invention designs a paranoid data knowledge flow as follows:
[0082]
[0083] in This represents a pair of images (real image, real image) input into the dual-knowledge-flow hybrid model. This represents the hybrid features obtained by the biased data knowledge flow through a hybrid model.
[0084] S3: Construct a single-scale multi-head self-attention processing module and introduce a single-channel attention fusion network to enable the model to adaptively select the required multi-scale fake latent features and multi-scale real image latent features.
[0085] This invention designs a unique single-channel attention fusion network, such as Figure 4 As shown, we capture three fake latent features at three different scales and adaptively combine them with three corresponding real image features at the same scale.
[0086] S31: The multi-scale features captured by the fake feature extractor are represented as V e The multi-scale features captured by the real image detector are represented as V a Through a single-channel attention fusion network F sc The obtained fusion feature is represented as V h The details are as follows:
[0087] V h =F hy (V a ,F sc (V e ))
[0088] Where F hy This is a feature mixing module.
[0089] S32: The single-channel attention fusion network first determines V... e Calculate the attention graph sc∈R1*1*c Through the function θ e and θ a Calculate V e The maximum and average values for each channel are as follows:
[0090] sc = sigmoid(conv(θ) e (V e ))+conv(θ a (V e )))
[0091] Where sigmoid represents the activation function and conv represents the convolution block.
[0092] S33: Use the obtained sc attention map to assign adaptive weights to the channels of the multi-scale fake features, as follows:
[0093] V sc =V e +sc*V e
[0094] Where V sc This represents multi-scale forged latent features enhanced by attention maps. Each element of the attention map represents V. e The attention map determines the level of information contained in each channel of the forged features. Therefore, forged semantic channels with a higher level of information are assigned greater weights to gain enhancement, and vice versa.
[0095] S34: Through the feature blending module F hy Mixed V sc and V a The details are as follows:
[0096] V h =F hy (V a ,F sc (V e ))=conv(concat(V sc V a ))
[0097] Where conv is a convolutional block and concat is a feature concatenation operation.
[0098] S4: Construct a high-pass filter specifically for misleading learning to preprocess the forged images.
[0099] S41: The SRM kernel is used to calculate the residual between a pixel value and the predicted value derived from its neighboring pixels. Images processed by the fake feature extractor are preprocessed using an SRM high-pass filter.
[0100] S42: In the misdirected learning phase, the SRM convolutional kernel is set to a gradient-learnable mode, and adaptive updates are performed through a misdirected process. Specifically:
[0101]
[0102] Where, β srm This refers to the initialization of the convolutional kernel before adaptive updates. L refers to the total misdirection learning loss, described in S5. γ is used to weight the regularization term of the loss function. β mis_srm This represents the convolution kernel weights obtained after adaptive updating.
[0103] S5: Introduce misleading learning loss to constrain the training of the fake feature extractor.
[0104] This invention designs two loss functions for the misleading learning stage: misleading classification loss and misleading contrast loss.
[0105] S51: Misclassification loss, as detailed below:
[0106]
[0107] Where L c This refers to the classification loss caused by misleading the flow of knowledge in learning, L c_n This refers to the classification loss of paranoid data knowledge flow, L ce H represents the cross-entropy loss function. c Indicates the category header, Y f =fake, Y k ∈(real,fake).
[0108] S52: The total misclassification loss is expressed as follows:
[0109] L c_m =α1*L c +α2*L c_n
[0110] Where α1 and α2 represent hyperparameters.
[0111] S52: A misleading contrastive regularization loss is employed to emphasize the difference between misleading blended features and the original image, thereby enhancing the face forgery feature extractor. The misleading contrastive loss is detailed below:
[0112] L con =max([m+||f anchor -f + ||2-||f anchor -f - ||2],0)
[0113] Where fanchor f represents the anchor feature of the image. + f represents a positive sample. - Let f represent a negative sample, and m represent a hyperparameter that measures the distance between anchor features, positive sample features, and negative sample features. + and f anchor If f represents a misleading mixed feature, then - This represents the original image features processed by a real face detector.
[0114] In particular, for the two data streams, L con The tendency is to make the encoder learn as many similar but slightly different forged feature representations as possible.
[0115] S53: The total loss function can be expressed as follows:
[0116] L = L c_m +βL con
[0117] Where β is a hyperparameter.
[0118] S6: Fine-tuning with an adaptive added to the fake feature extractor.
[0119] S61: This invention leads to a new feature adapter that uses a face forgery feature extractor D trained with a misleading learning loss L. e The extracted forgery features are fine-tuned across the entire dataset. This adapter employs the Xception model, reducing misleading learning of the high-pass filter β. mis_srm Information loss that occurs during the filtering process. For example... Figure 5 As shown.
[0120] S62: The final detector structure used for inference includes a face forgery feature extractor D trained with a misleading learning loss L. e And the fine-tuned adaptive. For example... Figure 1 The misleading learning inference process is shown below. The resulting forgery vectors are more sensitive to forged textures.
[0121] To verify the generalization ability of the proposed model, it was tested on four widely used large datasets: FaceForensics++, DeepfakeDetection, Deepfake Detection Challenge, and Celeb-DF. FaceForensics++ was used as the main training dataset, containing five forgery techniques for generating fake face images: Deepfakes, Face2Face, FaceSwap, NerualTexture, and FaceShifter. It is important to note that FaceForensics++ has three compressed versions (the highly compressed version is more challenging for fake face image detectors): original, highly compressed, and low-compressed. This invention uses the low-compressed version by default.
[0122] The effectiveness of the proposed model was verified using four widely used convolutional neural networks (Xception, Resnet-34, Resnet-50, and Efficientnet-b4). Fairness and generalization ability were also compared with existing state-of-the-art detectors. By default, all methods use Xception as the backbone network model.
[0123] Previous work has demonstrated excellent generalization performance on in-domain datasets. However, significant challenges remain in achieving fairness. The main reason is that harmful semantics negatively impact the fairness performance of the detector. Table 1 illustrates that the method of this invention can refine the detector's sensitivity to forged semantics, reducing hyperparameter problems caused by overfitting the model to specific forged semantics and demographic feature semantics. Compared with existing methods, the results demonstrate that the method of this invention can improve the model's fairness capability.
[0124] Table 1. Test results of the ff++ domain dataset.
[0125]
[0126]
[0127] To evaluate the generalization ability of the model of this invention on cross-domain datasets, Table 2 shows the comparative results. All models were trained on FF++ and validated on three other datasets. The results show that the method of this invention significantly improves the performance level by 6%, achieving higher generalization performance.
[0128] Table 2 Test Results of Out-of-Domain Dataset
[0129]
[0130] The above description is merely a preferred embodiment of the present invention, used to illustrate the technical solution of the present invention, and not to limit the present invention. It should be noted that for those skilled in the art, modifications and substitutions to some technical features without departing from the concept of the present invention will not depart from the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.
Claims
1. A method for detecting fake face images based on misleading learning, characterized in that, Includes the following steps: S1: Construct a prior knowledge acquisition module to build basic forgery artifact detection capabilities for forgery feature extractors and detectors; S2: Constructing a knowledge flow for misleading learning and a knowledge flow for biased data, including the following steps: S21: In misleading the flow of knowledge, a pair of images As input, where This represents a fake image, which serves as input to the forgery feature extractor. The image represents a real image and serves as input to the real image detector; the specific process is as follows: Where D hy This represents a hybrid model of dual knowledge flows, f i hy This represents the mixed features obtained through a mixture model to represent misleading learning knowledge flow; S22: The paranoid data knowledge flow is as follows: in This represents a pair of images input into the dual-knowledge-flow hybrid model. and All images are genuine. This represents the hybrid features obtained by the hybrid model of the paranoid data knowledge flow; S3: Introduce a single-channel attention fusion network to enable the model to adaptively select the required multi-scale fake latent features and multi-scale real image latent features; S4: Construct a high-pass filter specifically for misleading learning to preprocess the forged images; S5: Introduce misleading learning loss to constrain the training of the fake feature extractor; two loss functions are designed for the misleading learning stage: misleading classification loss and misleading contrast loss. S51: Misclassification loss, as detailed below: L c =L ce (h c (f i hy ),Y f ) L c_n =L ce (h c (f i hy ),Y k ) Where L c This refers to the classification loss caused by misleading the flow of knowledge in learning, L c_n This refers to the classification loss of paranoid data knowledge flow, L ce H represents the cross-entropy loss function. c Indicates the category header, Y f =fake, Y k ∈(real,fake); S52: The total misclassification loss is expressed as follows: L c_m =α1*L c +α2*L c_n Where α1 and α2 represent hyperparameters; S52: The specific losses due to misleading comparisons are as follows: L con =max([m+||f anchor -f + ||2-||f anchor -f - ||2],0) Where f anchor f represents the anchor feature of the image. + f represents a positive sample. - denoted as negative sample, and m represents the hyperparameter that measures the distance between anchor feature, positive sample feature, and negative sample feature; S53: The overall loss function is expressed as follows: L=L c_m +βL con Where β is a hyperparameter; S6: Fine-tuning with an adaptive added to the fake feature extractor.
2. The method for detecting fake face images based on misleading learning according to claim 1, characterized in that, S1 specifically includes the following steps: S11: For the prior knowledge acquisition module construction, select two backbone network architectures with the same structural design, and use them as face forgery feature extractors D. e and real face detector D a ; S12: Constructing the training dataset: Where X i Represents an image, Y i The label represents the identification tag: 0: real image, 1: fake image, and n represents the number of images. S13: The prior knowledge acquisition module is constructed as follows: f i e =D e (X i ) f i a =D a (X i ) Where f i e f i a These represent the latent features extracted from the face spoofing feature extractor and the real face detector, respectively.
3. The method for detecting fake face images based on misleading learning according to claim 1, characterized in that, S3 specifically includes the following steps: S31: The multi-scale features captured by the fake feature extractor are represented as V e The multi-scale features captured by the real image detector are represented as V a Through a single-channel attention fusion network F sc The obtained fusion feature is represented as V h The details are as follows: V h =F hy (V a ,F sc (V e )) Where F hy For feature mixing modules; S32: The single-channel attention fusion network first determines V... e Calculate the attention graph sc∈R 1*1*c Through the function θ e and θ a Calculate V e The maximum and average values for each channel are as follows: sc=sigmoid(conv(θ e (V e ))+conv(θ a (V e ))) Where sigmoid represents the activation function and conv represents the convolutional block; S33: Use the obtained sc attention map to assign adaptive weights to the channels of the multi-scale fake features, as follows: V sc =V e +sc*V e Where V sc This represents multi-scale forged latent features enhanced by attention maps; S34: Through the feature blending module F hy Mixed V sc and V a The details are as follows: V h =F hy (V a ,F sc (V e ))=conv(concat(V sc ,V a )) Where conv is a convolutional block and concat is a feature concatenation operation.
4. The method for detecting fake face images based on misleading learning according to claim 3, characterized in that, S4 specifically includes the following steps: S41: Preprocess the image that has passed the forgery feature extractor using the SRM high-pass filter; S42: In the misleading learning phase, the SRM convolution kernel is set to gradient learnable mode, and adaptive updates are performed through the misleading process.
5. The method for detecting fake face images based on misleading learning according to claim 4, characterized in that, S6 specifically includes the following steps: S61: Guide a feature adapter by using a face forgery feature extractor D trained with a misleading learning loss L. e The extracted forgery features are fine-tuned across the entire dataset; the adapter employs the Xception model, reducing misleading learning of the high-pass filter β. mis_srm Information loss that occurs during the filtering process; S62: The final detector structure used for inference includes a face forgery feature extractor D trained with a misleading learning loss L. e And the fine-tuned adaptive.