Fairness-aware deepfake detection method and apparatus, device, and medium

Abstract

The application provides a fair perception deep fake detection method, device, equipment and medium, including: acquiring a target image, performing detection on the target image by using a deep fake detection model to obtain a deep fake detection result; the training of the deep fake detection model includes: performing causal mask learning and causal feature decomposition through two causal reasoning branches and a data set to obtain a directed acyclic graph, stable causal features and non-causal features; determining counterfactual features across population statistics according to the causal feature decomposition result, and performing causal consistency constraint on the counterfactual features and original features by using KL divergence; performing population statistics label prediction on the non-causal features, and performing label prediction by using a gradient reversal layer; and obtaining the deep fake detection model by using a joint training method according to counterfactual consistency group classification. The embodiments of the application have at least the following beneficial effects: the face fake detection accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and image forgery detection, and particularly relates to a fair-aware deepfake detection method, device, equipment and medium. BACKGROUND

[0002] With the rapid development of generative models and the open source of synthesis tools, deepfake content has become accessible. Ordinary users can now generate highly realistic face images and videos in a few minutes. This low-threshold generation capability is often exploited on social media platforms for attacks, defamation or controversy, and more worryingly, this technology is used for identity theft, fake evidence construction and other high-risk scenarios.

[0003] Existing technologies use visual cues to achieve deepfake detection, such as boundary artifacts, frequency inconsistencies and lip motion analysis, and have achieved promising performance. However, the core challenge of deepfake detection is not only to identify perceptible artifacts, but also to model the underlying generation mechanism and ensure the generalization ability between different data distributions. Factors such as diverse source videos, changes in demographic attributes and environmental conditions all pose major challenges.

[0004] Mainstream models show significant performance differences between different demographic groups, and this unfairness is usually caused by false associations between fake labels and demographic attributes in the training data, and reflects the excessive sensitivity of the model to who the fake object is rather than how it is fake. SUMMARY

[0005] The main purpose of the embodiment of the present application is to provide a fair-aware deepfake detection method, device, equipment and medium, which improves the face forgery detection accuracy.

[0006] One aspect of the present application provides a fair-aware deepfake detection method, comprising:

[0007] Obtaining a target image, performing detection on the target image using a deepfake detection model to obtain a deepfake detection result;

[0008] The training of the deepfake detection model comprises:

[0009] Obtaining a data set, wherein the image samples in the data set include corresponding image labels, and the image labels include binary labels and demographic group labels;

[0010] Causal masking learning and causal feature decomposition are performed through two causal reasoning branches and the data set to obtain a directed acyclic graph, and the stable causal features related to manipulation are extracted through the directed acyclic graph, and the non-causal features related to the demographic group labels are separated;

[0011] obtaining a causal feature decomposition result, determining a counterfactual feature across the population statistics according to the causal feature decomposition result, and performing a causal consistency constraint on the counterfactual feature and the original feature by using a KL divergence;

[0012] performing a population statistics label prediction on the acyclic feature by using the group classifier and the gradient reversal layer, and performing a label prediction by using the gradient reversal layer;

[0013] According to the counterfactual consistency group classification, a joint training method is used to perform a back propagation learning through the backbone network, the fake classification loss, the causal structure learning loss, the counterfactual consistency loss and the group classification loss, so as to obtain the deep fake detection model.

[0014] According to the fair perception deep fake detection method, the causal mask learning and the causal feature decomposition are performed through the two causal reasoning branches and the data set, so as to obtain a directed acyclic graph, which includes:

[0015] Batch data is selected from the data set, and the batch data is equally divided, and the equally divided batch data includes the same number of real samples and fake samples;

[0016] The equally divided batch data is learned by the causal reasoning branch to learn a causal mask and perform a causal feature decomposition:

[0017]

[0018] wherein, is an identifier of the equally divided batch data, is a causal feature mask, is a causal feature decomposition, is an acyclic feature decomposition;

[0019] The fake classifier of the causal reasoning branch Optimization is performed by using a minimum classification loss:

[0020]

[0021] wherein represents a binary cross-entropy loss, wherein is a predicted fake probability, and a binary label , is the equally divided batch data;

[0022] The execution structure loss of the causal reasoning branch processing:

[0023]

[0024] wherein, is a sparsity regularization term, denotes an adjacency matrix in vectorized form, and is a reconstruction loss, denotes is:

[0025]

[0026] wherein, is an eigenvector, is a demographic group label, is a finite set of group classes, is the total number of groups, denotes the total number of nodes in the node set, denotes an L2 reconstruction loss applied to the feature dimension;

[0027] adopting acyclicity constraints to learn a directed acyclic graph, acyclicity constraints are:

[0028]

[0029] denotes the trace of a matrix.

[0030] According to the fair-aware deep forgery detection method, wherein adopting acyclicity constraints to learn a directed acyclic graph includes:

[0031] According to the causal stable graph, adopting a learnable adjacency matrix to obtain the causal effect of node to node , reconstruct the value of, and the reconstruction method is:

[0032]

[0033] wherein is a connected input vector, project the causal feature to the class logits.

[0034] According to the fair-aware deep forgery detection method, wherein the causal mask learning and causal feature decomposition include:

[0035] adopting a binary mask to highlight the causal features related to manipulation, and the binary mask is denoted as:

[0036]

[0037] decomposing the feature vector into causal and non-causal parts through the binary mask is:​​

[0038]

[0039] wherein the decomposition aligns the causal and acyclic causal structures of the cause and effect parts.

[0040] According to the fair-aware deep forgery detection method, wherein the causal feature decomposition result is obtained, the counterfactual features across the demographic are determined according to the causal feature decomposition result, and the KL divergence is used to constrain the causal consistency according to the counterfactual features and the original features, including:

[0041] obtaining the causal feature decomposition results of the two causal inference branches and ;

[0042] The counterfactual features across the demographic are constructed by the causal feature decomposition results as:

[0043]

[0044] The consistency loss of the KL divergence is used to maintain the prediction consistency of the original features and the counterfactual features, and the consistency loss of the KL divergence is:

[0045]

[0046] wherein, is the consistency loss, is the KL divergence, is the forgery classifier.

[0047] According to the fair-aware deep forgery detection method, wherein the demographic label prediction of the acyclic feature is performed by the group classifier and the gradient reversal layer, and the label prediction is performed by the gradient reversal layer, including:

[0048] The group classifier is trained by minimizing the standard multi-class cross-entropy loss, wherein the group classifier is , and the group classification loss is:

[0049]

[0050] wherein, is the acyclic feature, and the cross-entropy loss is:

[0051]

[0052] wherein, wherein indicates the predicted class probability, is the true label, is the number of classes, which corresponds to the number of demographic categories.

[0053] During forward propagation of training, non-causal features are included. Insert it into the population classifier D for training; during the backpropagation of training, perform gradient inversion processing through the gradient inversion layer before inserting it into the population classifier D for training. The gradient inversion is as follows:

[0054]

[0055] An adversarial module focused on fairness is trained by simply minimizing the group classification loss:

[0056]

[0057] in Indicates hyperparameters, and This represents the group classification loss for bicausal inference branches.

[0058] According to the fair perception deepfake detection method, based on counterfactual consistency group classification, a joint training method is used. Backpropagation learning is performed through a backbone network, forgery classification loss, causal structure learning loss, counterfactual consistency loss, and group classification loss to obtain the deepfake detection model, including:

[0059] Based on the overall training objective, a deep forgery detection model is obtained by backpropagation learning using a backbone network, forgery classification loss, causal structure learning loss, counterfactual consistency loss, and group classification loss, and by employing a sharpness perception minimization strategy. The overall training objective is:

[0060]

[0061] in For the first The spoofing classification loss of each DAG branch; The causal structure learning loss for the k-th branch includes feature reconstruction error and graph structure constraints; The counterfactual consistency loss is the cross-group counterfactual sample in the k-th branch. Group classification loss.

[0062] Another aspect of the present invention provides a fairness-aware deepfake detection device, comprising:

[0063] The first module is used to acquire a target image, perform detection on the target image using a deepfake detection model, and obtain a deepfake detection result.

[0064] The training of the deepfake detection model includes:

[0065] a second module configured to obtain a dataset, wherein an image sample in the dataset comprises a corresponding image label, and the image label comprises a binary label and a demographic group label;

[0066] a third module configured to perform causal mask learning and causal feature decomposition through two causal inference branches and the dataset, to obtain a directed acyclic graph, to extract a stable causal feature related to manipulation through the directed acyclic graph, and to separate a non-causal feature related to the demographic group label;

[0067] a fourth module configured to obtain a result of the causal feature decomposition, to determine a counterfactual feature across demographics according to the result of the causal feature decomposition, and to perform a causal consistency constraint using KL divergence according to the counterfactual feature and an original feature;

[0068] a fifth module configured to perform demographic label prediction on the non-causal feature through a group classifier and a gradient reversal layer, and to perform label prediction using the gradient reversal layer;

[0069] a sixth module configured to perform counterfactual consistency group classification using a joint training method, to perform backpropagation learning through a backbone network, a fake classification loss, a causal structure learning loss, a counterfactual consistency loss, and a group classification loss, and to obtain the deep fake detection model.

[0070] Another aspect of the embodiment of the present application provides an electronic device comprising a processor and a memory;

[0071] The memory is configured to store a program;

[0072] The processor executes the program to implement the method as described above.

[0073] The embodiment of the present application also discloses a computer readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the fair and aware deep fake detection method.

[0074] The embodiment of the present application has at least the following beneficial effects: the stable fake representation is learned based on the invariant causal inference branch, the false correlation is reduced, the fairness and generalization ability are improved, and the fake face image detection effect is improved; the counterfactual alignment mechanism is used to promote demographic invariance prediction by recombining causal and non-causal features, so that the detection is focused on the causal features related to the fake detection task; the adversarial decoupling branch is used to isolate the demographic clues through gradient reversal, prevent demographic information from leaking into the detection flow, improve the detection accuracy, and achieve enhanced robustness and fairness under demographic distribution changes.

[0075] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the attendant drawings or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0076] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0077] Figure 1 is a fair-aware deep forgery detection method flowchart diagram of an embodiment of the present application.

[0078] Figure 2 is a fair-aware directed acyclic graph-based causal modeling, counterfactual consistency constraint and group-invariant representation learning graph by gradient reversal diagram of an embodiment of the present application.

[0079] Figure 3 is a saliency map of PFG-DD and the method of an embodiment of the present application on four data sets. Figure 3 (a) is a saliency map generated by PFG-DD and an embodiment of the present application on four data sets, Figure 3 (b), Figure 3 (c) is a loss landscape comparison diagram of PFG-DD and an embodiment of the present application respectively shown at different angles.

[0080] Figure 4 is a fair-aware deep forgery detection device diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0081] Embodiments of the present application are described in detail below with reference to the attached drawings, which show by way of example embodiments in which the same or similar elements are denoted by the same or similar reference numerals throughout the drawings. In the following description, the suffixes "module", "part" or "unit" used to denote elements are used only for convenience of explanation of the present application, and have no particular meaning by themselves. Therefore, "module", "part" or "unit" can be used interchangeably. "First", "second", etc. are used only for the purpose of distinguishing technical features for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the order of the indicated technical features. In the following description, the consecutive numbers of the method steps are for the convenience of review and understanding, and in combination with the overall technical solution of the present application and the logical relationship between the steps, adjusting the implementation order between the steps will not affect the technical effects achieved by the technical solution of the present application. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0082] Reference Figure 1 , whereinFigure 1 is a fair-aware deep forgery detection method flowchart of an embodiment of the present application, which includes but is not limited to steps S100-S600:

[0083] Wherein S100 is a deep forgery detection step, including:

[0084] S100, obtaining a target image, performing detection on the target image using a deep forgery detection model to obtain a deep forgery detection result.

[0085] In some embodiments, the target image can be a forged image or a real image, where the images can be face images, certificate images, etc.

[0086] Wherein S200-S600 are training steps of the deep forgery detection model, including:

[0087] In some embodiments, the reference Figure 2 The fair-aware directed acyclic graph-based causal modeling, counterfactual consistency constraint and group-invariant representation learning graph through gradient reversal shown in the figure mainly includes:

[0088] (1) a causal reasoning module based on invariant DAGs, for learning stable forgery representations;

[0089] (2) a counterfactual alignment mechanism to promote demographic invariance prediction by reorganizing causal and acyclic features;

[0090] (3) an adversarial decoupling branch to isolate demographic cues through gradient reversal, preventing demographic information from leaking into the detection stream.

[0091] S200, obtaining a dataset, wherein the image samples in the dataset include corresponding image labels, and the image labels include binary labels and demographic group labels.

[0092] In some embodiments, wherein the dataset, the input image is represented as where H and W represent the height and width of the image respectively, each sample is associated with a binary label wherein represents a forged image, represents a real image. In addition, each sample is labeled with a demographic group label wherein is a finite set of group categories, is the total number of groups. For example, G can represent age groups and the like.

[0093] The training set of the embodiment of the present application is:

[0094]

[0095] wherein , test set includes demographic groups that were not seen during training, i.e. .

[0096] An objective of embodiments of the present invention is to learn a detection function for detecting distribution shifts and demographic aspects.

[0097] S300, by two causal inference branches and a dataset, causal mask learning and causal feature decomposition are performed to obtain a directed acyclic graph, and through the directed acyclic graph, a stable causal feature related to manipulation is extracted, and a non-causal feature related to a demographic group label is separated.

[0098] It can be understood that the manipulation-related refers to a fake trace, etc.

[0099] In some embodiments, for an invariant causal graph of fair deep fake detection, in order to enhance the fairness-focused deep fake detection, embodiments of the present invention adopt a causal perspective, and use a domain-invariant directed acyclic graph (DAG) to model the relationship between features and labels. Let the feature vector be , and the corresponding label be Embodiments of the present invention represent a unified node set as , wherein each represents the th feature component, represents a fake label.

[0100] In some embodiments, the causal stable graph is:

[0101] For each independent (e.g. age) domain e, embodiments of the present invention are in a structural causal model on the variable set , and the joint distribution is . According to the Markov condition, the distribution can be decomposed as:

[0102]

[0103] where denotes the parent node set (i.e. direct cause) of node in domain , and each SCM derives a DAG specific to the domain , wherein denotes a causal edge.

[0104] In some embodiments, the learning of the causal inference branch is: ​

[0105] An invariant graph G is defined as a shared structure across all domains, i.e. The intersection of G and G. According to Theorem 1, if G captures the common structure, it will remove domain-private dependencies and restore the true causal direction. .

[0106] In this embodiment of the invention, a learnable adjacency matrix is ​​used in each branch. To instantiate this theoretical framework, where Quantified from node To the node The causal effect. (The first...) Each node value is reconstructed in the following way:

[0107]

[0108] in It is the input vector of the connection. Project causal features onto category logits.

[0109] In some embodiments, causal feature identification includes:

[0110] based on This embodiment of the invention derives a binary mask. To highlight the causal characteristics associated with manipulation

[0111]

[0112] Then the eigenvector is decomposed into causal and non-causal components:

[0113]

[0114] This decomposition aligns with the domain-invariant causal structure G, filtering out spurious factors specific to the domain and retaining only those causally relevant to fabricated decisions. Through... By enforcing acyclicity and utilizing cross-domain graph merging, the model in this embodiment of the invention mitigates spurious associations and improves fairness and generalization ability.

[0115] In some embodiments, a two-branch causal reasoning method is used. To enhance the generalization ability of feature decoupling, embodiments of the present invention design two independent causal reasoning branches, denoted as follows: and During each training iteration, a mini-batch of data is split into two groups. ,in Each group includes the same number of real and fake samples from that batch and is fed into the corresponding DAG branch to independently learn the causal mask. And perform causal feature decomposition.

[0116]

[0117] Each branch is equipped with a dedicated forgery classifier optimized by minimizing the classification loss:

[0118]

[0119] Here, denotes the binary cross-entropy loss, where is the predicted forgery probability. In addition, to stabilize the learning of causal structures, each branch also minimizes a structure loss.

[0120]

[0121] Reconstruction loss to ensure that the learned causal graph can effectively reconstruct the original features and align with the true labels

[0122]

[0123] where denotes the L2 reconstruction loss applied to the feature dimensions. The sparsity regularization term encourages a sparse causal graph structure by penalizing redundant connections, where denotes the vectorized form of the adjacency matrix The acyclicity constraint ensures that the learned structure remains a strict directed acyclic graph (DAG).

[0124]

[0125] Here, denotes the trace of a matrix.

[0126] S400, obtaining a causal feature decomposition result, determining a counterfactual feature across the population statistics according to the causal feature decomposition result, and using KL divergence to constrain the causal consistency according to the counterfactual feature and the original feature.

[0127] In some embodiments, in order to enhance the robustness of the model in the cross-population statistics generalization scene, a counterfactual feature generation and causal consistency constraint mechanism based on double-branch causal feature decomposition is proposed.

[0128] Counterfactual generation, obtain causal feature decompositions from two branches and The present embodiment constructs a counterfactual feature across the population statistics as follows:

[0129]

[0130] The causal consistency constraint, the embodiments of the present application use a consistency loss based on KL divergence to enforce the predicted consistency between the original features and the corresponding counterfactual features, encouraging the model to focus more on the causal features related to the forgery detection task.

[0131]

[0132] S500, demographic label prediction of the acausal features is performed by the group classifier and the gradient reversal layer, and the label prediction is performed by using the gradient reversal layer.

[0133] It can be understood that, although the acausal features are excluded from the forgery prediction path, they still encode statistical clues related to demographic attributes. In order to prevent the model from inadvertently exploiting such spurious correlations and enhance its generalization ability under changes in demographic distribution, the embodiments of the present application introduce a gradient reversal mechanism to explicitly enforce group-invariant representation. This adversarial design ensures that demographic-related information is effectively decoupled from the main causal prediction stream.

[0134] For the group classifier and the gradient reversal layer, the embodiments of the present application design a group classifier that predicts demographic labels based on the acausal features . The classifier is trained by minimizing the standard multi-class cross-entropy loss:

[0135]

[0136] Here, denotes the demographic group label, denotes the current sample subset from a half of the training batch. The cross-entropy loss is defined as

[0137]

[0138] where denotes the predicted class probability, is the true label, is the number of classes, corresponding to the number of demographic categories. During forward propagation, is fed into the group classifier D through demographic mapping. During backpropagation, a gradient reversal layer (GRL) is inserted before D to reverse the gradient and prevent group-related information from being encoded into the shared representation.

[0139]

[0140] This mechanism ensures that the group classifier D is trained to minimize the group classification loss and accurately predict demographic labels. Meanwhile, the feature extractor receives backpropagated gradients through GRL, which implicitly maximizes so that the extracted features are encouraged to be group-invariant and less informative for demographic prediction. This adversarial mechanism enables the model to optimize group-invariant feature representations without introducing any additional loss terms.

[0141] The final fairness-aware adversarial module is trained by simply minimizing the group classification loss:

[0142]

[0143] Hyperparameters controls the weight of this term in the total loss function. This adversarial mechanism works in conjunction with the causal feature disentanglement module to effectively enhance the fairness performance of the model on unseen demographic groups.

[0144] S600, according to the counterfactual consistency group classification, a joint training method is adopted, and the deep fake detection model is obtained by backpropagation learning through the backbone network, the fake classification loss, the causal structure learning loss, the counterfactual consistency loss and the group classification loss.

[0145] The embodiment of the application adopts a joint training strategy, in which all module parameters are optimized simultaneously based on a shared backbone network. The total training target is defined as follows:

[0146]

[0147] fake classification causal graph structure counterfactual consistency group classification

[0148] Specifically:

[0149] : fake classification loss (cross-entropy) of the first DAG branch;

[0150] : causal structure learning loss of the first branch, including feature reconstruction error and graph structure constraint; : counterfactual consistency loss (KL divergence) of cross-group counterfactual samples in the first branch;

[0151] : group classification loss, applied only through the gradient reversal path.

[0152] : group classification loss, applied only through the gradient reversal path.

[0153] All loss components are jointly optimized through backpropagation. Importantly, due to the introduction of the gradient reversal mechanism, ​​The opposite direction gradients are provided for the group classifier and the feature extractor, so that the group invariance representation can be effectively learned. Without modifying the loss formula, the embodiments of the present application further add a sharpness-aware minimization (SAM) strategy during training to improve the generalization ability and optimization stability of the model. This helps to alleviate the adverse effects of sharp loss patterns on fairness and cross-domain performance.

[0154] Experiments of the embodiments of the present application are shown as follows:

[0155] Dataset. To evaluate the fairness generalization ability of the proposed method in deepfake detection, the embodiments of the present application perform experiments on several widely used benchmark datasets. The training data comes from the FaceForensics++ (FF++) dataset, which includes fake images generated by various face manipulation techniques, including DeepFakes, Face2Face, FaceSwap, NeuralTextures and FaceShifter.

[0156] To perform the evaluation, the embodiments of the present application test the model on FF++, DeepFake Detection (DFD), DeepFake Detection Challenge (DFDC) and Celeb-DF, which cover different fake methods and demographic distributions. To be able to perform fairness evaluation, the embodiments of the present application adopt the provided demographic attribute annotations and construct cross-group labels.

[0157] Evaluation metrics. To comprehensively evaluate the model of the embodiments of the present application, the embodiments of the present application consider both the overall detection performance and demographic fairness. The embodiments of the present application report AUC as the main accuracy indicator. To evaluate the fairness of the model of the embodiments of the present application among different demographic groups, the embodiments of the present application adopt the indicator suite proposed by Lin et al., which includes four standards: the fair false positive rate (FPFp), which measures the maximum difference in false positive rate among groups; the fair false negative rate (FNFn), which quantifies the difference in false negative rate; the fair detection parity (FPD), which evaluates the bias in class detection rate; and the fair overall accuracy parity (FOA), which evaluates the consistency of overall accuracy among subgroups.

[0158] Comparison methods. The embodiments of the present application compare the embodiments of the present application with the following methods: UCF, DAW-FDD and PFG-DD. All methods use the Xception backbone network to implement.

[0159] ​​​​Implementation details. All experiments are implemented using PyTorch and trained on an NVIDIA A100 GPU. The backbone network is Xception. During training, the batch size is set to 128 and the maximum number of training epochs is 100. Stochastic gradient descent (SGD) is used as the optimizer with an initial learning rate of . The loss weights are set as follows:

[0160] = 0.1

[0161] = 0.1

[0162] = 0.5

[0163] Main results:

[0164] Protocol 1, Performance on in-domain datasets. First, we evaluate the detection performance and fairness of our model on the five fake subsets (DF, F2F, FS, NT, FST) of FF++. To assess its fitting ability under known manipulation types, we evaluate the performance of our model on the FF++ dataset. As shown in Table 1, on the FF++ dataset, our model consistently outperforms DAW-FDD and PFG-DD on all five fake types (F2F, FS, NT, DF, and FST). Our model achieves the highest detection accuracy (AUC) and the lowest fairness bias metrics (e.g., and ) on all subsets. Compared to existing detection methods that focus on fairness (e.g., DAW-FDD and PFG-DD), our framework more effectively eliminates spurious associations related to demographic attributes while maintaining sensitivity to fake signals.

[0165] In particular, for more challenging manipulations, such as FaceSwap and NeuralTextures, our model significantly reduces the difference in false positive rates between different demographic groups ( ).

[0166] Protocol 2, Performance on cross-domain datasets. To evaluate the model’s generalization ability across different domains and unseen demographic groups, we perform fairness evaluation under the cross-dataset setting. All models are trained on FF++ and tested on DFD, DFDC, and Celeb-DF. We use four fairness metrics— , , and —along with AUC to measure overall detection performance. As shown in Table 2, our model outperforms DAW-FDD and PFG-DD on all fairness metrics (including 、 、 and ) while maintaining competitive or superior detection performance compared to all baseline methods.

[0167] In particular, on DFDC, the embodiments of the present application achieve an average fairness bias of 21.57, outperforming PFG-DD and all other baseline methods. On Celeb-DF, the embodiments of the present application further improve the fairness performance compared to PFG-DD while also achieving the highest AUC of 76.91. Although the fairness bias on DFD is relatively low for all methods, the embodiments of the present application still achieve the best trade-off. These results demonstrate the robustness of the embodiments of the present application in learning fair and transferable representations across unseen domains and demographic distributions.

[0168] Ablation study, To verify the effectiveness of the proposed method, the embodiments of the present application conduct an ablation study on four benchmark datasets. The goal is to evaluate the contribution of the three core components to fairness and detection performance.

[0169] Single-DAG: Using the single-branch causal graph module, without counterfactual consistency learning or adversarial debiasing through gradient reversal.

[0170] Dual-DAG: Including the dual-branch causal reasoning structure and cross-group counterfactual consistency learning, but excluding adversarial debiasing through gradient reversal.

[0171] Ours(Full): The complete model including dual-branch causal decomposition, counterfactual feature alignment, and adversarial debiasing through gradient reversal.

[0172] Table 1 Protocol 1, In-domain performance evaluation for different types of forgeries in FF++

[0173]

[0174] Table 2 Protocol 2, Inter-domain generalization evaluation

[0175]

[0176] Table 3 Ablation study comparison of the effect of each module

[0177]

[0178] As shown in Table 3, the Single-DAG baseline including only the basic causal graph module exhibits the highest fairness bias on all domains ( ​​) and the lowest robustness. The introduction of Dual-DAG structure and cross-group counterfactual consistency significantly improves fairness on all datasets. This shows that the combination of dual-branch causal decomposition and counterfactual reorganization enhances the stability and generalization ability of causal representation by mitigating spurious demographic correlations. This is further confirmed in the loss landscape of Figure 3 (b) and Figure 3 (c), where the Dual-DAG model exhibits a flatter and more stable surface.

[0179] After joining the adversarial debiasing module (OursFull), the model further reduces demographic prediction disparities and achieves better fairness generalization without sacrificing detection accuracy. These results show that the proposed components are complementary in fairness modeling, and their joint optimization effectively suppresses spurious correlations and improves robustness under demographic distribution changes.

[0180] Significance map visualization. As shown in Figure 3 (a), the PFG-DD and the embodiments of the present application visualize the significance maps generated on four datasets. PFG-DD usually focuses on broad facial regions including demographic sensitive areas, which indicates the presence of potential demographic bias. In contrast, the embodiments of the present application always localize attention on the manipulation artifacts (e.g., mouth, facial boundary), demonstrating improved fairness and robustness. This shows that causal decoupling and group invariance learning help the model focus on manipulation evidence rather than spurious cues.

[0181] FPR disparity across cross-demographic groups. The embodiments of the present application visualize the false positive rate across cross-demographic subgroups to assess fairness disparity, and compared with DAW-FDD and PFG-DD, the embodiments of the present application always reduce the FPR gap between the most and least affected groups, especially under challenging demographic conditions. This shows that fairness is improved at the sub-group level.

[0182] UMAP visualization of group-invariant representation. To further analyze how the gradient reversal mechanism promotes fairness, the embodiments of the present application use UMAP to visualize the feature representation before and after GRL.

[0183] In contrast, the bottom row presents features from the auxiliary group classification branch downstream of GRL. These features show clear clustering according to demographic attributes, indicating that group-related signals are preserved for adversarial supervision but successfully decoupled from the main task. This qualitative evidence highlights the key role of GRL in preventing demographic information from leaking into the causal prediction path, thereby improving fairness generalization across domains and subgroups.

[0184] Figure 4 is a defect detection analysis device diagram of interactive code coverage of an embodiment of the present application. The device includes a first module 610, a second module 620, a third module 630, a fourth module 640, a fifth module 650, and a sixth module 660.

[0185] The first module is configured to obtain a target image, perform detection on the target image using a deep forgery detection model, and obtain a deep forgery detection result. Training of the deep forgery detection model includes the following steps: the second module is configured to obtain a data set, wherein the image samples in the data set include corresponding image labels, and the image labels include binary labels and demographic group labels; the third module is configured to perform causal mask learning and causal feature decomposition through two causal reasoning branches and the data set, obtain a directed acyclic graph, extract stable causal features related to manipulation through the directed acyclic graph, and separate non-causal features related to the demographic group labels; the fourth module is configured to obtain the causal feature decomposition result, determine cross-demographic counterfactual features according to the causal feature decomposition result, and perform causal consistency constraint on the counterfactual features and original features using KL divergence; the fifth module is configured to perform demographic label prediction on the non-causal features through a group classifier and a gradient inversion layer, and perform label prediction using the gradient inversion layer; and the sixth module is configured to perform counterfactual consistency group classification using a joint training method, perform backpropagation learning through a backbone network, a forgery classification loss, a causal structure learning loss, a counterfactual consistency loss, and a group classification loss, and obtain the deep forgery detection model.

[0186] Exemplarily, under cooperation of the first module to the sixth module in the device, the embodiment device can implement any one of the defect detection methods of the interactive code coverage rate as described above, that is, obtaining a target image, performing detection on the target image by using a deep forgery detection model to obtain a deep forgery detection result; training of the deep forgery detection model comprises: obtaining a data set, wherein image samples in the data set comprise corresponding image labels, and the image labels comprise binary labels and demographic group labels; performing causal masking learning and causal feature decomposition through two causal inference branches and the data set to obtain a directed acyclic graph, extracting stable causal features related to manipulation through the directed acyclic graph, and separating non-causal features related to the demographic group labels; obtaining a causal feature decomposition result, determining cross-demographic counterfactual features according to the causal feature decomposition result, and performing causal consistency constraint on the counterfactual features and original features by using KL divergence; performing demographic label prediction on the non-causal features by using a group classifier and a gradient inversion layer, and performing label prediction by using the gradient inversion layer; and performing counterfactual consistency group classification, adopting a joint training method, and performing back propagation learning through a backbone network, a forgery classification loss, a causal structure learning loss, a counterfactual consistency loss, and a group classification loss to obtain the deep forgery detection model.The beneficial effects of the present application are as follows: program behaviors are analyzed through configuration pollution analysis, call graphs, and program call stack analysis, and configuration interaction code coverage rate is designed to improve configuration interaction defects, especially deep configuration interaction defects, and improve configuration interaction code coverage rate and overall code coverage rate.

[0187] The embodiment of the present application further provides an electronic device, which comprises a processor and a memory.

[0188] The memory stores a program.

[0189] The processor executes the program to perform the fair-aware deep forgery detection as described above; the electronic device has the function of carrying and running the software system of the fair-aware deep forgery detection method provided by the embodiment of the present application, for example, a personal computer, a mini computer, a mainframe, a workstation, a network or a distributed computing environment, a single or integrated computer platform, or communication with a charged particle tool or other imaging device, and the like.

[0190] The embodiment of the present application further provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the fair-aware deep forgery detection as described above.

[0191] In some alternative embodiments, the function / operations recited in the block diagrams can occur in a different order than the order described in the operational illustrations. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality / operations involved. Also, although the embodiments presented in the flow diagrams have been presented in serial fashion, it should be understood that the ordering of steps in the flow diagrams can not be essential. For example, where steps over partially overlapping states are at issue, described steps can sometimes be performed in the reverse order. Moreover, additional or fewer steps can be utilized. The disclosed methods are not limited to the operational and logical flow presented in the diagrams.

[0192] Embodiments of the present application also disclose a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the aforementioned fair-aware deepfake detection.

[0193] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the described functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the actual implementation of the modules, in light of the attributes, functions and internal relationships of the various functional modules disclosed herein, will be apparent to one of ordinary skill in the art, in view of the present disclosure. Accordingly, the disclosed application is not limited to the specific embodiments set forth herein, but is applicable to any apparatus that has the same functionality and achieves the same results. It is to be understood that the specific concepts disclosed herein are merely illustrative and are not intended to limit the scope of the present application.

[0194] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0195] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor or other system that can fetch the instructions from the instruction execution system, apparatus or device and execute the instructions, or in conjunction with these instructions execution systems, apparatus or devices. For the purpose of this specification, the "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in connection with an instruction execution system, apparatus or device, or in conjunction with these instruction execution systems, apparatus or devices.

[0196] More specific examples (non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CD ROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by editing, interpreting or otherwise processing, if necessary, in other suitable ways, to be electronically obtained and then stored in the computer memory.

[0197] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware which is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, can be used: a combination of discrete logic circuits having logic gates for implementing logic functions upon an application of data signals; application specific integrated circuits having logic gates, which are programmed configuration to perform specified logic functions; programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0198] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0199] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made therein without departing from the principles and spirit of the present application.

[0200] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application.

Claims

1. A fairness-aware deepfake detection method, characterized in that, include: Acquire a target image, perform detection on the target image using a deepfake detection model, and obtain a deepfake detection result; The training of the deepfake detection model includes: Obtain the dataset, in which the image samples in the dataset include corresponding image labels, and the image labels include binary labels and demographic group labels; Causal mask learning and causal feature decomposition are performed using two causal inference branches and the dataset to obtain a directed acyclic graph. Stable causal features related to manipulation are extracted from the directed acyclic graph, and non-causal features related to demographic group labels are separated. Obtain the causal feature decomposition results, determine the cross-demographic counterfactual features based on the causal feature decomposition results, and use KL divergence to constrain causal consistency based on the counterfactual features and the original features. Demographic label prediction of non-causal features is performed using a population classifier and gradient inversion layer, and the label prediction is performed using a gradient inversion layer. Based on the counterfactual consistency group classification, a joint training method is adopted, and backpropagation learning is performed through the backbone network, forgery classification loss, causal structure learning loss, counterfactual consistency loss and group classification loss to obtain the deep forgery detection model; The process of performing causal mask learning and causal feature decomposition using two causal inference branches and the dataset to obtain a directed acyclic graph includes: Select batches of data from the dataset and divide the batches of data equally, with each batch including the same number of real samples and fake samples. Learn the causal mask and perform causal feature decomposition on the evenly divided batch data through causal inference branches: in, This is an identifier for evenly distributed batch data. As a causal feature mask, Decomposition of causal characteristics, Decomposition of non-causal characteristics; A fake classifier using causal reasoning branches Optimization is performed by minimizing the classification loss: in Represents the binary cross-entropy loss, where It is the predicted probability of forgery, binary label. , The data is divided into evenly distributed batches. Loss of execution structure through causal reasoning branches deal with: in, For sparsity regularization, Representing the adjacency matrix The vectorized form, and , To reconstruct the loss, Represented as: in, For feature vectors, For demographic group labels, It is a finite set of group categories. It is the total number of people in the group. This represents the total number of nodes in the node set. This represents the L2 reconstruction loss applied to the feature dimension; Acyclic constraints are used We learn to obtain a directed acyclic graph with acyclic constraints. for: Represents the trace of a matrix; The causal mask learning and causal feature decomposition include: Using binary mask To highlight the causal characteristics related to manipulation, the binary mask is represented as: The feature vector is decomposed into causal and non-causal components using a binary mask: During decomposition, the causal and non-causal parts remain unchanged, and the causal structure is aligned.

2. The fairness-aware deepfake detection method according to claim 1, characterized in that, The use of acyclic constraints The directed acyclic graph obtained from learning includes: Based on the causal stable graph, a learnable adjacency matrix is ​​used. Get Node To the node causal effect For nodes The value is refactored, and the refactoring method is as follows: in It is the input vector of the connection. Project causal features onto category logits.

3. The fairness-aware deepfake detection method according to claim 1, characterized in that, The process of obtaining the causal feature decomposition results, determining cross-demographic counterfactual features based on the causal feature decomposition results, and applying KL divergence to constrain causal consistency based on the counterfactual features and the original features includes: Obtain the causal feature decomposition results of the two causal inference branches. and ; The counterfactual features of fixed-span population statistics are constructed based on the results of causal feature decomposition as follows: The consistency loss of KL divergence is used to maintain predictive consistency between the original features and the counterfactual features. The consistency loss of KL divergence is: in, For consistency loss, Let KL divergence be the KL divergence. To forge a classifier.

4. The fairness-aware deepfake detection method according to claim 3, characterized in that, The method of predicting demographic labels for non-causal features using a population classifier and a gradient inversion layer, and predicting labels using a gradient inversion layer, includes: A swarm classifier is trained by minimizing the standard multi-class cross-entropy loss, where the swarm classifier is... The group classification loss is: in, For non-causal features, cross-entropy loss for: Among them, Represents the predicted class probability. It's a real label. It refers to the number of categories, which corresponds to the number of demographic categories; During forward propagation of training, non-causal features are included. Insert into the population classifier Training is performed; during backpropagation of training, gradient inversion is performed through a gradient inversion layer before insertion into the population classifier. During training, the gradient is reversed as follows: An adversarial module focused on fairness is trained by simply minimizing the group classification loss: in Indicates hyperparameters, and This represents the group classification loss for bicausal inference branches.

5. The fairness-aware deepfake detection method according to claim 4, characterized in that, The deep forgery detection model, based on counterfactual consistency group classification, employs a joint training method. It utilizes a backbone network, forgery classification loss, causal structure learning loss, counterfactual consistency loss, and group classification loss for backpropagation learning to obtain the model. This includes: Based on the overall training objective, a deep forgery detection model is obtained by backpropagation learning using a backbone network, forgery classification loss, causal structure learning loss, counterfactual consistency loss, and group classification loss, and by employing a sharpness perception minimization strategy. The overall training objective is: in For the first The spoofing classification loss of each DAG branch; The causal structure learning loss for the k-th branch includes feature reconstruction error and graph structure constraints; The counterfactual consistency loss is the cross-group counterfactual sample in the k-th branch. Group classification loss.

6. A deepfake detection apparatus for fairness perception applying the deepfake detection method as described in any one of claims 1-5, characterized in that, include: The first module is used to acquire the target image, perform detection on the target image using a deepfake detection model, and obtain the deepfake detection result. The training of the deepfake detection model includes: The second module is used to obtain the dataset, in which the image samples in the dataset include corresponding image labels, and the image labels include binary labels and demographic group labels. The third module is used to perform causal mask learning and causal feature decomposition through two causal inference branches and the dataset to obtain a directed acyclic graph. Stable causal features related to manipulation are extracted through the directed acyclic graph, and non-causal features related to demographic group labels are separated. The fourth module is used to obtain the causal feature decomposition results, determine the counterfactual features across demographics based on the causal feature decomposition results, and use KL divergence to constrain causal consistency based on the counterfactual features and the original features. The fifth module is used to predict demographic labels for non-causal features using a population classifier and a gradient inversion layer, and to predict labels using a gradient inversion layer. The sixth module is used to obtain the deep forgery detection model by using a joint training method based on the counterfactual consistency group classification, through backpropagation learning via the backbone network, forgery classification loss, causal structure learning loss, counterfactual consistency loss and group classification loss.

7. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the fair perception deepfake detection method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the fair perception deepfake detection method as described in any one of claims 1-5.

Images