False information detection method and device based on plug-and-play counterfactual debiasing framework

Abstract

This application relates to a method and apparatus for detecting false information based on a plug-and-play counterfactual debiasing framework. The proposed plug-and-play counterfactual framework can be integrated with various traditional detection models as a flexible debiasing module. The plug-and-play counterfactual framework uses counterfactual reasoning to eliminate spurious correlations between claims, evidence, and predicted labels, thereby enhancing the model's causal reasoning capabilities. Counterfactual reasoning eliminates spurious correlations between claims, evidence, and predicted labels, thereby enhancing the detection model's causal reasoning capabilities. This significantly improves the accuracy of various basic detection models for false information detection.

Description

Technical Field

[0001] The present application relates to the technical field of false information detection, and in particular to a false information detection method and device based on a plug-and-play counterfactual debiasing framework. Background Art

[0002] The proliferation of electronic devices has prompted a gradual shift in how people access information from print to digital media. This shift has lowered the barriers to information dissemination, accelerated its spread, and subtly influenced people's perceptions and values. However, due to the inability to effectively and timely verify the information circulating on these platforms, social networks have become a breeding ground for fake news. Fake news is unfounded, fabricated information that often lacks factual support, remains unverified, is riddled with inaccuracies, or is intentionally deceptive. It may include fabricated facts, exaggerated descriptions, and misleading context. Fake news can pose a serious threat to social order and may even cause widespread panic among the public. Therefore, effective measures are urgently needed to curb the spread of fake news and maintain social harmony and stability.

[0003] In recent years, fake news detection has been extensively studied from various perspectives. Most fake news detection methods can be broadly categorized into two categories: pattern-based methods and evidence-aware methods. Pattern-based methods rely solely on the claim itself, without incorporating any external information. To detect fake news, different methods may emphasize different features, such as textual content, multimodal content, and communication structure.

[0004] In contrast, evidence-aware approaches incorporate relevant external information (such as evidence) to assist in fake news detection. They typically determine the authenticity of news by analyzing the correlation between evidence and claims. This approach involves extracting keywords from claims and searching for highly relevant terms in external sources, using these keywords to assist in determining the authenticity of the news. Based on evidence, evidence-aware fake news detection models demonstrate higher reliability and interpretability than other models.

[0005] Unlike straightforward text classification tasks, evidence-aware fake news detection methods require understanding context and identifying information requiring verification. After evidence is collected, causal reasoning is used to verify this information. For example, the input data consists not only of the textual claim but also of the top 30 pieces of relevant evidence. The claim is fed into a deep learning network (such as TextCNN) to extract features, and an attention mechanism is used to identify key information requiring verification. Finally, by concatenating and blending the feature vectors of the evidence and claim, accurate causal correlations are modeled. This feature fusion approach fully considers the interaction between evidence and claim, leading to more accurate judgments of news authenticity. Furthermore, this method emphasizes the interpretability of the detection process. By documenting the evidence retrieval and verification process, as well as the specific method of feature fusion, the model can provide detailed explanations for detection results, thereby improving reliability. This interpretability not only helps users understand the basis for predictions but also provides a reference for subsequent model optimization and improvement.

[0006] Furthermore, a significant challenge in the practical deployment of fake news detection systems is that training and test datasets often have different data distributions, a phenomenon known as the out-of-distribution (OOD) problem. In practice, fake news detection models are typically trained on data from a limited number of platforms, yet are applied to a variety of general-purpose platforms. Furthermore, training datasets are often limited to data on a specific topic or event within a short timeframe, while models are required to generalize to a wider range of contexts.

[0007] Given that the labels and associated evidence in the training dataset are manually collected, evidence-aware fake news detection models are prone to learning spurious correlations between claims and predicted labels. Furthermore, due to the correlation between claims and evidence, spurious correlations may also emerge between the evidence and the predicted label. These spurious correlations pose significant out-of-distribution challenges for fake news detection models. These spurious correlations can be defined as claim bias and evidence bias, respectively. Both biases prevent fake news detection models from effectively performing causal inference between claims and evidence. For example, if most events in a certain field are labeled "fake news" in the training dataset, the model may learn that topics and keywords in that field indicate fake news, rather than performing causal inference. As expected, this correlation does not hold when the data distribution changes.

[0008] To improve the accuracy and generalization of fake news detection models, it is necessary to mitigate the effects of claim bias and evidence bias and enhance the causal reasoning capabilities of detection models. However, the out-of-distribution problem in fake news detection has not been fully explored, especially for evidence-aware tasks. Some methods are evidence-based but simply concatenate claim and evidence features without performing true causal reasoning. Subsequently, various attention mechanisms have been applied to simulate the causal reasoning ability of the human brain. However, these methods primarily focus on identifying common features between different events or topics and fail to consider the specific reasoning paths required for evidence-aware fake news detection. Various debiasing methods have also been proposed to address this issue. ReW reduces the model's reliance on statistical features by downweighting high-frequency n-grams that are highly correlated with the label. However, it also ignores the patterns of biased features and may introduce new data imbalances. PoE utilizes the predictions of a "biased expert model" to suppress the influence of biased features on the main model. However, the independence assumption between the biased model and the main model in PoE is too strong, and dynamic weighting can lead to training instability. CF eliminates the influence of biased features through counterfactual reasoning and explicitly decouples biased features, but the computational complexity of the CF architecture significantly increases the computational cost and training difficulty. Summary of the Invention

[0009] Based on this, it is necessary to provide a false information detection method and device based on a plug-and-play counterfactual debiasing framework to address the above technical problems.

[0010] A method for detecting false information based on a plug-and-play counterfactual debiasing framework, the method comprising:

[0011] Obtain a data set of text information to be tested. Each piece of data information includes: a statement set, a corresponding evidence set, and a real information label.

[0012] A text embedding module is used to convert the statement set and the corresponding evidence set into statement embedding feature vectors and evidence embedding feature vectors.

[0013] According to the statement embedding feature vector and the evidence embedding feature vector, the traditional evidence-aware false information detection model is used to perform prediction and obtain the traditional prediction results.

[0014] The statement-embedded feature vector and the evidence-embedded feature vector are processed using a debiasing model to obtain statement-based prediction results and evidence-based prediction results; the debiasing model adopts a debiasing method of counterfactual reasoning.

[0015] The prediction results based on statements and the prediction results based on evidence are weighted and summed to obtain the prediction results under counterfactual conditions.

[0016] The feature fusion module is used to fuse the traditional prediction results and the prediction results under counterfactual conditions to obtain the false information debiased prediction results.

[0017] A false information detection device based on a plug-and-play counterfactual debiasing framework, the device comprising:

[0018] The unit for acquiring the text information data set to be detected is used to acquire the text information data set to be detected, where each piece of data information includes: a statement set, a corresponding evidence set, and a real information label.

[0019] The feature embedding unit is used to convert the statement set and the corresponding evidence set into a statement embedding feature vector and an evidence embedding feature vector by using a text embedding module.

[0020] The traditional prediction unit is used to perform prediction based on the statement embedding feature vector and the evidence embedding feature vector using a traditional evidence-aware false information detection model to obtain a traditional prediction result.

[0021] The prediction unit under counterfactual conditions is used to process the statement-embedded feature vector and the evidence-embedded feature vector using a debiasing model to obtain a statement-based prediction result and an evidence-based prediction result; the debiasing model uses a debiasing method of counterfactual reasoning; and the statement-based prediction result and the evidence-based prediction result are weighted and summed to obtain a prediction result under counterfactual conditions.

[0022] The false information debiasing prediction result determination unit is used to use the feature fusion module to fuse the traditional prediction results and the prediction results under counterfactual conditions to obtain the false information debiasing prediction results.

[0023] The aforementioned method and apparatus for disinformation detection based on a plug-and-play counterfactual debiasing framework proposes a flexible debiasing module that can be integrated with various traditional detection models. The plug-and-play counterfactual framework employs counterfactual reasoning to eliminate spurious correlations between claims, evidence, and predicted labels, thereby enhancing the model's causal reasoning capabilities. This significantly improves the accuracy of various basic detection models for disinformation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A causal diagram for evidence-aware false information detection in one embodiment;

[0025] Figure 2 1 is a flow chart of a false information detection method based on a plug-and-play counterfactual debiasing framework in one embodiment;

[0026] Figure 3An overview of a DP2CF debiasing method according to another embodiment;

[0027] Figure 4 is a flowchart of an evidence-based false information detection model in another embodiment;

[0028] Figure 5 This is a general flow of the DP2CF model in another embodiment;

[0029] Figure 6 The sensitivity analysis comparison diagram of DP2CF in another embodiment, wherein (a) is a BERT sensitivity analysis comparison diagram tested on Snopes, (b) is a BERT sensitivity analysis comparison diagram tested on Politifact, (c) is a DeClare sensitivity analysis comparison diagram tested on Snopes, (d) is a DeClare sensitivity analysis comparison diagram tested on Politifact, (e) is a MAC sensitivity analysis comparison diagram tested on Snopes, and (f) is a MAC sensitivity analysis comparison diagram tested on Politifact. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0031] A false information detection method based on the plug-and-play counterfactual debiasing framework, referred to as the DP2CF debiasing method.

[0032] This application uses Represents a dataset to be tested with a length of k, where each news By the corresponding statement , and the corresponding evidence set , and real news labels For each piece of evidence from the set Evidence , they are similar to statements, consisting of words . The components of Represents a single word. Considering that fake news detection is a binary classification problem, its true label , where 0 and 1 represent fake news and true news respectively. The goal of fake news is to develop a classification model to effectively calculate the test news The predicted label , based on the input declaration and evidence set .

[0033] The input dataset of news to be verified will be obtained directly before the fake news detection task. From the perspective of causal inference, the authenticity of news should be verified through causal reasoning between claims and evidence. After the interaction between evidence and claims, a latent embedding vector is generated, which is recorded as the interaction feature. In order to eliminate specific biases between claims, evidence, and predicted labels, causal graphs are used to study the relationships between them. Based on the insights gained from causal graphs, appropriate debiasing strategies can be designed. The causal graph of the evidence-aware fake news detection method is shown in Figure 2. Figure 1 As shown, Figure 1 The “×” means truncation.

[0034] In addition, each news By statement and evidence set The researchers intend to use the interaction characteristics Make predictions However, in the causal graph of traditional methods, there are false correlations that interfere with causal inference. Since the original dataset is collected manually, Specific keywords or fields may be related to the predicted labels There is a false correlation, denoted as .because and There is a strong correlation between Possibly related to Share the same keywords or specific fields, recorded as .therefore, and There may also be false correlations between During the training phase, when using traditional fake news detection methods, the model may learn from false correlations, resulting in Therefore, it is necessary to use counterfactual reasoning to alleviate and impact.

[0035] In one embodiment, Figure 2 As shown, a false information detection method based on a plug-and-play counterfactual debiasing framework is provided, which includes the following steps:

[0036] Step 100: Obtain a text information dataset to be tested, where each piece of data information includes: a statement set, a corresponding evidence set, and a real information label.

[0037] Specifically, in the field of natural language processing (NLP), word embedding technology plays a key role in capturing the semantic relationships between words. A core challenge in NLP lies in efficiently representing word semantics. In existing fake news detection methods, GloVe and BERT models are primarily used as embedding modules.

[0038] Step 102: Use a text embedding module to convert the statement set and the corresponding evidence set into a statement embedding feature vector and an evidence embedding feature vector.

[0039] Specifically, the text embedding module converts claims and evidence in the original text data into embedded feature vectors.

[0040] Step 104: A traditional evidence-aware false information detection model is used to perform prediction based on the statement embedding feature vector and the evidence embedding feature vector to obtain a traditional prediction result.

[0041] Step 106: A debiasing model is used to process the statement-embedded feature vector and the evidence-embedded feature vector to obtain a statement-based prediction result and an evidence-based prediction result; the debiasing model uses a debiasing method of counterfactual reasoning.

[0042] Specifically, due to data bias in the original dataset, traditional detection models may learn spurious correlations between specific fields or keywords and predicted labels. To mitigate this bias, counterfactual reasoning is employed as a debiasing method. Counterfactual reasoning is the process of inferring potential outcomes assuming certain variables are invalid, thereby assessing the contribution of the remaining variables to the predicted outcome. In the field of fake news detection, counterfactual reasoning requires that detection models make predictions based solely on claims or evidence.

[0043] Step 108: Perform a weighted summation of the prediction results based on the statement and the prediction results based on the evidence to obtain the prediction results under the counterfactual condition.

[0044] Step 110: Use a feature fusion module to fuse the traditional prediction results and the prediction results under counterfactual conditions to obtain a false information debiased prediction result.

[0045] Specifically, the feature fusion module fuses different feature vectors to obtain the final debiased prediction result.

[0046] The feature fusion module aims to enhance accuracy, stability, and generalization by integrating multidimensional data features. This module leverages fuzzy logic principles to map each output feature vector to a fuzzy membership space. This approach thus more effectively captures the intrinsic characteristics of the underlying patterns while mitigating the effects of noise.

[0047] This application proposes a novel counterfactual debiasing method for disinformation detection based on a plug-and-play counterfactual debiasing framework, namely the "Plug-and-Play Counterfactual Framework" (DP2CF). Although previous studies have applied CF to fake news detection, some of these studies assume that the average features of the evidence are essential when computing counterfactual predictions, rather than relying solely on claims without external sources. In contrast, DP2CF, as a flexible plug-and-play module, can be easily attached to various traditional evidence-aware fake news detection models and avoids these strong assumptions. Specifically: 1) For the embedding representation of claims or evidence, a multi-scale convolutional feature extraction module is used for feature extraction. A one-dimensional convolutional layer is used to convolve the input embedding sequence, and the convolved feature maps are then pooled into fixed-length feature vectors through an adaptive max pooling layer. Finally, the feature vectors of different sizes are concatenated to obtain a comprehensive feature representation. 2) To enhance the accuracy and adaptability of the detection model when testing on out-of-distribution datasets, a gradient reversal layer is introduced for feature processing. 3) To eliminate spurious correlations, hyperparameters α and β are introduced to adjust the computational process. Since counterfactual reasoning requires subtracting the traditional detection model’s logical value from the counterfactual logical value, hyperparameters may help balance this process, ensuring that the detection model retains important semantic information while removing spurious correlations.

[0048] Unlike existing methods that may be limited by strong assumptions or high computational complexity, the DP2CF framework in this application is a flexible plug-and-play module that can be easily integrated into various traditional detection models, providing a more efficient and effective debiasing solution, mitigating the impact of claim bias and evidence bias in evidence-aware fake news detection.

[0049] The overall structure of the DP2CF debiasing method is as follows: Figure 3 As shown in Figure 1, the framework consists of three modules. The first module is the text embedding module, which converts the claims and evidence in the raw text data into embedded feature vectors. The second module is the feature processing module, which includes a traditional detection model and a debiasing model for the claims and evidence. In addition to calculating traditional prediction results, this process can also generate counterfactual predictions. The final module is the feature fusion module, in which different feature vectors are fused to obtain the final debiased prediction result.

[0050] The aforementioned method and apparatus for detecting disinformation based on a plug-and-play counterfactual debiasing framework include a proposed plug-and-play counterfactual framework that can be integrated with various traditional detection models as a flexible debiasing module. The plug-and-play counterfactual framework employs counterfactual reasoning to eliminate spurious correlations between claims, evidence, and predicted labels, thereby enhancing the model's causal reasoning capabilities. This framework significantly improves the accuracy of various basic detection models for detecting disinformation.

[0051] In one embodiment, if the BERT model is selected as the basis of the traditional evidence-aware false information detection model, step 102 selects the BERT model as the text embedding module.

[0052] Specifically, BERT is a novel language representation pre-training method that achieves outstanding performance across a wide range of natural language processing tasks. BERT's key contributions lie in its bidirectional contextual representation and its pre-training and fine-tuning paradigm. BERT is based on the Transformer architecture, specifically leveraging the Transformer's encoder. BERT's encoder structure enables it to capture bidirectional context, meaning that each token in the input sequence can attend to all other tokens during the encoding process, both before and after it.

[0053] BERT’s input consists of three types of embeddings: (1) Token Embeddings: Each token in the input sequence is mapped to a vector in a high-dimensional space using the WordPiece method. (2) Segment Embeddings: These embeddings are used to distinguish different sentences in the input. For example, in a pair of sentences A and B, the token in sentence A is assigned a segment ID of 0, while the token in sentence B is assigned a segment ID of 1. (3) Position Embeddings: These embeddings provide information about the position of each token in the sequence, and BERT’s position embeddings are learnable. The final input representation is obtained by adding these three embeddings for each token.

[0054] BERT's basic architecture relies on a bidirectional Transformer encoder. Input text is processed sequentially through a series of sublayers, including multi-head self-attention layers and fully connected feed-forward layers. Each sublayer incorporates normalization and residual connections, and the output of each sublayer is calculated according to the following formula.

[0055] ;

[0056] Here, sublayer represents the specific operations performed by each sublayer. In the self-attention module, the query matrix (q), key matrix (k), and value matrix (v) are used to assess the similarity between words and assign weights accordingly. This process helps aggregate relevant information between words, enabling the model to capture dependencies between different positions. The calculation formula for this attention layer is as follows:

[0057] ;

[0058] in Represents a query and key The dimension of the matrix. To improve the performance of the model, a multi-head attention mechanism is often implemented. In this method, the input is linearly transformed into q, k, and v matrices. After generating multiple attention matrices, they are horizontally concatenated and then multiplied by a weight matrix To generate a unified matrix. The specific calculation process is as follows:

[0059] ;

[0060] ;

[0061] Among them, the weight vector 、 and The query q, key k, and value v matrices of the i-th attention head are corresponded respectively. Operation Cocat refers to the horizontal concatenation of the outputs of multiple attention heads, and Represents the weight matrix applied in the concatenation step. After the multi-head attention layer, its output is passed to a fully connected feed-forward network, which processes the data through several activation functions to generate the final output, as shown below:

[0062] ;

[0063] Among them, RELU represents the Rectified Linear Unit activation function, and and denote the weight and bias parameters of the multi-head attention layer, and This corresponds to the weights and biases of the fully connected feedforward network. In this application, the hidden state vector extracted from the last layer of the BERT model is used as the text embedding vector. In applications, if the BERT model is chosen as the basis for a traditional detection model, this method will serve as the text embedding module.

[0064] In one embodiment, when the traditional evidence-aware false information detection model is MAC or DeClare, step 102 selects the GloVe model as the text embedding module.

[0065] Specifically, GloVe (Global Word Vectors) is a powerful word embedding method that combines the advantages of count-based and prediction-based methods. By leveraging global co-occurrence statistics, the word vectors generated by GloVe are able to preserve meaningful linear substructure in the vector space. The core idea of ​​GloVe is to capture the semantic relationship between words through a co-occurrence matrix, which records the co-occurrence frequency of words in a large corpus. The objective function of GloVe is expressed as:

[0066] ;

[0067] in is the size of the vocabulary, Expressive words and words co-occurrence frequency. and The words and words The word vector of and is a word and words The additional bias term. is a weight function that adjusts the effect of co-occurrence frequency on the loss function and is defined as:

[0068] ;

[0069] in Is a threshold used to prevent excessive co-occurrence frequencies from adversely affecting the model during the training phase. is a hyperparameter. The model achieves optimal performance when set to 0.75. GloVe's key strengths lie in its ability to capture global context, scale to large vocabularies, and preserve linear structure in the vector space. These properties make GloVe an ideal tool for natural language processing tasks, such as detecting fake news. If DP2CF (assuming it's a detection framework or method) chooses MAC or Declare as its traditional detection model, the GloVe method will function as a text embedding module.

[0070] In one embodiment, the debiasing model includes: a statement debiasing model and an evidence debiasing model; the statement debiasing model is used to extract the relationship between the statement embedding feature vector and the prediction label to obtain a prediction result based on the statement; the evidence debiasing model is used to extract the relationship between the evidence embedding feature vector and the prediction label to obtain a prediction result based on the evidence; step 106 includes:

[0071] The statement embedding feature vector is input into the statement debiasing model to obtain the statement-based prediction result; the evidence embedding feature vector is input into the evidence debiasing model to obtain the evidence-based prediction result.

[0072] In one embodiment, the statement debiasing model includes: a TextCNN module, a maximum pooling layer, a gradient reversal layer and a full feedforward network layer; in the statement debiasing model: the statement embedded feature vector is feature extracted using the TextCNN module to obtain a convolution vector; the convolution vector is activated by the ReLU function and then processed by the maximum pooling layer to obtain |K| first intermediate feature vectors; K is the kernel set used for convolution; all intermediate feature vectors are spliced ​​through the Cocat operation to obtain a spliced ​​vector; the spliced ​​vector is processed by the gradient reversal layer and then input into the full feedforward network layer to obtain a statement-based prediction result.

[0073] In one embodiment, the evidence debiasing model has the same structure as the claim debiasing model; in the evidence debiasing model: the TextCNN module is used to extract features from the evidence embedded feature vector to obtain a first convolution vector; the first convolution vector is activated by the ReLU function and then processed by the maximum pooling layer to obtain |K| first intermediate feature vectors; where K is the kernel set used for convolution; all the first intermediate feature vectors are spliced ​​through the Cocat operation to obtain a first spliced ​​vector; the first spliced ​​vector is processed by the gradient reversal layer and then input into the full feedforward network layer to obtain an evidence-based prediction result.

[0074] In one embodiment, a feature fusion module is used to fuse the traditional prediction results and the prediction results under counterfactual conditions to obtain a false information debiased prediction result. The false information debiased prediction result is expressed as:

[0075] ;

[0076] in, represents the false information debiased prediction result, represents the prediction result of the traditional detection model, Represents the predicted results under counterfactual conditions.

[0077] Specifically, for the evidence-based false information detection method, the general flow chart is as follows: Figure 4shown.

[0078] (1) Traditional detection model:

[0079] Given a statement and its corresponding evidence dataset , the detection model first encodes the text content into word-level embedding vectors. The calculation process is as follows:

[0080] ;

[0081] ;

[0082] in, and are the word-level embedding matrices for claims and evidence, respectively. In addition, represents the dimension of word-level embedding, and represents the learnable parameters of the embedding module. Subsequently, the detection model performs word-level interactions between claims and evidence to generate sentence-level embeddings. Sentence-level embeddings are usually obtained through various pooling operations, as shown below:

[0083] ;

[0084] ;

[0085] in, and .and represents the dimension of sentence-level embedding. The learnable parameters of the word-level interaction module are expressed as Finally, the detection model conducts sentence-level interaction between claims and evidence to derive the predicted output The specific formula of this process can be expressed as follows:

[0086] ;

[0087] If the input data contains publisher embedding vectors and speaker embedding vectors To represent the credibility of the statement and evidence respectively, the above formula can be rewritten as:

[0088] ;

[0089] (2) Debiasing model

[0090] Due to data bias in the original dataset, traditional detection models may learn false correlations between specific fields or keywords and predicted labels. To mitigate this bias, counterfactual reasoning is used as a debiasing method. Counterfactual reasoning refers to the process of inferring potential outcomes under the assumption that some variables are invalid, thereby evaluating the contribution of the remaining variables to the predicted outcome. In the field of fake news detection, counterfactual reasoning requires the detection model to make predictions based only on claims or evidence. Specifically, by eliminating intermediate variables through causal intervention, the model can effectively detect the false positives and negative positives. o In this case, the detection model must rely on spurious correlations. In this method, the prediction under this counterfactual condition is defined as:

[0091] ;

[0092] in, is the prediction result under the counterfactual condition, and denote predictions based solely on claims and evidence, respectively. and are two hyperparameters introduced to avoid underestimation or overestimation. Therefore, the DP2CF model is introduced to calculate and .

[0093] like Figure 5 As shown in Figure 2, the main purpose of the DP2CF debiasing method is to extract the relationship between the statement or evidence and the predicted label. Considering the efficiency and generalization ability of the DP2CF debiasing method, TextCNN is used to extract the main features. Before entering the DP2CF model, each sentence is processed in the text embedding module to form a dimension of The embedding vector of represents the length of each sentence, and D represents the dimension of embedding. After the convolution operation, an embedding of dimension The convolution vector of represents the number of filters, and the kernel is the core set In addition, each convolution vector corresponds to a kernel Subsequently, the convolution vector passes through an activation function (ReLU) and a max pooling layer to identify the most important features, forming K intermediate feature vectors, which are then merged into a concatenated vector through the Cocat operation.

[0094] To more effectively mitigate the impact of bias on causal prediction output in a cross-platform setting, a gradient reversal layer (GRL) is employed. GRL works by reversing the gradient flow, enabling the model to learn both information related to the target variable and features unrelated to bias during training. This dual learning process enhances the model's generalization across different platforms and improves the accuracy of causal inference. This is followed by a fully feedforward network (FFN) layer. The FFN layer is the final stage of this application's model architecture. It takes the learned feature vector as input and performs a series of nonlinear transformations to generate the final prediction.

[0095] In one embodiment, the feature fusion module maps each output feature vector to a fuzzy membership space using fuzzy logic principles; the membership function expression is:

[0096] ;

[0097] in, is the membership function, m is the parameter that determines the center position of the curve, is the parameter that determines the degree of diffusion of the curve. x is the independent variable of the membership function.

[0098] The instance The fuzzy membership values ​​of all features of the instance are averaged to obtain For categories Membership value of For categories The membership value expression is:

[0099] ;

[0100] in, For example For categories The membership value of represents the dimension of the feature vector, It is i The first instance k Features, and corresponds to the feature Belong to category The two parameters of the membership function.

[0101] Specifically, the feature fusion module aims to enhance accuracy, stability, and generalization ability by integrating multidimensional data features. This module uses fuzzy logic principles to map each output feature vector to a fuzzy membership space. Therefore, this approach more effectively captures the intrinsic characteristics of the underlying pattern while mitigating the effects of noise. In the feature fusion module, the Gaussian membership function is given by and m, which determine the shape of the function. Each membership function is shown in the above membership function expression.

[0102] To calculate the instance For categories The membership value of , it is necessary to average the fuzzy membership values ​​of all features. The calculation method is as in the above example For categories The membership value expression is shown in Figure 2.

[0103] Therefore, based on the above description, the debiased prediction can be calculated as follows:

[0104] ;

[0105] in, represents the false information debiased prediction result, represents the prediction result of the traditional detection model, Represents the predicted results under counterfactual conditions.

[0106] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0107] In a validation example, experimental analysis was performed using the Snopes and PolitiFact datasets collected by Hansen. During preprocessing, the Snopes and PolitiFact datasets are characterized by binary labels ("True" or "False"), making them naturally suitable for binary classification tasks. The ratio of true news to false news in the Snopes dataset is approximately 1:3. In the PolitiFact dataset, the ratio of true news to false news is roughly 1:1. Furthermore, these datasets are divided into training, validation, and test sets. Detailed statistics for these datasets are shown in Table 1.

[0108] Table 1 Detailed data of Snopes and Politifact datasets

[0109]

[0110] (1) Baseline model

[0111] To more effectively evaluate the performance of the DP2CF method, DP2CF and CF debiasing methods are implemented on the following state-of-the-art evidence-aware fake news detection methods.

[0112] DeClare evaluates the credibility of claims via neural networks, using external evidence and avoiding artificial features.

[0113] MAC is able to understand explanations at both the word level and the evidence level due to its hierarchical multi-head attention mechanism.

[0114] BERT captures language patterns and relationships by performing unsupervised learning on large-scale text data. BERT's advantage lies in its bidirectional Transformer architecture, which can fully consider contextual information to generate accurate word embeddings.

[0115] Following previous research, experiments are conducted on two different datasets (PolitiFact and Snopes). When the training and test datasets are the same, the performance of each baseline model on the PolitiFact and Snopes datasets is shown in Table 2.

[0116] Table 2. Results of the baseline model under the same training and test set conditions.

[0117]

[0118] In a cross-platform setting, each baseline model will be applied to the PolitiFact and Snopes datasets with different debiasing strategies.

[0119] (2) Experimental setup

[0120] AUC, F1-Macro, and F1-Micro were selected as evaluation metrics. Following Hansen's cross-platform setup, each model was trained and validated on one dataset and tested on another. Because F1-Macro effectively handles imbalanced class distributions and considers both precision and recall, it achieves a balance among the key metrics for fake news detection. During training, if F1-Macro does not improve within 10 consecutive epochs, training for each model is terminated.

[0121] (3) Implementation details

[0122] The maximum lengths of claims and evidence are set to 30 and 100, respectively. Adam is used as the optimizer. Each claim is paired with a maximum of 30 pieces of evidence from the original dataset. The final output of each evaluation metric is aggregated based on five different random seeds. This paper uses hyperparameters α and β with values ​​ranging from {0.001, 0.005, 0.01, 0.05, 0.1, 0.5}. All experiments described in this paper were conducted using Python 3.9.19 and PyTorch 2.0.0, using an NVIDIA RTX 2080 Ti GPU.

[0123] (4) Experimental results and analysis

[0124] 1) Overall performance

[0125] In Tables 2 and 3, the performance of three baseline models is compared on two datasets (PolitiFact and Snopes), which have significantly different data distributions. Specifically, Table 2 shows the AUC, F1-Macro, and F1-Micro scores of the baseline models when the training and test datasets are the same. Meanwhile, Table 3 shows the performance metrics of the baseline models in a cross-platform setting. Each baseline model is evaluated using a different debiasing method. Among them, None indicates that no debiasing method is applied in the corresponding experiment, CF indicates the counterfactual reasoning debiasing method, and DP2CF indicates the new debiasing method proposed in this paper. From Tables 2 and 3, the following observations can be made.

[0126] First, the results in Tables 2 and 3 show that, without using debiasing methods, the evaluation metrics of the baseline models tend to decline when the test dataset differs from the training dataset. In particular, when the training dataset is Snopes, BERT's F1-Macro score drops significantly from 61.34% to 45.34%, DeClare from 69.87% to 49.05%, and MAC from 78.66% to 51.75%. Furthermore, other evaluation metrics, including AUC and F1-Micro, also show a clear downward trend. If the training dataset is switched to PolitiFact, although performance still declines in AUC and F1-Macro, this downward trend is no longer evident in F1-Micro.

[0127] Second, as shown in Table 3, DP2CF achieves significant improvements on all evaluation metrics for each baseline model, compared to the marginal improvements brought by the CF debiasing method. This demonstrates that DP2CF is more effective in mitigating bias in news and evidence content. Specifically, BERT improves AUC by 8% to 14%, F1-Macro by 10% to 11%, and F1-Micro by 5% to 10%, compared to the case without the debiasing method. For DeClare, AUC improves by 2% to 7%, F1-Macro by 3% to 6%, and F1-Micro by 3% to 10%. In contrast, MAC improves F1-Macro by 1% to 2% and F1-Micro by 1% to 5%. These results demonstrate that the DP2CF debiasing method can provide more significant performance improvements to baseline models with initially poor performance, compared to baseline models with initially good performance.

[0128] Based on the above results, the following conclusions can be drawn. First, the degradation in baseline model performance when the training and test datasets differ highlights the importance of addressing data bias. This suggests that a model trained on one dataset may not generalize well to another, especially when there are significant differences in data distribution between them. Second, the significant improvement (outperforming CF) achieved by the DP2CF debiasing method over all baseline models and evaluation metrics demonstrates its effectiveness in mitigating bias in claim and evidence content. Finally, among the three baseline models, DP2CF achieved the most significant improvement over BERT. This is because BERT's text embedding module, unlike the other two baseline models, is the most capable at capturing semantic information, which may also make it more susceptible to spurious correlations between claims, evidence, and predicted labels. Therefore, the DP2CF debiasing method achieves the most significant improvement on the BERT model.

[0129] 2) Ablation experiment

[0130] To further illustrate the effectiveness of the DP2CF debiasing method in mitigating claim bias and evidence bias, an ablation study was conducted on three baseline models in a cross-platform setting. The results are shown in Table 4. Figure 4As shown in Table 4, the DP2CF debiasing method is applied to two independent parts. Therefore, the debiasing strategies listed in Table 4 are designed as follows: (1) None: No DP2CF debiasing method is applied. (2) Evidence only: The DP2CF debiasing method is limited to the evidence part, excluding the embedded claim feature vectors and focusing on the features related to the evidence; (3) Claim only: The embedded evidence feature vectors are removed from the DP2CF model, thereby highlighting the features related to the claim while ignoring the evidence features; (4) DP2CF debiasing method: The feature vectors of both the claim and the evidence are included in the DP2CF model. Based on the results, the following key insights can be drawn.

[0131] First, all other strategies that include DP2CF debiasing show consistent improvements on all evaluation metrics compared to the “None” debiasing method, which does not use the DP2CF debiasing method. However, there are a few exceptions. When the “evidence-only” debiasing method is applied to DeClare using the Snopes training dataset, a slight decrease in the AUC metric is observed. Similarly, when the “evidence-only” debiasing method is applied to the MAC baseline model using the Snopes training dataset, a small decrease in the F1-Macro metric is observed. This may be due to the fact that the evidence bias is not obvious in the Snopes dataset compared to the PolitiFact dataset. Therefore, the debiasing method applied to the evidence part may have unexpectedly affected the evidence-aware causal reasoning method.

[0132] Secondly, compared to "evidence-only" and "claims-only" debiasing methods, the DP2CF debiasing method achieved significant improvements across all evaluation metrics. Specifically, focusing on the F1-Macro metric, BERT achieved a 2% to 4% improvement, DeClare achieved a 1% to 3% improvement, and MAC achieved a 0.6% to 3% enhancement. This result strongly demonstrates that when the DP2CF debiasing method effectively mitigates both claim bias and evidence bias, fake news detection models can achieve better performance. Both the claim and evidence components are beneficial to the DP2CF debiasing method.

[0133] Table 3 Comparison results under cross-platform settings

[0134]

[0135] Table 4 Ablation experiments under cross-platform settings

[0136]

[0137] 3) Hyperparameter sensitivity analysis

[0138] In this example, the sensitivity of DP2CF to the hyperparameters α and β on two datasets is discussed in detail. Figure 6 As shown, and The values ​​of come from the set {0.001, 0.005, 0.01, 0.05, 0.1, 0.5}. The vertical axis represents the F1-Macro score, and each point on the surface corresponds to a specific set of and The value represents a discrete set of combinations rather than a continuous surface. Specifically, in a cross-platform setting, BERT's best performance on Snopes appears , while the best performance in the PolitiFact test occurs when DeClare's best performance on Snopes' test came when , while the best performance in the PolitiFact test occurs when 、 The best performance of the Mac in Snopes testing was , while the best performance in the PolitiFact test occurs when These seemingly different results may be due to the differences in dataset distribution and the variations between baseline models.

[0139] Furthermore, experiments were conducted in two settings, applying the DP2CF debiasing method to three evidence-aware fake news detection models. The DP2CF debiasing method significantly outperformed both the original detection model and the CF debiasing method. This result highlights the potential of the DP2CF debiasing method as a powerful tool for developing more robust and debiased models to combat the spread of fake news.

[0140] In one embodiment, a false information detection device based on a plug-and-play counterfactual debiasing framework is provided, comprising: a text information dataset acquisition unit to be detected, a feature embedding unit, a traditional prediction unit, a prediction unit under counterfactual conditions, and a false information debiasing prediction result determination unit, wherein:

[0141] The unit for acquiring the text information data set to be detected is used to acquire the text information data set to be detected, where each piece of data information includes: a statement set, a corresponding evidence set, and a real information label.

[0142] The feature embedding unit is used to convert the statement set and the corresponding evidence set into a statement embedding feature vector and an evidence embedding feature vector by using a text embedding module.

[0143] The traditional prediction unit is used to perform prediction based on the statement embedding feature vector and the evidence embedding feature vector using a traditional evidence-aware false information detection model to obtain a traditional prediction result.

[0144] The prediction unit under counterfactual conditions is used to process the statement-embedded feature vector and the evidence-embedded feature vector using a debiasing model to obtain a statement-based prediction result and an evidence-based prediction result; the debiasing model uses a debiasing method of counterfactual reasoning; and the statement-based prediction result and the evidence-based prediction result are weighted and summed to obtain a prediction result under counterfactual conditions.

[0145] The false information debiasing prediction result determination unit is used to use the feature fusion module to fuse the traditional prediction results and the prediction results under counterfactual conditions to obtain the false information debiasing prediction results.

[0146] In one embodiment, if the BERT model is selected as the basis of the traditional evidence-aware false information detection model, the BERT model is selected as the text embedding module in the feature embedding unit.

[0147] In one embodiment, when the traditional evidence-aware false information detection model is MAC or DeClare, the GloVe model is selected as the text embedding module in the feature embedding unit.

[0148] In one embodiment, the debiasing model includes: a statement debiasing model and an evidence debiasing model; the statement debiasing model is used to extract the relationship between the statement embedding feature vector and the prediction label to obtain a prediction result based on the statement; the evidence debiasing model is used to extract the relationship between the evidence embedding feature vector and the prediction label to obtain a prediction result based on the evidence; the prediction unit under counterfactual conditions is also used to input the statement embedding feature vector into the statement debiasing model to obtain a prediction result based on the statement; the evidence embedding feature vector is input into the evidence debiasing model to obtain a prediction result based on the evidence.

[0149] In one embodiment, the statement debiasing model includes: a TextCNN module, a maximum pooling layer, a gradient reversal layer and a full feedforward network layer; the prediction unit under counterfactual conditions is also used in the statement debiasing model: using the TextCNN module to extract features from the statement embedded feature vector to obtain a convolution vector; activating the convolution vector through the ReLU function and then processing it through the maximum pooling layer to obtain |K| first intermediate feature vectors; where K is the kernel set used for convolution; splicing all the intermediate feature vectors through the Cocat operation to obtain a spliced ​​vector; processing the spliced ​​vector through the gradient reversal layer and then inputting it into the full feedforward network layer to obtain a statement-based prediction result.

[0150] In one embodiment, the evidence debiasing model has the same structure as the statement debiasing model; the prediction unit under counterfactual conditions is also used in the evidence debiasing model: the evidence embedded feature vector is subjected to feature extraction using the TextCNN module to obtain a first convolution vector; the first convolution vector is activated by the ReLU function and then processed by the maximum pooling layer to obtain |K| first intermediate feature vectors; K is the kernel set used for convolution; all the first intermediate feature vectors are spliced ​​through the Cocat operation to obtain a first spliced ​​vector; the first spliced ​​vector is processed by the gradient reversal layer and then input into the full feedforward network layer to obtain an evidence-based prediction result.

[0151] In one of the embodiments, the false information debiasing prediction result determination unit is also used to use a feature fusion module to fuse the traditional prediction results and the prediction results under counterfactual conditions to obtain the false information debiasing prediction result as described in the above-mentioned false information debiasing prediction result expression.

[0152] In one embodiment, the feature fusion module maps each output feature vector to a fuzzy membership space using fuzzy logic principles; the membership function is shown in the above membership function expression.

[0153] The instance The fuzzy membership values ​​of all features of the instance are averaged to obtain For categories The membership value of i For categories j The membership value expression is shown in Figure 2.

[0154] Regarding the specific limitations of the false information detection device based on the plug-and-play counterfactual debiasing framework, please refer to the limitations of the false information detection method based on the plug-and-play counterfactual debiasing framework above, which will not be repeated here. The various modules in the above-mentioned false information detection device based on the plug-and-play counterfactual debiasing framework can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0155] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the scope of the present application, and such modifications and improvements are all within the scope of protection of the present application.

Claims

1. A false information detection method based on a plug-and-play counterfactual debiasing framework, characterized in that: The method comprises: Obtain a dataset of text information to be tested. Each piece of data includes: a statement set, a corresponding evidence set, and a true information label. Using a text embedding module to convert the statement set and the corresponding evidence set into a statement embedding feature vector and an evidence embedding feature vector; Performing prediction using a traditional evidence-aware false information detection model based on the statement-embedded feature vector and the evidence-embedded feature vector to obtain a traditional prediction result; A debiasing model is used to process the statement-embedded feature vector and the evidence-embedded feature vector to obtain a statement-based prediction result and an evidence-based prediction result; the debiasing model adopts a debiasing method of counterfactual reasoning; Performing a weighted summation on the prediction result based on the statement and the prediction result based on the evidence to obtain a prediction result under the counterfactual condition; The feature fusion module is used to fuse the traditional prediction results and the prediction results under the counterfactual conditions, and the false information debiased prediction results are obtained as follows: ； in, represents the false information debiased prediction result, represents the prediction result of the traditional detection model, Represents the predicted results under counterfactual conditions.

2. The false information detection method based on the plug-and-play counterfactual debiasing framework according to claim 1 is characterized in that If the BERT model is chosen as the basis for the traditional evidence-aware disinformation detection model, then the BERT model is chosen as the text embedding module.

3. The false information detection method based on the plug-and-play counterfactual debiasing framework according to claim 1 is characterized in that When the traditional evidence-aware false information detection model is MAC or DeClare, the GloVe model is selected as the text embedding module.

4. The false information detection method based on the plug-and-play counterfactual debiasing framework according to claim 1 is characterized in that The debiasing model includes: a statement debiasing model and an evidence debiasing model; the statement debiasing model is used to extract the relationship between the statement embedding feature vector and the prediction label to obtain a prediction result based on the statement; the evidence debiasing model is used to extract the relationship between the evidence embedding feature vector and the prediction label to obtain a prediction result based on the evidence; The statement-embedded feature vector and the evidence-embedded feature vector are processed using a debiasing model to obtain a statement-based prediction result and an evidence-based prediction result, including: Inputting the statement embedding feature vector into the statement debiasing model to obtain a statement-based prediction result; The evidence-embedded feature vector is input into the evidence debiasing model to obtain an evidence-based prediction result.

5. The false information detection method based on the plug-and-play counterfactual debiasing framework according to claim 4 is characterized in that The declarative debiasing model includes: a TextCNN module, a maximum pooling layer, a gradient reversal layer, and a full feedforward network layer; In declaring a debiased model, The TextCNN module is used to extract features from the statement embedding feature vector to obtain the convolution vector; The convolution vector is activated by the ReLU function and then processed by the maximum pooling layer to obtain |K| intermediate feature vectors; where K is the kernel set used for convolution; All the intermediate feature vectors are concatenated by a Cocat operation to obtain a concatenated vector; The concatenated vector is processed by the gradient reversal layer and then input into the full feedforward network layer to obtain the prediction result based on the statement.

6. The false information detection method based on the plug-and-play counterfactual debiasing framework according to claim 4 is characterized in that The evidence debiasing model has the same structure as the claim debiasing model; In the evidence debiasing model: The TextCNN module is used to extract features from the evidence embedding feature vector to obtain the first convolution vector; Activating the first convolution vector with a ReLU function and then processing it with a maximum pooling layer to obtain |K| first intermediate feature vectors; where K is the kernel set used for convolution; Concatenate all the first intermediate eigenvectors through a Cocat operation to obtain a first concatenated vector; The first concatenated vector is processed by the gradient reversal layer and then input into the full feedforward network layer to obtain the evidence-based prediction result.

7. The false information detection method based on the plug-and-play counterfactual debiasing framework according to claim 1 is characterized in that The feature fusion module maps each output feature vector to a fuzzy membership space using fuzzy logic principles; the membership function is: ； in, is the membership function, m is the parameter that determines the center position of the curve, is the parameter that determines the degree of diffusion of the curve. x is the independent variable of the membership function; The instance The fuzzy membership values ​​of all features are averaged to obtain the instance For categories The membership value of is: ； in, For example For categories The membership value of represents the dimension of the feature vector, It is The first instance Features, and corresponds to the feature Belong to category The two parameters of the membership function.

8. A false information detection device based on a plug-and-play counterfactual debiasing framework, characterized in that: The device comprises: A unit for acquiring a text information data set to be tested, configured to acquire a text information data set to be tested, wherein each piece of data information includes: a statement set, a corresponding evidence set, and a true information label; a feature embedding unit, configured to convert the claim set and the corresponding evidence set into a claim embedding feature vector and an evidence embedding feature vector using a text embedding module; a traditional prediction unit, configured to perform prediction using a traditional evidence-aware false information detection model based on the statement-embedded feature vector and the evidence-embedded feature vector to obtain a traditional prediction result; a counterfactual prediction unit, configured to process the statement-embedded feature vector and the evidence-embedded feature vector using a debiasing model to obtain a statement-based prediction result and an evidence-based prediction result; the debiasing model employing a debiasing method of counterfactual reasoning; and performing a weighted summation of the statement-based prediction result and the evidence-based prediction result to obtain a counterfactual prediction result; The false information debiasing prediction result determination unit is used to fuse the traditional prediction result and the prediction result under the counterfactual condition using a feature fusion module to obtain a false information debiasing prediction result as follows: ； in, represents the false information debiased prediction result, represents the prediction result of the traditional detection model, Represents the predicted results under counterfactual conditions.

Images