Ffact verification task model for relieving deviation feature distribution and training method thereof

Abstract

The invention provides a fact verification task model for relieving deviation characteristic distribution, which comprises a fact verification basic module, an evidence deviation distribution module, a declaration deviation distribution module, a dynamic constraint loss module and a deviation perception prediction module, and is characterized in that the fact verification basic module is used for receiving declaration and evidence and judging the authenticity of the output declaration; the evidence deviation distribution module is used for acquiring deviation probability distribution which is only mapped to the authenticity label by the evidence; the declaration deviation distribution module is used for acquiring deviation probability distribution which is mapped to the authenticity label only from the declaration; the dynamic constraint loss module is used for reducing information differences among different modules; and the deviation perception prediction module is used for receiving the output of the fact checking basic module, the evidence deviation distribution module and the declaration deviation distribution module, and calculating to obtain the judgment of the authenticity of the final declaration. According to the invention, unbiased declaration evidence distribution characteristics can be obtained, and the accuracy of fact verification authenticity prediction is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of natural language processing and deep learning, and in particular, to a fact-checking task model for alleviating biased feature distribution and a training method thereof. Background Art

[0002] False information and misleading content have posed great challenges to social and political stability [1]. Therefore, the automated fact-checking task of evaluating the authenticity of statements by analyzing multiple pieces of evidence has received extensive attention from researchers [2]. As described in [3], existing fact-checking methods generally fall into three stages: first, retrieving relevant documents according to the statement whose authenticity needs to be verified, then selecting relevant evidence sentences from the relevant documents, and then verifying the authenticity of the statement based on the selected evidence sentences.

[0003] For evidence-based fact-checking tasks, existing methods mainly rely on evidence retrieved from Wikipedia [3], fact-checking websites [4], or search engines [5] to judge the authenticity of statements. However, most fact-checking models are mainly oriented to English texts, and the evidence sources they use are usually English corpora. However, the spread of false information is not limited to a single language. As Chinese is widely used in the world, there are significant differences in language styles between Chinese and English, but there is still relatively little research on Chinese fact-checking. Existing Chinese fact-checking research mainly conducts verification based on retrieved evidence or the gold-standard evidence provided in the dataset [6].

[0004] However, existing Chinese fact-checking datasets may be affected by cultural differences and the dominance of official news sources, resulting in certain distribution biases. Specifically, analyzing the most widely used dataset CHEF[6] in current Chinese fact-checking tasks, each claim in the dataset belongs to one of the fields of politics, society, health, science, culture, and life. By statistically analyzing the distribution relationship between these fields and truthfulness labels, it can be observed that 64% of the claims in the social field and 66% of the claims in the health field are labeled as "REFUTES" (contradicting the truth), while 55% of the claims in the political field and 72% of the claims in the cultural field are labeled as "SUPPORTS" (supporting the truth). Additionally, by analyzing the co-occurrence relationship between phrases and labels in the claims, it can be observed that claims containing words such as "Movie" are more likely to be classified as "SUPPORTS", while claims containing words such as "Vaccine" or "Virus" are more likely to be classified as "REFUTES". This indicates that there are certain distribution bias problems in the commonly used Chinese fact-checking dataset CHEF, which can lead to fact-checking models achieving relatively high prediction scores by relying solely on surface features such as field information or phrase co-occurrence relationships between claims and labels. However, in the real world, the truthfulness labels of claims in different fields are evenly distributed, and the probability of each phrase appearing in different truthfulness labels is also evenly distributed. Therefore, models that ignore the deep semantic interaction information between claims and evidence and rely solely on these distribution bias features cannot handle real-world situations. Thus, reducing the impact of distribution bias on the overall performance of the model and ensuring the fairness of the model are urgent problems to be solved in current fact-checking tasks.

[0005] References:

[0006] [1] Allcott, H., Gentzkow, M., 2017. Social media and fake news inthe 2016 election. Journal of economic perspectives 31, 211–236. DOI:https: / / doi.org / 10.1257 / jep.31.2.211.

[0007] [2] Chen, Z., Hui, S.C., Zhuang, F., Liao, L., Li, F., Jia, M., Li,J., 2022. Evidencenet: Evidence fusion network for fact verification, in:Proceedings of the ACM WebConference 2022, pp. 2636–2645. DOI: https: / / doi.org / 10.1145 / 3485447.3512135.

[0008] [3] Thorne, J., Vlachos, A., Christodoulopoulos, C., Mittal, A.,2018. Fever: a large-scale dataset for fact extraction and verification, in:Proceedings of the 2018 Conference of the North American Chapter of theAssociation for Computational Linguistics: Human Language Technologies,Volume 1 (Long Papers), pp. 809–819. DOI: https: / / doi.org / 10.48550 / arXiv.1803.05355

[0009] [4] Andreas Hanselowski, Christian Stab, Claudia Schulz, Zile Li, and Iryna Gurevych. 2019. A richly annotated corpus for different tasks in automated fact-checking. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 493–503, Hong Kong, China. Association for Computational Linguistics. https: / / doi.org / 10.18653 / v1 / K19-1046

[0010] [5] Popat, K., Mukherjee, S., Strötgen, J., Weikum, G., 2017. Where the truth lies: Explaining the credibility of emerging claims on the web and social media, in: Proceedings of the 26th International Conference on World Wide Web Companion, pp. 1003–1012. DOI: https: / / doi.org / 10.1145 / 3041021.3055133.

[0011] [6] Hu, X., Guo, Z., Wu, G., Liu, A., Wen, L., Philip, S.Y., 2022.Chef: A pilot chinese dataset for evidence-based fact-checking, in:Proceedings of the 2022 Conference of the North American Chapter of theAssociation for Computational Linguistics: Human Language Technologies, pp.3362–3376. DOI: https: / / doi.org / 10.48550 / arXiv.2206.11863. Summary of the Invention

[0012] To solve the above problems and enable the fact-checking process for problems with distribution bias to be free from the interference of false surface features of the data, the present invention analyzes the bias distribution of existing Chinese fact-checking datasets, focuses on the deeper semantic associations between claims and evidence, thereby enhancing the ability to verify the authenticity of claims based on evidence.

[0013] The fact-checking task model and its training method for alleviating the bias feature distribution proposed by the present invention include: a fact-checking basic module, an evidence bias distribution module, a claim bias distribution module, a dynamic constraint loss module, and a bias-aware prediction module. Among them,

[0014] The fact-checking basic module is used to receive a claim and evidence and judge the authenticity of the output claim;

[0015] The evidence bias distribution module is used to obtain the bias probability distribution that is only mapped from the evidence to the authenticity label;

[0016] The claim bias distribution module is used to obtain the bias probability distribution that is only mapped from the claim to the authenticity label;

[0017] The dynamic constraint loss module is used to reduce the information difference between different modules;

[0018] The bias-aware prediction module: receives the outputs of the fact-checking basic module, the evidence bias distribution module, and the claim bias distribution module, and calculates to obtain the judgment of the final claim authenticity.

[0019] Furthermore, the fact-checking basic module includes an encoder, a multi-layer perceptron, and a softmax activation function; the input claim and the evidence set , output the statement and evidence regarding the probability distribution of the tags.

[0020] Furthermore, the statement deviation distribution module includes an encoder, a multi-layer perceptron, and a softmax activation function; input the statement into the statement deviation distribution module, and output the deviation probability distribution based on the statement.

[0021] Furthermore, the evidence deviation distribution module includes an encoder, a multi-layer perceptron, and a softmax activation function; input the evidence into the evidence deviation distribution module, and output the deviation probability distribution based on the evidence.

[0022] Furthermore, the dynamic constraint loss module uses the negative KL divergence between the probability distributions output by the fact-checking basic module and the deviation branch module to limit the information difference between the two modules.

[0023] Furthermore, the deviation-aware prediction module receives the overall distribution features obtained by the fact-checking basic module and the statement and evidence deviation distribution features obtained by the deviation branch module and , and outputs the final authenticity prediction , and the formula is:

[0024] .

[0025] Furthermore, the statement deviation features include the deviation distribution features between the field to which the statement belongs and the tag, and the deviation distribution features between the phrases in the statement and the tag; the evidence deviation features include the deviation distribution features between the field to which the evidence belongs and the tag, and the deviation distribution features between the phrases in the evidence and the tag.

[0026] According to another aspect of the present invention, a training method for a fact-checking task model that mitigates the deviation feature distribution is characterized in that the fact-checking basic module, the evidence deviation distribution module, the statement deviation distribution module, and the dynamic constraint loss module are trained together. Input the statement and evidence, and the loss function of the fact-checking task model is:

[0027]

[0028] where represents the cross-entropy loss, is a hyperparameter used to balance the loss and of.

[0029] Furthermore, , .

[0030] The beneficial effects of the present invention are as follows:

[0031] (1) The present invention analyzes the deviation distribution problem existing in the Chinese fact-checking dataset, and designs a fact-checking basic module, a deviation branch module, a dynamic constraint module, and a deviation-aware prediction module to alleviate the deviation distribution problem existing in the Chinese fact-checking dataset, more effectively simulating the fact-checking scenario in the real world, and helping to improve the accuracy of fact-checking methods in real scenarios.

[0032] (2) The present invention obtains the surface deviation distribution characteristics from the claim and evidence, and uses the deviation-aware module to remove the deviation distribution characteristics from the overall distribution characteristics of the claim and evidence, so as to obtain the deviation-free claim-evidence distribution characteristics, so that it can pay more attention to the deep factual semantic characteristics between the claim and the evidence, and thus effectively improve the accuracy of fact-checking authenticity prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 It is a schematic structural diagram of a fact-checking task model for alleviating deviation feature distribution according to an embodiment of the present invention.

[0035] Figure 2 It is a schematic flowchart of calculating the training loss of a fact-checking task model for alleviating deviation feature distribution according to an embodiment of the present invention.

[0036] Figure 3 It is a schematic flowchart of predicting the final authenticity label according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0038] The deviation distribution characteristics existing between the claim, evidence, and label in the Chinese fact-checking dataset include: the deviation distribution characteristics between the fields to which the claim and evidence belong and the label, and the deviation distribution characteristics between the phrases in the claim and evidence and the label.

[0039] The proof methods include:

[0040] (1) For the deviation distribution characteristics between the fields to which the claims and evidence belong and the labels, statistical methods can be used for calculation. First, use the domain information of the claims and evidence in the Chinese fact-checking dataset to statistically calculate the probability of the co-occurrence of the fields to which the claims and evidence belong and different authenticity labels. Since the authenticity label distribution probabilities vary greatly for claims and evidence belonging to different fields, the authenticity label can be judged based on this deviation distribution characteristic according to the domain information of the claim.

[0041] (2) For the deviation distribution characteristics between the phrases in the claims and evidence and the labels, the Local Mutual Information (LMI) statistical method can be used for calculation. First, use the jieba word segmentation tool to segment the claim and evidence texts, and then use to calculate the distribution probabilities of each phrase after the given word segmentation and the labels. Because this measurement method has a bias towards phrases with low occurrence frequencies, the Local Mutual Information (LMI) method is used to identify high-frequency phrases that are strongly correlated with specific labels:

[0042]

[0043]

[0044] Since there is a strong correlation between some phrases and the corresponding labels, the authenticity label can then be judged based on this deviation distribution characteristic according to some phrases in the claim or evidence.

[0045] It should be noted that there may also be deviation distribution characteristics in the text expression and grammar structure in the dataset that are difficult to directly measure.

[0046] The above-mentioned characteristics are manifested as the authenticity label can be judged only through the claim or evidence. That is, the claim deviation characteristics include the deviation distribution characteristics between the field to which the claim belongs and the label, the deviation distribution characteristics between the phrases in the claim and the label, etc., which are manifested as the characteristics of judging the authenticity label only through the claim; the evidence deviation characteristics include the deviation distribution characteristics between the field to which the evidence belongs and the label, the deviation distribution characteristics between the phrases in the evidence and the label, which are manifested as the characteristics of judging the authenticity label only through the evidence.

[0047] For example, in the most commonly used Chinese fact-checking dataset CHEF, the claim of each piece of data belongs to one of the fields of politics, society, health, science, culture, and life. Among them, 64% of the claims in the social field and 66% of the claims in the health field have the authenticity label of "REFUTES", while 55% of the claims in the political field and 72% of the claims in the cultural field have the authenticity label of "SUPPORTS". Therefore, just judging the authenticity based on the information of the field to which the claim belongs can also achieve a correct rate of more than 50%. In addition, if the phrases "movie" or "release" appear in the claim, the probabilities that the claim has the authenticity label of "SUPPORTS" are 0.83 and 0.75 respectively. And if the phrases "vaccine" and "virus" appear in the claim, the probabilities that the claim has the authenticity label of "REFUTES" are 65% and 66% respectively. Therefore, without fully understanding the factual relationship expressed between the claim and the evidence, just based on some phrases that appear in the claim, a correct rate of more than 50% can also be achieved. However, this deviation feature distribution will interfere with the exploration of the factual connection between the claim and the evidence in the real world by fact-checking methods.

[0048] Regarding the problem of the deviation distribution characteristics existing between the claims, evidence, and labels in the Chinese fact-checking dataset, the present invention provides a fact-checking task model for alleviating the deviation feature distribution, as Figure 1 shown, including: a fact-checking basic module, an evidence deviation distribution module, a claim deviation distribution module, a dynamic constraint loss module, and a deviation-aware prediction module. Among them,

[0049] The fact-checking basic module is used to receive the claim and the evidence, and judge and output the authenticity of the claim;

[0050] The evidence deviation distribution module is used to obtain the deviation probability distribution that is mapped to the authenticity label only by the evidence;

[0051] The claim deviation distribution module is used to obtain the deviation probability distribution that is mapped to the authenticity label only by the claim;

[0052] The dynamic constraint loss module is used to reduce the information difference between different modules;

[0053] The deviation-aware prediction module: receives the outputs of the fact-checking basic module, the evidence deviation distribution module, and the claim deviation distribution module, and calculates to obtain the judgment of the final authenticity of the claim.

[0054] The fact-checking basic module includes an encoder, a multi-layer perceptron, and a softmax activation function. The input is the claim and the evidence set , and outputs the probability distribution of the claim and the evidence regarding the label. The training of the fact-checking basic module includes: using the claim c in the dataset and the corresponding evidence set Sequences are obtained by connecting using special delimiters , encoded using an encoder (e.g., the BERT - BASE - CHINESE pre - trained encoder). To obtain the overall features of the claim and evidence, the vector at the position is used as the representation of the claim - evidence sequence, and then it is input into a multi - layer perceptron and passed through the softmax activation function to obtain the probability distribution of the claim and evidence with respect to the labels. This process can be expressed as:

[0055]

[0056] where is the number of pieces of evidence, represents the output of the fact - checking basic module, represents the number of categories of the authenticity label ( when it is 2, it represents the SUPPORTS and REFUTES categories, indicating that the evidence supports the claim and the evidence refutes the claim respectively; when it is 3, it represents the SUPPORTS, REFUTES, and NEI categories, and the NEI category indicates that the evidence is insufficient to verify the authenticity of the claim). After training, the fact - checking basic module can obtain the probability distribution of the claim and evidence with respect to the labels, and this probability distribution can reflect the factual semantic probability distribution characteristics between the claim and the evidence, and may also include the deviation probability distribution characteristics that are only mapped from the claim or evidence to the label in the dataset.

[0057] To obtain the deviation probability distributions that are only mapped from the claim or evidence to the label respectively, the present invention constructs a claim deviation distribution module and an evidence deviation distribution module to achieve the acquisition of claim and evidence deviation characteristics.

[0058] The claim deviation distribution module includes an encoder, a multi - layer perceptron, and a softmax activation function. The claim is input into the claim deviation distribution module, and the deviation probability distribution based on the claim is output. Specifically, for a given claim , after encoding using an encoder (such as the BERT - BASE - CHINESE pre - trained encoder), it is input into a multi - layer perceptron and passed through the softmax activation function to obtain the deviation probability distribution of the claim. This process can be expressed as:

[0059]

[0060] where represents the output of the claim deviation sub - module, that is, the deviation probability distribution based on the claim, represents the number of categories of the authenticity label. Therefore, after training, this module can obtain the deviation probability distribution based on the claim.

[0061] Example: For the input statement "One of the causes of many diseases comes from bad eating habits at night", it is first converted into a vector representation by the encoder, then input into a multi-layer perceptron and passed through the softmax activation function to obtain the deviation probability distribution based on the statement. For example, the probability distribution for the three classification labels of SUPPORTS, REFUTES, and NEI is obtained: [0.3, 0.2, 0.5]. By comparing the probability values of the three categories, it can be found that the probability value corresponding to the NEI category is the largest at this time, so this statement can be classified into the NEI category. However, since this is a judgment based only on the statement and lacks the corroboration of corresponding evidence, the label that conforms to the actual situation cannot be obtained.

[0062] The evidence deviation distribution module includes an encoder, a multi-layer perceptron, and a softmax activation function. Input the evidence into the evidence deviation distribution module and output the deviation probability distribution based on the evidence. Specifically, for the given evidence , after encoding using an encoder (such as the BERT-BASE-CHINESE pre-trained encoder), it is input into a multi-layer perceptron and passed through the softmax activation function to obtain the deviation feature distribution of the evidence. This process can be expressed as:

[0063]

[0064] Among them, is the number of evidence items, represents the output of the fact-checking basic module, that is, the deviation probability distribution based on the evidence, represents the number of categories of authenticity labels. Therefore, after training, this module can obtain the deviation probability distribution based on the evidence.

[0065] Example: For the input evidence "Research shows that the factors leading to diseases are diverse, including genetics, environment, lifestyle, etc., and simple evening eating habits cannot directly cause most diseases", it is first converted into a vector representation by the encoder, then input into a multi-layer perceptron and passed through the softmax activation function to obtain the deviation probability distribution based on the statement. For example, the probability distribution for the three classification labels of SUPPORTS, REFUTES, and NEI is obtained: [0.2, 0.3, 0.5]. By comparing the probability values of the three categories, it can be found that the probability value corresponding to the NEI category is the largest at this time, so it can be classified into the NEI category. However, since this is a judgment based only on the evidence and lacks the corroboration of the corresponding statement, a judgment that conforms to the actual situation cannot be made.

[0066] The dynamic constraint loss module aims to reduce the uncertainty caused by the information difference between the two during the process of integrating the output probability distributions of the fact-checking basic module and the two deviation distribution modules, which can be achieved by calculating the information entropy difference between the probability distributions of the two modules. In one embodiment, the negative KL divergence between the output probability distributions of the fact-checking basic module and the deviation branch module is used to limit the information difference between the two modules. The specific process is as follows:

[0067]

[0068]

[0069]

[0070] where is the probability of the -th authenticity label output by the fact-checking basic module, are the probabilities of the -th authenticity label output by the claim and evidence deviation branch sub-modules respectively, is a hyperparameter used to balance the constraint losses based on the claim and evidence. The greater the distribution difference between the basic module and the deviation branch sub-module, the smaller the information uncertainty between the two, the smaller the degree of correction to the overall prediction result, corresponding to a larger KL divergence, a smaller overall loss, and less model correction; conversely, when the distribution difference is smaller, the information uncertainty between the two is greater, the degree of correction to the overall prediction result is greater, corresponding to a smaller KL divergence, a larger overall loss.

[0071] As Figure 2 shown, the fact-checking basic module, the two deviation distribution modules, and the dynamic constraint loss module are trained together. Given the input claim and evidence, the loss function of the fact-checking task model is:

[0072]

[0073] where represents the cross-entropy loss, is a hyperparameter used to balance the losses and . The training objective is to expect the model to reduce the gap between the predicted label and the true label without considering the deviation distribution of the dataset. Among them, in the case of and , the smaller the hyperparameters and , it means that the model considers less the influence of the deviation features; on the contrary, and are larger, more influence of the deviation features is considered. In one embodiment, they are set to 0.5 and 0.5 respectively.

[0074] During training, the fact-checking basic module and the two bias distribution modules are adjusted according to the total training loss. In the initial stage of training, the gap between the model prediction value and the true value is large, and the corresponding final loss value is also large. Through gradient descent and backpropagation, the model is prompted to adjust the training network in a large amplitude towards the true value; in the later stage of training, the gap between the model prediction value and the true value is small, and the corresponding final loss value is also small. Through gradient descent and backpropagation, the model is prompted to adjust the training network in a small amplitude towards the true value.

[0075] The bias-aware prediction module aims to alleviate the impact of the bias distribution of the dataset on the final authenticity prediction of the fact-checking task model. Specifically, it includes: inputting the overall distribution characteristics obtained from the fact-checking basic module and the statement and evidence bias distribution characteristics obtained from the bias branch module and , and outputting the final authenticity prediction . Through multi-source information integration, this module removes the bias probability distribution from the overall probability distribution to obtain the final unbiased probability distribution. As Figure 3 shown, the specific implementation of this process is as follows:

[0076]

[0077] Among them, is a hyperparameter used to balance the outputs of the basic module and the bias branch module, and is consistent with the hyperparameter in step S3. This bias-aware prediction module passes through and to adjust the degree of removing the bias probability distribution from the overall probability distribution of the model prediction. and The larger and , the greater the impact of the bias feature, and the more bias distribution features need to be removed; the smaller , the smaller the impact of the bias feature, and the fewer bias distribution features need to be removed. After passing through the bias-aware prediction module,

[0078] Example 1

[0079] For the input claim "One of the causes of many diseases comes from bad eating habits at night" and the corresponding evidence "Research shows that the factors leading to diseases are multifaceted, including genetics, environment, lifestyle, etc., and simple evening eating habits cannot directly cause most diseases". First, the claim and the evidence are concatenated and jointly input into the encoder of the fact-checking basic module to be converted into vector representations, and then input into the multi-layer perceptron of the fact-checking basic module and activated by softmax to obtain the probability distribution [0.5, 0.4, 0.1]. This probability distribution may be affected by the bias distribution of the affirmative expression and the SUPPORTS label in the claim, and the prediction result of SUPPORTS is obtained. Subsequently, the claim and the evidence are respectively input into the encoder of the bias sub-module to be converted into vector representations, and then input into the multi-layer perceptron of their respective bias sub-modules and activated by softmax to obtain the bias probability distributions based on the claim and the evidence respectively: [0.3, 0.1, 0.6] and [0.25, 0.3, 0.55]. Due to insufficient information for each, the prediction label of NEI is obtained. Through the bias-aware prediction module, the obtained probability distribution [-0.05, 0, -1.05] is normalized to obtain the final probability distribution [0.413, 0.435, 0.152], and the category with the largest probability value is selected as the final prediction category, that is: REFUTES. This is also consistent with the true label. Compared with being affected by the bias distribution of the training dataset, directly classifying the claim and the evidence initially may result in errors, while the prediction obtained after passing through the fact-checking basic module, the bias module, and the bias-aware prediction module can effectively alleviate the interference brought by the bias distribution in the training dataset.

[0080] Example 2: Experimental verification:

[0081] In one embodiment, the input text is the CHEF dataset of the corpus. The 2022 conference paper CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking first proposed the evidence-based Chinese fact-checking dataset.

[0082] The present invention effectively alleviates the distribution bias problem on the CHEF dataset, thereby improving the performance of the fact-checking model on this dataset. As shown in Table 1, the bidirectional debiasing model Dual-Debias Model of the present invention was experimented on the CHEF binary classification dataset (only including examples with labels of SUPPORTS and REFUTES), and claim-only, ChatGPT, EANN, MAC, QMFND, Conv-FFD, NEP, and DEP-FEND were used as baseline models for comparison in terms of Accuracy, Precision, Recall, and F1 metrics.

[0083] For claim-only, the present invention only encodes the claim into BERT and then sends it to a multi-layer perceptron to predict the final claim authenticity label, which is equivalent to the claim bias branch sub-module in the present invention. For ChatGPT, the present invention uses a zero-shot method to prompt ChatGPT to predict the authenticity of the input claim evidence through the chain of thought. EANN uses adversarial learning to extract shared features that can be generalized in various claims. MAC uses a hierarchical multi-head attention network to capture perceptual evidence to complete the fact-checking task. NEP constructs a claim environment perception network to improve the performance of fact-checking. QMFND uses quantum coding and quantum convolutional neural networks to achieve fact-checking. DEP-FEND combines fact-checking technology and text classification-based technology to achieve fact-checking, which is the previous state-of-the-art technology in the binary classification setting of the CHEF dataset. As shown in Table 1, the best results are in bold, and the second-best results are underlined.

[0084] Table 1

[0085]

[0086] The present invention also improves the effect of the model on the unbiased distribution dataset close to the real world. As shown in Table 2, the bidirectional debiasing model Dual-Debias Model of the present invention was experimented on the unbiased distribution symmetric-CHEF dataset constructed by the present invention, and claim-only, ChatGPT, Conv-FFD, NEP, and DEP-FEND were used as baseline models for comparison in terms of Accuracy, Precision, Recall, and F1 metrics.

[0087] Table 2

[0088]

[0089] Those of ordinary skill in the art can understand that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A fact-checking task model for alleviating the distribution of bias features, comprising: A fact-checking basic module, an evidence bias distribution module, a claim bias distribution module, a dynamic constraint loss module, and a bias-aware prediction module, where The fact-checking basic module is used to receive a claim and evidence and judge the authenticity of the output claim; The evidence bias distribution module is used to obtain the bias probability distribution that is only mapped from the evidence to the authenticity label; The claim bias distribution module is used to obtain the bias probability distribution that is only mapped from the claim to the authenticity label; The dynamic constraint loss module is used to reduce the information difference between different modules; The bias-aware prediction module: receives the outputs of the fact-checking basic module, the evidence bias distribution module, and the claim bias distribution module, and calculates to obtain the judgment of the final claim authenticity.

2. The fact-checking task model according to claim 1, wherein The fact-checking basic module includes an encoder, a multi-layer perceptron, and a softmax activation function; the input statement and the evidence set , and outputs the probability distributions of the statement and the evidence regarding the label.

3. The fact-checking task model according to claim 1, wherein The claim bias distribution module includes an encoder, a multi-layer perceptron, and a softmax activation function; inputting a claim to the claim bias distribution module outputs the bias probability distribution based on the claim.

4. The fact-checking task model according to claim 1, characterized in that, The evidence bias distribution module includes an encoder, a multi-layer perceptron, and a softmax activation function; inputting evidence to the evidence bias distribution module outputs the bias probability distribution based on the evidence.

5. The fact-checking task model according to claim 1, characterized in that, The dynamic constraint loss module uses the negative KL divergence between the probability distributions output by the fact-checking basic module and the bias branch module to limit the information difference between these two modules.

6. The fact-checking task model according to claim 1, wherein A deviation-aware prediction module that receives the overall distribution characteristics obtained by the fact-checking foundation module and the claim and evidence deviation distribution characteristics obtained by the deviation branch module and , and outputs the final authenticity prediction , and the formula is: 。 7. The fact-checking task model according to claim 1, characterized in that, The claim bias features include the bias distribution features between the field to which the claim belongs and the label, and the bias distribution features between the phrases in the claim and the label; the evidence bias features include the bias distribution features between the field to which the evidence belongs and the label, and the bias distribution features between the phrases in the evidence and the label.

8. The training method of the fact-checking task model according to any one of claims 1-7, characterized in that The fact-checking basic module, the two bias distribution modules, and the dynamic constraint loss module are trained together. Inputting a claim and evidence, the loss function of the fact-checking task model is: , where represents the cross-entropy loss, is a hyperparameter used to balance the loss and .

9. The training method according to claim 8, characterized in that ， 。

Images