A two-stage evidence compression method for long text false information detection based on declarative saliency guidance
By employing a two-stage evidence compression method guided by saliency of statements, key evidence sentences are screened and restored, and reconstructed in conjunction with boundary context. This solves the problems of input compression and information preservation in the detection of misinformation in long texts, and improves the accuracy and stability of detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF TECH
- Filing Date
- 2026-05-09
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies struggle to effectively compress input length while preserving key evidence and contextual information in long text misinformation detection, resulting in insufficient detection accuracy and stability.
A two-stage evidence compression method based on claim saliency is adopted. Core evidence sentences are selected through claim saliency assessment, and the input is reconstructed into a compact input by combining the boundary context. A pre-trained language model is used for unified encoding and decision calibration.
It improves the accuracy and stability of detecting false information in long texts, reduces interference from redundant information, and enhances the applicability of the model in complex text scenarios.
Smart Images
Figure CN122489766A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of natural language processing, artificial intelligence, information content security, and text classification technology, specifically to a method for detecting false information in long texts based on claim salience scoring, evidence compression and reconstruction, and classification using a pre-trained language model. Background Technology
[0002] With the widespread dissemination of online news, social media, and generative AI content, misinformation detection has become a significant issue in information security governance. Unlike short text detection, long text misinformation detection often faces challenges such as large amounts of information, numerous distracting elements, and scattered key evidence. The factual sentences relied upon for authenticity judgments typically constitute only a small proportion of the entire text, while extensive background narratives, commentary, and repetitive descriptions significantly interfere with the model's judgment.
[0003] In existing technologies, one approach typically employs pre-trained language models such as BERT, RoBERTa, and DeBERTa to uniformly encode the entire article or long evidence document and directly output the truth classification result. While this approach is straightforward, in long text scenarios, it tends to consume limited encoding capacity on a large amount of weakly relevant content, thereby reducing attention to key factual sentences.
[0004] Another approach controls the length of long text inputs through methods such as fixed-length truncation, sliding windows, or block aggregation. While this approach can reduce computational overhead to some extent, the truncation position often does not align with the key evidence relied upon for authenticity judgment, potentially leading to the omission of important factual sentences. On the other hand, while sliding windows and block aggregation can expand the coverage, they further introduce problems such as redundant calculations, window fusion errors, and unstable results.
[0005] In addition, some solutions attempt to highlight key content through evidence retrieval, sentence filtering, or attention allocation, but most of these technologies primarily serve the retrieval or interpretation stage and have not yet formed a complete processing flow for judging the authenticity of long texts, which first compresses core evidence and then performs unified classification. In particular, the collaborative utilization of title information, the original order of evidence, and boundary context is still insufficient.
[0006] Therefore, how to effectively compress long text inputs while preserving the core evidence for authenticity judgment, and at the same time maintain the necessary topic information and local context information, thereby improving the accuracy, stability and interpretability of long text misinformation detection, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0007] This application aims to address the aforementioned shortcomings in the prior art by providing a two-stage evidence compression method for detecting misinformation in long texts based on claim salience.
[0008] Therefore, the first objective of this application is to propose a two-stage evidence compression method for detecting misinformation in long texts based on claim salience.
[0009] To achieve the above objectives, the first aspect of this application proposes a two-stage evidence compression method for detecting misinformation in long texts based on claim salience, the technical solution of which is as follows: A two-stage evidence compression method for detecting misinformation in long texts based on claim salience guidance includes the following steps: S1. Obtain an input sample consisting of a title or short declaration and its corresponding long text, and preprocess the input sample; S2. Segment the preprocessed long text into sentences and perform a declaration saliency evaluation on the segmented sentences; S3. Select core evidence sentences based on the salience assessment results, and restore the order of the core evidence sentences; S4. Extract the boundary context information of the long text and reconstruct the title or short statement, core evidence sentence and boundary context into a compact input; S5. Use a pre-trained language model to uniformly encode the compact input and output the realism prediction result; S6. Perform training optimization during the model training phase and decision calibration during the prediction phase to improve the stability and accuracy of long text misinformation detection.
[0010] Step S1 includes: S11. Obtain input samples from a public database used for long text misinformation detection or assertion-evidence reasoning, the public database including one or more of PolitiFact++, GossipCop++, McIntire, WELFake, and FNID-FakeNewsNet; Each input sample is defined as:
[0011] in, Indicates a title or short statement. This refers to the news article or long evidence text corresponding to the aforementioned title or short statement; S12. The input samples are standardized and processed, including removing irrelevant symbols, unifying the text format, and filtering out missing samples, to form a basic input suitable for subsequent sentence segmentation, evidence screening, and authenticity classification.
[0012] Step S2 includes: S21. For long texts Perform sentence-level segmentation to obtain a set of sentences:
[0013] in, These represent the individual sentences obtained after segmentation; S22. Use a statement salience scoring model to analyze the sentence set. Each sentence in the document is scored, and the sentences and their corresponding scores are organized into a score result set:
[0014] in, Sentence The claim significance score is obtained by using the ClaimBuster claim detection model or a claim detection model that is functionally equivalent to it.
[0015] Step S3 includes: S31. Sort the sentences in the scoring results set according to their statement significance scores, and select the sentences with the highest scores. These sentences form the core set of evidence:
[0016] in, This represents the core evidence set obtained through screening, and the steps described herein are used to retain the key evidence sentences most relevant to the judgment of authenticity. S32. Restore the core evidence set according to its position order in the original long text to obtain the core evidence set after order restoration:
[0017] in, This represents the core set of evidence after the order has been restored, ensuring that the filtered evidence retains high-value information while maintaining the semantic coherence of the original text.
[0018] Step S4 includes: S41. From long text Extracting the header boundary context from the beginning part and from long text Extracting the tail boundary context from the end part ; S42. Title or short statement The core evidence set after the order was restored and boundary context Reconstructed into a compact input sequence:
[0019] The reconstruction method is used to retain topic information, core evidence information, and necessary context information while compressing the input length.
[0020] Step S5 includes: S51. Reconstruct the compact input sequence The pre-trained language model is input into a unified encoding process to obtain a contextual representation.
[0021] The pre-trained language model is preferably deberta-v3-base; S52. Extract sequence-level representation from the context representation. And by mapping the linear classifier head to the classification space, we obtain the classification vector:
[0022] The classification vector is normalized to obtain the predicted probability distribution for each true category:
[0023] in, This indicates the accuracy of the prediction.
[0024] Step S6 includes: S61. During the model training phase, focal loss is used to optimize the classification process, and its loss function is:
[0025] in, This represents the model's predicted probability for the target class. Represents the category balance coefficient. Indicates the adjustment parameter; S62. During the overall training process, model selection is performed based on the validation set; S63. In a binary classification scenario, further threshold search and threshold calibration are performed on the predicted probabilities to obtain the final authenticity determination result.
[0026] A second aspect of the present invention relates to a two-stage evidence compression apparatus for detecting misinformation in long texts based on claim salience guidance, comprising: An input sample acquisition and preprocessing unit is used to acquire an input sample consisting of a title or short declaration and a corresponding long text, and to preprocess the input sample. The declaration saliency evaluation unit is used to segment the preprocessed long text into sentences and perform declaration saliency evaluation on the segmented sentences; The core evidence screening and order restoration unit is used to screen core evidence sentences based on the claim salience assessment results and restore the order of the core evidence sentences. The compact input reconstruction unit is used to extract the boundary context information of long texts and reconstruct the title or short statement, core evidence sentence and boundary context into compact input; The unified encoding and realism prediction unit is used to uniformly encode the compact input using a pre-trained language model and output the realism prediction result. The training optimization and decision calibration unit is used to perform training optimization during the model training phase and decision calibration during the prediction phase to improve the stability and accuracy of long text misinformation detection.
[0027] The working principle of this invention is as follows: Addressing the problems of scattered effective evidence, excessive redundant information, and the ease with which direct full-text encoding introduces noise in long-text misinformation detection, this invention proposes a two-stage evidence compression method guided by declaration saliency. First, an input sample consisting of a title or short declaration and its corresponding long text is obtained and preprocessed. Second, the long text is segmented into sentences, and each sentence is scored using a declaration saliency evaluation model to select the core evidence sentences most relevant to the authenticity judgment. Then, the selected core evidence sentences are restored according to their position order in the original text, while simultaneously extracting the contextual information of the beginning and end boundaries of the long text, and reconstructed into a compact input together with the title or short declaration. Finally, the compact input is fed into a pre-trained language model for unified encoding and classification prediction, and the final misinformation detection result is output through loss optimization during the training phase and threshold calibration during the prediction phase. Through this method, this invention can compress the input length while retaining high-value evidence and necessary contextual information, thereby improving the accuracy and stability of long-text misinformation detection.
[0028] The innovation of this invention is: 1. This invention proposes a long text evidence screening mechanism based on declaration salience guidance. By evaluating the declaration salience of sentences and screening core evidence before encoding long texts, the authenticity judgment process is transformed from "passive encoding of the whole text" to "priority retention of key evidence".
[0029] 2. This invention proposes a core evidence order restoration and boundary context joint reconstruction mechanism. By performing original order restoration on the filtered core evidence sentences and combining the title or short declaration with the boundary context of the long text for joint reconstruction, the semantic coherence and necessary context information are maintained while compressing the input length.
[0030] 3. This invention proposes a two-stage detection framework of "evidence compression and reconstruction + unified coding and classification + training optimization / decision calibration", which organically combines evidence selection, input organization, authenticity classification and training and prediction optimization to form a complete processing flow suitable for detecting false information in long texts.
[0031] This application has at least the following beneficial effects: (1) The present invention can reduce the interference of redundant background information, descriptive information and repetitive information in long texts on the authenticity determination process.
[0032] (2) The present invention can retain key evidence and its contextual coherence while compressing the input length, thereby alleviating the problem of information fragmentation.
[0033] (3) The present invention can improve the stability and accuracy of long text false information detection and enhance the applicability of the model in complex text scenarios.
[0034] (4) The present invention can retain more effective information related to authenticity determination under the constraint of limited input length, thereby improving the efficiency of input representation.
[0035] (5) The present invention can reduce the problem of missing key evidence caused by fixed truncation and non-significance screening, thereby improving the detection capability of long text samples. Attached Figure Description
[0036] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the overall process of a two-stage evidence compression method for detecting misinformation in long texts based on declaration salience, as described in this invention. Figure 2 This is a schematic diagram of the core evidence screening and sequence restoration process of this invention; Figure 3 A schematic diagram illustrating the input reconstruction, unified coding, and authenticity classification process for this invention; Figure 4 This is a trend chart of the experimental results of the top-k parameters on the FNID-FakeNewsNet dataset of this invention. Detailed Implementation
[0037] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0038] Example 1
[0039] like Figure 1As shown, the overall processing flow of this invention includes steps such as input sample acquisition and preprocessing, claim saliency assessment, core evidence screening and order restoration, boundary context extraction and compact input reconstruction, unified coding classification, and training optimization and decision calibration.
[0040] The method for detecting misinformation in long texts using a two-stage evidence compression method based on declaration salience guidance, as described in this invention, includes the following steps: Step 101: Obtain an input sample consisting of a title or short declaration and the corresponding long text, and preprocess the input sample.
[0041] In this embodiment, the input samples are derived from publicly available databases used for long-text misinformation detection or assertion-evidence reasoning. These databases may include one or more of PolitiFact++, GossipCop++, McIntire, WELFake, and FNID-FakeNewsNet. To uniformly describe the objects to be processed, each input sample is defined as:
[0042] in, Indicates a title or short statement. This represents the news article or long evidence text corresponding to the title or short statement. After obtaining the input sample, the text content is standardized, including removing irrelevant symbols, standardizing the text format, and filtering out missing samples, to form a basic input suitable for subsequent sentence segmentation, evidence screening, and authenticity classification.
[0043] Table 1. Dataset partitioning for the present invention
[0044] As shown in Table 1, this embodiment covers small-scale datasets, medium-scale datasets, large-scale datasets, and datasets with a "short statement + long evidence" structure. Among them, the training set, validation set, and test set sizes of FNID-FakeNewsNet are 15212, 1058, and 1054, respectively, which better demonstrates the applicability of this invention for evidence compression and reconstruction in long evidence scenarios; the large sample size of WELFake is used to verify the stability of this invention under high data volume conditions.
[0045] Step 102: Segment the preprocessed long text into sentences and perform a declaration saliency evaluation on the segmented sentences.
[0046] First, for long texts Perform sentence-level segmentation to obtain a set of sentences:
[0047] After obtaining the set of sentences, a declaration saliency scoring model is used to score each sentence, forming a set of scoring results:
[0048] in, Sentence The claim salience score is used to measure the potential contribution of each sentence to the truthfulness judgment. The claim salience scoring model is preferably ClaimBuster or a functionally equivalent claim detection model, used to measure the potential contribution of each sentence to the truthfulness judgment.
[0049] Step 103: Select core evidence sentences based on the salience assessment results, and restore the order of the core evidence sentences.
[0050] like Figure 2 As shown, in the core evidence screening and order restoration stage, sentences in the scoring result set are sorted according to their claim significance scores, and the top k sentences with the highest scores are selected as the core evidence set:
[0051] This processing method allows for the retention of key facts most relevant to the determination of authenticity from long texts, while compressing a large amount of background and weakly relevant information. Considering that sorting by score might disrupt the original narrative order, the core evidence set is then restored according to its position in the original long text, resulting in a restored core evidence set:
[0052] This maintains the coherence of the evidence in terms of semantic expression and discourse organization.
[0053] Step 104: Extract the boundary context information of the long text and reconstruct the title or short statement, core evidence sentence and boundary context into a compact input.
[0054] like Figure 3 As shown, in the input reconstruction and classification stage, from long text Extracting the header boundary context from the beginning part From long text Extracting the tail boundary context from the end part Then, the title or short statement The core evidence set after the order was restored and boundary context Reconstructed into a compact input sequence:
[0055] Furthermore, the compression ratio of the reconstructed input can be expressed as:
[0056] This reconstruction method can retain as much topic information, key evidence information, and contextual information at the beginning and end of the text as possible while controlling the input length, thus providing a higher quality input structure for subsequent unified coding.
[0057] Step 105: Use a pre-trained language model to uniformly encode the compact input and output the realism prediction result.
[0058] In this embodiment, the reconstructed compact input sequence is input into the pre-trained language model for unified encoding to obtain the context representation:
[0059] The pre-trained language model is preferably deberta-v3-base, used to learn the joint semantic relationships between title information, core evidence information, and boundary context information. Subsequently, sequence-level representations are extracted from the context representation. And by mapping the linear classifier head to the classification space, we obtain the classification vector:
[0060] The classification vector is then normalized to obtain the predicted probability distribution for each true category:
[0061] Based on this, the authenticity of the sample to be tested can be determined.
[0062] Step 106: Perform training optimization during the model training phase and decision calibration during the prediction phase.
[0063] In this implementation, focal loss is used to optimize the classification process during the model training phase. The loss function is as follows:
[0064] By introducing this loss function, the dominant role of easily classified samples in the training process can be reduced, while the model's attention to difficult-to-classify and minority class samples can be increased. During the overall training process, model selection is performed based on the validation set; in binary classification scenarios, threshold search and threshold calibration are further performed on the predicted probabilities to obtain the final veracity determination result.
[0065] Table 2
[0066] As shown in Table 2, the main experimental parameters are configured as follows. This embodiment uses a unified parameter configuration for the experiment. The encoder is deberta-v3-base, the number of core evidences is set to k=12, the maximum input length is 512, the batch size is 6, and the learning rate is... The system is trained using a focal loss with gamma=1.5, four rounds of early stopping, and a maximum of 20 training rounds. This configuration indicates that the performance improvement of this invention mainly comes from evidence compression and input reconstruction mechanisms, rather than relying on complex parameter tuning for specific datasets.
[0067] After completing the above-mentioned method and unified parameter settings, in order to verify the implementation effect of the present invention, the main experimental results, input structure ablation results, and core evidence quantity parameter results are explained respectively.
[0068] Table 3
[0069] Table 3 shows a comparison of the main experimental results. As shown in Table 3, the present invention demonstrates good results on the main metrics of the five public datasets. Among them, the accuracy improved from 0.8558 to 0.9469 and the macro-average F1 score improved from 0.8548 to 0.9454 on the FNID-FakeNewsNet dataset, showing the most significant improvement. This indicates that the present invention can more effectively extract and organize key evidence in long evidence authenticity judgment scenarios. The results on PolitiFact++, GossipCop++, McIntire, and WELFake also show that the present invention is not only applicable to long evidence tasks but also to conventional fake information detection tasks.
[0070] Table 4
[0071] Table 4 compares the results of the input construction ablation experiments. As shown in Table 4, the model performance decreased after removing ClaimBuster scores, title information, or boundary context enhancements. The most significant decrease in macro-average F1 score was observed on FNID-FakeNewsNet after removing boundary context enhancements, indicating that this invention does not rely solely on core sentence selection but rather on the combined effects of "claim saliency selection, order restoration, and context reconstruction." Similarly, the decrease was significant on PolitiFact++ after removing title information, further demonstrating the substantial role of the joint organization of the title and core evidence in determining authenticity.
[0072] like Figure 4As shown, as the number of core evidence k increases from 4 to 12, the accuracy and macro average F1 score generally show an upward trend; however, performance begins to decline when k exceeds 12. This indicates that the innovation of this invention is not merely "sentence screening," but rather the establishment of a workable balance mechanism between "retaining sufficient key evidence" and "avoiding the reintroduction of long text noise," where k=12 corresponds to a better evidence budget in this embodiment.
[0073] As can be seen from the above specific implementation methods and their corresponding figures, this invention does not simply truncate or directly classify long texts. Instead, it first extracts high-value evidence from long texts based on claim saliency, then combines title information and boundary context to complete input reconstruction, and finally submits the data to a pre-trained language model for unified classification. Therefore, this invention can maintain high information density with a limited input length budget and effectively reduce the interference of redundant background content on authenticity judgment.
[0074] Example 2
[0075] This embodiment relates to a two-stage evidence compression device for detecting misinformation in long texts based on claim salience guidance, including: An input sample acquisition and preprocessing unit is used to acquire an input sample consisting of a title or short declaration and a corresponding long text, and to preprocess the input sample. The declaration saliency evaluation unit is used to segment the preprocessed long text into sentences and perform declaration saliency evaluation on the segmented sentences; The core evidence screening and order restoration unit is used to screen core evidence sentences based on the claim salience assessment results and restore the order of the core evidence sentences. The compact input reconstruction unit is used to extract the boundary context information of long texts and reconstruct the title or short statement, core evidence sentence and boundary context into compact input; The unified encoding and realism prediction unit is used to uniformly encode the compact input using a pre-trained language model and output the realism prediction result. The training optimization and decision calibration unit is used to perform training optimization during the model training phase and decision calibration during the prediction phase to improve the stability and accuracy of long text misinformation detection.
[0076] It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, and are not intended to limit the scope of protection of the present invention. For those skilled in the art, various modifications and substitutions can be made to the sequence of steps, model selection, parameter settings, or equivalent implementation methods without departing from the technical concept of the present invention, and all such modifications and substitutions should fall within the scope of protection of the present invention.
Claims
1. A two-stage evidence compression method for detecting misinformation in long texts based on claim salience guidance, characterized in that, Includes the following steps: S1. Obtain an input sample consisting of a title or short declaration and its corresponding long text, and preprocess the input sample; S2. Segment the preprocessed long text into sentences and perform a declaration saliency evaluation on the segmented sentences; S3. Select core evidence sentences based on the salience assessment results, and restore the order of the core evidence sentences; S4. Extract the boundary context information of the long text and reconstruct the title or short statement, core evidence sentence and boundary context into a compact input; S5. Use a pre-trained language model to uniformly encode the compact input and output the realism prediction result; S6. Perform training optimization during the model training phase and decision calibration during the prediction phase to improve the stability and accuracy of long text misinformation detection.
2. The method according to claim 1, characterized in that, Step S1 includes: S11. Obtain input samples from a public database used for long text misinformation detection or assertion-evidence reasoning, said public database including one or more of PolitiFact++, GossipCop++, McIntire, WELFake, and FNID-FakeNewsNet; Each input sample is defined as: in, Indicates a title or short statement. This refers to the news article or long evidence text corresponding to the aforementioned title or short statement; S12. The input samples are standardized and processed, including removing irrelevant symbols, unifying the text format, and filtering out missing samples, to form a basic input suitable for subsequent sentence segmentation, evidence screening, and authenticity classification.
3. The method according to claim 2, characterized in that, Step S2 includes: S21. For long texts Perform sentence-level segmentation to obtain a set of sentences: in, These represent the individual sentences obtained after segmentation; S22. Use a statement salience scoring model to analyze the sentence set. Each sentence in the document is scored, and the sentences and their corresponding scores are organized into a score result set: in, Sentence The claim significance score is obtained by using the ClaimBuster claim detection model or a claim detection model that is functionally equivalent to it.
4. The method according to claim 3, characterized in that, Step S3 includes: S31. Sort the sentences in the scoring results set according to their statement significance scores, and select the sentences with the highest scores. These sentences form the core set of evidence: in, This represents the core evidence set obtained through screening, and the steps described herein are used to retain the key evidence sentences most relevant to the judgment of authenticity. S32. Restore the core evidence set according to its position order in the original long text to obtain the core evidence set after order restoration: in, This represents the core set of evidence after the order has been restored, ensuring that the filtered evidence retains high-value information while maintaining the semantic coherence of the original text.
5. The method according to claim 4, characterized in that, Step S4 includes: S41. From long text Extracting the header boundary context from the beginning part and from long text Extracting the tail boundary context from the end part ; S42. Title or short statement The core evidence set after the order was restored and boundary context Reconstructed into a compact input sequence: The reconstruction method is used to retain topic information, core evidence information, and necessary context information while compressing the input length.
6. The method according to claim 5, characterized in that, Step S5 includes: S51. Reconstruct the compact input sequence The pre-trained language model is input into a unified encoding process to obtain a contextual representation. The pre-trained language model is preferably deberta-v3-base; S52. Extract sequence-level representation from the context representation. And by mapping the linear classifier head to the classification space, we obtain the classification vector: The classification vector is normalized to obtain the predicted probability distribution for each true category: in, This indicates the accuracy of the prediction.
7. The method according to claim 6, characterized in that, Step S6 includes: S61. During the model training phase, focal loss is used to optimize the classification process, and its loss function is: in, This represents the model's predicted probability for the target class. Represents the category balance coefficient. Indicates the adjustment parameter; S62. During the overall training process, model selection is performed based on the validation set; S63. In a binary classification scenario, further threshold search and threshold calibration are performed on the predicted probabilities to obtain the final authenticity determination result.
8. A two-stage evidence compression device for detecting misinformation in long texts based on claim salience guidance, characterized in that, include: An input sample acquisition and preprocessing unit is used to acquire an input sample consisting of a title or short declaration and a corresponding long text, and to preprocess the input sample. The declaration saliency evaluation unit is used to segment the preprocessed long text into sentences and perform declaration saliency evaluation on the segmented sentences; The core evidence screening and order restoration unit is used to screen core evidence sentences based on the claim salience assessment results and restore the order of the core evidence sentences. The compact input reconstruction unit is used to extract the boundary context information of long texts and reconstruct the title or short statement, core evidence sentence and boundary context into compact input; The unified encoding and realism prediction unit is used to uniformly encode the compact input using a pre-trained language model and output the realism prediction result. The training optimization and decision calibration unit is used to perform training optimization during the model training phase and decision calibration during the prediction phase to improve the stability and accuracy of long text misinformation detection.