A zero-new-facts anti-illusion method and system for audit summary generation

Abstract

The present application relates to the technical field of natural language processing, in particular to a zero-new-fact anti-illusion method and system for audit summary generation, which generates an atomic fact set by atomically disassembling the audit summary to be verified; performs cross-document evidence retrieval on each atomic fact to determine relevant evidence paragraphs from the audit original document set, and constructs an evidence candidate set for each atomic fact; checks each atomic fact and the evidence in the candidate set piece by piece to determine the result and detect cross-document evidence conflicts; based on the evidence candidate set and the conflict detection result, initiates automatic correction for the atomic fact determined as contradictory or without evidence, generates a corrected fact, and performs evidence retrieval and verification on the corrected fact in a loop; and reorganizes the atomic facts determined as supporting and the corrected facts in the original summary order to generate an output summary. The present application improves the accuracy of the generated summary content and meets the strict requirements of zero-new-fact audit.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a zero-new-fact anti-illusion method and system for generating audit summaries. Background Technology

[0002] With the development of natural language processing technology, large language models are being applied to tasks such as audit report generation and summary extraction. However, in practical applications, audit summaries generated by large language models are prone to containing illusory content that differs from the original documents, such as incorrect financial data, fictitious transactions, and incorrect regulatory citations. Auditing requires extremely high accuracy of facts, and such illusory content severely impacts the usability of the summary.

[0003] To address the large model illusion problem, existing technologies typically employ methods such as retrieval enhancement generation, cue engineering, or model fine-tuning to mitigate it. However, these existing technologies still have the following shortcomings in auditing scenarios: First, existing retrieval enhancement generation methods are mostly applicable to single-document question-and-answer scenarios, lacking cross-document alignment capabilities. When faced with multiple original audit documents, they cannot effectively handle information conflicts and inconsistencies between different documents. Second, existing fact-verification methods usually only output binary judgments of correct or incorrect, without providing an automatic correction mechanism for erroneous facts, and cannot generate a complete chain of evidence, making it difficult to meet the traceability requirements of audit working paper review. Third, auditing scenarios require the summary content to meet the zero new fact standard, meaning that every piece of information in the summary must have clear evidence in the original document; existing technologies have not yet provided an effective solution for this strict constraint. Summary of the Invention

[0004] To address the technical problem of inaccurate audit summary generation caused by the inability of large language models to automatically identify and correct factual errors, this invention aims to provide a zero-new-fact anti-illusion method and system for audit summary generation. The specific technical solution adopted is as follows: In a first aspect, the present invention provides a zero-new-fact anti-illusion method for audit summary generation, comprising the following steps: Step 1: Decompose the audit summary to be verified into atomic facts to generate a set of atomic facts; Step 2: Perform cross-document evidence retrieval for each atomic fact in the atomic fact set, determine relevant evidence paragraphs from the audit original document set, and construct an evidence candidate set for each atomic fact in the atomic fact set; Step 3: Verify each atomic fact in the atomic fact set against each piece of evidence in the evidence candidate set, determine the judgment result corresponding to each atomic fact in the atomic fact set, and detect cross-document evidence conflicts; Step 4: Based on the candidate evidence set and the detection results of cross-document evidence conflicts, automatically correct the atomic facts that are determined to be contradictory or unfounded, generate corrected facts, and re-execute steps 2 and 3 for evidence retrieval and verification. Step 5: Reassemble all atomic facts in the atomic fact set that are deemed to be supported and the corrected facts in the original digest order to generate an output digest.

[0005] Furthermore, in step 1, the audit summary to be verified is decomposed into atomic facts to generate a set of atomic facts, including: Based on the constructed decomposition prompt words, the audit summary to be verified is decomposed into indivisible subject-verb-object structure sentences using a large language model. Each subject-verb-object structure sentence is used to represent an independent factual information. Based on the decomposition of all the subject-verb-object structured sentences, an atomic fact set is formed.

[0006] Further, in step 2, cross-document evidence retrieval is performed on each atomic fact in the atomic fact set to determine relevant evidence paragraphs from the original audit document set, and to construct an evidence candidate set for each atomic fact in the atomic fact set, including: Each original document in the audit original document set is segmented by sentence or fixed window length to obtain a set of text blocks from the audit original document set; Determine the similarity score between each atomic fact in the set of atomic facts and each text block in the set of text blocks; The first set of text blocks with the highest similarity scores are selected to construct a candidate set of evidence for each atomic fact in the atomic fact set.

[0007] Furthermore, the similarity score between each atomic fact in the atomic fact set and each text block in the text block set is determined using edit distance similarity, Jaccard similarity, or cosine similarity.

[0008] Further, in step 3, each atomic fact in the atomic fact set is verified one by one with each piece of evidence in the evidence candidate set to determine the judgment result corresponding to each atomic fact in the atomic fact set, including: For each atomic fact in the atomic fact set, based on the judgment prompt word parameters, the large language model is used to judge the factual consistency of the atomic fact with each piece of evidence in the evidence candidate set, and the factual consistency judgment result is output. The factual consistency judgment result includes support, contradiction, and no basis. Based on the fact consistency determination result, a comprehensive determination result is determined for each atomic fact in the atomic fact set.

[0009] Furthermore, based on the factual consistency determination result, a comprehensive determination result is determined for each atomic fact in the atomic fact set, including: If all the factual consistency judgments are unfounded, then the overall judgment result will be set to unfounded. If any factual consistency determination result is contradictory, then the overall determination result will be set as contradictory; If any factual consistency determination result is supportive and all factual consistency determination results are not contradictory, then the overall determination result will be set to supportive.

[0010] Furthermore, step 3 involves detecting cross-document evidence conflicts, including: When using a large language model to determine the consistency between each atomic fact in the atomic fact set and the facts of each piece of evidence in the evidence candidate set, the specific evidence information corresponding to the evidence whose judgment result is supportive or contradictory is extracted. For any atomic fact in the set of atomic facts, if there are both contradictions and supporting evidence in its corresponding fact consistency determination result, or if there are at least two pieces of evidence with contradictory determination results in its corresponding fact consistency determination result, and the specific evidence information extracted from the at least two pieces of evidence with contradictory determination results is inconsistent, then the atomic fact is determined to have cross-document evidence conflict, and the corresponding evidence is marked as a set of conflicting evidence.

[0011] Furthermore, in step 4, automatic correction is initiated for atomic facts that are deemed contradictory or unfounded by the judgment result, generating corrected facts, including: For any atomic fact whose comprehensive judgment result is contradictory, if the detection result of cross-document evidence conflict indicates the existence of a cross-document evidence conflict set, then according to the preset evidence source priority rule, the evidence with the highest priority is selected from the cross-document evidence conflict set as the target evidence of the atomic fact, and based on the target evidence, the corrected fact is obtained using a large language model. For any atomic fact whose comprehensive judgment result is contradictory, if the detection result of cross-document evidence conflict indicates that there is no cross-document evidence conflict set, then all evidence with contradictory fact consistency judgment results of the atomic fact are obtained to obtain contradictory evidence set, and all evidence with supporting fact consistency judgment results of the atomic fact are obtained to obtain supporting evidence set. Then, based on the contradictory evidence set and the supporting evidence set, the corrected fact is obtained using a large language model. For any atomic fact that is deemed unfounded by the comprehensive judgment, the atomic fact is marked as to be deleted, and the corrected fact is set to null.

[0012] Furthermore, the evidence tracing information includes the original text of the evidence paragraph corresponding to the fact unit, the name of the source document, the paragraph position, and the consistency determination result of the evidence.

[0013] Secondly, the present invention also provides a zero-new-fact anti-illusion system for audit summary generation, the system comprising: The atomic fact decomposition module is used to decompose the audit summary to be verified into atomic facts and generate a set of atomic facts. The cross-document retrieval module is used to perform cross-document evidence retrieval for each atomic fact in the atomic fact set, determine relevant evidence paragraphs from the audit original document set, and construct an evidence candidate set for each atomic fact in the atomic fact set; The evidence alignment and verification module is used to verify each atomic fact in the atomic fact set and each piece of evidence in the evidence candidate set one by one, determine the judgment result corresponding to each atomic fact in the atomic fact set, and detect cross-document evidence conflicts. The conflict detection and automatic correction module is used to automatically correct atomic facts that are determined to be contradictory or unfounded based on the evidence candidate set and the detection results of cross-document evidence conflicts, generate corrected facts, and re-execute steps 2 and 3 for evidence retrieval and verification. The summary generation module is used to reorganize all atomic facts in the atomic fact set that are deemed to be supported by the judgment results and the corrected facts in the original summary order to generate an output summary.

[0014] Thirdly, the present invention also provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0015] Fourthly, the present invention also provides a computer-readable storage medium storing computer program code that, when executed on a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0016] This invention offers the following advantages: by decomposing the audit summary to be verified into atomic facts, the text is refined into independent units to ensure the accuracy of verification and tracing; by performing cross-document evidence retrieval, cross-document alignment of multi-document evidence is supported, providing comprehensive original evidence for fact verification; by verifying each atomic fact in the atomic fact set and each piece of evidence in its evidence candidate set and detecting cross-document evidence conflicts, it can accurately identify illusory content in the summary and effectively handle information conflicts between different original documents; for problematic facts, automatic correction is initiated and cyclical verification is performed to automatically correct errors, ensuring that the corrected facts are also supported by solid underlying evidence; finally, the verified facts are recombined and output in the original summary order, completely eliminating unfounded information and ensuring that every piece of content in the final summary has clear evidence in the original document, truly achieving the "zero new facts" goal required by the audit scenario, and significantly improving the accuracy and credibility of the summary facts. Attached Figure Description

[0017] To clearly illustrate the technical features of this solution, the embodiments of the present invention include the following figures: Figure 1 This is a flowchart illustrating the implementation of a zero-new-fact anti-illusion method for generating audit summaries according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a zero-new-fact anti-illusion system for generating audit summaries according to an embodiment of the present invention. Detailed Implementation

[0018] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings.

[0019] The following will provide a detailed description of a zero-new-fact anti-illusion method and system for generating audit summaries, provided by an embodiment of the present invention, with reference to the accompanying drawings.

[0020] Figure 1 This diagram illustrates the basic flowchart of a zero-new-fact anti-illusion method for generating audit summaries according to an embodiment of the present invention. Figure 1 As shown, the method specifically includes the following steps: Step 1: Decompose the audit summary to be verified into atomic facts to generate a set of atomic facts.

[0021] A set of atomic facts refers to a collection composed of indivisible, basic factual units (such as simple sentences with a subject, verb, and object expressing a single piece of information). Since the original audit summaries to be verified are often complex, long sentences containing overlapping information, processing them as a whole can lead to vague search targets and inaccurate verification positioning. Decomposing them into independent atomic facts enables fine-grained, precise cross-document retrieval, independent cross-validation, and targeted local error correction, avoiding the entire sentence being deemed invalid due to misinterpretations of local information.

[0022] In a specific example, step 1 involves decomposing the audit summary to be verified into atomic facts to generate a set of atomic facts. This includes: based on the constructed decomposition prompts, using a large language model to decompose the audit summary to be verified into indivisible subject-verb-object structure sentences, each of which represents an independent factual information; and constructing a set of atomic facts based on all the decomposed subject-verb-object structure sentences.

[0023] Specifically, using a large language model, the audit summary to be verified is decomposed into indivisible subject-verb-object simple sentences by constructing decomposition prompt words. Each sentence expresses an independent factual information and is output in JSON array format, thus obtaining a set of atomic facts composed of each sentence. : (1) in: Indicates the input summary; This represents the function for disassembling atomic facts; This indicates the parameters for disassembling the prompt words; Indicates the first Each is an independent unit of atomic fact; m represents the number of atomic facts obtained from the decomposition.

[0024] Taking the summary to be verified, "The company's revenue in the fourth quarter was 350 million yuan, accounting for 28% of total revenue, a year-on-year increase of 10%, and all contracts have been stamped," and three original audit documents as examples, the contents of the three original documents are as follows: Document 1 (Financial Report_2024Q4.txt): The company's actual revenue in the fourth quarter was 380 million yuan, accounting for 30.4% of total revenue, representing a year-on-year increase of 8.2%. All sales contracts are based on signed receipts.

[0025] Document 2 (Sales Ledger.txt): Q4 total revenue, 380M, accounting for 30.4%, year-on-year increase of 8.2%. Contract signing form: receipt.

[0026] Document 3 (Auditing Standards.txt): Revenue recognition must be based on a signed receipt; a stamp is not a necessary condition.

[0027] Calling the large language model, inputting the decomposition prompt "Please break the following summary into independent atomic facts, each fact being a complete simple sentence, and output a JSON array", yields the following 4 atomic facts: F1 = "The company's fourth-quarter revenue was 350 million yuan", F2 = "Fourth-quarter revenue accounted for 28% of total revenue", F3 = "Fourth-quarter revenue increased by 10% year-on-year", and F4 = "All contracts have been stamped".

[0028] Step 2: Perform cross-document evidence retrieval for each atomic fact in the atomic fact set, determine relevant evidence paragraphs from the audit original document set, and construct an evidence candidate set for each atomic fact in the atomic fact set.

[0029] Audit source documents include one or more of the following: auditing standards, laws and regulations, industry standards, auditee's financial statements, accounting vouchers, audit working papers, and historical audit cases. The evidence candidate set refers to a collection of text paragraphs initially selected from the audit source documents that are highly relevant to the atomic facts semantically or in content. In audit scenarios, source documents are typically lengthy, with key information scattered across multiple heterogeneous documents (such as financial statements, sales ledgers, and audit vouchers). Directly processing all documents with a large language model not only easily exceeds the model's context window limit but also leads to information redundancy, resulting in distraction and decreased validation accuracy. Pre-identifying highly relevant paragraphs through cross-document retrieval effectively filters out irrelevant noise, converging supporting materials from multiple perspectives to form a high signal-to-noise ratio validation context. This provides an objective and reliable hard factual benchmark for subsequent accurate local cross-validation by the large language model.

[0030] In a specific example, step 2 involves performing cross-document evidence retrieval for each atomic fact in the atomic fact set, determining relevant evidence paragraphs from the audit original document set, and constructing an evidence candidate set for each atomic fact in the atomic fact set. This includes: segmenting each original document in the audit original document set by sentence or fixed window length to obtain a text block set of the audit original document set; determining the similarity score between each atomic fact in the atomic fact set and each text block in the text block set; and selecting the top predetermined number of text blocks with the highest similarity scores to construct an evidence candidate set for each atomic fact in the atomic fact set.

[0031] Specifically, firstly, a collection of original audit documents is constructed. The set of text blocks C, To audit the nth document in the original document collection, each document is segmented into several text blocks based on sentences or a fixed window length, and an inverted index or vector index is built for each text block to support fast retrieval. This represents the m-th document. The j-th text block. Next, for each atomic fact... With each text block in the text block set C Calculate similarity score Next, the top K text blocks with the highest similarity scores are selected to construct a candidate set of evidence. As shown in formula (2): (2) Where: K represents the pre-set number, which is a preset positive integer, usually a positive integer between 3 and 5; This function represents the function that takes the K highest-valued elements. This represents the k-th piece of evidence retrieved for the i-th atomic fact. Each piece of evidence includes text content and a source document identifier.

[0032] In a specific example, edit distance similarity, Jaccard similarity, or cosine similarity is used to determine the similarity score between each atomic fact in the set of atomic facts and each text block in the set of text blocks.

[0033] Specifically, similarity score The similarity is calculated using a string-based similarity algorithm, which can be edit distance similarity, Jaccard similarity, or cosine similarity. The calculation of edit distance similarity is shown in formula (3): (3) in: Representing each atomic fact With each text block Edit distance similarity between them; Representing each atomic fact With each text block Edit distance between; Let |c| represent each atomic fact. With each text block The length of the corresponding string.

[0034] The cosine similarity is calculated as shown in formula (4): (4) in: Representing each atomic fact With each text block Cosine similarity between them; and Each atomic fact represents a separate fact. With each text block Semantic vectors obtained through mapping from a pre-trained language model; This represents the vector dot product operation; It represents the magnitude of the vector.

[0035] Taking the three original audit documents mentioned above as examples, to construct a text block set, each original document in the audit document set is divided into 5 text blocks by a period. For each atomic fact, the edit distance similarity (Equation (3)) is used to calculate the similarity with each text block, and K=3 is taken. The retrieval results are as follows: For F1, the three text blocks with the highest similarity are: "The company's actual revenue in the fourth quarter was 380 million yuan" in Document 1 (similarity 0.72), "Q4 total revenue was 380 million yuan" in Document 2 (similarity 0.68), and "Revenue recognition must be based on the signed receipt" in Document 3 (similarity 0.21). The evidence candidate set E(F1) contains these three pieces of evidence.

[0036] For F2, the search results included "accounting for 30.4% of total revenue" in Document 1, "accounting for 30.4%" in Document 2, and irrelevant content in Document 3.

[0037] For F3, the search results included "year-on-year growth of 8.2%" in Document 1, "year-on-year growth rate of 8.2%" in Document 2, and irrelevant content in Document 3.

[0038] For F4, the search results include "subject to the signed receipt" in Document 1, "signed receipt" in Document 2, and "stamping is not a necessary condition" in Document 3.

[0039] Step 3: Verify each atomic fact in the atomic fact set against each piece of evidence in the evidence candidate set, determine the consistency judgment result corresponding to each atomic fact in the atomic fact set, and detect cross-document evidence conflicts.

[0040] Consistency checks and cross-document evidence conflict detection are core steps in cutting through the illusion of large models and aligning with real audit business scenarios. Because large models often fabricate data out of thin air due to interference from their own pre-training weights when generating summaries, it is essential to refine the verification granularity to the "atomic fact - single piece of evidence" level through line-by-line consistency checks. This is a reliable means of implementing the "zero new facts" hard constraint. Simultaneously, real audit source materials are often multi-sourced and heterogeneous. Documents from different departments or versions are highly prone to underlying data inconsistencies. Without cross-document conflict detection, the system blindly passes verification based on a single supporting piece of evidence, masking significant underlying audit risks. Therefore, performing this step can accurately identify the illusionary errors in the summary itself and expose information contradictions in the source documents, preventing the system from making arbitrary erroneous corrections when underlying data conflicts occur, thereby thoroughly ensuring the rigor of audit review and the absolute reliability of data traceability.

[0041] In a specific example, step 3 involves verifying each atomic fact in the atomic fact set against each piece of evidence in the evidence candidate set, and determining the judgment result corresponding to each atomic fact in the atomic fact set. This includes: for each atomic fact in the atomic fact set, using a large language model based on the judgment prompt word parameters to determine the factual consistency between the atomic fact and each piece of evidence in the evidence candidate set, and outputting a factual consistency judgment result, which includes support, contradiction, and lack of evidence; and based on the factual consistency judgment result, determining the comprehensive judgment result corresponding to each atomic fact in the atomic fact set.

[0042] Specifically, for each atomic fact in the set of atomic facts Generate a fact containing that atom. Its evidence candidate set The judgment prompts for each piece of evidence e are determined, and the large language model is used to judge the atomic fact. The factual consistency between each piece of evidence e is determined, and a judgment label is output to characterize the factual consistency judgment result, as shown in formula (5): (5) in: Represents each atomic fact in the set of atomic facts. Its evidence candidate set The result of the factual consistency determination for each piece of evidence e; Represents a consistency judgment function; Indicates the parameter for the prompt word; Expressing support; This indicates a contradiction; This indicates there is no basis for the claim.

[0043] In determining this atomic fact After determining the decision labels among all evidence in its candidate evidence set, the atomic fact is further determined by combining all decision labels. The corresponding comprehensive judgment result.

[0044] In a specific example, based on the factual consistency determination results, the comprehensive determination result corresponding to each atomic fact in the atomic fact set is determined, including: if all factual consistency determination results are unfounded, then the comprehensive determination result is set to unfounded; if any factual consistency determination result is contradictory, then the comprehensive determination result is set to contradictory; if any factual consistency determination result is supportive and all factual consistency determination results are not contradictory, then the comprehensive determination result is set to supportive.

[0045] Specifically, for each atomic fact in the set of atomic facts The comprehensive judgment result Determined by formula (6): (6) Further, step 3 involves detecting cross-document evidence conflicts, including: when using a large language model to determine the factual consistency between each atomic fact in the atomic fact set and each piece of evidence in the evidence candidate set, extracting the specific evidence information corresponding to the evidence whose judgment result is supportive or contradictory; for any atomic fact in the atomic fact set, if its corresponding factual consistency judgment result contains both contradiction and support, or if its corresponding factual consistency judgment result contains at least two pieces of evidence with contradictory judgment results, and the specific evidence information extracted from the at least two pieces of evidence with contradictory judgment results is inconsistent, then it is determined that the atomic fact has a cross-document evidence conflict, and the corresponding evidence is marked as a conflict evidence set.

[0046] Specifically, when using a large language model to verify each atomic fact in the atomic fact set against each piece of evidence in its evidence candidate set, when the output judgment label is "support" or "contradiction," the large language model extracts the target value or key entity that led to that judgment from the corresponding evidence as specific supporting information. This process is applied to each atomic fact in the atomic fact set. If evidence exists at the same time Make and If the two pieces of evidence contradict each other, then it is determined that there is a Type I cross-document evidence conflict, and the corresponding contradictory evidence is marked as a set of conflicting evidence. If this atomic fact If the judgment labels of the corresponding evidence contain at least two contradictions, then the specific evidence information corresponding to the at least two contradictory pieces of evidence is compared using a large language model or character matching algorithm. If the semantics or values ​​are not equal, it indicates that the contradictory evidence is conflicting, and a second type of cross-document evidence conflict is determined to exist. All relevant contradictory evidence is then marked as a set of conflicting evidence. For example, document A records "the audited entity's violation amount is 1 million yuan," and document B records "the audited entity's violation amount is 1.2 million yuan." For the atomic fact "violation amount of 1 million yuan," if the system retrieves two contradictory pieces of evidence, then these two contradictory pieces of evidence are marked as a conflicting evidence set.

[0047] Using the three original audit documents mentioned above as examples, the large language model is invoked, and each atomic fact in the atomic fact set is compared with each piece of evidence in its evidence candidate set according to equation (5): F1 vs. Document 1 evidence: Model output "Contradictory" (350 million vs. 380 million); vs. Document 2 evidence: Output "Contradictory"; vs. Document 3 evidence: Output "No basis". Based on equation (6), the overall judgment is Y(F1) = contradictory.

[0048] F2 vs. Document 1 evidence: Output "Contradictory" (28% vs. 30.4%); vs. Document 2 evidence: Output "Contradictory"; Overall judgment: Y(F2) = Contradictory.

[0049] F3 vs. Document 1 evidence: Output "Contradictory" (10% vs. 8.2%); vs. Document 2 evidence: Output "Contradictory"; Overall judgment: Y(F3) = Contradictory.

[0050] F4 versus Document 1 and Document 2: Output "No basis" (no mention of stamp); versus Document 3: Output "Contradictory" (Document 3 states that stamp is not a necessary condition). Overall judgment: Y(F4) = No basis.

[0051] No cross-document conflicts were detected because the contradictory evidence for all contradictory facts comes from multiple documents and points to the same thing.

[0052] Step 4: Based on the candidate evidence set and the detection results of cross-document evidence conflicts, initiate automatic correction for atomic facts that are determined to be contradictory or unfounded, generate corrected facts, and re-execute steps 2 and 3 for evidence retrieval and verification.

[0053] By combining the candidate evidence set with conflict detection results, automatic correction of atomic facts is performed. The boundaries of fact reconstruction are defined using the underlying text, and conflict results are used to eliminate interference caused by conflicting content from multiple drafts. This forces the model to reconstruct sentences based on a single legitimate source, preventing large language models from generating data arbitrarily outside the text. After the facts are corrected, the semantic features of the original sentences change. The evidence set determined in the initial retrieval cannot fully cover the new features. Therefore, evidence retrieval is re-executed to relocate related paragraphs using the new features, and verification is re-executed to verify whether the reconstructed facts match the updated evidence. This cyclical verification mechanism can effectively prevent the model from generating new illusions during the correction process.

[0054] Further, in step 4, automatic correction is initiated for atomic facts whose judgment results indicate contradiction or lack of basis, and corrected facts are generated based on evidence. This includes: for any atomic fact whose comprehensive judgment result indicates contradiction, if the cross-document evidence conflict detection result indicates the existence of a cross-document evidence conflict set, then according to the preset evidence source priority rule, the highest priority evidence is selected from the cross-document evidence conflict set as the target evidence for the atomic fact, and the corrected facts are obtained using a large language model based on the target evidence; for any atomic fact whose comprehensive judgment result indicates contradiction, if the cross-document evidence conflict detection result indicates the absence of a cross-document evidence conflict set, then all evidence with contradictory fact consistency judgment results for the atomic fact are obtained, resulting in a contradictory evidence set, and all evidence with supporting fact consistency judgment results for the atomic fact are obtained, resulting in a supporting evidence set, and then the corrected facts are obtained using a large language model based on the contradictory evidence set and the supporting evidence set; for any atomic fact whose comprehensive judgment result indicates lack of basis, the atomic fact is marked as to be deleted, and the corrected facts are set to null values.

[0055] Specifically, an automatic correction loop is initiated for atomic facts that are found to be contradictory or unfounded by the determination results. The specific implementation steps are as follows: Step 4.1: For atomic facts whose comprehensive judgment results are contradictory If there is a conflict of evidence across documents, the single piece of evidence with the highest priority will be selected as the target evidence based on the preset evidence source priority rules (such as the latest signing date or the highest level of the original document). This is followed by forcing the large language model to generate corrected facts solely based on this target evidence, thereby resolving contradictions in the underlying documents. It should be understood that if the target evidence cannot be determined through pre-defined evidence source priority rules... If a conflicting evidence set does not exist across documents, then all evidence paragraphs marked as conflicting will be collected. And possible supporting evidence paragraphs .

[0056] Step 4.2: Generate revised facts based on this evidence using a large language model. As shown in formula (7): (7) in: For fact correction functions; This indicates that the prompt word parameters have been corrected.

[0057] Step 4.3: For atomic facts that are determined to be unfounded in the overall assessment Mark it as pending deletion and set the corrected facts. = NULL.

[0058] Step 4.4: The revised facts Re-enter steps 2 and 3 to perform a second evidence retrieval and verification. If the result is NULL, skip the second check. If the result of the second check is still contradictory or unfounded, repeat steps 4.1 to 4.3 for further correction.

[0059] Step 4.5: Initialize the current correction count iter=1, and increment iter=iter+1 each time a correction is performed, setting the maximum correction count. Maximum number of corrections The value is 3. When the number of corrections (iter) reaches... The loop stops when the time is up, and the facts that still fail the verification are marked as requiring manual review, as shown in formula (8): (8) By setting the maximum number of corrections For a person who has gone through If an atomic fact still fails verification after the second correction (e.g., the original document itself is ambiguous), the system will stop looping and mark it as "requiring manual review". This avoids infinite loops, ensures system efficiency, and leaves difficult cases to be handled manually, reflecting the design philosophy of human-computer collaboration.

[0060] Using the three original audit documents mentioned above as examples, automatic correction is initiated for atomic facts that are determined to be contradictory or unfounded. Based on the evidence, corrected facts are generated, including: For F1 (contradictory), collect contradictory evidence Econ(F1) = Document 1 evidence, Document 2 evidence, and call the large language model to generate the corrected fact according to formula (7): "The company's revenue in the fourth quarter was 380 million yuan". Re-enter the corrected fact into step 3 for secondary verification and determine it as supported.

[0061] For F2, a corrected fact is generated: "Fourth quarter revenue accounted for 30.4% of total revenue," and the second validation passes.

[0062] For F3, generate the corrected fact: "Fourth quarter revenue increased by 8.2% year-over-year," and the second validation passes. For F4 (no basis), delete it and set... .

[0063] In this example, the correction passed on the first attempt and did not trigger a cyclical correction.

[0064] Step 5: Reassemble all atomic facts in the atomic fact set that are deemed to be supported and the corrected facts in the original digest order to generate an output digest.

[0065] At its core, a natural language-generated summary is a semantic aggregate composed of several independent facts. Through the preceding deconstruction, retrieval, and iterative verification mechanisms, the original summary is deconstructed and "cleaned" at the atomic level, eliminating baseless illusions (facts without evidence) and correcting erroneous statements (contradictory facts). The final set of discrete factual units possesses a high degree of factual accuracy. These rigorously verified discrete factual units are then reassembled and pieced together according to their chronological and logical positions in the original summary. This ensures the absolute truthfulness of each information node at the micro level and restores the original coherent pragmatic logic and reading experience of the summary at the macro level. Thus, without compromising readability, a highly reliable audit output text that thoroughly meets the strict standard of "zero new facts" is reconstructed.

[0066] Specifically, the large language model API (Application Programming Interface) is used to reorganize all atomic facts in the atomic fact set that are deemed to be supported and the corrected facts in the original summary order to generate a zero new output summary, as shown in formula (9): (9) in: Indicates the output summary; Represents the set of all valid and non-empty fact units; Reconstruct represents the fact reconstructing function, which concatenates the fact units into a natural language summary according to the order in which the facts appear in the original summary.

[0067] Furthermore, the method also includes: when generating the output summary, outputting the evidence tracing information for each fact; the evidence tracing information includes the original text of the evidence paragraph corresponding to the fact unit, the name of the source document, the paragraph position, and the consistency judgment result of the evidence.

[0068] Specifically, when generating the output summary using the Large Language Model API, it is also necessary to output the source information for each fact. This source information is output in a structured format to generate a structured source report. The source information specifically includes each fact unit. The corresponding evidence paragraph, source document name, paragraph position, and judgment label of the evidence are shown in formula (10): (10) in: Representing a unit of fact Evidence tracing information; The original text of the evidence paragraph; Indicates the source document identifier; This indicates paragraph position information.

[0069] Using the three original audit documents mentioned above as examples, an output summary is generated, and the evidence tracing information for each fact is output, including: the fact that the restructuring has passed the verification. According to formula (9), the output summary is generated as: the company's revenue in the fourth quarter was RMB 380 million, accounting for 30.4% of the total revenue, an increase of 8.2% year-on-year. At the same time, a tracing report (JSON format) is generated according to formula (10). This example shows that the present invention can automatically identify and correct 3 illusory facts in the summary, delete 1 unfounded fact, and output a summary with zero new facts and traceability, which effectively improves the credibility of the audit summary.

[0070] Based on the method provided in this invention, through a complete process of "atomic fact decomposition—cross-document evidence retrieval—large model line-by-line verification—conflict detection and automatic correction—source tracing output," it can automatically identify and correct illusory facts in audit summaries, ensuring that every piece of information in the output summary is verifiable and achieving the goal of zero new facts. Compared with the prior art, this invention has the following advantages: First, it supports cross-document alignment and conflict detection of multi-document evidence; second, it has an automatic correction loop that can verify multiple times until the facts are correct; third, it outputs complete evidence source tracing information, meeting the traceability requirements of audit work. Experiments show that this invention can significantly improve the accuracy of summary facts, eliminate illusory facts in audit summaries, ensure traceability, and is suitable for scenarios such as audit report generation and working paper review.

[0071] Based on the same inventive concept, embodiments of the present invention also provide a zero-new-fact anti-illusion system for audit summary generation, such as... Figure 2 As shown, the system includes: The atomic fact decomposition module is used to decompose the audit summary to be verified into atomic facts and generate a set of atomic facts. The cross-document retrieval module is used to perform cross-document evidence retrieval for each atomic fact in the atomic fact set, determine relevant evidence paragraphs from the audit original document set, and construct an evidence candidate set for each atomic fact in the atomic fact set; The evidence alignment and verification module is used to verify each atomic fact in the atomic fact set and each piece of evidence in the evidence candidate set one by one, determine the judgment result corresponding to each atomic fact in the atomic fact set, and detect cross-document evidence conflicts. The conflict detection and automatic correction module is used to automatically correct atomic facts that are determined to be contradictory or unfounded based on the evidence candidate set and the detection results of cross-document evidence conflicts, generate corrected facts, and re-execute steps 2 and 3 for evidence retrieval and verification. The summary generation module is used to reorganize all atomic facts in the atomic fact set that are deemed to be supported by the judgment results and the corrected facts in the original summary order to generate an output summary.

[0072] In this embodiment of the invention, the system can be divided into functional modules according to the above method example. For example, each functional module can be assigned to a specific function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware.

[0073] Based on the same inventive concept, embodiments of the present invention also provide a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute any of the aforementioned zero-new-fact anti-illusion methods for generating audit summaries.

[0074] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing computer program code that, when executed on a computer, causes the computer to perform any of the aforementioned zero-new-fact anti-illusion methods for generating audit summaries.

[0075] It should be noted that the above embodiments are merely preferred embodiments of the present invention and do not limit the scope of protection of the present invention. Equivalent substitutions or modifications made based on the concept of the present invention are all within the scope of protection of the present invention.

Claims

1. A zero-new-fact anti-illusion method for audit summary generation, characterized in that, Includes the following steps: Step 1: Decompose the audit summary to be verified into atomic facts to generate a set of atomic facts; Step 2: Perform cross-document evidence retrieval for each atomic fact in the atomic fact set, determine relevant evidence paragraphs from the audit original document set, and construct an evidence candidate set for each atomic fact in the atomic fact set; Step 3: Verify each atomic fact in the atomic fact set against each piece of evidence in the evidence candidate set, determine the judgment result corresponding to each atomic fact in the atomic fact set, and detect cross-document evidence conflicts; Step 4: Based on the candidate evidence set and the detection results of cross-document evidence conflicts, initiate automatic correction for atomic facts that are determined to be contradictory or unfounded, generate corrected facts, and re-execute steps 2 and 3 for evidence retrieval and verification. Step 5: Reassemble all atomic facts in the atomic fact set that are deemed to be supported and the corrected facts in the original digest order to generate an output digest.

2. The zero-new-fact anti-illusion method for audit summary generation according to claim 1, characterized in that, In step 1, the audit summary to be verified is decomposed into atomic facts to generate a set of atomic facts, including: Based on the constructed decomposition prompt words, the audit summary to be verified is decomposed into indivisible subject-verb-object structure sentences using a large language model. Each subject-verb-object structure sentence is used to represent an independent factual information. Based on the decomposition of all the subject-verb-object structured sentences, an atomic fact set is formed.

3. The zero-new-fact anti-illusion method for audit summary generation according to claim 1, characterized in that, Step 2 involves performing cross-document evidence retrieval for each atomic fact in the atomic fact set, identifying relevant evidence paragraphs from the original audit document set, and constructing an evidence candidate set for each atomic fact in the atomic fact set, including: Each original document in the audit original document set is segmented by sentence or fixed window length to obtain a set of text blocks from the audit original document set; Determine the similarity score between each atomic fact in the set of atomic facts and each text block in the set of text blocks; The first set of text blocks with the highest similarity scores are selected to construct a candidate set of evidence for each atomic fact in the atomic fact set.

4. A zero-new-fact anti-illusion method for audit summary generation according to claim 3, characterized in that, The similarity score between each atomic fact in the atomic fact set and each text block in the text block set is determined using edit distance similarity, Jaccard similarity, or cosine similarity.

5. A zero-new-fact anti-illusion method for audit summary generation according to claim 1, characterized in that, Step 3 involves verifying each atomic fact in the atomic fact set against each piece of evidence in the evidence candidate set, determining the judgment result corresponding to each atomic fact in the atomic fact set, including: For each atomic fact in the atomic fact set, based on the judgment prompt word parameter, the large language model is used to judge the factual consistency of the atomic fact with each piece of evidence in the evidence candidate set, and the factual consistency judgment result is output. The factual consistency judgment result includes support, contradiction and no basis. Based on the fact consistency determination result, a comprehensive determination result is determined for each atomic fact in the atomic fact set.

6. A zero-new-fact anti-illusion method for audit summary generation according to claim 5, characterized in that, Based on the factual consistency determination result, a comprehensive determination result is determined for each atomic fact in the atomic fact set, including: If all the factual consistency judgments are unfounded, then the overall judgment result will be set to unfounded. If any factual consistency determination result is contradictory, then the overall determination result will be set as contradictory; If any factual consistency determination result is supportive and all factual consistency determination results are not contradictory, then the overall determination result will be set to supportive.

7. A zero-new-fact anti-illusion method for audit summary generation according to claim 5, characterized in that, Step 3 involves detecting cross-document evidence conflicts, including: When using a large language model to determine the consistency between each atomic fact in the atomic fact set and the facts of each piece of evidence in the evidence candidate set, the specific evidence information corresponding to the evidence whose judgment result is supportive or contradictory is extracted. For any atomic fact in the set of atomic facts, if there are both contradictions and supporting evidence in its corresponding fact consistency determination result, or if there are at least two pieces of evidence with contradictory determination results in its corresponding fact consistency determination result, and the specific evidence information extracted from the at least two pieces of evidence with contradictory determination results is inconsistent, then the atomic fact is determined to have cross-document evidence conflict, and the corresponding evidence is marked as a set of conflicting evidence.

8. A zero-new-fact anti-illusion method for audit summary generation according to claim 6, characterized in that, Step 4 involves automatically correcting atomic facts that are deemed contradictory or unfounded by the judgment results, generating corrected facts, including: For any atomic fact whose comprehensive judgment result is contradictory, if the detection result of cross-document evidence conflict indicates the existence of a cross-document evidence conflict set, then according to the preset evidence source priority rule, the evidence with the highest priority is selected from the cross-document evidence conflict set as the target evidence of the atomic fact, and based on the target evidence, the corrected fact is obtained using a large language model. For any atomic fact whose comprehensive judgment result is contradictory, if the detection result of cross-document evidence conflict indicates that there is no cross-document evidence conflict set, then all evidence with contradictory fact consistency judgment results of the atomic fact are obtained to obtain contradictory evidence set, and all evidence with supporting fact consistency judgment results of the atomic fact are obtained to obtain supporting evidence set. Then, based on the contradictory evidence set and the supporting evidence set, the corrected fact is obtained using a large language model. For any atomic fact that is deemed unfounded by the comprehensive judgment, the atomic fact is marked as to be deleted, and the corrected fact is set to null.

9. A zero-new-fact anti-illusion method for audit summary generation according to claim 1, characterized in that, The method further includes: when generating the output summary, outputting the evidence tracing information for each fact; the evidence tracing information includes the original text of the evidence paragraph corresponding to the fact unit, the name of the source document, the paragraph position, and the consistency judgment result of the evidence.

10. A zero-new-fact anti-illusion system for audit summary generation, characterized in that, The system includes: The atomic fact decomposition module is used to decompose the audit summary to be verified into atomic facts and generate a set of atomic facts. The cross-document retrieval module is used to perform cross-document evidence retrieval for each atomic fact in the atomic fact set, determine relevant evidence paragraphs from the audit original document set, and construct an evidence candidate set for each atomic fact in the atomic fact set; The evidence alignment and verification module is used to verify each atomic fact in the atomic fact set and each piece of evidence in the evidence candidate set one by one, determine the judgment result corresponding to each atomic fact in the atomic fact set, and detect cross-document evidence conflicts. The conflict detection and automatic correction module is used to automatically correct atomic facts that are determined to be contradictory or unfounded based on the evidence candidate set and the detection results of cross-document evidence conflicts, generate corrected facts, and re-execute steps 2 and 3 for evidence retrieval and verification. The summary generation module is used to reorganize all atomic facts in the atomic fact set that are deemed to be supported by the judgment results and the corrected facts in the original summary order to generate an output summary.

Images