Large model illusion elimination method and device, equipment and storage medium

Abstract

The invention discloses a large model illusion elimination method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence natural language processing. Carrying out atomization disassembly on the initial generation content to obtain an atomic fact set; performing internal knowledge base semantic matching and networking retrieval cross validation on each atomic fact in the atomic fact set to obtain an evidence coverage rate; when the evidence coverage rate is lower than a preset coverage rate threshold value, generating an evidence vacancy report; performing differentiation rewriting on the initial generation content based on an evidence vacancy report to obtain target generation content, performing evidence coverage rate quantitative checking on the generation content through atomization disassembly and an internal and external double-source cross validation mechanism, blocking illusion cascade in the early stage of reasoning, and feeding back a verification result for iteration rewriting, so as to obtain the target generation content. And high-confidence content subjected to dominant verification is obtained.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence natural language processing technology, and in particular to a method, apparatus, device and storage medium for eliminating large-scale illusions. Background Technology

[0002] With the widespread application of large language models in the field of knowledge question answering, the content they generate suffers from the problem of "illusion," that is, although the text output by the model appears logically coherent, it contains false information that is inconsistent with the facts or deviates from the reference materials, leading to frequent factual illusions and fidelity illusions, which seriously affect the credibility and practicality of the generated content.

[0003] Existing technologies mainly rely on retrieval enhancement to statically constrain the model to answer based on references in the prompts. However, when the model engages in complex reasoning, it may still deviate from the logical boundaries of the references and introduce false facts, creating a cascading illusion. Furthermore, there is a lack of quantitative verification of the degree of matching between each statement and the references. At the same time, self-correction methods based on introspection mechanisms suffer from a self-verification trap. The verification model and the generation model are logically similar, which makes it impossible to identify deep-seated factual errors. The correction effect is highly uncertain, and it is also unable to handle content that exceeds the scope of pre-trained knowledge.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a method, apparatus, device, and storage medium for eliminating large-scale model illusions, aiming to solve the technical problem of how to block the cascading illusion phenomenon in the generated content of large-scale models through atomic verification and iterative feedback mechanisms.

[0006] To achieve the above objectives, the present invention provides a method for eliminating large-scale hallucinations, the method comprising the following steps: Get the initial generated content; The initially generated content is atomized and decomposed to obtain a set of atomic facts; The evidence coverage rate is obtained by performing semantic matching with an internal knowledge base and cross-validation through online retrieval on each atomic fact in the atomic fact set. When the evidence coverage rate is lower than a preset coverage threshold, an evidence gap report is generated. Based on the evidence gap report, the initial generated content is rewritten differentially to obtain the target generated content.

[0007] In one embodiment, the step of atomizing and decomposing the initially generated content to obtain a set of atomic facts includes: Dependency parsing is performed on the initially generated content to obtain sentence segmentation results and part-of-speech tagging results; Based on the coordinating conjunctions and clause structures in the sentence segmentation results, the sentence segmentation results are decomposed into propositional units; By performing referential resolution on the propositional unit and restoring the pronoun to the entity noun, a self-explanatory proposition is obtained. The self-explanatory propositions are standardized using triples to obtain a set of atomic facts.

[0008] In one embodiment, the step of performing dependency parsing on the initially generated content to obtain sentence segmentation results and part-of-speech tagging results includes: Sentence boundary detection is performed on the initially generated text to obtain sentence segmentation results; Each word in the sentence segmentation result is tagged with part of speech to obtain part-of-speech tagging results. Subject, predicate and object components are identified based on the part-of-speech tagging results to construct the core logical framework.

[0009] In one embodiment, the step of performing internal knowledge base semantic matching and online retrieval cross-validation on each atomic fact in the atomic fact set to obtain the evidence coverage includes: Each atomic fact in the atomic fact set is converted into an atomic fact vector, and the semantic similarity of each atomic fact vector with the evidence fragments in the internal knowledge base is calculated to obtain the internal matching score. When the internal matching score is lower than the preset semantic matching score threshold, the atomic facts are sent to the network retrieval interface to obtain real-time retrieval fragments through the network retrieval interface, and the real-time retrieval fragments are converted into external evidence vectors. Calculate the cosine similarity between the atomic fact vector and the external evidence vector to obtain the external matching score; Based on the internal matching score and the external matching score, the coverage status of each atomic fact is determined, and the proportion of covered atomic facts in the atomic fact set is calculated to obtain the evidence coverage rate.

[0010] In one embodiment, the step of converting each atomic fact in the atomic fact set into an atomic fact vector, and calculating the semantic similarity between each atomic fact vector and evidence fragments in the internal knowledge base to obtain an internal matching score includes: Each atomic fact is mapped to an atomic fact vector in a high-dimensional vector space through a pre-trained word vector model; Based on the atomic fact vector, a preset number of target evidence fragments are retrieved from the internal knowledge base to obtain a candidate evidence set; Each candidate evidence fragment in the candidate evidence set is converted into an internal evidence vector; The similarity scores between the atomic fact vector and each internal evidence vector are calculated using the cosine similarity function, and the internal matching scores are determined based on these similarity scores.

[0011] In one embodiment, the step of obtaining a real-time search fragment through the network search interface and converting the real-time search fragment into an external evidence vector includes: Extract search keywords from the atomic facts and construct a search query statement; Send the search query to the search engine application programming interface; Receive the list of search results returned by the search engine application interface, and extract text fragments from the list of search results to obtain real-time search fragments; The real-time retrieval fragments are converted into external evidence vectors using a pre-trained word vector model.

[0012] In one embodiment, the step of generating an evidence gap report when the evidence coverage rate is lower than a preset coverage threshold includes: Identify uncovered target atomic facts in the set of atomic facts, and generate an evidence gap report based on the uncovered target atomic facts; The step of rewriting the initially generated content based on the evidence gap report to obtain the target generated content includes: The evidence gap report is sent to the generation end so that the generation end can make differentiated adjustments to the initial generated content based on the evidence gap report to obtain rewritten content. The differentiated adjustments include deleting uncovered atomic facts or converting uncovered atomic facts into speculative statements. The rewritten content is re-executed with atomic decomposition and cross-validation until the evidence coverage is greater than or equal to the preset coverage threshold, at which point the verification is deemed successful. The rewritten content that passes verification will be used as the target generated content.

[0013] Furthermore, to achieve the above objectives, the present invention also proposes a large-scale hallucination elimination device, the device comprising: The content acquisition module is used to acquire the initial generated content; An atomization decomposition module is used to atomize the initially generated content to obtain a set of atomic facts; The cross-validation module is used to perform internal knowledge base semantic matching and online retrieval cross-validation on each atomic fact in the atomic fact set to obtain the evidence coverage rate. The differential rewriting module is used to generate an evidence gap report when the evidence coverage rate is lower than a preset coverage threshold; and to perform differential rewriting on the initial generated content based on the evidence gap report to obtain the target generated content.

[0014] Furthermore, to achieve the above objectives, the present invention also proposes a large model illusion removal device, the device comprising: a memory, a processor, and a large model illusion removal program stored in the memory and executable on the processor, the large model illusion removal program being configured to implement the steps of the large model illusion removal method as described above.

[0015] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a large model illusion removal program, which, when executed by a processor, implements the steps of the large model illusion removal method as described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the large model illusion elimination method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: By atomizing and decomposing the generated content into the smallest verifiable units, each fact is independently cross-verified by both the internal knowledge base and online retrieval. This enables quantitative evaluation of evidence coverage based on vector semantic similarity, thereby accurately intercepting unfounded facts in the early stages of reasoning to prevent the cascading propagation of illusions. When the coverage rate does not reach the preset threshold, the system feeds back the evidence gap report to the generation end, driving it to delete unfounded facts or convert them into speculative statements and iteratively verify them until the coverage rate reaches the target, ultimately obtaining high-confidence content that has undergone explicit binary verification. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the large-scale illusion elimination method of this application. Figure 2This is a structural block diagram provided for Embodiment 2 of the large-scale illusion elimination method of this application; Figure 3 This is a schematic diagram of the module structure of the large-scale illusion elimination device according to an embodiment of this application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the large model illusion elimination method in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or a large-scale illusion removal device capable of performing the above functions. The following description uses a large-scale illusion removal device as an example to illustrate this embodiment and the subsequent embodiments.

[0025] Based on this, embodiments of this application provide a method for eliminating large-scale hallucinations, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the large-scale illusion elimination method of this application.

[0026] In this embodiment, the large-scale hallucination elimination method includes steps S10 to S50: Step S10: Obtain the initial generated content; It should be noted that the initial generated content refers to the original answer text generated by the Large Language Model (LLM) for a user query. This text has not undergone any fact-checking and may contain factual illusions or fidelity illusions.

[0027] The initial generated content is produced by the main generator based on the user's input query and its pre-trained parameters, and is the object of processing in all subsequent verification and correction processes. In this embodiment, the initial generated content can be natural language text of arbitrary length, containing one or more complete sentences, serving as the starting point of the entire processing chain.

[0028] Understandably, obtaining the initial generated content refers to receiving the generated response text from the output interface of the main generator and using this text as raw data to be verified in subsequent processing flows. This step captures the raw generated content through the data receiving interface, providing an input source for subsequent atomic decomposition and ensuring that the verification process has a clearly defined processing object.

[0029] Step S20: Atomize and decompose the initially generated content to obtain a set of atomic facts; It should be noted that atomization decomposition refers to the process of breaking down a complex unstructured text into the smallest, independently verifiable units of statement. This process uses natural language processing technology to separate the complex, interconnected sentences in the initial generated content, ensuring that each decomposed unit has independent fact-checking capabilities and avoiding interference from complex sentence structures in the verification logic. In this embodiment, atomization decomposition is achieved through a series of syntactic analysis and semantic transformation operations, and is a key step in connecting the original text with structured facts.

[0030] An atomic fact set refers to the collection of all atomic facts obtained after atomization. This set stores multiple atomic facts in the form of a data structure, where each atomic fact represents a minimal verifiable statement. In this embodiment, the elements in the atomic fact set are standardized fact units with a triple structure. Each atomic fact can be independently semantically matched and its coverage evaluated, providing an operational factual basis for subsequent two-source cross-validation.

[0031] Understandably, atomizing the initially generated content involves performing operations such as dependency parsing, propositional decoupling, pronoun resolution, and triple standardization on the acquired raw generated text, transforming the text into a set of atomic facts. In one feasible implementation, this process includes splitting complex long sentences containing multiple parallel or subordinate relationships into several simple declarative sentences, replacing pronouns in each declarative sentence with specific entity nouns, standardizing the replaced declarative sentences into triple structures, and finally aggregating all triple structures to form a set of atomic facts, thus achieving the transformation from unstructured text to structured facts.

[0032] In the specific implementation, atomic fact extraction is as follows: After the main generating agent provides the answer A, the verification agent uses Named Entity Recognition (NER) and triple extraction techniques to transform it into a fact set F = {f1, f2, ..., f...}. n}

[0033] In one feasible implementation, step S20 includes steps A11 to A14: A11: Perform dependency parsing on the initial generated content to obtain sentence segmentation and part-of-speech tagging results; It should be noted that dependency parsing refers to the process of using natural language processing tools to analyze the grammatical structure of text, identify the dependency relationships between words in a sentence, and determine the core logical framework of the sentence. This analysis can reveal key components such as the subject, predicate, and object in a sentence and their grammatical roles, providing a structural foundation for subsequent proposition decomposition. In this embodiment, dependency parsing is implemented using the SpaCy tool, which performs deep analysis on the input text through a pre-trained grammatical model and outputs a dependency relationship tree between words.

[0034] Sentence segmentation results refer to the collection of individual sentences or sentence fragments obtained by dividing the initially generated content according to sentence boundaries. Sentence segmentation results are the first output of dependency parsing, used to divide long text into the smallest independently processable sentence units, facilitating subsequent part-of-speech tagging and component recognition. In this embodiment, the sentence segmentation results contain one or more complete sentences, each of which serves as the basic unit for subsequent processing.

[0035] Part-of-speech tagging (POS) results refer to the set of tagging information obtained after marking each word in the sentence segmentation results with its grammatical category. POS tagging can identify the grammatical function of words in a sentence, such as nouns, verbs, and adjectives, providing a basis for identifying components such as subjects, predicates, and objects. In this embodiment, POS tagging results construct a grammatical feature layer of the text by adding part-of-speech tags to each word, supporting the extraction of the core logical framework.

[0036] Understandably, dependency parsing of the initially generated content refers to using natural language processing tools to perform sentence boundary detection and part-of-speech tagging on the original generated text. First, the text is segmented into independent sentences to obtain sentence segments. Then, each word in the sentence segments is tagged with a part-of-speech tag to obtain part-of-speech tagging results. In one feasible implementation, this process includes using the SpaCy tool to parse the initially generated content, identify sentence boundaries, and tag the part of speech of each word. The final output is an analysis result containing sentence boundaries and part-of-speech tags, providing a grammatical foundation for subsequent proposition decomposition.

[0037] Further, step A11 includes: Sentence boundary detection is performed on the initial generated text to obtain sentence segmentation results; Each word in the sentence segmentation results is tagged with its part of speech to obtain the part-of-speech tagging results. Based on the part-of-speech tagging results, the subject, predicate and object components are identified to construct the core logical framework.

[0038] Understandably, sentence boundary detection of the initial generated text refers to using sentence segmentation algorithms from natural language processing tools to scan the original generated text, identify sentence terminators and boundary markers, and divide the continuous text into several independent sentences, forming a segmentation result. Further, part-of-speech tagging of each word in the segmentation result involves analyzing each word in the segmentation result word by word, using a pre-trained part-of-speech tagging model to assign a corresponding part-of-speech label to each word, obtaining the part-of-speech tagging result. Going further, identifying subject, predicate, and object components based on the part-of-speech tagging results means extracting the nominal component acting as the subject, the verb component acting as the predicate, and the nominal component acting as the object from the sentence based on the part-of-speech tags and the dependency relationships between words. Finally, constructing the core logical framework refers to combining the identified subject, predicate, and object components into a core framework of triple structures, forming the basic factual forms of subject, predicate, and object, providing a structured foundation for subsequent triple standardization. In one feasible implementation, for the initially generated text, sentence boundary detection divides it into two sentence results. After part-of-speech tagging of each sentence, the subject, predicate and object components are identified in the first sentence to construct the core logical skeleton, thereby realizing the transformation from text to logical skeleton.

[0039] A12: Based on the coordinating conjunctions and clause structures in the sentence segmentation results, break down the sentence segmentation results into propositional units; It should be noted that coordinating conjunctions refer to words used to connect two or more parallel components, including, but not limited to, conjunctions such as "and," "furthermore," "as well as," "at the same time," and "in addition" in this embodiment. The presence of coordinating conjunctions indicates that the sentence contains multiple parallel factual statements, which need to be separated into independent propositions through decomposition. Coordinating conjunctions are key identification markers for proposition decoupling, used to locate the grammatical positions that need to be split.

[0040] Clause structure refers to the grammatical structure of subordinate clauses within a sentence, including complex syntactic structures such as relative clauses, adverbial clauses, and object clauses. Clauses typically modify the main clause or are nested within it as independent semantic units, requiring decomposition to extract them as independent propositions. Clause structure is a crucial basis for decomposing complex sentences, ensuring that each subordinate clause can be identified and separated.

[0041] A propositional unit refers to an independent declarative unit obtained by deconstructing coordinating conjunctions and clause structures. Each propositional unit expresses a complete semantic fact. A propositional unit is an intermediate form between the original sentence and atomic facts, and can be further processed into standardized atomic facts. In this embodiment, a propositional unit can be a simple declarative sentence or a pre-separated clause fragment, possessing independent semantic integrity.

[0042] Understandably, breaking down the sentence segmentation results into propositional units based on the coordinating conjunctions and subordinate clause structures in the sentence segmentation results means analyzing each sentence, identifying the coordinating conjunctions and subordinate clause structures, and breaking down complex sentences containing multiple coordinating components or clauses into several independent propositional units.

[0043] A13: By dissolving the pronouns in the propositional unit and restoring the pronouns to entity nouns, a self-explanatory proposition is obtained; It should be noted that reference resolution refers to the process of restoring pronouns in a sentence to the specific entities they refer to. This process determines the exact referent of a pronoun by analyzing the entity reference relationships in the context, thus eliminating referential ambiguity in the text. In this embodiment, the objects of reference resolution include pronouns such as "he," "it," and "the model," and the substitution operation enhances the self-containment of propositions, making each fact atom fully self-explanatory.

[0044] Pronouns are words in propositional units used to refer to previously mentioned content words, such as personal pronouns like he, she, it, that, and so on, or demonstrative pronouns. The use of pronouns causes propositional semantics to become context-dependent, reducing the ability to independently verify facts. In this embodiment, pronouns are target components that need to be identified and replaced; their presence is a marker that triggers the referential resolution process.

[0045] Entity nouns refer to specific nouns or noun phrases used to replace pronouns after pronoun resolution. These nouns explicitly refer to specific entities in the text. The introduction of entity nouns makes the proposition fully self-explanatory, allowing its complete semantics to be understood without relying on context. In this embodiment, entity nouns are derived from the names of specific objects already identified, ensuring that the semantics of the replaced proposition are accurate and independently verifiable.

[0046] Self-explanatory propositions are statements that, after referential resolution, can be understood independently without context. In self-explanatory propositions, all referential components have been replaced with explicit entity nouns, possessing complete semantic information and can be directly used for triple standardization. In this embodiment, self-explanatory propositions represent an upgraded form of propositional units, eliminating ambiguity caused by pronoun referencing and providing a clear semantic foundation for subsequent structured transformations.

[0047] Understandably, dereferencing propositional units involves analyzing the pronoun components within the propositional unit, tracing the referent of the pronoun in the context, and replacing the pronoun with the corresponding entity noun to obtain a self-explanatory proposition. In one feasible implementation, for propositional units containing high-performing models, dereferencing replaces the model with the name of a specific large language model, forming self-explanatory propositions with high-performing large language models, ensuring the semantic integrity and independence of the propositions.

[0048] A14: Standardize self-explanatory propositions using triples to obtain a set of atomic facts.

[0049] It should be noted that triple standardization refers to converting self-explanatory propositions into a standard structured form where Fact equals... Subject, Predicate, Object The processing procedure involves identifying the subject, predicate, and object components in a proposition and constructing a fact tuple consisting of three elements to achieve a standardized representation of the facts. In this embodiment, the triple standardization adopts a fixed three-element structure, ensuring that each atomic fact has a uniform format, facilitating subsequent verification processing.

[0050] Subject refers to the subject element in a triple, representing the main object of the factual statement. Subject originates from the subject component identified in a self-explanatory proposition and is the core entity of the atomic facts. In this embodiment, Subject can be any entity that acts as the subject of the statement, such as a person, organization, or concept.

[0051] A predicate is the verb element in a triple, representing a factual statement, relation, or action. Predicates originate from the predicate components identified in self-explanatory propositions, describing the association between the Subject and the Object. In this embodiment, a predicate can be various predicate components such as attribute relationships, actions, and state descriptions.

[0052] Object refers to the object element in a triple, representing the object or attribute value of a factual statement. Object originates from the object component identified in a self-explanatory proposition and is the target content of the factual statement. In this embodiment, Object can be a specific numerical value, another entity, attribute description, or other necessary elements that constitute the completeness of the fact.

[0053] Understandably, standardizing self-explanatory propositions into triples involves parsing the grammatical components of each self-explanatory proposition, extracting the subject, predicate, and object, constructing a structured fact tuple consisting of three elements: Subject, Predicate, and Object, and then aggregating all self-explanatory propositions to form an atomic fact set.

[0054] Step S30: Perform semantic matching with the internal knowledge base and cross-validation with online retrieval on each atomic fact in the atomic fact set to obtain the evidence coverage rate; It should be noted that internal knowledge base semantic matching refers to the verification process of calculating the semantic similarity between atomic facts and evidence fragments in a private knowledge base. This process evaluates whether supporting evidence exists for the atomic facts in the internal knowledge base through vector representation and similarity measurement, obtaining a quantified internal matching score. In this embodiment, internal knowledge base semantic matching uses a pre-trained word vector model to map text into high-dimensional vectors, and achieves semantic-level matching verification through cosine similarity calculation.

[0055] It should be noted that online retrieval cross-validation refers to a supplementary verification process where, when the internal knowledge base matching score falls below a preset threshold, an online retrieval plugin is activated to obtain real-time online information and perform semantic similarity calculations on the online retrieval fragments. This process obtains the latest information by calling a search engine interface, compensating for the deficiencies of the internal knowledge base in terms of timeliness and coverage, and obtaining a quantified external matching score. In this embodiment, online retrieval cross-validation constructs a retrieval query by extracting keywords from atomic facts, and then performs vector similarity calculations on the obtained relevant text fragments.

[0056] It should be noted that evidence coverage rate refers to the proportion of atomic facts in the atomic fact set that are verified as having supporting evidence after cross-verification through internal knowledge base semantic matching and online retrieval. Evidence coverage rate is a quantifiable verification indicator used to measure the proportion of verifiable content in the generated content. In this embodiment, evidence coverage rate is calculated by statistically analyzing the ratio of the number of covered atomic facts to the total number of atomic facts, and it is an important basis for triggering differentiated rewriting.

[0057] Understandably, performing internal knowledge base semantic matching and online retrieval cross-validation on each atomic fact in the atomic fact set refers to traversing the atomic fact set, sequentially performing internal knowledge base vector retrieval and similarity calculation for each atomic fact to obtain an internal matching score. When the internal score is lower than a preset threshold, online retrieval is further performed to obtain real-time evidence fragments and vector similarity calculation is performed to obtain an external matching score. The coverage status of each atomic fact is determined by combining the internal and external scores, and finally, the proportion of covered atomic facts is statistically analyzed to obtain the evidence coverage rate. In one feasible implementation, for an atomic fact, relevant evidence fragments are first retrieved in the internal knowledge base and similarity is calculated. If the score is lower than a threshold of 0.75, online retrieval is initiated to obtain the target data fragment, and an external matching score is calculated. If either the internal or external score exceeds the threshold, the atomic fact is considered to be covered.

[0058] In the specific implementation, for each f iFirst, a semantic matching score S{internal} is calculated in the private knowledge base. If the score is lower than the threshold, an online retrieval plugin is activated to obtain real-time search fragments and calculate the matching score S{external}. A coverage function C(f_i) is defined: if max(S{internal}, S{external}) > Threshold, then C(f_i) = 1; otherwise, it is 0. The overall coverage rate (ECR) is calculated.

[0059] Step S40: When the evidence coverage rate is lower than the preset coverage rate threshold, generate an evidence gap report; In one feasible implementation, step S40 includes step A21: identifying uncovered target atomic facts in the atomic fact set and generating an evidence gap report based on the uncovered target atomic facts; It should be noted that the preset coverage threshold refers to a pre-set critical value used to determine whether the evidence coverage meets the standard. The preset coverage threshold is the decision criterion for determining whether to trigger differentiated rewriting, and it is usually set according to the accuracy requirements of the application scenario. In this embodiment, the preset coverage threshold can be set to 0.9, which means that at least 90% of the atomic facts in the generated content must be supported by evidence.

[0060] An evidence gap report is an explanatory document generated when the evidence coverage rate fails to meet the standard, detailing the atomic facts for which records were not covered. The evidence gap report explicitly lists the atomic facts for which no supporting evidence was found in the internal knowledge base or online searches, providing the generating end with precise modification guidelines. In this embodiment, the evidence gap report includes the original text of the uncovered atomic facts, their triple representations, and the reasons for verification failure, thus explicitly expressing the verification results.

[0061] Understandably, generating an evidence gap report when the evidence coverage rate is lower than a preset coverage threshold means calculating the evidence coverage rate of the current set of atomic facts, comparing the calculated result with the preset coverage threshold, and if the calculated result is less than the threshold, identifying all uncovered atomic facts, and organizing these atomic facts into a structured list to generate an evidence gap report. In one feasible implementation, when the evidence coverage rate is 0.7 and the preset threshold is 0.9, 30% of the uncovered atomic facts are identified, and an evidence gap report containing details of these facts is generated.

[0062] In practice, if the ECR fails to meet the requirements, the verification agent will not only require a "rewrite", but will also generate an "evidence gap report" (e.g., "No supporting evidence was found for facts f3 and f5 in the database or online").

[0063] Step S50: Based on the evidence gap report, the initial generated content is rewritten in a differentiated manner to obtain the target generated content.

[0064] It should be noted that differentiated rewriting refers to a rewriting process that makes targeted adjustments to the initially generated content based on the list of uncovered atomic facts provided by the evidence gap report. Differential rewriting differs from full rewriting; it only modifies facts lacking supporting evidence, retaining content supported by evidence, thereby significantly reducing computational redundancy. In this embodiment, differentiated rewriting is performed by the generation end based on the evidence gap report, achieving precise content correction and avoiding unnecessary full-text reconstruction.

[0065] The target generated content refers to the final output text that achieves a preset coverage threshold after differentiated rewriting and iterative verification. The target generated content is the result of initial generated content after removing unsubstantiated facts, transforming unsubstantiated facts into speculative statements, and verifying them, thus possessing high evidence coverage and high credibility. In this embodiment, each statement in the target generated content undergoes explicit verification through internal knowledge base or online retrieval, greatly reducing the incidence of reasoning illusions and achieving a shift from blind generation to precise completion based on evidence gaps.

[0066] Understandably, differentiated rewriting of initial generated content based on evidence gap reports refers to sending the evidence gap report to the generation end. The generation end, based on the uncovered atomic facts listed in the report, performs deletion or transformation operations on the initial generated content to obtain rewritten content. This rewritten content is then subjected to atomic decomposition and cross-validation again until the evidence coverage rate reaches the target. The finally verified rewritten content is output as the target generated content. In one feasible implementation, when the initial generated content contains unverifiable facts, the generation end deletes the fact based on the evidence gap report or modifies the description to suggest that the fact may be true based on publicly available information. After forming the rewritten content, it is verified again to ensure that the evidence coverage rate reaches the preset threshold.

[0067] In one feasible implementation, step S50 includes steps A31 to A33: A31: Send the evidence gap report to the generation end so that the generation end can make differentiated adjustments to the initial generated content based on the evidence gap report to obtain rewritten content. The differentiated adjustments include deleting uncovered atomic facts or converting uncovered atomic facts into speculative statements. It should be noted that the generation end refers to the model or module that receives evidence gap reports and performs differentiated rewriting operations. The generation end is the main body of content generation and is responsible for adjusting the generation strategy based on the verification results. In this embodiment, the generation end is the main generation agent, which precisely modifies the initially generated content based on the evidence gap reports, realizing a closed-loop feedback from verification to rewriting.

[0068] Rewritten content refers to the new text obtained after the generator performs differentiated adjustments on the initial generated content based on the evidence gap report. Rewritten content is an intermediate product of the initial generated content after deleting unsubstantiated facts or transforming unsubstantiated facts into speculative statements, and its evidence coverage still needs to be re-verified. In this embodiment, the rewritten content retains the parts of the initial generated content that are supported by evidence, and only modifies the uncovered atomic facts, achieving targeted content optimization.

[0069] Speculative statements refer to expressions that mark uncertainties regarding facts lacking supporting evidence. This involves using keywords such as "possible," "speculated," and "unverified" to weaken factual statements. Speculative statements transform definitive factual statements into probable ones, reducing the confidence level of the content and avoiding the dissemination of unverified information. In this embodiment, speculative statements are a key means of differentiation adjustment, enabling the model to accurately represent the boundaries of knowledge.

[0070] Understandably, sending the evidence gap report to the generation end refers to transmitting the report, containing a list of uncovered atomic facts, to the generation end via an internal interface or messaging mechanism. In one feasible implementation, the evidence gap report is sent in a structured data format, containing the identifier, location, and suggested processing method for each uncovered atomic fact. Furthermore, enabling the generation end to differentiate the initial generated content based on the evidence gap report means that the generation end parses the evidence gap report, identifies the location of the marked uncovered atomic facts in the initial generated content, and performs a deletion operation for each marked fact—that is, directly removing the fact statement from the text—or performs a transformation operation, that is, modifying a deterministic statement into a speculative statement.

[0071] A32: Re-execute atomic decomposition and cross-validation on the rewritten content until the evidence coverage is greater than or equal to the preset coverage threshold, and then determine that the validation is successful. Understandably, re-performing atomization and cross-validation on the rewritten content means using the rewritten content obtained from the differential adjustments as new input, repeatedly executing the atomization process starting from step S20 and the cross-validation process in step S30, and recalculating the evidence coverage rate of the rewritten content. In one feasible implementation, the rewritten content is input into the atomization module to obtain a new set of atomic facts, and then the set is subjected to internal knowledge base semantic matching and online retrieval cross-validation to obtain a new evidence coverage rate. Further, judging that the verification is passed until the evidence coverage rate is greater than or equal to the preset coverage rate threshold means comparing the new evidence coverage rate with the preset threshold. If the new evidence coverage rate is greater than or equal to the preset threshold, the rewritten content is determined to meet the evidence support requirements, and the verification is passed; if the new evidence coverage rate is still lower than the preset threshold, an evidence gap report is generated again and differential rewriting is performed, forming an iterative loop until the verification is passed.

[0072] A33: Use the verified rewritten content as the target generated content.

[0073] Understandably, using verified rewritten content as target generated content means that when the evidence coverage of the rewritten content reaches or exceeds a preset coverage threshold after reverification, the rewritten content is determined as the final output. In one feasible implementation, after verification, the rewritten content is marked as target generated content. This content has excluded or transformed all uncovered atomic facts, possesses high evidence coverage and high credibility, and can be directly output to the user or enter subsequent application processes, completing the entire processing chain from initial generation to credible output.

[0074] This embodiment provides a method for eliminating large-scale model illusions. By atomizing facts, it intercepts non-existent facts in the early stages of reasoning, preventing the accumulation of errors. It establishes a "coverage rate" metric to clearly define the proportion of verifiable content in the generated content. It solves the problem that internal knowledge bases alone cannot falsify information by linking internal and external sources (internal library + network) for verification, improving the authority and real-time nature of verification. It feeds back information on insufficient coverage to the generation end, realizing the transformation from "blind generation" to "generation based on evidence gaps." Each output fact undergoes explicit binary (internal and external) verification, greatly reducing the occurrence of reasoning illusions. By accurately locating "which facts lack evidence," the model has strong targeting during secondary generation, avoiding the resource waste caused by full rewriting. The system can proactively identify which knowledge is neither local nor online, thereby guiding the model to express "I don't know" truthfully, rather than fabricating answers.

[0075] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S30 includes steps S301 to S304: Step S301: Convert each atomic fact in the atomic fact set into an atomic fact vector, and calculate the semantic similarity between each atomic fact vector and the evidence fragments in the internal knowledge base to obtain the internal matching score; It should be noted that atomic fact vectors refer to the numerical representation of atomic fact text in a high-dimensional vector space through a word vector model. Atomic fact vectors convert discrete text symbols into continuous vector forms, enabling semantic similarity calculations to be performed in the vector space. In this embodiment, atomic fact vectors are the mathematical expression of atomic facts, with each atomic fact corresponding to a fixed-dimensional vector, facilitating numerical comparison with evidence fragments.

[0076] An internal knowledge base refers to a database storing knowledge of a private domain. This database contains structured evidence fragments used for semantic matching with atomic facts. The internal knowledge base supports efficient vector retrieval through a pre-built index structure and is the primary knowledge source for verifying the authenticity of atomic facts. In this embodiment, the internal knowledge base can be a vector database or a graph database, and the evidence fragments it stores cover authoritative knowledge in a specific domain.

[0077] An evidence fragment refers to a reference text unit stored in the internal knowledge base used to verify atomic facts. Evidence fragments are the basic building blocks of the internal knowledge base, and each fragment contains a complete factual statement or knowledge entry. In this embodiment, the evidence fragment is pre-stored text data, and its similarity to the atomic fact vector is calculated to determine whether it supports the atomic fact.

[0078] Semantic similarity is a numerical metric that measures the degree of semantic similarity between two text units. It is implemented using a distance metric in vector space and can assess the semantic match between atomic facts and pieces of evidence. In this embodiment, cosine similarity is used for semantic similarity calculation, with values ​​ranging from 0 to 1; a higher value indicates a closer semantic similarity.

[0079] The internal matching score refers to the highest similarity value obtained after semantic similarity calculation between an atomic fact and evidence fragments in the internal knowledge base. The internal matching score quantifies the degree of support for an atomic fact in the internal knowledge base and serves as the basis for determining whether an atomic fact is covered by the internal knowledge base. In this embodiment, the internal matching score is obtained by comparing the similarity values ​​of the atomic fact vector with multiple internal evidence vectors, and taking the maximum value as the final score.

[0080] Understandably, converting each atomic fact in the atomic fact set into atomic fact vectors means traversing the atomic fact set and vectorizing each atomic fact using a pre-trained word vector model to obtain a set of atomic fact vectors. Further, calculating the semantic similarity between each atomic fact vector and evidence fragments in the internal knowledge base means retrieving relevant evidence fragments from the internal knowledge base for each atomic fact vector, converting the evidence fragments into vectors, and calculating their cosine similarity with the atomic fact vectors to obtain a set of similarity scores. Going further, obtaining the internal matching score means selecting the maximum value from the similarity score set as the internal matching score for that atomic fact, ultimately obtaining the internal matching score for each atomic fact.

[0081] In one feasible implementation, step S301 includes steps A41 to A43: A41: Each atomic fact is mapped to an atomic fact vector in a high-dimensional vector space through a pre-trained word vector model; It should be noted that a word vector model refers to a model trained using neural network technology that maps words or text into vector representations. This model learns distributed representations of words through training on large-scale corpora. Word vector models can map semantically similar text to similar positions in the vector space, forming the basis for semantic similarity calculation. In this embodiment, the word vector model uses pre-trained models such as BERT or BGE, converting text into vectors through fine-tuning or direct calls.

[0082] A high-dimensional vector space refers to the mathematical space in which the output vectors of a word vector model reside. This space typically has hundreds to thousands of dimensions, each representing a certain abstract semantic feature of the text. The high-dimensional vector space provides a numerical representation of text semantics, enabling semantic computation through vector operations. In this embodiment, the dimensions of the high-dimensional vector space are determined by the word vector model architecture, for example, a 1024-dimensional space.

[0083] Understandably, mapping each atomic fact to a high-dimensional vector space using a pre-trained word vector model means that for each atomic fact text, input into the pre-trained word vector model, the model compresses the semantic information of the text into a fixed-length vector representation through an encoding layer, and outputs the coordinate position of the atomic fact in the high-dimensional vector space, thus obtaining the atomic fact vector. This process realizes the transformation from text to numerical vectors, providing a mathematical foundation for subsequent similarity calculations.

[0084] A42: Based on atomic fact vectors, a predetermined number of target evidence fragments are retrieved from the internal knowledge base to obtain a candidate evidence set; It should be noted that the candidate evidence set refers to the collection of evidence fragments retrieved from the internal knowledge base that are semantically closest to the atomic fact vector. The candidate evidence set is an aggregation of potential supporting evidence related to the fact to be verified in the internal knowledge base, and each candidate evidence fragment may support the atomic fact. In this embodiment, the candidate evidence set is obtained through an approximate nearest neighbor retrieval algorithm, and the size of the set is controlled by a preset quantity parameter.

[0085] Understandably, retrieving a predetermined number of target evidence fragments from an internal knowledge base based on atomic fact vectors refers to using the atomic fact vectors as query vectors and performing a similarity search within the vector index of the internal knowledge base to find the predetermined number of evidence fragments closest to the query vector. In one feasible implementation, the predetermined number is a Top-K value, such as retrieving the 10 closest evidence fragments and aggregating these fragments to form a candidate evidence set, providing a candidate set for subsequent fine-grained matching.

[0086] A43: Convert each candidate evidence fragment in the candidate evidence set into an internal evidence vector; Understandably, converting each candidate evidence fragment in the candidate evidence set into an internal evidence vector means traversing the candidate evidence set, performing the same vectorization operation as step A41 on each candidate evidence fragment text in the set, mapping each text fragment to a vector representation through a pre-trained word vector model, and obtaining a set of internal evidence vectors that correspond one-to-one with the candidate evidence fragments, thus preparing data for subsequent similarity calculations.

[0087] A44: Calculate the similarity value between the atomic fact vector and each internal evidence vector using the cosine similarity function, and determine the internal matching score based on the similarity value.

[0088] It should be noted that the cosine similarity function is a mathematical function that calculates the cosine of the angle between two vectors. This function obtains the similarity value by dividing the vector dot product by the product of the vector magnitudes. The cosine similarity function is a standard method for measuring the semantic similarity of text in a vector space, with an output value range from -1 to 1; a larger value indicates a closer semantic similarity. In this embodiment, the cosine similarity function is used as a specific implementation of semantic similarity to quantify the degree of matching between atomic facts and evidence fragments.

[0089] The similarity score refers to the specific similarity value between an atomic fact vector calculated using the cosine similarity function and a single internal evidence vector. The similarity score is a quantitative indicator measuring the semantic closeness of a single piece of evidence to the atomic fact; each internal evidence vector corresponds to a similarity score. In this embodiment, the similarity score is a floating-point number between 0 and 1; a higher score indicates that the evidence piece more strongly supports the atomic fact.

[0090] Understandably, calculating the similarity between the atomic fact vector and each internal evidence vector using the cosine similarity function means that for each internal evidence vector, the cosine similarity function is called to calculate its similarity with the atomic fact vector, resulting in a set of similarity values. Furthermore, determining the internal matching score based on the similarity values ​​means selecting the maximum value from the calculated set of similarity values ​​and using this maximum value as the internal matching score of the atomic fact, representing the highest level of support from the internal knowledge base for that atomic fact.

[0091] In practical implementation, after obtaining the atomic facts, they need to be compared with the retrieved "reference evidence fragments". This invention uses the "vector semantic similarity" method to calculate the score.

[0092] Vector semantic similarity will convert atomic facts With retrieved evidence fragments Each word is mapped to a high-dimensional vector space using a pre-trained word embedding model (such as BERT or BGE). Semantic proximity is then calculated using cosine similarity.

[0093] Wherein, S in the formula im This represents the final similarity score, ranging from -1 to +1; a larger value indicates greater semantic similarity. i e represents the i-th atomic fact to be verified in the set of atomic facts. j V represents the j-th piece of evidence retrieved from a knowledge base or the web; fi and V ej They represent f respectively i With e j A high-dimensional vector is obtained through mapping using a pre-trained word vector model. During computation, V is first calculated. fi With V ej The dot product of vectors is the sum of the product of corresponding dimensions. Then, the magnitudes of the two vectors are calculated separately, which are the square roots of the sum of the squares of each dimension. Finally, the dot product result is divided by the product of the two magnitudes to achieve normalization, resulting in a similarity score that eliminates the influence of vector length and only reflects the directional relationship. This score is used to determine whether the atomic fact is supported by evidence.

[0094] Step S302: When the internal matching score is lower than the preset semantic matching score threshold, the atomic facts are sent to the network retrieval interface to obtain real-time retrieval fragments through the network retrieval interface and convert the real-time retrieval fragments into external evidence vectors. It should be noted that the preset semantic matching score threshold is a pre-set critical value used to determine whether the internal matching score is sufficient. The preset semantic matching score threshold is the decision criterion for initiating online retrieval; when the internal matching score is lower than this threshold, it indicates that the internal knowledge base cannot provide sufficient evidence. In this embodiment, the preset semantic matching score threshold can be set to 0.75; if it falls below this value, the online retrieval process is triggered.

[0095] The online retrieval interface refers to the application programming interface (API) that connects to external search engines. This interface supports sending retrieval queries and receiving real-time network information. As an access channel to external knowledge sources, the online retrieval interface expands the boundaries of verified knowledge and can obtain real-time information not covered by the internal knowledge base. In this embodiment, the online retrieval interface is implemented by calling a general-purpose search engine or a vertical domain search engine, providing dynamic knowledge acquisition capabilities.

[0096] Real-time retrieval fragments refer to the latest online text fragments related to atomic facts, retrieved from search engines via an online retrieval interface. These fragments represent real-time information on the internet and are used to verify the timeliness and widespread availability of atomic facts. In this embodiment, the real-time retrieval fragment is a summary or main text fragment from the search engine's returned results, serving as an external knowledge source for verification.

[0097] Understandably, sending the atomic fact to the online retrieval interface when the internal matching score is lower than the preset semantic matching score threshold means comparing the internal matching score with the preset threshold. If the internal matching score is less than the threshold, it indicates that the internal knowledge base cannot provide sufficient evidence. In this case, the online retrieval mechanism is triggered, and the atomic fact is sent to the online retrieval interface as a retrieval source. Furthermore, obtaining real-time retrieval fragments through the online retrieval interface means that after receiving the atomic fact, the online retrieval interface executes an information retrieval process, obtains the latest online information related to the atomic fact from an external search engine, and extracts text fragments as real-time retrieval fragments. Further still, converting the real-time retrieval fragments into external evidence vectors means performing the same vectorization operation as step A41 on the obtained real-time retrieval fragments. A pre-trained word vector model maps the text fragments into vector representations in a high-dimensional vector space, obtaining external evidence vectors to prepare data for subsequent external matching calculations.

[0098] In one feasible implementation, step S302 includes steps A51 to A54: A51: Extract search keywords from atomic facts and construct search query statements; It should be noted that search keywords refer to the core words or phrases extracted from the atomic fact text used to construct search queries. Search keywords summarize the core semantics of the atomic facts and are the foundation of information retrieval in online searches. In this embodiment, search keywords typically include the subject and predicate components of the atomic facts.

[0099] A search query is a query string constructed based on search keywords for use with a search engine. It is a standardized request format for interaction between the online search interface and the search engine, containing search keywords and possible search operators. In this embodiment, the search query can be a simple combination of keywords or a structured query containing logical operators, used to accurately obtain relevant evidence.

[0100] Understandably, extracting search keywords from atomic facts refers to analyzing the triple structure of atomic facts and extracting words with search value from the subject, predicate, or object components to form a keyword list. Furthermore, constructing a search query involves combining the extracted search keywords into a query string according to the search engine's query syntax rules. Operators such as quotation marks, AND, and OR can be added to optimize search precision, forming a complete search query statement ready to be sent to the search engine's application programming interface (API).

[0101] A52: Send the search query to the search engine application programming interface; It should be noted that a search engine application programming interface (API) refers to the programmatic access interface provided by a search engine. This interface receives search queries and returns search results data. The search engine API is the foundation for implementing online search functionality and supports automated information retrieval. In this embodiment, the search engine API can be a general search engine API or a vertical domain search API, providing keyword search and result return functions.

[0102] As you can understand, sending a search query to a search engine application interface (API) means sending the constructed search query as a parameter to the API via an HTTP request or SDK call. This triggers the search engine to execute keyword matching and ranking algorithms, retrieve relevant content from a vast amount of online resources, and then wait for the API to return a list of search results.

[0103] A53: Receives the list of search results returned by the search engine application interface, extracts text fragments from the list of search results, and obtains real-time search fragments; Understandably, receiving the search results list returned by the search engine application programming interface (API) means obtaining the search engine's processing response to the search query, parsing the data structure returned by the API, and obtaining a list containing multiple search results. Furthermore, extracting text fragments from the search results list means extracting text content such as summaries or main text fragments from each entry in the search results list, aggregating these text fragments to form real-time search fragments, which serve as external knowledge sources for subsequent verification.

[0104] A54: Convert real-time retrieved fragments into external evidence vectors using a pre-trained word vector model.

[0105] Understandably, converting real-time retrieval fragments into external evidence vectors using a pre-trained word vector model means performing the same vectorization operation as step A41 on the extracted real-time retrieval fragment text. The pre-trained word vector model maps each real-time retrieval fragment into a vector representation in a high-dimensional vector space, resulting in a set of external evidence vectors. These vectors are in the same vector space as the atomic fact vectors, supporting subsequent similarity calculations.

[0106] Step S303: Calculate the cosine similarity between the atomic fact vector and the external evidence vector to obtain the external matching score; It should be noted that the external matching score refers to the highest similarity value obtained after semantic similarity calculation between the atomic fact and the real-time retrieval fragment obtained from the network retrieval. The external matching score quantifies the degree of support for the atomic fact in external network information and serves as the basis for determining whether the atomic fact is covered by external knowledge sources. In this embodiment, the external matching score is obtained by calculating the cosine similarity between the atomic fact vector and multiple external evidence vectors and taking the maximum value, reflecting the highest degree of support for the atomic fact from external knowledge sources.

[0107] Understandably, calculating the cosine similarity between the atomic fact vector and the external evidence vector means that for each external evidence vector, the cosine similarity function is called to calculate its similarity to the atomic fact vector, resulting in a set of similarity values. Furthermore, obtaining the external matching score means selecting the maximum value from the calculated set of similarity values ​​and using this maximum value as the external matching score of the atomic fact, representing the highest level of support from the online retrieval for that atomic fact.

[0108] Step S304: Based on the internal matching score and the external matching score, determine the coverage status of each atomic fact, and calculate the proportion of covered atomic facts in the atomic fact set to obtain the evidence coverage rate.

[0109] It should be noted that coverage status refers to the evidence support status of atomic facts after dual-source verification through internal knowledge base and online retrieval. Coverage status is a binary label; when the internal matching score or external matching score exceeds a preset threshold, it is marked as covered; otherwise, it is marked as uncovered. In this embodiment, coverage status is the basis for determining whether atomic facts are included in coverage statistics, directly affecting the calculation result of evidence coverage.

[0110] Understandably, determining the coverage status of each atomic fact based on internal and external matching scores means comparing each atomic fact's internal and external matching scores with a preset semantic matching score threshold. If either the internal or external matching score is greater than or equal to the threshold, the atomic fact is determined to be covered; otherwise, it is determined to be uncovered. Further, calculating the proportion of covered atomic facts in the atomic fact set means traversing the atomic fact set, counting the number of covered atomic facts, and dividing this number by the total number of atomic facts to obtain the proportion of covered atomic facts. Further still, obtaining the evidence coverage rate means using the calculated proportion of covered atomic facts as the final evidence coverage rate. This coverage rate is the decision-making basis for triggering differentiated rewriting; when the coverage rate is lower than a preset coverage rate threshold, the rewriting process is initiated. In one feasible implementation, for a set containing 10 atomic facts, if 7 are marked as covered, the evidence coverage rate is 0.7. This value will be compared with a preset threshold to determine whether to proceed to step S40, which involves generating an evidence gap report.

[0111] This embodiment provides a method for eliminating illusions in large models. By atomizing the initially generated content, unstructured text is transformed into the smallest verifiable unit, enabling independent verification of each statement. This intercepts non-existent facts in the early stages of reasoning, preventing the cascading propagation of illusions. A dual-source mechanism, consisting of semantic matching from an internal knowledge base and cross-validation via online retrieval, quantitatively evaluates each atomic fact in the atomic fact set based on vector semantic similarity, ensuring authoritative and real-time verification. The proportion of covered atomic facts is statistically analyzed using an evidence coverage index, making the evidentiary support level of the generated content explicit, allowing the system to perceive knowledge boundaries and identify blind spots. When the coverage rate falls below a preset threshold, an evidence gap report is generated and fed back to the generation end, driving it to delete unsubstantiated facts or transform them into speculative statements. Iterative verification continues until the coverage rate reaches the target, achieving a shift from blind generation to precise completion based on evidence gaps. The final output is target generated content that has undergone explicit dual-source verification, significantly reducing computational redundancy and improving content credibility.

[0112] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the large-scale illusion elimination method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0113] This application also provides a large-scale illusion elimination device; please refer to [reference needed]. Figure 3 The large-scale illusion elimination device includes: Content acquisition module 10 is used to acquire the initial generated content; The atomization decomposition module 20 is used to atomize the initially generated content to obtain a set of atomic facts; The cross-validation module 30 is used to perform internal knowledge base semantic matching and online retrieval cross-validation on each atomic fact in the atomic fact set to obtain the evidence coverage. The differential rewriting module 40 is used to generate an evidence gap report when the evidence coverage rate is lower than a preset coverage rate threshold; and to perform differential rewriting on the initially generated content based on the evidence gap report to obtain the target generated content.

[0114] The large model illusion elimination device provided in this application, employing the large model illusion elimination method described in the above embodiments, can solve the technical problem of how to block the illusion cascade phenomenon in the generated content of large models through atomic verification and iterative feedback mechanisms. Compared with the prior art, the beneficial effects of the large model illusion elimination device provided in this application are the same as those of the large model illusion elimination method provided in the above embodiments, and other technical features in the large model illusion elimination device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0115] In one embodiment, the atomization decomposition module 20 is also used to perform dependency parsing on the initially generated content to obtain sentence segmentation results and part-of-speech tagging results; Based on the coordinating conjunctions and clause structures in the sentence segmentation results, the sentence segmentation results are broken down into propositional units; By dissolving the pronouns in the propositional unit and restoring the pronouns to entity nouns, a self-explanatory proposition is obtained. By standardizing self-explanatory propositions using triples, we obtain a set of atomic facts.

[0116] In one embodiment, the atomization decomposition module 20 is also used to perform sentence boundary detection on the initially generated text to obtain sentence segmentation results; Each word in the sentence segmentation results is tagged with its part of speech to obtain the part-of-speech tagging results. Based on the part-of-speech tagging results, the subject, predicate and object components are identified to construct the core logical framework.

[0117] In one embodiment, the cross-validation module 30 is further configured to convert each atomic fact in the atomic fact set into an atomic fact vector, and to perform semantic similarity calculation between each atomic fact vector and the evidence fragment in the internal knowledge base to obtain an internal matching score; When the internal matching score is lower than the preset semantic matching score threshold, the atomic facts are sent to the network retrieval interface to obtain real-time retrieval fragments through the network retrieval interface and convert the real-time retrieval fragments into external evidence vectors. Calculate the cosine similarity between the atomic fact vector and the external evidence vector to obtain the external matching score; Based on the internal matching score and the external matching score, the coverage status of each atomic fact is determined, and the proportion of covered atomic facts in the atomic fact set is calculated to obtain the evidence coverage rate.

[0118] In one embodiment, the cross-validation module 30 is further configured to map each atomic fact to an atomic fact vector in a high-dimensional vector space using a pre-trained word vector model; Based on atomic fact vectors, a predetermined number of target evidence fragments are retrieved from the internal knowledge base to obtain a set of candidate evidence. Convert each candidate evidence fragment in the candidate evidence set into an internal evidence vector; The similarity scores between the atomic fact vector and each internal evidence vector are calculated using the cosine similarity function, and the internal matching scores are determined based on these similarity scores.

[0119] In one embodiment, the cross-validation module 30 is further configured to extract search keywords from atomic facts and construct a search query statement; Send the search query to the search engine application programming interface; Receive the list of search results returned by the search engine application interface, and extract text fragments from the list of search results to obtain real-time search fragments; Real-time retrieved fragments are converted into external evidence vectors using a pre-trained word vector model.

[0120] In one embodiment, the differential rewriting module 40 is further configured to identify uncovered target atomic facts in the atomic fact set and generate an evidence gap report based on the uncovered target atomic facts; The evidence gap report is sent to the generation end so that the generation end can make differentiated adjustments to the initial generated content based on the evidence gap report to obtain rewritten content. The differentiated adjustments include deleting uncovered atomic facts or converting uncovered atomic facts into speculative statements. The rewritten content is re-executed with atomic decomposition and cross-validation until the evidence coverage is greater than or equal to the preset coverage threshold, at which point the validation is considered successful. The rewritten content that passes the verification will be used as the target content for generation.

[0121] This application provides a large model illusion removal device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the large model illusion removal method in Embodiment 1 above.

[0122] The following is for reference. Figure 4 The diagram illustrates a structural schematic suitable for implementing a large-scale illusion cancellation device according to embodiments of this application. The large-scale illusion cancellation device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The large-scale illusion removal device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0123] like Figure 4As shown, the large model illusion removal device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the large model illusion removal device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the large model illusion cancellation device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show large model illusion cancellation devices with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0124] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0125] The large model illusion removal device provided in this application, employing the large model illusion removal method described in the above embodiments, can solve the technical problem of how to block the illusion cascade phenomenon in the generated content of large models through atomic verification and iterative feedback mechanisms. Compared with the prior art, the beneficial effects of the large model illusion removal device provided in this application are the same as those of the large model illusion removal method provided in the above embodiments, and other technical features of this large model illusion removal device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0126] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0127] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0128] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the large model illusion elimination method described in the above embodiments.

[0129] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0130] The aforementioned computer-readable storage medium may be included in the large model illusion elimination device; or it may exist independently and not assembled into the large model illusion elimination device.

[0131] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the large-model illusion elimination device, the large-model illusion elimination device: responds to an opening trigger signal, parses the opening trigger signal to obtain an opening request; acquires initial generated content; atomically decomposes the initial generated content to obtain a set of atomic facts; performs internal knowledge base semantic matching and online retrieval cross-validation on each atomic fact in the set of atomic facts to obtain evidence coverage; when the evidence coverage is lower than a preset coverage threshold, generates an evidence gap report; and rewrites the initial generated content based on the evidence gap report to obtain target generated content.

[0132] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0134] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0135] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described large model illusion elimination method. This solves the technical problem of how to block the illusion cascade phenomenon in the generated large model content through atomic verification and iterative feedback mechanisms. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the large model illusion elimination method provided in the above embodiments, and will not be repeated here.

[0136] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the large model illusion elimination method described above.

[0137] The computer program product provided in this application can solve the technical problem of how to block the hallucination cascade phenomenon in the generated content of large models through atomic verification and iterative feedback mechanisms. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the large model hallucination elimination method provided in the above embodiments, and will not be repeated here.

[0138] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for eliminating large-scale hallucinations, characterized in that, The method includes: Get the initial generated content; The initially generated content is atomized and decomposed to obtain a set of atomic facts; The evidence coverage rate is obtained by performing semantic matching with an internal knowledge base and cross-validation through online retrieval on each atomic fact in the atomic fact set. When the evidence coverage rate is lower than a preset coverage threshold, an evidence gap report is generated. Based on the evidence gap report, the initial generated content is rewritten differentially to obtain the target generated content.

2. The method as described in claim 1, characterized in that, The step of atomically decomposing the initially generated content to obtain a set of atomic facts includes: Dependency parsing is performed on the initially generated content to obtain sentence segmentation results and part-of-speech tagging results; Based on the coordinating conjunctions and clause structures in the sentence segmentation results, the sentence segmentation results are decomposed into propositional units; By performing referential resolution on the propositional unit and restoring the pronoun to the entity noun, a self-explanatory proposition is obtained. The self-explanatory propositions are standardized using triples to obtain a set of atomic facts.

3. The method as described in claim 2, characterized in that, The step of performing dependency parsing on the initially generated content to obtain sentence segmentation results and part-of-speech tagging results includes: Sentence boundary detection is performed on the initially generated text to obtain sentence segmentation results; Each word in the sentence segmentation result is tagged with part of speech to obtain part-of-speech tagging results. Subject, predicate and object components are identified based on the part-of-speech tagging results to construct the core logical framework.

4. The method as described in claim 1, characterized in that, The step of performing internal knowledge base semantic matching and online retrieval cross-validation on each atomic fact in the atomic fact set to obtain the evidence coverage includes: Each atomic fact in the atomic fact set is converted into an atomic fact vector, and the semantic similarity of each atomic fact vector with the evidence fragments in the internal knowledge base is calculated to obtain the internal matching score. When the internal matching score is lower than the preset semantic matching score threshold, the atomic facts are sent to the network retrieval interface to obtain real-time retrieval fragments through the network retrieval interface, and the real-time retrieval fragments are converted into external evidence vectors. Calculate the cosine similarity between the atomic fact vector and the external evidence vector to obtain the external matching score; Based on the internal matching score and the external matching score, the coverage status of each atomic fact is determined, and the proportion of covered atomic facts in the atomic fact set is calculated to obtain the evidence coverage rate.

5. The method as described in claim 4, characterized in that, The step of converting each atomic fact in the atomic fact set into an atomic fact vector, and calculating the semantic similarity between each atomic fact vector and the evidence fragments in the internal knowledge base to obtain an internal matching score includes: Each atomic fact is mapped to an atomic fact vector in a high-dimensional vector space through a pre-trained word vector model; Based on the atomic fact vector, a preset number of target evidence fragments are retrieved from the internal knowledge base to obtain a candidate evidence set; Each candidate evidence fragment in the candidate evidence set is converted into an internal evidence vector; The similarity scores between the atomic fact vector and each internal evidence vector are calculated using the cosine similarity function, and the internal matching scores are determined based on these similarity scores.

6. The method as described in claim 4, characterized in that, The step of obtaining real-time search fragments through the network search interface and converting the real-time search fragments into external evidence vectors includes: Extract search keywords from the atomic facts and construct a search query statement; Send the search query to the search engine application programming interface; Receive the list of search results returned by the search engine application interface, and extract text fragments from the list of search results to obtain real-time search fragments; The real-time retrieval fragments are converted into external evidence vectors using a pre-trained word vector model.

7. The method as described in claim 1, characterized in that, The step of generating an evidence gap report when the evidence coverage rate is lower than a preset coverage threshold includes: Identify uncovered target atomic facts in the set of atomic facts, and generate an evidence gap report based on the uncovered target atomic facts; The step of rewriting the initially generated content based on the evidence gap report to obtain the target generated content includes: The evidence gap report is sent to the generation end so that the generation end can make differentiated adjustments to the initial generated content based on the evidence gap report to obtain rewritten content. The differentiated adjustments include deleting uncovered atomic facts or converting uncovered atomic facts into speculative statements. The rewritten content is re-executed with atomic decomposition and cross-validation until the evidence coverage is greater than or equal to the preset coverage threshold, at which point the verification is deemed successful. The rewritten content that passes verification will be used as the target generated content.

8. A large-scale illusion elimination device, characterized in that, The device includes: The content acquisition module is used to acquire the initial generated content; An atomization decomposition module is used to atomize the initially generated content to obtain a set of atomic facts; The cross-validation module is used to perform internal knowledge base semantic matching and online retrieval cross-validation on each atomic fact in the atomic fact set to obtain the evidence coverage rate. The differential rewriting module is used to generate an evidence gap report when the evidence coverage rate is lower than a preset coverage threshold; and to perform differential rewriting on the initial generated content based on the evidence gap report to obtain the target generated content.

9. A large-scale illusion elimination device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the large model illusion elimination method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the large model illusion elimination method as described in any one of claims 1 to 7.

Images