A model hallucination identification method and identification system based on long-term memory enhancement

Abstract

The application discloses a model hallucination recognition method and system based on long-term memory enhancement. The method includes the following steps: decomposing the target model to be detected into multiple atomic statements; matching the multiple atomic statements with historical hallucination records to obtain the matching degrees of the atomic statements, so as to divide the multiple atomic statements into a first set and a second set; performing knowledge retrieval on the atomic statements in the second set to obtain corresponding support evidence sets; calculating the hallucination probabilities of the atomic statements in the second set based on the matching degrees of the atomic statements in the second set, the support evidence sets corresponding to the atomic statements in the second set and the model generation confidence degrees of the atomic statements in the second set; and generating the hallucination recognition result of the target model according to the matching degrees of the atomic statements in the first set and the hallucination probabilities of the atomic statements in the second set. The method can realize statement-level matching and hierarchical retrieval, reuse historical hallucination memory and significantly improve the recognition efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a model illusion recognition method and system based on long-term memory enhancement. Background Technology

[0002] Large Language Models (LLMs), with their powerful semantic understanding and text generation capabilities, are being widely applied in text generation, intelligent question answering, and information summarization, becoming a core support for the industrialization of artificial intelligence technology. As their application deepens across various fields, the factual accuracy of the output content has become a key consideration. However, LLMs commonly suffer from the "illusion" problem when generating content, where the model confidently outputs information that is inconsistent with reality, unverifiable, or even fabricated. Therefore, efficient and accurate fact verification and illusion identification of LLM output content have become crucial technical requirements for the safe deployment of large language models.

[0003] Existing hallucination recognition solutions perform independent retrieval in each dialogue, which leads to the system's inability to recognize hallucinations on the same topic when they recur. This requires repeatedly triggering a full external retrieval, resulting in low efficiency and high latency. Furthermore, relying solely on a general factual knowledge base or temporal knowledge graph to store correct knowledge makes it difficult to continuously improve the accuracy and robustness of hallucination recognition, thus failing to meet the dual requirements of accuracy and efficiency for hallucination recognition in high-confidence scenarios. Summary of the Invention

[0004] The main objective of this disclosure is to propose a model hallucination recognition method and system based on long-term memory enhancement.

[0005] A first aspect of this application provides a model-based hallucination recognition method based on long-term memory enhancement, the method comprising: The target model's text to be detected is decomposed into multiple atomic statements; the atomic statements are used to characterize the smallest verifiable unit in the text to be detected. The multiple atomic statements are matched with historical hallucination records in a pre-built hallucination memory bank to obtain the matching degree of each atomic statement; Based on the matching degree of each atomic declaration, the plurality of atomic declarations are divided into a first set and a second set; the first set is used to characterize atomic declarations among the plurality of atomic declarations whose matching degree is greater than a preset matching degree threshold; the second set is used to characterize atomic declarations among the plurality of atomic declarations whose matching degree is less than or equal to the preset matching degree threshold. For each atomic statement in the second set, perform knowledge retrieval to obtain the supporting evidence set corresponding to each atomic statement in the second set; Based on the matching degree of each atomic statement in the second set, the supporting evidence set corresponding to each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set, the illusion probability of each atomic statement in the second set is calculated; the model generation confidence of each atomic statement in the second set is calculated through the token probability distribution when the target model generates each atomic statement in the second set. Based on the matching degree of each atomic statement in the first set and the illusion probability of each atomic statement in the second set, the illusion recognition result of the target model is generated.

[0006] The first aspect of this application provides a model hallucination recognition method based on long-term memory enhancement. This method involves decomposing the target model's text into multiple atomic statements; matching these atomic statements with historical hallucination records to obtain the matching degree of each atomic statement, thereby dividing the multiple atomic statements into a first set and a second set; performing knowledge retrieval on each atomic statement in the second set to obtain the corresponding supporting evidence set; calculating the hallucination probability of each atomic statement in the second set based on the matching degree of each atomic statement in the second set, the supporting evidence set corresponding to each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set; and generating the hallucination recognition result of the target model based on the matching degree of each atomic statement in the first set and the hallucination probability of each atomic statement in the second set. This method can significantly improve recognition efficiency and accuracy by reusing historical hallucination memories through statement-level matching and hierarchical retrieval.

[0007] To achieve the above objectives, a second aspect of this application provides a model hallucination recognition system based on long-term memory enhancement, the system comprising: The decomposition module is used to decompose the text to be detected in the target model into multiple atomic statements; the atomic statement is used to characterize the smallest verifiable unit in the text to be detected. A matching module is used to match the plurality of atomic statements with historical hallucination records in a pre-built hallucination memory bank to obtain the matching degree of each atomic statement; A partitioning module is used to partition the plurality of atomic declarations into a first set and a second set based on the matching degree of each atomic declaration; the first set is used to characterize atomic declarations among the plurality of atomic declarations whose matching degree is greater than a preset matching degree threshold; the second set is used to characterize atomic declarations among the plurality of atomic declarations whose matching degree is less than or equal to the preset matching degree threshold; The execution module is used to perform knowledge retrieval on each atomic statement in the second set to obtain the supporting evidence set corresponding to each atomic statement in the second set; The calculation module is used to calculate the illusion probability of each atomic statement in the second set based on the matching degree of each atomic statement in the second set, the supporting evidence set corresponding to each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set; the model generation confidence of each atomic statement in the second set is calculated by the token probability distribution when the target model generates each atomic statement in the second set; The generation module is used to generate the hallucination recognition result of the target model based on the matching degree of each atomic declaration in the first set and the hallucination probability of each atomic declaration in the second set.

[0008] To achieve the above objectives, a third aspect of this application provides an electronic device, including: at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform the above-described model illusion recognition method based on long-term memory enhancement.

[0009] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described model illusion recognition method based on long-term memory enhancement.

[0010] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0011] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating a model-based hallucination recognition method based on long-term memory enhancement provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a model hallucination recognition system based on long-term memory enhancement provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0013] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0014] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0015] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0016] Large Language Models (LLMs) excel in tasks such as text generation, question answering, and summarization, but they suffer from the "illusion" problem—that is, the model confidently outputs information that is inconsistent with reality, unverifiable, or even fabricated. Existing research has proven from a computational theory perspective that for any LLM based on existing computational paradigms, illusion is theoretically unavoidable: when the "absolute truth" function... With computable model functions When inconsistencies exist, it can be determined that a hallucination has occurred.

[0017] Existing hallucination mitigation solutions, including Retrieval-augmented Generation (RAG) and model fine-tuning, all have the following shortcomings: The single-turn limitation of RAG: Traditional RAG performs independent retrieval in each dialogue, lacks the use of historical interaction records, and cannot identify repetitive hallucination patterns across dialogues and sessions. Lack of hallucination memory: When hallucinations on the same topic recur at different times and with different users, the system cannot accumulate the characteristics of such "known hallucinations", leading to repeated mistakes; Coarse-grained verification: Existing solutions typically perform overall retrieval and verification at the sentence or paragraph level, which cannot achieve fine-grained illusion positioning at the claim-level. The feedback loop is missing: the user's error correction feedback on the output content is not stored in a structured manner, which prevents the formation of a closed loop for the continuous evolution of hallucination recognition capabilities.

[0018] Based on this, embodiments of this application provide a model hallucination recognition method and system based on long-term memory enhancement, which aims to significantly improve recognition efficiency and accuracy by reusing historical hallucination memories through declaration-level matching and hierarchical retrieval.

[0019] The model hallucination recognition method, system, electronic device and medium based on long-term memory enhancement provided in this application are specifically described through the following embodiments. First, the model hallucination recognition method based on long-term memory enhancement in this application is described.

[0020] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0021] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0022] The model illusion recognition method based on long-term memory enhancement provided in this application relates to the field of natural language processing technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the model illusion recognition method based on long-term memory enhancement, but is not limited to the above forms.

[0023] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0024] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0025] Therefore, referring to Figure 1 This application provides a model hallucination recognition method based on long-term memory enhancement. This method is applied to a central controller, which can be a server, an electronic device, or a mobile terminal, etc. There are no specific limitations here. The method includes the following steps S110 to S160.

[0026] Step S110: Decompose the text to be detected in the target model to obtain multiple atomic statements; atomic statements are used to characterize the smallest verifiable unit in the text to be detected. Step S120: Match multiple atomic declarations with historical hallucination records in a pre-built hallucination memory bank to obtain the matching degree of each atomic declaration; Step S130: Based on the matching degree of each atomic declaration, divide the multiple atomic declarations into a first set and a second set; the first set is used to characterize atomic declarations with a matching degree greater than a preset matching degree threshold; the second set is used to characterize atomic declarations with a matching degree less than or equal to the preset matching degree threshold. Step S140: Perform knowledge retrieval on each atomic statement in the second set to obtain the supporting evidence set corresponding to each atomic statement in the second set; Step S150: Based on the matching degree of each atomic statement in the second set, the supporting evidence set corresponding to each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set, calculate the illusion probability of each atomic statement in the second set; the model generation confidence of each atomic statement in the second set is calculated by the token probability distribution when the target model generates each atomic statement in the second set. Step S160: Generate the illusion recognition result of the target model based on the matching degree of each atomic statement in the first set and the illusion probability of each atomic statement in the second set.

[0027] First, let's analyze some of the terms used in this application: An atomic statement is the smallest verifiable unit obtained by decomposing the text to be detected from the output of the target model (such as a large language model) using a statement decomposition model. It is a basic semantic fragment of factual content that cannot be further divided, such as an independent subject-verb-object structure or entity relation statement. By decomposing text into atomic statements, fine-grained illusion detection far exceeding the sentence or paragraph level can be achieved, facilitating subsequent verification of the authenticity of each atomic fact.

[0028] A hallucination memory bank is a pre-built, dedicated database for persistently storing confirmed historical hallucination records. Unlike traditional general-purpose knowledge bases, this memory bank specifically stores erroneous facts rather than correct knowledge. Its core fields include hallucination identifier, claim text, claim vector, hallucination type, correction text, and occurrence count. The purpose of the hallucination memory bank is to accumulate historical hallucination experience, enabling rapid identification when the same or similar hallucinations recur, without requiring further external knowledge retrieval.

[0029] Knowledge retrieval refers to the process of obtaining supporting evidence for an atomic claim from at least two external knowledge sources in parallel. External knowledge sources include, but are not limited to, structured knowledge graphs, vector document repositories, real-time web retrieval interfaces, or relational databases. Preferably, during the retrieval process, the returned evidence fragments need to undergo validity screening, relevance calculation, authority assessment, and timeliness analysis, ultimately selecting the several pieces of evidence with the highest comprehensive scores as the supporting evidence set for the atomic claim.

[0030] Model generation confidence refers to the intrinsic confidence of the target model in the correctness of its own output when generating atomic statements. This confidence is obtained by extracting the probability distribution of each token during the generation of the atomic statement by the target model and calculating the average log probability. Model generation confidence is one of the important dimensions for calculating the probability of illusion. When the model is highly confident but the external evidence clearly contradicts it, it can be identified as a "confidence-based illusion," and this type of illusion requires special attention.

[0031] In this step, the text to be detected is first obtained from the target model (i.e., the large language model to be detected), and then preprocessed. Specifically, the text type is determined by an intent classification model. If it is factual or mixed, factual sub-texts are extracted, and then a declaration decomposition model is used to break down the factual sub-texts into multiple atomic declarations, laying the foundation for accurate illusion localization.

[0032] Furthermore, each atomic claim is semantically matched with historical hallucination records in a pre-built hallucination memory bank. Specifically, the atomic claim is first converted into a claim vector using an embedding model, and then the cosine similarity between this vector and each historical hallucination vector in the hallucination memory bank is calculated using an approximate nearest neighbor retrieval algorithm. The maximum value of this cosine similarity is taken as the matching degree of the atomic claim. The higher the matching degree, the more semantically similar the atomic claim is to a historically confirmed hallucination.

[0033] The hallucination memory bank is constructed through the following steps: Acquire initial historical data, including historical hallucination annotation data, manually confirmed hallucination cases, and user error correction data; extract the original error statement, the correct correction content corresponding to the original error statement, authoritative evidence links, and information on the technical field of the hallucination from the initial historical data; construct a hallucination memory bank based on the original error statement, the correct correction content corresponding to the original error statement, authoritative evidence links, and information on the technical field of the hallucination.

[0034] In this step, the construction of the hallucination memory bank first requires the acquisition of initial historical data, which specifically includes historical hallucination annotation data (i.e. samples that have been identified as hallucinations in past model outputs), manually confirmed hallucination cases (typical hallucination examples confirmed by experts), and user error correction data (records of user feedback and corrections to model output errors).

[0035] Furthermore, the core information required to construct the memory bank is extracted from the aforementioned initial historical data, specifically including: the original text of the error statement (i.e., the confirmed error fact text output by the model), the correct correction content corresponding to the original text of the error statement (objective facts taken from authoritative data sources such as encyclopedias, regulations, and papers), authoritative evidence links (source URLs used to support the correct correction content), and information on the technical field of the error statement (obtained by classifying the field of the error statement through a text classification model, such as medical, legal, and financial fields).

[0036] Furthermore, based on the extracted original error statements, corrected content, links to authoritative evidence, and information on the technical field, each historical hallucination record is persistently stored in the database according to a predefined storage structure (including ten fixed fields such as hallucination identifier, statement vector, hallucination type, confidence score, and frequency of occurrence). This constructs a persistent, searchable, and updatable dedicated hallucination memory bank. This memory bank is specifically used to store historically confirmed hallucination errors, distinguishing it from a general knowledge base that only stores correct knowledge. It achieves reverse knowledge construction, providing core data support for subsequent historical hallucination matching and rapid filtering.

[0037] Furthermore, based on the relationship between the matching degree of each atomic statement and a preset matching degree threshold, all atomic statements are divided into a first set and a second set. Atomic statements with a matching degree greater than the preset matching degree threshold are assigned to the first set; these statements are highly similar to historical illusions and can be directly considered as historical illusion hits. Atomic statements with a matching degree less than or equal to the preset matching degree threshold are assigned to the second set; these statements lack highly matching historical illusion records and require further verification through external retrieval. Preferably, the preset matching degree threshold can be further subdivided into a high threshold and a low threshold to achieve a three-level traffic split.

[0038] For each atomic statement included in the second set, a multi-source external knowledge retrieval is performed to obtain objective factual verification evidence. Specifically, during the retrieval, based on the domain tag of the atomic statement, at least two types of knowledge sources are selected from structured knowledge graphs, vector document libraries, real-time web retrieval interfaces, or relational databases. Retrieval requests are sent asynchronously and in parallel. Then, the returned evidence fragments are subjected to validity screening, relevance calculation, authority assessment, and timeliness scoring. Finally, the several pieces of evidence with the highest comprehensive scores are selected as the supporting evidence set for the atomic statement.

[0039] By combining information from three dimensions, the hallucination probability of each atomic statement in the second set is calculated. Preferably, the first dimension is the historical signal score determined based on the matching degree; the second dimension is the external evidence consistency score determined based on the natural language reasoning relationship between the supporting evidence set and the atomic statement; and the third dimension is the token probability distribution extracted when the target model generates the atomic statement, and the average log probability is calculated as the model's generation confidence.

[0040] Furthermore, the scores of the three dimensions are multiplied by their respective weighting coefficients and then summed to obtain the probability of hallucination, which ranges from 0 to 1. The higher the probability value, the greater the likelihood that the statement is a hallucination.

[0041] Finally, based on the matching degree of each atomic statement in the first set and the hallucination probability of each atomic statement in the second set, the hallucination recognition result of the target model is generated comprehensively. This result includes: determining the overall risk level of the text to be detected (e.g., high risk, medium risk, or low risk), and outputting the text according to the risk level. For example, for low-risk cases, the original text is output directly; for medium-risk cases, suspected hallucinations are highlighted and accompanied by notes; for high-risk cases, the hallucination parts are replaced with correct expressions and the corrected text is output, while an interpretable summary can be generated for user reference.

[0042] In some embodiments, step S110 includes the following steps S210 to S230: Step S210: Perform intent classification on the text to be detected to obtain the classification result; the classification result includes either factual text or mixed text. Step S220: If the classification result is factual text, decompose the text to be detected into multiple atomic declarations; Step S230: If the classification result is mixed text, extract factual subtext from the text to be detected, so as to decompose the factual subtext into multiple atomic statements.

[0043] In this embodiment, the text to be detected, output by the target model, is first input into a pre-trained text classification model (e.g., a fine-tuned BERT classifier) ​​to classify its intent and determine the intent type of the text to be detected. The executable classification result includes either factual text or mixed text.

[0044] If the classification result is factual text, meaning that all content of the text to be tested consists of verifiable factual statements, then the complete text to be tested is directly input into the statement decomposition model and broken down into multiple atomic statements. If the classification result is mixed text, meaning that the text to be tested contains both factual and opinion content, then factual sub-texts are first extracted from the text to be tested, and opinion content that does not need to be verified is filtered out. Then, the extracted factual sub-texts are input into the statement decomposition model and broken down into multiple atomic statements.

[0045] Therefore, by using the above-mentioned intention classification and selective extraction processing method, we can avoid invalid fact detection on opinion-based content, and only perform atomic declaration decomposition on factual content that needs to be verified, thereby improving processing efficiency and detection targeting.

[0046] In some embodiments, step S120 includes the following steps S310 to S330: Step S310: Convert multiple atomic declarations into data dimensions of a pre-built illusion memory library to obtain the semantic vector of each atomic declaration; Step S320: For the semantic vectors declared by each atom, perform an approximate nearest neighbor search in the pre-built illusion memory bank to calculate the semantic similarity between the semantic vectors declared by each atom and the semantic vectors of each historical illusion record in the pre-built illusion memory bank. Step S330: Determine the matching degree of each atomic declaration based on the semantic similarity between the semantic vector of each atomic declaration and the semantic vector of each historical hallucination record in the pre-built hallucination memory bank.

[0047] In this embodiment, each atomic declaration is first input into the same embedding model (e.g., text-embedding-ada-002 or BERT encoder) used when constructing the hallucination memory bank, and then converted into a semantic vector with the same dimensions as the historical hallucination records in the hallucination memory bank, to ensure consistency in subsequent comparisons.

[0048] Furthermore, for each semantic vector declared by an atom, an approximate nearest neighbor search is performed in the pre-built hallucination memory bank. The semantic similarity between the semantic vector and the semantic vectors corresponding to each historical hallucination record in the memory bank is calculated one by one, and cosine similarity is usually used as the metric.

[0049] Finally, based on all the calculated semantic similarity values, the maximum value is selected as the matching degree of the atomic statement. Specifically, the higher the matching degree, the closer the atomic statement is semantically to a historically confirmed hallucination record, thus providing a quantitative basis for subsequent hierarchical and segmented traffic based on matching degree.

[0050] In some embodiments, step S140 includes the following steps S410 to S440: Step S410: Obtain the retrieval results of external knowledge sources corresponding to the technical field information of each atom declaration in the second set; Step S420: Extract the set of valid evidence fragments for each atom statement from the second set of search results; Step S430: Perform a quality assessment on each valid evidence fragment in the set of valid evidence fragments of each atom statement in the second set to obtain the corresponding comprehensive quality score; Step S440: Determine the valid evidence fragments in the set of valid evidence fragments whose comprehensive quality score is greater than the preset quality threshold as supporting evidence for the corresponding atomic statement, so as to construct the supporting evidence set for each atomic statement in the second set.

[0051] In this embodiment, the technical field information (e.g., medical, legal, financial, etc.) of each atomic declaration in the second set is first obtained, and an appropriate external knowledge source (e.g., structured knowledge graph, vector document library, real-time web retrieval interface or relational database) is selected according to the field label. Retrieval requests are sent asynchronously and in parallel to at least two selected knowledge sources, and the original retrieval results returned by each knowledge source are obtained.

[0052] Furthermore, the search results for each atomic claim are screened for validity, removing evidence fragments that are semantically irrelevant to the atomic claim, have unreliable sources (such as low-authority sources like self-media), or contain obvious errors, thus extracting a set of valid evidence fragments. Then, each valid evidence fragment in the set is evaluated for quality, specifically including: The relevance score (based on semantic similarity), authority score (based on source domain name or institution level), and timeliness score (based on the degree of matching between the publication time and the declaration time requirement) of the evidence fragment are calculated separately. The scores are then weighted and summed according to preset weights (α, β, γ, which can be dynamically adjusted according to the domain) to obtain the comprehensive quality score of the evidence fragment.

[0053] Furthermore, valid evidence fragments with a comprehensive quality score greater than a preset quality threshold, or Top-M evidence fragments selected after sorting by comprehensive score from high to low, are determined as supporting evidence for the atomic statement, thereby constructing a supporting evidence set for each atomic statement in the second set, which is used for evidence consistency judgment and hallucination probability calculation in subsequent steps.

[0054] In some embodiments, step S120 includes the following steps S510 to S530: Step S510: Based on the matching degree of each atom declaration in the second set, obtain the historical signal score of the corresponding atom declaration; Step S520: Based on the supporting evidence set corresponding to each atomic statement in the second set, obtain the external evidence consistency score of the corresponding atomic statement; Step S530: The historical signal score of each atomic statement in the second set, the external evidence consistency score of each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set are weighted and calculated to obtain the illusion probability of the corresponding atomic statement.

[0055] In this embodiment, the historical signal score is first determined based on the matching degree of each atomic declaration in the second set. Specifically, if an atomic declaration is marked as "partially matched" (matching degree between the second threshold and the first threshold), the historical signal score is 0.5; if it is marked as "no match" (matching degree less than the second threshold), the historical signal score is 0.1. If the atomic declaration has a corresponding historical frequency weighting coefficient, the historical signal score is multiplied by the weighting coefficient and then limited to between 0 and 1.

[0056] Furthermore, based on the supporting evidence set corresponding to each atomic statement in the second set and the natural language reasoning relationship between each evidence fragment and the atomic statement, the consistency score of external evidence is determined. Specifically, each evidence fragment and atomic statement are input into a natural language reasoning model based on a bidirectional encoder representation technology using a transformer to obtain three types of reasoning relationship labels: supportive, contradictory, or neutral. If the relationship is supportive, the consistency score is reduced based on the overall evidence quality score; if the relationship is contradictory, the consistency score is increased based on the overall evidence quality score; if the relationship is neutral, the consistency score is slightly increased. Preferably, the initial value is set to 0.5, and after applying adjustments to all evidence fragments in sequence, the final score is limited to between 0 and 1.

[0057] Furthermore, the historical signal score, external evidence consistency score, and model generation confidence score extracted from the target model (obtained by calculating the average log probability from the token probability distribution when the atomic statement was generated) of the atomic statement are weighted and calculated. Specifically, the three dimensions are multiplied by their corresponding weight coefficients and then summed to obtain the hallucination probability of the atomic statement, with a value ranging from 0 to 1, where a higher score indicates a greater likelihood that the statement is a hallucination.

[0058] In some embodiments, step S160 includes the following steps S610 to S640: Step S610: Mark all atomic declarations in the first set as historically confirmed hallucinations, and extract the corresponding correction content and hallucination type from the hallucination memory bank; Step S620: Mark the atomic declarations in the second set whose hallucination probability is greater than or equal to the preset hallucination threshold as new hallucinations, and determine the corresponding hallucination type; Step S630: Mark the atomic declarations in the second set whose hallucination probability is less than the preset hallucination threshold as trustworthy content; Step S640: Based on the labeling information, correction content, and illusion type of each atom declaration, generate the illusion recognition result of the target model.

[0059] In this embodiment, all atomic statements in the first set are first directly marked as historically confirmed hallucinations. Since the atomic statements in the first set have been determined to be highly matched with the historical records in the hallucination memory bank (the matching degree is greater than a preset threshold), there is no need to perform external retrieval and hallucination probability calculation. The corrected content (correct factual statement) and hallucination type (such as factual error, temporal error, entity confusion, etc.) stored in the corresponding entries are directly extracted from the hallucination memory bank as the basis for subsequent output processing.

[0060] Furthermore, for the atomic statements in the second set, their illusion probability is compared with a preset illusion threshold: atomic statements with an illusion probability greater than or equal to the preset illusion threshold are marked as new illusions, and the specific illusion type (such as entity error, numerical fabrication, citation forgery, etc.) is determined according to their contradictory relationship with the supporting evidence set. At the same time, the correct expression is extracted from the supporting evidence set as the correction content.

[0061] Furthermore, atomic statements in the second set with a hallucination probability less than a preset hallucination threshold are marked as credible content, indicating that these statements have been verified by external evidence and no factual errors have been found, requiring no correction or annotation. Then, by combining the marking information of all atomic statements (historically confirmed hallucinations, newly added hallucinations, or credible content), the corrected content, and the hallucination type, a complete hallucination recognition result for the target model's text to be detected is generated.

[0062] Specifically, the identification result includes: the overall risk level of the text to be detected (high risk, medium risk, or low risk), and the final text after graded output according to the risk level. Specifically, for low risk, the original text is output directly; for medium risk, suspected hallucinations are highlighted and accompanied by notes; for high risk, the hallucination portion is replaced with the correct description and the corrected text is output, with the option to attach an interpretable summary and a structured hallucination report.

[0063] In some embodiments, the steps for applying the above-described hallucination recognition method based on long-term memory enhancement are as follows: The first step is to build and persistently store the Hallucination Memory Bank (HMS): Building a continuously updated and reusable long-term hallucination memory bank to store historically confirmed hallucination records provides data support for subsequent historical hallucination matching, enabling "once you make a mistake, you can avoid it for life". It is the prerequisite for all subsequent steps, and its output will directly serve as the retrieval data source for the third step.

[0064] Specifically, the process begins by acquiring historical hallucination annotation data, manually confirmed hallucination cases, and user error correction data. Then, a persistent storage structure is determined using MongoDB / MySQL, and 10 fixed fields are defined, each generated from a specific data source. hallucination_id: generated by "domain prefix + random string", for example, MED_HALL_001; claim_text: Directly taken from the original text of the confirmed erroneous facts output by the model (input data); claim_vector: Input claim_text into the text-embedding-ada-002 / BERT encoder and output a 512–2048 dimension vector; hallucination_type: Labeled by the human / verification model as one of five categories: factual error, temporal error, entity confusion, number fabrication, and reference forgery; correction: Correct factual text (input data) taken from authoritative data sources (encyclopedia / regulation / paper); confidence_score: Manual verification → assign a value of 0.9–1.0; Machine verification → assign a value of 0.7–0.9; occurrence_count: Initially 1, automatically incremented by 1 for each match; source_domain: Obtained by classifying claim_text using a text classification model; last_updated: Assigned by the current system timestamp (YYYY-MM-DDHH:MM:SS); evidence_links: Taken from authoritative source URLs (input data).

[0065] Thus, importing ≥10 confirmed hallucinations forms a usable HMS, resulting in a persistent, searchable, and updatable exclusive hallucination memory bank (HMS).

[0066] The second step, Declaration Decomposition and Vectorization (receiving and preprocessing the target text): preprocesses the target text generated by the large language model, distinguishes between opinion content that does not need to be detected and factual content that needs to be detected, decomposes the factual content into the smallest verifiable unit (atomic declaration) and quantizes it to generate standardized input, providing a searchable semantic vector for the historical illusion matching in the third step, which is the key link connecting the "input text" and "historical matching".

[0067] Specifically, the large model output text T (the text to be detected) is classified according to intent. T is then input into a fine-tuned BERT text classification model to obtain its classification labels: factual / opinion / mixed. The factual component is then extracted: opinion-based texts are skipped during detection; factual / mixed texts are extracted as factual subtexts T_factual.

[0068] Furthermore, T_factual is input into the Claim Decomposer to decompose the model into the smallest verifiable units: A1, A2, ..., A n Then declare vectorization, and convert each A i Input the same embedding model as in the first step, and obtain vectors V1, V2, ..., V n The dimension of this vector is exactly the same as that of the claim_vector in HMS.

[0069] Step 3, Historical Illusion Matching (Using Historical Memory to Quickly Filter Illusions): Using the illusion memory database built in Step 1, the semantic vectors of atomic claims output in Step 2 are quickly matched to identify historically confirmed illusions. No further external retrieval is required, which greatly reduces response latency and improves recognition efficiency. At the same time, atomic claims that are not matched or partially matched are diverted to provide clear objects for external retrieval in Step 4.

[0070] The specific operation process is as follows (based on the illusion memory bank output in the first step, and the atomic declarations and semantic vectors output in the second step): Vector V i Input HMS, perform ANN approximate nearest neighbor search, and calculate V. i The highest cosine similarity with all illusion vectors is denoted as sim_max. Then, a three-level splitting judgment is performed: If sim_max≥θ1→ mark it as HIT (historical illusion), retrieve correction, confidence_score, and hallucination_type from HMS; If θ2≤sim_max<θ1, mark it as PARTIAL (partial match) and add it to the list to be verified; If sim_max < θ2, mark it as MISS (no match) and add it to the list to be verified. Further, the corresponding hallucination entry `occurrence_count` in the HMS is obtained, and the weighting coefficient `w_freq = 1.0 + (occurrence_count / upper limit)` is determined, with a maximum value not exceeding 2.0, for weighted calculation to obtain the weighted risk coefficient. Finally, the matching results (HIT / PARTIAL / MISS), the list of items to be verified, and the weighted risk coefficient are obtained.

[0071] Step 4, Real-time External Knowledge Retrieval (Fact Verification of Unmatched Illusions): Real-time multi-source external knowledge retrieval is performed on the atomic claims in the list to be verified output in Step 3 to obtain objective evidence and verify their factuality, providing core support for the illusion scoring in Step 5; at the same time, a parallel retrieval method is adopted to reduce retrieval latency, balance efficiency and accuracy, and solve the problem that "historical memory cannot cover unknown illusions".

[0072] The specific operation process is as follows (based on the list of items to be verified output in step three, combined with the domain tags of the hallucination memory bank in step one, adapted to domain retrieval requirements): 1. Multi-source parallel retrieval: For each atomic declaration in the verification list output from step three, retrieval requests are simultaneously initiated to at least two types of external knowledge sources. This is executed asynchronously and in parallel, with a total retrieval latency of ≤500ms, ensuring real-time response and avoiding impact on overall recognition efficiency. Based on the domain tags of the atomic declaration (associated with domain tags from the illusion memory bank in step one), suitable external knowledge sources are flexibly selected. The applicable scenarios and retrieval methods for each type of knowledge source are as follows: Structured knowledge graphs, such as Wikidata, DBpedia, UMLS in the medical field, and legal knowledge graphs in the legal field, are suitable for verifying facts such as entity attributes and entity relationships (e.g., "Einstein's nationality" or "Newton's major achievements"). Standardized entity relationship data can be retrieved through the knowledge graph API interface. Vector document libraries: such as enterprise private document libraries, academic paper libraries, and industry standard libraries, are suitable for verifying facts in a professional field (such as "the technical parameters of a product" or "the supporting evidence for an academic viewpoint"). They use vector retrieval to match document fragments that are semantically similar to atomic declarations. Real-time Web Search: Obtain real-time online information through search engine interfaces such as Baidu API and Google API. It is suitable for verifying facts with high timeliness (such as "the latest GDP data in 2026" and "recent policy adjustments") and filtering authoritative source information within the past year. Relational databases, such as product parameter databases, regulatory clause databases, and user data dictionaries, are suitable for verifying precise values, version information, and regulatory clauses (such as "the specific content of a certain regulation" or "the factory parameters of a certain product"). They obtain precisely matched numerical or text information through SQL queries.

[0073] For example, evidence E1 is obtained from a structured knowledge graph (Wikidata / UMLS); evidence E2 is obtained from a private vector document library; evidence E3 is obtained from a real-time web retrieval interface; and evidence E4 is obtained from a relational database.

[0074] 2. Evidence Fragment Screening: For each atomic statement, at least 3 relevant evidence fragments are retrieved from various external knowledge sources (evidence fragments are objective text, data, clauses, etc. related to the atomic statement). Irrelevant and invalid evidence fragments are screened out—such as content unrelated to the statement, unreliable self-media content, and information with obvious errors. Valid evidence fragments are retained to ensure the authenticity and relevance of the evidence.

[0075] For example, 3. Evidence quality assessment: For each valid piece of evidence E k Three scores (each ranging from 0 to 1) are calculated to comprehensively assess the quality of the evidence and provide a basis for subsequent calculation of the overall score. The calculation method for the three scores is as follows: Correlation score R(A) i E k ): By calculating the atomic declaration A i With evidence fragment E k The semantic similarity is obtained by using the same semantic embedding model as in the first and second steps. The higher the similarity, the higher the relevance score, with a minimum of 0 and a maximum of 1. Authoritative score Auth(E) k The assessment is based on the source of the evidence fragments; the more authoritative the source, the higher the score. Specific standards are as follows: content published on government websites, authoritative academic journals, and leading industry enterprises has an authority score ≥ 0.8; content published on ordinary corporate websites and in mainstream media has an authority score of 0.5-0.8; content published by ordinary self-media and from unofficial sources has an authority score ≤ 0.5. Assessment dimensions include the source domain name, number of citations, and the qualifications of the publishing organization. Recency score (E) k The timeliness score is calculated based on the publication time of the evidence fragments and the time requirements of the atomic statement. If the atomic statement involves time-sensitive content (such as "latest data" or "recent policies"), the more recent the publication time, the higher the timeliness score. Evidence within the last 3 months has a score of ≥0.8, evidence within the last year has a score of ≥0.6, and evidence older than 1 year has a score of ≤0.5. If the atomic statement does not involve timeliness (such as "historical event dates"), the timeliness score is fixed at 0.9.

[0076] 4. Calculation of Overall Evidence Score: The overall score for each piece of evidence is obtained through weighted calculation. The calculation formula is as follows: ; in, This is a weighting coefficient, with values ​​ranging from 0 to 1 and summing to 1. It can be dynamically adjusted according to domain requirements to adapt to the evidence priority needs of different domains. Medical and legal fields: Authority weight Prioritize ensuring the authority of the evidence to avoid misjudgment due to non-authoritative evidence; News, government affairs, and finance: Timeliness weighting Prioritize the timeliness of evidence and ensure that verification results are consistent with the latest information; Academic Field: Relevance Weights Prioritize ensuring the relevance of evidence to statements to guarantee the relevance of verification.

[0077] 5. Evidence Aggregation: Select the top-M evidence fragments with the highest comprehensive scores (M ranges from 2 to 5, preferably 3) as the supporting evidence set for the atomic statement; the supporting evidence set must contain at least one piece of evidence with a high comprehensive score (≥0.8) to ensure the reliability of the verification and avoid misjudgment due to low-quality evidence; at the same time, sort the supporting evidence set, and the evidence with the higher the comprehensive score is ranked higher and has a higher weight in the subsequent fifth step of scoring.

[0078] 6. Special Case Handling: If no valid evidence is returned from any external knowledge source for a certain atomic statement (i.e. evidence vacuum), the atomic statement is marked as "unverifiable", recorded separately, and handled separately in the subsequent fifth step of scoring. It is not included in the regular illusion probability calculation, but it will affect the overall risk level.

[0079] Finally, we obtain the supporting evidence set (Top-M high-scoring evidence fragments, sorted by comprehensive score) for each atomic statement in the list to be verified, the comprehensive score and three individual scores (relevance, authority, and timeliness) for each evidence fragment, the list of atomic statements marked as "unverifiable" (if any), and the comprehensive quality score of evidence for each atomic statement (the average comprehensive score of the supporting evidence set).

[0080] Step 5, Multi-dimensional Hallucination Scoring (Determining Hallucination Probability and Risk Level): Integrating multi-dimensional signals (historical hallucination signals, external evidence signals, and internal model signals), the hallucination probability is calculated for each atomic statement to clarify the hallucination type. At the same time, the overall risk level of the target text is classified, providing a clear basis for the credible output processing in Step 6. This is the core judgment link in the entire hallucination recognition process.

[0081] The specific operation process is as follows (based on the matching status and weighting information output in the third step, the supporting evidence set and evidence quality score output in the fourth step, and combined with the internal signals generated when generating atomic statements using a large language model): 1. Single-declaration illusion probability calculation: For each atomic declaration A i Based on the following three dimensions, calculate its hallucination probability score P_hall(A i The value ranges from 0 to 1. The higher the score, the greater the probability that the statement is a hallucination. The calculation formula and explanation of each dimension are as follows: Dimension D1 – HMS Historical Signal: Determined based on the matching results of step three, it directly reflects the degree of association between the statement and historical hallucinations, and is an important basis for quickly judging hallucinations. Specific value rules: Historical hallucination matching (hit): D1=0.9 (high hallucination risk), indicating that the statement is highly similar to a historically confirmed hallucination, and is likely a hallucination; Partial match: D1=0.5 (moderate hallucination risk), indicating that the statement has some similarity to historical hallucinations, and further judgment is needed in conjunction with external evidence; No match (miss): D1=0.1 (low risk of hallucination), indicating that the statement has no historical hallucination reference and must rely entirely on external evidence for judgment; If the historical illusion entry corresponding to the atomic statement has frequency weighting, D1 needs to be multiplied by the weighting coefficient calculated in the third step to further adjust the risk weight (for example, when the weighting coefficient is 2.0, D1=0.9×2.0=1.8, and if it exceeds 1.0, it is calculated as 1.0).

[0082] Dimension D2 – External Evidence Consistency: Based on the supporting evidence set from step four, a fine-tuned BERT-NLI natural language inference model (inference accuracy ≥ 92%) is used to determine the inference relationship between each piece of evidence and the atomic statement, and then D2 is calculated, with a value ranging from 0 to 1. Specific calculation rules are as follows: Entailment: The evidence fragment clearly supports the authenticity of the atom statement. At this time, D2 decreases by 0.1-0.3 (the higher the overall quality score of the evidence, the greater the decrease. For example, when the quality score is 0.9, it decreases by 0.3). Contradiction: When the evidence fragment clearly contradicts the atomic statement, D2 increases by 0.3-0.5 (the higher the overall quality score of the evidence, the greater the increase; for example, when the quality score is 0.9, it increases by 0.5). Neutral: The evidence fragment has no clear connection with the atomic statement, neither supporting nor contradicting it. In this case, D2 increases slightly, by 0.05-0.1 (unrelated to the quality score of the evidence). The final value of D2 is calculated by combining the distribution of reasoning relationships among all evidence fragments and the overall quality score of the evidence. For example, if an atom's statement has 3 pieces of evidence, 2 of which are contradictory (quality score 0.9) and 1 is neutral, the initial value of D2 is 0.5. The contradictory evidence increases by 0.5 × 2 = 1.0, and the neutral evidence increases by 0.1. The final D2 = 0.5 + 1.0 + 0.1 = 1.6. If it exceeds 1.0, it is calculated as 1.0.

[0083] Dimension D3 – Model Confidence Calibration: Extract the token probability distribution when the large language model generates this atomic statement, and calculate the average log probability logP(Ai This value, ranging from 0 to 1, is used to assess the model's "confidence" when generating the statement. The specific calculation rules are as follows: If the mean log probability is ≥-0.5 (the model is highly confident), but the evidence consistency D2 is ≥0.8 (the evidence is clearly contradictory), it is judged as "confidence-based illusion". D3 is significantly increased to 0.8-0.9. This type of illusion needs to be closely monitored to avoid misbelief caused by the model's "confidence". If the average log probability is ≥-0.5, and the evidence consistency D2 ≤ 0.2 (clear evidence supports the claim), and D3 decreases to 0.1-0.2, it indicates that the model is confident and the evidence supports the claim, suggesting that the content is likely credible. In other cases, D3 is linearly adjusted according to the mean log probability, with a value ranging from 0.2 to 0.7. The higher the mean log probability, the closer D3 is to 0.7, and vice versa.

[0084] The comprehensive formula for calculating the probability score of hallucination is: ; in, These are weighting coefficients, ranging from 0 to 1, with a sum of 1. The default value is [value to be filled in]. (Historical signal weights) (Weight of consistency of evidence) (Model confidence weights) can be dynamically adjusted according to different fields (e.g., the medical field). It can be increased to 0.5, reinforcing the importance of the evidence.

[0085] 2. Hallucination type classification: When the hallucination probability of a certain atom declaration is P_hall(A i When the value is greater than or equal to the preset threshold φ (φ=0.60), the hallucination type sub-classifier is triggered, classifying the hallucination into the corresponding hallucination type. The classification criteria are completely consistent with the hallucination type in the first step. At the same time, combined with the supporting evidence chain in the fourth step, the specific error points are identified to facilitate subsequent correction. Factual illusions (most common type): Entity errors: Entities such as names of people, places, and organizations are misattributed, for example, "Newton discovered relativity" (error: entity confusion, the correct entity is Einstein); Chronological errors: Inconsistent time points or order, for example, "Einstein published relativity in 1921" (error: chronological error, the correct time is 1905); Numerical fabrication: Statistics, parameters, etc., are unverifiable or incorrect, for example, "China's population is 1.5 billion" (error: numerical fabrication, the correct value is approximately 1.41 billion); Citation forgery: Forged documents, regulations, and data sources, for example, "Citation of a 2025 paper in *Nature*" (error: citation forgery, no such paper exists).

[0086] Contextual illusion (closely related to the context): Confusion of reference: incorrect pronoun reference, for example, "Xiaoming and Xiaohong went to the park, and he bought a bottle of water" (error: confusion of reference, "he" does not clearly refer to Xiaoming or Xiaohong); logical contradiction: logical conflict in the context, for example, "Xiaoming is 10 years old this year, and he was 12 years old last year" (error: logical contradiction, conflict in age sequence).

[0087] 2. Overall Risk Level Aggregation: Based on the illusion probability score, illusion type, and "unverifiable" status of all atomic claims, the overall risk level of the target text is classified to clarify the credibility of the target text and provide a basis for the output processing in step six. The specific classification rules are as follows: High risk: There exists at least one atomic declaration P_hall(A i If the value is ≥0.85, it indicates that there is a clear hallucination in the target text and the risk is high. It needs to be forcibly corrected before output to avoid misleading users. Medium risk: P_hall(A) does not exist. i Atom declarations with a value ≥ 0.85, but multiple (≥ 2) atomic declarations exist: P_hall(A i If there is an atomic statement ∈ [0.60, 0.85), or there is one or more "unverifiable" statements, it indicates that there is a possible hallucination in the target text. It is necessary to mark and remind users to avoid being misled. Low risk: P_hall(A) for all atomic declarations i If the value is less than 0.60 and there are no "unverifiable" atomic declarations, it indicates that the target text is generally reliable and can be output normally.

[0088] The output of this step is as follows: the illusion probability score, illusion type label (if it is an illusion), and supporting / contradictory evidence chain for each atomic statement; the overall risk level of the target text; and a list of "unverifiable" atomic statements (if any). All outputs will be directly used as input for step six for trusted output processing.

[0089] Step 6, Trusted Output Processing (Graded Output and Correction): Based on the overall risk level defined in Step 5, the target text is processed in a targeted manner to achieve "graded control and trusted output." At the same time, interpretable content is generated to enhance users' trust in the output results and ensure that the output content meets the needs of high-trust scenarios.

[0090] The specific operation process is as follows (based on the overall risk level, hallucination probability score, hallucination type, evidence chain, and other results output in step five): 1. Low-risk processing: If the target text is marked as low-risk as a whole, it means that all atomic statements are trustworthy, there are no hallucinations or unverifiable content. In this case, the original target text generated by the large language model is directly output, and a "low-risk trustworthy" label is appended to the end of the text. The timestamp of this hallucination detection is marked after the label (format: "YYYY-MM-DD HH:MM:SS") to facilitate users to trace the detection record; no modifications are required to ensure output efficiency.

[0091] 2. Medium Risk Handling: If the target text is marked as medium risk overall, it indicates the presence of suspected hallucinations or unverifiable content. A warning should be issued, but the original text should not be forcibly modified. Specific steps are as follows: ① For all P_hall(A i ① Highlight suspected hallucination statements within the range [0.60, 0.85) using red text or underline, ensuring the highlighting position matches the original text's location for the statement; ② After each highlight, add a parenthetical note including: hallucination type, hallucination probability score (rounded to two decimal places), suggested revision (generated based on the supporting evidence set from step four), and 1-2 links to core evidence (from the supporting evidence set from step four); ③ Highlight "unverifiable" atomic statements with a yellow background and the note "Unverifiable: No valid supporting evidence found, caution advised"; ④ At the end of the target text, generate a brief interpretable summary (50-100 words) explaining the percentage of credible content, the number of suspected hallucinations, the number of unverifiable statements, and a key reminder (e.g., "2 suspected hallucinations were detected; please verify the highlighted content before use").

[0092] 3. High-risk handling: If the target text is marked as high-risk overall, it indicates the presence of a clear hallucination and requires forced correction before output to avoid misleading users. Specific steps are as follows: ① Trigger the regeneration process and extract all P_hall(A) from step 5. i ① Replace hallucination statements with a probability score ≥ 0.60 with correct statements verified in steps 3 and 4 (statements with historical hallucination matches use the correction field extracted in step 3; statements with partial or no matches generate correct statements based on the supporting evidence set in step 4); ② Reorganize the text structure to ensure logical coherence and fluent sentences in the corrected text, without changing the core intent of the original text (except for the hallucination part, other content remains unchanged); ③ At the end of the corrected text, generate a detailed interpretable summary (100-150 words) explaining the number of hallucinations found in this detection, the hallucination type, the basis for correction (link to core evidence), and credible content descriptions to enhance user trust; ④ Provide an optional structured hallucination report (for developer debugging), which includes the original text of each hallucination statement, hallucination probability score, hallucination type, corrected statement, and a list of supporting evidence, facilitating developers to optimize the large language model and this hallucination recognition method.

[0093] The output of this step includes: the final credible text after hierarchical processing / correction, an interpretable summary, and a detection timestamp; an optional structured hallucination report (for developers only); all outputs will be synchronously fed back to step seven for updating the hallucination memory bank.

[0094] Step 7, Hallucination Memory Bank Update and Maintenance (Dynamic Optimization): Asynchronously execute the updating, aging, distillation and cleaning of the hallucination memory bank to achieve dynamic optimization of the memory bank, continuously enrich historical hallucination records, improve the efficiency and accuracy of subsequent hallucination recognition, and form a closed loop of "recognition-correction-update-optimization", which is the core link of this embodiment to achieve long-term memory enhancement.

[0095] The specific operation process is as follows (based on the output of step 6, user feedback, and the hallucination memory bank built in step 1): 1. Automatic memory update: For atomic declarations identified as hallucinations in step 5 (P_hall≥φ=0.60) and clearly contradicted by external evidence in step 4, an automatic memory write operation is triggered. First, a similarity search is performed in the hallucination memory (using the Top-K nearest neighbor search method from step 3, with a threshold θ1=0.85). If an entry with a similarity ≥θ1 already exists, it means that the hallucination has been recorded. At this time, the relevant fields of the entry are updated: occurrence_count (historical occurrence count +1) and confidence_score (using weighted average). The update method is as follows: new confidence = (original confidence × original occurrence count + current hallucination probability score) ÷ (original occurrence count + 1)), last_updated (updated to the current timestamp); if there is no entry with similarity ≥ θ1, it means that the hallucination is a new hallucination, create a new hallucination memory entry, and write all 10 core fields defined in the first step to ensure information integrity. Initially, occurrence_count = 1, confidence_score = current hallucination probability score, and source_domain is labeled according to the domain attribute declared in the atom.

[0096] 2. User Feedback Enhancement: Explicit feedback from users submitted through the system interaction interface is received. Users can mark the output results of step 6 as "correct / incorrect / partially incorrect." The feedback information will be used to optimize the memory weights: ① If the user confirms the output result is incorrect (i.e., the system missed a hallucination), the corresponding atomic declaration is found, and the corresponding hallucination entry is retrieved or created in the memory, significantly increasing the weight of that entry—occurrence_count +5, confidence_score increased by 0.1-0.2 (not exceeding 1.0), ensuring that subsequent hallucinations of this type can be quickly identified; ② If the user confirms the output result is correct (i.e., the system falsely reported a hallucination), the corresponding historical hallucination entry (if any) is found, and the confidence score of that entry or similar entries is reduced (down by 0.1-0.2, not lower than 0.5), while its frequency weighting coefficient is reduced to avoid similar false alarms in the future; ③ If the user marks "partially incorrect," only the atomic declarations corresponding to the incorrect part are updated as described above, while the correct part is not affected in the memory.

[0097] 3. Memory Aging Process: A time decay function is introduced to age all entries in the hallucination memory bank, preventing long-term unreproduced hallucination entries from consuming resources and affecting recognition accuracy. The effective confidence score is calculated using the following formula: ; Where λ is the decay coefficient (configurable, ranging from 0.01 to 0.1), and Δt is the interval between the last_updated time of the entry and the current time (in days); when the effective confidence of an entry is ≤0.5, it is marked as "pending cleanup" and will not be deleted for the time being. If it is not reproduced later, it will be deleted during the periodic cleanup.

[0098] 4. Memory Distillation and Cleaning: Regularly distill and clean the hallucination memory bank. The distillation cycle is 1-3 months (configurable). The specific operations are as follows: ① Distillation: Distill high-frequency, high-confidence hallucination patterns (occurrence_count≥5 and confidence_score≥0.8) into lightweight classification rules (such as keyword combinations and semantic feature templates), and import them into the matching module in step 3 to improve the speed of historical hallucination matching and reduce the computational load of nearest neighbor retrieval; ② Cleaning: Delete entries in the "to be cleaned" state (effective confidence ≤0.5), occurrence_count≤2 and confidence_score≤0.7 to release storage resources; at the same time, check the duplicate entries in the memory bank (similarity≥0.95), retain entries with high confidence and high occurrence frequency, and delete duplicate entries to ensure the lightweight and accuracy of the memory bank.

[0099] The output of this step includes: HMS update confirmation log (containing the number of new entries, the number of updated entries, and the number of deleted entries), weight parameter update record (weight adjustments due to user feedback), and memory bank health report (including the total number of entries, average confidence, distribution of entries in each domain, and the top 10 most frequent hallucinations). The updated hallucination memory bank will serve as the basic data source for subsequent hallucination recognition, enabling long-term memory enhancement.

[0100] Based on this, this embodiment constructs a hallucination memory bank to persistently store historically confirmed hallucination records, achieving enhanced hallucination recognition across time and sessions, avoiding repeated verification, and significantly improving recognition efficiency. It integrates historical hallucination signals, consistency of external evidence, and multi-dimensional scoring of model confidence to effectively identify various types of hallucinations, especially significantly improving the recognition effect of "confidence-based hallucinations". Through user feedback loop and memory aging mechanism, the hallucination memory bank is dynamically optimized to avoid interference from outdated and erroneous data and adapt to the changing needs of different fields and scenarios. It generates text according to risk level classification, provides interpretable annotations and correction suggestions, and improves the security and usability of the output of large language models in high-confidence scenarios.

[0101] like Figure 2 As shown in some embodiments of this application, a model hallucination recognition system based on long-term memory enhancement is provided. The system includes a disassembly module 210, a matching module 220, a segmentation module 230, an execution module 240, a calculation module 250, and a generation module 260. Specifically: The decomposition module 210 is used to decompose the text to be detected in the target model to obtain multiple atomic statements; the atomic statement is used to characterize the smallest verifiable unit in the text to be detected. Matching module 220 is used to match multiple atomic declarations with historical hallucination records in a pre-built hallucination memory bank to obtain the matching degree of each atomic declaration; The partitioning module 230 is used to divide multiple atomic declarations into a first set and a second set based on the matching degree of each atomic declaration; the first set is used to represent atomic declarations among multiple atomic declarations whose matching degree is greater than a preset matching degree threshold; the second set is used to represent atomic declarations among multiple atomic declarations whose matching degree is less than or equal to the preset matching degree threshold. The execution module 240 is used to perform knowledge retrieval on each atomic statement in the second set to obtain the supporting evidence set corresponding to each atomic statement in the second set; The calculation module 250 is used to calculate the illusion probability of each atomic statement in the second set based on the matching degree of each atomic statement in the second set, the supporting evidence set corresponding to each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set; the model generation confidence of each atomic statement in the second set is calculated by the token probability distribution when the target model generates each atomic statement in the second set. The generation module 260 is used to generate the illusion recognition result of the target model based on the matching degree of each atomic declaration in the first set and the illusion probability of each atomic declaration in the second set.

[0102] It should be noted that the model hallucination recognition system based on long-term memory enhancement provided in this embodiment is based on the same inventive concept as the model hallucination recognition method based on long-term memory enhancement described above. Therefore, the relevant content of the model hallucination recognition method based on long-term memory enhancement described above also applies to the content of the model hallucination recognition system based on long-term memory enhancement. Therefore, it will not be repeated here.

[0103] The system decomposes the target model's text into multiple atomic statements; matches these atomic statements with historical hallucination records to obtain the matching degree of each atomic statement, thus dividing the atomic statements into a first set and a second set; performs knowledge retrieval on each atomic statement in the second set to obtain the corresponding supporting evidence set; calculates the hallucination probability of each atomic statement in the second set based on the matching degree, the corresponding supporting evidence set, and the model generation confidence of each atomic statement in the second set; and generates the hallucination recognition result of the target model based on the matching degree of each atomic statement in the first set and the hallucination probability of each atomic statement in the second set. In this way, by using statement-level matching and hierarchical retrieval, and reusing historical hallucination memories, the system can significantly improve recognition efficiency and accuracy.

[0104] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described model illusion recognition method based on long-term memory enhancement.

[0105] like Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes: At least one battery; At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the above-described model illusion recognition method based on long-term memory enhancement as described in this disclosure.

[0106] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0107] The electronic devices according to embodiments of this application will now be described in detail.

[0108] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to perform a model hallucination recognition method based on long-term memory enhancement according to an embodiment of this disclosure.

[0109] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0110] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the aforementioned model illusion recognition method based on long-term memory enhancement.

[0111] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0112] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0113] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0116] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any related variations, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0117] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

[0123] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A model-based hallucination recognition method based on long-term memory enhancement, characterized in that, The method includes: The target model's text to be detected is decomposed into multiple atomic statements; the atomic statements are used to characterize the smallest verifiable unit in the text to be detected. The multiple atomic statements are matched with historical hallucination records in a pre-built hallucination memory bank to obtain the matching degree of each atomic statement; Based on the matching degree of each atomic declaration, the plurality of atomic declarations are divided into a first set and a second set; the first set is used to characterize atomic declarations among the plurality of atomic declarations whose matching degree is greater than a preset matching degree threshold; the second set is used to characterize atomic declarations among the plurality of atomic declarations whose matching degree is less than or equal to the preset matching degree threshold. For each atomic statement in the second set, perform knowledge retrieval to obtain the supporting evidence set corresponding to each atomic statement in the second set; Based on the matching degree of each atomic statement in the second set, the supporting evidence set corresponding to each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set, the illusion probability of each atomic statement in the second set is calculated; the model generation confidence of each atomic statement in the second set is calculated through the token probability distribution when the target model generates each atomic statement in the second set. Based on the matching degree of each atomic statement in the first set and the illusion probability of each atomic statement in the second set, the illusion recognition result of the target model is generated.

2. The hallucination recognition method based on long-term memory enhancement according to claim 1, characterized in that, The text to be detected in the target model is decomposed into multiple atomic declarations, including: The text to be detected is classified according to intent to obtain a classification result; the classification result includes either factual text or mixed text. If the classification result is factual text, the text to be detected is decomposed into the multiple atomic statements; In the case where the classification result is mixed text, factual subtext is extracted from the text to be detected, and the factual subtext is decomposed into the multiple atomic statements.

3. The hallucination recognition method based on long-term memory enhancement according to claim 1, characterized in that, The step of matching the plurality of atomic statements with historical hallucination records in a pre-built hallucination memory bank to obtain the matching degree of each atomic statement includes: The multiple atomic statements are converted into data dimensions of the pre-built hallucination memory library to obtain the semantic vector of each atomic statement; For the semantic vector of each atomic declaration, an approximate nearest neighbor search is performed in the pre-built hallucination memory bank to calculate the semantic similarity between the semantic vector of each atomic declaration and the semantic vector of each historical hallucination record in the pre-built hallucination memory bank. The matching degree of each atomic statement is determined based on the semantic similarity between the semantic vector of each atomic statement and the semantic vector of each historical hallucination record in the pre-constructed hallucination memory bank.

4. The hallucination recognition method based on long-term memory enhancement according to claim 1, characterized in that, The hallucination memory bank was constructed through the following steps: Acquire initial historical data; the initial historical data includes historical hallucination annotation data, manually confirmed hallucination cases, and user error correction data; Extract the original error statement, the correct correction content corresponding to the original error statement, links to authoritative evidence, and information on the technical field of the error statement from the initial historical data; The hallucination memory bank is constructed based on the original text of the error statement, the correct correction content corresponding to the original text of the error statement, links to authoritative evidence, and information on the technical field to which it belongs.

5. The hallucination recognition method based on long-term memory enhancement according to claim 1, characterized in that, The step involves performing knowledge retrieval on each atomic statement in the second set to obtain the supporting evidence set corresponding to each atomic statement in the second set, including: Obtain the retrieval results of external knowledge sources corresponding to the technical field information of each atom declaration in the second set; Extract the set of valid evidence fragments for each atomic statement in the second set from the search results; A quality assessment is performed on each valid evidence fragment in the set of valid evidence fragments of each atom statement in the second set to obtain the corresponding comprehensive quality score; Valid evidence fragments with a comprehensive quality score greater than a preset quality threshold in the set of valid evidence fragments are identified as supporting evidence for the corresponding atomic statements, so as to construct the supporting evidence set for each atomic statement in the second set.

6. The hallucination recognition method based on long-term memory enhancement according to claim 1, characterized in that, The calculation of the illusion probability of each atomic statement in the second set, based on the matching degree of each atomic statement in the second set, the supporting evidence set corresponding to each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set, includes: Based on the matching degree of each atomic declaration in the second set, the historical signal score of the corresponding atomic declaration is obtained; Based on the supporting evidence set corresponding to each atomic statement in the second set, the external evidence consistency score of the corresponding atomic statement is obtained; The hallucination probability of each atomic statement is obtained by weighting the historical signal score, the external evidence consistency score, and the model generation confidence of each atomic statement in the second set.

7. The hallucination recognition method based on long-term memory enhancement according to claim 1, characterized in that, The step of generating the hallucination recognition result of the target model based on the matching degree of each atomic declaration in the first set and the hallucination probability of each atomic declaration in the second set includes: Mark all atomic declarations in the first set as historically confirmed hallucinations, and extract the corresponding correction content and hallucination type from the hallucination memory bank; Atom declarations in the second set whose hallucination probability is greater than or equal to a preset hallucination threshold are marked as new hallucinations, and the corresponding hallucination types are determined. Mark atomic statements in the second set whose hallucination probability is less than the preset hallucination threshold as believable content; Based on the tagging information, correction content, and hallucination type of each atomic declaration, the hallucination recognition result of the target model is generated.

8. A model hallucination recognition system based on long-term memory enhancement, characterized in that, The system includes: The decomposition module is used to decompose the text to be detected in the target model into multiple atomic statements; the atomic statement is used to characterize the smallest verifiable unit in the text to be detected. A matching module is used to match the plurality of atomic statements with historical hallucination records in a pre-built hallucination memory bank to obtain the matching degree of each atomic statement; A partitioning module is used to partition the plurality of atomic declarations into a first set and a second set based on the matching degree of each atomic declaration; the first set is used to characterize atomic declarations among the plurality of atomic declarations whose matching degree is greater than a preset matching degree threshold; the second set is used to characterize atomic declarations among the plurality of atomic declarations whose matching degree is less than or equal to the preset matching degree threshold; The execution module is used to perform knowledge retrieval on each atomic statement in the second set to obtain the supporting evidence set corresponding to each atomic statement in the second set; The calculation module is used to calculate the illusion probability of each atomic statement in the second set based on the matching degree of each atomic statement in the second set, the supporting evidence set corresponding to each atomic statement in the second set, and the model generation confidence of each atomic statement in the second set; the model generation confidence of each atomic statement in the second set is calculated by the token probability distribution when the target model generates each atomic statement in the second set; The generation module is used to generate the hallucination recognition result of the target model based on the matching degree of each atomic declaration in the first set and the hallucination probability of each atomic declaration in the second set.

9. An electronic device, characterized in that, The method includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform a model hallucination recognition method based on long-term memory enhancement as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the model hallucination recognition method based on long-term memory enhancement as described in any one of claims 1 to 7.

Images