AI model illusion suppression method and system based on retrieval enhancement generation and cue word engineering collaboration

Abstract

The invention discloses an AI model illusion suppression method and system based on retrieval enhancement generation and cue word engineering collaboration, and aims to solve the illusion problem of an existing large language model and the defects that a traditional scheme is high in cost, low in retrieval content utilization rate, universal knowledge conflicts, high in generation randomness and the like. The method comprises the following steps: constructing a high-precision domain vector index database, performing similarity retrieval and noise filtering on user query, generating a multi-level structured cue word containing role definition, thinking chain constraint and the like, dynamically switching a general knowledge shielding or fusion mode based on recall quality, and outputting a final result through multi-path reasoning sampling and consistency verification. According to the method, underlying model parameters do not need to be modified, the illusion rate is remarkably reduced, generated content is factual and accurate, logic coherence and traceability are achieved, deployment is flexible, cost is low, and the method is suitable for factual question and answer scenes in multiple fields such as medical treatment, industry and legal consultation.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a reliability control technology for the generated content of large language models, and more particularly to a method and system for accurately suppressing AI model illusions by deeply collaborating a retrieval-enhanced generation (RAG) architecture with prompt word engineering, combined with a general knowledge masking mechanism and self-consistency verification. Background Technology

[0002] With the rapid evolution of deep learning technology, large-scale language models (LLMs) based on the Transformer architecture, such as GPT-4 and the Llama series, have demonstrated powerful semantic understanding and content generation capabilities in natural language processing tasks such as text generation, question answering systems, and intelligent dialogue, driving the large-scale application of artificial intelligence technology across various industries. However, these models are essentially "next word prediction" systems based on probabilistic statistics, lacking a clear understanding of real-world facts and susceptible to biases and noise in training data. They commonly exhibit a "hallucination" phenomenon—generated text with fluent syntax but containing false information that contradicts objective facts, is logically self-contradictory, or lacks any basis. This problem severely limits the reliability of large-scale language models in sensitive fields such as medicine, law, and industry, where the accuracy of information is extremely important.

[0003] To address the model illusion problem, various technical solutions have been proposed, but all have significant drawbacks: First, the Reinforcement Learning Human Feedback (RLHF) approach optimizes the model by aligning with human preferences, but the training process requires substantial human and material resources, resulting in high costs, and still struggles to completely eliminate illusions. Second, rule-based post-processing filters can only check for specific types of factual errors, lacking adaptability to complex semantic logic and exhibiting insufficient flexibility. Third, Retrieval Augmentation Generation (RAG) technology provides factual evidence to the model by introducing external knowledge bases, mitigating factual errors to some extent, but traditional RAG technology suffers from several bottlenecks: low utilization of retrieval content, with the model tending to rely on internal pre-trained parameters rather than the input retrieval context; lack of refined logical constraints, often employing a simple "retrieval + concatenation" model, which fails to effectively distinguish when retrieval results are noisy or do not fully match the query intent; and the generated results are heavily influenced by randomness due to LLM (Limited Learning Model). The random sampling characteristic of the system means that even with correct context, incorrect logic may be generated accidentally. Furthermore, the interference from general knowledge is severe, and the model is prone to over-interpreting or conflict-correcting the retrieved content using built-in general knowledge, causing the generated content to deviate from the facts in the external knowledge base. In addition, existing similar technical solutions, such as "vector-based question-answering systems," have rudimentary prompt word designs, cannot force the model to reason step-by-step, have weak anti-interference capabilities, and lack consistency verification mechanisms, further exacerbating the illusion problem.

[0004] Therefore, there is an urgent need for a technical solution that can deeply couple the retrieval information with the generation process, and solve problems such as insufficient utilization of retrieval content, lack of logical constraints, interference from general knowledge, and randomness in generation through refined control methods, so as to achieve accurate and efficient suppression of AI model illusions. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an AI model hallucination suppression method and system based on the collaboration of retrieval enhancement generation and prompt word engineering, specifically solving the following technical problems:

[0006] 1. The "illusion" problem exists in the content generated by large-scale language models, including factual errors and logical contradictions;

[0007] 2. Existing RLHF solutions suffer from high costs and poor flexibility in rule filters;

[0008] 3. Traditional RAG technology suffers from low utilization of search content, lack of refined logical constraints, and high randomness in generated results;

[0009] 4. Conflicts between general knowledge and domain facts lead to the model ignoring or over-interpreting retrieved information;

[0010] 5. Existing solutions lack traceability, making it difficult to verify the generated results.

[0011] To address the aforementioned technical problems, this invention provides the following technical solution: an AI model hallucination suppression method based on the collaboration of retrieval enhancement generation and prompt word engineering, comprising the following steps:

[0012] Step 1: Construct a high-precision domain vector index library: Slice the documents in the domain knowledge base according to the parameters Chunk Size=512 tokens and Overlap=50 tokens. Use an embedding model that supports long text (such as bge-large-zh-v1.5 or OpenAI text-embedding-ada-002) to convert each document slice into a 1024-dimensional high-dimensional dense vector. Use the HNSW (Hierarchical Navigable Small World) algorithm to construct the vector index. Set the algorithm parameters to ef_construction=200 and M=16 to balance retrieval speed and recall. Store the constructed vector index in a vector database such as FAISS or Pinecone to provide a foundation for subsequent efficient retrieval.

[0013] Step 2, Retrieval Enhancement and Noise Filtering: After receiving the user query, the query is transformed into a high-dimensional vector using the same Embedding model as in Step 1. The cosine similarity between the query vector and the document vectors stored in the vector database is calculated, and the Top-K (K=5) relevant document fragments are retrieved. A first threshold of 0.75 is set, and only document fragments with similarity scores higher than this threshold are retained as valid context, while low-relevance noise documents are removed to ensure the quality of context information input to the model.

[0014] Step 3: Structured Prompt Generation: Construct a multi-layered structured prompt template, which specifically includes: a role definition ("You are a meticulous intelligent assistant; your task is to answer based on provided known information; fabrication is strictly prohibited"); and context injection, using...<context_info> Wrap the filtered valid context from step 2 with a special XML tag, in the format "<context_info> \n {Doc1}\n {Doc2}\n...< / context_info> "Thinking chain constraint: Please think according to the following steps: 1. Analyze"<context_info> 1. Identify key information; 2. Identify the connection between information and the problem; 3. Perform step-by-step logical reasoning; 4. Draw a conclusion; 5. Negative constraint "If..."<context_info> The answer is not included; please directly reply "I cannot answer based on the known information" and do not guess using your training data. The source tracing requirement is "Please indicate the document number cited with the [Source ID] after each statement in the answer". The effective context will be dynamically filled into the context injection area of ​​this template to form structured prompts for the current user's query.

[0015] Step 4: General Knowledge Masking Based on Recall Quality: Extract the highest similarity score of the effective context from Step 2, set a second threshold of 0.85, and compare the highest similarity score with this second threshold: If the highest similarity score ≥ 0.85, it is determined to be a strong recall mode, and the general knowledge masking instruction "System Alert: High relevance search result detected (similarity > 85%). [General knowledge masking has been activated]. The current answer must strictly and only rely on the provided [context information]. It is absolutely forbidden to call or mix your model's pre-trained knowledge. Any details not appearing in the context are considered 'unknown' and inference is prohibited," forcing the model to generate only based on the retrieved authoritative facts; If the highest similarity score is lower than 0.85 but higher than the first threshold of 0.75, it is determined to be a weak recall mode, the general knowledge masking is lifted, the model is allowed to combine effective context and pre-trained general knowledge for logical completion, and the model is required to label the uncertainty in the generated content.

[0016] Step 5, Self-consistent Decoding and Illusion Filtering: Input the optimized structured prompts from Step 4 into a large language model based on the Transformer architecture (such as GPT-4, Llama series). Set the Temperature parameter of the inference sampling to 0.7 to retain appropriate diversity, and perform N=5 independent inference samplings to obtain 5 candidate answers. First, perform compliance verification on the candidate answers and eliminate those that violate negative constraints or general knowledge masking instructions (such as fabricating information or mixing pre-trained knowledge). For the remaining compliant candidate answers, use a semantic clustering algorithm based on cosine similarity or a similarity comparison method after TF-IDF keyword extraction to count the conclusions that appear most frequently. Set a preset voting threshold of 3. If the frequency of the most frequent conclusion is ≥3, it is taken as the final output result; otherwise, trigger a retry mechanism (re-execute steps 2-5) or output "uncertain" to ensure the reliability of the generated results.

[0017] This invention also provides an AI model hallucination suppression system based on the collaboration of retrieval enhancement generation and prompt word engineering, including a vector index construction module, a retrieval filtering module, a prompt word generation module, a knowledge masking module, a decoding and verification module, an output module, and a storage module.

[0018] Vector index building module: used to execute step 1 of the above method, and complete the slicing, vector embedding and index building of domain knowledge base documents;

[0019] Search and filtering module: used to execute step 2 of the above method, realize similarity retrieval and noise filtering of user queries, and obtain effective context;

[0020] Prompt word generation module: used to execute step 3 of the above method, construct structured prompt words and dynamically populate valid context;

[0021] Knowledge masking module: used to execute step 4 of the above method, switch the generation mode based on the recall quality assessment results and inject corresponding instructions;

[0022] Decoding and verification module: used to execute step 5 of the above method, to realize multi-path reasoning sampling, candidate answer filtering and consistency verification;

[0023] Output module: Used to output the final result obtained by the decoding and verification module, or to re-call the process from the retrieval and filtering module to the decoding and verification module when the retry mechanism is triggered;

[0024] Storage module: Used to store domain knowledge base documents, constructed vector indexes, retrieval logs and generated results. The generated results include labeled source IDs and generation mode information, which facilitates subsequent traceability and review.

[0025] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages compared with the prior art:

[0026] Significant hallucination suppression effect: By introducing an external authoritative domain knowledge base through the RAG architecture, combined with multi-level constraints of structured prompts and a general knowledge shielding strategy, the information path of model hallucination is cut off from the entire process of retrieval source, constraint generation, and result verification. Experimental verification shows that on an internal test set containing 500 factual questions and answers, the hallucination rate is reduced from 18.5% of the traditional RAG to 4.2%, and the hallucination rate is as low as 5.2% in the TruthfulQA benchmark test, which is significantly better than the native LLM (18.5%) and the traditional RAG (10.3%).

[0027] The utilization rate of retrieval information has been greatly improved: the thought chain constraint in the structured prompt words forces the model to analyze the retrieval context step by step, and the special label marking and semantic alignment design enhance the model's attention weight to the retrieval information. The citation accuracy has increased from 65% in traditional RAG to 91%, effectively solving the problem that the model ignores the retrieval information and relies on pre-trained parameters in traditional RAG.

[0028] Strong traceability and controllability: The traceability requirement mandates that each assertion be labeled with a source ID, and the generated results can be directly linked to specific search documents, facilitating manual review or automated verification. The storage module synchronously records the generation mode and search logs, greatly improving the usability of the AI ​​system in sensitive fields such as medicine and law.

[0029] Low cost and high flexibility: No need to modify the underlying parameters of large language models, no need to conduct large-scale RLHF training, hallucination suppression is achieved only through external retrieval, prompt word control and process optimization. It has low deployment cost, strong adaptability and can be flexibly applied to knowledge base scenarios in different fields.

[0030] Outstanding robustness: Threshold filtering eliminates retrieval noise, a general knowledge masking strategy resolves knowledge conflicts, and self-consistent decoding and majority voting mechanisms combat generation randomness. Even in complex query scenarios, it can maintain the logical coherence of the generated content. According to manual evaluation, the logical coherence score is 15% higher than that of traditional solutions.

[0031] Wide adaptability to various scenarios: By dynamically switching between strong and weak recall modes, it not only meets the factual accuracy requirements in high-confidence retrieval scenarios, but also adapts to the logical completion needs when the retrieval information is insufficient. It can be widely used in multiple fields such as medical Q&A, industrial fault diagnosis, and legal consultation.

[0032] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the vector index construction process of an AI model illusion suppression method and system based on retrieval enhancement generation and prompt word engineering collaboration, as described in this invention.

[0034] Figure 2 This is a flowchart illustrating the structured cue word construction process of an AI model hallucination suppression method and system based on retrieval enhancement generation and cue word engineering collaboration, as described in this invention.

[0035] Figure 3 This is a system module interaction architecture diagram of an AI model hallucination suppression method and system based on retrieval enhancement generation and prompt word engineering collaboration according to the present invention;

[0036] Figure 4 This is a sequence diagram of the overall method flow of the AI ​​model illusion suppression method and system based on the collaboration of retrieval enhancement generation and prompt word engineering of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0040] like Figure 1-4 As shown, the AI ​​model hallucination suppression system of the present invention achieves data interaction and functional connection according to the following logic, and the specific connection relationship is as follows:

[0041] The storage module establishes a one-way data transmission connection with the vector index building module. The storage module outputs domain knowledge base documents to the vector index building module. After the vector index building module completes document slicing, vector embedding and index building, it sends the generated vector index back to the storage module for persistent storage.

[0042] The vector index building module establishes a one-way data transmission connection with the retrieval filtering module. The vector index building module outputs a high-precision vector index to the retrieval filtering module to provide data support for the retrieval operation.

[0043] The user establishes a two-way data interaction connection with the search and filtering module. The user inputs a query request to the search and filtering module. After completing the search and noise filtering, the search and filtering module outputs the valid context to the subsequent modules.

[0044] The retrieval filtering module establishes a one-way data transmission connection with the prompt word generation module. The valid context output by the retrieval filtering module is used as the input data for the prompt word generation module to fill the structured prompt word template.

[0045] The prompt word generation module and the knowledge masking module establish a two-way data interaction connection. The prompt word generation module outputs the initial prompt word to the knowledge masking module, and the knowledge masking module feeds back optimization instructions to the prompt word generation module based on the recall quality assessment results.

[0046] The prompt word generation module and the decoding verification module establish a one-way data transmission connection, and the optimized prompt words output by the prompt word generation module are used as the input data for the decoding verification module.

[0047] The decoding and verification module establishes a two-way data interaction connection with the large language model. The decoding and verification module outputs optimized prompt words to the large language model and triggers inference sampling. The large language model returns multiple candidate answers to the decoding and verification module.

[0048] The decoding and verification module establishes a one-way data transmission connection with the output module, and the decoding and verification module outputs the filtered final result or a retry trigger signal to the output module.

[0049] The output module establishes bidirectional and unidirectional data connections with the user and storage modules respectively. The output module feeds back the final generated result to the user and outputs generation logs and retrieval logs containing source ID and generation mode to the storage module.

[0050] The output module establishes a one-way control connection with the retrieval and filtering module. When the decoding and verification module triggers the retry mechanism, the output module sends a retry command to the retrieval and filtering module, initiating a new round of retrieval, filtering, and decoding verification processes. In this implementation scheme,

[0051] In this implementation scheme, the present invention forms a closed-loop control mechanism through a combination of steps: "vector index construction - retrieval filtering - structured prompt word generation - general knowledge shielding - self-consistency decoding verification". The steps are not simply superimposed, but rather produce a technical effect of 1+1>2 through deep collaboration, as detailed below:

[0052] The core benefits of the overall process are: through the deep coupling of retrieval enhancement and prompt word engineering, combined with dynamic knowledge shielding and multi-path verification, a full-chain illusion suppression of "pre-guidance - in-process constraint - post-verification" is achieved without modifying the underlying parameters of large language models. While reducing deployment costs, the illusion rate is reduced to 4.2%-5.2%, the citation accuracy is increased to 91%-92.4%, and the logical coherence score is improved by 15%. This solves the pain point that existing technologies cannot balance illusion suppression effect, deployment cost, and scenario adaptability.

[0053] The function and specific benefits of each step:

[0054] Vector index construction steps: By accurately setting slicing parameters, selecting a high-performance embedding model, and optimizing HNSW algorithm parameters, a high-precision vector index library was constructed, which solved the problems of low retrieval accuracy and insufficient recall in traditional RAG technology, laid the foundation for subsequent effective context acquisition, significantly improved the matching degree between the retrieved relevant document fragments and the user's query intent, and provided a reliable source of facts for hallucination suppression;

[0055] Search filtering steps: By using cosine similarity calculation and a dual threshold filtering mechanism, low-relevance noisy documents are eliminated, which solves the problem of model misjudgment caused by noisy search results in traditional RAG technology. This ensures that the context information of the input model has high relevance and reliability, and reduces the logical contradiction illusion caused by noisy information.

[0056] Structured prompt generation steps: Through multi-level constraint design and special labeling, the model is forced to reason according to fixed logic and generate content based on the retrieval context. This solves the problems of rudimentary prompt design and "free play" of the model in traditional RAG technology, improves the utilization rate of retrieval information, and achieves the traceability of generated results through traceability requirements, which facilitates manual review and automated verification and meets the application needs of sensitive fields.

[0057] General knowledge shielding steps: Innovatively, the generation mode is dynamically switched based on recall quality. In the strong recall mode, general knowledge is shielded to avoid conflicts, while in the weak recall mode, knowledge fusion is allowed to complete the logic. This solves the core pain points of conflicts between general knowledge and domain facts and the model's over-reliance on pre-trained knowledge in existing technologies. It enables the model to strictly follow authoritative facts in high-confidence scenarios and to take into account the integrity of information in low-confidence scenarios.

[0058] Self-consistent decoding verification steps: By using multi-path sampling and majority voting mechanisms, the occasional illusions caused by random sampling of the model are filtered out, which solves the problem of strong randomness and insufficient reliability of the generated results of large language models. This ensures that the final output result is the optimal solution that has been verified through multiple rounds. At the same time, the retry mechanism further improves the robustness of the system and avoids erroneous output caused by single sampling deviation.

[0059] Through complementary functions and synergistic cooperation, each step comprehensively addresses the shortcomings of existing technologies from four core dimensions: fact sources, generation constraints, knowledge conflicts, and result reliability. This invention enables it to stably exert its hallucination suppression effect in factual question-and-answer scenarios in multiple fields such as medicine, industry, and law, and significantly improves the reliability and usability of content generated by large-scale language models.

[0060] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make various modifications and alterations without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the claims.

Claims

1. A method for suppressing hallucinations using an AI model based on the synergy of retrieval enhancement generation and cue word engineering, characterized in that, Includes the following steps: (1) Construct a high-precision domain vector index library: slice the domain knowledge base documents according to the preset slicing parameters, transform the slices into high-dimensional dense vectors through the Embedding model, construct the vector index using the specified indexing algorithm and store it in the vector database; (2) Retrieval enhancement and noise filtering: After receiving the user query, calculate the semantic similarity between the query vector and the document vector in the index, retrieve the Top-K relevant document fragments, and retain only the document fragments with similarity scores higher than the first threshold as valid context; (3) Structured prompt generation: Based on the preset template, construct structured prompts that include role definition, context injection, thought chain constraint, negation constraint and tracing requirements, and dynamically fill the context injection area of ​​the prompts with the effective context filtered in step (2); (4) General knowledge masking based on recall quality: Calculate the highest similarity score of the effective context and compare it with the second threshold. Switch the generation mode and inject the corresponding instruction according to the comparison result. If the highest similarity score is ≥ the second threshold, start the strong recall mode and the general knowledge masking instruction to prohibit the model from calling the pre-trained general knowledge. If the highest similarity score is lower than the second threshold but higher than the first threshold, start the weak recall mode to allow the model to combine the effective context and general knowledge to generate content and label the uncertainty. (5) Self-consistent decoding and illusion filtering: Perform N independent reasoning sampling on the structured prompt words generated in step (3) to obtain N candidate answers; eliminate the illegal answers that violate the negation constraint or the general knowledge shielding instruction, perform semantic clustering or keyword matching on the remaining candidate answers, and count the conclusions that appear most frequently; If the frequency of the most frequent conclusion is greater than or equal to the preset voting threshold, it will be used as the final output. Otherwise, trigger the retry mechanism or output "Uncertain".

2. The method according to claim 1, characterized in that, The preset slice parameters in step (1) are ChunkSize=512 tokens and Overlap=50 tokens; the Embedding model is bge-large-zh-v1.5 or OpenAItext-embedding-ada-002, and the generated high-dimensional dense vector has a dimension of 1024.

3. The method according to claim 1, characterized in that, The specified indexing algorithm in step (1) is the HNSW algorithm, and the algorithm parameters are set to ef_construction=200 and M=16; the vector database is FAISS or Pinecone.

4. The method according to claim 1, characterized in that, In step (2), the Top-K value is 5, the first threshold is 0.75, and the semantic similarity is calculated by the cosine similarity algorithm.

5. The method according to claim 1, characterized in that, The specific structure of the structured prompt words in step (3) is as follows: Role definition: "You are a meticulous intelligent assistant. Your task is to answer questions based on the provided known information. Fabrication is strictly prohibited." Context injection: using<context_info> The tag encloses the valid context and is in the format "<context_info> \n{Doc1}\n{Doc2}\n...< / context_info> ”; Mind chain constraint: "Please think according to the following steps:

1. Analyze"<context_info> 1. Identify the key information in the problem; 2. Identify the connection between the information and the problem; 3. Perform step-by-step logical reasoning; 4. Draw a conclusion. Negative constraint: "If<context_info> The answer is not provided; please answer directly with "I cannot answer based on the given information" and do not guess using your training data. Source tracing requirement: "Please indicate the document number cited with the [Source ID] after each statement in your answer." 6. The method according to claim 1, characterized in that, In step (4), the second threshold is 0.85; the general knowledge shielding instruction is: "System alarm: High relevance search results detected (similarity > 85%). [General knowledge shielding has been activated]. The current answer must strictly and only rely on the provided [context information]. It is absolutely forbidden to call or mix your model's pre-trained knowledge. Any details not appearing in the context are considered 'unknown' and cannot be deduced." 7. The method according to claim 1, characterized in that, In step (5), the Temperature parameter for inference sampling is set to 0.7, and N is 5; the preset voting threshold is 3.

8. The method according to claim 1, characterized in that, In step (5), semantic clustering adopts a cosine similarity-based clustering algorithm, and keyword matching adopts TF-IDF keyword extraction followed by similarity comparison.

9. The method according to any one of claims 1-8, characterized in that, The AI ​​model is a large-scale language model based on the Transformer architecture, including GPT-4 and Llama series models.

10. A hallucination suppression system based on the synergy of retrieval enhancement generation and cue word engineering using an AI model, characterized in that, include: Vector index building module: used to execute step (1) of any of the methods described in claims 1-9, and to complete the slicing, vector embedding and index building of the domain knowledge base; Search filtering module: used to perform step (2) of the method according to any one of claims 1-9, to realize similarity retrieval and noise filtering of user queries; Prompt word generation module: used to perform step (3) of the method according to any one of claims 1-9, constructing structured prompt words and filling in valid context; Knowledge shielding module: used to execute step (4) of the method described in any of claims 1-9, switch the generation mode based on the recall quality assessment result and inject the corresponding instruction; Decoding and verification module: used to perform step (5) of the method described in any one of claims 1-9, to realize multi-path sampling, candidate answer filtering and consistency verification; Output module: Used to output the final result obtained by the decoding and verification module, or to re-call the process from the retrieval and filtering module to the decoding and verification module when the retry mechanism is triggered. It also includes a storage module, which is used to store domain knowledge base documents, constructed vector indexes, retrieval logs and generated results, wherein the generated results include labeled source IDs and generation mode information.

Images