Large model self-checking-based method for relieving large language model illusion by RAG system
By using large-model self-checking technology in the RAG system, the comprehensive probability of hallucination and checking probability is solved, and the credibility of the answers and the reliability of the system is improved.
Patent Information
- Application Number
- CN202410958130.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-12
AI Technical Summary
Large language models are prone to hallucinations when generating answers, resulting in inaccurate and unreliable answers, and it is difficult for the existing technology to effectively solve this problem.
Using a method based on big model self-checking, the generated answers are checked by using the same large language model in the RAG system, the comprehensive probability of hallucination probability and inspection probability are calculated, and the credibility of the answer is judged based on the comparison results, and the credibility of the answer is adjusted. Generate model parameters to reduce hallucination risk.
It improves the quality and credibility of generated answers, reduces the risk of the system generating false answers, and enhances the reliability and user trust of the system.
Smart Images

Figure CN118940839A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of auxiliary retrieval technology, and more specifically, to a method for alleviating large language model hallucination in a RAG system based on large model self-verification. Background Art
[0002] The RAG system, whose full name is Result Augment Generation, is a system that uses auxiliary retrieval technology based on a large language model to enable the large language model to obtain more contextual information related to user questions, thereby reducing the probability of hallucinations, adapting to vertical scenario applications, and making up for the lack of real-time data.
[0003] A large language model, also known as a massive language model, is a deep learning model that learns the grammar, semantics, and contextual information of a language by analyzing large amounts of text data. Compared with traditional natural language processing models, large language models have stronger language generation and understanding capabilities and can handle more complex language problems such as text generation, dialogue systems, and machine translation. The emergence of large language models has greatly promoted the development of the field of natural language processing, enabling machines to better understand human language and apply it to various practical scenarios. Summary of the invention
[0004] In view of the technical problems existing in the prior art, the present invention provides a method for alleviating large language model hallucination in a RAG system based on large model self-verification, so as to solve the problems raised in the above background technology.
[0005] The technical solution of the present invention to solve the above technical problem is as follows: a method for alleviating large language model hallucination in a RAG system based on large model self-verification, specifically comprising the following steps:
[0006] Step 101: After the user inputs a question, the RAG system retrieves relevant information from the database and generates a prompt project to guide the large language model to generate an answer;
[0007] Step 102: Generate an answer using a large language model based on the generated prompt project, and verify the generated answer using the same large language model to determine whether it meets the requirements of the question and related information raised by the user;
[0008] Step 103: Calculate the hallucination probability and the inspection probability according to the verification result, and multiply the calculated results to obtain the comprehensive probability of the answer with hallucination, and compare the calculated comprehensive probability with the hallucination probability to determine whether the generated answer is credible.
[0009] In a preferred embodiment, in step 101, after the user inputs a question, the RAG system retrieves relevant information from the database and generates a prompt project to guide the large language model to generate an answer. The RAG system fills the knowledge gaps of the generated model by using the information in the database to help the model better understand the user's question and generate a more relevant and coherent answer. The specific steps are as follows:
[0010] Step A1: Question analysis: Analyze the question input by the user to determine the user's intention and needs;
[0011] Step A2, information retrieval: Based on the parsed user questions, relevant information is retrieved from a pre-established database by using a retrieval algorithm and keyword matching technology, and the retrieved relevant information is screened and organized to ensure that the extracted information is relevant to the user questions and is usable;
[0012] Step A3, generate a prompt project: Based on the filtered and organized information, generate a prompt project, which contains the user question and related information, and is used to guide the large language model to generate answers. The RAG system dynamically adjusts the prompt project according to different user inputs to meet different user information needs and question types;
[0013] Step A4, prompt engineering transfer: The generated prompt engineering is transferred to the large language model as a guide and basis for generating answers.
[0014] In a preferred embodiment, in step 102, based on the generated prompt project, a large language model is used to generate an answer, and the generated answer is verified using the same large language model to determine whether it meets the requirements of the question and related information raised by the user. The specific steps are as follows:
[0015] Step B1, generate answer: Based on the generated prompt project as input, provide it to the large language model, and use the large language model to generate a preliminary answer;
[0016] Step B2, answer verification: Use the same large language model to verify the generated answers. By inputting the generated answers into the model, the quality of the answers is checked to ensure that the generated answers are consistent with the prompt project in terms of style, grammar, and logic, thereby improving the coherence and quality of the answers.
[0017] Step B3, Verification Indicator Definition: Define indicators used to verify the quality of answers, including semantic consistency, logical rationality, and information accuracy;
[0018] Step B4, verification result evaluation: Analyze the verification results of the large language model on the generated answers, and evaluate whether the answers meet the requirements of the questions and related information raised by the users based on the verification indicators. The generated answers have undergone double verification and validation, and are more in line with the questions and needs raised by users, thereby improving users' satisfaction and trust in the system.
[0019] In a preferred embodiment, in step 103, the hallucination probability and the inspection probability are calculated according to the verification result, and the calculated results are multiplied to obtain the comprehensive probability of the answer with hallucination, and the calculated comprehensive probability is compared with the hallucination probability to determine whether the generated answer is credible. By calculating the comprehensive probability, the information of the hallucination probability and the inspection probability is combined, and the credibility of the answer can be more comprehensively evaluated, and the degree of trust in the answer can be improved. The specific steps are as follows:
[0020] Step C1: Calculate the probability of hallucination: The proportion of hallucinations generated by the statistical model on the data set is expressed as P 1 , the hallucination probability represents the possibility that the answer is a hallucination;
[0021] Step C2: Check probability calculation: This is calculated by measuring the accuracy of the model in detecting hallucinations. Using a set of labeled data sets, the accuracy of the model in detecting hallucinations is expressed as P 2 , the inspection probability represents the possibility that the answer passes the inspection;
[0022] Step C3, comprehensive probability calculation: The calculated hallucination probability P 1 and the inspection probability P 2 The two are multiplied to obtain the comprehensive probability P, which represents the relationship between the probability of hallucination when the large language model generates answers and the probability of hallucination being detected in the inspection stage. The specific calculation formula is as follows:
[0023] P=P 1 ×P 2
[0024] Among them, P 1 represents the probability of hallucination, P 2 represents the inspection probability, and P represents the comprehensive probability.
[0025] In a preferred embodiment, the calculated comprehensive probability is compared with the hallucination probability to determine whether the generated answer is credible. Calculating and comparing the comprehensive probability of the generated answer can effectively reduce the risk of the system generating false answers and improve the reliability and security of the system. The specific steps are as follows:
[0026] Step D1: Determine the credibility of the answer: When P < P 1, indicating that the comprehensive probability is less than the hallucination probability, and the generated answer is credible; when P ≥ P 1 , indicating the existence of hallucinations, and judging that the generated answer is unreliable;
[0027] Step D2, output result: Based on the judgment result, output the credibility of the generated answer. When the generated answer is judged to be unreliable, regenerate the answer by adjusting the generation model parameters.
[0028] In a preferred embodiment, the step C2 checks the probability calculation by measuring the accuracy of the model in detecting hallucinations. Using a set of labeled data sets, the accuracy of the model in detecting hallucinations is expressed as P 2 , further comprising the following steps:
[0029] Step C201, labeling dataset: create a labeling dataset containing known hallucination cases and their correct answers;
[0030] Step C202, model training: train the model on this dataset to evaluate its performance in detecting hallucinated answers;
[0031] Step C203, accuracy calculation: Calculate the accuracy of the model in detecting the labeled hallucination cases. The specific calculation formula is as follows:
[0032]
[0033] Among them, P 2 represents the inspection probability, T rue P is the number of cases that the model correctly identifies as hallucinations, T otal P i is the total number of predictions made by the model over all hallucination cases.
[0034] The beneficial effects of the present invention are as follows: after a user inputs a question, the RAG system retrieves relevant information from a database and generates a prompting project for guiding a large language model to generate an answer; based on the generated prompting project, the large language model is used to generate an answer; and the generated answer is verified using the same large language model to determine whether it meets expectations; based on the verification result, the hallucination probability and the inspection probability are calculated, and the calculated results are multiplied to obtain a comprehensive probability of an answer with a hallucination; and the calculated comprehensive probability is compared with the hallucination probability to determine whether the generated answer is credible; when the generated answer is determined to be unreliable, the answer is regenerated by adjusting the generation model parameters; a method for alleviating large language model hallucinations in a RAG system based on large model self-verification, which alleviates the large language model hallucination problem, improves the quality and credibility of generated answers, and the robustness of the system, optimizes user experience, and reduces risks and costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0037] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.
[0038] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.
[0039] Example 1
[0040] This embodiment provides Figure 1 A method for alleviating large language model hallucination in a RAG system based on large model self-verification is shown, which specifically includes the following steps:
[0041] Step 101: After the user inputs a question, the RAG system retrieves relevant information from the database and generates a prompt project to guide the large language model to generate an answer;
[0042] Step 102: Generate an answer using a large language model according to the generated prompt project, and verify the generated answer using the same large language model to determine whether it meets expectations;
[0043] Step 103: Calculate the hallucination probability and the inspection probability according to the verification result, and multiply the calculated results to obtain the comprehensive probability of the answer with hallucination, and compare the calculated comprehensive probability with the hallucination probability to determine whether the generated answer is credible.
[0044] Preferably, in step 101, after the user inputs a question, the RAG system retrieves relevant information from the database and generates a prompting project to guide the large language model to generate an answer. The RAG system fills the knowledge gaps of the generated model by using the information in the database to help the model better understand the user's question and generate a more relevant and coherent answer. The specific steps are as follows:
[0045] Step A1, question analysis: Analyze the questions input by the user to determine the user's intention and needs, avoid giving the user an unsatisfactory experience, and enhance the user's satisfaction with the service;
[0046] Step A2, information retrieval: Based on the parsed user questions, relevant information is retrieved from a pre-established database by using a retrieval algorithm and keyword matching technology, and the retrieved relevant information is screened and organized to ensure that the extracted information is relevant to the user questions and is usable;
[0047] Step A3, generate a prompt project: Based on the filtered and organized information, generate a prompt project, which contains the user question and related information, and is used to guide the large language model to generate answers. The RAG system dynamically adjusts the prompt project according to different user inputs to meet different user information needs and question types;
[0048] Step A4, prompt engineering transfer: The generated prompt engineering is transferred to the large language model as a guide and basis for generating answers.
[0049] Preferably, in step 102, according to the generated prompt project, a large language model is used to generate an answer, and the generated answer is verified using the same large language model to determine whether it meets the requirements of the question and related information raised by the user. The specific steps are as follows:
[0050] Step B1, generate answer: Based on the generated prompt project as input, provide it to the large language model, and use the large language model to generate a preliminary answer;
[0051] Step B2, answer verification: Use the same large language model to verify the generated answers. By inputting the generated answers into the model, the quality of the answers is checked to ensure that the generated answers are consistent with the prompt project in terms of style, grammar, and logic, thereby improving the coherence and quality of the answers.
[0052] Step B3, Verification Indicator Definition: Define indicators used to verify the quality of answers, including semantic consistency, logical rationality, and information accuracy;
[0053] Step B4, verification result evaluation: Analyze the verification results of the large language model on the generated answers, and evaluate whether the answers meet the requirements of the questions and related information raised by the users based on the verification indicators. The generated answers have undergone double verification and validation, and are more in line with the questions and needs raised by users, thereby improving users' satisfaction and trust in the system.
[0054] Preferably, in step 103, according to the verification result, the hallucination probability and the inspection probability are calculated, and the calculated results are multiplied to obtain the comprehensive probability of the answer with hallucination, and the calculated comprehensive probability is compared with the hallucination probability to determine whether the generated answer is credible. The specific steps are as follows:
[0055] Step C1: Calculate the probability of hallucination: The proportion of hallucinations generated by the statistical model on the data set is expressed as P 1 The hallucination probability indicates the possibility that the answer is hallucinated. The hallucination probability calculation helps to evaluate the risk of the statistical model generating hallucinations on the data set, avoids users' excessive trust in the model output, and reduces the spread of misleading information and decision-making errors caused by hallucinations.
[0056] Step C2: Check probability calculation: This is calculated by measuring the accuracy of the model in detecting hallucinations. Using a set of labeled data sets, the accuracy of the model in detecting hallucinations is expressed as P 2 , the inspection probability represents the possibility that the answer passes the inspection;
[0057] Step C3, comprehensive probability calculation: The calculated hallucination probability P 1 and the inspection probability P 2 The two are multiplied to obtain the comprehensive probability P, which represents the relationship between the probability of hallucination when the large language model generates answers and the probability of hallucination being detected in the inspection stage. The specific calculation formula is as follows:
[0058] P=P 1 ×P 2
[0059] Among them, P 1 represents the hallucination probability, which is the probability of providing wrong information when the large language model generates an answer, P 2 represents the inspection probability, which represents the probability that the hallucination can be correctly detected in the verification stage, and P represents the comprehensive probability.
[0060] Preferably, the calculated comprehensive probability is compared with the hallucination probability to determine whether the generated answer is credible. Calculating and comparing the comprehensive probability of the generated answer can effectively reduce the risk of the system generating false answers and improve the reliability and security of the system. The specific steps are as follows:
[0061] Step D1: Determine the credibility of the answer: When P < P 1 , indicating that the comprehensive probability is less than the hallucination probability, and the generated answer is credible; when P ≥ P 1 , indicating the existence of hallucinations, and judging that the generated answer is unreliable;
[0062] Step D2, output result: Based on the judgment result, output the credibility of the generated answer. When the generated answer is judged to be unreliable, regenerate the answer by adjusting the generation model parameters.
[0063] Preferably, in the calculation of the inspection probability in step C2, the accuracy of the model in detecting hallucinations is measured, and a set of labeled data sets is used to express the accuracy of the model in detecting hallucinations as P 2 , further comprising the following steps:
[0064] Step C201, labeling dataset: create a labeling dataset containing known hallucination cases and their correct answers;
[0065] Step C202, model training: train the model on this dataset to evaluate its performance in detecting hallucinated answers;
[0066] Step C203, accuracy calculation: Calculate the accuracy of the model in detecting the labeled hallucination cases, and evaluate the performance of the model in identifying hallucination cases.
[0067] In this embodiment, it should be specifically explained that the specific calculation formula of the accuracy of the model in detecting the marked hallucination cases is as follows:
[0068]
[0069] Among them, P 2 represents the inspection probability, T rue P is the number of cases that the model correctly identifies as hallucinations, T otal P i is the total number of predictions made by the model over all hallucination cases.
[0070] In this embodiment, it should be specifically noted that the performance of the model in generating hallucinations is affected by the bias of the data set. If the number of hallucination cases in the data set is small, the model will tend to generate fewer hallucinations, even if there are actually more hallucinations.
[0071] The hallucination probability generated by the model only represents the model's confidence in the existence of the hallucination, rather than the true probability of existence. The hallucination probability generated by the model cannot be directly interpreted as the absolute probability that the answer is a hallucination.
[0072] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
[0073] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0074] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0075] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0076] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0079] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0080] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for alleviating large language model hallucination in a RAG system based on large model self-verification, characterized in that: The specific steps include: Step 101: After the user inputs a question, the RAG system retrieves relevant information from the database and generates a prompt project to guide the large language model to generate an answer; Step 102: Generate an answer using a large language model according to the generated prompt project, and verify the generated answer using the same large language model to determine whether it meets expectations; Step 103: Calculate the hallucination probability and the inspection probability according to the verification result, and multiply the calculated results to obtain the comprehensive probability of the answer with hallucination, and compare the calculated comprehensive probability with the hallucination probability to determine whether the generated answer is credible.
2. According to claim 1, a method for alleviating large language model hallucination in a RAG system based on large model self-verification is characterized by: In step 101, after the user inputs a question, the RAG system retrieves relevant information from the database and generates a prompt project to guide the large language model to generate an answer. The specific steps are as follows: Step A1: Question analysis: Analyze the question input by the user to determine the user's intention and needs; Step A2, information retrieval: according to the parsed user questions, relevant information is retrieved from a pre-established database by using a retrieval algorithm and keyword matching technology, and the retrieved relevant information is screened and organized; Step A3, generating a prompt project: Based on the filtered and organized information, a prompt project is generated, which includes the user question and related information, and is used to guide the large language model to generate answers; Step A4, prompt engineering transfer: The generated prompt engineering is transferred to the large language model as a guide and basis for generating answers.
3. According to claim 1, a method for alleviating large language model hallucination in a RAG system based on large model self-verification is characterized in that: In step 102, according to the generated prompt project, a large language model is used to generate an answer, and the generated answer is verified using the same large language model to determine whether it meets expectations. The specific steps are as follows: Step B1, generate answer: Based on the generated prompt project as input, provide it to the large language model, and use the large language model to generate a preliminary answer; Step B2, answer verification: Use the same large language model to verify the generated answer, and check the quality of the answer by inputting the generated answer into the model; Step B3, Verification Indicator Definition: Define indicators used to verify the quality of answers, including semantic consistency, logical rationality, and information accuracy; Step B4, verification result evaluation: Analyze the verification results of the large language model on the generated answer, and evaluate whether the answer meets the requirements of the question raised by the user and the relevant information based on the verification indicators.
4. According to claim 1, a method for alleviating large language model hallucination in a RAG system based on large model self-verification is characterized in that: In step 103, according to the verification result, the hallucination probability and the inspection probability are calculated, and the calculated results are multiplied to obtain the comprehensive probability of the answer with hallucination, and the calculated comprehensive probability is compared with the hallucination probability to determine whether the generated answer is credible. The specific steps are as follows: Step C1, hallucination probability calculation: the proportion of hallucinations generated by the statistical model on the data set is expressed as P1; Step C2, check probability calculation: calculated by measuring the accuracy of the model in detecting hallucinations, using a set of labeled data sets, and denoting the accuracy of the model in detecting hallucinations as P2; Step C3, comprehensive probability calculation: Multiply the calculated hallucination probability P1 and the inspection probability P2 to obtain the comprehensive probability P. The specific calculation formula is as follows: P=P1×P2 Among them, P1 represents the probability of hallucination, P2 represents the probability of inspection, and P represents the comprehensive probability.
5. According to claim 4, a method for alleviating large language model hallucination in a RAG system based on large model self-verification is characterized in that: The calculated comprehensive probability is compared with the hallucination probability to determine whether the generated answer is credible. The specific steps are as follows: Step D1, judging the credibility of the answer: when P < P1, it means that the comprehensive probability is less than the hallucination probability, and the generated answer is judged to be credible; When P ≥ P1, it means that hallucination exists and the generated answer is judged to be unreliable; Step D2, output result: Based on the judgment result, output the credibility of the generated answer. When the generated answer is judged to be unreliable, regenerate the answer by adjusting the generation model parameters.
6. According to claim 4, a method for alleviating large language model hallucination in a RAG system based on large model self-verification is characterized in that: In the calculation of the inspection probability in step C2, the accuracy of the model in detecting hallucinations is measured and calculated, and a set of labeled data sets are used to represent the accuracy of the model in detecting hallucinations as P2, and further includes the following steps: Step C201, labeling dataset: create a labeling dataset containing known hallucination cases and their correct answers; Step C202, model training: train the model on this dataset to evaluate its performance in detecting hallucinated answers; Step C203, accuracy calculation: Calculate the accuracy of the model in detecting the labeled hallucination cases. The specific calculation formula is as follows: Among them, P2 represents the inspection probability, T rue P is the number of cases that the model correctly identifies as hallucinations, T otal P i is the total number of predictions made by the model over all hallucination cases.
Citation Information
Cited By
Automobile intelligent question-answering system and method based on large model and retrieval enhancement
CN119202206A
System for monitoring and intervening large model to generate illusion content in real time
CN119357759A
Error positioning method for retrieval enhancement generation system, electronic equipment and medium
CN119474276A
Fine-grained large model illusion processing method and system based on zero resource framework
CN119670760A
Electronic control parameter intelligent verification method and system based on multi-mode AI fusion
CN120353695A
