Large model illusion relieving method and device, equipment and storage medium

By introducing the thinking chain reasoning process and evidence set correction mechanism in the generative model, the problems of insufficient accuracy in complex problems and external illusions in RAG technology are solved, and the accuracy and reliability of the generated content of large language models are improved. .

CN118964583AInactive Publication Date: 2024-11-15SHENZHEN FARBEN INFORMATION TECH CO LTD

Patent Information

Application Number
CN202411427263.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-14
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When traditional generative models deal with complex, knowledge-intensive or time-sensitive problems, it is difficult to accurately capture details, resulting in insufficient accuracy and detail of generated answers. The introduced RAG technology has external hallucinations, resulting in reliability of generated content. Decreased in sexuality and accuracy.

Method used

By introducing a chain of thought (CoT) reasoning process, initial answers are generated and hallucinatory judgments are made, potential errors are corrected using evidence sets, and RAG technology is combined to improve the accuracy and reliability of generated content.

Benefits of technology

It significantly improves the accuracy and reliability of generated content, reduces hallucinatory problems caused by internal errors or knowledge blind spots of the model, and reduces the frequency of external retrieval, maintains the advantages of information retrieval, and improves the richness of answers to complex questions and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964583A_ABST
    Figure CN118964583A_ABST
Patent Text Reader

Abstract

The invention discloses a large model illusion relieving method and device, equipment and a storage medium, and the method comprises the steps: generating an initial answer and a corresponding reasoning process according to initial input information, and enabling the reasoning process to be obtained through employing a thinking chain prompt word to extract a generation process of the initial answer; performing illusion judgment on an initial question and answer pair to obtain an illusion judgment result, the initial question and answer pair including the initial input information and the initial answer; and when the illusion judgment result is that illusion possibly exists, performing illusion correction on the initial answer through an evidence set in combination with the reasoning process, and outputting correction content. According to the method, the illusion is reduced by optimizing the RAG technology, so that the reliability and the accuracy of large language model generated content are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a method, device, equipment and storage medium for alleviating large model hallucinations. Background Art

[0002] In traditional text generation tasks, generative models mainly rely on patterns that directly learn and generate text from training data. This model often achieves good results when dealing with simple, routine text tasks. However, when faced with more complex, knowledge-intensive, or time-sensitive problems, its limitations gradually become apparent. Complex problems often require the integration of knowledge from multiple fields and a deep understanding of background information, and traditional generative models often find it difficult to accurately capture these details due to the lack of external information supplementation, resulting in the lack of accuracy and detail in the generated answers.

[0003] To overcome this challenge, RAG (Retrieval Augmented Generation) technology was introduced. Combining the advantages of information retrieval, it enriches the input of the generative model by retrieving relevant information from an external knowledge base, thereby improving the quality and accuracy of its output. Under the RAG framework, retrieval models (such as BM25, DPR, etc.) are first used to filter out documents or fragments that are highly relevant to the input question from a large knowledge base. Subsequently, this retrieved information is integrated into the input of the generative model as additional knowledge support, allowing the model to refer to a wider and deeper source of information when generating answers.

[0004] Although RAG technology has alleviated the "hallucination" problem of large language models to a certain extent, that is, the erroneous or misleading output produced by the model when there is a lack of sufficient information or understanding bias, it also brings a new challenge - external hallucinations. This external hallucination mainly stems from the uncertainty of the retrieval process. Since the information in the external knowledge base is complex and not all information is directly related to the user's question, the retrieval model will inevitably introduce some noise or irrelevant information when screening information. When this irrelevant information is provided to the large language model for processing, the model may incorrectly integrate it into the generated answer, resulting in inaccurate, irrelevant or even misleading content in the answer.

[0005] Specifically, external hallucinations can manifest in various forms. Sometimes, the model may introduce incorrect facts or opinions in the answer because it retrieves information that is close to the question topic but does not match the details. Sometimes, the model may use irrelevant information retrieved as useful information, causing the answer to deviate from the core of the question. The existence of such external hallucinations not only reduces the reliability and accuracy of the content generated by the large language model, but also brings unnecessary trouble and misleading to users. Summary of the invention

[0006] The main purpose of this application is to provide a large model hallucination mitigation method, device, equipment and storage medium, aiming to optimize RAG technology to improve the reliability and accuracy of content generated by large language models.

[0007] To achieve the above object, the present application provides a large model hallucination mitigation method, the large model hallucination mitigation method comprising the following steps: Generate an initial answer and a corresponding reasoning process according to the initial input information, wherein the reasoning process is obtained by extracting the generation process of the initial answer using a thinking chain prompt word; Performing hallucination judgment on an initial question-answer pair to obtain a hallucination judgment result, wherein the initial question-answer pair includes the initial input information and the initial answer; When the hallucination judgment result is that a hallucination may exist, the initial answer is hallucinated and corrected by combining the evidence set with the reasoning process, and the corrected content is output.

[0008] In one embodiment, the step of performing hallucination judgment on the initial question-answer pair and obtaining the hallucination judgment result includes: Generating a target language variant question-answer pair of the initial question-answer pair, and calculating a consistency score between the initial question-answer pair and the target language variant question-answer pair; If the consistency score is lower than a first preset threshold, the hallucination judgment result is that a hallucination may exist.

[0009] In one embodiment, the step of generating a target language variant question-answer pair of the initial question-answer pair and calculating a consistency score between the initial question-answer pair and the target language variant question-answer pair comprises: Generating source variant input information semantically consistent with the initial input information, and translating the source variant input information into target language variant input information; Inputting the target language variant input information into the big model, generating a target language variant answer, and obtaining a target language variant question-answer pair; A consistency score is calculated between the initial question-answer pair and the target language variant question-answer pair.

[0010] In one embodiment, the step of performing hallucination judgment on the initial question-answer pair and obtaining the hallucination judgment result further includes: Predicting the token in the initial answer to obtain the entropy value of the token in the initial answer; Obtaining an uncertainty score of the initial answer using the entropy value of the token; If the uncertainty score is higher than a second preset threshold, the hallucination judgment result is that a hallucination may exist.

[0011] In one embodiment, the step of performing hallucination judgment on the initial question-answer pair and obtaining the hallucination judgment result further includes: Inputting the initial input information and the preset prompt words into the large model to generate a response result; Performing uncertainty analysis on the response result to obtain an uncertainty score of the response result; If the uncertainty score is higher than a third preset threshold, the hallucination judgment result is that a hallucination may exist.

[0012] In one embodiment, when the hallucination judgment result is that a hallucination may exist, the step of performing hallucination correction on the initial answer by combining the evidence set with the reasoning process and outputting the correction content further includes: Retrieve external knowledge sources to collect a set of evidence; Using the evidence set, reviewing the reasoning process and the initial answer to screen for potential errors; Based on the evidence set, the potential error is corrected and a correction content is output.

[0013] In one embodiment, when the hallucination judgment result is that a hallucination may exist, after the step of performing hallucination correction on the initial answer by combining the evidence set with the reasoning process and outputting the correction content, the step further includes: When the hallucination judgment result is that there is no hallucination, an initial answer is output.

[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a large model hallucination relief device, the large model hallucination relief device comprising: A first acquisition module is used to generate an initial answer and a corresponding reasoning process according to initial input information, wherein the reasoning process is obtained by extracting the generation process of the initial answer using a thinking chain prompt word; A second acquisition module is used to perform hallucination judgment on an initial question-answer pair and obtain a hallucination judgment result, wherein the initial question-answer pair includes the initial input information and the initial answer; The correction module is used to correct the initial answer to the hallucination by combining the evidence set with the reasoning process and output the correction content when the hallucination judgment result is that the hallucination may exist.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a terminal device, which includes a memory, a processor, and a large model hallucination mitigation program stored in the memory and executable on the processor, and the large model hallucination mitigation program, when executed by the processor, implements the steps of the large model hallucination mitigation method as described above.

[0016] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a large model hallucination mitigation program is stored. When the large model hallucination mitigation program is executed by a processor, the steps of the large model hallucination mitigation method as described above are implemented.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: By introducing the Chain of Thought (CoT) reasoning process, the accuracy and reliability of the generated content are significantly improved. Specifically, before generating the initial answer, this application first uses the chain of thought capability of the large language model to generate a detailed thought process through step-by-step reasoning. This process not only improves the accuracy of the answer, but also enhances the interpretability of the answer.

[0018] Compared with the prior art, the technical solution of the present application has achieved remarkable results in the following aspects: First, through the thought chain reasoning process, it effectively reduces the illusion problems caused by internal errors or knowledge blind spots in the model, and improves the accuracy and detail of the generated answers; Second, the technical solution of the present application corrects the internal illusion by judging in advance that the external knowledge source will be retrieved only when there is an internal illusion, thereby reducing the frequency of external retrieval and the occurrence of external illusions. At the same time, it also maintains the original information retrieval advantage of RAG technology, so that the model can make full use of the relevant information in the external knowledge base when generating answers to complex, knowledge-intensive or time-sensitive questions, thereby further improving the richness and accuracy of the generated content. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a first exemplary embodiment of a large model hallucination mitigation method of the present application; Figure 2 A flowchart of a second exemplary embodiment of the large model hallucination mitigation method of the present application; Figure 3 This is a schematic diagram of the module structure of a large-scale model hallucination relief device according to an embodiment of the present application; Figure 4 Schematic diagram of the device structure of the hardware operating environment involved in the large model hallucination mitigation method in the embodiment of the present application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] The main solution of the embodiment of the present application is: generate an initial answer and a corresponding reasoning process according to the initial input information, the reasoning process uses a thinking chain prompt word to extract the generation process of the initial answer; perform hallucination judgment on the initial question and answer pair to obtain the hallucination judgment result, the initial question and answer pair includes the initial input information and the initial answer; when the hallucination judgment result is that a hallucination may exist, perform hallucination correction on the initial answer through the evidence set combined with the reasoning process and output the correction content. Compared with the technical problems of insufficient accuracy and detail that the generative model is prone to when dealing with complex, knowledge-intensive or time-sensitive problems, based on the solution of this application, by introducing the thinking chain reasoning process and the multilingual detection module, and combining the RAG technology, the accuracy and reliability of the generated content are improved.

[0023] Technical terms involved in the embodiments of this application: Chain of Thought (CoT): is a method for improving the reasoning ability of large language models. This method guides the model to show detailed reasoning steps when answering questions, similar to the thinking process of humans when solving problems. These steps are logical and coherent, which helps the model better understand the complexity of the problem and provide more accurate answers.

[0024] Large Language Model (LMM): refers to machine learning models with large parameters and capable of processing natural language text. These models usually learn the statistical laws and patterns of language by training on large corpora. Large language models are able to perform a variety of natural language processing tasks, such as text generation, text classification, language translation, etc. Their capabilities increase as the model size increases, and they are able to handle more complex and diverse language tasks.

[0025] Retrieval Augmented Generation (RAG): is a technical method that combines information retrieval and text generation. RAG technology introduces the ability of information retrieval. It first uses the retrieval model to retrieve documents related to the input question from the external knowledge base, and then feeds these documents into the generation model together with the original input to enhance the output of the generation model. This method can alleviate the hallucination problem of large language models to a certain extent and improve the reliability and accuracy of generated content.

[0026] The executor of the method of this embodiment can be a large model illusion mitigation device, or a large model illusion mitigation terminal device or server. This embodiment takes a large model illusion mitigation device as an example. The large model illusion mitigation device can be integrated in terminal devices such as smart phones and tablet computers with data processing functions.

[0027] Specifically, the following are the detailed steps of an exemplary embodiment of the large model hallucination mitigation method: Reference Figure 1 , Figure 1 This is a flow chart of the first exemplary embodiment of the large model hallucination mitigation method of the present application. In this embodiment, the large model hallucination mitigation method includes steps S10 to S30: Step S10, generating an initial answer and a corresponding reasoning process according to the initial input information, wherein the reasoning process is obtained by extracting the generation process of the initial answer using thought chain prompt words.

[0028] Specifically, we first construct an initial input information to ensure that it is a complex problem that requires reasoning analysis to answer. For example, the question can be: "In a bag consisting of three colors of balls, red, yellow, and blue, how many different color combinations can there be when two balls are randomly taken out?" Then, construct a suitable chain of thought prompt for the LLM. This prompt should guide the model to analyze the problem step by step and show its reasoning process. For the above example, you can set a prompt like: "Please list your thinking steps in detail, first analyze the colors of the balls in the bag, and then consider all possible color combinations of two balls." Next, the constructed thinking chain prompts are input to the LLM together with the initial question. After receiving the input, the LLM will use its parameterized knowledge and strong reasoning ability to analyze the problem step by step and generate detailed reasoning steps according to the requirements of the thinking chain prompts. For the above problem, the LLM may first list the colors of the balls in the bag (red, yellow, and blue), and then analyze all possible combinations of two balls (red-red, red-yellow, red-blue, yellow-yellow, yellow-blue, and blue-blue), and exclude repeated or non-conforming combinations (such as red-red, yellow-yellow, and blue-blue, because the question requires different color combinations).

[0029] After that, the complete reasoning process is extracted from the output of the LLM. Based on the extracted reasoning process, the LLM can be asked to provide a concise and direct answer. For the above question, the answer will be: "There are three different ways to combine colors: red and yellow, red and blue, and yellow and blue." For example, for the above question about the combination of colored balls, by setting appropriate thinking chain prompts for LLM, the following reasoning process and initial answer are obtained: Reasoning process: First, we know that there are three colors of balls in the bag: red, yellow, and blue. Next, consider all possible color combinations of two balls. Since the question requires different color combinations, we can rule out the case where the two balls have the same color (such as red-red, yellow-yellow, blue-blue). Therefore, the remaining combinations are: red-yellow, red-blue, and yellow-blue. These are all possible different color combinations.

[0030] Initial answer: There are three different color combinations: red and yellow, red and blue, and yellow and blue.

[0031] Step S20, performing hallucination judgment on the initial question-answer pair to obtain a hallucination judgment result, wherein the initial question-answer pair includes the initial input information and the initial answer.

[0032] In a feasible implementation, step S20 may include steps A1-A2: Step A1, generating a target language variant question-answer pair of the initial question-answer pair, and calculating a consistency score between the initial question-answer pair and the target language variant question-answer pair; In a feasible implementation, step A1 may include steps A11 to A13: Step A11, generating source variant input information semantically consistent with the initial input information, and translating the source variant input information into target language variant input information; It should be noted that LLM is first used to generate multiple source variant inputs that are semantically consistent with the initial input information. The purpose of this step is to increase the diversity of input questions so that the accuracy of the model's response can be more comprehensively evaluated in subsequent steps.

[0033] Specifically, the LLM is provided with initial input information and given explicit instructions to generate k source variant input information that is highly similar in semantics but slightly different in expression. In order to further enhance the diversity of input problems and introduce changes at the language level, the source variant input information generated in step A11 is further translated into different target language variant inputs. This step also utilizes the cross-language capabilities of the LLM by giving explicit translation instructions to require the model to translate each source variant input information into a specified target language. For example, if the initial input information is in Chinese, you can choose to translate the source variant input information into English, Japanese, or French, etc., to simulate input situations in different language environments.

[0034] It should be noted that after the target language variant input is generated and translated, these variants are screened to ensure that they are semantically consistent with the initial input. A semantic equivalence checking mechanism can be designed to use LLM to determine whether two input information (i.e., the original input information and a variant input information) are semantically equivalent. Specifically, two input information are provided to LLM, and a prompt is given: "Are the following two input information semantically equivalent?" Then, according to the response of LLM (usually "true" or "false"), those variant inputs that are semantically inconsistent with the original input are filtered out. After this step, a set of variant input information that is highly semantically consistent with the initial input and contains language-level changes can be obtained. These variant input information will be used to evaluate the consistency and accuracy of the model response in subsequent steps.

[0035] Step A12, inputting the target language variant input information into the large model, generating a target language variant answer, and obtaining a target language variant question-answer pair; Specifically, the large model is prompted to generate a unique response for each variant input, and for each target language variant input information, a corresponding target language variant answer is generated, thereby obtaining multiple target language variant question and answer pairs, each question and answer pair consisting of a variant input and a variant answer. In addition, an initial question and answer pair is obtained based on the initial input and initial answer of step S10.

[0036] Step A13, calculating the consistency score of the initial question-answer pair and the target language variant question-answer pair.

[0037] It should be noted that the semantic equivalence check operator C is used to obtain the semantic equivalence between different question-answer pairs. This operator accepts two question-answer pairs as input and compares their semantic equivalence. If the two question-answer pairs are semantically equivalent, C returns "true" (corresponding to the value 1.0); otherwise, it returns "false" (corresponding to the value 0.0).

[0038] Specifically, for each variant question-answer pair (x s , r s ), compare it with the initial question-answer pair (x0, r0), and use the C operator to calculate the semantic equivalence between them. Then, each comparison result is mapped to a numerical score, and the weighted average of the scores between all variant question-answer pairs and the initial question-answer pair is calculated to obtain a multilingual consistency score. This score reflects the stability and consistency of the initial answer r0 under different representations.

[0039] The mathematical formula for the above consistency score can be expressed as:

[0040] Among them, Z is the consistency score, C is the semantic equivalence check operator, and P j s and P j t are two question-answer pairs input to C.

[0041] For example, suppose the original query x0 is “What is artificial intelligence?”, and the initial answer r0 generated by LLM is “Artificial intelligence is a technology that simulates human intelligence”. Subsequently, two variant inputs x0 and r1 are generated. s1 and x s2 , through semantic equivalence checking, we find that x s1 is semantically equivalent to x0, but x s2 It is not semantically consistent with x0, so x is filtered out. s2 . Get the variant answer r s1, for variant question-answer pairs (x s1 , r s1 ), calculate the consistency score between it and the initial question-answer pair (x0, r0), and use it to evaluate the accuracy and reliability of the initial answer r0.

[0042] Step A2: If the consistency score is lower than a first preset threshold, the hallucination judgment result is that a hallucination may exist.

[0043] It should be noted that, in this embodiment, the calculation of the consistency score is intended to evaluate and determine whether the preliminary answer contains hallucination content. If the above consistency score is lower than the preset threshold, it is considered that the initial answer may contain hallucinations and subsequent hallucination correction is required.

[0044] In another feasible implementation, step S20 may include steps B1 to B3: Step B1, predicting the token in the initial answer and obtaining the entropy value of the token in the initial answer; Specifically, the initial answer is first decomposed into a series of tokens, which can be words, characters, or subword units, depending on the word segmentation method used. For example, if the initial answer r0 is "Apple is a fruit", it can be decomposed into ["apple", "is", "a", "fruit"].

[0045] Subsequently, LLM is used to predict each token. During the prediction process, the model not only outputs the probability distribution of each token, but also calculates the entropy value of each token based on this distribution. Entropy is an indicator of information uncertainty. A high entropy value indicates that the prediction result of the token has a high uncertainty.

[0046] Taking the token "apple" as an example, the model may predict multiple candidate words, but in theory, if the model's prediction result for a certain token is not certain enough, that is, there are multiple candidate words with similar probabilities, then the entropy value of the token will be higher.

[0047] Step B2, obtaining an uncertainty score of the initial answer using the entropy value of the token; After obtaining the entropy value of each token, it is necessary to aggregate or average them to obtain the uncertainty score of the entire initial answer. The specific aggregation or averaging method can be selected according to the actual situation. For example, the entropy values ​​of all tokens can be directly arithmetic averaged, or a weighted average method can be used, where the weight can be set according to factors such as the importance and position of the token.

[0048] Step B3: If the uncertainty score is higher than a second preset threshold, the hallucination judgment result is that a hallucination may exist.

[0049] Finally, the calculated uncertainty score is compared with the preset second threshold. This threshold is set according to the actual situation and experimental results. If the uncertainty score is higher than the threshold, it is considered that the initial answer may contain hallucination content, so a subsequent RAG process is needed to retrieve external knowledge to correct possible errors.

[0050] In another feasible implementation, step S20 may include steps C1 to C3: Step C1, inputting the initial input information and the preset prompt words into the large model to generate a response result; Specifically, an initial query or question is first submitted to the large language model, such as: "Please explain the principle of quantum entanglement." At the same time, in order to guide the model to express the uncertainty of its answer, a series of preset prompt words can be attached as part of the input. These prompt words can be designed as: "When answering, please also give your confidence in this answer and point out any possible ambiguity or uncertainty." After such a combined input (initial question + prompt word) is fed into the large language model, the model will generate a response result containing the answer and its uncertainty description. For example, the model may answer: "Quantum entanglement is a phenomenon in quantum mechanics that describes a strong correlation between two or more particles, making their states impossible to describe individually. I am about 85% confident in this answer because although the theoretical basis of quantum entanglement is solid, its practical application and interpretation still have some ambiguity." Step C2, performing uncertainty analysis on the response result to obtain an uncertainty score of the response result; Specifically, natural language processing (NLP) technology is used to analyze the response results generated by the model. This includes identifying and extracting keywords, phrases, or sentences related to uncertainty, such as "the confidence level is about 85%" or "there is a certain degree of ambiguity." Then, based on this information, an uncertainty score is quantified. In this example, "85%" can be used directly as the direct basis for the uncertainty score.

[0051] Step C3: if the uncertainty score is higher than a third preset threshold, the hallucination judgment result is that a hallucination may exist.

[0052] Specifically, the calculated uncertainty score is compared with the preset third threshold. This threshold is set in advance based on factors such as actual application scenarios, model performance, and user acceptance. Assuming that the set threshold is 70%, in the above example, since the uncertainty score is 85%, which is higher than the threshold, it can be judged that there may be hallucination content in the answer, that is, although the model gives a relatively complete answer, its confidence is not enough to completely rule out the possibility of misunderstanding or error in the answer.

[0053] Step S30, when the hallucination judgment result is that a hallucination may exist, the initial answer is hallucinated and corrected by combining the evidence set with the reasoning process and the corrected content is output.

[0054] In a feasible implementation, step S30 may include steps S31 to S33: Step S31, searching external knowledge sources to collect evidence sets; Specifically, RAG technology is used to retrieve relevant information from external knowledge sources to build an evidence set. Taking "the number of planets in the solar system" as an example, when the number given in the preliminary answer does not match the known facts, the system will search for relevant documents in large knowledge bases (such as online encyclopedias, NASA official websites, academic databases, etc.) through preset retrieval models (such as BM25 or DPR). These models will deeply analyze the keywords of the question ("solar system", "number of planets") and their context to accurately match and retrieve a collection of documents containing the correct number of planets to form an evidence set.

[0055] Step S32, using the evidence set, reviewing the reasoning process and the initial answer to screen out potential errors; Specifically, the collected evidence set is used to conduct a detailed review of the preliminary answer and the reasoning behind it. During this process, the model will review the documents in the evidence set one by one to evaluate their consistency and relevance with the preliminary answer. The model will pay special attention to evidence that is inconsistent with the preliminary answer but comes from reliable sources and accurate information, because they are likely to reveal potential errors in the preliminary answer. Through comparative analysis, the model can identify possible logical errors, misunderstandings, or information omissions in the reasoning process, thereby locking in specific points that need to be corrected.

[0056] Step S33: Based on the evidence set, correct the potential error and output the correction content.

[0057] Specifically, based on the information provided by the evidence set, potential errors in the preliminary answer are corrected. This process is not just a simple replacement of incorrect numbers, but also a process of in-depth understanding and re-reasoning. The model will review and adjust its reasoning path based on the authoritative information in the evidence set to ensure that each step is based on reliable facts and data. In the example of "the number of planets in the solar system", the model will correct the incorrect number in the preliminary answer to the correct number of planets based on consistent evidence from multiple sources, and may include explanations to increase the transparency and credibility of the answer. Ultimately, the corrected answer is the final response output of the system, which not only improves the accuracy of the answer, but also enhances the user's confidence in the reliability of the system.

[0058] Further, refer to Figure 2 , Figure 2 This is a flow chart of the second embodiment of the large model hallucination mitigation method of this application. Figure 1 In the embodiment shown, in this embodiment, step S30 may include step S40: Step S40: When the hallucination judgment result is that there is no hallucination, output an initial answer.

[0059] Specifically, when the hallucination judgment result is that there is no hallucination, it indicates that the preliminary answer is semantically consistent and there is no obvious external hallucination content. At this time, the preliminary answer will be directly output as the final response to the user input question without further introducing the RAG process to retrieve and correct external knowledge.

[0060] It should be noted that the preliminary answer is directly used as the final response to the user's input question. Since the accuracy and consistency of the preliminary answer have been verified by the multilingual detection module and the CoT reasoning process, directly outputting the answer can effectively avoid the external hallucination problem caused by irrelevant information introduced by external knowledge sources. At the same time, when the preliminary answer is confirmed to be correct, direct output reduces unnecessary processing time, improves the system's response speed, and also improves the user experience because users can get accurate and reliable answers faster.

[0061] In addition, the present application also proposes a large model hallucination relief device, the large model hallucination relief device comprising: The first acquisition module 10 is used to generate an initial answer and a corresponding reasoning process according to the initial input information, wherein the reasoning process is obtained by extracting the generation process of the initial answer using a thinking chain prompt word; A second acquisition module 20 is used to perform hallucination judgment on an initial question-answer pair and obtain a hallucination judgment result, wherein the initial question-answer pair includes the initial input information and the initial answer; The correction module 30 is used to correct the initial answer to the hallucination by combining the evidence set with the reasoning process and output the correction content when the hallucination judgment result is that the hallucination may exist.

[0062] The large model hallucination mitigation device provided by the present application adopts the large model hallucination mitigation method in the above embodiment, and can optimize the RAG technology to improve the reliability and accuracy of the content generated by the large language model. Compared with the prior art, the beneficial effects of the large model hallucination mitigation device provided by the present application are the same as the beneficial effects of the large model hallucination mitigation method provided by the above embodiment, and the other technical features in the large model hallucination mitigation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0063] The present application provides a large model hallucination mitigation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the large model hallucination mitigation method in the above-mentioned embodiment one.

[0064] Reference below Figure 4 , which shows a schematic diagram of the structure of a large-model illusion mitigation device for implementing an embodiment of the present application. The large-model illusion mitigation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The large-scale model hallucination relief device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0065] like Figure 4As shown, the large model illusion mitigation device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the large model illusion mitigation device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the large model illusion mitigation device to communicate with other devices wirelessly or by wire to exchange data. Although the large model illusion mitigation device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0066] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0067] The large model hallucination mitigation device provided by the present application adopts the large model hallucination mitigation method in the above embodiment, and can optimize the RAG technology to improve the reliability and accuracy of the content generated by the large language model. Compared with the prior art, the beneficial effects of the large model hallucination mitigation device provided by the present application are the same as the beneficial effects of the large model hallucination mitigation method provided by the above embodiment, and the other technical features in the large model hallucination mitigation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0068] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0069] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0070] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the large model hallucination mitigation method in the above-mentioned embodiment.

[0071] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0072] The computer-readable storage medium may be included in the large-model illusion alleviation device; or may exist independently without being assembled into the large-model illusion alleviation device.

[0073] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0074] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0075] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0076] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned large model hallucination mitigation method, and can optimize the RAG technology to improve the reliability and accuracy of the content generated by the large language model. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the large model hallucination mitigation method provided in the above-mentioned embodiment, and will not be repeated here.

[0077] The present application also provides a computer program product, including a computer program, which implements the steps of the large model hallucination mitigation method as described above when the computer program is executed by a processor.

[0078] The computer program product provided by the present application can optimize the RAG technology to improve the reliability and accuracy of the content generated by the large language model. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the large model hallucination mitigation method provided by the above embodiment, which will not be repeated here.

[0079] Compared with the prior art, the large model illusion mitigation method, device, equipment, medium and computer product proposed in the embodiment of the present application extracts the business feature information of the target business, performs data standardization processing on the business feature information to obtain standard feature data, performs hash processing on the standard feature data to obtain unique feature data, performs numerical processing and splicing processing on the unique feature data to obtain the first business feature value, accumulates the first business feature value of the target business to obtain the target business feature value, and finally compares the target business feature value with the feature value set to obtain the large model illusion mitigation result. Compared with the traditional method of identifying duplicate businesses by generating a unique key value or a continuous serial number for each business, it is more efficient, flexible and reliable. Based on the solution of the present application, the business of complex scenarios is transformed into a comparison of two numbers through a series of simple transformations, so that the comparison process is very intuitive and efficient. The system only needs to simply compare whether the two values ​​are equal to quickly determine whether the two businesses are exactly the same.

[0080] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0081] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0083] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for alleviating large model hallucinations, characterized in that: The large model hallucination mitigation method comprises: Generate an initial answer and a corresponding reasoning process according to the initial input information, wherein the reasoning process is obtained by extracting the generation process of the initial answer using a thinking chain prompt word; Performing hallucination judgment on an initial question-answer pair to obtain a hallucination judgment result, wherein the initial question-answer pair includes the initial input information and the initial answer; When the hallucination judgment result is that a hallucination may exist, the initial answer is hallucinated and corrected by combining the evidence set with the reasoning process, and the corrected content is output.

2. The large model hallucination mitigation method according to claim 1, characterized in that: The step of performing hallucination judgment on the initial question-answer pair and obtaining a hallucination judgment result comprises: Generating a target language variant question-answer pair of the initial question-answer pair, and calculating a consistency score between the initial question-answer pair and the target language variant question-answer pair; If the consistency score is lower than a first preset threshold, the hallucination judgment result is that a hallucination may exist.

3. The large model hallucination mitigation method according to claim 2, characterized in that: The step of generating a target language variant question-answer pair of the initial question-answer pair and calculating a consistency score between the initial question-answer pair and the target language variant question-answer pair comprises: Generating source variant input information semantically consistent with the initial input information, and translating the source variant input information into target language variant input information; Inputting the target language variant input information into the big model, generating a target language variant answer, and obtaining a target language variant question-answer pair; A consistency score is calculated between the initial question-answer pair and the target language variant question-answer pair.

4. The large model hallucination mitigation method according to claim 1, characterized in that: The step of performing hallucination judgment on the initial question-answer pair and obtaining the hallucination judgment result further includes: Predicting the token in the initial answer to obtain the entropy value of the token in the initial answer; Obtaining an uncertainty score of the initial answer using the entropy value of the token; If the uncertainty score is higher than a second preset threshold, the hallucination judgment result is that a hallucination may exist.

5. The large model hallucination mitigation method according to claim 1, characterized in that: The step of performing hallucination judgment on the initial question-answer pair and obtaining the hallucination judgment result further includes: Inputting the initial input information and the preset prompt words into the large model to generate a response result; Performing uncertainty analysis on the response result to obtain an uncertainty score of the response result; If the uncertainty score is higher than a third preset threshold, the hallucination judgment result is that a hallucination may exist.

6. The large model hallucination mitigation method according to claim 1, characterized in that: When the hallucination judgment result is that a hallucination may exist, the step of performing hallucination correction on the initial answer by combining the evidence set with the reasoning process and outputting the correction content also includes: Retrieve external knowledge sources to collect a set of evidence; Using the evidence set, reviewing the reasoning process and the initial answer to screen for potential errors; Based on the evidence set, the potential error is corrected and a correction content is output.

7. The large model hallucination mitigation method according to claim 1, characterized in that: When the hallucination judgment result is that a hallucination may exist, the step of performing hallucination correction on the initial answer by combining the evidence set with the reasoning process and outputting the correction content also includes: When the hallucination judgment result is that there is no hallucination, an initial answer is output.

8. A large-scale model hallucination relief device, characterized in that: The device comprises: A first acquisition module is used to generate an initial answer and a corresponding reasoning process according to initial input information, wherein the reasoning process is obtained by extracting the generation process of the initial answer using a thinking chain prompt word; A second acquisition module is used to perform hallucination judgment on an initial question-answer pair and obtain a hallucination judgment result, wherein the initial question-answer pair includes the initial input information and the initial answer; The correction module is used to correct the initial answer to the hallucination by combining the evidence set with the reasoning process and output the correction content when the hallucination judgment result is that the hallucination may exist.

9. A large-scale model hallucination relief device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the large model hallucination mitigation method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the large model hallucination mitigation method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Result output method and device, equipment and medium

    CN117454980A

  • Method and system for solving illusion problem of large legal language model

    CN117744802A

  • Method and device for judging whether large language model output is illusion or not

    CN118333149A

  • Response data reasoning method and device, electronic equipment and storage medium

    CN118410135A

  • Automated fact checking using iterative knowledge base querying

    US20230401213A1

Cited By

  • Large language model generation stability evaluation method, computer equipment and storage medium

    CN119312069A

  • Illusion text relieving method and device, equipment, storage medium and program product

    CN119378692A

  • Subject education large model knowledge boundary exploration method

    CN119539055A

  • Big language model illusion elimination method and device, equipment and storage medium

    CN119849509A

  • Large model prompt design method for social governance knowledge graph construction based on CoT chain thinking

    CN119918677A