Method for reducing answer illusion of large language model

Abstract

The invention provides a large language model answer hallusion reduction method, which is characterized by comprising the following steps: S1, constructing a hallusion evaluation data set containing a plurality of hallusion categories according to existing data; s2, analyzing the output of the hidden layer and the attention head of the large language model according to the illusion evaluation data set to obtain the hidden layer and the attention head, which have abnormal response to each illusion category, of the large language model, and high contribution words corresponding to each illusion category; s3, inputting a specified question into the large language model to obtain an answer selection probability of the large language model to the specified question and intermediate generation data of the large language model; and S4, according to the high contribution word, the intermediate generation data and the answer selection probability, in combination with the hidden layer with the abnormal response and the attention head, calculating through a selection function to obtain an adjusted probability corresponding to the answer selection probability. In short, the method can reduce the generation illusion of the large language model.

Description

Technical Field

[0001] The invention belongs to the field of large language model answer quality control, and in particular relates to a method for reducing hallucinations of large language model answers. Background Art

[0002] Hallucinations are when a large language model generates content that is inconsistent with user input, contradicts previously generated content, or is inconsistent with known world knowledge. Hallucinations pose a major challenge to the reliability of large language models in real-world scenarios. Hallucinations are unacceptable in scenarios where accuracy is very high, such as in the medical and financial fields, so reducing hallucinations is very important.

[0003] Existing technologies have made some achievements in reducing hallucinations. For example, Peng et al. improved the accuracy of predictions by using external knowledge; Zhang et al. adopted thought chain prompts to enable the model to generate reasoning before providing the final answer; Chen et al. constructed an "I don't know" dataset to train the model not to answer questions they don't know; Sun et al. proposed fact-enhanced RL to enhance the reward model and thus reduce hallucinations.

[0004] Although the above methods can make large language models generate results that are more consistent with basic facts in some cases, they still have many problems. For example, improving training data cannot completely solve the problem of large language models producing hallucinations when facing new or rare situations; prompt engineering requires fine-tuning of prompt words and has high requirements on users' input query skills; reinforcement learning requires a large amount of manually annotated data and a complex training process; knowledge retrieval depends on the quality and coverage of external knowledge bases, which may not fully cover all fields and have great limitations on the latest news.

[0005] In short, there is still much room for improvement in how to reduce hallucinations in large language models. Summary of the invention

[0006] The present invention is made to solve the above-mentioned problem, and its purpose is to provide a method for reducing answer hallucinations of large language models.

[0007] The present invention provides a method for reducing hallucinations of answers of a large language model, which is used to reduce the hallucinations of a large language model for a specified question, and has the following characteristics, including the following steps: step S1, constructing a hallucination evaluation data set containing multiple hallucination categories according to existing data; step S2, analyzing the outputs of the hidden layer and the attention head of the large language model according to the hallucination evaluation data set, and obtaining the hidden layer and the attention head of the large language model with abnormal responses to each hallucination category, and the high contribution words corresponding to each hallucination category; step S3, inputting the specified question into the large language model, and obtaining the answer selection probability of the large language model for the specified question and the intermediate generated data of the large language model; step S4, according to the high contribution words, the intermediate generated data, the answer selection probability, combined with the hidden layer and the attention head with abnormal response, obtaining the adjusted probability corresponding to the answer selection probability by selecting a function, wherein the answer corresponding to the largest probability in the adjusted probabilities is the answer generated by the large language model for the specified question.

[0008] In the method for reducing hallucinations of answers in a large language model provided by the present invention, there may also be such a feature: wherein the specified question includes a corresponding hallucination category and a correct answer, and the calculation expression of the selection function is: P n =P o exp(-α·P), P=P l +P dl +P a +P ca +P wa , Where P n is the adjusted probability, P o is the probability of answer selection, α is a preset constant, P is the total penalty function, N is the total number of hallucination categories, L is the total number of hidden layers, and o ij is the absolute value of the output of the i-th hidden layer with an abnormal response to the j-th type of hallucination in the intermediate generated data, is the average output value of the same layer of the correct answer, x j Select the output point after probability reduction for the answer corresponding to the jth type of hallucination, is the center of the spatial distribution of the correct answer, Dist() is the Euclidean distance calculation function, H is the total number of attention heads, and A ijk is the attention output of the k-th attention head in the i-th layer that has an abnormal response to the j-th type of hallucination in the intermediate generated data, is the average attention output matrix of the correct answer, A j is the attention output matrix of the j-th hallucination, W j is the set of high contribution words corresponding to the j-th type of hallucination, Con(w j ) is the set Wj The contribution of the jth high contribution word to the hallucination text, λ l , dl , a , ca and λ wa Both are adjustment factors.

[0009] The method for reducing hallucinations of answers using a large language model provided by the present invention may also have the following features: when a specified question does not contain a corresponding hallucination category and a correct answer, the optimal answer is taken as the correct answer, and any hallucination category is taken as the hallucination category corresponding to the specified question, and the optimal answer is the answer generated by the best existing model for the specified question or the answer given manually to the specified question.

[0010] The method for reducing hallucinations of answers from a large language model provided by the present invention may also have the following features: wherein the hallucination categories include factual errors, logical errors, conceptual confusion, fuzziness, lack of common sense, over-generalization, emotional bias and lack of uncertainty. Factual errors refer to incorrect facts contained in the text generated by the large language model. Logical errors refer to illogical or self-contradictory texts generated by the large language model. Concept confusion refers to the problem of confusing different concepts in the text generated by the large language model. Fuzziness refers to overly vague or unclear texts generated by the large language model. Lack of common sense refers to the lack of some common knowledge in the large language model and the inability to generate correct texts. Over-generalization refers to overly broad answers generated by the large language model and lack of details related to a specific context. Emotional bias refers to bias or emotional tone in the text generated by the large language model. Lack of uncertainty refers to overly affirmative answers to uncertain events in the text generated by the large language model.

[0011] The method for reducing hallucinations of large language model answers provided by the present invention may also have the following features: wherein the hallucination evaluation data set includes evaluation questions and corresponding correct answers, and in step S2, a hidden layer whose output when the large language model generates an answer of a specific hallucination type is significantly different from that when the large language model generates a correct answer, is used as a hidden layer with an abnormal response to the hallucination category, and an attention head whose output when the large language model generates an answer of a specific hallucination type is significantly different from that when the large language model generates a correct answer, is used as an attention head with an abnormal response to the hallucination category.

[0012] In the method for reducing hallucination of answers of a large language model provided by the present invention, the hallucination evaluation dataset may also have the following features: wherein the hallucination evaluation dataset includes evaluation questions and corresponding correct answers, and in step S2, the evaluation questions are converted into differentiable embeddings as inputs of the large language model, the gradients of the embeddings are L2 normalized to generate normalized embeddings, and the gradients and the normalized embeddings are multiplied by the Input X Gradient method to obtain the significance scores of the words in the evaluation questions corresponding to the embeddings, and the words with high significance scores are selected as high contribution words.

[0013] Functions and Effects of the Invention

[0014] According to the method for reducing hallucinations of answers of a large language model involved in the present invention, a hallucination evaluation data set is constructed, and the hallucination evaluation data set is used to analyze the activity of the hidden layer and attention head of the large language model for each hallucination, so that the intermediate generated data of the large language model under the specified question is combined with the selection function to calculate the corrected adjusted probability, thereby reducing the possibility that the large language model has hallucinations for the specified question. Therefore, the method for reducing hallucinations of answers of a large language model of the present invention can reduce the generation of hallucinations by the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of a method for reducing answer hallucinations of a large language model in an embodiment of the present invention;

[0016] Figure 2 It is a schematic diagram of the principle of a method for reducing answer hallucinations of a large language model in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate the method for reducing answer hallucinations of a large language model of the present invention.

[0018] This embodiment provides a method for reducing hallucinations of answers to large language models, which is used to reduce the hallucinations of a large language model about a specified question.

[0019] Figure 1 It is a flowchart of a method for reducing answer hallucinations of a large language model in an embodiment of the present invention.

[0020] like Figure 1 As shown, the method for reducing hallucination of large language models includes the following steps:

[0021] Step S1, constructing a hallucination evaluation dataset containing multiple hallucination categories based on existing data. The hallucination evaluation dataset includes evaluation questions and corresponding correct answers. In this embodiment, the existing data is the existing RelQA-Cate dataset, and multiple hallucination categories are obtained by summarizing answers generated by multiple existing large language models.

[0022] The hallucination categories of this embodiment include eight categories, namely factual errors, logical errors, conceptual confusion, ambiguity, lack of common sense, over-generalization, emotional bias and lack of uncertainty. Factual errors refer to incorrect facts contained in the text generated by the large language model. Logical errors refer to illogical or self-contradictory texts generated by the large language model. Concept confusion refers to the problem of confusing different concepts in the text generated by the large language model. Ambiguity refers to the text generated by the large language model that is too vague or unclear. Lack of common sense refers to the lack of some common knowledge in the large language model and the inability to generate correct text. Over-generalization refers to the answers generated by the large language model that are too broad and lack details related to the specific context. Emotional bias refers to the text generated by the large language model that shows bias or emotional tone. Lack of uncertainty refers to the text generated by the large language model that gives overly positive answers to uncertain events.

[0023] Step S2, analyzing the outputs of the hidden layers and attention heads of the large language model according to the hallucination evaluation dataset, and obtaining the hidden layers and attention heads of the large language model that have abnormal responses to each hallucination category, as well as the high contribution words corresponding to each hallucination category.

[0024] Among them, the hidden layer whose output when the large language model generates an answer of a specific hallucination type is significantly different from the output when the large language model generates a correct answer is used as the hidden layer with an abnormal response to the hallucination category. The attention head whose output when the large language model generates an answer of a specific hallucination type is significantly different from the output when the large language model generates a correct answer is used as the attention head with an abnormal response to the hallucination category.

[0025] The evaluation problem is converted into a differentiable embedding as the input of the large language model, the gradient of the embedding is L2 normalized to generate a normalized embedding, and the gradient and the normalized embedding are multiplied by the Input XGradient method to obtain the significance score of the word in the evaluation problem corresponding to the embedding, and finally the words with high significance scores are selected as high contribution words.

[0026] Step S3, input the specified question into the large language model, and obtain the probability of the large language model selecting the answer to the specified question and the intermediate generated data of the large language model. The intermediate generated data includes the outputs of each hidden layer and attention head generated by the large language model for the specified question.

[0027] Step S4, based on the high contribution words, the intermediate generated data, the answer selection probability, combined with the hidden layer and the attention head with abnormal response, the adjusted probability corresponding to the answer selection probability is calculated by the selection function. Among them, the answer corresponding to the largest probability in the adjusted probability is the answer generated by the large language model for the specified question.

[0028] Among them, the specified question contains the corresponding hallucination category and the correct answer, and the calculation expression of the selection function is:

[0029] P n =P o exp(-α·P),

[0030] P=P l +P dl +P a +P ca +P wa ,

[0031]

[0032] Where P n is the adjusted probability, P o is the probability of answer selection, α is a preset constant, P is the total penalty function, N is the total number of hallucination categories, L is the total number of hidden layers, and o ij is the absolute value of the output of the i-th hidden layer with an abnormal response to the j-th type of hallucination in the intermediate generated data, is the average output value of the correct answer in the same layer of the large language model, x j Select the output point after probability reduction for the answer corresponding to the jth type of hallucination, is the center of the spatial distribution of the correct answer, Dist() is the Euclidean distance calculation function, H is the total number of attention heads, and A ijk is the attention output of the k-th attention head in the i-th layer that has an abnormal response to the j-th type of hallucination in the intermediate generated data, is the average attention output matrix of the correct answer, A j is the attention output matrix of the j-th hallucination, W j is the set of high contribution words corresponding to the j-th type of hallucination, Con(w j ) is the set W j The contribution of the jth high contribution word to the hallucination text, λ l , dl , a , ca and λ wa The optimal answer in this embodiment is the answer generated by the best existing model to the specified question or the answer given manually to the specified question. The best model is a large language model that is recognized to have the best performance in answering questions.

[0033] When the specified question does not contain the corresponding hallucination category and correct answer, the best answer is taken as the correct answer, and any hallucination category is taken as the hallucination category corresponding to the specified question. The best answer is the answer generated by the best existing model to the specified question or the answer given by humans to the specified question.

[0034] Figure 2 It is a schematic diagram of the principle of a method for reducing answer hallucinations of a large language model in an embodiment of the present invention.

[0035] like Figure 2 As shown in the figure, given the question "Historic Merton Street is parallel to the High Street in the university town southeast of Oxford, England, what is its surface?", the large language model generates multiple candidate answers: A1. This is a cobblestone street, A2. There are many people and A3. This is an old cement road, as well as the selection probability of each candidate answer. Then, the answer selection probability includes each candidate answer and its corresponding selection probability. Among them, the selection probability of A1 is 0.5, the selection probability of A2 is 0.2, and the selection probability of A3 is 0.3, and the candidate answers corresponding to A2 and A3 obviously have hallucination problems.

[0036] Then, the penalty value is calculated by selecting the function, that is, the total penalty function is calculated, and then the answer selection probability is adjusted according to the total penalty function to generate a new answer selection probability, that is, the adjusted probability. For the above specified question, the selection probability of A1 in the adjusted probability is 0.8, the selection probability of A2 is 0.1, and the selection probability of A3 is 0.1. Finally, the large language model selects the candidate answer with the highest selection probability, that is, A1, as the response to the specified question.

[0037] It can be seen that the method for reducing hallucinations of answers of large language models in this embodiment can reduce the probability of the large language model selecting candidate answers with hallucinations, thereby increasing the probability of the large language model generating answers without hallucinations.

[0038] Functions and Effects of the Embodiments

[0039] According to the method for reducing hallucinations of answers of a large language model involved in this embodiment, a hallucination evaluation dataset is constructed, and the hallucination evaluation dataset is used to analyze the activity of the hidden layer and attention head of the large language model for each hallucination, so as to obtain the corrected adjusted probability based on the intermediate generated data of the large language model under the specified question and the selection function calculation, thereby reducing the possibility that the large language model has hallucinations for the specified question. In short, this method can reduce the generation of hallucinations by the large language model.

[0040] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for reducing hallucination of answers of a large language model, for reducing the hallucination of a large language model to a specified question, characterized in that: The following steps are involved: Step S1, constructing a hallucination assessment dataset containing multiple hallucination categories based on existing data; Step S2, analyzing the outputs of the hidden layer and the attention head of the large language model according to the hallucination evaluation data set, and obtaining the hidden layer and the attention head of the large language model having abnormal responses to each of the hallucination categories, and the high contribution words corresponding to each of the hallucination categories; Step S3, inputting the specified question into the large language model to obtain the probability of the large language model selecting an answer to the specified question and the intermediate generated data of the large language model; Step S4, according to the high contribution words, the intermediate generated data, the answer selection probability, combined with the hidden layer and the attention head with abnormal response, the adjusted probability corresponding to the answer selection probability is calculated by selecting a function, Among them, the answer corresponding to the largest probability among the adjusted probabilities is the answer generated by the large language model to the specified question.

2. The method for reducing answer hallucinations using a large language model according to claim 1, wherein: in, The specified question includes the corresponding hallucination category and the correct answer, The calculation expression of the selection function is: P n =P o ·exp(-α·P), P=P l +P dl +P a +P ca +P wa , Where P n is the adjusted probability, P o is the probability of selecting the answer, α is a preset constant, P is the total penalty function, N is the total number of hallucination categories, L is the total number of hidden layers, o ij is the absolute value of the output of the i-th hidden layer with an abnormal response to the j-th type of hallucination in the intermediate generated data, is the average output value of the same layer of the correct answer, x j Select the output point after probability reduction for the answer corresponding to the jth type of hallucination, is the center of the spatial distribution of the correct answer, Dist() is the Euclidean distance calculation function, H is the total number of attention heads, and A ijk is the attention output of the k-th attention head in the i-th layer that has an abnormal response to the j-th type of hallucination in the intermediate generated data, is the average attention output matrix of the correct answer, A j is the attention output matrix of the j-th hallucination, W j is the set of high contribution words corresponding to the j-th type of hallucination, Con(w j ) is the set W j The contribution of the jth high contribution word to the hallucination text, λ l , dl , a , ca and λ wa Both are adjustment factors.

3. The method for reducing answer hallucinations using a large language model according to claim 2, wherein: in, When the designated question does not contain the corresponding hallucination category and the correct answer, the best answer is used as the correct answer, and any one of the hallucination categories is used as the hallucination category corresponding to the designated question. The optimal answer is the answer generated by the best existing model to the specified question or the answer given manually to the specified question.

4. The method for reducing answer hallucinations using a large language model according to claim 1, wherein: in, The categories of illusions include factual errors, logical errors, conceptual confusion, vagueness, lack of common sense, overgeneralization, emotional bias, and lack of uncertainty. The factual error is that the text generated by the large language model contains incorrect facts. The logical error is that the text generated by the large language model is illogical or self-contradictory. The concept confusion is that the text generated by the large language model has the problem of confusing different concepts. The fuzziness is that the text generated by the large language model is too fuzzy or unclear, The lack of common sense means that the large language model lacks some common knowledge and cannot generate correct text. Overgeneralization means that the answers generated by the large language model are too broad and lack details related to the specific context. The emotional bias is that the text generated by the large language model exhibits a bias or emotional tone, The lack of uncertainty is that the text generated by the large language model gives overly confident answers to uncertain events.

5. The method for reducing answer hallucinations using a large language model according to claim 1, wherein: in, The hallucination evaluation dataset includes evaluation questions and corresponding correct answers. In step S2, the hidden layer whose output when the large language model generates an answer of a specific hallucination type is significantly different from the output when the large language model generates a correct answer is used as the hidden layer having an abnormal response to the hallucination category. The attention head whose output when the large language model generates an answer of a specific hallucination type is significantly different from the output when the large language model generates a correct answer is regarded as an attention head with an abnormal response to the hallucination category.

6. The method for reducing answer hallucinations using a large language model according to claim 1, wherein: in, The hallucination evaluation dataset includes evaluation questions and corresponding correct answers. In the step S2, the evaluation problem is converted into a differentiable embedding as the input of the large language model, the gradient of the embedding is L2 normalized to generate a normalized embedding, and the gradient and the normalized embedding are multiplied by the Input X Gradient method to obtain a significance score of a word in the evaluation problem corresponding to the embedding. The words with high significance scores are selected as the high contribution words.

Images