Big language model illusion relieving method based on contrast decoding

By using contrast decoding and single-linear layer hallucination information screening models in large language models, the problems of low efficiency and high resource consumption in hallucination mitigation of large language models are solved, and more efficient hallucination mitigation and output authenticity are achieved.

CN118964552AActive Publication Date: 2024-11-15CHENGDU UNIV OF INFORMATION TECH

Patent Information

Application Number
CN202410955938.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-15
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

The prior art has problems of inefficiency in alleviating the hallucination of large language models, relying on a large amount of manpower and computing resources, inability to completely clear problem data from the corpus, prolonged decoding time, and poor output uncertainty and poor interpretability.

Method used

Using a contrast decoding-based method, by subtracting hallucination information from the prediction information of the final layer of the large language model, using additional hallucination information to filter the model to fuse all the prediction information of the lower layer, and training a single-linear layer model to identify and mitigate hallucination information. Ensure the authenticity and reliability of the output.

Benefits of technology

It effectively reduces the illusion of large language models, improves the factuality and reliability of outputs, saves computing resources and time costs, and ensures model integrity and data consistency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964552A_ABST
    Figure CN118964552A_ABST
Patent Text Reader

Abstract

The invention relates to a big language model illusion relieving method based on comparative decoding. The authenticity of output content is enhanced by removing potential illusion components from prediction information of a final layer of a big model, a multi-layer fusion mechanism is designed, and probability distribution information of prediction of each layer of the model is fully utilized. An illusion information screening model is introduced, and illusion information possibly contained in a low layer of a large language model is integrated and analyzed in a simple and efficient mode. The illusion information screening model is trained on the illusion question and answer data set, so that the illusion information screening model can accurately capture and fit illusion data features commonly existing in the data set, and effective recognition and filtering of the illusion information are achieved. In the decoding stage, the probability distribution output by the final layer of the model and the probability distribution output by the illusion information screening model are compared and decoded, and the generation probability distribution of the next mark is dynamically adjusted by taking the difference as guidance, so that the big language model is guided to more tend to generate real and illusion-free contents in the decoding process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a large language model hallucination alleviation method based on contrast decoding. Background Art

[0002] The rapid development of large language models has brought revolutionary changes to the field of artificial intelligence research. Large language models have demonstrated powerful natural language understanding and the ability to handle complex tasks. Their widespread application has provided a huge impetus for the development of natural language processing and artificial intelligence technology. However, although large models perform well on many tasks, they sometimes generate texts that deviate from the facts. This phenomenon is called hallucination. Hallucinations can lead to the spread of misleading information and undermine the accuracy of decision-making, especially in high-risk fields such as healthcare and finance, where hallucinatory information may have irreversible consequences. In summary, solving the hallucination problem of large language models has great application value in practice.

[0003] At present, the mainstream methods in this field include manual or automatic elimination of noise training data, reinforcement learning based on human feedback, design of decoding strategies, introduction of external knowledge, etc. The method of eliminating noise training data acts on the data set. Initially, researchers used the method of manually screening data, but with the increase in the size of pre-training corpora in recent years, it is generally based on rules or large language models (such as ChatGPT) to automatically organize pre-training corpora to minimize unverifiable or unreliable data; reinforcement learning based on human feedback is currently a more commonly used method. In order to alleviate hallucinations, researchers have designed a special reward function to encourage the model to express "I don't know" for uncertain answers. This honest-oriented reward function is more advantageous in alleviating hallucinations; another is to design a decoding strategy. This method does not need to change the knowledge trained by the original large language model, but tends to change the final prediction of the model. The most commonly used method is to compare decoding with models with hallucination information, as well as decoding with inference time intervention, etc.; the method of introducing external knowledge generally includes two steps, the first is to obtain external knowledge, and the second is to use external knowledge to guide response generation. This method passes real-time updated knowledge to the large language model and is interpretable.

[0004] The existing technical solutions include the following technical solutions. Corpus filtering, manually eliminating hallucination data in the data set or filtering unreal data in the data set based on rule retrieval. Supervised fine-tuning, first annotating or collecting a large amount of factual data, and then using a training target based on hallucination settings to replace the maximum likelihood estimation (for example: minimum risk training) to fine-tune the pre-trained basic large language model. By adopting a carefully designed supervised fine-tuning strategy, the purpose of alleviating the hallucination of the large model can be achieved. Design a hallucination scorer, design an additional fact scorer, and the scorer model is usually designed based on an understanding model represented by BERT, so that the language model generates multiple outputs for the same question, and then the multiple outputs are factually scored, and the result with the highest score is selected as the final output. Induce hallucination first and then compare and decode, first induce hallucination from the original model, and by fine-tuning the original model on the hallucination data set, a large model that is relatively weak in fact is constructed. Then, by comparing and decoding the original model with the model with weak facts, the non-factual knowledge internalized in the weak model is eliminated from the output space of the original model, and the model is guided to generate more factual content. Prompt Engineering: Large language models change significantly based on the prompts given by users, allowing the model to understand and perform various tasks through predefined prompts without the need for large-scale retraining or fine-tuning of the model.

[0005] The above-mentioned existing technical solutions have deficiencies. Manual elimination of false data in a data set may be affected by personal subjective judgment, resulting in insufficient objectivity and accuracy in data filtering. At the same time, manual filtering requires a lot of time and human resources, especially when processing large-scale data sets, and the efficiency is low. The rule-based retrieval method may not cover all types of untrue data, resulting in the inability to completely remove the problematic data in the corpus. Although supervised fine-tuning can significantly improve the factual ability of model output, it also has some significant disadvantages. On the one hand, it relies on a large amount of labeled high-quality data, but it is difficult to obtain data in this regard; on the other hand, the fine-tuning process has high requirements for hardware resources. Taking the common NVIDIA RTX3090 graphics card as an example, its video memory can usually only support fine-tuning of models with a maximum of about 1.5 billion parameters, but most of the models actually used in the current environment have parameters between 6 billion and 13 billion. The training of the hallucination scorer is still a supervised training process, and the requirements for labeled high-quality training data are still high, and the accuracy of the hallucination scorer has a significant impact on the final output. If the scorer accuracy is not high, the hallucination scorer may not be able to accurately identify and evaluate hallucinations, thereby affecting its overall performance and the reliability of the output results. At the same time, since the whole process needs to generate multiple replies and then score them, the whole process is relatively long and time-consuming and computationally resource-intensive. The comparative decoding of the two large models does show significant results in alleviating the illusion of the large language model. However, this method also brings a problem that cannot be ignored, that is, the decoding time increases exponentially. This is because during the comparative decoding process, the system needs to process the information of two large models at the same time. This time delay not only affects the overall performance of the system, but also leads to a poor user experience in actual use. The positive impact on the output results can be achieved through the prompt engineering to build prompt words, although it is relatively easy to implement. However, designing an effective prompt template requires deep domain knowledge and is difficult. Secondly, the current large language model is a probabilistic generative model. The uncertainty of its output and poor interpretability also make it very difficult to correct the response based on the prompt word. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a large language model hallucination relief method based on contrastive decoding, which uses the prediction information of the final layer of the large language model to subtract the hallucination information to achieve the purpose of alleviating hallucinations, makes full use of the probability distribution of the predictions of all layers of the large language model, uses an additional hallucination information screening model to fuse the hallucination information existing in all lower layers, and trains the hallucination information screening model on some hallucination question-answering data sets to make the hallucination information screening model fit the characteristics of most hallucination data in the data sets, and takes the difference between the probability distribution of the final layer and the probability distribution obtained by the hallucination information screening model as the distribution of the next tag. The hallucination relief method specifically includes:

[0007] Step 1: Load the pre-trained hallucination information screening model. If the hallucination information screening model does not exist, execute the following subdivided training steps 11 to 17, otherwise execute step 2;

[0008] Step 11: Design a hallucination information screening model, set the input of the hallucination information screening model to N dimensions, where N is the number of layers of the large language model, and the output is 1 dimension;

[0009] Step 12: Prepare a hallucination question-answering dataset, which can be a public dataset or a self-constructed question-answering dataset. It needs to contain two labels: question and hallucination answer.

[0010] Step 13: Input the hallucination question answering dataset into the large language model, obtain the labels of the hallucination replies in the hallucination question answering dataset, use a tokenizer to segment the hallucination replies of each data into tokens, and then map the tokens to indexes in the vocabulary of the large language model for use in calculating the loss;

[0011] Step 14: Obtain the probability distribution of the hallucination reply predictions of the remaining layers in the large language model except the final layer, take the question of each data in the data set as the input of the large language model, and according to the word-by-word prediction characteristics of the large language model, obtain the probability distribution of each word predicted by the 0 to N-1 layers of the large prediction model except the final layer for the hallucination answer;

[0012] Step 15: Obtain the label predicted by the hallucination information screening model, stack the probability distribution predicted by the large language model 0 to N-1 layers for the hallucination response as the input of the hallucination information screening model, obtain the probability distribution calculated by the hallucination information screening model, and select the token with the highest probability as the predicted label of the hallucination information screening model based on greedy decoding;

[0013] Step 16: Train the hallucination information screening model, calculate the loss of the predicted label in step 15 and the label of the hallucination answer in step 13, specifically use the cross entropy loss function, and set parameters including learning rate and number of learning rounds for training;

[0014] Step 17: Back-propagate the loss to the hallucination information screening model, repeat step 16, save the hallucination information screening model when the number of learning times in step 16 is reached, and then execute step 2;

[0015] Step 2: Get the probability distribution of the next token prediction by the final layer of the large language model;

[0016] Step 3: Obtain the probability distribution of the next token prediction for layers 0 to N-1 of the large language model except the final layer;

[0017] Step 4: Obtain the probability distribution result of the hallucination information screening model, stack the probability distribution obtained in step 3 as the input of the hallucination information screening model, and output the probability distribution of the hallucination information screening model;

[0018] Step 5: Compare and decode, and take the difference between the probability distribution predicted by the final layer of the large language model and the probability distribution obtained by the hallucination information screening model as the probability distribution of the next token;

[0019] Step 6: Decode the probability distribution after comparison decoding into text, and decode the probability distribution into text through greedy decoding.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. The present invention achieves the technical effect of reducing hallucinations and increasing the factuality of output by eliminating hallucination information from the prediction of the final layer of a large language model through comparative decoding.

[0022] 2. The present invention alleviates hallucinations through an additional hallucination information screening model, and a simple linear layer can meet the needs, saving computing resources and time costs.

[0023] 3. The present invention designs a hallucination information screening model that is independent of the large language model and is used as an external part. It does not need to change the knowledge originally learned by the large language model, which ensures the integrity of the model and the consistency of the data, while effectively improving the authenticity and reliability of the generated content. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a structural schematic diagram of the IEAH model proposed in the present invention;

[0025] Figure 2 It is a processing flow chart of the method of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0027] The following is a detailed description with reference to the accompanying drawings.

[0028] The token of the present invention refers to: a basic unit when a large language model processes input text, which can be a Chinese character, a word, a subword, a character or other text fragment.

[0029] The hallucination question answering dataset of the present invention is a hallucination data set focusing on question answering scenarios, which covers misleading information or "hallucination" phenomena that large language models may encounter when generating text. Each sample consists of a real question and a corresponding response that seems reasonable but is actually inconsistent with the question context or logically wrong.

[0030] The present invention proposes a new decoding strategy, the core idea of ​​which is to make the prediction of the final layer of the large language model subtract the prediction with hallucination information to achieve the purpose of reducing the hallucination of the large model. Therefore, the present invention designs an additional hallucination information screening model (preferably a single linear layer model) to fuse the hallucination information present in the low-level prediction of the model. This decoding strategy effectively reduces the hallucination information in the model output and enhances the factuality and reliability of the model output. More importantly, the method of the present invention has a low reliance on the complexity of the hallucination information screening model. For example, the preferred single linear layer model structure in the present invention is very simple, the training process is relatively easy, and the training cost is low, making the model more efficient and economical.

[0031] The hallucination information screening model of the present invention can be a single linear layer model, a convolutional neural network, a recurrent neural network, a Bert model or a large language model. It is not recommended to use a larger model because as the model size increases, the required computing resources, memory usage and energy consumption will increase significantly.

[0032] This case takes the large language model LLaMA-7B as an example. The output of the 32nd layer is the original output of the model. When the model is asked: Which goddess is Easter related to? The model incorrectly outputs the goddess "Ishtar", while the correct answer is the goddess "Ister". One possible reason is that the pre-training goal of the large language model, that is, the prediction of the next tag based on maximum likelihood, the goal of maximum likelihood is not always consistent with the facts of the real world, but tends to rely on surface patterns in the training corpus. Simply put, "Ishtar" may appear more times in the training corpus, and the model will tend to output it. However, other knowledge is not completely discarded inside the model, but is stored in the lower layer of the model. The purpose of the technical solution of the present invention is to extract useful information from the lower layer of the large language model by constructing an additional hallucination information screening model (the present invention is preferably a hallucination information screening model composed of a single linear layer model) to achieve the purpose of alleviating hallucinations.

[0033] Below, the technical solution of the invention is described in detail with reference to the accompanying drawings.

[0034] Figure 1The present invention proposes a model structure schematic for alleviating the large model hallucination, which is referred to as the IEAH model (Integrate the hallucination information existing in all Evolving layers to Alleviate Hallucinations). Figure 1 The LLaMA-7B model is used as an example of a large language model, but the actual application is not limited to the LLaMA-7B model. The core idea of ​​the method of the present invention is to remove the hallucination information from the probability distribution of the final layer prediction of the large language model to reduce the hallucination. The IEAH model makes full use of the prediction information of all layers of the large language model. Figure 1 As shown, first, for the problem of inputting a large language model, the probability distribution of the next token prediction of all layers of the large language model (including layer 0, i.e., the word embedding layer) is obtained. Next, the present invention stacks up the remaining layers except the final layer as the input of the single linear layer model. The work done internally by this single linear layer model is to perform weighted summation on these input probability distributions to obtain the result of the single linear layer model. Finally, the present invention compares and decodes the probability distribution predicted by the final layer of the large language model with the probability distribution obtained by the single linear layer model, and the difference obtained is used as the probability distribution of the next token.

[0035] from Figure 1 As can be seen in Figure 2, the IEAH model corrects the originally incorrect output of the large language model.

[0036] Step 1: Load the pre-trained hallucination information screening model. The preferred hallucination information screening model of the present invention is a fully connected network model composed of a single linear layer. If the hallucination information screening model does not exist, perform the following subdivided training step 11, otherwise perform step 2.

[0037] Step 11: Hallucination information screening model Taking the single linear layer network model as an example, a single linear layer model is designed, and the input of the hallucination information screening model is set to N dimensions, where N is the number of layers of the large language model, and the output is 1 dimension.

[0038] Step 12: Prepare a hallucination question-answering dataset. It can be a self-constructed question-answering dataset that needs to contain two labels: questions and hallucination answers, or it can be a public dataset such as the HaluEval dataset.

[0039] Step 13: Input the hallucination question-answering dataset into the large language model, obtain the labels of the hallucination replies in the dataset, use the tokenizer to split the hallucination replies of each data into tokens, and then map these tokens to the indexes in the vocabulary of the model for use in calculating the loss.

[0040] Step 14: Obtain the probability distribution of the hallucination response predictions of the remaining layers in the large language model except the final layer, take the question of each data in the data set as the input of the large language model, and according to the characteristics of the large language model's word-by-word prediction, obtain the probability distribution of each word predicted by the large prediction model from 0 to N-1 layers except the final layer for the hallucination answer. The mathematical expression is as follows:

[0041] q(x t |x <t ,j)=softmax(logit(x t |x <t ,j)),j∈{0,1,2…N-1} (1)

[0042] Among them, logit(x t ∣x <t ,j) is the next token x predicted by the jth layer of the large language model (j∈{0,1,2,…,N-1}) t Finally, it is normalized into a probability distribution through the softmax operation.

[0043] Step 15: Obtain the label predicted by the hallucination information screening model, stack the probability distribution predicted by the large language model 0 to N-1 layers for the hallucination response as the input of the hallucination information screening model, obtain the probability distribution calculated by the hallucination information screening model, and select the token with the highest probability as the predicted label of the single linear layer model based on greedy decoding.

[0044] Step 16: Train the hallucination information screening model, calculate the loss of the predicted label in step 15 and the label of the hallucination answer in step 13, preferably the cross entropy loss function, and set the learning rate, number of learning rounds and other parameters for training.

[0045] Step 17: Back-propagate the loss to the hallucination information screening model, repeat step 16, save the hallucination information screening model when the number of learning times in step 16 is reached, and then execute step 2.

[0046] Step 2: Get the probability distribution of the next token predicted by the final layer of the large language model.

[0047] The formula here is similar to that in step 14, but for the convenience of subsequent description, the probability of the last layer prediction of the large language model is listed separately here.

[0048] p(x t |x <t )=softmax(logit(x t |x <t ,N)) (2)

[0049] Step 3: Obtain the probability distribution of the next token prediction for the remaining layers of the large language model except the final layer. The formula is the same as step 14 in the training phase.

[0050] q(x t |x <t ,j)=softmax(logit(x t |x <t ,j)),j∈{0,1,2…N-1} (3)

[0051] Step 4: Obtain the probability distribution results of the hallucination information screening model. Stack the probability distributions obtained in step 3 as the input of the hallucination information screening model, and obtain the results predicted by the hallucination information screening model. The mathematical expression is as follows:

[0052]

[0053] in, It indicates stacking the probability distribution of the predictions from layer 0 to layer N-1. LinearModel() indicates the operation of the hallucination information screening model, and LM indicates the probability distribution after the operation of the hallucination information screening model.

[0054] Step 5: Comparative decoding. The difference between the probability distribution predicted by the final layer of the large language model and the probability distribution obtained by the hallucination information screening model is used as the distribution of the next token.

[0055] Here, in order to avoid the syntax problem after comparison decoding, the present invention introduces an adaptive rationality constraint V head , only retaining a few tags with higher probabilities predicted by the final layer of the model, where the range of α is [0,1]. The larger the α, the fewer tags are retained. The mathematical expression is as follows:

[0056]

[0057] Step 6: Decode the probability distribution after contrast decoding into text. Through greedy decoding, the probability distribution is decoded into words. The probability distribution obtained by the hallucination information screening model is a probability distribution with hallucination information. Contrast decoding is equivalent to removing the hallucination information from the prediction of the large language model. This method can effectively reduce the hallucination information and generate more accurate and logical output.

[0058] It should be noted that the above specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the protection scope of the present invention. Those skilled in the art should understand that the present invention description and its drawings are illustrative and do not constitute a limitation of the claims. The protection scope of the present invention is defined by the claims and their equivalents.

Claims

1. A large language model hallucination mitigation method based on contrastive decoding, characterized in that: The purpose of alleviating hallucinations is achieved by using the predicted information of the final layer of the large language model minus the hallucination information, making full use of the probability distribution predicted by all layers of the large language model, using an additional hallucination information screening model to fuse the hallucination information existing in all lower layers, and training the hallucination information screening model on some hallucination question-answering data sets to make the hallucination information screening model fit the characteristics of most hallucination data in the data sets, and taking the difference between the probability distribution of the final layer and the probability distribution obtained by the hallucination information screening model as the distribution of the next tag. The hallucination mitigation method specifically includes: Step 1: Load the pre-trained hallucination information screening model. If the hallucination information screening model does not exist, execute the following subdivided training steps 11 to 17, otherwise execute step 2; Step 11: Design a hallucination information screening model, set the input of the hallucination information screening model to N dimensions, where N is the number of layers of the large language model, and the output is 1 dimension; Step 12: Prepare a hallucination question-answering dataset, which can be a public dataset or a self-constructed question-answering dataset. It needs to contain two labels: question and hallucination answer. Step 13: Input the hallucination question answering dataset into the large language model, obtain the labels of the hallucination replies in the hallucination question answering dataset, use a tokenizer to segment the hallucination replies of each data into tokens, and then map the tokens to indexes in the vocabulary of the large language model for use in calculating the loss; Step 14: Obtain the probability distribution of the hallucination reply predictions of the remaining layers in the large language model except the final layer, take the question of each data in the data set as the input of the large language model, and according to the word-by-word prediction characteristics of the large language model, obtain the probability distribution of each word predicted by the 0 to N-1 layers of the large prediction model except the final layer for the hallucination answer; Step 15: Obtain the label predicted by the hallucination information screening model, stack the probability distribution predicted by the large language model 0 to N-1 layers for the hallucination response as the input of the hallucination information screening model, obtain the probability distribution calculated by the hallucination information screening model, and select the token with the highest probability as the predicted label of the hallucination information screening model based on greedy decoding; Step 16: Train the hallucination information screening model, calculate the loss of the predicted label in step 15 and the label of the hallucination answer in step 13, specifically use the cross entropy loss function, and set parameters including learning rate and number of learning rounds for training; Step 17: Back-propagate the loss to the hallucination information screening model, repeat step 16, save the hallucination information screening model when the number of learning times in step 16 is reached, and then execute step 2; Step 2: Get the probability distribution of the next token prediction by the final layer of the large language model; Step 3: Obtain the probability distribution of the next token prediction for layers 0 to N-1 of the large language model except the final layer; Step 4: Obtain the probability distribution result of the hallucination information screening model, stack the probability distribution obtained in step 3 as the input of the hallucination information screening model, and output the probability distribution of the hallucination information screening model; Step 5: Compare and decode, and take the difference between the probability distribution predicted by the final layer of the large language model and the probability distribution obtained by the hallucination information screening model as the probability distribution of the next token; Step 6: Decode the probability distribution after comparison decoding into text, and decode the probability distribution into text through greedy decoding.

Citation Information

Patent Citations

  • Multi-modal large model fine tuning method and device, computer equipment and storage medium

    CN117095257A

  • Method and apparatus for generating appropriate responses based on the user intent in an ai chatbot through retrieval-augmented generation

    KR102681147B1

Cited By

  • Method and system for relieving illusion of large language model

    CN119149718A

  • HAE rare disease risk prediction method and device based on AI

    CN119230116A

  • System for monitoring and intervening large model to generate illusion content in real time

    CN119357759A

  • Large language model fact enhancement decoding method based on part-of-speech evaluation

    CN119558280A

  • Large model illusion removal method based on multilayer fusion contrast decoding framework

    CN119886323A