Large language model fact enhancement decoding method based on part-of-speech evaluation

Abstract

The invention discloses a large language model fact enhancement decoding method based on part-of-speech evaluation. The method mainly comprises the following steps: carrying out part-of-speech distribution analysis of an illusion part on a text in each layer of a large language model, and determining an intervention target part-of-speech; based on the intervention target part of speech, determining an output probability trend of the fact vocabulary and the illusion vocabulary in the same sentence; establishing an evaluator for different intervention target part-of-speech, and training the evaluator based on the output probability trend of the fact vocabulary and the illusion vocabulary in the same sentence; in the decoding process of the text generated by the large language model, the trained evaluator is used for monitoring model output in real time, potential illusion signals are detected, and options in the fact text generation process of the large language model are dynamically adjusted according to the detected illusion signals. Compared with a traditional method, the method has the advantages that the fact accuracy of the generated content can be remarkably improved, and external resources or models are not needed.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a large language model fact-enhanced decoding method based on part-of-speech judgment. Background Art

[0002] In recent years, large language models (LLMs) have demonstrated unprecedented performance in a variety of tasks, from machine translation to text generation to question-answering systems. However, despite their excellent performance in various application scenarios, LLMs tend to produce hallucinations, that is, when generating text, the model may generate information that is inconsistent with the facts or contradicts the previous context. This feature will undermine users' trust in the model and may bring misleading information, posing a significant risk to practical applications.

[0003] In order to deal with the problem of hallucination in large language models, previous research work has mainly focused on enhancing the authenticity and consistency of model-generated content by introducing external knowledge sources. For example, some researchers have tried to integrate knowledge graphs or professional databases into the model to correct the erroneous information generated by the model. However, this method not only increases the computational overhead, but also has high requirements on the quality of the knowledge source. High-quality knowledge sources are often difficult to obtain and need to be updated frequently to maintain timeliness. In addition, even with such a knowledge base, how to effectively integrate this knowledge into the model itself is also a challenge, because it involves complex model design and training processes. Summary of the invention

[0004] To solve the above technical problems, the present invention provides a large language model fact enhancement decoding method based on part-of-speech judgment, which improves the factual accuracy of generated text by dynamically monitoring and intervening in the behavior of the model in the decoding stage. In this way, even without relying on external knowledge bases or models, the factuality of generated text can be significantly improved.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows: A large language model fact-enhanced decoding method based on part-of-speech judgment, the method comprising the following steps: Perform part-of-speech distribution analysis on the hallucinated parts of the text in each layer of the large language model to determine the target part-of-speech for intervention; Based on the intervention target part of speech, determine the output probability trend of factual words and hallucination words in the same sentence; Construct a judge for different intervention target parts of speech, and train the judge based on the output probability trend of factual words and hallucination words in the same sentence; During the decoding process of text generated by a large language model, the trained judge is used to monitor the model output in real time, detect potential hallucination signals, and dynamically adjust the options in the factual text generation process of the large language model based on the detected hallucination signals.

[0006] The beneficial effects of the present invention are: This paper focuses on using part-of-speech attributes as the basis for controlling the generation mechanism. Starting from the inside of the model, it adaptively detects potential hallucinations for different parts of speech (such as nouns, verbs, etc.), and improves the factual accuracy of the generated content by dynamically monitoring and intervening in the decoding process of the model. The advantage of this method is that it does not require additional knowledge sources, but relies on the internal mechanism of the model itself and combines grammatical characteristics to enhance the reliability of the generated content. Experimental results show that the proposed method significantly improves factuality without introducing additional knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A network block diagram of a large language model fact enhancement decoding method based on part-of-speech judgment of the present invention; Figure 2 Dynamic graphs that output probabilities for different parts of speech; Figure 3 The experimental results of different search space sizes are shown in Figure 2. DETAILED DESCRIPTION

[0008] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0009] Figure 1 This is a network block diagram of a large language model fact-enhanced decoding method based on part-of-speech judgment of the present invention. The specific steps of the method are as follows: Step 1: Analyze the part-of-speech distribution of the hallucinated part of the text in each layer of the large language model to determine the target part-of-speech for intervention. Statistical analysis is performed based on two datasets, FAVABENCH and HADES, which provide fine-grained annotations. In the statistical experiment, the annotated hallucinated part of the dataset is selected, and Spacy is used to perform part-of-speech tagging, and the proportion of each part-of-speech category is calculated. Without considering functional words, the top 5 parts of speech are selected according to the average proportion ranking for in-depth analysis. In addition, it is worth noting that since there are only a few examples of NUM-type hallucinations in HADES, simple replacement samples were deleted during the manual annotation stage, but since NUM is also an important part of the content, it is also taken into account. Therefore, a total of six parts of speech, VERB, NOUN, ADJ, PROPN, PRON, and NUM (i.e., verbs, nouns, adjectives, proper nouns, pronouns, and numerals) are selected as the intervention target parts of speech for in-depth analysis.

[0010] Step 2. Based on the intervention target part of speech, determine the output probability trend of factual words and hallucination words in the same sentence; follow the data format of HADES, use the original data sampled from the English WIKI40B dataset, and directly prompt ChatGPT to disturb each sentence in the paragraph to construct the research dataset. Specifically, select sentences with at least 10 words, select words that belong to the intervention target part of speech category, and then prompt the model to replace the selected word with another word with the same part of speech and similar semantics but actually wrong. In order to observe the internal trend of LLM, the observations of the six part-of-speech categories are visualized through Logit Lens. As Figure 2 As shown in Figure 2, by observing the output probability trends of hallucinations and non-hallucinations, it is found that factual words and hallucination words show almost the same dynamic trends in the lower layers, but show different trends in the higher layers. At the same time, there are also differences in the internal trends of different part-of-speech categories. Some categories (such as NOUN) show the most significant differences in the middle layers, while other categories (such as NUM and VERB) show a blurred boundary between factual words and hallucination words.

[0011] Step 3. Construct and train judges for different intervention target parts of speech. Based on the above observations, cross-layer probability trends are used for hallucination detection. Since internal trends show differences under different parts of speech, adaptively selecting hierarchical features can produce better detection results. Grid search is performed on different parts of speech to obtain the optimal selection of layer information corresponding to the best classification accuracy, and a simple SVM model is trained. The SVM model is implemented using the SVC class and default hyperparameters from scikit-learn. A negative logarithmic transformation is applied to the output probability and used as the input of the SVM model. The results are shown in Table 1, indicating that even with the simplest SVM classifier, the classification performance for various part-of-speech categories can still reach more than 80%. In addition, training part-of-speech specific classifiers can achieve better performance than ordinary classifiers that use information from all layers without distinguishing parts of speech, highlighting the necessity of adaptive detection.

[0012] Table 1

[0013]

[0014] Step 4: During the decoding process of the text generated by the large language model, the trained judge is used to monitor the model output in real time to detect potential hallucination signals. According to the detected hallucination signals, the options in the fact text generation process of the large language model are dynamically adjusted. The judge constructed by the method has proven to be reliable in distinguishing the nuances of fine-grained facts, providing an effective detection signal for model generation. By using this signal as a guide, real-time fact generation can be achieved. The specific operations are as follows: Given prompt ,in Representing each token in the prompt, the probability distribution of the next token generated by the large language model is recorded as Then cut off the top-k tokens and perform part-of-speech tagging, select the token corresponding to the most frequent part-of-speech and obtain the candidate set: , , in, Indicates the mark for truncating top-k. represents the part of speech of the candidate tag t, That is to perform the part-of-speech tagging process. To count the number of occurrences of each part of speech, To select the most frequent part of speech, is to select the set of tags corresponding to the most frequent part of speech, is the indicator function, when The value is 1 when it is, otherwise it is 0. Indicates that the tag set corresponding to the most frequent part of speech is selected; Next, for each candidate marker , select the corresponding layer information and judge The detection is performed, and the token with the highest score will be the next generated token. Repeat the above steps until the stopping condition (such as the maximum generation length) is reached: , in, is the candidate marker t in the The output probability of the layer, Indicates that the candidate tag t has a part of speech The optimal number of layers corresponding to is the SVM model for the candidate tag t corresponding to the part of speech, receiving the negative log probability of the selection layer as input and outputting the fact score, Indicates the final selection as the tag to be generated in the next step.

[0015] In order to illustrate the effectiveness of the system, the present invention conducted experiments to verify the performance of the method.

[0016] (1) Test benchmark; TruthfulQA dataset: TruthfulQA is a benchmark for measuring the authenticity of language models in question-answering tasks. This paper focuses on the hallucinations produced by the model in open text generation and evaluates its performance on generation tasks. According to the evaluation provided by TruthfulQA, the metrics used include "Truth", "Info" and "Truth Info”, they represent the percentage of true, informative, or both of the generated answers.

[0017] TriviaQA dataset: TriviaQA is a commonly used question-answering benchmark. It includes 95K annotated question-answer pairs. We use Exact Match and F1 score to evaluate the correctness of the generation.

[0018] (2) Comparative method; The proposed method is compared with the following decoding methods, including: 1) Greedy Decoding, which greedily selects the next token with the highest probability; 2) Inference Time Intervention (ITI), which attempts to improve factuality by changing model activations; 3) Fact Kernel Sampling, which dynamically adjusts the sampling kernel to reduce randomness; 4) Contrastive Decoding (CD), which compares the output distribution between expert and amateur models.

[0019] About implementation details: All experiments were implemented on a server with 10 RTX3090 GPUs. Experiments were conducted with the LLaMA family, all of which baselines use LLaMA-7b and LLaMA-13b as their backbones. In addition, LLaMA-13b is used for comparison with LLaMA-7b. When using the proposed method on TruthfulQA and TriviaQA, the judges are trained with a train / dev / test (8 / 1 / 1) split on a self-constructed dataset without any overlap with the benchmark data.

[0020] About the experimental results: The results are shown in Table 2. On TruthfulQA, the proposed method is slightly inferior to the fact core sampling on LLaMA-7b. As the model capacity increases, the performance of the method continues to improve, and the best Truth and Truth on LLaMA-13b are obtained. Info score. On TriviaQA, the proposed method achieved the best results on both LLaMA-7b and LLaMA-13b. In general, the proposed method outperforms other decoding methods on most benchmark indicators and does not use additional knowledge resources or models, which shows the advancement and effectiveness of the method.

[0021] Table 2

[0022]

[0023] In order to verify the effectiveness of the critic, the present invention compares the task performance under different critics. Experiments were conducted on TruthfulQA using LLaMA-7b and LLaMA-13b. General critics do not consider the difference in POS, while the present invention uses specific critics to provide guidance for tokens with different parts of speech. For layer-wise critics, the present invention further selects different inter-layer information on different parts of speech as the critic input. The task performance under the three critics is shown in Table 3. The layer-wise critic achieved the most significant improvements in authenticity and informativeness generation on both models, and most critics helped with authenticity. The results show that the guidance provided by the critic can improve the performance of the model.

[0024] Table 3

[0025]

[0026] To determine the impact of the size of the search space k on task performance, the performance of using the inter-layer judge on TruthfulQA is measured at different search beam values. Figure 3 As shown, it can be observed that the performance of LLaMA-7b first improves and then decreases as k increases, indicating that the search beam plays an important role in the reasoning process and can directly affect the authenticity of large language models. However, experiments on LLaMA-13b show smaller performance fluctuations and only drop at very large k values, indicating that larger models are less sensitive to k because they effectively prioritize high-quality tokens. All results are better than greedy decoding, demonstrating the effectiveness of expanding the search space.

[0027] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large language model fact-enhanced decoding method based on part-of-speech judgment, characterized in that: The method comprises the following steps: Perform part-of-speech distribution analysis on the hallucinated parts of the text in each layer of the large language model to determine the target part-of-speech for intervention; Based on the intervention target part of speech, determine the output probability trend of factual words and hallucination words in the same sentence; Construct a judge for different intervention target parts of speech, and train the judge based on the output probability trend of factual words and hallucination words in the same sentence; During the decoding process of text generated by a large language model, the trained judge is used to monitor the model output in real time, detect potential hallucination signals, and dynamically adjust the options in the factual text generation process of the large language model based on the detected hallucination signals.

2. According to the large language model fact enhancement decoding method based on part-of-speech judgment according to claim 1, it is characterized in that: The method of performing part-of-speech distribution analysis on the hallucinated part of the text in each layer of the large language model and determining the intervention target part-of-speech includes, in a statistical experiment, selecting the annotated hallucinated part in the data set, and using Spacy to perform part-of-speech tagging, calculating the proportion of each part-of-speech category, and selecting six parts of speech, including verbs, nouns, adjectives, proper nouns, pronouns, and numerals, as the intervention target parts-of-speech according to the average proportion ranking.

3. The large language model fact-enhanced decoding method based on part-of-speech judgment according to claim 1 is characterized in that: The method of determining the output probability trend of factual vocabulary and hallucination vocabulary in the same sentence based on the intervention target part of speech includes: selecting a sentence with at least 10 words, selecting words belonging to the intervention target part of speech category, and then prompting the large language model to replace the selected words with another word that has the same intervention target part of speech and is semantically similar but actually wrong, namely, the hallucination vocabulary, and counting the output probability trends of factual vocabulary and hallucination vocabulary at different layers of the large language model.

4. The large language model fact-enhanced decoding method based on part-of-speech judgment according to claim 1 is characterized in that: The method of constructing an evaluator for different intervention target parts of speech and training the evaluator based on the output probability trend of factual words and hallucination words in the same sentence includes: using the output probability trend across layers to detect hallucination words, performing grid search on different parts of speech to obtain the optimal selection of layer information corresponding to the best classification accuracy; using the SVC class and default hyperparameters from scikit-learn to construct and train an SVM model, applying a negative logarithmic transformation to the output probability trend, and using the output probability trend as the input of the SVM model.

5. The large language model fact-enhanced decoding method based on part-of-speech judgment according to claim 1 is characterized in that: The method of using a trained judge to monitor the model output in real time, detecting potential hallucination signals, and dynamically adjusting the options in the fact text generation process of the large language model according to the detected hallucination signals includes: Step a: Given a prompt ,in Representing each token in the prompt, the probability distribution of the next token generated by the large language model is recorded as , truncate the top-k tokens and perform part-of-speech tagging, select the token corresponding to the most frequent part-of-speech and obtain the candidate set; Step b, for each candidate tag, select the corresponding hierarchical information and judge for detection, the tag with the highest score will be the next generated tag, and repeat steps a-b until the stopping condition is reached.

Images