A large language model fact enhancement decoding method based on part-of-speech evaluation

Abstract

The application discloses a large language model fact enhancement decoding method based on part-of-speech evaluation. The method includes the following main steps: performing part-of-speech distribution analysis on the hallucination part of the text in each layer of the large language model to determine the intervention target part-of-speech; based on the intervention target part-of-speech, determining the output probability trend of the fact vocabulary and the hallucination vocabulary in the same sentence; constructing an evaluator for different intervention target part-of-speech, and training the evaluator based on the output probability trend of the fact vocabulary and the hallucination vocabulary in the same sentence; in the decoding process of the large language model generating text, the trained evaluator is used to monitor the model output in real time, detect potential hallucination signals, and dynamically adjust the options in the fact text generation process of the large language model according to the detected hallucination signals. Compared with the traditional method, the application can significantly improve the fact accuracy of the generated content without the help of external resources or models.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a large language model fact-enhanced decoding method based on part-of-speech judgment. Background Art

[0002] In recent years, large language models (LLMs) have demonstrated unprecedented performance across a wide range of tasks, achieving remarkable success in fields ranging from machine translation to text generation to question-answering systems. However, despite their impressive performance across a wide range of application scenarios, LLMs are prone to hallucinations, where the models may generate text that is inconsistent with facts or contradicts prior context. This characteristic undermines user trust in the models and can lead to misleading information, posing a significant risk to real-world applications.

[0003] To address the problem of hallucinations in large language models, previous research has focused on enhancing the authenticity and consistency of model-generated content by introducing external knowledge sources. For example, some researchers have attempted to integrate knowledge graphs or professional databases into the model to correct the erroneous information generated by the model. However, this approach not only increases computational overhead, but also places high demands on the quality of the knowledge source. High-quality knowledge sources are often difficult to obtain and require frequent updates to maintain timeliness. In addition, even with such a knowledge base, how to effectively integrate this knowledge into the model itself is also a challenge, as it involves complex model design and training processes. Summary of the Invention

[0004] To address the above technical issues, the present invention provides a method for fact-enhanced decoding of large language models based on part-of-speech analysis. This method improves the factual accuracy of generated text by dynamically monitoring and intervening in the model's behavior during the decoding phase. This approach significantly improves the factuality of generated text, even without relying on external knowledge bases or models.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A large language model fact-enhanced decoding method based on part-of-speech judgment, the method comprising the following steps:

[0007] Perform part-of-speech distribution analysis on the hallucinated text in each layer of the large language model to determine the target part-of-speech for intervention;

[0008] Based on the intervention target part of speech, determine the output probability trend of factual words and hallucination words in the same sentence;

[0009] Construct a discriminator for different intervention target parts of speech and train it based on the output probability trend of factual words and hallucination words in the same sentence;

[0010] During the decoding process of text generated by a large language model, a trained judge is used to monitor the model output in real time, detect potential hallucination signals, and dynamically adjust the options in the fact text generation process of the large language model based on the detected hallucination signals.

[0011] The beneficial effects of the present invention are:

[0012] This paper focuses on leveraging part-of-speech attributes as the basis for controlling the generation mechanism. Starting from within the model, it adaptively detects potential hallucinations for different parts of speech (such as nouns and verbs). This approach improves the factual accuracy of generated content by dynamically monitoring and intervening in the model's decoding process. This approach has the advantage of not requiring additional knowledge sources, but instead relying on the model's internal mechanisms and incorporating grammatical properties to enhance the reliability of generated content. Experimental results show that the proposed method significantly improves factuality without introducing additional knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a network diagram of a large language model fact-enhanced decoding method based on part-of-speech judgment in the present invention;

[0014] Figure 2 A dynamic graph outputting probabilities for different parts of speech;

[0015] Figure 3 The experimental results of different search space sizes are shown in Figure 2. DETAILED DESCRIPTION

[0016] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0017] Figure 1 This is a network diagram of a large language model fact-enhanced decoding method based on part-of-speech judgment in the present invention. The specific steps of the method are as follows:

[0018] Step 1: Analyze the part-of-speech distribution of hallucinated text in each layer of the large language model to determine the target part-of-speech for intervention. Statistical analysis was conducted based on two datasets with fine-grained annotations: FAVABENCH and HADES. In this statistical experiment, annotated hallucinated text from the datasets was selected and tagged using Spacy. The proportion of each part-of-speech category was calculated. Without considering function words, the top five parts-of-speech were ranked by average proportion for further analysis. It is also worth noting that while HADES contains only a few examples of NUM-type hallucinations, simple replacement samples were removed during the manual annotation phase. However, since NUM is a significant component of the content, it was also taken into account. Therefore, a total of six parts-of-speech (VERB, NOUN, ADJ, PROPN, PRON, and NUM) (verb, noun, adjective, proper noun, pronoun, and numeral) were selected as target parts-of-speech for further analysis.

[0019] Step 2. Based on the intervention target part of speech, determine the output probability trend of factual words and hallucination words in the same sentence; follow the data format of HADES, use the original data sampled from the English WIKI40B dataset, and directly prompt ChatGPT to disturb each sentence in the paragraph to construct the research dataset. Specifically, select sentences with at least 10 words, select words that belong to the intervention target part of speech category, and then prompt the model to replace the selected word with another word with the same part of speech and similar semantics but actually wrong. In order to observe the internal trend of LLM, the observations of the six part of speech categories are visualized by Logit Lens. Figure 2 As shown in Figure 2, by observing the output probability trends for hallucinated and non-hallucinated words, we find that factual and hallucinated words exhibit nearly identical dynamic trends at lower layers, but differ at higher layers. Furthermore, internal trends within different part-of-speech categories also differ. Some categories (such as NOUN) show the most significant differences at intermediate layers, while other categories (such as NUM and VERB) exhibit a blurred boundary between factual and hallucinated words.

[0020] Step 3: Construct and train a discriminator for different intervention target parts of speech. Based on the above observations, we exploit cross-layer probabilistic trends for hallucination detection. Because internal trends exhibit differences across different parts of speech, adaptively selecting layer-by-layer features can produce better detection results. A grid search was performed across different parts of speech to obtain the optimal selection of layer information corresponding to the best classification accuracy, and a simple SVM model was trained. The SVM model was implemented using the SVC class with default hyperparameters from scikit-learn. A negative logarithmic transformation was applied to the output probabilities, and this was used as input to the SVM model. The results, shown in Table 1, demonstrate that even with the simplest SVM classifier, classification performance for various part-of-speech categories can still reach over 80%. Furthermore, training part-of-speech-specific classifiers can achieve better performance than a general classifier that utilizes information from all layers without distinguishing between parts of speech, highlighting the necessity of adaptive detection.

[0021] Table 1

[0022]

[0023] Step 4: During the decoding process of the text generated by the large language model, the trained judge is used to monitor the model output in real time to detect potential hallucination signals. Based on the detected hallucination signals, the options for the large language model's factual text generation process are dynamically adjusted. The judge constructed by this method has demonstrated the reliability of distinguishing the nuances of fine-grained facts, providing an effective detection signal for model generation. By using this signal as a guide, real-time fact generation can be achieved. The specific operation is as follows:

[0024] Given prompt ,in Representing each token in the prompt, the probability distribution of the next token generated by the large language model is recorded as Then we cut off the top-k tokens and perform part-of-speech tagging, select the token corresponding to the most frequent part-of-speech and obtain the candidate set:

[0025] ,

[0026] ,

[0027] in, Indicates the mark for truncating top-k. represents the part of speech of the candidate tag t, This is to perform the part-of-speech tagging process. To count the number of occurrences of each part of speech, To select the most frequent part of speech, is to select the set of tags corresponding to the most frequent part of speech, is the indicator function, when If yes, it takes the value 1, otherwise it takes the value 0. Indicates that the tag set corresponding to the most frequent part of speech is selected;

[0028] Next, for each candidate marker , select the corresponding layer information and judge The detection is performed, and the token with the highest score will be the next generated token. Repeat the above steps until the stopping condition (such as the maximum generated length) is reached:

[0029] ,

[0030] in, Is the candidate marker t in the The output probability of the layer, Indicates that the candidate tag t has a part of speech The optimal number of layers corresponding to is an SVM model for the part of speech of the candidate tag t, which receives the negative log probability of the selection layer as input and outputs the fact score, Indicates the final tag selected as the next step to be generated.

[0031] In order to illustrate the effectiveness of the system, the present invention conducted experiments to verify the performance of the method.

[0032] (1) Test benchmark;

[0033] TruthfulQA dataset: TruthfulQA is a benchmark for measuring the authenticity of language models in question-answering tasks. This paper focuses on the hallucinations produced by the model in open text generation and evaluates its performance on generation tasks. According to the evaluation provided by TruthfulQA, the metrics used include "Truth", "Info" and "Truth Info”, which represent the percentage of true, informative, or both true and informative answers.

[0034] TriviaQA dataset: TriviaQA is a commonly used question-answering benchmark. It includes 95K annotated question-answer pairs. We use Exact Match and F1 score to evaluate the correctness of the generated pairs.

[0035] (2) Comparative method;

[0036] The method of the present application is compared with the following decoding methods, including: 1) greedy decoding: greedily selecting the next token with the highest probability; 2) inference-time intervention (ITI), trying to improve factualness by changing model activations; 3) factored kernel sampling, dynamically adjusting the sampling kernel to reduce randomness; 4) contrastive decoding (CD), comparing the output distribution between expert and amateur models.

[0037] Regarding implementation details:

[0038] All experiments were implemented on a server with 10 RTX3090 GPUs. Experiments were conducted with the LLaMA family, and all these baselines used LLaMA-7b and LLaMA-13b as their backbone. In addition, LLaMA-13b was used for comparison with LLaMA-7b. When using the method of the present application on TruthfulQA and TriviaQA, the evaluator will accept training on a self-constructed dataset with training, development, and testing (8 / 1 / 1) partitions, and there is no overlap with the benchmark data.

[0039] Regarding experimental results:

[0040] The results are shown in Table 2. On TruthfulQA, the method of the present application is slightly inferior to factored kernel sampling on LLaMA-7b. With the increase of model capacity, the method performance is constantly improved, and the best Truth and Truth Info scores are obtained on LLaMA-13b. On TriviaQA, the method of the present application achieves the best results on both LLaMA-7b and LLaMA-13b. Overall, the method of the present application outperforms other decoding methods on most benchmark indicators, and no additional knowledge resources or models are used, which demonstrates the advancement and effectiveness of the method.

[0041] Table 2

[0042]

[0043] To verify the effectiveness of the critics, the invention compares the task performance under different critics. Experiments are conducted on TruthfulQA using LLaMA-7b and LLaMA-13b. General critics do not consider the distinction of POS, while the invention uses specific critics to provide guidance for tokens with different parts of speech. For layer-wise critics, the invention further selects different inter-layer information on different parts of speech as critic input. The task performance under the three kinds of critics is shown in Table 3. The layer-wise critics achieve the most significant improvement in the truthfulness and informativeness generation on both models, and most of the critics help the truthfulness. The results show that the guidance provided by the critics can improve the performance of the model.

[0044] Table 3

[0045]

[0046] To determine the impact of the size k of the search space on the task performance, the performance of the layer-wise critics on TruthfulQA is measured under different search beam values. The results, as shown in Figure 3 , it can be observed that the performance of LLaMA-7b first improves and then declines with the increase of k, which indicates that the search beam plays an important role in the inference process and can directly affect the truthfulness of large language models. However, the experiments on LLaMA-13b show smaller performance fluctuations and only decline at very large k values, which indicates that larger models are less sensitive to k because they effectively prioritize high-quality tokens. All results are superior to greedy decoding, proving the effectiveness of expanding the search space.

[0047] The specific embodiments described above further illustrate the objects, technical solutions, and advantages of the invention. It should be understood that the above description is only a specific embodiment of the invention and is not intended to limit the invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the invention should be included in the protection scope of the invention.

Claims

1. A large language model fact-enhanced decoding method based on part-of-speech judgment, characterized in that: The method comprises the following steps: Perform part-of-speech distribution analysis on the hallucinated text in each layer of the large language model to determine the target part-of-speech for intervention; Based on the intervention target part of speech, determine the output probability trend of factual words and hallucination words in the same sentence; Construct a discriminator for different intervention target parts of speech and train it based on the output probability trend of factual words and hallucination words in the same sentence; During the decoding process of the text generated by the large language model, the trained judge is used to monitor the model output in real time, detect potential hallucination signals, and dynamically adjust the options in the fact text generation process of the large language model based on the detected hallucination signals.

2. The large language model fact-enhanced decoding method based on part-of-speech judgment according to claim 1 is characterized in that: The part-of-speech distribution analysis of the hallucinated parts of the text in each layer of the large language model and the determination of the intervention target part-of-speech include, in a statistical experiment, selecting the annotated hallucinated parts in the dataset and using Spacy to perform part-of-speech tagging, calculating the proportion of each part-of-speech category, and selecting six parts of speech, including verbs, nouns, adjectives, proper nouns, pronouns, and numerals, as the intervention target parts-of-speech according to the average proportion ranking.

3. The large language model fact-enhanced decoding method based on part-of-speech judgment according to claim 1 is characterized in that: The method of determining the output probability trends of factual words and hallucination words in the same sentence based on the intervention target part of speech includes: selecting a sentence with at least 10 words, selecting words belonging to the intervention target part of speech category, and then prompting the large language model to replace the selected words with another word that has the same intervention target part of speech and is semantically similar but actually incorrect, namely, the hallucination word, and counting the output probability trends of the factual words and the hallucination words at different layers of the large language model.

4. The large language model fact-enhanced decoding method based on part-of-speech judgment according to claim 1 is characterized in that: The method of constructing an evaluator for different intervention target parts of speech and training the evaluator based on output probability trends of factual words and hallucination words in the same sentence includes: using output probability trends of different layers to detect hallucination words, performing grid search on different parts of speech to obtain the optimal selection of layer information corresponding to the best classification accuracy; using the SVC class and default hyperparameters from scikit-learn to construct and train an SVM model, applying a negative logarithmic transformation to the output probability trend, and using the output probability trend as input to the SVM model.

5. The large language model fact-enhanced decoding method based on part-of-speech judgment according to claim 1 is characterized in that: The method of using a trained judge to monitor the model output in real time, detecting potential hallucination signals, and dynamically adjusting the fact text generation process of the large language model based on the detected hallucination signals includes: Step a: Given a prompt ,in Representing each token in the prompt, the probability distribution of the next token generated by the large language model is recorded as , truncate the top-k tokens and perform part-of-speech tagging, select the token corresponding to the most frequent part-of-speech and obtain the candidate set; In step b, for each candidate tag, the corresponding hierarchical information and judge are selected for detection. The tag with the highest score will be the next generated tag. Repeat steps a-b until the stopping condition is reached.

Images