Large language model illusion detection method and system based on adaptive sampling strategy

Abstract

The invention discloses a large language model illusion detection method and system based on an adaptive sampling strategy, and the method comprises the steps: carrying out the word segmentation and coding of a to-be-detected text and each sampling text, obtaining the respective token identifiers, carrying out the splicing, obtaining a plurality of text pair sequences, and carrying out the coding processing, and obtaining the coding results of a plurality of text pairs; the coding results of the text pairs are input into a consistency evaluation model to be processed, and consistency scores of the multiple text pairs are obtained; and obtaining an embedded vector of a to-be-detected text, calculating statistical information of consistency scores of the plurality of text pairs, inputting the embedded vector and the statistical information into the sampling decision model for decision making, determining whether to perform sampling again, if so, performing sampling again, and if not, obtaining a hallucination detection result based on the consistency scores. A sampling decision is innovatively designed after consistency scoring, and the sampling decision realizes adaptive sampling and scoring in the illusion detection process by means of a near-end strategy gradient optimization training sampling decision model in reinforcement learning.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method and system for detecting hallucinations in large language models based on an adaptive sampling strategy. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] The rise of Large Language Models (LLMs) has revolutionized the field of natural language processing, bringing unprecedented machine text generation, translation, and comprehension capabilities and greatly expanding the boundaries of technological applications. However, the learning from massive corpora and the complex training process are a double-edged sword. While endowing models with powerful generative and comprehension abilities, they also introduce a significant limitation: the "illusion phenomenon." This phenomenon stems from the fact that massive corpora inevitably contain "dirty" data that is difficult to thoroughly identify and remove. Furthermore, generative models may produce outputs that do not reflect reality due to biases in data understanding or algorithmic limitations during training and inference. These factors combine to lead to the "illusion phenomenon." Specifically, LLMs may generate content that contradicts user input, conflicts with previously generated context, or is inconsistent with accepted world knowledge in certain situations. This inconsistency can not only lead to user misunderstanding but also seriously question the reliability of the model.

[0004] Model illusions can be categorized into two types: internal illusions, where the content generated by the large model contradicts the source content; and external illusions, where the content generated by the large model cannot be verified against the source content (the output is neither supported nor refuted by the source content). Compared to internal illusions, external illusions lack reliable sources of information, which undoubtedly increases the difficulty of detecting them. Therefore, detecting external illusions has gradually become an important research focus and trend.

[0005] Currently, computational methods for large-scale hallucination detection have become a focal point of in-depth exploration for many researchers. Early research focused on detecting text-level hallucinations, such as the FAST model developed by Zhong et al., which represents the factual structure of a given document as an entity graph and further utilizes graph neural networks to learn sentence representations to detect whether the text is fake news generated by the model. However, this method can only identify whether the text is generated by the model, but cannot identify the specific content that contains hallucinations.

[0006] Later research focused more on sentence-level and word-level analysis. Some studies were based on supervised learning techniques; for example, Liu et al. proposed a no-reference, word-level hallucination detection baseline, HADES, and fine-tuned models such as GPT-2 and BERT, with GPT-2 achieving the best results, reaching an F1-score of 68.40% for hallucination detection on this baseline. Nouha et al. provided high-quality datasets for knowledge-based dialogues through systematic data cleaning and rigorous annotation, and trained a RoBERTa-based classifier, FAITHCRITIC, improving accuracy on multiple test sets. However, these methods are mostly designed for specific task datasets, lack generalization ability, require large amounts of labeled data, and may perform poorly when handling complex or ambiguous errors.

[0007] Some research is based on external knowledge bases. For example, Zhou et al. developed a graph-based fact verification method that constructs a fully connected evidence graph to facilitate information propagation between evidence. The constructed graph serves as a feature, and a classifier is used to determine whether the evidence supports or refutes a given statement. The problem with this method is its low accuracy; it achieved 67.10% accuracy on the FEVER dataset. Sewon et al. proposed the FactScore method, which segments long texts into a series of smaller text fragments and calculates the percentage of these smaller fragments that have been verified as correct by reliable knowledge sources. Furthermore, the authors developed automated models based on LLAMA or ChatGPT that can estimate FactScore with an error rate below 2%. RAGTruth is a hallucination detection corpus containing approximately 18,000 responses generated by the RAG model, with the hallucination level of each response manually annotated to evaluate the model's accuracy when referencing knowledge base content. Experimental results show that training a small-scale model using high-quality knowledge base data can achieve hallucination detection performance comparable to larger models. Methods based on external knowledge bases have an advantage in accuracy, but ensuring the reliability of external knowledge bases requires additional work.

[0008] Recently, some methods have emerged that leverage model uncertainty, utilizing multiple rounds of results generated by a large model to detect hallucinations by comparing the consistency of the generated content. These methods assume that the model should exhibit consistency when answering the same or similar questions; inconsistencies may indicate hallucinations. For example, Manakul et al. developed a method called selfcheckGPT. This method uses the consistency of multiple model sampling to evaluate generated hallucinations, achieving a sentence-level AUC-PR of 87.33% and a paragraph-level Pearson relevance of 69% on the wiki_bio_gpt3_hallucintaion dataset (hereinafter referred to as the WBGH dataset). This method has been used as a baseline model in many subsequent hallucination detection works. In contrast, Mündler et al. used the contradictions in the model's multiple generated texts as a means of detecting hallucinations. Specifically, they triggered self-contradictions through constraints, generated related sentence pairs, used various cueing strategies to detect hallucinations, and developed an iterative hallucination mitigation program that edited the local text, removing conflicting information while maintaining the fluency and information content of the original text. Manakul's method has a major limitation: it requires a significant investment of resources, needing 20 generation iterations for a single concept. In contrast, the method proposed by Mündler et al. only requires two generation iterations, which may also lead to some misjudgments.

[0009] Output feature analysis methods focus on detecting whether a feature-generated model using natural language produces hallucinations. For example, Jiang et al., addressing the hallucination problem caused by LLMs based on known facts, proposed constructing a hallucination classifier by analyzing features such as text fluency, grammar, and logical coherence, achieving an accuracy of 85% in hallucination detection on the adjusted COUNTERFACT dataset. Meanwhile, Zhang et al. achieved hallucination detection by simulating human fact-checking methods, focusing on keywords in the text, checking for unreliable markers in the historical context, and considering marker attributes (such as type and frequency) to calculate a hallucination score. Using LLAMA at different scales as the base model, this model achieved an AUC-PR of 89.94% at the sentence level and a Pearson correlation of 77.15% at the paragraph level on the WBGH dataset. A drawback of output feature analysis methods is their reliance on manually constructed features, which may be biased or have unclear classification categories in some cases.

[0010] Furthermore, some studies have incorporated knowledge graph techniques to address the hallucination problem. For example, Hannah et al. explored the effect of training language models based on knowledge graphs on reducing hallucinations. This study systematically checked whether each triple in the response was consistent with the given context by structurally representing entities and relations in the text. The study showed that training with knowledge graphs could effectively reduce the hallucination rate on seen data, but it still suffered from poor generalization on unseen data.

[0011] In summary, the design process of hallucination detection methods is relatively uniform, mostly involving encoding text data, followed by feature extraction or enhancement, and then training a deep learning model to detect hallucinatory text. However, the techniques used in hallucination detection methods are diverse, including retrieval-augmented generation (RAG), knowledge graphs, and supervised learning. Furthermore, benchmarks are highly inconsistent; in many studies, researchers propose their own hallucination datasets or modify existing datasets. This results in trained models that may perform significantly within the dataset's distribution but exhibit poor generalization ability outside of it.

[0012] Existing hallucination detection methods based on sampled text consistency suffer from two major problems. First, they consume too many resources. To ensure detection performance, SelfCheckGPT uses 20 samples, meaning each judgment requires calling large model resources and consistency evaluation methods up to 20 times. Currently, there is limited research on reasonably reducing the number of samples. Research shows that only Wang et al. used Bayesian methods to reduce the number of recalled documents. Second, the consistency evaluation methods have shortcomings. SelfCheckGPT combines three consistency evaluation methods, but its performance still has significant room for improvement. Summary of the Invention

[0013] To overcome the shortcomings of the prior art, this invention provides a large language model hallucination detection method and system based on an adaptive sampling strategy. It innovatively designs a sampling decision after consistency scoring. The consistency scoring adopts a consistency evaluation method that combines unified modeling and hybrid expert networks. The sampling decision uses the proximal policy gradient optimization in reinforcement learning to train the sampling decision model, thereby realizing adaptive sampling and scoring in the hallucination detection process.

[0014] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention provides a large language model hallucination detection method based on an adaptive sampling strategy, comprising: Obtain the text to be detected and its multiple corresponding sampled texts; The text to be detected and each sampled text are segmented and encoded to obtain their respective token identifiers; the token identifier of the text to be detected is concatenated with the token identifier of each sampled text to obtain multiple text pair sequences, which are then encoded to obtain the encoding results of multiple text pairs. The encoding results of the text pairs are input into the consistency evaluation model for processing to obtain consistency scores for multiple text pairs; The embedding vector of the text to be detected is obtained, and the statistical information of the consistency score of multiple text pairs is calculated. The embedding vector and statistical information are input into the sampling decision model for decision-making. The sampling decision model decides whether to perform sampling again. If so, sampling is performed again. Otherwise, the hallucination detection result is obtained based on the consistency score.

[0015] A further technical solution yields the token identifier, specifically: Word segmentation technology is used to break down a single text into sub-word units; The sub-word units are transformed using a word segmenter to obtain the corresponding token identifier.

[0016] A further technical solution yields the encoding results of multiple text pairs, specifically: The token identifiers of the text to be detected and each sampled text are padded or truncated to obtain token identifiers of uniform length. The token identifier of the text to be detected is concatenated with the token identifier of a sampled text to obtain multiple text pair sequences; Encoding multiple text pairs is performed using a pre-trained language model to obtain the encoding results of multiple text pairs.

[0017] A further technical solution is that the consistency evaluation model includes a basic model layer and a hybrid expert layer. The basic model layer uses DeBERTa-V3 as the backbone, and the hybrid expert layer is a hybrid expert network spliced ​​after the last hidden layer of DeBERTa-V3.

[0018] In a further technical solution, the reward function of the sampling decision model takes the changing trend of the consistency score as the core and combines the standard deviation to measure the consistency convergence of the sampled text.

[0019] In a further technical solution, the total loss of the sampling decision model is a weighted sum of the strategy loss and the value loss, wherein the strategy loss is the generalized advantage estimation loss.

[0020] A further technical solution, wherein obtaining the hallucination detection result based on the consistency score specifically involves: calculating the average consistency score of multiple text pairs to obtain an average consistency score; comparing the average consistency score with a set hallucination threshold, and outputting non-hallucination if the score is greater than the set hallucination threshold, and outputting hallucination if the score is less than the set hallucination threshold.

[0021] Secondly, this invention provides a large language model hallucination detection system based on an adaptive sampling strategy, comprising: The text acquisition module is configured to acquire the text to be detected and its corresponding multiple sampled texts. The text encoding module is configured to: segment and encode the text to be detected and each sampled text to obtain their respective token identifiers; concatenate the token identifier of the text to be detected with the token identifier of each sampled text to obtain multiple text pair sequences; and after encoding processing, obtain the encoding results of multiple text pairs. The consistency evaluation module is configured to input the encoding results of the text pairs into the consistency evaluation model for processing, and obtain consistency scores for multiple text pairs. An adaptive sampling decision module is configured to: acquire the embedding vector of the text to be detected, calculate the statistical information of the consistency scores of multiple text pairs, input the embedding vector and statistical information into the sampling decision model for decision-making, and the sampling decision model decides whether to perform sampling again. If so, sampling is performed again; otherwise, the hallucination detection result is obtained based on the consistency score.

[0022] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the large language model illusion detection method based on an adaptive sampling strategy as described in the first aspect.

[0023] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the large language model illusion detection method based on an adaptive sampling strategy as described in the first aspect.

[0024] The above one or more technical solutions have the following beneficial effects: This invention, based on the idea of ​​using multiple sampling for consistency in hallucination detection, innovatively proposes a method to control the multiple sampling process. To ensure consistency with the text encoding method in the subsequent consistency scoring process, DeBERTa's word segmenter is used for word segmentation after collecting sample pairs. To more accurately calculate semantic consistency after removing text, the consistency evaluation model UDeMoE is used to calculate the consistency score of sample pairs. In experiments using different consistency scoring methods for hallucination detection, UDeMoE achieved the best results, proving its rationality as a consistency scoring method. To reduce the resource waste caused by multiple sampling and evaluation, an adaptive sampling decision module is constructed using a policy model trained on PPO. Through multiple sampling, evaluation, and hallucination score conversions, hallucination classification is ultimately achieved.

[0025] This invention proposes an adaptive sampling strategy that abandons the traditional fixed sampling method and dynamically selects more representative samples based on features such as the consistency score of text pairs, effectively improving the efficiency and accuracy of hallucination detection.

[0026] This invention innovatively proposes a method for training decision models using PPO (Procedure-Based Learning). By setting the PPO environment, reward function, and GAE (Government-Based Algorithm) loss function, the sampling strategy of the policy model can be efficiently optimized. In this process, the designed custom reward mechanism cleverly achieves semi-supervised learning, and successfully implements a decision-making process using text features as input through neural networks.

[0027] Experimental results from this invention show that using a decision model can significantly reduce the number of samples and improve detection performance. Compared with existing methods, HADAS exhibits superior performance on independent test sets. Resource consumption analysis shows that HADAS performs well in terms of sampling frequency, time, and inference resource requirements, fully demonstrating its high efficiency in practical applications.

[0028] This invention constructs an efficient closed-loop detection process from data collection, preprocessing (text encoding), consistency assessment, strategy model training to test evaluation, achieving fast and accurate hallucination detection, and has high practicality and scalability. Attached Figure Description

[0029] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0030] Figure 1 This is a framework diagram of the large language model illusion detection method based on an adaptive sampling strategy of the present invention; Figure 2 This is the strategy model and value model architecture of the embodiments of the present invention; Figure 3This refers to the AUC-PR of the model on the training and test sets under different discount factors in embodiments of the present invention. Figure 4 This refers to the AUC-PR of the model on the training and test sets under different cropping ranges in this embodiment of the invention. Figure 5 This refers to the AUC-PR of the model on the training and test sets under different training epochs in this embodiment of the invention. Figure 6 This is the hallucination detection effect of the model at the sentence level with and without a sampling decision module in this embodiment of the invention; Figure 7 This is the hallucination detection effect of the model at the paragraph level with and without a sampling decision module in this embodiment of the invention; Figure 8 This is an example of the hallucination detection performance at the sentence level using different algorithms employed in the sampling decision module of this invention. Figure 9 This is an example of the hallucination detection effect at the paragraph level using different algorithms in the sampling decision module of this invention. Detailed Implementation

[0031] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0032] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0033] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0034] Example 1 like Figure 1 As shown, this embodiment discloses a large language model hallucination detection method based on an adaptive sampling strategy. The method includes the following steps: S1: Obtain the text to be detected and its corresponding multiple sampled texts; In this embodiment, the hallucination detection method based on sampled text consistency requires multiple samplings of the text to be detected. This embodiment uses 19 samples from the WBGH dataset (wiki_bio_gpt3_hallucintaion) to provide the text to be detected. This dataset, proposed by SelfCheckGPT, randomly selects 238 articles from the top 20% of the longest data in the WikiBio dataset (composed of the first paragraphs of Wikipedia biographies), and then uses the concepts described in these articles to prompt the GPT3 model, ultimately obtaining 20 responses. The prompt paradigm is such as "This is a Wikipediapassage about {concept}." Subsequently, each sentence in the first sample for all concepts is labeled with whether it is true or not; the labels are divided into three-category labels and two-category labels.

[0035] The three-class labels are: "Mostly hallucinations (1)", "Minor hallucinations (0.5)", and "Non-hallucinations (0)". The two-class labels are: "Hallucinations" (1) and "Non-hallucinations" (0). It can be seen that the three-class labels are more accurate than the two-class labels. However, the consistency for the three-class labels is only 0.59, while the consistency for the two-class labels is 0.75. In order to train and evaluate the model's performance more reliably, this embodiment selects the two-class labels. The sentence label information of the WBGH dataset is shown in Table 1.

[0036] Table 1. Overview of sentence samples in the WBGH dataset

[0037] WBGH samples were collected and cleaned. The text to be detected, along with 19 sampled texts, was used to construct a benchmark dataset for hallucination detection, which was then split into training and test sets. Some unescaped newline characters appeared in the WBGH dataset samples; these were matched and removed in this embodiment to ensure the model focuses on the semantic information of the text. To verify the model's generalization ability, 20% of the samples were randomly selected to construct an independent test dataset, and the remaining samples were used as the training dataset.

[0038] S2: Segment and encode the text to be detected and each sampled text to obtain their respective token identifiers; concatenate the token identifier of the text to be detected with the token identifier of each sampled text to obtain multiple text pair sequences, and after encoding processing, obtain the encoding results of multiple text pairs; In this embodiment, the text sequence encoding is processed using the tokenizer of the DeBERTa (Decoding-enhanced BERT with Disentangled Attention) model. First, Byte Pair Encoding (BPE) technology is used to break down individual original texts (such as the text to be detected or a sampled text) into smaller sub-word units. For example, the word "unbelievable" can be broken down into sub-word fragments such as "un," "belie," and "vable." This decomposition method not only helps preserve the semantic information of the text but also significantly improves the model's ability to handle rare words. Subsequently, the SentencePiece-based tokenizer further converts these sub-word units into corresponding token identifiers (Token IDs). A Token ID is a numerical identifier that corresponds one-to-one with a word or sub-word in the pre-trained vocabulary. Through this conversion process, the original text is encoded into a fixed-length numerical sequence.

[0039] However, the varying lengths of different texts pose a challenge to the model's ability to handle them uniformly. To address this issue, padding or truncation operations were performed on the token ID sequences. For shorter text sequences, a special "[PAD]" marker was added to the end to bring their length to the maximum value set by the model; while for text sequences exceeding the set length, truncation was performed, retaining only the first part. In this way, the sequence lengths of all input models were standardized.

[0040] In the context of text consistency evaluation, the encoding process for text sequences was further optimized. Specifically, the method of processing text sequences during encoding was improved. The token identifiers obtained from encoding the two text segments involved in the evaluation (the text to be detected and its corresponding sampled text) are concatenated into a unified text pair sequence, separated by a special marker in the format "[CLS] Text 1 [SEP] Text 2 [SEP]". The [CLS] marker serves as the start identifier of the sequence, and the model aggregates the semantic information of the entire sequence through its hidden state. The [SEP] marker is used to explicitly distinguish between the two text segments, ensuring that the model can accurately identify the boundaries between different text paragraphs.

[0041] Finally, the text pair sequence (input sequence) obtained after the above series of processing and the corresponding attention mask are input into the DeBERTa model, which outputs the "hidden state at the [CLS] marker position". In other words, it outputs the encoded result of the text pair, namely a fixed-length sequence of Token IDs (including [CLS] / [SEP] markers), along with the attention mask. This hidden state aggregates the semantic information of the entire input sequence and is used to comprehensively analyze the semantic relationship between the two text segments, providing a key basis for the consistency score of the subsequent UDeMoE model.

[0042] Attention masks are generated synchronously during the text sequence encoding process when the token IDs sequence is padded or truncated. They are used to mark which positions in the sequence are actually valid tokens (not [PAD]) and which are padded invalid tokens ([PAD]), so as to ensure that the model ignores invalid positions when calculating attention.

[0043] S3: Input the encoding results of the text pairs into the consistency evaluation model for processing to obtain the consistency scores of multiple text pairs; In this embodiment, the large language model hallucination detection method based on an adaptive sampling strategy is named HADAS (Hallucination Detection based Adaptive Sampling), which employs a consistency evaluation method called UDeMoE. This method transforms data from different natural language processing tasks (such as natural language inference, question answering systems, fact verification, etc.) into consistency evaluation samples in a unified format, and then constructs a diverse training corpus based on these samples.

[0044] The consistency evaluation model UDeMoE (UnifiedDeBERTa with MoE) of Mixture-of-Experts (MoE) networks consists of a base model layer and a mixed expert layer.

[0045] The base model layer uses DeBERTa-V3 as its backbone, receives the encoding results of text pairs, extracts semantic features through a self-attention mechanism, and outputs the [CLS] label hidden state of the last hidden layer (aggregating the overall semantics of the text pairs).

[0046] The hybrid expert layer is a MoE network concatenated after the last hidden layer of DeBERTa-V3. It contains multiple expert sub-networks (each expert focusing on different task features, such as natural language reasoning and fact verification). Input features are dynamically routed to the corresponding experts through a gating network. The weighted expert outputs are then connected to the classifier, ultimately outputting a consistency score of 0-1 (1 for complete agreement, 0 for complete disagreement). This design allows different expert networks to specialize in processing textual information with different features, thereby enhancing the model's adaptability to various text formats.

[0047] Specifically, the encoding results of the text pairs are input into the consistency evaluation model, and the consistency scoring steps are as follows: (1) Feature extraction is performed in the base model layer to obtain the overall semantic representation of the text pair. Input the Token IDs and attention mask into DeBERTa-V3, calculate the hidden state of each layer, and take the hidden state at the [CLS] position of the last layer as the overall semantic representation of the text pair.

[0048] (2) Dynamic routing and scoring are performed in the hybrid expert layer to obtain a consistency score. Expert activation: The gating network calculates the activation probability of each expert based on the [CLS] hidden state and selects highly relevant experts; Expert calculation: The selected expert sub-networks process the [CLS] features respectively and output expert-specific consistency scores; Weighted combination: The gating weights are weighted and summed on the expert scores, and the final consistency score is output through the classifier.

[0049] The core of the consistency assessment model UDeMoE is to score the consistency between the text to be detected and a corresponding sampled text. The text pair obtained by splicing the above texts does not merge the two texts into a new text, but uses a structured format to allow the model to capture the semantics of the two texts at the same time and evaluate their consistency (such as whether the semantics match, whether there is a contradiction, etc.).

[0050] S4: Obtain the embedding vector of the text to be detected, and calculate the statistical information of the consistency scores of multiple text pairs. Input the embedding vector and statistical information into the sampling decision model for decision-making. The sampling decision model decides whether to perform sampling again. If so, sample again; otherwise, obtain the hallucination detection result based on the consistency score.

[0051] In this embodiment, the variance of consistency scores varies significantly across multiple samples for texts exhibiting different degrees of hallucination. For example, texts without hallucination show high consistency across multiple samples, resulting in a low variance in their consistency scores; however, texts with severe hallucination may show consistently low consistency scores across multiple samples, or they may fluctuate considerably, with a higher variance in the latter case. Therefore, uniformly reducing the number of samples is inappropriate; a method that can adjust the number of samples according to different characteristics and circumstances, i.e., an adaptive sampling strategy, should be adopted.

[0052] The main goal of adaptive sampling is to improve sampling efficiency and effectiveness, reduce unnecessary computation or resource consumption, and ensure the quality of sampling results. In this embodiment, the adaptive sampling strategy flow is as follows: (1) Sample the text to be detected to obtain a sampled text that is from the same source as the text to be detected.

[0053] (2) Calculate the consistency score between the text to be detected and the sampled text.

[0054] (3) Input the obtained information into the sampling decision model to determine whether to perform another sampling and calculation. The determination of whether supplementary sampling is needed is made by quantifying the "uncertainty of the current decision". If the uncertainty is too high (i.e. the existing information is insufficient to support a reliable decision), then resampling is triggered. Information refers to the comprehensive features that can reflect the current consistency assessment status, including the embedding vector of the text to be detected: a 768-dimensional vector generated by the pre-trained language model to capture the semantic and structural information of the text; consistency score and statistical information: the consistency score between the text to be detected and each sampled text that has been calculated, and the statistical features of these scores (such as mean and standard deviation); number of samplings: the number of samplings and consistency assessments that have been completed so far.

[0055] (4) If resampling is required, repeat steps (1)-(3); if resampling is not required, output the final result after hallucination score conversion. Based on the preset rules, the consistency score is converted to obtain the hallucination detection result. This result directly reflects whether the text (sentence or paragraph) to be detected is hallucinatory and the degree of hallucination. The judgment result is different depending on the detection level (sentence level or paragraph level). At the sentence level, by setting a threshold (such as 0.5), the average consistency score is converted into a binary label - "non-hallucination" (0) or "hallucination" (1); at the paragraph level, the hallucination scores of all sentences in the paragraph (sentence level result) are averaged to obtain a continuous score (not a classification label, but a quantitative indicator of the degree of hallucination) representing the overall degree of hallucination of the paragraph.

[0056] The sampling decision process is the core of the adaptive sampling process, as it determines whether the current evaluation state is reliable enough.

[0057] The sampling decision model is based on Proximal Policy Optimization (PPO). PPO can solve the typical sequential decision problem of dynamically adjusting the number of samples, and balances the accuracy of evaluation with the sampling cost through policy learning. In this embodiment, only weak labels such as generating consistency scores and text features exist; there are no explicit ground truth labels to indicate whether the decision is good or bad. PPO does not require ground truth labels; instead, it optimizes the policy by obtaining reward signals through interaction with the environment. To balance the accuracy of evaluation with the sampling cost, a reinforcement learning environment and reward function based on PPO were designed.

[0058] The sampling decision model based on near-end policy optimization is as follows: (1) Enhance the learning environment A reinforcement learning environment based on PPO is used to dynamically control the evaluation process of text consistency detection. The environment's input features include the embedding vector of the text to be detected and a consistency score based on the generated results (sampled text). Specifically, the text embedding, generated by a pre-trained language model, has a dimension of 768 and is used to capture the semantic and structural information of the text; the consistency score is quantified by the semantic distribution of multiple generation results, providing important information on detection reliability. The state of the environment is composed of the text embedding and statistical information of the consistency evaluation (including the mean score, standard deviation, and number of samples), forming a 771-dimensional feature vector. In each decision step, the sampling decision model can choose to continue sampling to improve evaluation reliability or stop evaluation to reduce computational cost.

[0059] By calculating the semantic consistency score between the text to be detected and multiple generated source texts (sampled texts), and then analyzing the distribution characteristics of these scores (such as mean, standard deviation, etc.), the reliability of the text to be detected can be quantitatively assessed. If the consistency between the multiple generated results and the text to be detected is high and stable (centralized distribution), the text to be detected is more likely not a hallucination; if the consistency is low or fluctuates greatly (dispersed distribution), it is more likely to be a hallucination.

[0060] (2) Reward function The reward function is designed by defining rules to dynamically balance the reliability of consistency evaluation with computational cost. This reward function focuses on the trend of consistency score changes, combined with the standard deviation to measure the consistency convergence of the generated results (sampled text). When the standard deviation of the consistency score decreases, the model receives a positive reward, with rewards segmented according to the absolute value of the standard deviation: the highest reward (2 points) is given when the standard deviation is below 0.2, a moderate reward (1 point) is given when it is between 0.2 and 0.3, and a penalty (-1 point) is given when it is above 0.3. Furthermore, if the direction of change in the standard deviation is unfavorable to convergence (e.g., an increase compared to the previous time), a further penalty (-1 point) is applied. Simultaneously, to limit computational cost, an increase in the number of evaluations is also penalized (-1 point).

[0061] In the early stopping mechanism of policy decision-making, when the standard deviation of the consistency score falls below a set early stopping threshold (e.g., 0.1), the model receives an additional high reward (10 points) to encourage the decision-making model to stop in a timely manner when the evaluation is sufficiently reliable, thus saving computational resources. Conversely, for actions that choose to stop before accumulating sufficient evaluation information (taking a "stop" action when the standard deviation evaluation is insufficient), a large negative reward (-10 points) is applied to prevent the policy from terminating the evaluation prematurely. Through the design of this reward function, the reinforcement learning agent can achieve an effective balance between detection reliability and evaluation cost.

[0062] (3) Advantage calculation Advantage measures the merit of an action relative to the average behavior in the current state, helping the policy network decide whether to take that action. This embodiment calculates the Generalized Advantage (GAE) using reward, state value, discount factor, and smoothing factor. It calculates the temporal difference error (TD error) and advantage at each time step through a backward iterative process. The specific calculation process is as follows: 1) For each time step, calculate the timing difference error using the difference between the current reward and the value of the next state and the current state value:

[0063] in, It is a discount factor. It is the value of the current state. It is the value of the next state. It is the immediate reward at the current time step. The value of each state is given by the value model.

[0064] 2) Calculate the current dominance value by weighting the timing difference error and the dominance value of the previous time step:

[0065] in, It is an advantage at this moment. It is an advantage in the next moment. It is a smoothing factor used to control the weighting of future advantages; Used to control the dominance function The calculation method, the bias and variance of the equilibrium estimate, and the range of values ​​are as follows: The advantages obtained from the above calculations are used to optimize the parameters of the PPO decision model.

[0066] (4) Loss Calculation 1) Calculate the action probability ratio between the current policy and the old policy:

[0067] in, Is the current strategy in the state? Select action The probability, Is the old strategy in the state? Select action The probability of.

[0068] 2) Calculate the cut probability ratio:

[0069] in, These are the pruning parameters. To avoid excessively large policy updates, the pruning probability ratio is set within a certain range. between.

[0070] 3) Calculate the loss function of the policy network:

[0071] Among them, advantages To measure the relative merits of actions, The operation ensures that the minimum value is selected within the clipping range, avoiding drastic changes to the strategy.

[0072] 4) Calculate the loss of the value function:

[0073] in, It's a discount reward. It is the state value predicted by the model.

[0074] The total loss of PPO is the weighted sum of strategy loss and value loss:

[0075] in, It is the weight of the adjustment term.

[0076] (5) Strategy network and value network structure The policy network and value network share the first few feature extraction layers of the network, and then calculate the policy and value separately. The model structure is as follows: Figure 2 As shown, this network contains five fully connected layers, with dimensions gradually decreasing from 771 to 64 to extract high-level features. Each layer is followed by batch normalization to stabilize the training process for vectors. The normalization calculation formula is:

[0077] in, It is a vector The i-th sample, It is after normalization , yes The mean, yes The variance. The normalized data is then scaled and shifted to restore its expressive power, as shown in the following expression:

[0078] in, It is the converted illusion score. and These are learnable parameters.

[0079] Simultaneously, the LeakyReLU activation function is employed to enhance the network's nonlinear expressive power for the input vector. LeakyReLU can be represented as:

[0080] in, It is a very small constant used to control the slope in the negative region.

[0081] The network ends with two heads: a Policy Head, which predicts the action probability distribution using Softmax activation; and a Value Head, which estimates the state-value function. This design, by increasing network depth and capacity, can more effectively capture complex state representations, thereby improving the stability and generalization ability of policy learning.

[0082] In this embodiment, hallucination detection is divided into sentence-level and paragraph-level. At the sentence level, after the text to be detected is processed by the sampling decision model, multiple samples and multiple consistency scores are obtained. Subsequently, the average of all sample consistency scores is calculated. Next, a hallucination threshold is set. If the average consistency score (the arithmetic mean of the consistency scores between the text to be detected and multiple sampled texts) is... If the output is "not a hallucination" (0), then the output will be "not a hallucination"; if the output is If the value is 1, then the output is "hallucination" (1). Paragraph-level hallucination detection cannot be simply classified into two levels. It is more reasonable to output a continuous score to represent the degree of hallucination. The value is derived by averaging the hallucination scores of each sentence in the paragraph.

[0083] In another implementation, a sampling decision model based on confidence intervals is used. The confidence interval is calculated based on the standard error and the Student's t-distribution. This method allows determination of the possible fluctuation range of scores at a given confidence level (e.g., 95%), thereby assessing the reliability of the consistency scores. The consistency score ranges from [0, 1], where 1 represents complete consistency and 0 represents complete inconsistency. A score boundary, such as 0.5, can be set to convert the consistency score into a hallucination category. If the lower limit of the confidence interval for multiple consistency scores is greater than the set boundary, it is determined that there is no hallucination; if the upper limit is less than the set boundary, it is determined that there is a hallucination; if the confidence interval exceeds the set range, it indicates insufficient model confidence, and rescoring is required. If the list of consistency scores is x, the confidence interval CI is:

[0084] in, It is the mean of the consistency score. is the critical value of the t-distribution, s is the fractional standard deviation, and n is the length of the list.

[0085] In another embodiment, a sampling decision model based on Bayesian optimization is employed. The Bayesian optimization-based consistency sampling decision method treats the sampling process as an optimization problem. Specifically, by employing surrogate models such as Gaussian process regression, the relationship between the quality of sampling decisions and sampling points is modeled, predicting the expected returns of different sampling strategies. In this invention, a sampling point is defined as... Let n, c, and v represent the number of samples, the context vector, and the consistency score, respectively. The quality of a decision is defined as the observed value, and its value is determined in the same way as the reward function of the PPO.

[0086] Suppose there is a set of sampling points The corresponding observed value is Gaussian processes assume that these observations follow a multivariate Gaussian distribution:

[0087] in, It is a kernel matrix, whose values ​​represent sampling points. and The similarity between them, where 0 is the zero vector. For a new input point... Gaussian processes can predict their target values. And its uncertainties.

[0088] In each round of sampling, a balance is struck between exploration and utilization using the acquisition function, thereby dynamically determining whether to continue sampling. The acquisition function, based on the desired improvement, can be expressed as:

[0089] in, , and These are the probability density function and the cumulative distribution function, respectively. yes standard deviation yes The mean, It is the maximum value of the observed value.

[0090] Guided by the acquisition function, the next sampling point is selected. Finally, the merits of the sampling decision are calculated at the new sampling point, and the new data points are added to the dataset. The Gaussian process model is then refitted using the updated dataset. This method effectively improves sampling quality, maximizes information gain, and avoids unnecessary oversampling.

[0091] In another implementation, a sampling decision model based on dynamic programming optimization is employed. The application of dynamic programming in consistent sampling decision-making models the sampling process as an optimization problem, recursively solving for the optimal sampling strategy at each state based on historical sampling results and consistency scores. Dynamic programming defines the value of each state, calculates the expected reward of each possible sampling action using the Bellman equation, and selects the action that maximizes future rewards. By recursively updating state values, dynamic programming can make the optimal sampling decision at each decision point until a stopping condition is met.

[0092] Specifically, the state includes the standard deviation of the consistency score, the number of evaluations, and historical information, and can be represented as follows: Actions include continuing sampling and stopping sampling. The state transition equation describes how to reach the next state given the current state and decision, and is defined as follows: If you choose to continue sampling, the state changes from... Transferred to ,in It is the new standard deviation. This becomes the new historical standard deviation. If sampling is stopped, the state will not change, but the decision-making process ends. It remains unchanged.

[0093] The payoff function is used to evaluate the merits of each state, similar in concept to the reward function in PPO. Value function. This represents the maximum payoff obtainable starting from state S and adopting the optimal strategy. In the dynamic programming solution process, based on the state transition and payoff function, the Bellman equation is established to recursively calculate the value function:

[0094] in, It is the immediate benefit of taking action a in state S. It is a discount factor. It is the value function after the state transition.

[0095] This method solves for the optimal strategy through iteration or backtracking, that is, by selecting the action with the greatest benefit in each state.

[0096] The experiment will be described below, and the results will be analyzed.

[0097] (1) Experimental setup In this experiment, the batch size was set to 32 to accommodate the small number of samples in the dataset, avoid overfitting, and improve the flexibility of model learning. The learning rate was chosen as 1e-5, combined with the adaptive characteristics of the AdamW optimizer, to fine-tune model parameters and ensure training stability and convergence. Ten training epochs were conducted to fully evaluate model performance with limited resources, preventing overfitting caused by overtraining. The training samples consisted of the first 1526 sentences from the WBGH dataset, forming 190 complete paragraph samples. The remaining 382 sentence samples (corresponding to 48 complete paragraph samples) formed the test set, ensuring independence between the test and training sets. After hyperparameter tuning experiments, the optimal hyperparameter size was selected. The hyperparameter settings for PPO are as follows: discount factor. and GAE parameters All values ​​are taken as 0.95, and the cutting range is... Set the threshold to 0.2. Set the value function loss weight and entropy loss weight to 0.5 and 0.01 respectively, and set the gradient clipping threshold to 0.5.

[0098] (2) Performance evaluation indicators To maintain consistency with previous hallucination detection methods, this embodiment uses AUC-PR as the evaluation index for sentence-level hallucination detection, and Pearson correlation coefficient and Spearman's rank correlation coefficient as evaluation indexes for paragraph-level hallucination detection.

[0099] The Precision (PR) curve is plotted with Precision on the y-axis and Recall (TPR) on the x-axis, also plotted by adjusting the threshold. It is more sensitive to model performance evaluation in cases of imbalanced samples, highlighting the change in precision under high recall. AUC-PR, the area under the PR curve, measures the model's overall performance on the PR plane. In scenarios where accurate identification of positive samples is crucial, AUC-PR better reflects the model's strengths.

[0100] The Pearson correlation coefficient and the Spearman correlation coefficient are two commonly used methods for measuring the correlation between two variables. The Pearson correlation coefficient measures the linear relationship between two continuous variables, with values ​​ranging from -1 to 1. A value of 1 indicates a perfect positive correlation; a value of -1 indicates a perfect negative correlation; and a value of 0 indicates no linear correlation. The Spearman correlation coefficient primarily measures the monotonic relationship between two variables, that is, the tendency for one variable to increase (or decrease) as the other increases (or decreases). This coefficient is calculated based on the variable's rank rather than its original value, making it insensitive to outliers. Rank refers to the position of the data after sorting it from smallest to largest.

[0101] The Pearson correlation coefficient is calculated as follows:

[0102] in, and These are the i-th observations of the two variables. and These are the averages of the two variables, and n is the number of observations.

[0103] The Spearman correlation coefficient is calculated as follows:

[0104] in, It is the rank difference of the i-th observation of two variables, where n is the number of observations.

[0105] (3) Optimal selection of hyperparameters In PPO algorithm training, there are numerous adjustable hyperparameters. However, this invention is characterized by a small decision space, sparse rewards, and a high-dimensional state space, which can easily lead to training instability. Therefore, a discount factor... Cutting range The number of training epochs is particularly important. Among these, the discount factor... Controlling the impact of future rewards on current decisions affects the trade-off between long-term planning and short-term returns; limiting scope. Controlling the magnitude of policy changes during each policy update is a key factor in ensuring the stability of PPO training and avoiding over-updates; the number of training epochs determines the optimization effect and convergence of the model.

[0106] To find the optimal hyperparameter settings, we will... , , Hyperparameter tuning experiments were conducted within the range of values. To clearly present the results, this embodiment only shows the AUC-PR line graphs of sentence-level illusion classes on the training and test sets. Figure 3 The graph illustrates the impact of different discount factors on the AUC-PR of the training and test sets. As shown, the test set AUC-PR initially increases and then decreases with increasing discount factor. When the discount factor is 0.95, the test set AUC-PR reaches its highest value of 88.26%, then gradually decreases, indicating that a smaller discount factor may be more helpful in improving the model's generalization performance on the test set. The training set AUC-PR gradually decreases with increasing discount factor. This may be because this embodiment is more suitable for smaller discount factors that focus on short-term gains; a larger discount factor results in lower training efficiency, leading to a decrease in the training set AUC-PR. Therefore, the discount factor is set to 0.95.

[0107] Figure 4 The figure illustrates the impact of different pruning ranges on the AUC-PR of the training and test sets. As shown, the AUC-PR of both the training and test sets initially increases and then decreases with increasing pruning range. When the pruning range approaches 0.2, the AUC-PR of the test set reaches its highest value of 88.26%, and then begins to decline. This indicates that moderate pruning can effectively improve the model's generalization performance, while excessive pruning may lead to information loss, thereby reducing model performance. Therefore, the pruning range is set to 0.2 during model training.

[0108] Figure 5 The graph illustrates the impact of different epochs on the AUC-PR of the training and test sets. As shown, the AUC-PR of the training set gradually increases with the number of training epochs, stabilizing after 10 epochs and reaching 93.23% at 20 epochs. The AUC-PR of the test set follows the same trend as the training set before 10 epochs, then decreases slightly, reaching 88.26% at 10 epochs. This indicates that appropriately increasing the number of training epochs allows the model to better learn the features of the dataset, improving the metrics of both the training and test sets; however, too many training epochs can lead to overfitting. Therefore, the number of training epochs is set to 10.

[0109] (4) Selection analysis of consistency scoring module To verify the effectiveness of selecting the UDeMoE model as the consistency scoring module, this embodiment uses other consistency evaluation methods and the UDeMoE model as consistency scoring modules in the construction of HADAS, and conducts training and testing on the hallucination detection dataset WBGH. The results are shown in Table 2.

[0110] From the sentence-level AUC-PR metrics, the UDeMoE model achieved 72.22% accuracy in the non-hallucination category. Compared to other methods, the AUC-PR improved by 1.73%, 5.09%, 23.08%, and 27.09%, respectively, indicating that UDeMoE can more effectively identify non-hallucination information. In the hallucination category, UDeMoE reached 88.26%, although the AUC-PR decreased by 1.22% compared to the TRUE method, it still improved by 8.00%, 3.85%, and 7.20% compared to other methods. It is worth noting that the differences in detection performance among multiple consistency evaluation methods in the hallucination category were not significant compared to the non-hallucination category. This may be because the amount of hallucination data in the dataset is relatively small, making it difficult for the model to fully learn its feature distribution during training.

[0111] At the paragraph level, the effectiveness of the consistency assessment methods was measured using Pearson and Spearman correlation coefficients. UDeMoE significantly improved the Pearson correlation coefficient from 58.63% to 77.29% and the Spearman correlation coefficient from 56.60% to 74.47%, demonstrating the best performance among all methods.

[0112] Table 2. Hallucination detection results (%) of existing consistency assessment methods and the UDeMoE model

[0113] Note: In the table, "Pearson C" and "Spilman C" represent the evaluation indicators, respectively. "Pearson correlation coefficient" and "Spearman correlation coefficient".

[0114] Overall, UDeMoE performs exceptionally well at both the sentence and paragraph levels. At the sentence level, it demonstrates significant advantages in non-hallucination category identification and also performs well in hallucination category detection. At the paragraph level, both Pearson and Spearman correlation coefficients are significantly improved and are the best among all methods. This fully demonstrates the superiority and reliability of UDeMoE in the consistency evaluation module, providing more accurate support for the model's decision sampling.

[0115] (5) Effectiveness analysis of the sampling decision module The effectiveness analysis of the sampling decision module is divided into two parts: whether there is a difference in the effect of the module, and the difference in the effect of the module using different algorithms.

[0116] 1) Differences in performance with and without a sampling decision module This experiment performed 3, 5, 10, and 20 direct samplings on each sample, and then tested the hallucination detection effect with and without the sampling decision module.

[0117] Depend on Figure 6 , Figure 7 It is known that increasing the number of samples generally leads to better experimental results, whether at the sentence or paragraph level. However, when the number of samples exceeds 10, the improvement becomes very slow. The sampling decision module proposed in this invention achieves an average of 4.38 samples in sentence-level hallucination detection, with AUC-PR scores of 88.26% for hallucination classes and 72.22% for non-hallucination classes, outperforming the effect of direct sampling of 5 times and approaching the level of direct sampling of 10 and 20 times. In paragraph-level detection, the module achieves an average of 4.59 samples across all samples, with Pearson correlation coefficients and Spearman correlation coefficients of 77.29% and 74.47%, respectively. The Pearson correlation coefficient is significantly better than the effect of direct sampling of 5 times, very close to the level of direct sampling of 20 times; the Spearman correlation coefficient is also significantly better than the effect of direct sampling of 5 times, approaching the level of direct sampling of 10 times.

[0118] This demonstrates that the sampling decision module maintains an efficient and stable sampling process at both the sentence and paragraph levels, resulting in good and consistent performance for HADAS. Under various conditions, the average number of samples is less than 5, yet the performance approaches that of direct sampling of 20. Using this module allows the HADAS method to maintain good hallucination detection results while significantly reducing resource consumption, fully demonstrating the effectiveness of the sampling decision module in HADAS.

[0119] 2) Differences in the effectiveness of different algorithms used in the sampling decision module This experiment compares the sampling decision model based on confidence interval judgment, the sampling decision model based on Bayesian optimization, and the sampling decision model based on dynamic programming with the PPO-based sampling decision model proposed in this invention. To ensure the robustness of the experiment, all experiments were repeated 10 times and the average value was taken.

[0120] Depend on Figure 8 , Figure 9 It can be seen that in both sentence-level and paragraph-level experiments, the PPO-based model had the fewest sampling times, at 4.38 for the sentence level and 4.59 for the paragraph level. Furthermore, this model achieved an AUC-PR of 85.56% in sentence-level hallucination detection, outperforming other methods; and at the paragraph level, its Pearson correlation coefficient and Spearman correlation coefficient reached 72.29% and 71.47% respectively, also outperforming other methods.

[0121] The confidence interval-based method performed exceptionally well at the sentence level, achieving an AUC-PR of 72.55% for non-hallucination detection, 0.33% higher than the PPO-based method. However, it performed poorly at the paragraph level, with average sampling times of 7.28 and 6.92 at the sentence and paragraph levels, respectively, significantly higher than other methods. This contradicts the study's initial goal of balancing resource consumption with experimental results. The dynamic programming-based method performed second best among the four methods overall, with average sampling times of 5.67 and 5.82. In sentence-level non-hallucination detection, its AUC-PR was 0.03% higher than the PPO-based method, and its Spearman correlation coefficient at the paragraph level was 71.46%, only 0.01% lower than the PPO-based method. While the Bayesian optimization-based method had an advantage in sampling times, its evaluation metrics were subpar.

[0122] In summary, the PPO-based sampling decision model achieves a good balance between reliability and efficiency, and has significant advantages in illusion detection tasks based on generative consistency.

[0123] (6) Performance comparison with existing models To verify the effectiveness of the HADAS method, this embodiment compares HADAS with several current state-of-the-art hallucination detection methods on the WBGH independent test set. Specifically, SelfCheckGPT and HaloCheck detect hallucinations through the consistency of multiple samplings; SelfContradictory detects hallucinations by leveraging the contradictions generated multiple times (using GPT4, the best-performing model in the original paper, as the contradiction detection model); HDBSE calls an external knowledge base and uses a Bayesian optimization method to determine the number of calls; EUBHD uses a surrogate model (using LLAMA30B, the best-performing model in the original paper, for experiments) to obtain the probability of text generation and then performs hallucination detection.

[0124] Analysis of the experimental results in Table 3 shows that in the sentence-level hallucination detection task, the HADAS method performs exceptionally well, achieving an AUC-PR value of 88.26%, second only to EUBHD-LLAMA30B (89.79%), and exceeding the other methods in the table by 0.93%, 1.81%, 0.48%, and 1.79%, respectively. In non-hallucination detection, HADAS's AUC-PR value reaches 72.22%, surpassing all compared methods, and exceeding the other methods in the table by 10.39%, 10.26%, 2.54%, 6.53%, and 8.68%, respectively. These results demonstrate that HADAS has strong discriminative ability in overall hallucination detection, especially excelling in non-hallucination sample detection, highlighting its ability to improve model robustness. HADAS also performs excellently in the paragraph-level detection task. On the Pearson correlation coefficient, HADAS scored 77.29%, second only to SelfContradictory-GPT4 (81.18%), and higher than the other methods in the table by 8.24%, 3.62%, 0.14%, and 4.01%, respectively. On the Spearman correlation coefficient, HADAS scored 74.47%, higher than the other methods in the table by 6.70%, 0.27%, 2.95%, 1.23%, and 3.91%, respectively. These results further demonstrate that HADAS can effectively capture semantic associations and contextual consistency when dealing with more complex paragraph-level illusion detection tasks, reflecting its comprehensive cross-level detection capabilities.

[0125] Table 3. Hallucination detection results (%) of HADAS method and existing methods on the WBGH test set.

[0126] Note: In the table, "Pearson C" and "Spilman C" represent the evaluation indicators, respectively. "Pearson correlation coefficient" and "Spearman correlation coefficient".

[0127] (7) Resource consumption analysis The HADAS method proposed in this invention, while ensuring the effectiveness of hallucination detection, also needs to balance the resource consumption caused by multiple sampling and evaluation. This section statistically compares and contrasts the time consumption, computing power, and API call costs of various methods in application scenarios. In real-world applications, the text to be detected usually does not have multiple samples; therefore, this section only uses the text to be detected from the WBGH test set. The sampled text needs to be obtained in real time when applying each method. The text to be detected in the WBGH dataset is generated by GPT3, so the real-time sampled text is also generated using GPT3, using the original dataset's query paradigm: "This is a Wikipedia passage about {concept}." Table 4 shows the number of samplings and time taken for each hallucination detection method to infer the WBGH test set.

[0128] Table 4. Number of samplings and time taken for each hallucination detection method to infer the WBGH test set.

[0129] As shown in Table 4, the experimental results demonstrate that the HADAS method proposed in this invention achieves a good balance between sampling frequency and inference time, taking into account both detection performance and the practicality requirements of resource consumption. In sentence-level detection, the HADAS method has an average sampling frequency of 4.38 times and an inference time of 3.06 seconds, which is significantly reduced compared to SelfCheckGPT (20 samples, 12.59 seconds) and HDBSE (6.22 samples, 7.63 seconds). Although the inference time of HADAS is slightly longer than that of EUBHD-LLAMA30B (1 sample, 1.25 seconds), which currently has the shortest inference time, it still maintains high detection efficiency.

[0130] Meanwhile, the computational resources required for inference are also a significant cost. Table 5 shows the computational resources required for the application of HADAS and existing methods.

[0131] Table 5 Resource Requirements of HADAS Method and Existing Methods in the Inference Process

[0132] As shown in Table 5, the HADAS method exhibits a good balance in terms of computational resource requirements during inference. Regarding memory requirements, the HADAS method only requires 2GB, the same as HDBSE and HaloCheck, and significantly lower than the 64GB required by the EUBHD-LLAMA30B. This means that HADAS can run on devices with limited computing power without the need for high-end hardware support. In terms of API requirements, the HADAS method is similar to HDBSE and HaloCheck, requiring the API of the model under test for inference. However, compared to SelfContradictory-GPT4, which relies on the GPT-4 API, the HADAS method avoids dependence on high-cost external APIs.

[0133] Overall, the HADAS method achieves a balance between model performance and efficiency thanks to its efficient consistency scoring and sampling decision modules. Compared to EUBHD-LLAMA30B, which requires high-end hardware support, and SelfContradictory-GPT4, which relies on GPT-4, the HADAS method is more resource-friendly, more economically feasible, and can be more widely applied in computationally limited environments.

[0134] The "hallucination phenomenon" of large language models (LLMs) severely hinders their application in high-risk domains. To address the issues of high resource consumption and insufficient consistency evaluation in LLM hallucination detection, this invention proposes HADAS, a large language model hallucination detection method based on an adaptive sampling strategy. First, the WBGH dataset is cleaned, and the DeBERTa model's tokenizer is used for text sequence segmentation and encoding. A unified modeling and hybrid expert network approach is combined to evaluate text pair consistency scores. Then, Proximal Policy Optimization (PPO) is introduced to train the sampling decision module, dynamically balancing detection accuracy and sampling cost to achieve an adaptive sampling strategy. Finally, a threshold is set to convert the consistency score into a hallucination category. Experimental results show that HADAS achieves an AUC-PR of 88.26% in sentence-level hallucination detection, and a Pearson correlation coefficient of 77.29% and a Spearman correlation coefficient of 74.47% at the paragraph level. It also shows significant advantages in non-hallucination detection, with an average sampling count of less than 5, approaching the performance of direct sampling of 20 times. Compared with existing methods such as SelfCheckGPT and EUBHD, HADAS exhibits superior performance on independent test sets, with lower requirements for sampling times, time, and inference resources. It effectively improves the efficiency and accuracy of hallucination detection, providing a practical solution for the reliable application of LLMs in high-risk fields.

[0135] Example 2 This embodiment discloses a large language model hallucination detection system based on an adaptive sampling strategy, including: The text acquisition module is configured to acquire the text to be detected and its corresponding multiple sampled texts. The text encoding module is configured to: segment and encode the text to be detected and each sampled text to obtain their respective token identifiers; concatenate the token identifier of the text to be detected with the token identifier of each sampled text to obtain multiple text pair sequences; and after encoding processing, obtain the encoding results of multiple text pairs. The consistency evaluation module is configured to input the encoding results of the text pairs into the consistency evaluation model for processing, and obtain consistency scores for multiple text pairs. An adaptive sampling decision module is configured to: acquire the embedding vector of the text to be detected, calculate the statistical information of the consistency scores of multiple text pairs, input the embedding vector and statistical information into the sampling decision model for decision-making, and the sampling decision model decides whether to perform sampling again. If so, sampling is performed again; otherwise, the hallucination detection result is obtained based on the consistency score.

[0136] Example 3 The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method of Embodiment 1.

[0137] Example 4 The purpose of this embodiment is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method of Embodiment 1.

[0138] The steps and methods involved in the apparatuses of Embodiments 3 and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0139] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0140] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0141] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for large language model hallucination detection based on adaptive sampling strategy, characterized in that, The method comprises the following steps: obtaining a to-be-detected text and a plurality of sampling texts corresponding to the to-be-detected text; performing tokenization and encoding on the to-be-detected text and each sampling text to obtain token identifiers of the to-be-detected text and each sampling text; and splicing the token identifiers of the to-be-detected text with the token identifiers of each sampling text to obtain a plurality of text pair sequences, and performing encoding processing to obtain a plurality of encoding results of the text pairs; inputting the encoding results of the text pairs into a consistency evaluation model to obtain consistency scores of the plurality of text pairs; obtaining an embedding vector of the to-be-detected text, calculating statistical information of the consistency scores of the plurality of text pairs, inputting the embedding vector and the statistical information into a sampling decision model to make a decision, and determining whether to perform re-sampling according to the decision; if yes, re-sampling is performed; and if no, a hallucination detection result is obtained based on the consistency scores.

2. The method for large language model hallucination detection based on adaptive sampling strategy according to claim 1, wherein, The token identifiers are obtained in the following manner: performing tokenization on a single text to obtain subword units; converting the subword units based on a tokenizer to obtain corresponding token identifiers.

3. The method for large language model hallucination detection based on adaptive sampling strategy according to claim 2, wherein, The encoding results of the plurality of text pairs are obtained in the following manner: performing padding or truncation on the token identifiers of the to-be-detected text and each sampling text to obtain token identifiers with uniform lengths; splicing the token identifiers of the to-be-detected text with the token identifiers of a certain sampling text to obtain a plurality of text pair sequences; performing encoding on the plurality of text pair sequences based on a pre-trained language model to obtain a plurality of encoding results of the text pairs.

4. The method for large language model hallucination detection based on adaptive sampling strategy of claim 1, wherein, The consistency evaluation model comprises a base model layer and a mixed expert layer, the base model layer takes DeBERTa-V3 as a backbone, and the mixed expert layer is spliced with a mixed expert network after the last hidden layer of DeBERTa-V3.

5. The method for large language model hallucination detection based on adaptive sampling strategy according to claim 1, wherein, The reward function of the sampling decision model takes the change trend of the consistency scores as the core, and combines the standard deviation to measure the consistency convergence of the sampling texts.

6. The method for large language model hallucination detection based on adaptive sampling strategy according to claim 5, wherein, The total loss of the sampling decision model is a weighted sum of a policy loss and a value loss, and the policy loss is a generalized advantage estimation loss.

7. The method for large language model hallucination detection based on adaptive sampling strategy according to claim 1, wherein, The hallucination detection result based on the consistency scores is obtained in the following manner: calculating an average value of the consistency scores of the plurality of text pairs to obtain an average consistency score; comparing the average consistency score with a set hallucination threshold, and outputting non-hallucination if the average consistency score is greater than the set hallucination threshold, and outputting hallucination if the average consistency score is less than the set hallucination threshold.

8. A large language model hallucination detection system based on adaptive sampling strategy, characterized in that, The method comprises the following steps: a text acquisition module configured to obtain a to-be-detected text and a plurality of sampling texts corresponding to the to-be-detected text; a text encoding module configured to perform tokenization and encoding on the to-be-detected text and each sampling text to obtain token identifiers of the to-be-detected text and each sampling text; and splicing the token identifiers of the to-be-detected text with the token identifiers of each sampling text to obtain a plurality of text pair sequences, and performing encoding processing to obtain a plurality of encoding results of the text pairs; a consistency evaluation module configured to input the encoding results of the text pairs into a consistency evaluation model to obtain consistency scores of the plurality of text pairs; An adaptive sampling decision module configured to: obtain an embedding vector of a text to be detected, and calculate statistical information of consistency scores of a plurality of text pairs, input the embedding vector and the statistical information to a sampling decision model for decision, the sampling decision model deciding whether to sample again, if yes, sampling again, if no, obtaining an illusion detection result based on the consistency score.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps in the large language model illusion detection method based on the adaptive sampling strategy in any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps in the large language model illusion detection method based on the adaptive sampling strategy in any one of claims 1-7.

Images