Big language model illusion detection method and system, terminal and medium

Abstract

The invention belongs to the technical field of large language models, and particularly discloses a large language model illusion detection method and system, a terminal and a medium. Comprising the steps of obtaining a middle hidden state of each token in a process of generating an answer by a large language model, and organizing and forming a hidden state sequence according to a generation sequence; performing time sequence preprocessing on the hidden state sequence to complete the preprocessing of the sequence; inputting the pre-processed hidden state sequence and a dynamic feature formed by a difference vector, a trend drift degree or local fluctuation intensity into a dynamic sequence modeling network, and enabling the network to extract a time dependence feature reflecting an internal state change mode based on an evolution relation of a hidden state along with a generation process; and generating an illusion risk score according to the comprehensive time sequence characteristics output by the dynamic sequence modeling network, and determining that the answer contains the illusion content when the score exceeds a preset threshold. The reliability of the output content of the large language model can be improved.

Description

Technical Field

[0001] This invention belongs to the field of large language model technology, specifically relating to a method, system, terminal, and medium for detecting illusions using large language models. Background Technology

[0002] With the widespread deployment of large language models in scenarios such as question answering, content generation, and decision support, the credibility of their generated results has attracted increasing attention. Due to the large number of parameters and the complexity of their internal modeling mechanisms, large language models are prone to generating content that does not conform to the facts when dealing with knowledge-intensive problems, i.e., producing so-called "illusions".

[0003] Existing hallucination detection methods can be broadly divided into two categories: One type is the comparison and verification method based on external knowledge bases, which judges the consistency of the model's generated results by searching the knowledge base or external literature. However, this type of method relies on external knowledge sources, has high system deployment costs, and suffers from large latency and insufficient coverage in real-time interactive scenarios. Another type is the post-processing method based on the model's output content itself, such as analyzing the logical structure, consistency, or confidence of the generated text. While this type of method does not rely on external knowledge bases, it often only identifies explicit text anomalies and lacks sensitivity to potential errors caused by the internal generation mechanisms of large language models, making it difficult to accurately address the diverse manifestations of illusions.

[0004] In recent years, some research approaches have emerged that attempt to utilize the intermediate activation features of large language models for hallucination recognition. For example, some schemes perform static classification only on the hidden state corresponding to the final token to determine whether the generated content is a hallucination.

[0005] However, hallucinations in large language models often stem from the gradual shifts or anomalous evolutions of internal states during the generation process. Static analysis of the hidden state at a single moment is insufficient to fully reflect the dynamic changes within the model. Existing detection methods are mostly based on static features or the final generated result, resulting in low accuracy in hallucination identification. Summary of the Invention

[0006] This invention addresses the problems in the prior art by providing a method, system, terminal, and medium for detecting hallucinations in large language models. The solution addresses the issues in the prior art where hallucinations in large language models often originate from the gradual shift or abnormal evolution of internal states during the generation process. Static analysis of the hidden state at a single moment is insufficient to fully reflect the dynamic changes within the model, and hallucination recognition accuracy is low based on static features or the final generated result.

[0007] The technical solution adopted in this invention is as follows: Firstly, this application provides a method for detecting hallucinations using a large language model, the method comprising the following steps: During the process of generating answers using a large language model, the intermediate hidden states corresponding to each token are obtained according to the generation order of the tokens, and the hidden states are organized into a hidden state sequence in chronological order. Perform time-series preprocessing on the hidden state sequence; The preprocessed hidden state sequence is input into the dynamic sequence modeling network. In the dynamic sequence modeling network, temporal features are extracted based on the evolution relationship of the hidden state with the generation process to characterize the internal state changes of the model. Based on temporal features, the responses generated by the large language model are subjected to hallucination determination. The hallucination determination is based on the overall dynamic change pattern of the hidden state sequence to determine whether the response contains hallucination content.

[0008] Further time series preprocessing includes: The hidden state sequence is divided into sliding windows according to a preset window length, and the hidden state sequence in each window is truncated, padded and normalized in the length dimension. The hidden state sequence is dynamically stabilized based on the rate of change between adjacent hidden states within the window. The preprocessed hidden state sequence is obtained.

[0009] Furthermore, before inputting the preprocessed hidden state sequence into the dynamic sequence modeling network, the following steps are also included: Calculate the rate of change sequence of the hidden state sequence based on the difference vector of the hidden state at adjacent time steps; The trend drift or local fluctuation intensity of the hidden state sequence is calculated based on a preset local window. The rate of change sequence and trend drift or local fluctuation intensity are used as additional dynamic features and input together with the preprocessed hidden state sequence into the dynamic sequence modeling network.

[0010] Furthermore, the dynamic sequence modeling network includes a temporal neural network structure for characterizing the evolution of the hidden state sequence during the generation process; The temporal neural network structure is at least one of a multilayer long short-term memory network, a gated recurrent unit network, or a temporal Transformer network based on an attention mechanism; Temporal neural network structures model the rate of change, trend drift, or local fluctuation features in the hidden state sequence through recursive propagation across time steps or dynamic allocation of attention weights over time, and extract time-dependent features that can reflect the change patterns of the internal state of the large language model before and after the hallucination.

[0011] Furthermore, the output of the dynamic sequence modeling network includes a first temporal dependency feature reflecting the long-term dependencies of the hidden state sequences and a second temporal dependency feature reflecting the local change patterns. The first time-dependent feature is obtained from the final hidden state of the last layer of the dynamic sequence modeling network or the sequence compression result of the hidden states at each time step; The second time-dependent feature is obtained based on the difference between the hidden states of adjacent time steps, the convolution operation within the local window, or the local attention distribution. The first time-dependent feature is fused with the second time-dependent feature to form a comprehensive temporal feature for hallucination determination.

[0012] Furthermore, hallucination determination based on temporal characteristics includes: By inputting the comprehensive temporal features into a pre-trained dynamic sequence modeling network, a hallucination risk score is obtained to characterize whether the answer is hallucinatory. The hallucination risk score is compared with a preset risk threshold. When the hallucination risk score exceeds the risk threshold, it is determined that the answer generated by the large language model contains hallucination content.

[0013] Furthermore, the pre-trained dynamic sequence modeling network learns the correspondence between the temporal evolution pattern of the hidden state during the generation process and the occurrence of hallucination by supervising the training of hidden state sequence samples labeled with hallucination. The hidden state sequence samples contain labels obtained by manual annotation, indicating either "hallucination exists" or "hallucination does not exist"; Dynamic sequence modeling networks model the dynamic changes in hidden state sequences during training using recurrent neural networks or attention-based temporal network structures.

[0014] Secondly, this application provides a large language model illusion detection system for implementing the large language model illusion detection method as described in the first aspect. The system includes: The hidden state acquisition unit is configured to acquire the intermediate hidden states corresponding to each token in the order of token generation during the process of generating answers in the large language model, and organize the hidden states in chronological order to form a hidden state sequence. The temporal preprocessing unit is configured to perform temporal preprocessing on the hidden state sequence, including dividing the hidden state sequence into sliding windows according to a preset window length, performing truncation, padding and normalization processing on the hidden state sequence in each window in the length dimension, and performing dynamic stabilization processing on the hidden state sequence based on the rate of change between adjacent hidden states in the window, so as to obtain the preprocessed hidden state sequence. The dynamic feature construction unit is configured to calculate the rate of change sequence based on the difference vector of the hidden state of adjacent time steps, calculate the trend drift degree or local fluctuation intensity based on a preset local window, and output the rate of change sequence and the trend drift degree or local fluctuation intensity as additional dynamic features together with the preprocessed hidden state sequence. The dynamic sequence modeling unit is configured to use a multi-layer long short-term memory network, a gated recurrent unit network, or an attention-based temporal Transformer network to model the rate of change, trend drift, or local fluctuation features of the hidden state sequence, and extract time-dependent features that can reflect the change pattern of the internal state of the large language model before and after the hallucination. The comprehensive feature fusion unit is configured to obtain a first time dependency feature reflecting the long-term dependency relationship of the hidden state sequence and a second time dependency feature reflecting the local change pattern from the output of the dynamic sequence modeling unit, and fuse the first time dependency feature and the second time dependency feature to form a comprehensive temporal feature for hallucination determination. The hallucination determination unit is configured to input comprehensive temporal features into a pre-trained dynamic sequence modeling network to obtain a hallucination risk score, compare the hallucination risk score with a preset risk threshold, and determine that the answer contains hallucination content when the hallucination risk score exceeds the threshold.

[0015] Thirdly, this application provides a terminal, including: Memory, used to store the large language model illusion detection program; A processor is configured to implement the steps of the large language model illusion detection method as described in the first aspect when executing the large language model illusion detection device.

[0016] Fourthly, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the large language model illusion detection method as described in the first aspect.

[0017] As can be seen from the above technical solutions, the advantages of the present invention are: By acquiring the hidden states of each token in real time during the generation of a large language model and forming a sequence of hidden states, illusion detection is no longer limited to the final output text itself, but is directly analyzed based on the dynamic representation process inside the model.

[0018] Its advantage lies in its ability to reflect the continuous evolution of the model's internal state during the generation process, thereby enabling timely identification of abnormal changes before hallucinations manifest in the text content, thus improving the accuracy and real-time performance of hallucination detection.

[0019] The hidden state sequence is structured in the time dimension through windowing, truncation, padding and normalization, and noise interference is suppressed and the main change trend of the sequence is highlighted through dynamic stabilization steps based on the rate of change.

[0020] This preprocessing method improves the stability and learnability of the hidden state sequence when it is input into the dynamic model, enabling the subsequent temporal network to more accurately capture the real change pattern of the hidden state sequence and reduce the probability of misjudgment. Attached Figure Description

[0021] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the steps of the large language model hallucination detection method in the embodiment; Figure 2 This is a flowchart illustrating the training process of the dynamic sequence modeling network in this embodiment. Figure 3 This is a structural diagram of the large language model illusion detection system in the embodiment. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Please see Figure 1 As shown, this application provides a method for detecting hallucinations using a large language model, including: Step S1: During the process of generating answers using the large language model, obtain the intermediate hidden states corresponding to each token according to the generation order of the tokens, and organize the hidden states into a hidden state sequence in chronological order. In one implementation, the system can intercept intermediate results during the inference phase of a large language model and extract the hidden state corresponding to each token from a designated intermediate layer of the model when generating the next token. This hidden state is typically composed of a multi-dimensional vector, representing the internal semantic representation of the token within the current generation context. In actual operation, the system continuously monitors the model's generation process and extracts the hidden state at each time step. Finally, all hidden states are concatenated in the generation order to obtain a sequence reflecting the trajectory of internal state changes within the model. In one specific embodiment, a specific intermediate layer of the model can be selected as the acquisition layer, and the dimension of each hidden state is consistent with the model structure. A unified data caching module records the hidden states of each token sequentially, allowing subsequent steps to directly process the complete time-series data.

[0025] Step S2: Perform time-series preprocessing on the hidden state sequence; In one specific implementation, temporal preprocessing may include segmenting and organizing the hidden state sequence to ensure that the data input to the dynamic model has a consistent length structure. To avoid data misalignment caused by different response lengths, a fixed-length window approach can be used, dividing the original sequence into several windows according to time order and padding windows that are not long enough. In another implementation, outliers in the sequence can be smoothed to reduce noise interference during model calculation and make the main trends of hidden state changes more prominent. In one specific embodiment, the system judges the relationship between changes between adjacent hidden states. When excessively large jumps occur, transition points are generated through interpolation to maintain the overall smoothness of the sequence, enabling the dynamic sequence modeling network to better identify continuous evolution patterns.

[0026] Step S3: Input the preprocessed hidden state sequence into the dynamic sequence modeling network, and extract temporal features to characterize the internal state changes of the model based on the evolution relationship of the hidden state with the generation process in the dynamic sequence modeling network. In one specific implementation, the dynamic sequence modeling network can be a deep network model containing multiple temporal structure layers, used to extract long-term dependency information and local variation features from the input sequence. During operation, the network sequentially receives each preprocessed temporal segment, gradually establishing the correlation between each time step through internal state recursion or attention computation mechanisms. In one specific embodiment, the network automatically learns the changing patterns of the model's internal representations during processing, such as identifying trends where hidden states gradually deviate from their original semantic trajectories, or capturing unstable state changes occurring in specific contexts, thereby forming a feature vector structure capable of representing hallucination tendencies. This feature is then used for subsequent hallucination risk assessment.

[0027] Step S4: Based on temporal features, perform hallucination determination on the answers generated by the large language model. The hallucination determination is based on the overall dynamic change pattern of the hidden state sequence to determine whether the answer contains hallucination content.

[0028] In one implementation, the system inputs the feature vectors generated by the dynamic sequence modeling network into the classification or scoring module, and determines whether the feature distribution pattern in the high-dimensional space belongs to the hallucination-related internal state pattern. In practical applications, when the hidden state sequence exhibits a change behavior similar to the dynamic trajectory corresponding to the "hallucination label" in the training samples, the system outputs a higher hallucination risk score. In another implementation, the system compares the risk score with a preset threshold. When the score exceeds the threshold, a prompt is triggered in the user interface, making the user aware of potential information bias before continuing to read or use the model-generated content, thereby reducing the risk caused by model hallucinations.

[0029] In some embodiments, timing preprocessing includes: The hidden state sequence is divided into sliding windows according to a preset window length, and the hidden state sequence in each window is truncated, padded and normalized in the length dimension. The hidden state sequence is dynamically stabilized based on the rate of change between adjacent hidden states within the window. The preprocessed hidden state sequence is obtained.

[0030] In one specific embodiment, the preprocessing module first divides the hidden state sequence into a sliding window of fixed length, dividing the sequence into several continuous segments to accommodate generated responses of different lengths. Within the window, if the sequence length is less than the preset length, padding is achieved by adding placeholder vectors to the end of the sequence; if the sequence is too long, the portion closest to the current generation position is truncated to maintain sequence consistency. Normalization processing adjusts the numerical range of each hidden state within the window to make the feature scale more stable. Based on this, the system further calculates the rate of change between adjacent states within the window and applies smoothing processing to regions of rapid change, making the main trends of hidden state changes clearer and thus improving the reliability of subsequent modeling.

[0031] In some embodiments, before inputting the preprocessed hidden state sequence into the dynamic sequence modeling network, the method further includes: Calculate the rate of change sequence of the hidden state sequence based on the difference vector of the hidden state at adjacent time steps; The trend drift or local fluctuation intensity of the hidden state sequence is calculated based on a preset local window. The rate of change sequence and trend drift or local fluctuation intensity are used as additional dynamic features and input together with the preprocessed hidden state sequence into the dynamic sequence modeling network.

[0032] In one specific embodiment, the system can calculate the vector differences between hidden states in consecutive time steps to reflect the degree of state change over time. Furthermore, the system can analyze the average direction of change of hidden states within a small local window to obtain the drift degree reflecting semantic shift trends, or it can obtain the fluctuation intensity based on the discreteness of states within the window. These change rate sequences, trend drift degrees, or fluctuation intensities are input as additional features along with the original preprocessed sequence, enabling the network to understand the internal state change patterns from richer dimensions. For example, when the model hesitates or deviates from the task objective during inference, these dynamic features can anticipate this trend, thereby improving the sensitivity of hallucination detection.

[0033] In some embodiments, the dynamic sequence modeling network includes a temporal neural network structure for characterizing the evolution of hidden state sequences as they are generated; The temporal neural network structure is at least one of a multilayer long short-term memory network, a gated recurrent unit network, or a temporal Transformer network based on an attention mechanism; Temporal neural network structures model the rate of change, trend drift, or local fluctuation features in the hidden state sequence through recursive propagation across time steps or dynamic allocation of attention weights over time, and extract time-dependent features that can reflect the change patterns of the internal state of the large language model before and after the hallucination.

[0034] In one specific embodiment, the dynamic sequence modeling network can employ a two-layer long short-term memory network and record the dependencies between different time steps of the sequence through an internal state update mechanism, such as long-term semantic dependencies or short-term mutation features. In another embodiment, a temporal Transformer network based on an attention mechanism can be used. By assigning different attention weights to different time positions in the hidden state sequence, the network can focus on time segments that are more closely related to hallucination formation. During model inference, the network can extract discriminative temporal patterns from dynamic features such as rate of change and trend drift, for example, identifying the process of slow shift of the hidden state or local unstable regions, which can serve as an important basis for hallucination detection.

[0035] In some embodiments, the output of the dynamic sequence modeling network includes a first time dependency feature reflecting the long-term dependency of the hidden state sequence and a second time dependency feature reflecting the local change pattern. The first time-dependent feature is obtained from the final hidden state of the last layer of the dynamic sequence modeling network or the sequence compression result of the hidden states at each time step; The second time-dependent feature is obtained based on the difference between the hidden states of adjacent time steps, the convolution operation within the local window, or the local attention distribution. The first time-dependent feature is fused with the second time-dependent feature to form a comprehensive temporal feature for hallucination determination.

[0036] In one specific embodiment, the system can extract the final hidden state from the last layer of the network as a long-term dependency feature to characterize the overall evolution trend of the hidden state throughout the generation process. Simultaneously, the system can perform differential analysis on the hidden states of adjacent time steps to capture local anomalous changes, such as sudden shifts or unstable fluctuations. In another embodiment, a lightweight convolutional structure can be used to scan local windows, forming local change features, enabling the system to focus on specific anomalous signals within a short time range. In practical applications, the comprehensive feature fusion module will merge long-term and short-term features according to task requirements to obtain a more discriminative feature representation, thereby more accurately identifying hallucination-related internal state patterns.

[0037] In some embodiments, hallucination determination based on temporal characteristics includes: By inputting the comprehensive temporal features into a pre-trained dynamic sequence modeling network, a hallucination risk score is obtained to characterize whether the answer is hallucinatory. The hallucination risk score is compared with a preset risk threshold. When the hallucination risk score exceeds the risk threshold, it is determined that the answer generated by the large language model contains hallucination content.

[0038] In one specific embodiment, the system can learn the correspondence between comprehensive temporal features and hallucinations based on labeled samples during the training phase, and input the comprehensive features into the trained model during the inference phase to obtain a risk score. If the score exceeds a set judgment threshold, the system can consider the current answer to have a high probability of hallucination. In use cases, this risk score can be output to the user interface in real time. When the model generates a longer answer, a stage score can be obtained at each stage, allowing the user to promptly notice potential information biases when high-risk signals appear.

[0039] In some embodiments, a pre-trained dynamic sequence modeling network learns the correspondence between the temporal evolution pattern of the hidden state during the generation process and the occurrence of hallucination by supervised training on hidden state sequence samples labeled with hallucination. The hidden state sequence samples contain labels obtained by manual annotation, indicating either "hallucination exists" or "hallucination does not exist"; Dynamic sequence modeling networks model the dynamic changes in hidden state sequences during training using recurrent neural networks or attention-based temporal network structures.

[0040] In one specific embodiment, human annotators can first evaluate the responses generated by the large language model, forming a dataset containing two types of labels, and simultaneously recording the hidden state sequences corresponding to the generation process. During the training phase, the system inputs these sequences into the model, which learns the evolutionary patterns of the hidden states of different labeled samples to grasp the statistical characteristics of the internal state changes before and after the hallucination occurs. In another embodiment, temporal networks of different depths can be used during training, and the model structure can be adjusted according to the size of the training set and the task objective, enabling the model to identify both obvious abnormal change patterns and relatively implicit slow shift phenomena, thereby improving the overall detection performance.

[0041] Please see Figure 2 As shown, in one specific embodiment, the dynamic sequence modeling network of the present invention can be obtained through the following training process to ensure that the model can learn the temporal evolution characteristics of the hidden state before and after the hallucination occurs.

[0042] First, the system collects several question-and-answer samples from a small-scale data source, inputs each question into a large language model to obtain the corresponding generated answer and the hidden state sequence of each token during the generation process. Then, human annotators with professional backgrounds manually annotate each sample based on whether the answer content contains factual errors or logical deviations, forming an initial high-quality dataset containing labels for "illusion of existence" and "illusion of non-existence".

[0043] After manual annotation, the system trains a multilayer perceptron classifier based on the annotated data, enabling it to perform binary classification of samples using the hidden state features at the end of the answer. Once the performance of the multilayer perceptron classifier meets a preset accuracy threshold, the system further collects a larger unannotated question-answer dataset and automatically annotates the subset using the trained multilayer perceptron classifier to improve annotation efficiency and significantly expand the sample size available for training.

[0044] After automated annotation is completed, human annotators sample and verify a portion of the samples to correct any potential classification biases, thereby obtaining a large-scale training sample set of high quality. Subsequently, the system inputs the hidden state sequences of the samples into temporal models such as Long Short-Term Memory networks in chronological order, and learns the correspondence between the dynamic change patterns of the hidden states and the hallucination labels through supervised training. The trained temporal classifier can reliably identify hallucinations based on the evolutionary characteristics of the hidden state sequences and serves as the core model of the hallucination determination module of this invention.

[0045] By introducing additional dynamic features such as rate of change sequence, trend drift degree, and local fluctuation intensity, the input data not only contains the absolute value of the hidden state, but also descriptors reflecting the state evolution behavior.

[0046] These dynamic features can enhance the model's sensitivity to phenomena such as sudden changes, drifts, and abnormal fluctuations in the hidden state, forming a key basis for identifying changes in the internal state before and after the formation of hallucinations, thereby significantly improving the ability to distinguish hallucinations.

[0047] Temporal network structures with time-dependent modeling capabilities, such as LSTM, GRU, or Temporal Transformer, are used to establish cross-time step relationships through recursion or attention mechanisms.

[0048] This structure can deeply characterize the long-term dependencies and short-term dynamic changes of hidden state sequences, and is especially suitable for capturing the gradual shifts or sudden instabilities of the model's internal representation before and after the hallucination occurs, thereby improving the overall performance of hallucination pattern recognition.

[0049] By fusing long-term dependency features output by dynamic sequence modeling networks with short-term features reflecting local changes, a more comprehensive temporal feature set can be formed.

[0050] These multi-scale fusion features can simultaneously reveal global trends and local fluctuations in the generation process, enabling hallucination detection models to have stronger adaptability and robustness when facing different types of hallucinations (such as semantic deviation, erroneous reasoning, and local anomalies).

[0051] By inputting comprehensive temporal features into a pre-trained dynamic sequence modeling network to obtain a hallucination risk score, and judging whether hallucination exists based on a threshold, the judgment mechanism has the advantages of being quantifiable, interpretable, and adjustable.

[0052] This method can achieve stable hallucination detection output based on sequence-level dynamic analysis, is suitable for real-time interactive scenarios, and can effectively reduce false alarms and false negatives, thereby improving the actual usability of the system.

[0053] By training a dynamic sequence modeling network based on hidden state sequence samples labeled with hallucinations, the model can learn the temporal dependency between the changes in hidden states during the generation process and the occurrence of hallucinations.

[0054] This training method enables the detection model to recognize "hallucination-related internal dynamic patterns" rather than relying on text features, fundamentally improving the accuracy, generalization ability, and adaptability to different model structures in hallucination detection.

[0055] Please see Figure 3As shown, in some embodiments, this application provides a large language model illusion detection system for implementing a large language model illusion detection method. The system includes: The hidden state acquisition unit is configured to acquire the intermediate hidden states corresponding to each token in the order of token generation during the process of generating answers in the large language model, and organize the hidden states in chronological order to form a hidden state sequence. The temporal preprocessing unit is configured to perform temporal preprocessing on the hidden state sequence, including dividing the hidden state sequence into sliding windows according to a preset window length, performing truncation, padding and normalization processing on the hidden state sequence in each window in the length dimension, and performing dynamic stabilization processing on the hidden state sequence based on the rate of change between adjacent hidden states in the window, so as to obtain the preprocessed hidden state sequence. The dynamic feature construction unit is configured to calculate the rate of change sequence based on the difference vector of the hidden state of adjacent time steps, calculate the trend drift degree or local fluctuation intensity based on a preset local window, and output the rate of change sequence and the trend drift degree or local fluctuation intensity as additional dynamic features together with the preprocessed hidden state sequence. The dynamic sequence modeling unit is configured to use a multi-layer long short-term memory network, a gated recurrent unit network, or an attention-based temporal Transformer network to model the rate of change, trend drift, or local fluctuation features of the hidden state sequence, and extract time-dependent features that can reflect the change pattern of the internal state of the large language model before and after the hallucination. The comprehensive feature fusion unit is configured to obtain a first time dependency feature reflecting the long-term dependency relationship of the hidden state sequence and a second time dependency feature reflecting the local change pattern from the output of the dynamic sequence modeling unit, and fuse the first time dependency feature and the second time dependency feature to form a comprehensive temporal feature for hallucination determination. The hallucination determination unit is configured to input comprehensive temporal features into a pre-trained dynamic sequence modeling network to obtain a hallucination risk score, compare the hallucination risk score with a preset risk threshold, and determine that the answer contains hallucination content when the hallucination risk score exceeds the threshold.

[0056] In some embodiments, this application provides a terminal, including: Memory, used to store the large language model illusion detection program; A processor is configured to implement the steps of the large language model illusion detection method when executing the large language model illusion detection system.

[0057] In some embodiments, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the large language model illusion detection method.

[0058] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A method for detecting hallucinations using a large language model, characterized in that, Includes the following steps: During the process of generating answers using a large language model, the intermediate hidden states corresponding to each token are obtained according to the generation order of the tokens, and the hidden states are organized into a hidden state sequence in chronological order. Perform time-series preprocessing on the hidden state sequence; The preprocessed hidden state sequence is input into the dynamic sequence modeling network. In the dynamic sequence modeling network, temporal features are extracted based on the evolution relationship of the hidden state with the generation process to characterize the internal state changes of the model. Based on temporal features, the responses generated by the large language model are subjected to hallucination determination. The hallucination determination is based on the overall dynamic change pattern of the hidden state sequence to determine whether the response contains hallucination content.

2. The large language model hallucination detection method according to claim 1, characterized in that, Timing preprocessing includes: The hidden state sequence is divided into sliding windows according to a preset window length, and the hidden state sequence in each window is truncated, padded and normalized in the length dimension. The hidden state sequence is dynamically stabilized based on the rate of change between adjacent hidden states within the window. The preprocessed hidden state sequence is obtained.

3. The large language model hallucination detection method according to claim 1, characterized in that, Before inputting the preprocessed hidden state sequence into the dynamic sequence modeling network, the following steps are also included: Calculate the rate of change sequence of the hidden state sequence based on the difference vector of the hidden state at adjacent time steps; The trend drift or local fluctuation intensity of the hidden state sequence is calculated based on a preset local window. The rate of change sequence and trend drift or local fluctuation intensity are used as additional dynamic features and input together with the preprocessed hidden state sequence into the dynamic sequence modeling network.

4. The method for detecting hallucinations using a large language model according to any one of claims 1-3, characterized in that, Dynamic sequence modeling networks include temporal neural network structures used to characterize the evolution of hidden state sequences as they are generated; The temporal neural network structure is at least one of a multilayer long short-term memory network, a gated recurrent unit network, or a temporal Transformer network based on an attention mechanism; Temporal neural network structures model the rate of change, trend drift, or local fluctuation features in the hidden state sequence through recursive propagation across time steps or dynamic allocation of attention weights over time, and extract time-dependent features that can reflect the change patterns of the internal state of the large language model before and after the hallucination.

5. The large language model hallucination detection method according to claim 4, characterized in that, The output of the dynamic sequence modeling network includes a first temporal dependency feature reflecting the long-term dependencies of the hidden state sequences and a second temporal dependency feature reflecting the local change patterns. The first time-dependent feature is obtained from the final hidden state of the last layer of the dynamic sequence modeling network or the sequence compression result of the hidden states at each time step; The second time-dependent feature is obtained based on the difference between the hidden states of adjacent time steps, the convolution operation within the local window, or the local attention distribution. The first time-dependent feature is fused with the second time-dependent feature to form a comprehensive temporal feature for hallucination determination.

6. The large language model hallucination detection method according to claim 5, characterized in that, Hallucination determination based on temporal characteristics includes: By inputting the comprehensive temporal features into a pre-trained dynamic sequence modeling network, a hallucination risk score is obtained to characterize whether the answer is hallucinatory. The hallucination risk score is compared with a preset risk threshold. When the hallucination risk score exceeds the risk threshold, it is determined that the answer generated by the large language model contains hallucination content.

7. The large language model hallucination detection method according to claim 6, characterized in that, A pre-trained dynamic sequence modeling network learns the correspondence between the temporal evolution pattern of the hidden state during the generation process and the occurrence of hallucination by supervising the training of hidden state sequence samples labeled with hallucination. The hidden state sequence samples contain labels obtained by manual annotation, indicating either "hallucination exists" or "hallucination does not exist"; Dynamic sequence modeling networks model the dynamic changes in hidden state sequences during training using recurrent neural networks or attention-based temporal network structures.

8. A large language model illusion detection system, used to implement the large language model illusion detection method as described in claim 1, characterized in that, The system includes: The hidden state acquisition unit is configured to acquire the intermediate hidden states corresponding to each token in the order of token generation during the process of generating answers in the large language model, and organize the hidden states in chronological order to form a hidden state sequence. The temporal preprocessing unit is configured to perform temporal preprocessing on the hidden state sequence, including dividing the hidden state sequence into sliding windows according to a preset window length, performing truncation, padding and normalization processing on the hidden state sequence in each window in the length dimension, and performing dynamic stabilization processing on the hidden state sequence based on the rate of change between adjacent hidden states in the window, so as to obtain the preprocessed hidden state sequence. The dynamic feature construction unit is configured to calculate the rate of change sequence based on the difference vector of the hidden state of adjacent time steps, calculate the trend drift degree or local fluctuation intensity based on a preset local window, and output the rate of change sequence and the trend drift degree or local fluctuation intensity as additional dynamic features together with the preprocessed hidden state sequence. The dynamic sequence modeling unit is configured to use a multi-layer long short-term memory network, a gated recurrent unit network, or an attention-based temporal Transformer network to model the rate of change, trend drift, or local fluctuation features of the hidden state sequence, and extract time-dependent features that can reflect the change pattern of the internal state of the large language model before and after the hallucination. The comprehensive feature fusion unit is configured to obtain a first time dependency feature reflecting the long-term dependency relationship of the hidden state sequence and a second time dependency feature reflecting the local change pattern from the output of the dynamic sequence modeling unit, and fuse the first time dependency feature and the second time dependency feature to form a comprehensive temporal feature for hallucination determination. The hallucination determination unit is configured to input comprehensive temporal features into a pre-trained dynamic sequence modeling network to obtain a hallucination risk score, compare the hallucination risk score with a preset risk threshold, and determine that the answer contains hallucination content when the hallucination risk score exceeds the threshold.

9. A terminal, characterized in that, include: Memory, used to store the large language model illusion detection program; A processor is configured to implement the steps of the large language model illusion detection method as described in any one of claims 1-7 when executing the large language model illusion detection device.

10. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions. When the computer reads the computer instructions in the storage medium, the computer executes the large language model illusion detection method as described in any one of claims 1-7.

Images