A method, system, and program for detecting and correcting large-scale hallucinations.

Abstract

This invention discloses a method, system, and program for detecting and correcting large-scale hallucinations, relating to the field of artificial intelligence technology. The method includes: extracting a quantitative representation of the internal cognitive state at each time step of the response text generated by the language model, and generating a cognitive trajectory sequence based on the quantitative representation; calculating an internal inconsistency score based on the cognitive trajectory sequence to obtain potential hallucination points; acquiring multi-source external data for the potential hallucination points and the entire generated text, and calculating external data support based on the multi-source external data; fusing the internal inconsistency score and the external data support to obtain a comprehensive hallucination score; determining the final hallucination point based on the comprehensive hallucination score, and correcting the final hallucination point. Through the processing scheme disclosed in this invention, interpretable and highly efficient hallucination recognition and content correction can be achieved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, and program for detecting and correcting large model illusions. Background Technology

[0002] As large language models become more widespread in daily life, their impact on production and daily life is increasing, and the requirements for their accuracy are also rising.

[0003] The illusion problem inherent in large language models has become a core obstacle restricting their reliable deployment in high-risk domains. Existing illusion detection and correction technologies mainly revolve around external knowledge verification and post-processing of output, but they have significant limitations in terms of real-time performance, coverage, interpretability, and correction quality, and cannot meet the requirements of high-reliability applications.

[0004] Current mainstream solutions typically rely on fact-checking with external knowledge bases. These methods compare the output text with structured knowledge bases (such as Wikidata) or document repositories after generation. However, they suffer from several drawbacks: First, poor real-time performance; the complete retrieval and verification process introduces significant latency, making it difficult to meet the immediate feedback requirements of interactive applications. Second, incomplete coverage; the completeness and timeliness of the knowledge base cannot be guaranteed, often resulting in missing evidence for the latest information or specialized domain knowledge, leading to missed detections. Furthermore, these methods are essentially black-box operations, only able to determine the correctness of the output, completely failing to explain the internal cognitive roots of the model's illusions, thus limiting the possibility of subsequent targeted optimization.

[0005] Another type of approach focuses on analyzing the consistency of the model's output, such as checking the stability of generated content through multiple sampling or hinting at variations. While these methods are more efficient, they can only capture explicit uncertainties in the model and are weakly capable of detecting systematic illusions (i.e., illusions of confidence) where the model is highly confident but actually erroneous. They also lack insight into the generation process and cannot distinguish between contradictions caused by a lack of knowledge or reasoning errors.

[0006] In terms of correction, existing technologies mostly employ simple strategies, such as directly deleting suspected hallucination content, using generic expressions for ambiguity, or calling a model to rewrite the entire text. These methods severely damage the original text's information content, stylistic coherence, and logical structure, resulting in a poor user experience.

[0007] While the aforementioned hallucination detection methods can perform hallucination detection on large language models to a certain extent, several shortcomings have been found in practical applications, preventing them from achieving optimal results. These shortcomings can be summarized as follows: 1) Lagging and one-sided detection: Over-reliance on external knowledge makes it impossible to achieve efficient and real-time process monitoring, and the source of evidence is singular; 2) Diagnostic black box: Only examining the final output fails to delve into the cognitive mechanisms underlying hallucinations, resulting in unexplainable and unclear test results; 3) Crude correction: lacks analysis of the causes and context of hallucinations, and the correction methods are simplistic and highly destructive.

[0008] Therefore, it is evident that the existing hallucination detection methods described above still have inconveniences and shortcomings in use, and urgently need further improvement. How to create a new method for large-scale hallucination detection and correction has become a pressing goal for the industry. Summary of the Invention

[0009] In view of this, the present disclosure provides a method for detecting and correcting large model illusions, which at least partially solves the problems existing in the prior art.

[0010] In a first aspect, embodiments of this disclosure provide a method for detecting and correcting large model illusions, the method comprising the following steps: A quantitative representation of the internal cognitive state at each time step of the response text generated by the language model is extracted, and a cognitive trajectory sequence is generated based on the quantitative representation; Based on the cognitive trajectory sequence, calculate the internal inconsistency score to obtain potential hallucination points; For the potential hallucination points and the entire generated text, multi-source external data is obtained, and the external data support is calculated based on the multi-source external data; The internal inconsistency score is fused with the external data support to obtain the comprehensive illusion score; The final hallucination point is determined based on the comprehensive hallucination score, and then the final hallucination point is corrected.

[0011] According to a specific implementation of this disclosure, the step of extracting a quantitative representation of the internal cognitive state of each time step of the language model generating the response text, and generating a cognitive trajectory sequence based on the quantitative representation, includes: A quantitative representation of the internal cognitive state at each time step of the language model generating the response text is extracted; the quantitative representation of the internal cognitive state includes: the attention weight matrix, the hidden state vector of the decoder layer, and the probability distribution on the vocabulary; The average attention entropy and attention concentration ratio are obtained based on the attention weight matrix. Based on the hidden state vector, extract the vector and activation sparsity after dimensionality reduction by principal component analysis; Extract normalized entropy and top-1 probability based on the probability distribution on the vocabulary; The average attention entropy, attention concentration ratio, vector after dimensionality reduction by principal component analysis, activation sparsity, normalized entropy, and top-1 probability are concatenated as features and mapped to generate a cognitive trajectory sequence through a neural network encoder.

[0012] According to a specific implementation of this disclosure, the step of calculating the internal inconsistency score based on the cognitive trajectory sequence includes: Analyze the abnormal patterns of the cognitive trajectory sequence; wherein, the abnormal patterns include temporal abruptness features, attentional distraction features, confidence abnormality features, and hidden state abnormality features; The temporal abruptness characteristic is calculated based on the following formula: ; in, This is a temporal abrupt change characteristic. ; In time step The original temporal abrupt change degree; For all time steps; The attention distractibility feature is calculated based on the following formula: ; in, For the feature of attentional distractibility, ; The slope parameter of the Sigmoid function; In time step The relative dispersion; For time step index; The confidence level anomaly feature is calculated based on the following formula: ; in, For confidence level anomaly features, ; This is an indicator of uncertainty anomalies; Weights for uncertainty anomaly indicators; For decision-making fuzziness indicators; Weights for indicators of decision-making fuzziness; This is an indicator of distribution offset anomalies; Weights for distribution offset anomaly indicators; ; The hidden state anomaly feature is calculated based on the following formula: ; in, To hide the anomaly degree feature of the state, ; This is a normalized sparsity anomaly. Weights for normalized sparsity anomalies; The normalized Mahalanobis distance anomaly; The weights of the normalized Mahalanobis distance anomalies; The normalized hidden state change; The Euclidean distance between the current hidden state vector and the hidden state vector at the previous time step; The mean of the entire current sequence; This represents the standard deviation over the entire current sequence. The weights of the normalized hidden state changes; ; The internal inconsistency score is calculated based on the following formula: ; in, For internally inconsistent scores, ; For time step index; For index variables, At that time, the corresponding temporal abrupt change index, , At that time, the corresponding attention distraction index, , At that time, the corresponding confidence level outlier index, , At that time, the corresponding hidden state anomaly index, ; For the first The fusion weights of the anomalous indicators satisfy the following: .

[0013] According to a specific implementation of an embodiment of this disclosure, obtaining potential hallucination points includes: For each time step A preliminary determination is made based on the following formula: ; in, For time steps Internal hallucination indicator function; Internal inconsistency score; This is the internal inconsistency threshold; when Time step Preliminary assessment identifies it as a potential hallucination point. Time step The point was initially determined to be non-hallucination.

[0014] According to a specific implementation manner of an embodiment of the present disclosure, the obtaining of multi-source external data for the potential hallucination points and the entire generated text, and calculating the external data support degree based on the multi-source external data includes: Calculating the external data support degree based on the following formula: ; Wherein, is the external data support degree; is the weight contributed by the entity; is the weight contributed by the relationship; is the weight contributed by the claim; is the support degree score contributed by entity evidence at time step ; is the support degree score contributed by relationship evidence at time step ; is the support degree score contributed by claim evidence at time step ; Calculating the entity contribution based on the following formula: ; Wherein, is the support degree score contributed by entity evidence at time step ; is the total number of entities identified in the entire generated text; is the index variable; is an element of the time step - entity association matrix When , it indicates that the token generated at time step is within the text range of entity , and when , it indicates that time step has nothing to do with entity ; is the external data support degree of entity ; is the entity weight coefficient; is a small positive number to prevent division by zero; Calculating the relationship contribution based on the following formula: ; Wherein, is the support degree score contributed by relationship evidence at time step ; is the total number of relationship triples extracted from the entire generated text; is the relationship index; is an element in the time step - relationship association matrix. When , it indicates that time step The generated token is within the text range of the subject, predicate, or object of the relational triple When represents the time step is unrelated to the relational triple ; is the external data support degree of the relational triple ; is the relational weight coefficient; is a small positive number to prevent division by zero; The claim contribution is calculated based on the following formula: ; where is the support degree score contributed by the claim evidence at the time step ; is the total number of claims segmented from the entire generated text; is the claim index; is an element in the time step - claim association matrix. When it means that the token generated at the time step is within the text range of the claim . When it means that the time step is not within the text range of the claim ; is the external data support degree of the entire claim ; is the importance score of the claim ; is a small positive number to prevent division by zero.

[0015] According to a specific implementation manner of an embodiment of the present disclosure, the fusion of the internal inconsistency score and the external data support degree to obtain a comprehensive hallucination score includes: The comprehensive hallucination score is calculated based on the following formula: ; where is the comprehensive hallucination score, ; is the combined fusion score at the time step ; The combined fusion score at the time step is calculated based on the following formula : ; where is the combined fusion score at the time step ; is the combined fusion score at the time step The posterior probability; In time step The score calculated by the linear weighted fusion strategy; In time step The score calculated by the product fusion strategy; In time step The score calculated by the maximum evidence fusion strategy; The weights are the posterior probabilities; The score weights are calculated for the linear weighted fusion strategy; The weights of the scores calculated for the product fusion strategy; The weights of the scores calculated for the maximum evidence fusion strategy.

[0016] According to a specific implementation of an embodiment of this disclosure, the method further includes: Merge consecutive potential hallucination points into hallucination segments; Calculate the overall confidence level for each hallucination segment; The hallucination segments are classified based on the contribution ratio of internal and external evidence to obtain hallucination types; the hallucination types include: internal-dominant, external-dominant, and mixed types.

[0017] According to a specific implementation of this disclosure, the step of determining the final hallucination point based on the comprehensive hallucination score and correcting the final hallucination point includes: Obtain hallucination data and constraint types; the hallucination data includes: the original content of the hallucination segment, and the content of the context of the hallucination segment; the constraint types include: semantic constraints, external evidence constraints, and hallucination type constraints; Generate a modified constraint set based on illusion data and constraint type; Based on the set of modified constraints, modification candidates are generated; The evaluation metrics for each candidate correction are calculated separately, and the evaluation metrics are weighted to obtain a weighted comprehensive score; wherein, the evaluation metrics include: factual accuracy score, grammatical consistency score, style consistency score, contextual coherence score, and information retention score; Candidates for revision are selected based on the weighted comprehensive score.

[0018] Secondly, embodiments of this disclosure provide a system for detecting and correcting large-scale illusions, the system comprising: According to a specific implementation of an embodiment of this disclosure, the system further includes: The data acquisition module is configured to extract a quantitative representation of the internal cognitive state at each time step of the language model generating the response text, and to generate a cognitive trajectory sequence based on the quantitative representation; The potential hallucination point module is configured to calculate an internal inconsistency score based on the cognitive trajectory sequence to obtain potential hallucination points; The support module is configured to acquire multi-source external data for the potential illusion points and the entire generated text, and to calculate the external data support based on the multi-source external data; A fusion module is configured to fuse the internal inconsistency score with the external data support to obtain a comprehensive illusion score; The correction module is configured to determine the final illusion point based on the comprehensive illusion score and to correct the final illusion point.

[0019] Thirdly, embodiments of this disclosure also provide a computer program product, the computer program product including a computing program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method for detecting and correcting large model illusions in the first aspect or any implementation thereof.

[0020] The method for detecting and correcting large-scale hallucinations in this embodiment of the present disclosure achieves a fundamental shift from black-box output inspection to white-box process monitoring by extracting multi-dimensional internal states such as attention, hidden states, and output probabilities in real time and encoding them into analyzable cognitive trajectories. By fusing four types of cognitive anomaly features—temporal abruptness, attention distraction, confidence anomaly, and hidden state anomaly—it can efficiently and in real-time initially locate potential hallucination points, providing accurate targets for subsequent external verification. By finely aligning the time steps of evidence and generated text, it can calculate precise external support for each generated lemma, greatly improving the coverage and accuracy of fact-checking, and effectively identifying evidence conflicts and quantifying uncertainty, avoiding misjudgments caused by the lack or error of a single knowledge source. By fusing internal inconsistency scores with the external data support through a Bayesian framework and a multi-strategy fusion model, and making decisions through adaptive thresholds, it automatically classifies hallucinations into internally dominant, externally dominant, or hybrid types. The system intelligently selects correction strategies based on the type and context of the hallucination, and injects correct information in real time during the generation process through factual constraint decoding. By evaluating multiple candidates from multiple dimensions to select the optimal solution and conducting iterative verification, it ensures that the corrected result is not only factually accurate, but also maintains a high degree of grammatical, stylistic, and logical coherence with the original text. This maximizes the preservation of the original meaning while eliminating errors, forming a quality closed loop of detection-correction-verification. Attached Figure Description

[0021] Figure 1 A schematic flowchart of a method for detecting and correcting large-scale illusions provided in this embodiment of the disclosure; Figure 2 A flowchart illustrating a method for detecting and correcting large-scale illusions provided in this disclosure embodiment; Figure 3 This is a schematic diagram of a system structure for detecting and correcting large-scale illusions, provided in an embodiment of this disclosure. Detailed Implementation

[0022] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0023] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0024] It should be noted that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. Furthermore, this device and / or this method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0025] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0026] This invention provides a method for detecting and correcting large-scale hallucinations. By extracting multi-dimensional internal states such as attention, hidden states, and output probabilities in real time and encoding them into analyzable cognitive trajectories, it achieves a fundamental shift from black-box output inspection to white-box process monitoring. By fusing four types of cognitive anomaly features—temporal abruptness, attentional distraction, confidence anomaly, and hidden state anomaly—it can efficiently and in real time initially locate potential hallucination points, providing accurate targets for subsequent external verification. By finely aligning the time steps of evidence with the generated text, it can calculate precise external support for each generated lexical unit, greatly improving the coverage and accuracy of fact-checking, and effectively identifying evidence conflicts and quantifying uncertainty, avoiding misjudgments caused by the absence or error of a single knowledge source. By fusing internal inconsistency scores with the external data support through a Bayesian framework and a multi-strategy fusion model, and making decisions through adaptive thresholds, it automatically classifies hallucinations into internally dominant, externally dominant, or hybrid types. Based on the hallucination type and context, it intelligently selects correction strategies and injects correct information in real time during the generation process through fact constraint decoding. By evaluating multiple candidates from multiple dimensions to select the optimal solution and conducting iterative verification, the corrected result is ensured to be not only factually accurate, but also to maintain a high degree of grammatical, stylistic, and logical coherence with the original text. This preserves the original meaning to the greatest extent while eliminating errors, forming a quality closed loop of detection-correction-verification.

[0027] Figure 1 This is a schematic diagram of the method flow for detecting and correcting large-scale illusions provided in the embodiments of this disclosure.

[0028] Figure 2 To and Figure 1 The flowchart of the corresponding large-scale illusion detection and correction method.

[0029] like Figure 1 As shown, in step S110, a quantitative representation of the internal cognitive state of each time step of the language model generating the response text is extracted, and a cognitive trajectory sequence is generated based on the quantitative representation.

[0030] In this embodiment of the invention, the step of extracting a quantitative representation of the internal cognitive state at each time step of the language model generating the response text, and generating a cognitive trajectory sequence based on the quantitative representation, includes: extracting a quantitative representation of the internal cognitive state at each time step of the language model generating the response text; the quantitative representation of the internal cognitive state includes: an attention weight matrix, a hidden state vector of the decoder layer, and a probability distribution on the vocabulary; obtaining the average attention entropy and attention concentration ratio based on the attention weight matrix; extracting the vector after dimensionality reduction by principal component analysis and the activation sparsity based on the hidden state vector; extracting the normalized entropy and top-1 probability based on the probability distribution on the vocabulary; concatenating the average attention entropy, attention concentration ratio, vector after dimensionality reduction by principal component analysis, activation sparsity, normalized entropy, and top-1 probability into features, and mapping them through a neural network encoder to generate a cognitive trajectory sequence.

[0031] More specifically, it includes the following steps: I. Cognitive Trajectory Extraction and Encoding At each time step in the language model's generation of response text, a quantitative representation of its internal cognitive state is extracted, and these high-dimensional, heterogeneous internal states are encoded into a unified, analyzable cognitive trajectory sequence, forming a traceable cognitive trajectory.

[0032] 1. Receive user queries The word-by-word generation process of the language model, and initialization of empty sequences. Used to store cognitive feature vectors. ,in, For the query of the first Each word element, .

[0033] 2. Generate response text using the language model Each time step In the process, three types of internal states are extracted simultaneously: (1) Extract the attention weight matrix of the last layer.

[0034] Let the model have Each attention head, for each attention head Extracting the attention weight matrix ;in This indicates that the response text is generated. At that time, in the In the attention of the head, the position Position The level of attention given to it.

[0035] (2) Extract the hidden state vector of the last layer (or key layer). ,in, For the hidden layer dimension.

[0036] (3) Extract the probability distribution from the vocabulary list.

[0037] Extract the probability distribution of the softmax layer output. ,satisfy ,and ,in, For vocabulary size, This is a lexical index variable.

[0038] 3. Attention Feature Extraction Extracting statistical features from high-dimensional attention weights to characterize the model at time steps The attention pattern.

[0039] (1) Multi-head attention aggregation, including: Calculate the average attention weights: ; Preserve the standard deviation of attention for each head: .

[0040] in, For time steps The average attention weight matrix; The total number of attention heads; For the index variable of the attention head; For time steps No. Attention weight matrix for each attention head; For time steps The standard deviation of attention.

[0041] (2) Focus on concentration calculation, including: For each query position The entropy of its attention distribution is calculated based on the following formula:

[0042] middle, For time steps No. Attention distribution entropy at each location; This is the current time step generated; For index variables; For time steps Total length of the context; The index of the key position; Position in the average attention matrix The value; To prevent the logarithm from being a negative infinitesimal constant, it is preferable to... .

[0043] The average attention entropy is calculated based on the following formula:

[0044] The variance of attention entropy is calculated using the following formula:

[0045] (3) Key location identification, including: The total attention weight obtained at each location is calculated based on the following formula:

[0046] in, For time steps No. The total attention weight obtained at each location; Position in the average attention matrix The value of .

[0047] The following formula is used to calculate the pre-identification value. The most watched position:

[0048] in, For time steps The top attention weight A set of indices for each position; For index Perform sorting operations; For the first Total attention weight for each position; To retrieve the sorted order before Each element.

[0049] Calculate the time step based on the following formula. forward Percentage of attention focused on each location :

[0050] (4) Cross-head consistency measures, including: The cosine similarity matrix between attention heads is calculated using the following formula:

[0051] in, For time steps No. and Cosine similarity between attention heads; This is a vectorization operation used to flatten a matrix into a vector. For time steps No. The average attention matrix of each attention head; For time steps No. The average attention matrix of each attention head.

[0052] The average cross-head similarity is calculated based on the following formula:

[0053] in, For time steps Mean cosine similarity of all attention heads.

[0054] Output attention feature vector :

[0055] in, For time steps Attention feature vector; For time steps Average attention entropy; For time steps The variance of attention entropy; For time steps The percentage of attention focused; For time steps Average head-to-head similarity; For time steps Attention weight ranking A set of indices for each position; This is the set of indexes for all query positions at the current moment; It is a one-hot encoding function; This represents the attention feature dimension.

[0056] 4. Hidden state feature extraction, including: Extract features representing the thinking state of the model from the high-dimensional hidden state.

[0057] (1) Principal component extraction: Time step Hidden state vector Perform principal component analysis (PCA) for dimensionality reduction:

[0058] in, The hidden state vector after dimensionality reduction by principal component analysis; This is the transpose of the PCA transformation matrix (principal component matrix); For PCA projection matrix, (Preferred) =50); For time steps The original hidden state vector; This is the mean vector of the hidden states.

[0059] (2) Activation mode analysis, including: The activation intensity of each dimension of the hidden state is calculated based on the following formula:

[0060] in, This is the feature vector after processing with the ReLU activation function.

[0061] The activation sparsity metric is calculated based on the following formula:

[0062] in, For time steps The sparsity index is typically between [0,1], where 0 indicates that it is not sparsy at all (all dimensions have values) and 1 indicates that it is sparsy (only one dimension has a value). Indicates the vector Each element in the array is mapped to its symbol; The total dimension of the hidden state.

[0063] Through calculation mean Standard deviation and kurtosis The activation intensity distribution is obtained.

[0064] (3) Time series rate of change: when At that time, the hidden state change is calculated based on the following formula:

[0065] in, For time steps The first-order difference of the hidden state; For time steps The hidden state vector; For time steps The hidden state vector.

[0066] Calculate the changing acceleration using the following formula: ( ) in, For time steps The second-order difference of the hidden state; For time steps The first-order difference of the hidden state; For time steps The first-order difference of the hidden state.

[0067] Output hidden state feature vector :

[0068] in, For time steps The hidden state feature vector; The hidden state before PCA dimensionality reduction One principal component; , indicating taking the previous one One principal component; For time steps Sparsity index; For time steps The mean of the activation values; For time steps The standard deviation of the activation values; For time steps kurtosis of activation values; For time steps The first-order difference of the hidden state; For time steps Second-order difference of the hidden state.

[0069] 5. Output distribution feature extraction, including: Extract features that characterize the decision determinism of the model from the lexical probability distribution.

[0070] (1) Measurement of distribution uncertainty: Calculate the entropy of the probability distribution:

[0071] in, For time steps Information entropy; For time steps Model prediction categories The probability value; The size of the vocabulary list; It is a very small constant.

[0072] Normalized entropy :

[0073] (2) Decision-making sharpness: calculate Probability value:

[0074] in, For time steps Predict the maximum value in the probability distribution.

[0075] calculate Probability ratio:

[0076] in, For time steps Confidence ratio; For time steps The second highest probability value in the predicted probability distribution.

[0077] calculate Cumulative probability:

[0078] in, For time steps forward The cumulative sum of the highest probability; For the first High prediction probability; To take the front The threshold with the highest probability is preferably... .

[0079] (3) Distribution anomaly detection: The KL divergence with a uniform distribution is calculated using the following formula:

[0080] The KL divergence from the historical average distribution is calculated using the following formula. :

[0081] (4) Candidate lexical analysis: Extracting the top-3 candidate terms:

[0082] in, For time steps Model predictions Candidate category set The index / identifier for the top three categories (or words) that the model considers to have the highest probability; .

[0083] Calculate candidate lexical units and semantic similarity between and (Through pre-trained word embeddings).

[0084] Output distribution feature vector :

[0085] 6. Feature Fusion and Encoding The three types of features are integrated into a unified cognitive state representation.

[0086] (1) Feature splicing:

[0087] in, For time steps The concatenated feature vectors; For time steps Attention feature vector; For time steps The hidden state feature vector; For time steps The output layer feature vector; Dimension of the attention feature vector; is the dimension of the hidden state feature vector; This specifies the dimension of the output feature vector.

[0088] (2) Normalization process: For each feature dimension Calculate the mean on the training set and standard deviation .

[0089] standardization:

[0090] in, For time steps The first concatenated vector The standardized values ​​of each dimension; For time steps splicing vectors The first in One original feature value; In the training dataset, the concatenated vector is... The mean of each dimension; In the training dataset, the concatenated vector is... Standard deviation of each dimension; To prevent small positive numbers from being divided by zero.

[0091] (3) Dimensional compression: Features are further compressed using a small neural network encoder:

[0092] in, For time steps The final feature vector after encoder processing; This is the encoder function, which can be a multilayer perceptron (MLP) or a fully connected neural network; For time steps The standardized value of the concatenated vector; The weights are for the first layer of linear transformation. ; The weights for the second-level linear transformation. , The final feature dimension (preferably 128); The learnable bias vector for the first layer; This is the learnable bias vector for the second layer.

[0093] (4) Add to cognitive trajectory sequence: After extracting the states at all time steps, a complete cognitive trajectory sequence is obtained: .

[0094] This step provides the data foundation for the entire hallucination detection and correction method, enabling subsequent analysis to be based on the cognitive processes within the model rather than just the final output, thereby improving the sensitivity and interpretability of the detection.

[0095] More specifically, we now proceed to step S120.

[0096] In step S120, an internal inconsistency score is calculated based on the cognitive trajectory sequence to obtain potential hallucination points.

[0097] In this embodiment of the invention, calculating the internal inconsistency score based on the cognitive trajectory sequence includes: analyzing the abnormal patterns of the cognitive trajectory sequence; wherein, the abnormal patterns include temporal abruptness features, attentional distraction features, confidence anomaly features, and hidden state anomaly features; The temporal abruptness characteristic is calculated based on the following formula: ; in, This is a temporal abrupt change characteristic. ; In time step The original temporal abrupt change degree; For all possible time steps; The attention distractibility feature is calculated based on the following formula: ; in, For the feature of attentional distractibility, ; The slope parameter of the Sigmoid function; In time step The relative dispersion; For time step index; The confidence level anomaly feature is calculated based on the following formula: ; in, For confidence level anomaly features, ; This is an indicator of uncertainty anomalies; Weights for uncertainty anomaly indicators; For decision-making fuzziness indicators; Weights for indicators of decision-making fuzziness; This is an indicator of distribution offset anomalies; Weights for distribution offset anomaly indicators; ; The hidden state anomaly feature is calculated based on the following formula: ; in, To hide the anomaly degree feature of the state, ; This is a normalized sparsity anomaly. Weights for normalized sparsity anomalies; The normalized Mahalanobis distance anomaly; The weights of the normalized Mahalanobis distance anomalies; The normalized hidden state change; The Euclidean distance between the current hidden state vector and the hidden state vector at the previous time step; The mean of the entire current sequence; This represents the standard deviation over the entire current sequence. The weights of the normalized hidden state changes; ; The internal inconsistency score is calculated based on the following formula: ; in, For internally inconsistent scores, ; For time step index; For index variables, At that time, the corresponding temporal abrupt change index, , At that time, the corresponding attention distraction index, , At that time, the corresponding confidence level outlier index, , At that time, the corresponding hidden state anomaly index, ; For the first The fusion weights of the anomalous indicators satisfy the following: .

[0098] In this embodiment of the invention, obtaining potential hallucination points includes: For each time step A preliminary determination is made based on the following formula: ; in, For time steps Internal hallucination indicator function; Internal inconsistency score; This is the internal inconsistency threshold; when Time step Preliminary assessment identifies it as a potential hallucination point. Time step The point was initially determined to be non-hallucination.

[0099] More specifically, it includes the following steps: I. Internal Inconsistency Detection Based on cognitive trajectory sequence By analyzing abnormal patterns in feature sequences, we can identify potential internal cognitive inconsistencies during model generation, thereby locating potential hallucination points, reducing over-reliance on external knowledge sources, and improving detection efficiency.

[0100] 1. The cognitive feature sequences are processed for missing values, standardized, denoised, and smoothed using a sliding window to provide stable and comparable feature representations for subsequent analysis.

[0101] Sliding window smoothing is based on the following formula: To reduce noise fluctuations, a moving average is applied to each feature:

[0102] in, For time steps Dimensions The value after smoothing; The window radius is preferably 2-3; This is the offset of the time step; These are the feature values ​​that have shifted in the time dimension after standardization.

[0103] Boundary handling: handling boundary locations or Use the average within the valid window.

[0104] Obtain the smoothed standardized feature sequence .

[0105] 2. Calculation of temporal abrupt change degree When a model generates hallucinatory content, its internal cognitive state often undergoes sudden and abnormal changes, which are discontinuous with the preceding and following generation steps.

[0106] (1) First-order difference calculation: Calculate the characteristic changes of adjacent time steps:

[0107] (2) Calculation of the range of change: The norm of the changing vector is used as the magnitude of the change:

[0108] in, Preferably, it is 2 (Euclidean distance) or 1 (Manhattan distance). (3) Adaptive threshold setting: Calculate the moving average of the change: Moving average:

[0109] Moving standard deviation:

[0110] in, The window size is preferably 10-20.

[0111] Define temporal abrupt change degree:

[0112] in, The sensitivity parameter is preferably 2-3.

[0113] (4) Normalize the mutation degree to the [0,1] interval:

[0114] Focusing only on positive anomalies (sudden increases), the temporal abrupt change characteristic is: .

[0115] 3. Calculation of attention distractibility When the model is uncertain or contains fictional content, its attention mechanism often exhibits a scattered and unfocused pattern, failing to concentrate on the relevant context.

[0116] (1) Extracting attention-related features: From cognitive characteristics Extract relevant components, including: average attention entropy. Attention concentration ratio and cross-head consistency .

[0117] (2) Calculation of attention distractibility: Define the attention distraction index:

[0118] in, For time steps Attention distractibility index; , , Let be the weighting coefficient, satisfying Preferred .

[0119] (3) Context normalization: Calculate the baseline dispersion in the currently generated sequence:

[0120] in, Baseline dispersion; This represents the length of the historical time window.

[0121] Calculate the relative dispersion:

[0122] (4) Nonlinear mapping: Applying the sigmoid function to enhance the sensitivity of anomaly detection:

[0123] in, For time steps The normalized threshold after mapping by the sigmoid function, when When the baseline is exactly the same, the output value is 0.5. When the value deviates from the baseline, the output value approaches 1. This is an adjustment coefficient used to control the steepness of the curve, preferably 3-5.

[0124] Characteristics of attention distractibility: .

[0125] 4. Calculation of confidence level and outlier Hallucination content is often accompanied by abnormal probability distribution patterns, such as low confidence but being selected, multiple candidate words with similar probabilities, and distributions that are significantly different from historical patterns.

[0126] (1) Calculate the uncertainty anomaly based on entropy :

[0127] High entropy ( And low top-1 probability ( )hour, Close to 1.

[0128] Low entropy ( or high top-1 probability ( )hour, Close to 0.

[0129] (2) Calculate decision fuzziness:

[0130] in, For decision-making fuzziness indicators; This is an adjustment coefficient used to control the transition speed, preferably 5; For time steps The ratio indicator represents the ratio of a key statistic (such as prediction error, variance, etc.) at the current time step to the historical baseline.

[0131] when ( and When the probabilities are similar, This indicates ambiguity in decision-making.

[0132] when ( The probability is significantly higher than )hour, This indicates that the decision-making was clear.

[0133] (3) Calculate the distribution offset anomaly:

[0134] in, These are outlier values ​​due to distribution shift. The offset threshold is determined through statistical analysis of the training data; For time steps Historical cumulative measures represent long-term system stress or cumulative deviation, as previously stated. The sum of abnormal scores at each time step, cumulative delay, etc.

[0135] (4) Overall confidence level outlier: Three abnormal indicators for weighted fusion:

[0136] in, For time steps The overall confidence level and outlier score; Weights for uncertainty anomaly indicators; For decision-making fuzziness indicators; Weights for indicators of decision-making fuzziness; This is an indicator of distribution offset anomalies; Weights for distribution offset anomaly indicators; , , preferred .

[0137] Confidence level anomaly characteristics: .

[0138] 5. Calculation of Anomaly Degree in Hidden State Anomalous patterns in hidden states can reflect disorder or inconsistency in the internal representation of the model.

[0139] (1) Calculate sparsity anomalies:

[0140] in, For time steps Sparsity outliers; For time steps Sparsity statistics; The average sparsity of the sequence.

[0141] After normalization, we get: .

[0142] (2) Detection of principal component trajectory anomalies: Construct the principal component trajectory matrix:

[0143] Calculate the point-by-point principal component Mahalanobis distance:

[0144] in, For time steps Mahalanobis distance; For time steps PCA feature vectors; The mean vector of the PCA features; It is the mean vector. Let covariance matrix be the variance matrix. for The inverse matrix.

[0145] Calculate Mahalanobis distance anomaly:

[0146] in, It has degrees of freedom. The 99th percentile of the chi-square distribution.

[0147] (4) Overall hidden state anomaly degree: Weighted fusion:

[0148] in, The score is based on the overall anomaly level of the hidden state. For time steps The normalized sparsity anomaly is . For time steps The normalized Mahalanobis distance; For time steps The trend change measure represents the amount of change in the current feature value relative to the previous moment; The historical standard deviation of the trend change ; This represents the historical average of the trend change, indicating the trend within a historical time window. The average value represents the normal level of variation. ; Weights for normalized sparsity anomalies; The weights of the normalized Mahalanobis distance anomalies; The weights of the normalized hidden state changes; Preferred .

[0149] Hidden state anomaly characteristics: .

[0150] 6. Multi-indicator fusion and internal inconsistency scoring The four abnormal indicators mentioned above are merged into a unified internal inconsistency score.

[0151] (1) Indicator normalization: Ensure all indicators are within the range [0,1] and in the same direction (larger values ​​indicate greater anomalies):

[0152]

[0153]

[0154]

[0155] (2) Assigning weights based on the reliability of indicators: Normalized discriminant values ​​of each metric on the training set As a basis for weighting:

[0156] in, For the first The basic weight of each indicator; For the first The AUROC value of each indicator.

[0157] Calculate the internal inconsistency score:

[0158] in, For time steps Internal inconsistency score; For time steps No. Normalized signal values ​​of each indicator; .

[0159] Internal inconsistency score: .

[0160] 7. Preliminary identification of potential hallucination points Based on the internal inconsistency score, possible hallucination points are initially identified.

[0161] (1) Set the initial threshold :

[0162] in, , The sensitivity parameter is preferably 1.5-2.5.

[0163] (2) For each time step Preliminary assessment: ; in, For time steps Internal hallucination indicator function; Internal inconsistency score; This is the internal inconsistency threshold; when Time step Preliminary assessment identifies it as a potential hallucination point. Time step The point was initially determined to be non-hallucination.

[0164] (3) Connecting consecutive potential hallucination points Merged into hallucination segment .

[0165] The merging rules are as follows: If and If the length is less than 1, then they are merged into one segment. A fragment.

[0166] (4) For each hallucination segment, calculate the average inconsistency score as the confidence level:

[0167] in, The overall confidence level for each hallucination segment; This is the end time of the time window; This is the start time of the time window; For a moment The internal inconsistency score.

[0168] This step provides important internal evidence for hallucination detection by analyzing abnormal patterns in the model's internal cognitive state from multiple dimensions. Subsequent steps will combine external knowledge sources for verification to improve the accuracy of the detection.

[0169] Next, proceed to step S130.

[0170] In step S130, multi-source external data is obtained for the potential illusion points and the entire generated text, and the external data support is calculated based on the multi-source external data.

[0171] In this embodiment of the invention, the step of acquiring multi-source external data for the potential hallucination points and the entire generated text, and calculating the external data support based on the multi-source external data, includes: External data support is calculated using the following formula: ; in, External data support; The weight contributed to the entity; The weight of contribution to the relationship; Weighting of the claimed contribution; In time step The support score contributed by physical evidence; In time step The support score contributed by the relationship evidence; In time step The score is determined by the support score of the evidence contributing to the claim. Entity contribution is calculated based on the following formula: ; in, In time step The support score contributed by physical evidence; This represents the total number of entities identified in the entire generated text. For index variables; For the elements in the time step-entity association matrix, when When it represents a time step The generated token is within the text range of the entity When it represents a time step It has nothing to do with the entity ; is the external data support degree of the entity ; is the entity weight coefficient; is a small positive number to prevent division by zero; Calculate the relationship contribution based on the following formula: ; where is the support degree score contributed by the relationship evidence at the time step ; is the total number of relationship triples extracted from the entire generated text; is the relationship index; is an element in the time step - relationship association matrix. When it represents that the token generated at the time step is within the text range of the subject, predicate or object of the relationship triple . When it represents that the time step has nothing to do with the relationship triple ; is the external data support degree of the relationship triple ; is the relationship weight coefficient; is a small positive number to prevent division by zero; Calculate the claim contribution based on the following formula: ; where is the support degree score contributed by the claim evidence at the time step ; is the total number of claims segmented from the entire generated text; is the claim index; is an element in the time step - claim association matrix. When it represents that the token generated at the time step is within the text range of the claim . When it represents that the time step is not within the text range of the claim ; is the external data support degree of the entire claim ; is the importance score of the claim ; To prevent small positive numbers from being divided by zero.

[0172] More specifically, for the initially identified potential hallucination points and the entire generated text, supporting or refuting evidence is collected from multi-source external knowledge bases, and this evidence is aligned with the time steps and semantic units of the generated content. Quantitative external data support is calculated for each time step, specifically including the following steps: I. Semantic parsing and information unit extraction of generated text The generated text is broken down into verifiable semantic units (entities, relations, claims), providing structured queries for targeted evidence collection.

[0173] 1. Lexical-to-lexical mapping: The word sequence of the generated text Sub-words and lexical elements are combined into a complete vocabulary: ,in, .

[0174] Establish mapping Indicates time step Corresponding vocabulary index j.

[0175] 2. Named Entity Recognition: Identify entities in text using a pre-trained NER model:

[0176] in, For a collection of entities; For the first The text content of each entity; For the first Type tags for each entity (including people, locations, times, organizations, etc.); For the first The start and end positions (boundary positions) of an entity in the word sequence; For set The total number of entities in; It is an index variable.

[0177] 3. Relationship triple extraction: Identify relationships between entities using relation extraction models:

[0178] in, For a set of relations; As the main entity; It is a relation type; For the object entity; For the location index of the main entity; For the location index of the object entity; For set The total number of relational triples in the equation.

[0179] 4. Advocating for division: Segment the generated text into independent claims:

[0180] in, For the purpose of the set; For the set of Each claim For each complete statement of fact, each claim is associated with its lexical position range: ; For set The total number of claims in the group; It is an index variable.

[0181] 5. Obtain the claim importance score based on the following formula:

[0182] in, For the first Each claim's importance score; For the first The amount of information contained in each claim; For the first The degree of certainty of a claim; For the first The entity density of the claim; For the first The weight of the information content of each claim For the first The weight of the degree of certainty of each claim; For the first The weight of the entity density claimed by each entity.

[0183]

[0184] in, P For words The probability of word frequency; To advocate Words in the text;

[0185] in, To assert the length in the word sequence; For the time step within the scope of the claim; To assert the span in the sequence; For the model at time step The probability value for a prediction of a specific category (such as true or fact).

[0186]

[0187] II. Collection of Multiple Sources of Evidence Evidence related to semantic units is collected in parallel from multiple external knowledge sources, covering information of varying granularity and reliability. These external knowledge sources include: structured knowledge bases, unstructured document libraries, real-time information sources, and internal model knowledge.

[0188] 1. Query construction: For each entity Perform structured and unstructured queries.

[0189] Structured query: Retrieve entity attributes (such as date of birth, date of death, occupation, etc.).

[0190]

[0191] in, For entities The generated structured query statement; This is a query command that retrieves the attributes and corresponding values ​​of the entity. For the entity; This is an entity linking function that maps text entities to knowledge base IDs; These are attribute variables, representing a certain attribute of an entity (such as date of birth or occupation). For object variables, it represents the value corresponding to the attribute (e.g., 1990 or doctor).

[0192] Unstructured query: Construct search query terms.

[0193]

[0194] in, For entities The generated text search query terms; For entity text; It is an entity type.

[0195] For each relation triple Check if the triple exists in the knowledge base:

[0196] in, A Boolean query used to verify whether there is an association between subjects, relations, and objects; The main entity is mapped to a KG ID via a link function; It is a relational mapping function; The subject of the research.

[0197] Each claim It is broken down into multiple queries, covering all components of the claim.

[0198] 2. Parallel query execution: Establish a connection pool for each knowledge source, and send queries to all relevant sources in parallel for each semantic unit.

[0199] Configure timeout and retry mechanisms: Maximum number of retries .

[0200] For unstructured queries, use BM25 or vector retrieval to obtain top-K relevant fragments. 3. Evidence Collection: Structured evidence: ,in, To query entities / query topics; For attribute names, such as date of birth, occupation, etc.; For attribute values, such as 1990, doctor; This represents the total number of attribute-value pairs associated with the queried entity.

[0201] Unstructured evidence: ,in, For the first A text fragment (a paragraph of text extracted from a retrieved webpage or document, containing context relevant to the query). For the first The relevance score of each segment; The total number of fragments retrieved and retained (Top-K results).

[0202] Real-time evidence: ,in, For the first Real-time information, such as current stock price, today's weather, and the latest news headlines; For the first The timestamp of each message; For the first The source of this information; The number of real-time information entries collected.

[0203] Internal model evidence: ,in, For the model's first An internal response or search result; For the model to the first The confidence level of each response; The number of candidate responses generated for the model.

[0204] 4. Evidence quality assessment: Each knowledge source is assigned a basic credibility level. Structured knowledge base (High-precision) authoritative document (Verification required), Real-time source: (May be inaccurate), internal model (This should be used with caution).

[0205] Calculating evidence freshness based on time-sensitive information:

[0206] in, As evidence Freshness; The half-life (e.g., news is 1 day, scientific facts are 1 year); The current time; As evidence Generate / get timestamps.

[0207] The final set of original evidence was obtained. .

[0208] III. Evidence and Time Step Alignment By mapping the collected evidence to specific time steps in the generated text, a fine-grained time series of evidence support is established.

[0209] 1. Mapping matrix from time steps to semantic units: Construct a mapping matrix , ,in, This represents the number of time steps.

[0210] Entity Association Matrix When time steps In entity When the location is within range ,otherwise .

[0211] Relationship matrix When time steps In relation When the subject or object or relation word is within the range of position ,otherwise .

[0212] Advocating the correlation matrix When time steps In advocacy When the location is within range ,otherwise .

[0213] 2. Semantic unit evidence scoring: For each entity Calculate the support level of entity evidence:

[0214] in, For entities External data support; For all available data sources; For data source For entities The level of support for its existence; For data source The weight.

[0215] For each relation triple Calculate relation support :

[0216] in, For the first A relational triple, As the main entity, For relational types, For the object entity; For natural language statements transformed from triples, the NLI function calculates the text implication score; This is a document evidence set (text fragments obtained by retrieving relevant web pages).

[0217] For each claim Calculate the support for the claim :

[0218] in, For the first One proposition; For real-time evidence (the latest news, social media updates, or live data streams).

[0219] The text entailment probability is calculated using a Transformer-based NLI model.

[0220] 3. For each time step The evidence from aggregated semantic units is used to calculate the overall support from external data.

[0221] in, For time steps Comprehensive external data support; For time steps The overall contribution of entities; Weighting of the entity's overall contribution; For time steps Overall contribution of the relationship; Weighting of overall contribution to the relationship; For time steps The overall contribution of the claims; To advocate for a comprehensive contribution weight; Preferred .

[0222]

[0223] in, This represents the total number of entities involved in the current context. For each element in the entity-time step association matrix, the element represents the time step. ,entity Whether it is activated, mentioned, or related to the current processing unit; For entities Support score; This is a global weight parameter at the entity level, used to adjust the overall size of the entity's contribution; It is a very small constant.

[0224]

[0225] in, This represents the total number of relation triples involved in the current context. For each element in the relation-time step incidence matrix, the element represents the time step. Relational Triples Is it active or connected to the current node? For relational triples Support score; These are global weight parameters at the relational level; It is a very small constant.

[0226]

[0227] in, This represents the total number of claims involved in the current context. For the elements in the claim-time step correlation matrix; To advocate Support score; The function indicates the importance of the claim; It is a very small constant.

[0228] To ensure contextual coherence, evidence support can be smoothed over time, and boundary processing can be performed using mirror padding or truncation.

[0229] IV. Analysis of Evidence Conflicts and Uncertainties Analyze the consistency between different sources of evidence to assess the certainty and reliability of the evidence.

[0230] 1. Conflict of evidence detection: For each time step Collect supporting scores from all relevant evidence sources:

[0231] in, For time steps The amount of relevant evidence.

[0232] Calculate the consistency of evidence:

[0233] in, For time steps Consistency score; For time steps The amount of relevant evidence; In time step The The degree of support for each piece of evidence; In time step The average of all scores ;when A value close to 1 indicates a high degree of consistency in the evidence. A value close to 0 indicates inconsistent evidence.

[0234] Identifying conflicting evidence:

[0235] in, In time step score set The maximum value in; In time step score set The minimum value in; The conflict threshold is set (preferably 0.5). 2. Quantification of Evidence Uncertainty: Uncertainty weighted by the reliability of the source of evidence:

[0236] in, For time steps Below, the uncertainty score is based on the source of evidence; For the first The weight of the source of each piece of evidence; In time step The The degree of support for each piece of evidence.

[0237] Due to the uncertainty of the amount of evidence:

[0238] in, An uncertainty score based on the amount of evidence; This refers to the current actual amount of evidence collected. The preset baseline quantity (preferably 5).

[0239] Overall uncertainty:

[0240] in, For time steps Total score for overall uncertainty; Weights for uncertainty scores based on the source of evidence; Weights for uncertainty scores based on the amount of evidence; The score is based on the consistency of the evidence. Weights for the consistency score of evidence; .

[0241] 3. Evidence support level correction: Evidence support level after considering uncertainty:

[0242] in, For time steps Adjusted external support; For time steps The original external support; For time steps The overall uncertainty is as follows.

[0243] Reduce support when there is high uncertainty Consider the correction for consistency of evidence:

[0244] in, For time steps The final external entity support; For time steps Adjusted external support; The score is based on the consistency of the evidence.

[0245] 4. Time step to the source record of evidence: For each time step Record the main sources of evidence:

[0246] in, For time steps The collection of sources of evidence; For support; To assess source credibility; As a type of evidence; This is a summary of the evidence.

[0247] This step enables the systematic collection, alignment, and quantification of external evidence, providing objective factual basis for subsequent fusion decisions. Furthermore, combined with internal inconsistency detection, it can significantly improve the accuracy and interpretability of hallucination detection.

[0248] Next, proceed to step S140.

[0249] In step S140, the internal inconsistency score is fused with the external data support to obtain a comprehensive illusion score.

[0250] In this embodiment of the invention, fusing the internal inconsistency score with the external data support to obtain a comprehensive illusion score includes: The overall hallucination score is calculated based on the following formula: ; in, To calculate the overall illusion score, ; In time step The combined fusion score; Calculate the time step based on the following formula. Combination and fusion scores : ; in, In time step The combined fusion score; In time step The posterior probability; In time step The score calculated by the linear weighted fusion strategy; In time step The score calculated by the product fusion strategy; In time step The score calculated by the maximum evidence fusion strategy; The weights are the posterior probabilities; The score weights are calculated for the linear weighted fusion strategy; The weights of the scores calculated for the product fusion strategy; The weights of the scores calculated for the maximum evidence fusion strategy.

[0251] More specifically, a single detection source may produce false positives or false negatives. Internal detection is sensitive but may falsely report rare but correct information, while external detection is accurate but relies on knowledge coverage and is time-sensitive. This invention achieves complementary advantages and improves the accuracy, recall, and interpretability of detection by fusing internal inconsistency scores with external data support to calculate a comprehensive hallucination score.

[0252] I. Feature Preparation and Preprocessing Aligning and standardizing internal inconsistency scores with external data support prepares consistent feature representations for fusion computation.

[0253] 1. Verify sequence length consistency: Ensure When discrepancies exist, linear interpolation is used to fill in missing values. For internally inconsistent scores, External data support.

[0254] 2. Distribution standardization: Standardized internal inconsistency score:

[0255] in, The mean of the training set; The standard deviation over the training set; To prevent small positive numbers from being divided by zero.

[0256] External data support is converted into external contradiction, and external contradiction is positively correlated with the probability of hallucination:

[0257] in, This indicates the degree of contradiction in external evidence; External data support.

[0258] Standardized external contradiction degree:

[0259] in, For time steps The degree of standardized external contradictions; For time steps The original external contradiction degree below; This represents the mean of the degree of external contradiction. The standard deviation of the degree of external contradiction.

[0260] 3. Confidence-weighted: The confidence level of internal inconsistency can be obtained based on the confidence level assessment:

[0261] in, For time steps The weighting of internal evidence; For time steps Below, the confidence score of internal evidence.

[0262] The confidence level of external evidence can be obtained based on uncertainty analysis:

[0263] in, For time steps The weight of external evidence; For time steps The overall uncertainty is as follows; For time steps Consistency of evidence.

[0264] Normalized weights:

[0265]

[0266] II. Bayesian Fusion Framework 1. Based on Bayesian inference, calculate the posterior probability of hallucination at each time step:

[0267] in, Let be the posterior probability, representing the probability of observing internal evidence. and external evidence Then, assuming The probability of it being true; Likelihood represents the probability of observing the current combination of internal and external evidence, assuming the hypothesis is true. This is the prior probability; The marginal probability of evidence represents the total probability of observing the current combination of evidence (regardless of whether the hypothesis is true or not).

[0268] The prior probability of hallucination can be estimated from the training set; Indicates time step The hallucination exists (1) or does not exist (0); This indicates evidence of internal inconsistency; This indicates external contradictory evidence.

[0269] 2. Conditional independence assumption: Suppose that in a given hallucination state Under the following conditions, internal and external evidence are independent:

[0270] in, For joint likelihood; Conditional likelihood indicates that, when the hypothesis is true, only internal evidence is observed. The probability of; Conditional likelihood represents the likelihood of finding external evidence when the hypothesis is true. The probability of.

[0271] 3. Likelihood function modeling: Model the distribution of evidence in hallucination and non-hallucination states using a Gaussian distribution: a. Internal inconsistency likelihood:

[0272] in, Assumption Values When (1 or 0), internal evidence is observed alone. The probability of; The mean parameter represents the value under the assumed state. At that time, the ideal or expected value of internal evidence; Here is the variance parameter, representing the variance under the assumed state. In this context, the smaller the variance of the internal evidence, the more stable the evidence is.

[0273] b. External contradiction likelihood:

[0274] in, As a hypothesis Values When (1 or 0), external evidence is observed alone. The probability of; The mean parameter represents the value under the assumed state. At that time, the ideal or expected value of external evidence; Here is the variance parameter, representing the variance under the assumed state. In this context, the smaller the variance of external evidence fluctuations or uncertainties, the more stable the evidence is.

[0275] 4. Calculation of posterior probability: Substituting the likelihood function into Bayes' formula:

[0276] To avoid numerical underflow, logarithmic calculation is used, resulting in:

[0277] The final result is:

[0278] in, It is a logarithmic probability.

[0279] 5. Parameter learning: The parameters are trained using a labeled hallucination dataset to obtain the Bayesian posterior probability:

[0280] in, For time steps The final Bayesian posterior probability.

[0281] III. Linear Weighted Fusion Basic form:

[0282] in, Linear fusion score; The dynamic weighting coefficient represents the proportion of internal evidence; Internal support level after uncertainty correction or standardization; This represents the degree of external contradiction after adjustment for uncertainty or standardization.

[0283] Dynamic weight calculation:

[0284] in, These are dynamic weighting coefficients; Weighting of processed internal evidence; Weights for processed external evidence.

[0285] Normalize to [0,1]:

[0286] 2. Product fusion: Calculate the product fusion score based on the principle of evidence accumulation:

[0287] in, The product fusion score; Internal support level after uncertainty correction or standardization; This represents the degree of external contradiction after adjustment for uncertainty or standardization.

[0288] The product value is high when both types of evidence are strong; the product value is low when either type of evidence is weak.

[0289] The product fusion score is normalized.

[0290] 3. Maximum Evidence Fusion: To improve recall, take the higher value of the two types of evidence.

[0291] in, for and The maximum value in.

[0292] fusion strategy selection based on contextual features: when When the value is high, choose to apply linear weighting; when... When the value is high, choose to apply product fusion.

[0293] Increase the weight of external evidence when the query domain is known and the knowledge coverage is good.

[0294] Decision function:

[0295] 5. Final fusion score: Combining multiple fusion results:

[0296] in, For time steps The overall score; Time step Bayesian posterior probability; Linear fusion score; The product fusion score; The maximum evidence fusion score; These are the Bayesian posterior probability weights; The weights are for linear fusion scores; The weights for the product-fusion score; The optimal weight for maximizing the evidence fusion score is [missing information]. .

[0297] Finally, normalization is performed to obtain the overall illusion score. :

[0298] The higher the value, the higher the likelihood of hallucination.

[0299] IV. Multi-level threshold decision Based on the comprehensive hallucination score, the final hallucination point is determined through adaptive thresholding and context-aware decision-making.

[0300] 1. Global threshold calculate:

[0301] in, The baseline threshold (preferably 0.6) is determined based on the training set; The average comprehensive hallucination score for the current sequence; The standard deviation of the overall hallucination score for the current sequence; To control sensitivity, a value of 1.0-1.5 is preferred; A baseline threshold set by the individual (preferably 0.5).

[0302] 2. Local threshold adjustment: Considering the influence of context, different thresholds are applied to different types of text regions: a. Entity reference area: Lowering the threshold has a significant impact on entity errors.

[0303] (Within the physical area) b. High-certainty region: Increase the threshold so that the model is confident that it may be correct even if the score is high.

[0304]

[0305] in, For time steps The dynamic confidence threshold; This represents the top-1 probability.

[0306] c. Low evidence coverage area: Raise the threshold to avoid misjudgment due to insufficient evidence.

[0307]

[0308] in, For time steps Coverage perception threshold; To account for overall uncertainty.

[0309] Comprehensive local threshold:

[0310] in, For time steps The final local threshold used in practice; all adjustments are limited to the range of [-0.15, 0.15].

[0311] 3. For each time step Make a preliminary decision:

[0312] in, For time steps Decision-making labels.

[0313] 4. Context consistency check: Hallucinations are usually continuous within a context, while isolated points may be noise.

[0314] Filtering out-of-place points: For each Check its neighborhood , The minimum length of the illusion segment. If within the neighborhood If all are 0, then let .

[0315] For the gap between two hallucination segments, if the length is less than (Preferably 3) and the average fraction within the gap is greater than If so, then it is filled with illusion.

[0316] 5. Final Decision Sequence :

[0317] V. Hallucination Segment Extraction and Confidence Assessment Consecutive hallucination points are merged into hallucination segments, and the confidence and type of each hallucination segment are evaluated.

[0318] 1. Merging of continuous segments: Scan the final decision sequence and merge segments that are consecutively 1 ( ):

[0319] in, The set of time periods during which hallucinations were detected; For the first The start and end timestamps of a detected series of consecutive abnormal segments.

[0320] Length less than The segment may be noise and can be filtered or marked as low confidence.

[0321] 2. Segment confidence calculation: For each hallucination segment Calculate the overall confidence level separately:

[0322] in, For the first The average hallucination score of the segment; For the first Intra-segment consistency.

[0323] a. Average hallucination score:

[0324] in, For the first The average hallucination score of the segment; For the first The starting time step of each hallucination segment; For the first The end time step of each illusion segment; In time step The final comprehensive abnormal score.

[0325] b. Strength of internal evidence:

[0326] in, For the first The strength of internal evidence within the paragraph; For time steps The internally corrected score.

[0327] c. Strength of external evidence:

[0328] in, For the first The strength of external evidence for the segment; For time steps The external feature score.

[0329] d. Intra-segment consistency:

[0330] in, For the first Intra-segment consistency.

[0331] 3. Classification of hallucination types: Classification based on the proportion of contribution from internal and external evidence:

[0332] in, For the first The proportion of different types of hallucination fragments; For the first The strength of internal evidence within the paragraph; For the first Strength of external evidence for the segment.

[0333] when When the hallucination type is determined to be internally dominant, the model's cognitive abnormalities are observed, but external evidence may be insufficient, such as rare but correct knowledge, leading to model uncertainty; when When the hallucination type is determined to be externally dominant, the model confidently generates it, but this contradicts reality, for example, the model learns incorrect knowledge; when When the hallucination is judged to be of a mixed type, both internal and external evidence are strong, such as: obvious hallucination, model of cognitive confusion and inconsistent with the facts.

[0334] 4. Severity assessment of hallucinations: The severity score is calculated using the following formula:

[0335] in, For the first A comprehensive severity score for each hallucination segment; The proportion of entity words in the hallucination segment; For position weighting functions; It is uncorrectable; These are the weighting coefficients corresponding to the proportions of the entities. These are the weight coefficients for the corresponding position weights; These are the weighting coefficients corresponding to the uncorrectability.

[0336] The final result is: List of hallucination segments:

[0337] Illusion score sequence:

[0338] Decision sequence:

[0339] VI. Generation of Interpretability Reports Generate an interpretable detection report for each hallucination segment, explaining the basis of the detection and the source of evidence.

[0340] This step completes the core decision-making process for hallucination detection, organically combining internal cognitive analysis with external fact-checking to achieve high-precision, interpretable hallucination localization, providing clear target areas and guidance information for subsequent corrective steps.

[0341] Next, proceed to step S150.

[0342] In step S150, the final illusion point is determined based on the comprehensive illusion score, and the final illusion point is corrected.

[0343] In this embodiment of the invention, the method further includes: merging consecutive potential hallucination points into hallucination segments; calculating the overall confidence level of each hallucination segment; classifying the hallucination segments based on the contribution ratio of internal evidence and external evidence to obtain hallucination types; the hallucination types include: internally dominant, externally dominant, and mixed types.

[0344] In this embodiment of the invention, determining the final hallucination point based on the comprehensive hallucination score and correcting the final hallucination point includes: acquiring hallucination data and constraint types; the hallucination data includes: the original content of the hallucination segment and the content of the context of the hallucination segment; the constraint types include: semantic constraints, external evidence constraints, and hallucination type constraints; generating a set of corrected constraints based on the hallucination data and constraint types; generating correction candidates based on the set of corrected constraints; calculating the evaluation index for each correction candidate and weighting the evaluation index to obtain a weighted comprehensive score; wherein, the evaluation index includes: factual accuracy score, grammatical consistency score, style consistency score, contextual coherence score, and information retention score; and selecting correction candidates based on the weighted comprehensive score.

[0345] More specifically, based on the hallucination segments located through the above steps, the causes of hallucination and contextual constraints are analyzed to achieve accurate, fluent, and factually correct corrections, maintaining the original style and coherence, and improving the reliability of the generated text.

[0346] I. Revise Context Analysis and Constraint Definitions Analyze the context surrounding the hallucination segment and define the modified multi-dimensional constraints.

[0347] 1. Extract the preceding and following context windows (preferably 20 lexical units) and the original content of the hallucination segment for each hallucination segment.

[0348] 2. Semantic constraint analysis: Grammatical role analysis: Based on the grammatical function of the illusion segment in the sentence (subject, predicate, object, attributive, etc.), a dependency parser is used to identify grammatical relations and constrain the modified content to maintain the same grammatical role.

[0349] Semantic relation constraints: Analyze the semantic relations (cause and effect, temporal sequence, modification, etc.) between the hallucination segment and the preceding and following segments, and maintain the same semantic relations after correction.

[0350] Style consistency constraint: Analyze the contextual text style (formal / colloquial, academic / popular, tense, voice, etc.) and constrain the content style correction to match.

[0351] 3. Extract relevant facts from external evidence, including: Set of correct facts: , of which each These are correct facts that have high support.

[0352] List of related entities: (Entities related to the hallucination segment).

[0353] Collection of Forbidden Facts: (False facts that have been proven false should be avoided during the correction process).

[0354] 4. Define correction priorities based on hallucination type, including: a. Factual error type (externally dominated type): Replacing incorrect facts with correct facts.

[0355] Constraints: Factual accuracy > Stylistic consistency > Fluency.

[0356] b. Ambiguous expression type (internal-dominated, insufficient external evidence): Express in a more accurate and unambiguous way.

[0357] Constraints: Clarity > Factuality (preferably no explicit facts) > Stylistic consistency.

[0358] c. Logical contradiction type (hybrid type): Correcting logical inconsistencies.

[0359] Constraints: Logical consistency > Factual accuracy > Coherence.

[0360] 5. Formalizing constraints, including: Encode the constraints as weight vectors:

[0361] in, In order to target the A hallucination segment The corrected constraint weight vector; The weighting coefficient for the accuracy of the facts; This is the weighting coefficient for syntactic consistency; This represents the weighting coefficient for style consistency. This is a weighting factor for contextual coherence; The weighting coefficients represent logical consistency; the sum of all weights is 1, and they are adaptively adjusted according to the type of hallucination.

[0362] The final output is for each illusion segment. The set of modified constraints:

[0363] in, In order to target the A hallucination segment The set of modified constraints; hallucination segment The preceding text; hallucination segment The following text; For the first The original text content of the illusion segment; For the hallucination segment The relevant set of correct facts; For the hallucination segment List of relevant external evidence entities; For the hallucination segment The relevant set of erroneous facts; In order to target the A hallucination segment The corrected constraint weight vector.

[0364] II. Revise the content generation strategy Based on the constraints, different strategies are used to generate correction candidates.

[0365] 1. A generation strategy based on hallucination segment features, including: a. Fact substitution strategy (suitable for short passages and explicit factual errors): when and At that time, from Select the most relevant fact and replace it directly.

[0366] b. Local regeneration strategy (default strategy): Regenerate illusion segments based on context and constraints.

[0367] c. Extended regeneration strategy (suitable for high context dependencies): When the illusion segment is highly intertwined with the context, or the grammatical structure is complex, regenerate a longer segment that includes context extensions.

[0368] d. Obfuscation strategy (suitable for cases with insufficient external evidence but high internal uncertainty): when and When internal factors dominate, replace erroneous, deterministic statements with more conservative, probabilistic ones.

[0369] 2. Project notifications, including: a. Partial Regeneration Prompt Template:

[0370] in, For the first A hallucination segment The constructed local regeneration prompt text; hallucination segment The preceding text; These are placeholder markers used to specify the locations where the model needs to be filled in or regenerated; hallucination segment The following text; For the hallucination segment The relevant set of correct facts; This is a string concatenation operator.

[0371] b. Expand the regeneration prompt template:

[0372] in, For the first A hallucination segment The constructed extended regeneration prompt text; hallucination segment The expanded preceding text; hallucination segment The expanded text that follows; These are placeholder markers used to specify the locations where the model needs to be rewritten.

[0373] c. Specify the constraints in the prompt, and generate the code that must satisfy: (1) Must include facts: [list of facts].

[0374] (2) Must avoid: [List of incorrect facts].

[0375] (3) Grammatical roles: [subject / predicate, etc.].

[0376] (4) Style: [Style description].

[0377] 3. Multiple candidate generation: Use the original model Or a dedicated correction model For each strategy, generate Candidates (preferred) ):

[0378] in, For the first The candidate set generated by each generation strategy; The number of candidate words (or candidate fragments) generated for each strategy; For the first The first strategy 10 candidate words.

[0379] 4. Decoding Factual Constraints: Imposing factual constraints during the decoding process: For each generation time step t′, calculate the constraint enhancement probability of candidate lexical v:

[0380] in, To enhance post-prediction words The probability of; The original predicted words of the language model The probability of; The factual consistency score is used to measure the semantic units. The degree of matching with correct facts.

[0381] Reunification: Entity consistency constraints: Ensure that the entities in the generated content are consistent with... To maintain consistency, if the same entity is mentioned, the same name / reference must be used.

[0382] III. Candidate Evaluation and Selection Select the optimal candidate from multiple modified candidates to ensure that all constraints are met.

[0383] 1. Calculation of evaluation indicators: For each candidate Calculate the following indicators: a. Factual accuracy score: Consistency with external evidence:

[0384] in, For the first Factual consistency score of each candidate segment; The set of correct facts retrieved; For natural language reasoning scores; The sum of the NLI scores for the candidate fragments and all correct facts.

[0385] Degree of conflict with erroneous facts:

[0386] in, For the first The degree of conflict among candidate segments; The set of known erroneous facts; For natural language reasoning scores; The maximum NLI score for the candidate fragment and all erroneous facts.

[0387] The conflict level should be close to 0 after the correction.

[0388] Overall fact score :

[0389] b. Syntactic consistency score: Syntax role matching:

[0390] in, Syntactic correctness is measured using syntactic tree similarity.

[0391] in, c. Style consistency score: Extracting style feature vectors Calculate the cosine similarity:

[0392] in, Score for style consistency; To extract the style vector of the text; This is the context vector generated before the fragment is generated.

[0393] d. Contextual coherence score: Probabilities of the language model before and after:

[0394] in, Score for overall consistency; The probability function of the main language model; The current original input text; This is the context vector generated after the fragment is created.

[0395] When calculating the sequence probability using model M, the difference should be greater than 0.

[0396] Sentence encoders are used to calculate semantic fluency.

[0397] e. Information retention score: Degree of retention of non-hallucinatory information:

[0398] in, For non-hallucination information retention; It represents the intersection of semantic information.

[0399] 2. Weighted composite score:

[0400] in, This is the final weighted total score; These are the weighting coefficients.

[0401] Normalize to [0,1]:

[0402] 3. Diversity Guarantee: To avoid selecting overly similar candidates, calculate the similarity matrix between candidates:

[0403] If the similarity between the first two candidates is greater than 0.9, consider selecting a third candidate to ensure a diverse perspective.

[0404] 4. Final choice: Selection criteria:

[0405] in, (Preferably 0.7) If no candidate meets the threshold, an alternative strategy (e.g., fuzzing) is triggered.

[0406] Output: The optimal correction candidate for each hallucination segment .

[0407] IV. Context Integration and Fluency Optimization The revised candidates were integrated into the original text to ensure overall fluency and coherence.

[0408] Boundary smoothing: Check the boundary fluency of the corrected candidates and context: a. Adjustment of conjunctions: if Ending with a specific conjunction, and If a conjunction begins with a mismatch, adjust the conjunction. For example, if the preceding text ends with "because," correct it to begin with "therefore" instead of "and."

[0409] b. Reference consistency check: Check whether the pronouns in the correction candidates are consistent with the antecedents in the context, and replace the pronouns with explicit references if necessary.

[0410] c. Tense and voice consistency: Ensure that the tense and voice of the correction candidate are consistent with the context.

[0411] 2. Transition segment generation: When revisions result in significant content changes, use a small generative model to generate transition sentences / words for a natural transition. For example, the original illusion "Therefore the Earth is flat" could be revised to "Therefore the Earth is approximately sphere," with the transition "But modern science has proven..." added. 3. Overall coherence assessment: The perplexity is calculated by inputting the corrected full text into the model:

[0412] in, The perplexity of the corrected text; To compute the complete text after replacing the hallucination content using language model M The level of confusion; This is the complete text after replacing the illusion section.

[0413] Comparison of perplexity with the original text:

[0414] in, The difference in perplexity; The perplexity of the original text.

[0415] expect (Smoother after correction).

[0416] 4. Local readjustment: If the overall perplexity increases, perform local optimization on the boundary region: expand the boundary by a few tokens and regenerate it using constraint generation; keep the core correction content unchanged and adjust the connection part.

[0417] 5. Format consistency check: Check punctuation, capitalization, number formatting, etc., for consistency with the context; ensure consistency in list and quotation formatting; output the corrected complete text. .

[0418] V. Iterative Correction and Verification Verify the revised text and iterate as needed to ensure the quality of the correction.

[0419] 1. Correct the verification loop: a. to Rerun the hallucination detection, focusing on whether the hallucination still exists in the corrected section itself, whether the correction introduces new inconsistencies, and whether the context creates new problems due to the correction.

[0420] b. Validation metrics: Changes in the number of hallucination segments: ,in, The total number of hallucination segments detected in the original text; This represents the total number of hallucination segments detected in the corrected text.

[0421] Changes in average hallucination scores: ,in, The average illusion score of the original text; The average illusion score for the corrected text.

[0422] Verification with external evidence can improve the accuracy of facts.

[0423] c. Iterative decision-making: If new hallucinations are detected (especially in the revised segment): (1) Analyze the reasons: Is the correction incomplete, or has the correction introduced new problems? (2) Adjust or modify the strategy.

[0424] (3) Regenerate corrections.

[0425] (4) Set the maximum number of iterations (Preferably 3 times) to avoid infinite loops.

[0426] (5) Convergence condition: two consecutive iterations and .

[0427] 2. Coordination of multiple hallucination segments: When the original text contains multiple hallucination paragraphs, the revisions may affect each other: a. Sequential Correction: Correct each element sequentially according to its position. After each correction, update the entire text before processing the next one.

[0428] b. Coordination after parallel corrections: If contradictions arise after multiple illusion segments are corrected, they should be coordinated. For example, if the first illusion is corrected to "1821", the second one, which mentions "two years later", should be adjusted accordingly.

[0429] c. Global consistency check: Check the internal consistency of all modified sections to ensure global consistency in time, logic, and reference.

[0430] 3. Revise the quality assessment: Final evaluation indicators: a. Accuracy of facts:

[0431] b. Smoothness retention:

[0432] A value close to 1 indicates good fluency. If the perplexity remains almost unchanged before and after the correction, it means that the model has maintained the original language fluency very well while eliminating illusions. The article does not read like a machine translation or be awkward due to the abrupt replacement.

[0433] c. Information retention:

[0434] Information is extracted and measured through key entities and relationships.

[0435] Final output corrected text .

[0436] This step completes the entire closed loop from hallucination detection to correction. Through multi-strategy generation, multi-dimensional evaluation, and iterative optimization, it achieves high-quality, interpretable text correction, significantly improving the reliability and usability of content generated by the large language model.

[0437] Figure 3 The present invention illustrates a large-scale hallucination detection and correction system 300, which includes a data acquisition module 310, a potential hallucination point module 320, a support module 330, a fusion module 340, and a correction module 350.

[0438] The data acquisition module 310 is used to extract a quantitative representation of the internal cognitive state of each time step of the language model generating response text, and to generate a cognitive trajectory sequence based on the quantitative representation; The potential hallucination point module 320 is used to calculate the internal inconsistency score based on the cognitive trajectory sequence to obtain potential hallucination points; The support module 330 is used to acquire multi-source external data for the potential illusion points and the entire generated text, and to calculate the external data support based on the multi-source external data; The fusion module 340 is used to fuse the internal inconsistency score with the external data support to obtain a comprehensive illusion score; The correction module 350 is used to determine the final illusion point based on the comprehensive illusion score and to correct the final illusion point.

[0439] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0440] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0441] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0442] It should be understood that the various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof.

[0443] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for detecting and correcting large-scale hallucinations, characterized in that, The method includes the following steps: A quantitative representation of the internal cognitive state at each time step of the response text generated by the language model is extracted, and a cognitive trajectory sequence is generated based on the quantitative representation; Based on the cognitive trajectory sequence, calculate the internal inconsistency score to obtain potential hallucination points; For the potential hallucination points and the entire generated text, multi-source external data is obtained, and the external data support is calculated based on the multi-source external data; The internal inconsistency score is fused with the external data support to obtain the comprehensive illusion score; The final hallucination point is determined based on the comprehensive hallucination score, and then the final hallucination point is corrected.

2. The method for detecting and correcting large-scale hallucinations according to claim 1, characterized in that, The step of extracting a quantitative representation of the internal cognitive state at each time step of the response text generated by the language model, and generating a cognitive trajectory sequence based on the quantitative representation, includes: A quantitative representation of the internal cognitive state at each time step of the language model generating the response text is extracted; the quantitative representation of the internal cognitive state includes: the attention weight matrix, the hidden state vector of the decoder layer, and the probability distribution on the vocabulary; The average attention entropy and attention concentration ratio are obtained based on the attention weight matrix. Based on the hidden state vector, extract the vector and activation sparsity after dimensionality reduction by principal component analysis; Extract normalized entropy and top-1 probability based on the probability distribution on the vocabulary; The average attention entropy, attention concentration ratio, vector after dimensionality reduction by principal component analysis, activation sparsity, normalized entropy, and top-1 probability are concatenated as features and mapped to generate a cognitive trajectory sequence through a neural network encoder.

3. The method for detecting and correcting large-scale hallucinations according to claim 1, characterized in that, The calculation of the internal inconsistency score based on the cognitive trajectory sequence includes: Analyze the abnormal patterns of the cognitive trajectory sequence; wherein, the abnormal patterns include temporal abruptness features, attentional distraction features, confidence abnormality features, and hidden state abnormality features; The temporal abruptness characteristic is calculated based on the following formula: ； in, This is a temporal abrupt change characteristic. ; In time step The original temporal abrupt change degree; For all time steps; The attention distractibility feature is calculated based on the following formula: ； in, For the feature of attentional distractibility, ; The slope parameter of the Sigmoid function; In time step The relative dispersion; For time step index; The confidence level anomaly feature is calculated based on the following formula: ； in, For confidence level anomaly features, ; This is an indicator of uncertainty anomalies; Weights for uncertainty anomaly indicators; For decision-making fuzziness indicators; Weights for indicators of decision-making fuzziness; This is an indicator of distribution offset anomalies; Weights for distribution offset anomaly indicators; ; The hidden state anomaly feature is calculated based on the following formula: ； in, To hide the anomaly degree feature of the state, ; This is a normalized sparsity anomaly. Weights for normalized sparsity anomalies; The normalized Mahalanobis distance anomaly; The weights of the normalized Mahalanobis distance anomalies; The normalized hidden state change; The Euclidean distance between the current hidden state vector and the hidden state vector at the previous time step; The mean of the entire current sequence; This represents the standard deviation over the entire current sequence. The weights of the normalized hidden state changes; ; The internal inconsistency score is calculated based on the following formula: ； in, For internally inconsistent scores, ; For time step index; For index variables, At that time, the corresponding temporal abrupt change index, , At that time, the corresponding attention distraction index, , At that time, the corresponding confidence level outlier index, , At that time, the corresponding hidden state anomaly index, ; For the first The fusion weights of the anomalous indicators satisfy the following: .

4. The method for detecting and correcting large-scale hallucinations according to claim 1, characterized in that, The acquisition of potential hallucination points includes: For each time step A preliminary determination is made based on the following formula: ； in, For time steps Internal hallucination indicator function; Internal inconsistency score; This is the internal inconsistency threshold; when Time step Preliminary assessment identifies it as a potential hallucination point. Time step The point was initially determined to be non-hallucination.

5. The method for detecting and correcting large-scale hallucinations according to claim 1, characterized in that, The process of acquiring multi-source external data for the potential hallucination points and the entire generated text, and calculating external data support based on the multi-source external data, includes: External data support is calculated using the following formula: ； in, External data support; The weight contributed to the entity; The weight of contribution to the relationship; Weighting of the claimed contribution; In time step The support score contributed by physical evidence; In time step The support score contributed by the relationship evidence; In time step The score is determined by the support score of the evidence contributing to the claim. Entity contribution is calculated based on the following formula: ； Among them, is the support score contributed by entity evidence at time step ; is the total number of entities identified in the entire generated text; is an index variable; is an element in the time step - entity association matrix. When , it means that the token generated at time step is within the text range of entity . When , it means that time step has nothing to do with entity ; is the external data support degree of entity ; is the entity weight coefficient; is a small positive number to prevent division by zero; The relationship contribution is calculated based on the following formula: ； Among them, is the support score contributed by relational evidence at time step . is the total number of relational triples extracted from the entire generated text; is the relational index; is an element in the time step-relational association matrix. When , it means that the token generated at time step is within the text range of the subject, predicate, or object of the relational triple . When , it means that time step has nothing to do with the relational triple ; is the external data support degree of the relational triple ; is the relational weight coefficient; is a small positive number to prevent division by zero; The claim contribution is calculated based on the following formula: ； where is the support score contributed by the claimed evidence at time step ; is the total number of claims segmented from the entire generated text; is the claim index; is an element in the time step-claim association matrix. When , it means that the token generated at time step is within the text range of claim . When , it means that time step is not within the text range of claim ; is the external data support degree of the entire claim ; is the importance score of claim ; is a small positive number to prevent division by zero.

6. The method for detecting and correcting large-scale hallucinations according to claim 1, characterized in that, The process of fusing the internal inconsistency score with the external data support to obtain a comprehensive illusion score includes: The overall hallucination score is calculated based on the following formula: ； in, To calculate the overall illusion score, ; In time step The combined fusion score; Calculate the time step based on the following formula. Combination and fusion scores : ； in, In time step The combined fusion score; In time step The posterior probability; In time step The score calculated by the linear weighted fusion strategy; In time step The score calculated by the product fusion strategy; In time step The score calculated by the maximum evidence fusion strategy; The weights are the posterior probabilities; The score weights are calculated for the linear weighted fusion strategy; The weights of the scores calculated for the product fusion strategy; The weights of the scores calculated for the maximum evidence fusion strategy.

7. The method for detecting and correcting large-scale hallucinations according to claim 1, characterized in that, The method further includes: Merge consecutive potential hallucination points into hallucination segments; Calculate the overall confidence level for each hallucination segment; The hallucination segments are classified based on the contribution ratio of internal and external evidence to obtain hallucination types; the hallucination types include: internal-dominant, external-dominant, and mixed types.

8. The method for detecting and correcting large-scale hallucinations according to claim 7, characterized in that, The process of determining the final hallucination point based on the comprehensive hallucination score and then revising the final hallucination point includes: Obtain hallucination data and constraint types; the hallucination data includes: the original content of the hallucination segment, and the content of the context of the hallucination segment; the constraint types include: semantic constraints, external evidence constraints, and hallucination type constraints; Generate a modified constraint set based on illusion data and constraint type; Based on the set of modified constraints, modification candidates are generated; The evaluation metrics for each candidate correction are calculated separately, and the evaluation metrics are weighted to obtain a weighted comprehensive score; wherein, the evaluation metrics include: factual accuracy score, grammatical consistency score, style consistency score, contextual coherence score, and information retention score; Candidates for revision are selected based on the weighted comprehensive score.

9. A system for detecting and correcting large-scale illusions, characterized in that, The system includes: The data acquisition module is configured to extract a quantitative representation of the internal cognitive state at each time step of the language model generating the response text, and to generate a cognitive trajectory sequence based on the quantitative representation; The potential hallucination point module is configured to calculate an internal inconsistency score based on the cognitive trajectory sequence to obtain potential hallucination points; The support module is configured to acquire multi-source external data for the potential illusion points and the entire generated text, and to calculate the external data support based on the multi-source external data; A fusion module is configured to fuse the internal inconsistency score with the external data support to obtain a comprehensive illusion score; The correction module is configured to determine the final illusion point based on the comprehensive illusion score and to correct the final illusion point.

10. A computer program product, characterized in that, The computer program product includes a computing program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the method for detecting and correcting large model illusions as described in any one of claims 1 to 8.

Images