Large language model hallucination detection method and detection system based on dynamic entropy change trajectory

Abstract

This application discloses a method and system for hallucination detection based on a large language model with dynamic entropy change trajectory. The method determines the instantaneous entropy value of the current generation step by analyzing the vocabulary probability distribution output by the current generation step during autoregressive generation of the input prompt text. It then arranges the instantaneous entropy values ​​from the first generation step to the current generation step in chronological order to obtain a temporal entropy change trajectory. A dynamic entropy change trajectory is obtained by extracting multiple consecutive instantaneous entropy values ​​adjacent to the current generation step. A semantic consistency score is determined based on the difference between the generated text sequence and the semantic vector, and a matching score is determined between the dynamic entropy change trajectory and a pre-built hallucination precursor pattern library. An initial hallucination risk value is obtained based on the semantic consistency score and the matching score. The initial hallucination risk value is calibrated according to the semantic complexity parameter to obtain the hallucination risk probability, thereby generating the hallucination detection result for the current generation step. This method enables real-time hallucination detection during the generation process.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and natural language processing technology, and in particular to a method and system for detecting hallucinations based on a large language model with dynamic entropy change trajectory. Background Technology

[0002] Large Language Models (LLMs), with their autoregressive generative paradigm, have been widely applied in scenarios such as intelligent question answering, text creation, and professional content generation. However, when there is no factual basis or the logic is flawed, the model may still output erroneous, fabricated, or contradictory content with high confidence, i.e., the "illusion" problem, which seriously restricts its large-scale implementation in high-credibility scenarios such as healthcare, law, finance, and government affairs.

[0003] Existing hallucination detection techniques mostly rely on static probability features from model outputs for screening, typically such as calculating the Shannon entropy or perplexity of a single token and judging the uncertainty of generated content based on a fixed threshold. These methods only calculate instantaneous entropy values ​​for the probability distribution of each token, breaking down the autoregressive generation process into isolated static data points, failing to fully utilize the evolutionary information of uncertainty over time. Since the generation process is serialized token by token, the emergence of hallucinations often manifests in the continuous changes in entropy values, and single-step static detection cannot capture this temporal dynamic characteristic, resulting in a lack of ability to identify abnormal precursors in the generation process. Summary of the Invention

[0004] The main purpose of this application is to propose a method and system for hallucination detection based on a large language model with dynamic entropy change trajectory, which can realize real-time detection of hallucinations during the generation process.

[0005] The first aspect of this application provides a hallucination detection method based on a large language model with dynamic entropy-varying trajectories, the method comprising: During the autoregressive generation process of the large language model to be detected based on the input prompt text, the vocabulary probability distribution output by the current generation step, the sequence of texts generated before the current generation step, the semantic vector of the input prompt text, and the semantic complexity parameters of the generation task corresponding to the input prompt text are obtained. The instantaneous entropy value of the current generation step is determined based on the vocabulary probability distribution. The instantaneous entropy values ​​from the first generation step to the current generation step are arranged in the order of generation of the corresponding generation steps to obtain the temporal entropy change trajectory; The dynamic entropy change trajectory is obtained by extracting multiple consecutive instantaneous entropy values ​​adjacent to the current generation step from the temporal entropy change trajectory. A semantic consistency score is determined based on the difference between the previously generated text sequence and the semantic vector before the current generation step. Determine the matching score between the dynamic entropy change trajectory and a pre-constructed library of hallucination precursor patterns; Based on the semantic consistency score and the matching score, an initial hallucination risk value is obtained; The initial hallucination risk value is calibrated based on the semantic complexity parameter to obtain the hallucination risk probability; Based on the hallucination risk probability, generate the hallucination detection result of the large language model to be detected in the current generation step.

[0006] The first aspect of this application provides a hallucination detection method based on a large language model with dynamic entropy change trajectory. This method determines the instantaneous entropy value of the current generation step by analyzing the vocabulary probability distribution output during the autoregressive generation process based on the input prompt text. The instantaneous entropy values ​​from the first generation step to the current step are arranged in chronological order to obtain a temporal entropy change trajectory. Multiple consecutive instantaneous entropy values ​​adjacent to the current generation step are extracted to obtain a dynamic entropy change trajectory. A semantic consistency score is determined based on the difference between the generated text sequence and the semantic vector, and a matching score is determined between the dynamic entropy change trajectory and a pre-constructed hallucination precursor pattern library. An initial hallucination risk value is obtained based on the semantic consistency score and the matching score. The initial hallucination risk value is calibrated according to a semantic complexity parameter to obtain a hallucination risk probability, thereby generating the hallucination detection result for the current generation step. This method enables real-time hallucination detection during the generation process.

[0007] To achieve the above objectives, a second aspect of the present invention provides a large language model hallucination detection system based on dynamic entropy-varying trajectories, the system comprising: The acquisition module is used to acquire the vocabulary probability distribution output by the current generation step, the sequence of texts generated before the current generation step, the semantic vector of the input prompt text, and the semantic complexity parameter of the generation task corresponding to the input prompt text during the autoregressive generation process of the large language model to be detected based on the input prompt text. The first module is used to determine the instantaneous entropy value of the current generation step based on the vocabulary probability distribution; The second module is used to arrange the instantaneous entropy values ​​from the first generation step to the current generation step in the order of generation of the corresponding generation steps to obtain the temporal entropy change trajectory. The third module is used to extract multiple consecutive instantaneous entropy values ​​adjacent to the current generation step from the temporal entropy change trajectory to obtain the dynamic entropy change trajectory; The fourth module is used to determine the semantic consistency score based on the difference between the generated text sequence before the current generation step and the semantic vector; The fifth module is used to determine the matching score between the dynamic entropy change trajectory and the pre-built hallucination precursor pattern library; The sixth module is used to obtain an initial hallucination risk value based on the semantic consistency score and the matching score; The seventh module is used to calibrate the initial hallucination risk value according to the semantic complexity parameter to obtain the hallucination risk probability; The eighth module is used to generate the hallucination detection result of the large language model to be detected in the current generation step based on the hallucination risk probability.

[0008] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: at least one processor and a memory for communicatively connecting to the at least one processor; the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the above-described method for detecting hallucinations based on a large language model with dynamic entropy change trajectory.

[0009] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described hallucination detection method based on a large language model with dynamic entropy change trajectory.

[0010] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0011] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating a large language model-based hallucination detection method based on dynamic entropy trajectory provided in this application; Figure 2 This is a schematic diagram of the structure of a large language model hallucination detection system based on dynamic entropy trajectory provided in this application; Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in this application. Detailed Implementation

[0012] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0013] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0014] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0015] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0016] Based on this, the embodiments of this application provide a method and system for detecting hallucinations based on a large language model with dynamic entropy change trajectory. The specific implementation is described in the following embodiments. First, the method for detecting hallucinations based on a large language model with dynamic entropy change trajectory in the embodiments of this application is described.

[0017] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0018] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0019] The large language model illusion detection method based on dynamic entropy change trajectory provided in this application relates to the fields of artificial intelligence and natural language processing technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the large language model illusion detection method based on dynamic entropy change trajectory, etc., but is not limited to the above forms.

[0020] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0021] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0022] Therefore, referring to Figure 1This application provides a method for hallucination detection based on a large language model with dynamic entropy change trajectory. This method is applied to a central controller, which can be a server, an electronic device, or a mobile terminal, etc. There are no specific limitations here. The method includes the following steps S110 to S190.

[0023] Step S110: Obtain the vocabulary probability distribution output by the current generation step, the generated text sequence before the current generation step, the semantic vector of the input prompt text, and the semantic complexity parameters of the generation task corresponding to the input prompt text during the autoregressive generation process of the large language model to be detected based on the input prompt text. Step S120: Determine the instantaneous entropy value of the current generation step based on the vocabulary probability distribution; Step S130: Arrange the instantaneous entropy values ​​from the first generation step to the current generation step in the order of generation of the corresponding generation steps to obtain the temporal entropy change trajectory; Step S140: Extract multiple consecutive instantaneous entropy values ​​adjacent to the current generation step from the temporal entropy change trajectory to obtain the dynamic entropy change trajectory; Step S150: Determine the semantic consistency score based on the difference between the generated text sequence and the semantic vector before the current generation step; Step S160: Determine the matching score between the dynamic entropy change trajectory and the pre-built hallucination precursor pattern library; Step S170: Based on the semantic consistency score and the matching score, obtain the initial hallucination risk value; Step S180: Calibrate the initial hallucination risk value according to the semantic complexity parameter to obtain the hallucination risk probability; Step S190: Based on the hallucination risk probability, generate the hallucination detection result of the large language model to be detected in the current generation step.

[0024] In this step, four types of basic input data are obtained during the autoregressive generation process of the large language model to be detected based on the input prompt text: the vocabulary probability distribution output by the current generation step, the sequence of texts generated before the current generation step, the semantic vector of the input prompt text, and the semantic complexity parameters of the generation task corresponding to the input prompt text.

[0025] The acquisition of the four types of basic input data mentioned above can be used to collect and cache a unified input benchmark required for subsequent steps in real time during the word-by-word autoregressive generation process of a large language model. The four types of data are: first, the vocabulary probability distribution of the current generation step, denoted as data A; second, the text sequence generated before the current generation step, denoted as data B; third, the fixed semantic anchor vector corresponding to the input prompt text, denoted as data C; and fourth, the semantic complexity parameter set of the current generation task, denoted as data D.

[0026] The above vocabulary probability distribution can be the probability distribution of the large language model to be detected in the first... The full vocabulary size generated by autoregression and normalized (e.g., softmax) is... The probability distribution on is denoted as ,in This represents the total size of the model vocabulary. For the first Step 1 The probability of generating each word is the data that is updated in real time as the model generates word by word. Specifically, the word probability distribution reflects the model's uncertainty judgment about the next word in the current generation step and is the sole input source for subsequent entropy calculations.

[0027] The above-generated text sequence can be the model up to the [number]th [section / section]. The complete text content that has been generated in the previous step is denoted as: It is used to correlate the semantics of the original input and to complete the consistency verification of the generated content. Specifically, the generated text sequence is the accumulation of the actual output results of the model and is the direct basis for judging whether the generated content deviates from the constraints of the input facts.

[0028] The aforementioned semantic vector can be obtained by encoding the original input prompt text using a unified semantic coding model. A fixed-dimensional vector serves as the core alignment benchmark for determining whether generated content deviates from factual constraints, remaining constant throughout a single generation task. Specifically, the semantic vector solidifies the semantics and factual constraints of the input prompt text into a stable representation, enabling subsequent generated content to be quantitatively compared with it.

[0029] The aforementioned semantic complexity parameters can be a set of prior parameters describing the difficulty of the current generation task, including three types of parameters: domain rarity, cumulative reasoning steps, and factual requirement level. These parameters are used for adaptive dynamic calibration of the subsequent hallucination judgment threshold. Specifically, domain rarity characterizes the degree of professional scarcity in the domain to which the task belongs, cumulative reasoning steps characterize the length of the reasoning chain, and factual requirement level characterizes the severity of the consequences of output errors.

[0030] The above method of determining the instantaneous entropy value based on the vocabulary probability distribution can be based on the first... Probability distribution of step word list As the only input, the Shannon entropy formula is used to quantify the generation uncertainty of the model step by step in this generation step, so as to obtain the instantaneous entropy value of the current generation step. Specifically, instantaneous entropy measures the dispersion of the model's output probability distribution at a single generation step. A lower entropy value indicates a more certain model, while a higher entropy value indicates a more uncertain model.

[0031] ; in, The instantaneous entropy value of the current generation step. This represents the total size of the model vocabulary. For the first Step 1 The generation probability of each word.

[0032] In numerical calculations, it is agreed that when hour, This is to avoid numerical anomalies caused by the meaninglessness of logarithms.

[0033] The aforementioned temporal entropy trajectory can be derived from the model's first step to the second step. All instantaneous entropy values ​​of the step A continuous vector formed by concatenating vectors in the order they are generated is denoted as a trajectory vector. Specifically, the temporal entropy change trajectory solidifies the originally isolated stepwise entropy value into a curve that evolves over time, thus preserving the complete evolutionary information of the uncertainty in the generation process.

[0034] The aforementioned dynamic entropy change trajectory can be represented by a fixed-length sliding window. Extract the complete temporal entropy change trajectory and obtain the nearest neighbor to the current generation step. The step window entropy change subsequence is denoted as E1. Specifically, the sliding window allows the model to focus only on the entropy change evolution within the most recent time period, which preserves the temporal dynamics characteristics while avoiding interference from earlier irrelevant steps.

[0035] It should be noted that, in order to fully characterize the evolution of uncertainty in model generation, after obtaining the dynamic entropy-change trajectory E1, third-order entropy-change dynamic features can be extracted layer by layer from three dimensions: static amplitude, dynamic rate of change, and acceleration of change, to obtain an eight-dimensional feature vector E2, which serves as an enhanced input for subsequent precursor pattern matching and joint evaluation. The specific calculation methods for each order of features are given below.

[0036] The first order is the instantaneous entropy statistical feature, which calculates three indicators based on the entropy value sequence within the window: the average entropy of the window, the standard deviation of the window entropy, and the deviation of the maximum entropy of the window, to characterize the static amplitude of the uncertainty of single-step generation: Window average entropy: ; Window entropy standard deviation: ; Maximum entropy deviation of the window: ( (the maximum entropy value within the window). in, For window length, For the first in the window The instantaneous entropy value of the step, This represents the average entropy of the window.

[0037] The second-order characteristic is the entropy rate of change (first-order difference). The entropy rate of change sequence is obtained by performing a first-order difference on the entropy value sequence within the window. Further calculations are made of three indices: average entropy rate of change, maximum positive entropy rate of change, and entropy rate variance, characterizing the rate of change and fluctuation of entropy values ​​during the generation process. Average entropy change rate: ; Maximum positive entropy change rate (entropy surge): ; Variance of entropy rate of change: ; in, The average entropy change rate, This represents the maximum positive entropy change rate (i.e., the magnitude of the entropy surge).

[0038] The third order is the entropy acceleration feature (second-order difference). The entropy acceleration sequence is obtained by calculating the second-order difference based on the entropy rate of change sequence, which characterizes the degree of oscillation of entropy change: Based on this, two characteristics are calculated: the absolute value of the acceleration extremum and the number of positive and negative acceleration reversals, which characterize the severity of entropy fluctuations and the characteristics of sudden state changes. Absolute value of acceleration extrema: ; Acceleration Flip Count: The total number of times the sign of the entropy acceleration within the window changes. ; The above 8 features are concatenated into a feature vector, namely data E2 (third-order entropy change feature set): ; in, The absolute value of the acceleration extremum. This represents the total number of times the sign of the entropy acceleration changes within the window.

[0039] The aforementioned semantic consistency score can be a comprehensive alignment score obtained after establishing a strong quantitative correlation between the original input and the model-generated content. It is used to filter out invalid entropy change anomaly signals and reduce the false alarm rate. Specifically, relying solely on entropy change features cannot distinguish between "reasonable entropy increase caused by normal complex reasoning" and "abnormal entropy increase caused by content deviating from the facts." Introducing semantic consistency verification can make up for this deficiency. The higher the semantic consistency score, the more closely the generated content fits the facts and semantic constraints of the input.

[0040] For example, first generate content semantic encoding: up to the [number]th [section / section / etc.] t The generated sequence of the step By inputting the same semantic encoding model, we obtain the semantic representation vector of the generated content. Then, global semantic alignment is calculated by determining the cosine similarity between the generated semantic vector and the input anchor vector, which serves as the global alignment score. ,in, For globally aligned scores, This is used to generate semantic representation vectors for the content. A fixed semantic anchor vector corresponding to the input prompt text.

[0041] Furthermore, entity-level fact alignment verification extracts key entities and factual constraints (such as time, value, and subject) from the input Prompt, matches them against the generated sequence, and calculates the entity matching rate. : ; Furthermore, by integrating the comprehensive alignment score, the global semantic alignment and entity alignment are weighted and fused to obtain the final input-generated alignment score: ; in, Generate aligned scores for the input. For entity matching rate, For globally aligned scores, weights Take 0.6, which can be adjusted according to the factual requirements of the task.

[0042] Alignment score determination rule: When If the alignment is deemed "not in compliance with requirements", it is marked as a high-risk concern and weighted amplification is applied in subsequent risk calculations to strengthen the penalty weight for deviations from the input anchor point.

[0043] The matching score mentioned above can be the highest similarity obtained by matching the dynamic entropy change trajectory (or its derived third-order features) with each standard precursor pattern in a pre-constructed hallucination precursor pattern library. Specifically, the matching score measures the degree of agreement between the entropy change pattern of the current generation process and known hallucination precursors, and is the core dynamic signal for identifying the initiation of hallucinations.

[0044] The aforementioned initial hallucination risk value can be the original risk score obtained by weighted fusion of the semantic consistency score after deviation calculation and the matching score. Specifically, relying solely on entropy change dynamics features cannot distinguish between normal reasoning fluctuations and abnormal hallucination fluctuations in complex scenarios. By introducing input-generation semantic consistency constraints, true / false identification is achieved. Only when the entropy change features are abnormal and accompanied by the generated content continuously deviating from the original input semantic facts is it determined to be a hallucination precursor.

[0045] For example, the Dynamic Time Warping (DTW) algorithm is first used to match the entropy change trajectory within the current window with a pre-built library of hallucination prodrome patterns (4 standard patterns) one by one, and the matching similarity of each pattern is obtained. The highest similarity score is taken as the pattern matching score. .

[0046] Furthermore, the decrease in the current alignment score compared to the initial alignment benchmark (the average alignment score generated in the first 3 steps) is calculated as the alignment deviation. : ; If the current alignment score does not meet the threshold Then, a 1.5-fold weighted amplification is applied to the deviation, i.e. Strengthen the risk weights of low-aligned states.

[0047] Furthermore, the entropy change pattern matching score and alignment deviation are weighted and fused to obtain the initial hallucination risk value. : ; Among them, weight The default value is 0.5, which means that the entropy change dynamics feature is as important as the input consistency verification.

[0048] The aforementioned hallucination risk probability can be obtained by dynamically mapping a normal risk baseline based on semantic complexity parameters, and then normalizing the initial hallucination risk value using this baseline as a boundary to obtain a standardized risk probability in the range of 0 to 1. Specifically, this step solves the problem of poor generalization of traditional fixed thresholds, enabling the judgment threshold to adapt to generation scenarios with different domains and different inference depths.

[0049] For example, firstly, based on the semantic complexity parameter D, the baseline value of normal risk for the current task is calculated using a pre-trained linear mapping function: ; in, The mapping parameters are obtained from pre-training. For domain rarity, To accumulate the number of reasoning steps, For factual requirements level, This is a bias term.

[0050] Furthermore, using the baseline value as a boundary, the original risk score is mapped to a final hallucination risk probability in the range of 0 to 1: ; When the risk probability exceeds 0.5, it indicates that the generated state deviates from the normal baseline and there is a risk of hallucination.

[0051] The hallucination detection results described above can be used to output corresponding warning levels and intervention strategies based on fixed risk level determination rules and the number of steps the risk continues to generate. Specifically, the detection results map continuous risk probabilities into actionable business actions, enabling tiered warnings and proactive intervention for hallucinations.

[0052] For example, based on the established risk level determination rules and the number of steps the risk continues to generate, corresponding early warning instructions and intervention strategies are output: When the risk probability is in the mild risk range of 0.3 to less than 0.6, the currently generated segment is marked as pending verification, and the risk changes are continuously tracked without interrupting the normal generation process. When the risk probability is in the moderate risk range of 0.6 to less than 0.8 and lasts for more than 3 steps, it is judged as a moderate hallucination risk, triggering the knowledge anchoring correction mechanism and injecting authoritative factual information to correct the generation direction. When the risk probability is greater than or equal to 0.8 and matches a typical hallucination precursor pattern, it is judged as a high-risk hallucination state, the current abnormal generation branch is immediately terminated, alternative reasoning paths are started or confidence level prompts are output, and the entire process of hallucination warning and intervention is completed.

[0053] To facilitate understanding of the overall method in this embodiment, a specific application example of the sequential execution of the above steps is given below in a medical question-and-answer scenario. In a certain medical intelligent question-and-answer system, the user inputs a prompt: "Please briefly describe the recommended daily salt intake standard for adult hypertensive patients in my country." During the autoregressive generation of the answer by the large model, a factual bias occurs at the output key fact value node (the correct conclusion is no more than 5 grams per day, but the model tends to generate the incorrect value "10 grams"), thus gradually tracking and calculating and triggering an alert. Taking the 12th generation step as the core observation node, with a sliding window size w=8, the entire data flow and intermediate results are as follows: Step 1: Collect raw input data (data A / B / C / D):

[0054] Step 2: Calculate the dynamic entropy change trajectory based on data A (output: data E1): [Input] Step-by-step vocabulary probability distribution from steps 1 to 12; [Calculation process]: Calculate the entropy value in step 12 using the instantaneous entropy formula: ; Extract the instantaneous entropy values ​​of the most recent 8 steps (steps 5 to 12) to form a window entropy variable sequence.

[0055] Output: Data E1 Window dynamic entropy change trajectory sequence: ; Feature description: The entropy value drops sharply in step 9 (the model forcibly selects an incorrect number token), and the entropy value rises rapidly in steps 11-12, which is consistent with the evolutionary characteristics of hallucination precursors.

[0056] Step 3: Calculate the third-order entropy change feature set based on data E1 (output: data E2): [Input] An 8-dimensional window entropy change sequence of data E1; [Calculation process]: 1. Statistical characteristics of first-order instantaneous entropy: Window average entropy: ; Window entropy standard deviation: ; Maximum entropy deviation: ; 2. Second-order entropy rate characteristic: First calculate the entropy rate sequence of adjacent steps: ; Average entropy change rate: ; Maximum positive entropy change rate: ; Variance of entropy rate of change: ; 3. By differencing the entropy rate sequence using the third-order entropy acceleration characteristics, the acceleration sequence is obtained: ; Absolute value of acceleration extrema: ; Number of positive and negative acceleration flips: 3 times; Output: Data E2 8-dimensional third-order entropy change eigenvectors: Step 4: Calculate the input-generated semantic alignment score based on data B and data C (output: data E3): [Input] Data B (generated text), Data C (input semantic anchor vector); [Calculation process]: 1. Encode the generated text to obtain the semantic vector of the generated content. Calculate the cosine similarity with the input anchor vector: ; 2. Extract the core entity set of the input {hypertension, salt intake, standard}, and calculate the entity matching rate of the generated text: ; 3. Weighted fusion yields the overall alignment score. : ; 4. Alignment determination: Items that do not meet the alignment threshold are marked as high-risk items, and their deviation is weighted at 1.5 times. The average alignment benchmark in the initial generation stage is 0.91. The weighted alignment deviation is calculated as follows: ;

Output: Data E3

[0057] Step 5: Based on E2 and E3, complete the joint assessment and output the raw hallucination risk score (data E4):

Input

Input

[0058] Step 7: Output the final early warning and intervention results based on E5: [Input] Data E5 (Risk probability 0.65): [Judgment Logic] The risk probability is in the medium risk range of 0.6 to less than 0.8, and the risk status has continued for 3 consecutive steps (steps 10, 11, and 12), which meets the triggering conditions for a level 2 warning.

[0059] [Final Output]: Warning level: Level 2 (moderate risk of hallucinations); Intervention action: Trigger the knowledge anchoring mechanism to automatically retrieve the entry "salt intake of patients with hypertension" from the authoritative medical knowledge base, inject the factual information "recommended daily intake ≤ 5 grams" into the generation context, guide the model to correct the subsequent generated content, and finally output the correct conclusion.

[0060] Based on this, the hallucination detection method based on a large language model with dynamic entropy change trajectory provided in this application collects four types of data: vocabulary probability distribution, generated text sequences, semantic vectors of input prompt text, and semantic complexity parameters. It quantifies instantaneous entropy step by step and constructs temporal and dynamic entropy change trajectories, extracts third-order entropy change features, and completes precursor matching and joint evaluation by combining input-generation semantic consistency verification. Then, it outputs standardized risk probabilities through semantic complexity adaptive calibration, ultimately providing graded warnings and interventions. This method strongly couples the uncertainty of internal model generation with external input factual constraints, solving the technical problems of existing static single-point detection methods, such as inability to capture temporal evolution precursors, input-output semantic decoupling, poor generalization of fixed thresholds, lack of real-time intervention capabilities, and inability to identify confident latent hallucinations. It can identify hallucination precursor patterns in real time during autoregressive generation, achieving advanced warning and preventative control of hallucinations.

[0061] In some embodiments, determining semantic consistency based on the difference between the previously generated text sequence and the semantic vector before the current generation step includes the following steps S210 to S250: Step S210: Extract key entities and factual constraints from the input prompt text.

[0062] In this step, the aforementioned key entities and factual constraints can be core semantic elements extracted from the input prompt text to measure whether the generated content faithfully reflects the input, including main entities, time, values, relationships, etc. Specifically, the key entities and factual constraints constitute a factual benchmark list on the input side, which serves as the basis for subsequent entity-level alignment.

[0063] Step S220: Calculate the matching ratio of key entities and fact constraints in the generated text sequence up to the current generation step to obtain the entity-level alignment score.

[0064] In this step, the aforementioned entity-level alignment score can be calculated by matching key entities with factual constraints one by one in the generated text sequence, using the ratio of the number of matches to the total number of constraints to measure entity-level fidelity. Specifically, the entity-level alignment score quantifies the degree to which the generated content covers the input at the level of discrete factual elements, and is a hard constraint in the semantic consistency score that combats factual deviation.

[0065] Step S230: Encode the generated text sequence up to the current generation step to obtain the generated semantic vector.

[0066] In this step, the above-mentioned generation of semantic vectors can be achieved by taking the vectors up to the specified number of lines. The generated sequence input semantic encoding model of the step is obtained The semantic representation vector is a dimensional semantic representation vector. Specifically, the generated semantic vector and the input semantic anchor vector reside in the same semantic space, and their similarity can be directly compared.

[0067] Step S240: Calculate the cosine similarity between the generated semantic vectors and the semantic vectors.

[0068] In this step, the cosine similarity mentioned above can be the cosine of the angle between the generated semantic vector and the input semantic anchor vector, serving as a global semantic alignment score. Specifically, the cosine similarity quantifies the consistency between the generated content and the input prompt at the overall semantic representation level; the closer the value is to 1, the better the semantic fit.

[0069] Step S250: Calculate the semantic consistency score based on the entity-level alignment score and cosine similarity.

[0070] In this step, the semantic consistency score mentioned above can be a comprehensive score obtained by weighted fusion of global semantic alignment (cosine similarity) and entity-level alignment. Specifically, the fusion takes into account both overall semantic consistency and discrete factual consistency, making it more robust than a single indicator.

[0071] By extracting key entities and factual constraints separately, statistically analyzing entity-level matching ratios, encoding semantic vectors, calculating cosine similarity, and performing weighted fusion, the input-generation semantic consistency is quantified into a computable comprehensive score. This allows for the filtering of invalid abnormal signals caused by simple entropy change features using dual alignment verification, significantly reducing the false alarm rate in complex reasoning scenarios and enabling the distinction between "normal reasoning entropy increase" and "factual deviation entropy increase."

[0072] In some embodiments, determining the matching score between the dynamic entropy change trajectory and the pre-built hallucination precursor pattern library includes the following steps S310 to S320: Step S310: Perform dynamic time warping matching between the dynamic entropy change trajectory and each standard precursor pattern in the hallucination precursor pattern library to obtain the matching similarity corresponding to each standard precursor pattern. The hallucination precursor pattern library includes at least the entropy sudden drop followed by sudden rise type, continuous high-level oscillation type, monotonically slow rise type, and low-entropy mutation type.

[0073] In this step, the aforementioned dynamic time warping matching can be achieved using the Dynamic Time Warping (DTW) algorithm. This algorithm compares the dynamic entropy change trajectory within the current window with the entropy change patterns of various standard precursor patterns in the pattern library one by one to obtain the matching similarity corresponding to each standard pattern. Specifically, DTW can tolerate the scaling and shifting of two sequences on the time axis, making it suitable for comparing entropy change patterns of similar length but different rhythms, and is more robust than fixed point-to-point comparisons.

[0074] The four types of standard precursor patterns mentioned above are as follows: 1. Entropy drop followed by sudden rise: The model first forcibly selects a certain word to cause a sudden drop in entropy, and then the entropy rises rapidly due to the rebound of uncertainty; 2. Continuous high-level oscillation: The entropy value remains at a high level for a long time within the window and fluctuates repeatedly; 3. Monotonically rising: The entropy value rises smoothly and continuously with the number of generation steps; 4. Low-entropy mutation: The model is in a low-entropy steady state but suddenly jumps at a certain step.

[0075] Step S320: Take the highest value of the matching similarity corresponding to each standard precursor pattern as the matching score.

[0076] In this step, the matching score mentioned above can be the maximum value of the similarity scores for each standard precursor pattern. Specifically, the highest similarity score is taken as the entropy change pattern of the current generation process that is considered closest to that type of precursor, and it is used as the dynamic confidence score for subsequent risk calculation.

[0077] By dynamically warping the time, the dynamic entropy change trajectory is matched one by one with a precursor pattern library containing four standard patterns, and the highest similarity is taken as the matching score. In this way, the abstract entropy change pattern recognition is transformed into a quantifiable precursor matching confidence, which can capture the temporal dynamic precursors in the budding stage of hallucination. In particular, low-entropy confident latent hallucinations (whose single-point entropy value does not fluctuate significantly) can be identified through pattern matching, making up for the missed detection defects of static single-point detection.

[0078] In some embodiments, the initial hallucination risk value is obtained based on the semantic consistency score and the matching score, including the following steps S410 to S430: Step S410: Determine the semantic deviation score based on the decrease in the semantic consistency score of the current generation step and the preset initial alignment benchmark.

[0079] In this step, the aforementioned semantic deviation score can be the decrease in the semantic consistency score of the current generation step relative to the preset initial alignment benchmark. Specifically, the initial alignment benchmark characterizes the alignment level when the content is still faithful to the input in the early stages of generation, and the subsequent decrease directly reflects the degree to which the generated content deviates from the constraints of the input facts.

[0080] Step S420: If the semantic consistency score of the current generation step is lower than the preset alignment threshold, apply a preset multiple of weighted amplification to the semantic deviation score to obtain the weighted semantic deviation score.

[0081] In this step, the aforementioned preset alignment threshold and preset multiplier can be pre-set judgment thresholds and amplification coefficients. Specifically, when the semantic consistency score is lower than the threshold, it indicates that the generated content has significantly deviated from the input facts. At this time, applying weighted amplification to its deviation can strengthen the risk weight of the low alignment state and prevent it from being diluted by the high matching score.

[0082] Step S430: Combine the matching score with the semantic deviation score or the weighted semantic deviation score to obtain the initial hallucination risk value.

[0083] In this step, the initial hallucination risk value mentioned above can be the original risk score obtained by fusing the matching score and the (weighted) semantic deviation score according to their weights. Specifically, the fusion makes the risk determination rely on both the entropy change dynamic precursor (matching score) and the input semantic deviation (deviation score) to achieve true / false identification.

[0084] By determining the semantic deviation score based on the alignment decrease, applying weighted amplification at low alignment, and then fusing it with the matching score to obtain the initial illusion risk value, the semantic consistency constraint can be used as a verification gate for entropy change anomalies. The risk is only increased when entropy change precursors and semantic deviations occur simultaneously, thereby reducing both false positive and false negative rates.

[0085] In some embodiments, the above-mentioned calibration of the initial hallucination risk value based on the semantic complexity parameter to obtain the hallucination risk probability includes the following steps S510 to S520: Step S510: Calculate the normal risk baseline value of the generated task corresponding to the input prompt text based on the semantic complexity parameter.

[0086] In this step, the aforementioned normal risk baseline value can be the normal risk boundary value under the current task, calculated by a pre-trained linear mapping function based on semantic complexity parameters (domain rarity, cumulative inference steps, and factual requirement level). Specifically, the entropy baselines for different tasks are inherently different. Tasks with scarce professional domains or high factual requirements naturally have a higher reasonable risk level, so a complexity-adaptive baseline should be used instead of the globally fixed threshold.

[0087] Step S520: Using the normal risk baseline value as the boundary, normalize the initial hallucination risk value to obtain the hallucination risk probability.

[0088] In this step, the aforementioned hallucination risk probability can be a standardized probability that maps the initial hallucination risk value to the interval between 0 and 1, using the normal risk threshold under the current task as the boundary. Specifically, normalization decouples the risk representation from the specific task baseline. When the risk score exceeds the baseline, it is mapped to a probability greater than 0.5, representing that the generated state deviates from the normal baseline and there is a risk of hallucination.

[0089] By mapping the normal risk baseline to the semantic complexity parameter and normalizing the initial risk value with the baseline as the boundary to obtain the hallucination risk probability, it can automatically adapt to generation tasks of different domains, different inference depths, and different text types. It does not require manual adjustment of the threshold for a single scenario, and the cross-scenario detection performance is stable, solving the pain points of poor cross-scenario adaptability and threshold migration failure of traditional methods.

[0090] In some embodiments, generating the hallucination detection result of the large language model to be detected in the current generation step based on the hallucination risk probability includes the following steps S610 to S630: Step S610: If the probability of hallucination risk is within the preset first risk range, mark the text generated in the current generation step as pending verification.

[0091] In this step, the aforementioned preset first risk range can be a low-risk range (e.g., 0.3 to less than 0.6). Specifically, being in this range indicates that there is a certain risk but it has not yet reached the level requiring intervention. Therefore, it is only marked as pending verification and the risk changes are continuously tracked without interrupting the normal generation process.

[0092] Step S620: If the hallucination risk probability is within the preset second risk range and the number of consecutive generation steps within the preset second risk range reaches the preset number of steps, determine that the hallucination detection result of the current generation step is a moderate hallucination risk.

[0093] In this step, the aforementioned preset second risk range and preset number of steps can be a moderate risk range (e.g., 0.6 to less than 0.8) and a continuous step threshold (e.g., 3 steps). Specifically, a stable moderate risk is only confirmed when the risk remains within the moderate range for multiple consecutive steps, thus avoiding misjudgments caused by single-step fluctuations.

[0094] Step S630: If the hallucination risk probability is in the preset third risk range and the standard precursor pattern corresponding to the matching score belongs to the preset typical precursor pattern, determine the hallucination detection result of the current generation step as a high-risk hallucination.

[0095] In this step, the aforementioned preset third risk range and preset typical precursor patterns can be a high-risk range (e.g., greater than or equal to 0.8) and a set of typical precursor patterns (e.g., entropy sudden drop followed by sudden rise, low-entropy mutation). Specifically, the high-risk determination must simultaneously satisfy both high-risk probability and typical precursor pattern to prevent simple high-entropy fluctuations from being misjudged as high-risk.

[0096] By using the first, second, and third risk intervals and combining continuous steps with typical precursor patterns, three types of test results—pending verification, moderate, and high risk—are generated in a hierarchical manner. This allows continuous risk probabilities to be mapped into hierarchical business actions, supporting flexible risk prevention and control in high-reliability scenarios.

[0097] In some embodiments, after generating the hallucination detection result of the large language model to be detected in the current generation step based on the hallucination risk probability, the method further includes the following steps S710 to S730: Step S710: If the hallucination detection result of the current generation step indicates a pending verification state, monitor the hallucination risk probability of the next generation step of the large language model to be detected.

[0098] In this step, monitoring the probability of hallucination risk in the next generation step can be done without interrupting generation. This involves continuing to calculate the risk probability and track its changing trend for subsequent generation steps according to steps S110 to S190 of Example 1. Specifically, the state awaiting verification only needs to be observed; no active intervention is required to avoid disturbing normal generation.

[0099] Step S720: If the hallucination detection result of the current generation step indicates a moderate hallucination risk, retrieve authoritative factual information that matches the input prompt text from the preset knowledge base and inject it into the large language model to be detected to obtain an optimized large language model.

[0100] In this step, the aforementioned retrieval and injection of authoritative factual information can trigger a knowledge anchoring correction mechanism. This mechanism retrieves authoritative factual entries related to the input prompt text from a pre-defined knowledge base and injects them into the generation context to guide the model in correcting subsequent generation. Specifically, moderate risks can still be mitigated through factual correction; therefore, a soft intervention is used instead of a hard termination.

[0101] Step S730: If the hallucination detection result of the current generation step indicates a high risk of hallucination, terminate the generation of lexical units in the large language model to be detected and generate an abnormal intervention instruction.

[0102] In this step, the aforementioned termination of lexical generation and generation of anomaly intervention instructions can be implemented by immediately terminating the current anomalous generation branch in high-risk situations, initiating alternative inference paths, or outputting confidence level prompts. Specifically, a high-risk situation has been determined to be a severe hallucination, requiring hard intervention to prevent the continued generation of erroneous content. For example, when a high-risk hallucination risk is determined, the current anomalous generation branch is immediately terminated, and anomaly intervention instructions or confidence level prompts are output.

[0103] By configuring three intervention actions—monitoring, knowledge anchoring injection, and generation termination—for the three types of results—verification, moderate, and high-risk—it is possible to match differentiated intervention intensities based on risk levels. This can reduce disruptions in mild cases and prevent erroneous outputs in high-risk cases, supporting large-scale implementation in high-risk scenarios such as healthcare, law, finance, and government.

[0104] like Figure 2 As shown, some embodiments of this application provide a hallucination detection system based on a large language model with dynamic entropy-varying trajectories, specifically: The acquisition module 210 is used to acquire the vocabulary probability distribution output by the current generation step, the sequence of texts generated before the current generation step, the semantic vector of the input prompt text, and the semantic complexity parameters of the generation task corresponding to the input prompt text during the autoregressive generation process of the large language model to be detected based on the input prompt text. The first module 220 is used to determine the instantaneous entropy value of the current generation step based on the vocabulary probability distribution; The second module 230 is used to arrange the instantaneous entropy values ​​from the first generation step to the current generation step in the order of generation of the corresponding generation steps to obtain the temporal entropy change trajectory. The third module 240 is used to extract multiple consecutive instantaneous entropy values ​​adjacent to the current generation step from the temporal entropy change trajectory to obtain the dynamic entropy change trajectory; The fourth module 250 is used to determine the semantic consistency score based on the difference between the generated text sequence and the semantic vector before the current generation step; Module 5, 260, is used to determine the matching score between the dynamic entropy change trajectory and a pre-built library of hallucination precursor patterns; Module 6, 270, is used to obtain the initial hallucination risk value based on the semantic consistency score and the matching score; Module 7, 280, is used to calibrate the initial hallucination risk value based on the semantic complexity parameter to obtain the hallucination risk probability. Module 8, 290, is used to generate the hallucination detection results of the large language model to be detected at the current generation step based on the hallucination risk probability.

[0105] It should be noted that the large language model hallucination detection system based on dynamic entropy trajectory provided in this embodiment is based on the same inventive concept as the large language model hallucination detection method based on dynamic entropy trajectory described above. Therefore, the relevant content of the large language model hallucination detection method based on dynamic entropy trajectory described above is also applicable to the content of the large language model hallucination detection system based on dynamic entropy trajectory. Therefore, it will not be repeated here.

[0106] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described hallucination detection method based on a large language model with dynamic entropy change trajectory.

[0107] like Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the above-described hallucination detection method based on a large language model with dynamic entropy trajectory as described in this application.

[0108] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0109] The electronic devices according to embodiments of this application will now be described in detail.

[0110] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and called and executed by the processor 1600 to execute the large language model hallucination detection method based on dynamic entropy trajectory of this application.

[0111] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0112] This application also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described hallucination detection method based on a large language model with dynamic entropy change trajectory.

[0113] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0114] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0115] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0117] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0118] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any related variations, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0119] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0121] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0122] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0123] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0124] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

[0125] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A hallucination detection method based on a large language model with dynamic entropy-varying trajectories, characterized in that, The method includes: During the autoregressive generation process of the large language model to be detected based on the input prompt text, the vocabulary probability distribution output by the current generation step, the sequence of texts generated before the current generation step, the semantic vector of the input prompt text, and the semantic complexity parameters of the generation task corresponding to the input prompt text are obtained. The instantaneous entropy value of the current generation step is determined based on the vocabulary probability distribution. The instantaneous entropy values ​​from the first generation step to the current generation step are arranged in the order of generation of the corresponding generation steps to obtain the temporal entropy change trajectory; The dynamic entropy change trajectory is obtained by extracting multiple consecutive instantaneous entropy values ​​adjacent to the current generation step from the temporal entropy change trajectory. A semantic consistency score is determined based on the difference between the previously generated text sequence and the semantic vector before the current generation step. Determine the matching score between the dynamic entropy change trajectory and a pre-constructed library of hallucination precursor patterns; Based on the semantic consistency score and the matching score, an initial hallucination risk value is obtained; The initial hallucination risk value is calibrated based on the semantic complexity parameter to obtain the hallucination risk probability; Based on the hallucination risk probability, generate the hallucination detection result of the large language model to be detected in the current generation step.

2. The hallucination detection method based on a large language model with dynamic entropy variation trajectory according to claim 1, characterized in that, The step of determining the semantic consistency score based on the difference between the previously generated text sequence and the semantic vector before the current generation step includes: Extract the key entities and factual constraints from the input prompt text; The entity-level alignment score is obtained by statistically analyzing the matching ratio of the key entities and fact constraints in the generated text sequence up to the current generation step. Encode the generated text sequence up to the current generation step to obtain a generated semantic vector; Calculate the cosine similarity between the generated semantic vector and the semantic vector; The semantic consistency score is calculated based on the entity-level alignment score and the cosine similarity.

3. The hallucination detection method based on a large language model with dynamic entropy variation trajectory according to claim 1, characterized in that, Determining the matching score between the dynamic entropy change trajectory and a pre-built library of hallucination precursor patterns includes: The dynamic entropy change trajectory is dynamically time-warped and matched with each standard precursor pattern in the hallucination precursor pattern library to obtain the matching similarity corresponding to each standard precursor pattern. The hallucination precursor pattern library includes at least the entropy sudden drop followed by sudden rise type, continuous high-level oscillation type, monotonous slow rise type and low-entropy sudden change type. The highest value among the matching similarities corresponding to each standard precursor pattern is taken as the matching score.

4. The hallucination detection method based on a large language model with dynamic entropy variation trajectory according to claim 1, characterized in that, The process of obtaining the initial hallucination risk value based on the semantic consistency score and the matching score includes: The semantic deviation score is determined based on the decrease in the semantic consistency score of the current generation step compared to the preset initial alignment benchmark. If the semantic consistency score of the current generation step is lower than a preset alignment threshold, the semantic deviation score is weighted by a preset multiple to obtain a weighted semantic deviation score. The initial hallucination risk value is obtained by combining the matching score with the semantic deviation score or the weighted semantic deviation score.

5. The hallucination detection method based on a large language model with dynamic entropy variation trajectory according to claim 1, characterized in that, The step of calibrating the initial hallucination risk value according to the semantic complexity parameter to obtain the hallucination risk probability includes: Based on the semantic complexity parameter, calculate the normal risk baseline value of the generated task corresponding to the input prompt text; Using the normal risk baseline value as a boundary, the initial hallucination risk value is normalized to obtain the hallucination risk probability; The semantic complexity parameters include domain rarity, cumulative reasoning steps, and factual requirement level.

6. The hallucination detection method based on a large language model with dynamic entropy variation trajectory according to claim 1, characterized in that, The step of generating the hallucination detection result of the large language model to be detected in the current generation step based on the hallucination risk probability includes: If the probability of hallucination risk is within a preset first risk range, the text generated in the current generation step is marked as pending verification. If the hallucination risk probability is within a preset second risk range and the number of consecutive generation steps within the preset second risk range reaches a preset number, the hallucination detection result of the current generation step is determined to be a moderate hallucination risk. If the hallucination risk probability is within a preset third risk range, and the standard precursor pattern corresponding to the matching score belongs to a preset typical precursor pattern, then the hallucination detection result of the current generation step is determined to be a high-risk hallucination.

7. The hallucination detection method based on a large language model with dynamic entropy variation trajectory according to claim 6, characterized in that, The method further includes generating the large language model to be detected based on the hallucination risk probability after the hallucination detection result of the current generation step: If the hallucination detection result of the current generation step indicates a pending verification state, monitor the hallucination risk probability of the next generation step of the large language model to be detected; If the hallucination detection result of the current generation step indicates a moderate hallucination risk, authoritative factual information matching the input prompt text is retrieved from the preset knowledge base and injected into the large language model to be detected to obtain an optimized large language model. If the hallucination detection result of the current generation step indicates a high risk of hallucination, terminate the generation of lexical units in the large language model to be detected and generate an abnormal intervention instruction.

8. A hallucination detection system based on a large language model with dynamic entropy-varying trajectories, characterized in that, The system includes: The acquisition module is used to acquire the vocabulary probability distribution output by the current generation step, the sequence of texts generated before the current generation step, the semantic vector of the input prompt text, and the semantic complexity parameter of the generation task corresponding to the input prompt text during the autoregressive generation process of the large language model to be detected based on the input prompt text. The first module is used to determine the instantaneous entropy value of the current generation step based on the vocabulary probability distribution; The second module is used to arrange the instantaneous entropy values ​​from the first generation step to the current generation step in the order of generation of the corresponding generation steps to obtain the temporal entropy change trajectory. The third module is used to extract multiple consecutive instantaneous entropy values ​​adjacent to the current generation step from the temporal entropy change trajectory to obtain the dynamic entropy change trajectory; The fourth module is used to determine the semantic consistency score based on the difference between the generated text sequence before the current generation step and the semantic vector; The fifth module is used to determine the matching score between the dynamic entropy change trajectory and the pre-built hallucination precursor pattern library; The sixth module is used to obtain an initial hallucination risk value based on the semantic consistency score and the matching score; The seventh module is used to calibrate the initial hallucination risk value according to the semantic complexity parameter to obtain the hallucination risk probability; The eighth module is used to generate the hallucination detection result of the large language model to be detected in the current generation step based on the hallucination risk probability.

9. An electronic device, characterized in that, The method includes at least one processor and a memory for communicatively connecting to the at least one processor; the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the large language model hallucination detection method based on dynamic entropy trajectory as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the hallucination detection method based on a large language model with dynamic entropy change trajectory as described in any one of claims 1 to 7.

Images