Hidden layer activation-based prejudice illusion detection method

Abstract

The invention discloses a prejudice illusion detection method based on hidden layer activation. The method comprises the steps that a belief state is evaluated according to behavior characteristics of answers of a language model, a belief state evaluation result is obtained, and the belief state reflects the uncertainty degree of the answers of the language model; based on the belief state evaluation result, a training set is constructed, a belief state classifier is trained, and the training set reflects the corresponding relation between the belief state of the language model and hidden layer activation; acquiring hidden layer activation in a target language model generation process in real time, and judging a belief state by utilizing the trained belief state classifier according to the hidden layer activation; and for the detected illusion, in combination with the output correctness of the target language model and the belief state judged by the belief state classifier, judging the type of the illusion. According to the invention, belief state determination is carried out based on model hidden layer activation, and the accuracy of prejudice hallucination detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of large model hallucination detection, and more specifically, to a bias hallucination detection method based on hidden layer activation. Background Art

[0002] Hallucination in large language models (LLMs) refers to the phenomenon that LLMs produce coherent but factually inaccurate responses. The purpose of existing LLMs hallucination detection is to identify possible factual errors in the output. Hallucination detection methods include methods that retrieve external knowledge and detection methods based on model uncertainty. Methods that retrieve external knowledge verify the correctness of the model output through external knowledge sources or reference answers. If the output of the model does not match the objective facts, it is regarded as a potential hallucination, such as detecting factual errors by using external tools for evidence collection. Detection methods based on model uncertainty do not rely on external knowledge and detect hallucinations by estimating the uncertainty of the model output.

[0003] Previous studies have shown that the hidden activations of LLMs encapsulate internal beliefs related to the uncertainty of the statements they generate. These hidden activations can serve as an effective basis for hallucination detection. Although current hallucination detection methods have made significant progress in identifying false information in the output of LLMs, there are still limitations. First, whether it is the method of retrieving external knowledge or the method based on model uncertainty, existing works only consider the correctness of the model output, but lack attention to the causes of hallucinations. These works usually focus on detecting the existence of hallucinations and ignore the differences between hallucinations. For example, the method based on model uncertainty assumes that the high certainty of the model means that the output is accurate, but the high certainty of the model does not guarantee the accuracy of the output because the model itself may have erroneous biases. These limitations not only hinder the reliability of hallucination detection for LLMs, but also limit the possibility of targeted improvement of model performance. Therefore, it is necessary to consider different bases to improve the quality of hallucination detection work, such as the correctness of the model output and the confidence of the model in its own answers. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a bias hallucination detection method based on hidden layer activation. The method comprises the following steps:

[0005] Evaluate the belief state according to the behavioral characteristics of the language model's answer, and obtain a belief state evaluation result, wherein the belief state reflects the uncertainty degree of the language model for the answer;

[0006] Constructing a training set based on the belief state evaluation result and training a belief state classifier, wherein the training set reflects the corresponding relationship between the belief state of the language model and the hidden layer activation;

[0007] Acquire the hidden layer activation in the target language model generation process in real time, and judge the belief state using the trained belief state classifier according to the hidden layer activation;

[0008] For the detected hallucination, the hallucination category is determined by combining the output correctness of the target language model with the belief state determined by the belief state classifier.

[0009] Compared with the prior art, the advantage of the present invention is that, compared with the hallucination detection methods based on uncertainty and based on retrieving external knowledge, the present invention provides a biased hallucination detection method based on belief state, which solves the technical problem of capturing the belief state of a large model. The present invention uses a large model and a feedforward neural network (FNN) as core components, combines the correctness of the model output and the degree of confidence in its own output, divides the hallucination more finely, and finally obtains a fine-grained and highly reliable hallucination detection result.

[0010] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0012] Figure 1 is a schematic diagram of distinguishing hallucination categories according to an embodiment of the present invention;

[0013] Figure 2 is a flow chart of bias hallucination detection according to one embodiment of the present invention;

[0014] Figure 3 is a flow chart of a bias hallucination detection method based on hidden layer activation according to an embodiment of the present invention;

[0015] Figure 4 is a flowchart of cognitive state assessment and cognitive state training set construction according to one embodiment of the present invention;

[0016] Figure 5 A schematic diagram of an ablation experiment according to an embodiment of the present invention;

[0017] Figure 6 is a schematic diagram of a data distribution statistical sample according to an embodiment of the present invention;

[0018] Figure 7 Schematic diagram of a visualization example of hidden layer activation of a model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0020] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0021] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0022] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0023] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0024] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0025] In general, the provided bias hallucination detection method based on hidden layer activation includes: evaluating the belief state according to the behavioral characteristics of the model's answer; constructing a belief state classifier based on hidden layer activation according to the model's hidden layer activation and belief state evaluation results; using an illusion evaluation method based on external knowledge to judge the correctness of the model output, and obtaining the hidden layer activation of the model generation process in real time, and judging the belief state according to the model's hidden layer activation; combining the output correctness with the belief state to judge the hallucination category.

[0026] In one embodiment, belief state evaluation includes: inputting question text q into model M to obtain model response a; sampling multiple model responses and extracting valid keywords in the responses using an answer extraction model; calculating response consistency and evaluating the model belief state.

[0027] In one embodiment, constructing a belief state classifier includes: for each question q in an existing open domain question answering benchmark, obtaining a model response a and obtaining the model hidden layer activation in real time during the model generation process; evaluating the consistency of the model response and calculating the belief state label s i, and combined with the model hidden layer activation to construct a belief state classification training set, and use the constructed training set to train a small feedforward neural network to obtain a belief state classifier.

[0028] In one embodiment, based on the model belief state and output correctness, detecting bias hallucinations includes: for a given question, the model gives a response and obtains the hidden layer activation h of the model generation process; based on a method of retrieving external knowledge, the model response is compared with the correct answer to evaluate the model output quality; using the model's hidden layer activation, the belief state of the model about its own answer is judged through a belief state classifier; for the detected hallucination r, the hallucination type v is judged in combination with the model's belief state.

[0029] Hereinafter, detailed embodiments of the present invention will be specifically described.

[0030] This paper explores the large model bias hallucination detection task and proposes an effective bias hallucination detection model based on hidden layer activation. Figure 1 As shown in the figure, in order to determine the reason why LLMs generate hallucinations, the bias hallucination detection task is defined as a binary classification problem, that is, to determine whether the hallucinations generated by large language models (LLMs) are bias hallucinations or knowledge-deficient hallucinations. Based on this goal, a belief-based bias hallucination detection framework BABID is proposed. Figure 2 As shown in the figure, the framework consists of two core modules: a belief state classifier and an evidence-based hallucination detection module. For each question in the dataset, the large model extracts the hidden layer activations in the model generation process while answering. The evidence-based hallucination detection module adopts the large model evaluation method, such as using ChatGPT to judge the correctness of the model's answer. This module takes the question and the model answer as input and outputs the hallucination detection result. The belief state classifier uses a trained classifier model, takes the hidden layer activations in the generation process as input, and outputs the belief state of the model, such as a confident state (confident) or an uncertain state (uncertain). Finally, BABID combines the hallucination detection results and the belief state to classify the hallucination as a bias hallucination or a knowledge deficiency hallucination.

[0031] Specifically, see Figure 3 As shown, the provided bias hallucination detection method based on hidden layer activation includes the following steps:

[0032] Step 110 , evaluate the belief state of the large language model.

[0033] For example, we can evaluate the belief state of the model about the text it generates. In this article, belief state refers to the uncertainty of the model about the answer, rather than the answerability of the question or the correctness of the model's answer. The construction of the belief state dataset can be completed based on the labels after the belief state evaluation. The evaluation process is as follows: Figure 4 shown.

[0034] For example, the belief state of the model for its own answers is divided into two categories: confident state and uncertain state. The belief state of the model is judged by evaluating the self-consistency of the model's answers. Specifically, for each question q, multiple answers are obtained from the model. In order to balance statistical reliability and computational efficiency, the number of questions and answers is set to 10. In order to automatically process the free-form answers generated by the model, the answer extraction technology in the reading comprehension task is adopted, and the core answers are extracted from the free-form answers using the debta-v3-large model fine-tuned on squda2.0. This extraction process can standardize answers of different forms. After extracting the answers, the repetition rate of the answers obtained from multiple answers is calculated, and the frequency of occurrence of the answer a with the highest repetition rate, freq(a), is used as a measure of the model's confidence in the question. For example, the determination of the belief state is based on the following formula:

[0035]

[0036] Where freq(a) is the number of times answer a appears in 10 answers. s is the belief state of the model for question q. In order to distinguish the belief state more accurately, two thresholds k and m (k>m) are introduced: if the answer generated by the model for a certain question is highly consistent, it means that the model has high confidence and stability in its own answer and is in the confident state (S con ); On the contrary, if the answers are scattered and lack consistency, it means that the model is highly uncertain about its own answers and is in an uncertain state (S unc ).

[0037] Step 120 , constructing a training set using the obtained belief evaluation results, and then training a belief state classifier, wherein the training set reflects the corresponding relationship between hidden layer activation and belief state.

[0038] Based on the belief state evaluation results of step 1, we first construct a training set and then train the belief state classifier. In order to obtain the hidden layer activation of the model during the generation process, we concatenate the question with the model’s answer and select the hidden layer activation of the i-th token in the l-th layer, expressed as Where l is the layer number and d is the dimension of the hidden layer activation. The hidden layer activation is used as the input of the classifier, and the corresponding belief state s i As labels, we get the dataset used to train the belief state classifier

[0039] In the main experiment, the last hidden layer activation of the model is used as the reference feature. The basic rule for this choice is that the higher layers of the model, especially those close to the output layer, usually contain richer semantic information. Considering that different layers may contain different levels of semantic information, the last few hidden layers of the model are selected as candidates, and the effects of these layers are compared through ablation experiments.

[0040] After the training set is constructed, it can be used to train the belief state classifier. For example, the belief state classifier uses a three-layer feedforward neural network structure, with 256, 128, and 64 hidden layer neurons, and uses the ReLU activation function. The belief state classifier is trained using the training set, and the input of the classifier is the hidden layer activation value matrix The output layer uses a sigmoid function, and the output is a binary label (confident / uncertain), which represents the belief state of the model. It should be noted that for the same question, different models will have different answers and belief states. Therefore, it is necessary to construct a model-specific dataset and use the corresponding classifier for training and testing.

[0041] Step 130 , detect bias hallucinations by analyzing the belief state of the large language model when generating answers.

[0042] The core idea of ​​bias hallucination detection is to distinguish bias hallucinations from knowledge deficiency hallucinations by analyzing the belief state of the model when generating answers. The detection process combines the belief state of the model with advanced hallucination detection methods to distinguish whether the cause of the model's hallucination is bias or lack of knowledge by judging the category of the hallucination. The following algorithm shows the process of bias hallucination detection.

[0043]

[0044] Specifically, bias illusion detection mainly includes the following steps:

[0045] S1: Get model answers and hidden layer activations.

[0046] For example, taking question q as input, we obtain the model’s hidden layer activation h while generating the answer a (see rows 1 and 2).

[0047] S2: Input the question q and the answer a generated by the model into the hallucination detection model to determine whether the answer a generated by the model is correct (see line 3). At the same time, use the hidden layer activation h in the model generation process as the input of the belief state classifier T to determine whether the belief state of the model is confident or uncertain (see line 4).

[0048] S3: For the detected hallucination R, the hallucination type v is judged in combination with the model’s belief state about its own answer (see lines 5 to 14).

[0049] For example, when the model is in the confident state (s con ), the model expresses high confidence in its answer even though the output may be wrong. The error generated in this case is called bias illusion. This type of bias can arise from many factors, such as flawed training data, spurious correlations, or flaws in the model generation strategy. When the model is in an uncertain state (s unc ), it indicates that the model may lack relevant knowledge in this field and is in an unknown state when answering questions, thus generating wrong answers. This type of hallucination belongs to the knowledge deficiency hallucination.

[0050] As an important evaluation indicator for large model output, the hallucination type detection results will guide the model's training, fine-tuning, and reasoning. In the pre-training and fine-tuning stages, the data cleaning process and training strategy can be guided according to the ratio of the two types of hallucinations; in the reasoning stage, the hallucination type evaluation technology will be widely used in tasks such as retrieval enhancement generation and complex logical reasoning, such as external knowledge selection for assisting retrieval enhancement generation to improve generation quality; or in the process of logical reasoning, the correctness of the reasoning process is evaluated to improve the final result.

[0051] It should be understood that, without violating the spirit and scope of the present invention, those skilled in the art may make appropriate changes or modifications to the above embodiments. For example, the belief state classifier may use other types of neural networks. For another example, the number of hidden layers, the number of neurons, the type of activation function, the number of hallucination categories, the number of categories of belief states, etc. can all be determined according to actual needs.

[0052] Furthermore, in order to verify the effectiveness of the bias illusion detection method based on belief state provided by the present invention, comparative experiments were conducted with other methods, and the AUC (area under the curve) evaluation method was used to evaluate the bias illusion detection effect.

[0053] In the comparative experiment, TrivialQA, NQOPEN, and ALCUNA data were used as data sets. The present invention was compared with several other bias hallucination detection methods on eight models and three question-answering data sets. The results are shown in Table 1 below.

[0054] Table 1: Comparative experimental results

[0055]

[0056] It can be seen from the comparative experimental results that among all the models, the BABID proposed by the present invention is always better than the baseline method in the three data sets under different evaluation indicators, which shows that LLMs do have different belief states for hallucination answers. The hidden layer activation of LLMs can effectively judge the belief state of the model, thereby judging the type of hallucination. In addition, when the parameters of the classifier remain unchanged, for the Gemma series of models, the larger the model scale, the higher the accuracy, which may be due to the fact that the hidden layer activation of the larger-scale model has richer features. For the llama series of models, the larger the model scale, the lower the accuracy, which may be because after exceeding a certain scale, the current classifier cannot make full use of more complex internal features.

[0057] In order to better explain the effectiveness of the relevant strategies proposed in this invention in the bias hallucination detection task, an ablation experiment was conducted on the model. For the belief state detection module, the variables related to threshold selection and hidden layer activation selection were controlled for experimental analysis. See Table 2.

[0058] Table 2: Ablation experiment results

[0059]

[0060] Each data in Table 2 represents the AUC scores of tokens at different positions on hidden layer activations of different layers, using data generated by llama3-8B-Instruct on the NQOPEN dataset. In the experiment, we selected the token at the end of the question sequence (Qend) and the token at the end of the entire sequence (Aend), and focused on the hidden layer close to the output layer. The results show that the effect of tokens at the same position is slightly different between different layers, and the classification accuracy of tokens at different positions is significantly different. The last token (Aend) performs best, probably because its hidden layer activation retains more information related to the belief state. To explain the role of the threshold, different belief state thresholds are selected for experiments, and the effect is compared. Figure 5 As shown in the figure, as the gap between the k and m thresholds increases, the AUC of the classifier increases significantly, indicating that in the dataset with a larger gap between the thresholds, the data of the two belief states (bias illusion and knowledge deficiency illusion) show better discrimination, which directly improves the accuracy of bias illusion detection. Therefore, the consistency of the answer reflects the belief state of the model to a certain extent, and the stronger the consistency, the higher the degree of confidence of the model in the answer.

[0061] In order to explain the application value of the bias illusion detection method proposed in this invention, the proportion of bias illusions of different models on different data sets is counted. The number of model answers of three categories, bias illusions, unknown illusions, and correct answers, is counted, and the proportion of different categories in the entire data set is compared, such as Figure 6As shown. For each model, the proportion of bias hallucinations to illusions is relatively small, and there is no obvious relationship between the proportion of bias hallucinations and the model size, indicating that the occurrence of bias hallucinations is more strongly correlated with the quality of the training corpus. In addition, the proportion of bias hallucinations is usually lower for models with high accuracy. On this basis, the hidden layer activations of the knowledge deficiency illusion and bias hallucination in the present invention are visualized, and the final hidden layer of the last generated token is embedded in a 3-D plane to perform PCA projection. As shown Figure 7 As shown, it can be observed that the boundary between the bias illusion (represented by pink dots in the figure) and the correct answer is not obvious, but the boundary with the knowledge deficiency illusion is clearer. In addition, there is a boundary in the space between the correct answer (represented by the blue dots in the figure) and the other two types of answers, but it is not obvious. This positioning shows that the internal characteristics of the model can distinguish the belief state to a certain extent, and the bias illusion and the correct answer have a strong similarity in the belief state. This distinction proves the effectiveness of the present invention.

[0062] In summary, the present invention provides a bias hallucination detection method based on hidden layer activation. The method generally includes: using the hidden layer activation of a large model, constructing a belief state classification data set based on an existing benchmark, and training a model-specific belief state classifier; using an external knowledge retrieval hallucination detection method to judge the model output quality, and obtaining the hidden layer activation in the model generation process in real time; using the model hidden layer activation to judge the belief state, and judging the hallucination category in combination with the model generation quality. The present invention performs belief state judgment based on the model hidden layer activation, and combines the text quality from multiple angles to evaluate the accuracy of bias hallucination detection. Compared with the prior art, the present invention has a greater advantage in capturing the inherent bias errors of the model, and the belief state judgment method based on hidden layer activation can make full use of the internal features in the model generation process, thereby improving the hallucination detection effect.

[0063] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0064] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0065] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0066] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0067] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0068] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0069] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0070] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0071] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A bias hallucination detection method based on hidden layer activation, comprising the following steps: Evaluate the belief state according to the behavioral characteristics of the language model's answer, and obtain a belief state evaluation result, wherein the belief state reflects the uncertainty degree of the language model for the answer; Constructing a training set based on the belief state evaluation result and training a belief state classifier, wherein the training set reflects the corresponding relationship between the belief state of the language model and the hidden layer activation; Acquire the hidden layer activation in the target language model generation process in real time, and judge the belief state using the trained belief state classifier according to the hidden layer activation; For the detected hallucination, the hallucination category is determined by combining the output correctness of the target language model and the belief state determined by the belief state classifier.

2. The method according to claim 1, characterized in that The belief state of the language model is evaluated according to the following steps: For a given question q, obtain multiple free-format answers from the language model; Extracting answers from the multiple free-format answers and calculating the repetition rate of the answers; The frequency of occurrence of the answer a with the highest repetition rate is used as a measure of the confidence of the language model for the question, and the belief state is determined according to the following formula: Where freq(a) is the number of times answer a appears in the total number of set answers, s is the belief state of the language model for question q, k and m are set thresholds, k>m, S con Indicates a state of confidence, S unc Indicates an uncertain state.

3. The method according to claim 1, characterized in that The belief state classifier is a feedforward neural network, including an input layer, multiple hidden layers and an output layer, the multiple hidden layers use the ReLU activation function, the output layer uses the sigmoid function, and the output is a binary label reflecting the belief state, including a confident state and an uncertain state.

4. The method according to claim 1, characterized in that: The training set is constructed according to the following steps: Concatenate the question with the answer of the language model, select the i-th token in the hidden layer activation of the l-th layer, and express it as Where l is the layer number, d is the dimension of hidden layer activation; Hidden layer activation As the input of the belief state classifier, the corresponding belief state s i As the label, the training set is obtained, which is expressed as N is the number of samples in the training set.

5. The method according to claim 4, characterized in that The training set is constructed using the last hidden layer activation of the language model as a reference feature.

6. The method according to claim 1, characterized in that The correctness of the output of the target language model is determined by comparing the response of the target language model with the correct answer by retrieving external knowledge.

7. The method according to claim 3, characterized in that The multi-layer hidden layer is set to three layers, and the number of neurons is set to 256, 128 and 64 respectively.

8. The method according to claim 1, characterized in that The categories of hallucinations include illusions of bias or illusions of lack of knowledge.

9. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images