A bias hallucination detection method based on hidden layer activations

Abstract

The application discloses a bias hallucination detection method based on hidden layer activation. The method comprises the following steps: evaluating a belief state according to the behavior characteristics of a language model answer, obtaining a belief state evaluation result, and reflecting the uncertainty degree of the language model to the answer; constructing a training set based on the belief state evaluation result, training a belief state classifier, and reflecting the corresponding relationship between the belief state of the language model and the hidden layer activation; obtaining the hidden layer activation in the generation process of a target language model in real time, and judging the belief state by using the trained belief state classifier according to the hidden layer activation; and judging the hallucination category by combining the output correctness of the target language model and the belief state judged by the belief state classifier for the detected hallucination. The application judges the belief state based on the model hidden layer activation, and improves the accuracy of the bias hallucination detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of large model hallucination detection, and more particularly, to a bias hallucination detection method based on hidden layer activation. BACKGROUND

[0002] Hallucination in large language models (LLMs) refers to the phenomenon where LLMs generate coherent but factually inaccurate responses. Existing LLM hallucination detection focuses on identifying potential factual errors in the output. Hallucination detection methods include methods that retrieve external knowledge and methods based on model uncertainty. Methods that retrieve external knowledge verify the correctness of the model output by external knowledge sources or reference answers. If the output of the model does not match the objective facts, it is considered a potential hallucination, such as detecting factual errors by collecting evidence using external tools. Methods based on model uncertainty do not rely on external knowledge, and detect hallucinations by estimating the uncertainty of the model output.

[0003] Existing research has shown that the hidden layer activation of LLMs encapsulates internal beliefs related to the uncertainty of the statements they generate. These hidden layer activations can serve as an effective basis for hallucination detection. Although current hallucination detection methods have made significant progress in identifying incorrect information in LLM outputs, there are still limitations. First, whether it is a method that retrieves external knowledge or a method based on model uncertainty, existing work only considers the correctness of the model output, and lacks attention to the causes of hallucination. These works usually focus on detecting the presence of hallucinations, ignoring the differences between hallucinations. For example, methods based on model uncertainty assume that high model certainty means accurate output, but high model certainty does not guarantee accurate output, because the model itself may have incorrect biases. These limitations not only hinder the reliability of hallucination detection for LLMs, but also limit the possibility of targeted improvement of model performance. Therefore, it is necessary to consider different bases to improve the quality of hallucination detection work, such as the correctness of the model output and the model's confidence in its own answers. SUMMARY

[0004] The purpose of the present application is to overcome the above-mentioned deficiencies of the prior art and provide a bias hallucination detection method based on hidden layer activation. The method comprises the following steps:

[0005] Evaluating the belief state according to the behavioral characteristics of the language model answer, obtaining a belief state evaluation result, the belief state reflecting the uncertainty of the language model in answering;

[0006] Based on the belief state evaluation result, a training set is constructed, and a belief state classifier is trained, the training set reflecting the correspondence between the belief state of the language model and the hidden layer activation;

[0007] obtaining the hidden layer activation in a target language model generation process in real time, and judging the belief state by using the trained belief state classifier according to the hidden layer activation;

[0008] For the detected hallucination, the hallucination category is judged in combination with the output correctness of the target language model and the belief state judged by the belief state classifier.

[0009] Compared with the prior art, the advantage of the present application is that, compared with the hallucination detection method based on uncertainty and the hallucination detection method based on retrieval of external knowledge, the present application provides a hallucination detection method based on belief state, which solves the technical problem of capturing the belief state of a large model. The present application uses a large model and a feedforward neural network (FNN) as core components, combines the correctness of the model output and the degree of confidence in the output of the model, and makes a more detailed division of hallucinations, so as to finally obtain a hallucination detection result with fine granularity and high reliability.

[0010] Other features and advantages of the present application will become apparent from the following detailed description of exemplary embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings incorporated in and forming a part of the specification illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0012] Figure 1 is a hallucination category distinguishing schematic diagram according to an embodiment of the present application;

[0013] Figure 2 is a bias hallucination detection flowchart according to an embodiment of the present application;

[0014] Figure 3 is a bias hallucination detection method based on hidden layer activation flowchart according to an embodiment of the present application;

[0015] Figure 4 is a cognitive state evaluation and cognitive state training set construction flowchart according to an embodiment of the present application;

[0016] Figure 5 is an ablation experiment schematic diagram according to an embodiment of the present application;

[0017] Figure 6 is a data distribution statistical sample schematic diagram according to an embodiment of the present application;

[0018] Figure 7 is a model hidden layer activation visualization sample schematic diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the present application unless specifically stated otherwise.

[0020] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the scope of the application its application or uses.

[0021] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein. However, where appropriate, such techniques, methods, and apparatus should be considered as being part of the specification.

[0022] In all of the examples shown and discussed herein, any specific values should be interpreted as merely exemplary, and not as a limitation. Thus, other examples of exemplary embodiments can have different values.

[0023] It is to be understood that the drawings shown in the following embodiments are only schematic and are non-limiting examples of embodiments of the application, as a result, the present application can have any other desired shape and form, and the number, shape, and size of components shown in the drawings can be changed as desired, and the layout of components can be more complex.

[0024] It should be noted that like numbers and letters refer to like items throughout the drawings, and once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0025] In general, the provided bias hallucination detection method based on hidden layer activation includes: belief state evaluation according to the behavior characteristics of the model answer; constructing a belief state classifier based on hidden layer activation according to the model hidden layer activation and the belief state evaluation result; using hallucination evaluation means based on external knowledge to judge the correctness of the model output, and obtaining the hidden layer activation of the model generation process in real time, judging the belief state according to the model hidden layer activation; combining the output correctness and the belief state to judge the hallucination category.

[0026] In one embodiment, the belief state evaluation includes: inputting the question text q into the model M to obtain the model response a; sampling multiple model responses and extracting effective keywords in the response using an answer extraction model; calculating the response consistency and evaluating the model belief state.

[0027] In one embodiment, constructing the belief state classifier includes: for each question q in the existing open domain question answering benchmark, obtaining the model response a, and obtaining the model hidden layer activation in real time during the model generation process; evaluating the model response consistency, calculating the belief state label s iAnd a belief state classification training set is constructed in combination with model hidden layer activation, a small feedforward neural network is trained by using the constructed training set, and a belief state classifier is obtained.

[0028] In one embodiment, according to the model belief state and the output correctness, the bias hallucination detection includes: for a given question, a response is given by the model, and the hidden layer activation h of the model generation process is obtained; based on the method of retrieving external knowledge, the model response is compared with the correct answer to evaluate the quality of the model output; the belief state of the model to its own answer is judged by the belief state classifier by using the hidden layer activation of the model; for the detected hallucination r, the hallucination type v is judged in combination with the belief state of the model.

[0029] In the following, the detailed embodiments of the application will be specifically introduced.

[0030] The application explores the bias hallucination detection task of large models, and proposes an effective bias hallucination detection model based on hidden layer activation. Referring to Figure 1 As shown, in order to judge the reason of LLMs generating hallucination, the bias hallucination detection task is defined as a binary classification problem, that is, to judge whether the hallucination generated by the large language model (LLMs) is a bias hallucination or a knowledge deficiency hallucination. Based on this goal, a bias hallucination detection framework BABID based on belief state is proposed. As shown, Figure 2 The framework includes two core modules: a belief state classifier and an evidence-based hallucination detection module. For each question in the dataset, the model extracts the hidden layer activation in the model generation process while answering. The evidence-based hallucination detection module uses the method of large model evaluation, such as using ChatGPT to judge the correctness of the model answer. The module takes the question and the model answer as input, and outputs the hallucination detection result. The belief state classifier uses the trained classifier model to take the hidden layer activation in the generation process as input, and outputs the belief state of the model, such as including the confident state or the uncertain state. Finally, BABID classifies the hallucination as a bias hallucination or a knowledge deficiency hallucination in combination with the hallucination detection result and the belief state.

[0031] Specifically, referring to Figure 3 As shown, the provided bias hallucination detection method based on hidden layer activation includes the following steps:

[0032] Step 110, evaluating the belief state of the large language model.

[0033] For example, the belief state of the model to the generated text is evaluated. In this paper, the belief state refers to the uncertainty of the model to the answer, rather than the answerability of the question or the correctness of the model answer. The construction of the belief state dataset can be completed based on the label after the belief state evaluation. The evaluation process is as shown in Figure 4 As shown.

[0034] For example, the belief state of the model answering itself is divided into two categories: a confident state and an uncertain state. The belief state of the model is determined by evaluating the self-consistency of the answers of the model. Specifically, for each question q, multiple answers are obtained from the model. To balance statistical reliability and computational efficiency, the number of answers is set to 10. In order to automatically process the free-form answers generated by the model, the answer extraction technology in the reading comprehension task is adopted, and the deberta-v3-large model fine-tuned on squda2.0 is used to extract the core answer from the free-form answer, and this extraction process can standardize different forms of answers. After extracting the answer, the repetition rate of the answers obtained from multiple answers is calculated, and the frequency freq(a) of the answer a with the highest repetition rate is taken as the measure of the confidence of the model for the question. For example, the determination of the belief state is based on the following formula:

[0035]

[0036] wherein freq(a) is the number of occurrences of answer a in 10 answers. s is the belief state of the model for question q. In order to more accurately distinguish the belief state, two thresholds k and m (k > m) are introduced: if the answers generated by the model for a question are highly consistent, indicating that the model has high confidence and stability in answering itself, it is in a confident state (S con ); on the contrary, if the answers are scattered and lack consistency, indicating that the model has a high degree of uncertainty in answering itself, it is in an uncertain state (S unc ).

[0037] Step 120, using the obtained belief evaluation results to construct a training set, and then training a belief state classifier, the training set reflecting the correspondence between the hidden layer activation and the belief state.

[0038] Based on the belief state evaluation results of step 1, first, a training set is constructed, and then a belief state classifier is trained. In order to obtain the hidden layer activation of the model in the generation process, the question and the answer of the model are spliced, and the hidden layer activation of the ith token in the lth layer is selected, denoted as wherein l is the layer number, and d is the dimension of the hidden layer activation. Taking the hidden layer activation as the input of the classifier, the corresponding belief state s i is taken as the label, and the data set for training the belief state classifier is obtained

[0039] In the main experiment, the last layer of the model's hidden layer activation is used as the reference feature. The basic principle of this selection is that the high layers of the model, especially the part close to the output layer, usually contain richer semantic information. Considering that different layers may contain different levels of semantic information, the last few hidden layers of the model are selected as candidates, and the effects of these layers are compared through ablation experiments.

[0040] After the training set is constructed, it can be used to train the belief state classifier. For example, the belief state classifier adopts a three-layer feedforward neural network structure, with 256, 128, and 64 hidden layer neurons, respectively, using ReLU activation function. The training set is used to train the belief state classifier, and the input of the classifier is the hidden layer activation value matrix The output layer uses a sigmoid function, and the output is a binary label (confident / uncertain) representing the belief state of the model. It should be noted that for the same question, different models will have different answers and belief states, so a model-specific dataset needs to be constructed and the corresponding classifier needs to be trained and tested.

[0041] Step 130, detecting bias hallucination by analyzing the belief state of large language models when generating answers.

[0042] The core idea of bias hallucination detection is to distinguish between bias hallucination and knowledge deficiency hallucination by analyzing the belief state of the model when generating answers. The detection process combines the model's belief state and advanced hallucination detection methods to distinguish between bias and knowledge deficiency as the reason for the model's hallucination. The following algorithm shows the process of bias hallucination detection.

[0043]

[0044] Specifically, bias hallucination detection mainly includes the following steps:

[0045] S1: Obtain the model answer and hidden layer activation.

[0046] For example, taking question q as input, obtain the hidden layer activation h of the model while generating answer a (see lines 1 and 2).

[0047] S2: Input the question q and the answer a generated by the model to the hallucination detection model to determine whether the answer a generated by the model is correct (see line 3). At the same time, use the hidden layer activation h in the model generation process as the input of the belief state classifier T to determine whether the belief state of the model is confident or uncertain (see line 4).

[0048] S3: For the detected hallucination R, the model judges the hallucination type v of the belief state of its own answer (see lines 5-14).

[0049] For example, when the model is in the confident state (s con ), it still shows high confidence in its answer even if the output can be wrong. The error produced in this case is called a biased hallucination. Such bias can be caused by multiple factors, such as flawed training data, spurious correlations, or defects in the model generation strategy. When the model is in the uncertain state (s unc ), it indicates that the model may lack relevant knowledge in this field and is in an unknown state when answering questions, thus generating an incorrect answer. This type of hallucination is called a knowledge deficiency hallucination.

[0050] As an important evaluation indicator of large model output, the hallucination type detection result will guide the training, fine-tuning, and reasoning of the model. In the pre-training and fine-tuning stages, the proportion of the two types of hallucinations can guide the data cleaning process and training strategy; in the reasoning stage, the hallucination type evaluation technology will be widely used in tasks such as retrieval augmented generation and complex logical reasoning, such as assisting external knowledge selection for retrieval augmented generation to improve generation quality; or in the process of logical reasoning, evaluating the correctness of the reasoning process to improve the final result.

[0051] It should be understood that the above embodiments can be appropriately changed or modified by those skilled in the art without departing from the spirit and scope of the present application. For example, the belief state classifier can use other types of neural networks. For example, the number of hidden layers, the number of neurons, the type of activation function, the number of hallucination categories, and the number of belief state categories can be determined according to actual needs.

[0052] Further, in order to verify the effectiveness of the belief state-based biased hallucination detection method provided by the present application, comparative experiments were conducted with other methods, and the AUC (area under the curve) evaluation method was used to evaluate the biased hallucination detection effect.

[0053] In the comparative experiment, TrivialQA, NQOPEN, and ALCUNA data were used as data sets, and the present application was compared with several other biased hallucination detection methods on eight models and three question and answer data sets. The results are shown in Table 1 below.

[0054] Table 1: Results of comparative experiment

[0055]

[0056] From the comparison experiment results, it can be seen that in all models, the BABID proposed in the application is always better than the baseline method in different evaluation indexes in three data sets, which shows that the LLMs have different belief states for hallucinated answers, and the hidden layer activation of the LLMs can effectively judge the belief state of the model, thereby judging the hallucination type. In addition, under the condition that the parameters of the classifier are unchanged, for the models of the Gemma series, the larger the model size, the higher the accuracy, which may be due to the fact that the hidden layer activation of the larger model has more rich features. For the models of the llama series, the larger the model size, the lower the accuracy, which may be because after exceeding a certain size, the current classifier cannot fully utilize the more complex internal features.

[0057] In order to better explain the effectiveness of the related strategy proposed in the application in the bias hallucination detection task, an ablation experiment is performed on the model. For the belief state detection module, the variables related to threshold selection and hidden layer activation selection are controlled respectively for experimental analysis. Referring to Table 2.

[0058] Table 2: Ablation experiment results

[0059]

[0060] Each piece of data in Table 2 represents the data generated by using llama3-8B-Instruct on the NQOPEN data set, and the AUC score of the token at different positions on the hidden layer activation of different layers. In the experiment, the token at the end of the question sequence (Qend) and the token at the end of the entire sequence (Aend) are selected, and the hidden layer close to the output layer is focused on. The results show that the effect of the same position token in different layers is small, and the classification accuracy of different position tokens is significantly different, and the last token (Aend) performs best, which may be because its hidden layer activation retains more information related to the belief state. In order to explain the effect of the threshold, different belief state thresholds are selected for experiments, and the effect comparison is shown in Figure 5 As k and m threshold difference increases, the AUC of the classifier significantly improves, indicating that the data with larger threshold difference between the two belief states (bias hallucination and knowledge deficiency hallucination) shows better discrimination, which directly improves the accuracy of bias hallucination detection. Therefore, the consistency of the answer reflects the belief state of the model to some extent, and the stronger the consistency, the higher the confidence of the model to the answer.

[0061] In order to explain the application value of the bias hallucination detection method proposed in the application, the proportion of bias hallucination of different models on different data sets is counted. The number of model answers of three types of models, bias hallucination, unknown hallucination and correct answer, is counted, and the proportion of different categories in the whole data set is compared, as shown in Figure 6As shown. For each model, the proportion of bias hallucination is smaller than the proportion of hallucination, and the proportion of bias hallucination and the model size have no obvious relationship, which shows that the occurrence of bias hallucination has a stronger correlation with the quality of the training corpus. In addition, the proportion of bias hallucination is usually lower for the model with high accuracy. On this basis, the hidden layer activation of knowledge deficiency hallucination and bias hallucination in the present application is visualized, and the final hidden layer embedding of the last generated token is performed PCA projection on the 3-D plane. As shown in the figure, it can be observed that the boundary between bias hallucination (represented by pink dots in the figure) and correct answer is not obvious, but the boundary with knowledge deficiency hallucination is relatively clear. In addition, the correct answer (represented by blue dots in the figure) and the other two types of answers have boundaries in the space, but they are not obvious. This positioning shows that the internal features of the model can distinguish the belief state to some extent, and the bias hallucination and the correct answer have strong similarity in the belief state. This distinction proves the effectiveness of the present application. Figure 7 As shown, it can be observed that the boundary between bias hallucination (represented by pink dots in the figure) and correct answer is not obvious, but the boundary with knowledge deficiency hallucination is relatively clear. In addition, the correct answer (represented by blue dots in the figure) and the other two types of answers have boundaries in the space, but they are not obvious. This positioning shows that the internal features of the model can distinguish the belief state to some extent, and the bias hallucination and the correct answer have strong similarity in the belief state. This distinction proves the effectiveness of the present application.

[0062] In summary, the present application provides a bias hallucination detection method based on hidden layer activation. The method generally includes: using the hidden layer activation of the large model, constructing the belief state classification data set based on the existing benchmark, and training the belief state classifier specific to the model; using the hallucination detection method of external knowledge retrieval to judge the quality of the model output, and obtaining the hidden layer activation in the model generation process in real time; using the model hidden layer activation to judge the belief state, and combining the model generation quality to judge the hallucination category. The present application judges the belief state based on the model hidden layer activation, and combines the multi-angle text quality evaluation to improve the accuracy of bias hallucination detection. Compared with the prior art, the present application has great advantages in capturing the inherent bias errors of the model, and the belief state judgment method based on the hidden layer activation can fully utilize the internal features in the model generation process, thereby improving the hallucination detection effect.

[0063] The present application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon, which are used to enable a processor to implement various aspects of the present application.

[0064] A computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0065] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0066] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0067] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0068] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0069] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0070] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0071] Embodiments of the present application have been described above, and the description is intended to be illustrative, and not restrictive, of the disclosed embodiments. Many modifications and variations of the disclosed embodiments are possible in light of the above teachings. It is therefore to be understood that within the scope of the disclosed embodiments, modifications and variations of the disclosed embodiments can be practiced. It is also to be understood that the specific order or hierarchy of steps in the processes disclosed is an illustration of exemplary processes. Based upon the description and illustrations provided herein, those skilled in the art will understand that changes can be made to the order of steps in the processes and that many of the individual steps can be modified or eliminated. Additionally, the description and illustrations provided herein are not meant to limit the scope of the disclosed embodiments. The scope of the disclosed embodiments is limited only by the claims.

Claims

1. A bias hallucination detection method based on hidden layer activation, comprising the following steps: behavioral feature evaluation of belief state according to the answer of the language model, obtaining belief state evaluation results, the belief state reflecting the uncertainty of the language model to the answer; constructing a training set based on the belief state evaluation results, and training a belief state classifier, the training set reflecting the corresponding relationship between the belief state of the language model and the hidden layer activation; real-time acquisition of hidden layer activation in the generation process of the target language model, and judgment of the belief state by using the trained belief state classifier according to the hidden layer activation; for the detected hallucination, combining the output correctness of the target language model with the belief state judged by the belief state classifier to judge the hallucination category; wherein the belief state of the language model is evaluated according to the following steps: For the set question obtain a free-form multiple answer from the language model; extracting answers from the free-form multiple answers and calculating the repetition rate of the answers; With the highest repetition rate answer The frequency of occurrence as a measure of the language model's confidence in the answer to the question, and the belief state is judged according to the following formula: wherein, is the answer is the number of occurrences in the set of answers, is the belief state of the language model for the question , and is a set threshold, > , represents a confident state, represents an uncertain state; wherein the training set is constructed according to the following steps: concatenating the question with the language model's answer, selecting the i first token's hidden layer activation at the l first layer, denoted as where is the layer number, is the dimension of the hidden layer activation; with hidden layer activations as the belief state classifier, a corresponding belief state as labels, the training set is obtained, denoted as N is the number of samples in the training set.

2. The method of claim 1, wherein, The belief state classifier is a feedforward neural network, including an input layer, multiple hidden layers and an output layer, the multiple hidden layers use ReLU activation function, the output layer uses sigmoid function, and the output is a binary label reflecting the belief state, including self-confidence state and uncertain state.

3. The method of claim 1, wherein, The last layer of the hidden layer activation of the language model is used as the reference feature to construct the training set.

4. The method of claim 1, wherein, The output correctness of the target language model adopts the method of retrieving external knowledge, which is determined by comparing the response of the target language model with the correct answer.

5. The method of claim 2, wherein, The multiple hidden layers are set to three layers, and the number of neurons is set to 256, 128 and 64 respectively.

6. The method of claim 1, wherein, The hallucination category includes bias hallucination or knowledge deficiency hallucination.

7. A computer readable storage medium having stored thereon a computer program, wherein, The computer program is executed by the processor to realize the steps of the method according to any one of claims 1 to 6.

8. A computer device comprising a memory and a processor, having stored on the memory a computer program capable of running on the processor, characterized in that, The processor executes the computer program to realize the steps of the method according to any one of claims 1 to 6.

Images