Knowledge cross validation question and answer method and system for reducing illusion of large language model

Abstract

The invention discloses a knowledge cross validation question-answering method and system for reducing hallusion of a large language model, and belongs to the technical field of artificial intelligence, and the method is implemented by the following steps: generating results through multiple times of sampling: when a user puts forward a question, controlling the large model to perform multiple times of sampling, and generating a specified number of results; calculating hidden state related indexes: extracting the hidden state of the last token of the middle layer of the large model corresponding to the result, and calculating covariance matrixes and answer discrete feature values of the hidden states; mLP model prediction: inputting the discrete feature value of the answer and the length of the answer into a multilayer perceptron MLP, and outputting a hallucination-free probability; querying and summarizing a knowledge graph; and calculating a final illusion-free score and outputting a result. According to the method, the answer quality and credibility of a large language model can be remarkably improved, and the method is particularly suitable for application scenes with extremely high requirements on the accuracy of single-mode text generation contents.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a knowledge cross-validation question and answer method and system for reducing hallucination of a large language model. BACKGROUND

[0002] In the field of artificial intelligence and natural language processing (NLP) today, large language models (LLMs) such as GPT-3, LLaMA, etc. have shown great application potential in many fields due to their powerful text generation, understanding and reasoning capabilities. However, despite the remarkable progress of LLMs, their inherent "hallucination" problem (i.e. the model generates seemingly reasonable but actually inconsistent with facts or logically contradictory content) remains a key challenge that hinders their widespread application and credibility. The hallucination problem not only reduces the reliability of the LLM-generated information, but also can lead to incorrect decisions, especially in professional fields (such as medicine, law, finance, etc.) where accuracy is highly required, its harm is more prominent.

[0003] To alleviate the hallucination problem of LLMs, existing technologies have proposed various methods. For example, the Retrieval-Augmented Generation (RAG) system combines the retrieval capabilities of external knowledge bases to improve the accuracy of generated content to some extent. However, the RAG system mainly focuses on providing external information for the model to reference, and it cannot directly judge or suppress the potential hallucinations generated by the model during the generation process. In addition, some methods attempt to detect hallucinations by analyzing the model's output results. For example, some technologies use perplexity to evaluate the fluency of generated text, considering that low perplexity may indicate higher credibility. Other methods judge whether there is hallucination by comparing the consistency of the model's multiple generation results, considering that highly consistent output may be more reliable. However, these output result-based analysis methods have obvious shortcomings. Perplexity alone can only reflect the grammatical and semantic fluency of the language model, and cannot effectively capture factual errors or deep logical contradictions. Simple consistency checks may overlook subtle semantic differences in the model, or when faced with complex and open-ended questions, even if the model produces hallucinations, the multiple generation results may exhibit surface consistency. These methods often fail to capture the deep reasons for hallucination within the model, and lack an effective mechanism for cross-verification of generated content with external facts, making it difficult to comprehensively and accurately identify and suppress the hallucinations of LLMs.

[0004] These challenges are particularly prominent in some applications that require high accuracy and credibility. For example, in professional question and answer systems, users expect LLMs to provide accurate and error-free answers, and any hallucination can lead to serious consequences. Existing hallucination detection methods often fail to provide satisfactory solutions when faced with such tasks, and cannot effectively guarantee the authenticity and reliability of the generated content. SUMMARY

[0005] The technical task of the present application is to provide a knowledge cross-validation question and answer method and system for reducing hallucination of a large language model, which can significantly improve the answer quality and credibility of a large language model, and is especially suitable for application scenarios with extremely high requirements for the accuracy of single-modal text generation content.

[0006] The technical solution adopted by the present application to solve its technical problems is:

[0007] A knowledge cross-validation question and answer method for reducing hallucination of a large language model, the implementation of the method includes the following steps:

[0008] Multiple sampling generation results: when a user raises a question, control the large model to perform multiple sampling to generate a specified number of results;

[0009] Calculate hidden state related indicators: extract the hidden state of the last token of the intermediate layer of the large model corresponding to the results, and calculate the covariance matrix and answer discrete eigenvalue (ADEV) of the hidden state;

[0010] MLP model prediction: input the answer discrete eigenvalue and the length of the answer into a multilayer perceptron (MLP) as input, output the probability of no hallucination, and the hidden layer has a specific structure and number of neurons;

[0011] Knowledge graph query and summary: extract the entities involved in the model answer by the hallucination checking agent, drive the knowledge graph query module to query information, and summarize the queried information; perform embedding processing on the summary of the agent and the answer of the large model, and calculate the cosine similarity between them; the similarity is used to measure the matching degree of the answer and the factual information in the knowledge graph;

[0012] Calculate the final hallucination-free score and output the result: the final hallucination-free score is obtained by weighted summation of the cosine similarity and the hallucination-free probability output by the multilayer perceptron; the content returned to the user is determined according to the comparison result of the score and the threshold.

[0013] The method organically combines multiple sampling generation results, neural network model prediction, knowledge graph query verification, and other technologies to construct a large model question and answer hallucination suppression system, which can effectively reduce hallucination by comprehensively evaluating whether the large model can accurately answer user questions, thereby solving the hallucination problem of the large model in the question and answer process and improving the accuracy, relevance, and credibility of the generated content.

[0014] Further, the multilayer perceptron has two hidden layers, each with 8 neurons.

[0015] Further, the content returned to the user is determined according to the comparison result of the score and the threshold value,

[0016] A double threshold mechanism is set, and according to the comparison result of the illusion-free score and the main threshold value, the result generated by the greedy strategy or the expression of "unable to answer this question" is determined to be returned to the user; if the score is within the range of the secondary threshold value, the sample is stored in the database for subsequent analysis.

[0017] Further, the training process of the MLP model is as follows:

[0018] First, use the large model to perform inference on the question and answer data set, collect the corresponding answer of each question and the corresponding answer discrete feature value ADEV;

[0019] Then, use ROUGE-L to determine whether the answer is correct, to determine whether the large model has illusion on the question, and thus construct the {(ADEV, answer length), whether there is illusion} data set;

[0020] Finally, use the data set to train the MLP to obtain a model capable of illusion detection.

[0021] Further, the illusion checking agent is a small parameter illusion checking agent.

[0022] Further, the multiple sampling generation result is that the large model performs 10 independent samplings to generate 10 answer results.

[0023] Further, the answer discrete feature value calculation formula is represented as:

[0024]

[0025] Wherein, n=10, mu i represents the eigenvalue of the regularized covariance matrix C+beta*I n , beta is a small regularization term, I n is an n*n unit matrix.

[0026] The present application also claims a knowledge cross-validation question and answer system for reducing the illusion of a large language model, comprising:

[0027] A multiple sampling generation module: when a user raises a question, control the large model to perform multiple samplings to generate a specified number of results;

[0028] A hidden state analysis module: extract the hidden state of the last token in the middle layer of the large model corresponding to each answer, and calculate the covariance matrix of the hidden state and the answer discrete feature value (ADEV);

[0029] MLP module: taking the answer discrete feature value and the length of the answer as input, outputting the probability of no hallucination;

[0030] Hallucination checking agent module: responsible for extracting the entities involved in the model answer, driving the knowledge graph query module to query information, and summarizing the queried information; performing embedding processing on the summary of the agent and the answer of the large model respectively, and calculating the cosine similarity of the two;

[0031] Result fusion and discrimination module: by weighting and summing the cosine similarity and the no hallucination probability output by the MLP module, the final no hallucination score is obtained; according to the comparison result of the score and the threshold, the content returned to the user is determined;

[0032] The system can realize the above-mentioned knowledge cross-validation question and answer method for reducing hallucination of a large language model.

[0033] The application also claims to protect a knowledge cross-validation question and answer device for reducing hallucination of a large language model, comprising: at least one memory and at least one processor;

[0034] The at least one memory is used to store machine readable programs;

[0035] The at least one processor is used to call the machine readable programs to realize the above-mentioned method.

[0036] The application also claims to protect a computer readable medium, which stores computer instructions, and the computer instructions realize the above-mentioned method when executed by a processor.

[0037] Compared with the prior art, the knowledge cross-validation question and answer method and system for reducing hallucination of a large language model has the following beneficial effects:

[0038] Firstly, the results generated by multiple sampling can fully utilize the internal information in the generation process of the large model, effectively capture the uncertainty and discreteness of the model answer, and provide a reliable basis for subsequent hallucination judgment. Compared with the traditional method of relying only on a single generation result for judgment, this multiple sampling analysis method can more comprehensively reflect the degree of grasping of the model for the problem.

[0039] Secondly, by combining ADEV and the length of the answer and using MLP to predict the probability of no hallucination, the existing features and model capabilities are fully utilized, and the accuracy of probability prediction is improved. ADEV, as an effective discrete feature measurement index, combined with the length of the answer, can characterize the features of the answer from multiple dimensions, making the prediction result of MLP more valuable.

[0040] Further, the hallucination checking intelligent agent and the knowledge graph query verification mechanism are introduced to compare the answer of the large model with the factual information in the knowledge graph, thereby further improving the accuracy of hallucination judgment. By calculating the cos similarity of embedding, the matching degree of the answer and the factual information can be quantified, thereby providing an important basis for the final hallucination-free score.

[0041] Finally, by setting a threshold to determine whether to return the answer, the hallucination content generated by the large model when it cannot accurately answer the question can be effectively avoided, thereby improving the credibility and reliability of the system output. When the hallucination-free score is higher than the threshold, the high-quality generated result is returned; when the score is lower than the threshold, the user is explicitly informed that the question cannot be answered, thereby enhancing the user's trust in the system. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a knowledge cross-validation question and answer method provided by an embodiment of the present application to reduce hallucination of a large language model.

[0043] Figure 2 is a schematic diagram of a hallucination detection intelligent agent module. DETAILED DESCRIPTION

[0044] The present application will be further described below in conjunction with specific embodiments.

[0045] The embodiment of the present application provides a knowledge cross-validation question and answer method for reducing hallucination of a large language model. By fusing multiple rounds of generation diversity analysis, hidden state semantic feature mining, knowledge graph fact verification, and dynamic threshold judgment, etc. Key modules, accurate hallucination detection and suppression of large model output are realized.

[0046] The implementation of the method includes the following steps:

[0047] Multiple sampling generation results: when a user raises a question, the large model is controlled to be sampled multiple times to generate a specified number of results.

[0048] Calculate the hidden state related indicators: extract the hidden state of the last token in the intermediate layer of the large model corresponding to the result, and calculate the covariance matrix and the answer discrete eigenvalue (ADEV) of the hidden state.

[0049] MLP model prediction: the answer discrete eigenvalue and the length of the answer are input into the multi-layer perceptron MLP, and the probability of hallucination-free is output, and the hidden layer has a specific structure and neuron number.

[0050] Knowledge graph query and summary: the hallucination checking agent module contains a small parameter agent responsible for extracting the entities involved in the model's answer and driving the knowledge graph query module to query information, and summarizing the queried information; after the agent extracts the entity, it retrieves related information in the knowledge graph and returns it to the agent; then the agent's summary and the large model's answer are respectively embedded, and the cosine similarity of the two is calculated; this similarity is used to measure the matching degree of the answer and the factual information in the knowledge graph.

[0051] Calculate the final hallucination score and output the result: the final hallucination score is obtained by weighted sum of the cosine similarity and the hallucination probability output by the multi-layer perceptron; according to the comparison result of the score and the threshold, the content returned to the user is decided. Set double threshold mechanism, according to the comparison result of hallucination score and main threshold, decide to return the result generated by greedy strategy or the expression of "can not answer this question" to the user; if the score is in the range of the secondary threshold, the sample is stored in the database for subsequent analysis.

[0052] The main steps of the specific implementation of the method are as follows:

[0053] 1. Multiple sampling to generate results: when the user asks the large model a question, the backend controls the large model to sample multiple times to generate 10 results.

[0054] 2. Calculate hidden state related indicators: extract the hidden state of the last token in the middle layer of the large model corresponding to each answer; calculate the covariance matrix C of the 10 answer hidden states, where is used to capture the relationship between different answer hidden states; according to the covariance matrix, calculate the answer discrete eigenvalue (ADEV), the calculation of ADEV can refer to the existing method, and its calculation formula is expressed as:

[0055]

[0056] Where, n = 10, μ i represents the eigenvalue of the regularized covariance matrix C+β·I n , β is a small regularization term, I n is an n×n identity matrix.

[0057] 3. MLP model prediction: input ADEV and the length of the answer into MLP to get the probability of no hallucination. The MLP hidden layer has two layers, each with 8 neurons, and the input is ADEV and the length of the output answer, and the output is the probability of positive samples (no hallucination).

[0058] 4. Knowledge graph query and summary: the hallucination detection intelligent agent with small parameters extracts the entities involved in the model's answer, queries the information in the knowledge graph, and summarizes the information queried by the intelligent agent.

[0059] 5. Calculate the cosine similarity: both the summary of the intelligent agent and the answer of the large model are embedded, and the cosine similarity between the two is calculated. This similarity is used to measure the matching degree of the answer and the factual information in the knowledge graph.

[0060] 6. Calculate the final hallucination-free score: the probability obtained in step 3 and the similarity obtained in step 5 are weighted and summed to obtain the final hallucination-free score. This score comprehensively reflects the degree of hallucination-free answer.

[0061] 7. Output results: when the final hallucination-free score is higher than the set threshold, it indicates that the model has the ability to accurately answer the question, and the result generated by the greedy strategy is returned to the user; if the hallucination-free score is lower than the threshold, it indicates that the large model cannot accurately answer the question, and the expression that the model cannot answer the question is returned to the user.

[0062] The training process of the MLP model is as follows:

[0063] First, use the large model to infer on the question and answer data set, collect the corresponding answer and answer discrete feature value ADEV for each question;

[0064] Then, use ROUGE-L to determine whether the answer is correct to determine whether the large model hallucinates on this question, thereby constructing a {(ADEV, answer length), whether hallucination} data set;

[0065] Finally, use the data set to train the MLP to obtain a model capable of hallucination detection.

[0066] This method constructs a robust hallucination detection and suppression mechanism by deeply analyzing the internal state of the large language model (LLM) and the dual dimensions of external knowledge. Not only can it effectively evaluate the internal uncertainty of the model for the current question through deep semantic dispersion analysis of the internal hidden state of the LLM multiple sampling generation results, but also innovatively combines external knowledge graphs to cross-verify key entities involved in the model's answer. By weighting and fusing the internal consistency measure and the external knowledge verification result, the system can more accurately determine whether the LLM's answer contains hallucinations and alert users in a timely manner when the model cannot provide reliable answers, thereby significantly improving the answer quality and credibility of the LLM, especially suitable for scenarios where the accuracy of single-modal text generation content is extremely high, and multiple sources of information, cross-domain knowledge, and strict output authenticity requirements are required for intelligent question and answer scenarios.

[0067] To better understand this method, we will use a financial knowledge question-answering system as an example to illustrate its implementation. This case study will detail how to handle a typical financial knowledge query to suppress the illusions that large models may generate during the answering process.

[0068] Suppose a user submits the following request to the system: "Please explain what interest rate swaps are, and describe their main risk types and management methods." Upon receiving this text query, the system will activate the "Knowledge Cross-Validation Large Model Illusion Suppression System (KCVS)" to process it.

[0069] 1. First, the multi-round sampling generation module begins operation. The system backend controls the large model to perform 10 independent samplings of the financial question, generating 10 responses regarding the definition, risk type, and management methods of interest rate swaps. These results are preliminary responses generated by the large model based on its own knowledge reserves and may contain some uncertainty or speculative content.

[0070] 2. The hidden state extraction module processes the aforementioned 10 answers. This module extracts the hidden state of the last token in the intermediate layer of the large model corresponding to each answer. These hidden states are considered as vector representations that can effectively capture the semantic information of each answer, laying the foundation for subsequent semantic consistency analysis.

[0071] 3. The covariance matrix calculation module constructs a matrix using the aforementioned 10 hidden state vectors (as sentence embeddings) and processes it using a centering matrix to eliminate the influence of the mean. Subsequently, the covariance matrix of these 10 sentence embeddings is calculated. This matrix reflects the interrelationships and distribution characteristics of different responses in the semantic embedding space.

[0072] 4. The ADEV calculation module performs eigenvalue decomposition based on the covariance matrix mentioned above to obtain the corresponding eigenvalues. By performing logarithmic operations on these eigenvalues ​​and taking the average, the discrete eigenvalue (ADEV) of the answer is calculated. The magnitude of ADEV reflects the semantic dispersion of these 10 answers. The higher the dispersion, the worse the consistency of the results generated by the large model, and the higher the potential risk of hallucination.

[0073] 5. The no-hallucination probability prediction module inputs the calculated ADEV and the length features of the 10 answers into a pre-trained MLP (Multilayer Perceptron) model. This MLP model contains two hidden layers, each with 8 neurons, and is trained on a financial question-and-answer dataset. The training data consists of sample pairs of {(ADEV, answer length), whether or not there is a hallucination}, where the "whether or not there is a hallucination" label is derived by the ROUGE-L metric, which judges the consistency between the answer and the reference standard answer. Finally, the MLP outputs the probability that the answer to the question is not a hallucination.

[0074] 6、Entity extraction and knowledge graph query module launches a small parameter hallucination checking agent, extracts entities such as "interest rate swap", "credit risk", "market risk" and other key financial entities from the 10 answers generated by the large model, and then queries related information in the preset financial knowledge graph using these entities. The knowledge graph contains rich structured knowledge such as financial terminology definitions, risk type descriptions, and risk management methods, and the agent summarizes the information found to form factual reference content based on the knowledge graph.

[0075] 7、Semantic similarity calculation module processes the knowledge graph information summarized by the hallucination checking agent and the 10 answers generated by the large model into vector representations in a unified vector space through embedding processing, and then calculates the cosine similarity between each pair of vectors. This similarity reflects the matching degree of the large model's answers and the factual information in the knowledge graph.

[0076] 8、Final hallucination-free score calculation module weights and sums the hallucination-free probability output by the above MLP and the semantic similarity according to the preset weights to obtain the final hallucination-free score. This score comprehensively considers the internal consistency of the large model's generated results and the matching with external knowledge, and fully evaluates the reliability of the answers.

[0077] 9、Result fusion and discrimination module compares the final hallucination-free score with the preset main threshold value. When the score is higher than the main threshold value, it indicates that the large model has the ability to give reliable answers to the financial problem, and the system returns the result generated by the greedy strategy to the user (i.e. the most likely answer obtained by the large model in the deterministic generation mode); if the score is lower than the main threshold value, it indicates that the large model's answer to the problem has a high hallucination risk, and the system returns the expression "unable to answer this question" to the user.

[0078] 10、Dual threshold mechanism and sample storage module work simultaneously. Based on the judgment of the main threshold value, if the final hallucination-free score is within the preset secondary threshold value range, it indicates that the system's hallucination discrimination for the problem is ambiguous. At this time, the system will store the sample (including the question, 10 generated results, ADEV, similarity, final score, etc.) into a special database for subsequent analysis of system performance, optimization of the model, and adjustment of threshold parameters to continuously improve the hallucination suppression ability of the system.

[0079] Through these series of steps of coordinated work, the knowledge cross-validation question and answer system with reduced hallucination of large language model can effectively perform knowledge cross-validation on financial knowledge question and answer, significantly reduce the hallucination phenomenon in the large model's answers, and provide users with more reliable and accurate financial knowledge information.

[0080] The method combines semantic consistency analysis (based on ADEV) of generated content with external knowledge fact verification (based on a knowledge graph), and through multi-dimensional feature fusion and a double threshold mechanism, it realizes accurate identification and flexible processing of large model hallucinations, while taking into account the reliability of the results and the sustainability of sample accumulation.

[0081] Through application examples in intelligent question answering, information retrieval and other scenarios, the method demonstrates its ability to improve the accuracy of large model output and reduce hallucination risk, especially in fields with high factual requirements, such as professional consulting, knowledge services, etc. By implementing the method, users can obtain more reliable and factually accurate generated content, thereby improving the quality of decision-making and information service based on large models.

[0082] The embodiment of the application also provides a knowledge cross-validation question answering system for reducing hallucinations of a large language model, comprising:

[0083] Multiple sampling generation module: when a user asks a question, control the large model to sample multiple times to generate a specified number of results.

[0084] Hidden state analysis module: extract the hidden state of the last token in the middle layer of the large model corresponding to each answer, and calculate the covariance matrix of these hidden states and the answer discrete eigenvalue (ADEV).

[0085] MLP module: takes the answer discrete eigenvalue and the length of the answer as input, and outputs the probability of no hallucination, with two layers in its hidden layer, each layer having 8 neurons.

[0086] Hallucination checking agent module: contains a small parameter agent responsible for extracting entities involved in the model's answer, driving the knowledge graph query module to query information, and summarizing the queried information; the summary of the agent and the answer of the large model are respectively embedded, and the cosine similarity of the two is calculated.

[0087] Result fusion and discrimination module: the cosine similarity and the no hallucination probability output by the MLP module are weighted and summed to obtain the final no hallucination score; the module has a double threshold mechanism, which determines whether to return the result generated by the greedy strategy or the expression "cannot answer this question" to the user according to the comparison result of the no hallucination score and the main threshold; if the score is within the range of the secondary threshold, the sample is stored in the database for subsequent analysis.

[0088] The system can implement the knowledge cross-validation question answering method for reducing hallucinations of a large language model described in the above embodiments. The main steps of its implementation are as follows:

[0089] 1. Multiple sampling generates results: when the user asks the large model a question, the backend controls the large model to sample multiple times to generate 10 results.

[0090] 2. Calculate hidden state related indicators: extract the hidden state of the last token in the middle layer of the large model corresponding to each answer; calculate the covariance matrix C of the 10 answer hidden states, where is used to capture the relationship between different answer hidden states; according to the covariance matrix, calculate the answer discrete eigenvalue (ADEV), and the calculation formula of ADEV can refer to the existing method, which is expressed as:

[0091]

[0092] where n = 10, μ i represents the eigenvalue of the regularized covariance matrix C+β·I n , β is a small regularization term, and I n is an n×n identity matrix.

[0093] 3. MLP model prediction: input ADEV and the length of the answer into MLP to get the probability of no hallucination. The MLP hidden layer has two layers, each with 8 neurons, and the input is ADEV and the length of the output answer, and the output is the probability of positive samples (no hallucination).

[0094] 4. Knowledge graph query and summary: extract the entities involved in the model's answer by the hallucination checking intelligent agent with smaller parameters, query the information in the knowledge graph, and summarize the information queried by the intelligent agent.

[0095] 5. Calculate the cos similarity: perform embedding processing on the summary of the intelligent agent and the answer of the large model, and calculate the cos similarity between the two, which is used to measure the matching degree of the answer and the factual information in the knowledge graph.

[0096] 6. Calculate the final no hallucination score: weight and sum the probability obtained in step 3 and the similarity obtained in step 5 to get the final no hallucination score, which reflects the degree of no hallucination of the answer.

[0097] 7. Output results: when the final no hallucination score is higher than the set threshold, it means that the model has the ability to accurately answer the question, and the results generated by the greedy strategy are returned to the user; if the no hallucination score is lower than the threshold, it means that the large model cannot accurately answer the question, and the expression that cannot answer the question is returned to the user.

[0098] where the training process of the MLP model is as follows:

[0099] First, use the large model to infer on the question and answer dataset, collect the answer corresponding to each question and the corresponding answer discrete feature value ADEV;

[0100] Then, use ROUGE-L to determine whether the answer is correct, to determine whether the large model has hallucination on the question, thereby constructing the {(ADEV, answer length), whether there is hallucination} dataset;

[0101] Finally, use the dataset to train MLP to obtain a model capable of hallucination detection.

[0102] The system effectively solves the hallucination problem in the large model question and answer process by innovatively combining large model multiple sampling generation technology, neural network prediction (MLP) and knowledge graph query verification technology, and improves the accuracy, relevance and credibility of the generated content. The system adopts a "multiple sampling feature analysis, neural network probability prediction, knowledge graph fact verification and result fusion discrimination collaborative strategy", calculates the discrete feature (ADEV) by extracting the hidden state of multiple sampling, predicts the hallucination-free probability using MLP, combines the hallucination check intelligent agent driven knowledge graph query verification and calculates the semantic similarity, and finally obtains the hallucination-free score based on the threshold decision, improves the reliability, credibility and hallucination suppression accuracy level of the large model output content.

[0103] The system can be used in knowledge question and answer scenes such as financial knowledge and medical knowledge that have strict requirements on answer accuracy. It can effectively suppress large model hallucination and improve product competitiveness.

[0104] The embodiment of the application also provides a knowledge cross-validation question and answer device for reducing hallucination of a large language model, comprising: at least one memory and at least one processor;

[0105] The at least one memory is used to store a machine readable program;

[0106] The at least one processor is used to call the machine readable program, and realize the knowledge cross-validation question and answer method for reducing hallucination of a large language model described in the above embodiment.

[0107] The embodiment of the application also provides a computer readable medium, and the computer readable medium stores computer instructions, and the computer instructions make the processor execute the knowledge cross-validation question and answer method for reducing hallucination of a large language model described in the above embodiment when the processor executes the computer instructions. Specifically, a system or device equipped with a storage medium can be provided, and the storage medium stores software program code realizing the functions of any one of the above embodiments, and makes the computer (or CPU or MPU) of the system or device read and execute the program code stored in the storage medium.

[0108] In this case, the program code itself read out from the storage medium can realize the functions of any of the above-described embodiments, and therefore the program code and the storage medium storing the program code constitute a part of the present application.

[0109] Embodiments of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk such as a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, a DVD- RW, a DVD+RW, a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0110] Further, it should be understood that not only the program code read out by the computer, but also the operating system or the like operating on the computer based on the instructions of the program code can perform part or all of the actual operations to realize the functions of any of the above-described embodiments.

[0111] Further, it should be understood that the program code read out from the storage medium can be written into a memory provided in an expansion board inserted into the computer or a memory provided in an expansion unit connected to the computer, and then part or all of the actual operations can be performed by a CPU or the like mounted on the expansion board or the expansion unit based on the instructions of the program code to realize the functions of any of the above-described embodiments.

[0112] The present application has been described in detail based on the preferred embodiments thereof, with reference to the attached drawings, but it should be understood that the present application is not limited to these disclosed embodiments, and that one skilled in the art can know various modifications and changes of the above-described embodiments, and that the present application encompasses all such modifications and changes.

Claims

1. A knowledge cross-validation question answering method for reducing hallucination of a large language model, characterized in that, The method comprises the following steps: The multiple sampling generation result: when the user raises a question, the large model is controlled to perform multiple sampling to generate a specified number of results; Calculate the hidden state related indicators: extract the hidden state of the last token in the middle layer of the large model corresponding to the result, and calculate the covariance matrix of the hidden state and the answer discrete eigenvalue; MLP model prediction: the answer discrete eigenvalue and the length of the answer are input into the multilayer perceptron MLP as input, and the probability of no hallucination is output, and the hidden layer has a specific structure and a certain number of neurons; Knowledge graph query and summary: the hallucination checking agent extracts the entities involved in the model answer, drives the knowledge graph query module to query information, and summarizes the queried information; the summary of the agent and the answer of the large model are respectively embedded, and the cos similarity is calculated; Calculate the final no hallucination score and output the result: the final no hallucination score is obtained by weighted sum of the cos similarity and the no hallucination probability output by the multilayer perceptron; the content returned to the user is determined according to the comparison result of the score and the threshold.

2. The method of claim 1, wherein the method further comprises: The multilayer perceptron, the hidden layer of which contains two layers, each layer having 8 neurons.

3. The method of claim 1, wherein the method further comprises: The content returned to the user is determined according to the comparison result of the score and the threshold, A double threshold mechanism is set, and according to the comparison result of the no hallucination score and the main threshold, the result generated by the greedy strategy or the expression "unable to answer this question" is returned to the user; if the score is within the range of the secondary threshold, the sample is stored in the database for subsequent analysis.

4. The method of claim 1, wherein the method further comprises: The training process of the MLP model is as follows: First, use the large model to reason on the question and answer data set, collect the corresponding answer and the corresponding answer discrete eigenvalue ADEV for each question; Then, use ROUGE-L to judge whether the answer is correct, so as to determine whether the large model has hallucination on the question, so as to build {(ADEV, answer length), whether there is hallucination} data set; Finally, use the data set to train MLP to obtain a model capable of hallucination detection.

5. The method of claim 1, wherein the method further comprises: The hallucination checking agent is a small parameter hallucination checking agent.

6. The method of claim 1, wherein the method further comprises: The multiple sampling generation result, the large model performs 10 independent samplings to generate 10 answer results.

7. The method of claim 1, wherein the method further comprises: The answer discrete eigenvalue calculation formula is represented as: where n = 10, μ i denotes the eigenvalues of the regularized covariance matrix C+β·I n , β is a small regularization term, and I n is the n x n identity matrix.

8. A knowledge cross-validation question answering system that reduces hallucinations of large language models, characterized in that, It includes: Multiple sampling generation module: when the user raises a question, the large model is controlled to perform multiple sampling to generate a specified number of results; Hidden state analysis module: extract the hidden state of the last token in the middle layer of the large model corresponding to each answer, and calculate the covariance matrix of the hidden state and the answer discrete eigenvalue; MLP module: taking the answer discrete eigenvalue and the length of the answer as input, outputting the probability of no hallucination; Hallucination checking agent module: responsible for extracting the entities involved in the model answer, driving the knowledge graph query module to query information, and summarizing the queried information; the summary of the agent and the answer of the large model are respectively embedded, and the cos similarity is calculated; A result fusion module: a final no-hallucination score is obtained by weighted sum of the cos similarity and the no-hallucination probability output by the MLP module; Decide the content returned to the user according to the comparison result of the score and the threshold value; The system can implement the method of any one of claims 1 to 7.

9. A device for reducing hallucination of a large language model knowledge cross-validation question and answer, characterized by, Comprise: At least one memory and at least one processor; The at least one memory is used to store machine readable programs; The at least one processor is used to call the machine readable programs to implement the method of any one of claims 1 to 7.

10. A computer readable medium characterized by The computer readable medium stores computer instructions, and the computer instructions are executed by the processor to implement the method of any one of claims 1 to 7.

Images