A knowledge cross-validation question answering method and system for reducing hallucinations of a large language model

Abstract

The application discloses a knowledge cross-validation question and answer method and system for reducing hallucination of a large language model, and belongs to the technical field of artificial intelligence. The implementation of the method comprises the following steps: multiple sampling to generate results: when a user raises a question, a large model is controlled to perform multiple sampling to generate a specified number of results; calculating hidden state related indexes: the hidden states of the last token of the intermediate layer of the large model corresponding to the results are extracted, and the covariance matrix of the hidden states and the answer discrete eigenvalue are calculated; MLP model prediction: the answer discrete eigenvalue and the length of the answer are input into a multi-layer perceptron MLP as input, and the probability of no hallucination is output; knowledge graph query and summary; calculating the final hallucination-free score and outputting the result. The application can significantly improve the answer quality and credibility of the large language model, and is especially suitable for application scenarios with extremely high requirements for the content accuracy of single-modal text generation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a knowledge cross-validation question-answering method and system for reducing the illusion of large language models. Background Technology

[0002] In the fields of artificial intelligence and natural language processing (NLP), large language models (LLMs), such as GPT-3 and LLaMA, have demonstrated enormous application potential across multiple domains due to their powerful text generation, understanding, and reasoning capabilities. However, despite significant progress in LLMs, their inherent "illusion" problem (i.e., the model generates content that appears reasonable but is actually inconsistent with reality or logically contradictory) remains a key challenge limiting their widespread application and credibility. The illusion problem not only reduces the reliability of information generated by LLMs but can also lead to erroneous decisions, especially in professional fields with extremely high accuracy requirements (such as medicine, law, and finance), where its harm is even more pronounced.

[0003] To mitigate the illusion problem in LLM (Language Modeling), various methods have been proposed in existing technologies. For example, Retrieval-Enhanced Generation (RAG) systems improve the accuracy of generated content to some extent by combining the retrieval capabilities of external knowledge bases. However, RAG systems primarily focus on providing external information for the model's reference; they cannot directly identify or suppress potential illusions generated during the model's generation process. Furthermore, some methods attempt to detect illusions by analyzing the model's output. For instance, some techniques use perplexity to assess the fluency of generated text, suggesting that low perplexity may indicate higher credibility. Other methods determine the presence of illusions by comparing the consistency of multiple generation results, suggesting that highly consistent outputs may be more reliable. However, these output-based analytical methods have significant limitations. Simple perplexity only reflects the syntactic and semantic fluency of the language model and cannot effectively capture factual errors or deep logical contradictions. Simple consistency checks may overlook subtle differences in the model's semantics, or, when facing complex, open-ended questions, even if the model exhibits illusions, its multiple generation results may still show superficial consistency. These methods often struggle to delve into the underlying causes of hallucinations within the model and lack mechanisms for effectively cross-validating generated content with external facts, making it difficult to comprehensively and accurately identify and suppress LLM hallucinations.

[0004] These challenges are particularly pronounced in applications where accuracy and reliability are paramount. For example, in professional question-answering systems, users expect LLM to provide accurate and error-free answers; any hallucination could have serious consequences. Existing hallucination detection methods often fail to provide satisfactory solutions for such tasks, and cannot effectively guarantee the authenticity and reliability of the generated content. Summary of the Invention

[0005] The technical objective of this invention is to address the above-mentioned shortcomings by providing a knowledge cross-validation question-answering method and system that reduces the illusion of large language models. This method can significantly improve the quality and credibility of responses from large language models, and is particularly suitable for application scenarios where the accuracy of single-modal text generation content is extremely important.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A knowledge cross-validation question answering method to reduce the illusion of large language models, the implementation of which includes the following steps:

[0008] Multiple sampling to generate results: When a user asks a question, the large model is controlled to perform multiple samplings to generate a specified number of results;

[0009] Calculate hidden state related metrics: Extract the hidden state of the last token in the intermediate layer of the large model corresponding to the result, and calculate the covariance matrix of these hidden states and the discrete eigenvalues ​​(ADEV) of the answer;

[0010] MLP model prediction: The discrete feature values ​​of the answer and the length of the answer are input into a multilayer perceptron (MLP), and the output is the probability of no hallucination. Its hidden layers have a specific structure and number of neurons.

[0011] Knowledge Graph Query and Summarization: The hallucination checking agent extracts entities involved in the model's answer, drives the knowledge graph query module to query information, and summarizes the queried information; the agent's summary and the large model's answer are respectively embedded and their cosine similarity is calculated; this similarity is used to measure the degree of matching between the answer and the factual information in the knowledge graph;

[0012] Calculate and output the final hallucination-free score: The final hallucination-free score is obtained by weighted summation of the cosine similarity and the hallucination-free probability output by the multilayer perceptron; the content returned to the user is determined based on the comparison result of the score and the threshold.

[0013] This method organically combines techniques such as multiple sampling to generate results, neural network model prediction, and knowledge graph query verification to construct a large-scale model question-answering illusion suppression system. By comprehensively evaluating whether the large model can accurately answer user questions, it effectively reduces the illusion phenomenon. This addresses the illusion problem that occurs in the question-answering process of large models, improving the accuracy, relevance, and credibility of the generated content.

[0014] Furthermore, the multilayer perceptron has two hidden layers, each with eight neurons.

[0015] Furthermore, the content returned to the user is determined based on the comparison result between the score and the threshold.

[0016] A dual-threshold mechanism is set up. Based on the comparison between the no-illusion score and the main threshold, the system decides whether to return the result generated by the greedy strategy or the statement "This question cannot be answered" to the user. If the score is within the range of the secondary threshold, the sample is stored in the database for subsequent analysis.

[0017] Furthermore, the training process of the MLP model is as follows:

[0018] First, a large model is used to perform inference on the question-answering dataset, collecting the answer to each question and the corresponding discrete feature value ADEV of the answer.

[0019] Then, ROUGE-L is used to determine whether the answer is correct, thereby determining whether the large model is hallucinating on this question, thus constructing a dataset of {(ADEV, answer length), whether there is hallucination};

[0020] Finally, the MLP was trained using this dataset to obtain a model capable of hallucination detection.

[0021] Furthermore, the hallucination checking agent is a hallucination checking agent with small parameters.

[0022] Furthermore, in the multiple sampling to generate results, the large model performs 10 independent samplings to generate 10 response results.

[0023] Furthermore, the formula for calculating the discrete eigenvalues ​​of the answer is expressed as:

[0024]

[0025] Where n = 10, μ i The regularized covariance matrix C+β·I represents... n The eigenvalues ​​of I, where β is a small regularization term. n It is an n×n identity matrix.

[0026] This invention also claims a knowledge cross-validation question-answering system that reduces the illusion of large language models, comprising:

[0027] Multiple sampling generation module: When a user asks a question, it controls the large model to perform multiple samplings to generate a specified number of results;

[0028] Hidden State Analysis Module: Extracts the hidden state of the last token in the intermediate layer of the large model corresponding to each answer, and calculates the covariance matrix of these hidden states and the discrete eigenvalues ​​(ADEV) of the answer;

[0029] MLP module: Takes the discrete feature values ​​of the answer and the length of the answer as input, and outputs the probability of no hallucination;

[0030] The hallucination detection agent module is responsible for extracting entities involved in the model's answers, driving the knowledge graph query module to perform information queries, and summarizing the queried information; it performs embedding processing on the agent's summary and the large model's answers respectively, and calculates the cosine similarity between the two.

[0031] The result fusion and discrimination module obtains the final hallucination-free score by weighted summing of the cosine similarity and the hallucination-free probability output by the MLP module; and determines the content to be returned to the user based on the comparison between the score and the threshold.

[0032] This system can implement the aforementioned method for knowledge cross-validation to reduce the illusion of large language models.

[0033] The present invention also claims a knowledge cross-validation question-answering device for reducing the illusion of large language models, comprising: at least one memory and at least one processor;

[0034] The at least one memory is used to store a machine-readable program;

[0035] The at least one processor is used to call the machine-readable program to implement the above method.

[0036] The present invention also claims a computer-readable medium storing computer instructions that, when executed by a processor, implement the above-described method.

[0037] The knowledge cross-validation question-answering method and system for reducing large language model illusion of the present invention has the following advantages compared with the prior art:

[0038] First, generating results through multiple samplings fully utilizes the internal information from the large model's generation process, effectively capturing the uncertainty and dispersion of the model's responses, thus providing a reliable basis for subsequent hallucination judgments. Compared to traditional methods that rely solely on a single generated result, this multiple sampling analysis approach more comprehensively reflects the model's grasp of the problem.

[0039] Secondly, by combining ADEV and response length and using MLP for hallucination-free probability prediction, the existing features and model capabilities are fully utilized, improving the accuracy of probability prediction. ADEV, as an existing and effective measure of discrete features, combined with response length, can characterize the response from multiple dimensions, making the prediction results of MLP more valuable.

[0040] Furthermore, by introducing a hallucination-checking agent and a knowledge graph query verification mechanism, the responses from the large model are compared with factual information in the knowledge graph, further improving the accuracy of hallucination judgment. By calculating the cosine similarity of the embedding, the degree of matching between the response and factual information can be quantified, providing an important basis for the final hallucination-free score.

[0041] Finally, setting a threshold to determine whether to return an answer effectively prevents large models from generating hallucinatory content when they cannot accurately answer questions, thus improving the credibility and reliability of the system's output. When the no-hallucination score is above the threshold, a high-quality generated result is returned; when the score is below the threshold, the user is clearly informed that the answer cannot be given, enhancing the user's trust in the system. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the knowledge cross-validation question-answering method for reducing large language model illusion provided in one embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of an illusion detection intelligent agent module provided in one embodiment of the present invention. Detailed Implementation

[0044] The present invention will be further described below with reference to specific embodiments.

[0045] This invention provides a knowledge cross-validation question-answering method to reduce illusions in large language models. By integrating key modules such as multi-round generative diversity analysis, hidden state semantic feature mining, knowledge graph fact verification, and dynamic threshold judgment, it achieves accurate detection and suppression of illusions in the output of large models.

[0046] The implementation of this method includes the following steps:

[0047] Multiple sampling to generate results: When a user asks a question, the large model is controlled to perform multiple samplings to generate a specified number of results.

[0048] Calculate hidden state related metrics: Extract the hidden state of the last token in the intermediate layer of the large model corresponding to the result, and calculate the covariance matrix of these hidden states and the discrete eigenvalues ​​(ADEV) of the answer.

[0049] MLP model prediction: The discrete feature values ​​of the answer and the length of the answer are used as inputs to a multilayer perceptron (MLP), and the output is the probability of no hallucination. Its hidden layers have a specific structure and number of neurons.

[0050] Knowledge Graph Query and Summarization: The hallucination checking agent module contains a small-parameter agent responsible for extracting entities involved in the model's answer and driving the knowledge graph query module to perform information queries, while summarizing the retrieved information. After extracting entities, the agent retrieves relevant information from the knowledge graph and returns it to the agent. Then, the agent's summary and the large model's answer are respectively embedded, and their cosine similarity is calculated. This similarity is used to measure the degree of matching between the answer and the factual information in the knowledge graph.

[0051] The final hallucination-free score is calculated and output: The final hallucination-free score is obtained by weighted summation of the cosine similarity and the hallucination-free probability output by the multilayer perceptron. The content returned to the user is determined based on the comparison between the score and a threshold. A dual-threshold mechanism is set up: based on the comparison between the hallucination-free score and the primary threshold, it determines whether to return the result generated by the greedy strategy or a "this question cannot be answered" statement to the user; if the score is within the secondary threshold range, the sample is stored in the database for subsequent analysis.

[0052] The main steps of implementing this method are as follows:

[0053] 1. Multiple sampling to generate results: When a user asks a question to the large model, the backend controls the large model to perform multiple samplings to generate 10 results.

[0054] 2. Calculate hidden state related metrics: Extract the hidden state of the last token in the intermediate layer of the large model corresponding to each answer; calculate the covariance matrix C of the hidden states of these 10 answers, where... This is used to capture the relationship between different hidden states of the answer; the discrete eigenvalues ​​(ADEV) of the answer are calculated based on the covariance matrix. The calculation of ADEV can refer to existing methods, and its formula is expressed as follows:

[0055]

[0056] Where n = 10, μ i The regularized covariance matrix C+β·I represents... n The eigenvalues ​​of I, where β is a small regularization term. n It is an n×n identity matrix.

[0057] 3. MLP Model Prediction: The ADEV and the length of the response are input into the MLP to obtain the probability of no hallucination. The MLP has two hidden layers, each with 8 neurons. Its inputs are the ADEV and the length of the output response, and its output is the probability of a positive sample (no hallucination).

[0058] 4. Knowledge Graph Query and Summarization: The illusion-checking agent with smaller parameters extracts the entities involved in the model's answers, queries the knowledge graph for information, and then the agent summarizes the queried information.

[0059] 5. Calculate cosine similarity: Both the agent's summary and the large model's answer are embedded, and the cosine similarity between the two is calculated. This similarity is used to measure the degree of matching between the answer and the factual information in the knowledge graph.

[0060] 6. Calculate the final hallucination-free score: The probability obtained in step 3 and the similarity obtained in step 5 are weighted and summed to obtain the final hallucination-free score, which comprehensively reflects the degree to which the answer is hallucination-free.

[0061] 7. Output results: When the final no-illusion score is higher than the set threshold, it indicates that the model is capable of answering the question accurately, and the result generated by the greedy strategy is returned to the user; if the no-illusion score is lower than the threshold, it indicates that the model cannot answer the question accurately, and the statement that the question cannot be answered is returned to the user.

[0062] The training process of an MLP model is as follows:

[0063] First, a large model is used to perform inference on the question-answering dataset, collecting the answer to each question and the corresponding discrete feature value ADEV of the answer.

[0064] Then, ROUGE-L is used to determine whether the answer is correct, thereby determining whether the large model is hallucinating on this question, thus constructing a dataset of {(ADEV, answer length), whether there is hallucination};

[0065] Finally, the MLP was trained using this dataset to obtain a model capable of hallucination detection.

[0066] This method constructs a robust illusion detection and suppression mechanism by deeply analyzing both the internal state and external knowledge dimensions of a large language model (LLM). It not only effectively assesses the model's inherent uncertainty in the current problem through deep semantic discrete analysis of the internal hidden states of the LLM's multiple sampling results, but also innovatively combines external knowledge graphs to perform factual cross-validation on key entities involved in the model's answers. By weightedly fusing internal consistency metrics with external knowledge validation results, the system can more accurately determine whether an LLM's answer contains illusions and promptly alert the user when the model cannot provide a reliable answer. This significantly improves the quality and credibility of LLM answers, making it particularly suitable for intelligent question-answering scenarios with extremely high accuracy requirements for single-modal text generation and those that require the integration of multi-source information, cross-domain knowledge, and stringent requirements for output authenticity.

[0067] To better understand this method, we will use a financial knowledge question-answering system as an example to illustrate its implementation. This case study will detail how to handle a typical financial knowledge query to suppress the illusions that large models may generate during the answering process.

[0068] Suppose a user submits the following request to the system: "Please explain what interest rate swaps are, and describe their main risk types and management methods." Upon receiving this text query, the system will activate the "Knowledge Cross-Validation Large Model Illusion Suppression System (KCVS)" to process it.

[0069] 1. First, the multi-round sampling generation module begins operation. The system backend controls the large model to perform 10 independent samplings of the financial question, generating 10 responses regarding the definition, risk type, and management methods of interest rate swaps. These results are preliminary responses generated by the large model based on its own knowledge reserves and may contain some uncertainty or speculative content.

[0070] 2. The hidden state extraction module processes the aforementioned 10 answers. This module extracts the hidden state of the last token in the intermediate layer of the large model corresponding to each answer. These hidden states are considered as vector representations that can effectively capture the semantic information of each answer, laying the foundation for subsequent semantic consistency analysis.

[0071] 3. The covariance matrix calculation module constructs a matrix using the aforementioned 10 hidden state vectors (as sentence embeddings) and processes it using a centering matrix to eliminate the influence of the mean. Subsequently, the covariance matrix of these 10 sentence embeddings is calculated. This matrix reflects the interrelationships and distribution characteristics of different responses in the semantic embedding space.

[0072] 4. The ADEV calculation module performs eigenvalue decomposition based on the covariance matrix mentioned above to obtain the corresponding eigenvalues. By performing logarithmic operations on these eigenvalues ​​and taking the average, the discrete eigenvalue (ADEV) of the answer is calculated. The magnitude of ADEV reflects the semantic dispersion of these 10 answers. The higher the dispersion, the worse the consistency of the results generated by the large model, and the higher the potential risk of hallucination.

[0073] 5. The no-hallucination probability prediction module inputs the calculated ADEV and the length features of the 10 answers into a pre-trained MLP (Multilayer Perceptron) model. This MLP model contains two hidden layers, each with 8 neurons, and is trained on a financial question-and-answer dataset. The training data consists of sample pairs of {(ADEV, answer length), whether or not there is a hallucination}, where the "whether or not there is a hallucination" label is derived by the ROUGE-L metric, which judges the consistency between the answer and the reference standard answer. Finally, the MLP outputs the probability that the answer to the question is not a hallucination.

[0074] 6. The entity extraction and knowledge graph query module initiates a small-parameter illusion-checking agent to extract entities from the 10 responses generated by the large model, identifying key financial entities such as "interest rate swaps," "credit risk," and "market risk." Subsequently, the agent uses these entities to query relevant information in a pre-defined financial knowledge graph. This knowledge graph contains rich structured knowledge such as financial terminology definitions, risk type descriptions, and risk management methods. The agent then summarizes the queried information to form factual reference content based on the knowledge graph.

[0075] 7. The semantic similarity calculation module performs embedding processing on the knowledge graph information summarized by the hallucination checking agent and the 10 answers generated by the large model, converting them into vector representations in a unified vector space. Then, it calculates the cosine similarity between each pair of vectors, which reflects the degree of matching between the large model's answers and the factual information in the knowledge graph.

[0076] 8. The final hallucination-free score calculation module calculates the weighted sum of the hallucination-free probability and semantic similarity output by the MLP according to preset weights, resulting in the final hallucination-free score. This score comprehensively considers the internal consistency of the large model's generated results and its matching with external knowledge, thus fully evaluating the reliability of the answer.

[0077] 9. The result fusion and discrimination module compares the final no-illusion score with a preset main threshold. When the score is higher than the main threshold, it indicates that the large model is capable of providing a reliable answer to the financial question, and the system returns the result generated by the greedy strategy (i.e., the most likely answer obtained by the large model in the deterministic generation mode) to the user; if the score is lower than the main threshold, it indicates that the large model's answer to the question has a high risk of illusion, and the system returns the statement "This question cannot be answered" to the user.

[0078] 10. The dual-threshold mechanism and the sample storage module work together. Based on the primary threshold used to determine the presence or absence of hallucinations, if the final hallucination-free score falls within the preset secondary threshold range, it indicates that the system's hallucination detection for that question is ambiguous. In this case, the system stores the sample (including the question, 10 generated results, ADEV, similarity, final score, etc.) in a dedicated database for subsequent system performance analysis, model optimization, and threshold parameter adjustment, continuously improving the system's hallucination suppression capability.

[0079] Through this series of collaborative steps, knowledge cross-validation question answering that reduces the illusion of large language models can effectively cross-validate financial knowledge questions, significantly reducing the illusion phenomenon in large model answers and providing users with more reliable and accurate financial knowledge information.

[0080] This method combines semantic consistency analysis of generated content (based on ADEV) with external knowledge fact verification (based on knowledge graph). Through multi-dimensional feature fusion and a dual threshold mechanism, it achieves accurate identification and flexible processing of large model illusions, while taking into account the reliability of the results and the sustainability of sample accumulation.

[0081] Through application examples in scenarios such as intelligent question answering and information retrieval, this method demonstrates its ability to improve the accuracy of large model outputs and reduce the risk of illusions, making it particularly suitable for fields with high factual requirements, such as professional consulting and knowledge services. By implementing this method, users can obtain more reliable and fact-based generated content, thereby improving the quality of decision-making and the level of information services based on large models.

[0082] This invention also provides a knowledge cross-validation question-answering system to reduce the illusion of large language models, comprising:

[0083] Multiple sampling generation module: When a user asks a question, the module controls the large model to perform multiple samplings to generate a specified number of results.

[0084] Hidden State Analysis Module: Extracts the hidden state of the last token in the intermediate layer of the large model corresponding to each answer, and calculates the covariance matrix of these hidden states and the discrete eigenvalues ​​(ADEV) of the answer.

[0085] The MLP module takes the discrete feature values ​​of the answer and the length of the answer as input and outputs the probability of no hallucination. Its hidden layer contains two layers, each with 8 neurons.

[0086] The hallucination checking agent module contains an agent with relatively small parameters, which is responsible for extracting entities involved in the model's answer, driving the knowledge graph query module to perform information queries, and summarizing the queried information. The agent's summary and the large model's answer are both embedded and their cosine similarity is calculated.

[0087] The result fusion and discrimination module calculates the final hallucination-free score by weighting and summing the cosine similarity and the hallucination-free probability output by the MLP module. The module has a built-in dual threshold mechanism. Based on the comparison between the hallucination-free score and the main threshold, it decides to return the result generated by the greedy strategy or the statement "This question cannot be answered" to the user. If the score is within the range of the secondary threshold, the sample is stored in the database for subsequent analysis.

[0088] This system can implement the knowledge cross-validation question-answering method for reducing large language model illusions as described in the above embodiments. The main steps of its implementation are as follows:

[0089] 1. Multiple sampling to generate results: When a user asks a question to the large model, the backend controls the large model to perform multiple samplings to generate 10 results.

[0090] 2. Calculate hidden state related metrics: Extract the hidden state of the last token in the intermediate layer of the large model corresponding to each answer; calculate the covariance matrix C of the hidden states of these 10 answers, where... This is used to capture the relationship between different hidden states of the answer; the discrete eigenvalues ​​(ADEV) of the answer are calculated based on the covariance matrix. The calculation of ADEV can refer to existing methods, and its formula is expressed as follows:

[0091]

[0092] Where n = 10, μ i The regularized covariance matrix C+β·I represents... n The eigenvalues ​​of I, where β is a small regularization term. n It is an n×n identity matrix.

[0093] 3. MLP Model Prediction: The ADEV and the length of the response are input into the MLP to obtain the probability of no hallucination. The MLP has two hidden layers, each with 8 neurons. Its inputs are the ADEV and the length of the output response, and its output is the probability of a positive sample (no hallucination).

[0094] 4. Knowledge Graph Query and Summarization: The illusion-checking agent with smaller parameters extracts the entities involved in the model's answers, queries the knowledge graph for information, and then the agent summarizes the queried information.

[0095] 5. Calculate cosine similarity: Both the agent's summary and the large model's answer are embedded, and the cosine similarity between the two is calculated. This similarity is used to measure the degree of matching between the answer and the factual information in the knowledge graph.

[0096] 6. Calculate the final hallucination-free score: The probability obtained in step 3 and the similarity obtained in step 5 are weighted and summed to obtain the final hallucination-free score, which comprehensively reflects the degree to which the answer is hallucination-free.

[0097] 7. Output results: When the final no-illusion score is higher than the set threshold, it indicates that the model is capable of answering the question accurately, and the result generated by the greedy strategy is returned to the user; if the no-illusion score is lower than the threshold, it indicates that the model cannot answer the question accurately, and the statement that the question cannot be answered is returned to the user.

[0098] The training process of the MLP model is as follows:

[0099] First, a large model is used to perform inference on the question-answering dataset, collecting the answer to each question and the corresponding discrete feature value ADEV of the answer.

[0100] Then, ROUGE-L is used to determine whether the answer is correct, thereby determining whether the large model is hallucinating on this question, thus constructing a dataset of {(ADEV, answer length), whether there is hallucination};

[0101] Finally, the MLP was trained using this dataset to obtain a model capable of hallucination detection.

[0102] This system innovatively combines large-model multiple sampling generation technology, neural network prediction (MLP), and knowledge graph query verification technology to effectively solve the hallucination problem in the large-model question-answering process, improving the accuracy, relevance, and credibility of the generated content. The system employs a collaborative strategy of "multiple sampling feature analysis, neural network probability prediction, knowledge graph fact verification, and result fusion discrimination." It extracts hidden states from multiple sampling to calculate discrete features (ADEV), uses MLP to predict the probability of no hallucination, combines knowledge graph query verification driven by a hallucination-checking agent, and calculates semantic similarity. Finally, a weighted fusion is performed to obtain a no-hallucination score, and a threshold-based decision is made, improving the reliability, credibility, and accuracy of hallucination suppression in the output content of the large model.

[0103] This system can be used in knowledge-based question-and-answer scenarios such as financial and medical knowledge, where the accuracy of answers is strictly required. It can effectively suppress the illusion of large models and enhance product competitiveness.

[0104] This invention also provides a knowledge cross-validation question-answering device for reducing the illusion of large language models, comprising: at least one memory and at least one processor;

[0105] The at least one memory is used to store a machine-readable program;

[0106] The at least one processor is used to call the machine-readable program to implement the knowledge cross-validation question-answering method for reducing the illusion of large language models as described in the above embodiments.

[0107] This invention also provides a computer-readable medium storing computer instructions. When executed by a processor, the computer instructions cause the processor to perform the knowledge cross-validation question-answering method for reducing large language model illusion as described in the above embodiments. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the above embodiments can be provided, and the computer (or CPU or MPU) of the system or apparatus can read and execute the program code stored in the storage medium.

[0108] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0109] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0110] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0111] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0112] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the code review methods in the different embodiments. These embodiments are also within the protection scope of the present invention.

Claims

1. A knowledge cross-validation question-answering method to reduce the illusion of large language models, characterized in that, The implementation of this method includes the following steps: Multiple sampling to generate results: When a user asks a question, the large model is controlled to perform multiple samplings. The large model performs 10 independent samplings to generate 10 answer results. Calculate hidden state related metrics: Extract the hidden state of the last token in the intermediate layer of the large model corresponding to the result, and calculate the covariance matrix of these hidden states and the discrete eigenvalues ​​of the answer; the formula for calculating the discrete eigenvalues ​​of the answer is expressed as: ； in, , Represents the regularized covariance matrix The eigenvalues ​​are given by , and C is the covariance matrix of the 10 hidden states with different answers. ; For a small regularization term, for The identity matrix; MLP model prediction: The discrete feature value of the answer and the length of the answer are used as inputs to a multilayer perceptron (MLP), and the output is the probability of no hallucination. The hidden layer of the MLP contains two layers, each with 8 neurons. The training process of an MLP model is as follows: First, a large model is used to perform inference on the question-answering dataset, collecting the answer to each question and the corresponding discrete feature value ADEV of the answer. Then, ROUGE-L is used to determine whether the answer is correct, thereby determining whether the large model is hallucinating on this question, thus constructing a dataset of {(ADEV, answer length), whether there is hallucination}; Finally, the MLP was trained using this dataset to obtain a model capable of hallucination detection; Knowledge Graph Query and Summarization: The hallucination checking agent extracts entities involved in the model's answer, drives the knowledge graph query module to perform information queries, and summarizes the queried information; the agent's summary and the large model's answer are respectively embedded and their cosine similarity is calculated. Calculate and output the final hallucination-free score: The final hallucination-free score is obtained by weighted summation of the cosine similarity and the hallucination-free probability output by the multilayer perceptron; the content returned to the user is determined based on the comparison result of the score and the threshold.

2. The knowledge cross-validation question-answering method for reducing large language model illusion according to claim 1, characterized in that, The content returned to the user is determined based on the comparison result between the score and the threshold. A dual-threshold mechanism is set up. Based on the comparison between the no-illusion score and the main threshold, the system decides whether to return the result generated by the greedy strategy or the statement "This question cannot be answered" to the user. If the score is within the range of the secondary threshold, the sample is stored in the database for subsequent analysis.

3. A knowledge cross-validation question-answering system for reducing the illusion of large language models, characterized in that, include: Multiple sampling generation module: When a user asks a question, the large model is controlled to perform multiple samplings. The large model performs 10 independent samplings to generate 10 answer results. Hidden State Analysis Module: Extracts the hidden state of the last token in the intermediate layer of the large model corresponding to each answer, and calculates the covariance matrix of these hidden states and the discrete eigenvalues ​​of the answer; The formula for calculating the discrete eigenvalues ​​of the answer is expressed as: ； in, , Represents the regularized covariance matrix The eigenvalues ​​are given by , and C is the covariance matrix of the 10 hidden states with different answers. ; For a small regularization term, for The identity matrix; MLP module: The discrete feature value of the answer and the length of the answer are input into the multilayer perceptron MLP, and the output is the probability of no hallucination. The hidden layer of the multilayer perceptron contains two layers, each with 8 neurons. The training process of an MLP model is as follows: First, a large model is used to perform inference on the question-answering dataset, collecting the answer to each question and the corresponding discrete feature value ADEV of the answer. Then, ROUGE-L is used to determine whether the answer is correct, thereby determining whether the large model is hallucinating on this question, thus constructing a dataset of {(ADEV, answer length), whether there is hallucination}; Finally, the MLP was trained using this dataset to obtain a model capable of hallucination detection; The hallucination detection agent module is responsible for extracting entities involved in the model's answers, driving the knowledge graph query module to perform information queries, and summarizing the queried information; it performs embedding processing on the agent's summary and the large model's answers respectively, and calculates the cosine similarity between the two. The result fusion and discrimination module obtains the final hallucination-free score by weighted summing of the cosine similarity and the hallucination-free probability output by the MLP module; and determines the content to be returned to the user based on the comparison between the score and the threshold. The system is capable of implementing the method described in any one of claims 1 to 2.

4. A knowledge cross-validation question-answering device for reducing the illusion of large language models, characterized in that, include: At least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to invoke the machine-readable program to implement the method of any one of claims 1 to 2.

5. A computer-readable medium, characterized in that, The computer-readable medium stores computer instructions that, when executed by a processor, implement the method described in any one of claims 1 to 2.

Images