Large language model illusion detection method based on dynamic subgraph retrieval

Abstract

The invention discloses a large language model illusion detection method based on dynamic subgraph retrieval, relates to the technical field of model illusion detection, and aims to solve the problem of poor detection effect of a current illusion detection method. Comprising the following steps: acquiring and preprocessing a data set to obtain a training set and a test set; performing fine tuning according to the preprocessed data set to obtain a hop count predictor, a relation classifier, an entity classifier and a fact classifier; using cue words to extract an entity list from the question answer pair by means of a large language model; starting from entities in the entity list, extracting related evidence sub-graphs from the knowledge graph by utilizing a hop count predictor, a relation classifier and an entity classifier in a cooperative manner; and judging the correctness of the answer pair of the original question by using a fact classifier according to the obtained evidence sub-graph. According to the method, the evidence sub-graphs are dynamically retrieved through cooperation of a plurality of classifiers, and related sub-graphs of complex reasoning questions can be retrieved more accurately so as to judge illusion of question answer pairs.

Description

Technical Field

[0001] This invention relates to the field of model illusion detection technology, specifically to a method for detecting large language model illusions based on dynamic subgraph retrieval. Background Technology

[0002] In recent years, large-scale language models based on the Transformer architecture have made groundbreaking progress in text generation, question answering systems, code writing, and knowledge reasoning tasks. These models, through pre-training on massive amounts of text data, have learned rich language patterns and world knowledge, enabling them to generate fluent, coherent, and seemingly reasonable text.

[0003] However, the essence of large language models is a probabilistic text generator. Its goal is to predict the next most likely token based on the preceding context, rather than strictly adhering to factual or logical truths. This mechanism often leads to "illusions" in the generated content. "Illusions" specifically refer to content generated by the model that appears reasonable but is actually erroneous, fabricated, or contradicts the provided information source. The existence of these illusions severely restricts the application of large models in high-risk, high-reliability scenarios such as healthcare, finance, law, and news, becoming one of the core bottlenecks to their industrialization.

[0004] Current hallucination detection methods (such as CN117972033A) include those based on internal model knowledge, such as identifying hallucinations through self-examination, self-questioning, or internal consistency checks; those based on internal model signals, such as using the confidence level and attention heatmap when the model generates answers to determine if a hallucination has occurred; and those based on external knowledge, which generally involves retrieving relevant text through similarity and using the knowledge in the text to determine whether a hallucination has occurred. Summary of the Invention

[0005] This invention addresses the problem of poor detection performance in current hallucination detection methods by proposing a large language model hallucination detection method based on dynamic subgraph retrieval. By using multiple classifiers to collaboratively and dynamically retrieve evidence subgraphs, this method can more accurately retrieve relevant subgraphs for complex reasoning problems to determine if the answer to the question is correct.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a large language model illusion detection method based on dynamic subgraph retrieval, comprising the following steps: S1. Obtain the dataset and preprocess it to obtain the training set and test set; S2, fine-tunes the hop count predictor, relation classifier, entity classifier and fact classifier based on the preprocessed dataset; S3 uses prompt words and a large language model to extract a list of entities from question-answer pairs; S4, starting from the entities in the entity list, uses the hop count predictor, relation classifier and entity classifier to extract relevant evidence subgraphs from the knowledge graph in collaboration; S5. Based on the obtained evidence subgraph, use a fact classifier to determine the correctness of the original question answer pair.

[0007] In the technical solution of this invention, the dataset is first preprocessed, and then the hop count predictor, relation classifier, entity classifier and fact classifier are fine-tuned based on the processed dataset. Then, the entity list is extracted from the question-answer pair, and the hop count predictor, relation classifier and entity classifier are used in collaboration to extract the relevant evidence subgraph from the knowledge graph. Finally, the fact classifier is used to judge the correctness of the original question-answer pair based on the evidence subgraph.

[0008] The present invention is further configured such that the training set and the test set can be constructed from existing datasets through screening or converted from multi-hop knowledge graph question-answering datasets according to application requirements.

[0009] In this technical solution, the training set and test set mentioned above can be obtained through two methods.

[0010] The present invention is further configured such that: the hop count predictor performs a multi-classification task, the relation classifier, the entity classifier and the fact classifier perform a binary classification task, and the fine-tuning is a full-parameter fine-tuning based on BERT.

[0011] The present invention is further configured such that: the jump count predictor can predict the distance from the entity to the target answer entity; the relation classifier can determine whether the combination of "question + entity + relation" is related; the entity classifier can filter out semantically dissimilar redundant entities; and the fact classifier can determine whether the answer is an illusion.

[0012] The present invention is further configured such that step S3 includes: S31, Generate sub-questions based on the input text question-answer pairs, obtain the answer to the current sub-question based on the original question-answer pairs, and extract the entity information from them; S32, based on the information in S31, further generate sub-problems, repeat the process of S31 until the model has obtained all entity information; S33, deduplicate the entity information to obtain the corresponding entity list.

[0013] The present invention is further configured such that step S4 includes: S41, Input the entities and issues in the entity list into the hop count predictor and output the hop count; S42, input the relation classifier in the form of "problem + entity + relation", and output the binary classification result to determine whether to retain the reasoning path where the current relation is located; S43. Input the entity classifier with the form of "entity list + entity", evaluate the semantic alignment between the current entity and the entity list, and determine whether the information is needed.

[0014] The present invention is further configured such that step S4 also includes: A complete triple is extracted and added to the evidence subgraph only when both classifiers confirm its usefulness for fact verification; for multi-hop queries, the algorithm recursively processes tail entities, dynamically updating the hop count prediction at each step to guide graph expansion; the recursion terminates when a direct relationship is found or the traversal depth reaches the predefined maximum retrieval depth.

[0015] The present invention is further configured such that step S5 includes: concatenating the question, the answer, and the retrieved evidence subgraph into a text sequence, inputting it into a fact classifier model, and having the model determine whether the question-answer pair is supported by evidence, thereby obtaining the final result of correct or hallucinatory.

[0016] The present invention is further configured such that: the text sequence obtained by concatenating the question, the answer and the retrieved evidence subgraph is input into a word segmenter for word segmentation and special markers are added, and the word segmenter is encoded into a tensor, which is then input into a fact classifier model.

[0017] The present invention is further configured such that: after the tensor is input into the fact classifier model, the model performs forward propagation and finally outputs a two-dimensional raw score; the Softmax function is used to convert the raw score into a probability distribution of two categories, and the sum of the two probabilities is 1; the final classification result is obtained by comparing the probabilities of the two categories.

[0018] The present invention provides a large language model illusion detection method based on dynamic subgraph retrieval, which can bring the following beneficial effects: To address the issue that hallucination detection methods relying on model-learned knowledge depend on internal model knowledge, this method retrieves evidence subgraphs from the knowledge graph as supplementary knowledge, thus compensating for the lack of knowledge. To address the issue of insufficient interpretability in hallucination detection methods based on model internal states, this paper proposes using dynamically retrieved evidence subgraphs as strong evidence to support hallucination detection. The entire evidence subgraph contains the reasoning path and relevant information of the problem, thus exhibiting strong interpretability. To address the issue of redundant information leading to incorrect judgments in hallucination detection methods based on retrieved text, knowledge graph retrieval yields less redundant information compared to text, thus having less impact on hallucination judgment. To address the inherent limitations of models based on internal states and the inability of traditional subgraph retrieval methods to retrieve complex problem subgraphs, which leads to poor detection of complex problem illusions, this invention utilizes multiple classifiers to collaboratively and dynamically retrieve evidence subgraphs. This enables more accurate retrieval of relevant subgraphs for complex reasoning problems, thus helping to identify question-answer pair illusions. Attached Figure Description

[0019] Figure 1 This is a flowchart of a large language model illusion detection method based on dynamic subgraph retrieval according to the present invention.

[0020] Figure 2 This is a schematic diagram of a large language model illusion detection system based on dynamic subgraph retrieval according to the present invention. Detailed Implementation

[0021] Example 1 To address the issue of unsatisfactory detection performance in current hallucination detection methods, this embodiment proposes a hallucination detection method based on a large language model using dynamic subgraph retrieval, referencing... Figure 1 and Figure 2 It mainly includes the following steps.

[0022] Step S1: First, obtain the dataset and preprocess it to obtain the corresponding training set and test set. The training set and test set can be constructed from the existing dataset and selected according to the actual application requirements, or transformed using a multi-hop knowledge graph question answering dataset.

[0023] In this technical solution, the training set and test set mentioned above can be obtained through two methods.

[0024] One approach is to construct the dataset by selecting relevant data entries from existing datasets; for example, if the application requirement is to retrieve subgraphs as evidence chains to determine the factuality of question-answer pairs, then text question-answer pairs constructed using knowledge graphs can be extracted from the dataset as training and test sets.

[0025] Another approach is to transform existing multi-hop knowledge graph question-answering datasets. For example, PathQuestion, as a knowledge graph-based multi-hop dataset, can use a large model to modify its answers to other entities of the same type as the original answer as negative examples, and use the original data as positive examples to train the corresponding classifier to solve the problem.

[0026] Step S2: Fine-tune the hop count predictor, relation classifier, entity classifier, and fact classifier using the preprocessed dataset. The hop count predictor performs a multi-class classification task, while the relation classifier, entity classifier, and fact classifier perform a binary classification task. All fine-tuning is based on full parameter fine-tuning of BERT.

[0027] The aforementioned fine-tuning is based on BERT's full-parameter fine-tuning, which can mainly predict the distance between the entity and the target answer entity; for the relation classifier, it can accurately determine whether the combination of "question + entity + relation" is relevant; for the entity classifier, it can filter out semantically dissimilar redundant entities; and the fact classifier can determine whether the answer is an illusion.

[0028] In this embodiment, during the fine-tuning process, the text to be classified is directly input into the BertForSequenceClassification (an architecture specifically designed for performing classification tasks using BERT) model architecture. The text sequence obtains a deep semantic representation through the BERT encoder, where the final layer output vector of the first [CLS] tag is used as an aggregate summary of the entire sequence. Subsequently, this vector is fed into the model's built-in classification output layer, which is randomly initialized after pre-training, to directly calculate the probability distribution of each category. The model obtains a classifier suitable for a specific task by minimizing the cross-entropy loss between the predicted result and the true label, and by using the backpropagation algorithm to update all parameters of the entire BERT model (including its encoder and built-in classification layer).

[0029] The input format of the hop count predictor is "question + entity". The output of the hop count predictor is the distance between the entity and the target answer entity, that is, how many relations are needed to retrieve the answer.

[0030] The input to the relation classifier is "question + entity + relation", and the output is 0 or 1. An output of 1 indicates that the information on the link where the current entity and relation are located helps to detect the illusion of the current question-answer pair, while 0 indicates the opposite.

[0031] The input to the entity classifier is "entity list + entity". It determines whether the currently retrieved entity is semantically similar to the entities in the entity list. If they are similar, the entity is retained; otherwise, it is deleted.

[0032] The input to the fact classifier is "question + answer + evidence subgraph". The output of the fact classifier is the result of hallucination detection. An output of 1 indicates that the current evidence subgraph is sufficient to support that the question and answer are facts. An output of 0 indicates that the evidence is insufficient or that the evidence points out that the current answer is wrong, that is, a hallucination has occurred.

[0033] Step S3 involves extracting an entity list from the question-answer pairs based on the prompt words and with the help of a large language model; this step specifically includes the following sub-steps.

[0034] Step S31: Based on the input text question-answer pair, consider what information needs to be obtained, then generate a sub-question, obtain the answer to the current sub-question based on the original question-answer pair, and extract the entity information from it.

[0035] Step S32: Based on the information from step S31, further generate sub-problems, and repeat the process of step S31 until the extraction model has obtained all entity information.

[0036] Step S33: Deduplicate the entity information to obtain the corresponding entity list.

[0037] In this embodiment, step S3 includes the following: The prompt word specifically assigns a professional role to the large model, clarifying its task as entity extraction. The specific process involves considering what information needs to be obtained in the first step based on the input text question-answer pair, generating corresponding sub-questions, obtaining the answer to the current sub-question based on the original question-answer pair, and then extracting the entity information. Then, based on the information obtained in the first step, it continues to consider what information is needed in the second step, repeating the above steps until the model believes it has obtained all the information and stops thinking. Then, all entities obtained in all steps are extracted, and after deduplication, a list of entities corresponding to the original question-answer pair is obtained. This entity list needs to be passed into the subsequent subgraph retrieval method to retrieve relevant evidence subgraphs.

[0038] The above-mentioned entity list acquisition process includes the following steps: role setting and task initialization, iterative loop, and result integration and output.

[0039] The role setting and task initialization process mainly involves assigning a specific role to the large language model and clarifying the core task. Specifically, it involves assigning a professional role to the large model and instructing it to extract all relevant entities from a given question-answer pair.

[0040] For the iterative loop process, a single loop mainly includes the following steps: 1. Based on the currently available information, the model autonomously considers what information needs to be obtained next to more completely answer the original question, thus generating a specific sub-question that needs to be solved. 2. The model answers its own sub-question based on all the information contained in the answer pair of the original question, generating the answer to the sub-question. 3. From the sub-question answers generated in step 2 above, new entities are extracted and added to a temporary list. 4. The model determines whether all necessary information has been obtained. If it determines yes, the loop stops; if it determines no, it returns to step 2 with the newly obtained information to start the next round of the loop.

[0041] For the result integration and output process, it merges all the entities obtained in the above iterative loop, removes duplicates from these entities, and finally generates the corresponding entity list.

[0042] Step S4: Starting from the entities in the entity list, relevant evidence subgraphs are extracted from the knowledge graph through the collaboration of the hop count predictor, relation classifier, and entity classifier. This step mainly includes the following sub-steps.

[0043] Step S41: Input the entities and questions from the entity list into the hop count predictor and output the hop count; specifically, starting from an entity in the entity list, input it and the question into the hop count predictor and output the hop count, which represents the distance required from the current entity to the target entity.

[0044] Step S42: Input the relation classifier in the form of "question + entity + relation" and output the binary classification result to determine whether to retain the reasoning path where the current relation is located; specifically, obtain all relations connected to the entity in the graph, input them and the entity and question in the form of "question + entity + relation" into the relation classifier, output the binary classification result to determine whether to retain the reasoning path where the current relation is located, and continue to search the depth along the relation.

[0045] Step S43: Input the entity classifier in the form of "entity list + entity" to evaluate the semantic alignment between the current entity and the entity list, and determine whether the information is needed; specifically, extract all tail entities connected to the relationship from the knowledge graph, and input them into the entity classifier in the form of "entity list + entity" along with the entity list extracted in the previous step, evaluate the semantic alignment between the current entity and the entity list, and determine whether the information is needed.

[0046] Only when both classifiers confirm that it is helpful for fact verification can a complete triple be extracted and added to the evidence subgraph. For multi-hop queries (predicted hop count greater than 2), the algorithm recursively processes tail entities, dynamically updating the hop count prediction at each step to guide the expansion of the graph.

[0047] The recursion terminates when a direct relationship is found (predicted hop count less than or equal to 1) or the traversal depth reaches a predefined maximum retrieval depth, thus achieving a balance between comprehensiveness and computational efficiency. After processing all entities in the entity list extracted in step S3 through step S4, a complete evidence subgraph will be obtained, which can be used to verify the factuality of the answer to the question.

[0048] Step S5: Use a fact classifier to determine the correctness of the original question answer pair based on the obtained evidence subgraph.

[0049] Specifically, the question, the answer, and the retrieved evidence subgraph are concatenated into a single text sequence. This text sequence is then input into a fact classifier model, which determines whether the question-answer pair is supported by evidence, thus arriving at the final result of whether it is correct or an illusion.

[0050] The text sequence obtained by concatenating the question, answer, and retrieved evidence subgraphs is input into the corresponding word segmenter of the model for word segmentation and the addition of special tags (such as [CLS] and [SEP]), and encoded into tensors. The tensors are then input into the fact classifier model.

[0051] The tensor is input into the model, which is already in evaluation mode. Through forward propagation, the model directly outputs a two-dimensional raw score. Applying the Softmax function to this score yields the probability distribution of the text belonging to either positive or negative categories; the sum of the two probabilities is 1. Comparing the probabilities of the two categories gives the final classification result. In this embodiment, if the probability is greater than a set threshold, the answer is considered correct; if the probability is less than or equal to the set threshold, it is determined that a hallucination exists.

[0052] The fundamental assumption of hallucination detection methods based on model internal knowledge is that a model with a large and well-developed internal knowledge base can identify hallucination outputs that contradict its learned knowledge through self-examination, self-questioning, or internal consistency checks. However, models have certain knowledge blind spots; for knowledge that the model has not learned, this method cannot detect whether it is a hallucination.

[0053] The fundamental assumption of hallucination detection methods based on internal model signals is that when a model generates hallucinations, its internal state (such as low confidence, scattered attention, and abnormal neuronal activation) exhibits quantifiable patterns different from those when generating real content. Methods based on internal model signals can only detect low-level errors; they are completely ineffective against complete hallucination narratives derived from a flawed premise and rigorous attention logic. Furthermore, this method requires extensive feature engineering and annotation for each model, and even each new version, resulting in extremely high costs and making large-scale deployment and application difficult in practice. Additionally, the method lacks interpretability.

[0054] The fundamental assumption of hallucination detection methods based on external knowledge is the existence of an objective, accurate, and complete external knowledge source (such as knowledge graphs, databases, or search engine results) that can serve as the gold standard to determine whether the model output is a hallucination. One approach is to retrieve textual knowledge. However, textual knowledge is usually segmented into individual text segments, and the most similar segments are matched to determine if it's a hallucination. This method often results in redundant knowledge, affecting the final judgment. Another approach is to retrieve graph knowledge, which typically involves extracting entities through Neural Query Analyzer (NER) and then generating SQL (Structured Query Language) for retrieval. The SQL can be predefined or generated using a large language model. However, both methods can only handle simple problems. For complex problems, there are too many scenarios, and the definitions are inevitably incomplete. Model-generated SQL in complex situations is of poor quality and is unlikely to retrieve any information.

[0055] This invention proposes a large-scale hallucination detection method based on dynamic subgraph retrieval, which overcomes the shortcomings of the aforementioned hallucination detection methods. The method employs the following technical solution: First, the dataset is preprocessed. Then, based on the preprocessed dataset, a hop count predictor, relation classifier, entity classifier, and fact classifier are fine-tuned. Next, an entity list is extracted from the question-answer pairs, and the hop count predictor, relation classifier, and entity classifier are used collaboratively to extract relevant evidence subgraphs from the knowledge graph. Finally, based on the evidence subgraphs, the fact classifier is used to determine the correctness of the original question-answer pairs.

[0056] Based on the aforementioned large language model illusion detection method based on dynamic subgraph retrieval, this embodiment also proposes a large language model illusion detection system based on dynamic subgraph retrieval, referencing... Figure 2The system includes a dataset processing module, a fine-tuning module, an entity list extraction module, an evidence subgraph extraction module, and an illusion judgment module. The dataset processing module is connected to the fine-tuning module, which in turn is connected to both the entity list extraction module and the evidence subgraph extraction module. Both the entity list extraction module and the evidence subgraph extraction module are connected to the illusion judgment module. The dataset processing module executes step S1; the fine-tuning module executes step S2; the entity list extraction module and the evidence subgraph extraction module extract entity lists and evidence subgraphs, respectively; and the illusion judgment module uses a fact classifier to determine the correctness of the original question's answer.

[0057] Example 2 To apply the above Example 1 to a specific use case, in order to detect whether the answers generated by the large model are illusions, this example uses the question "In what year was the school that the spouse of a certain celebrity Yao graduated from established?" as an example. The model generates the answer "The spouse of a certain celebrity Yao, Ye, graduated from a certain university, which was established in a certain year and month." The first step is entity extraction. The current question-and-answer pair is input into the main model. The model's first step is to determine who the spouse of a certain celebrity, Yao, is. The extracted information from the question-and-answer pair is that Yao's spouse is Ye. This sub-question-and-answer pair contains the entities "Yao, Ye". Then, in the second round, the information from the first round and the question-and-answer pair are input into the model. The second round considers which school Ye graduated from. The corresponding answer is that Ye graduated from a certain university, containing the entities "Ye, a certain university". Next, the model considers the year the university was founded, and the corresponding answer is that the university was founded in a certain year and month, extracting the entities "a certain university, a certain year and month". Finally, the model outputs all the information obtained.

[0058] The entity list obtained after sorting and deduplicating the above entities is "a certain celebrity Yao, Ye, a certain university, a certain year and month".

[0059] The first entity in the entity list is fed into the hop count predictor along with the question. The hop count predictor returns 3, indicating that the current entity is 3 units away from the target entity. Then, using "a certain celebrity Yao" as the head entity, the knowledge graph is searched for related relationships. These relationships, along with the entity and the question, are fed into the relationship classifier to determine whether to continue searching deeper along a particular relationship. For example, the output for the "lover" relationship is 1, and the output for age is 0. Next, the knowledge graph is searched for the entity "Ye" corresponding to the "lover" relationship of "a certain celebrity Yao". "Ye" is then appended to the entity list and fed into the entity classifier, which determines its semantic similarity to the entities in the list. Clearly, this entity exists in the entity list, so the return value is 1 (names are generally identical and easy to identify; however, some other types of entities extracted by the model may not be exactly the same as those in the knowledge graph). Then, it is determined that the hop count for "a certain celebrity Yao" is greater than 1, so the current tail entity ("Ye") is used to continue searching the knowledge graph until the predicted hop count is less than or equal to 1 or the maximum exploration depth is reached. Then, the entities in the entity list are also passed into the subgraph for retrieval (because the first entity may not be able to explore the correct target entity, and the process may stop early due to model errors; this is to prevent such randomness).

[0060] After the retrieval, an evidence subgraph is obtained: [(Celebrity Yao, spouse Ye), (Ye graduated from a certain university), (the university was established in a certain year and month)]. This subgraph is connected to the question and answer, separated by a separator. Specifically, it takes the form: "In what year was the university that celebrity Yao's spouse graduated from established? [Separator] Celebrity Yao's spouse Ye graduated from a certain university, established in a certain year and month. [Separator][(Celebrity Yao, spouse Ye), (Ye graduated from a certain university), (the university was established in a certain year and month)]". This is input into the fact classifier. The fact classifier outputs 1, indicating that the current evidence chain proves that the answer generated by this model did not produce a hallucination.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting hallucinations in a large language model based on dynamic subgraph retrieval, characterized in that, Includes the following steps: S1. Obtain the dataset and preprocess it to obtain the training set and test set; S2, fine-tunes the hop count predictor, relation classifier, entity classifier and fact classifier based on the preprocessed dataset; S3 uses prompt words and a large language model to extract a list of entities from question-answer pairs; S4, starting from the entities in the entity list, uses the hop count predictor, relation classifier and entity classifier to extract relevant evidence subgraphs from the knowledge graph in collaboration; S5. Based on the obtained evidence subgraph, use a fact classifier to determine the correctness of the original question answer pair.

2. The method for detecting hallucinations in a large language model based on dynamic subgraph retrieval according to claim 1, characterized in that, The training and test sets can be constructed from existing datasets through filtering or transformed using multi-hop knowledge graph question-answering datasets, according to application requirements.

3. The method for detecting hallucinations in a large language model based on dynamic subgraph retrieval according to claim 1 or 2, characterized in that, The hop count predictor performs a multi-class classification task, while the relation classifier, entity classifier, and fact classifier perform a binary classification task. The fine-tuning is a full-parameter fine-tuning based on BERT.

4. The hallucination detection method for a large language model based on dynamic subgraph retrieval according to claim 3, characterized in that, The jump count predictor can predict the distance from an entity to the target answer entity; the relation classifier can determine whether the combination of "question + entity + relation" is relevant; the entity classifier can filter out semantically dissimilar redundant entities; and the fact classifier can determine whether the answer is an illusion.

5. A method for detecting hallucinations in a large language model based on dynamic subgraph retrieval according to claim 1, 2, or 4, characterized in that, Step S3 includes: S31, Generate sub-questions based on the input text question-answer pairs, obtain the answer to the current sub-question based on the original question-answer pairs, and extract the entity information from them; S32, based on the information in S31, further generate sub-problems, repeat the process of S31 until the model has obtained all entity information; S33, deduplicate the entity information to obtain the corresponding entity list.

6. The method for detecting hallucinations in a large language model based on dynamic subgraph retrieval according to claim 5, characterized in that, Step S4 includes: S41, Input the entities and issues in the entity list into the hop count predictor and output the hop count; S42, input the relation classifier in the form of "problem + entity + relation", and output the binary classification result to determine whether to retain the reasoning path where the current relation is located; S43. Input the entity classifier with the form of "entity list + entity", evaluate the semantic alignment between the current entity and the entity list, and determine whether the information is needed.

7. The method for detecting hallucinations in a large language model based on dynamic subgraph retrieval according to claim 6, characterized in that, Step S4 further includes: A complete triple is extracted and added to the evidence subgraph only when both classifiers confirm its usefulness for fact verification; for multi-hop queries, the algorithm recursively processes tail entities, dynamically updating the hop count prediction at each step to guide graph expansion; the recursion terminates when a direct relationship is found or the traversal depth reaches the predefined maximum retrieval depth.

8. A method for detecting hallucinations in a large language model based on dynamic subgraph retrieval according to claim 1, 6, or 7, characterized in that, Step S5 includes: concatenating the question, the answer, and the retrieved evidence subgraph into a text sequence, inputting it into a fact classifier model, and having the model determine whether the question-answer pair is supported by evidence, thereby arriving at the final result of whether it is correct or an illusion.

9. The method for detecting hallucinations in a large language model based on dynamic subgraph retrieval according to claim 8, characterized in that, The text sequence obtained by concatenating the question, the answer, and the retrieved evidence subgraph is input into a word segmenter for word segmentation and special tagging, and then encoded into a tensor. The tensor is then input into the fact classifier model.

10. The method for detecting hallucinations in a large language model based on dynamic subgraph retrieval according to claim 9, characterized in that, After the tensor is input into the fact classifier model, the model performs forward propagation and finally outputs a two-dimensional raw score. The Softmax function is used to convert the raw score into a probability distribution of two classes, and the sum of the two probabilities is 1. The final classification result is obtained by comparing the probabilities of the two classes.

Images