Big model double-channel trusted reasoning method based on fusion of knowledge graph

Abstract

A dual-channel reliable reasoning method for medical questions, integrating a large-scale language model with a knowledge graph. The invention includes the following steps: (1) Receiving a medical query question, performing entity recognition using a large-scale language model and aligning it with a medical knowledge graph; (2) In the first channel, performing path planning and preliminary retrieval to obtain a reasoning path; calling a pre-trained knowledge graph embedding model to calculate the prediction score of each path and determining the highest prediction score; determining whether the highest prediction score is lower than a preset confidence threshold; if not, generating an answer based on the current reasoning path; if so, starting the second channel; (3) The second channel performs iterative multi-hop graph retrieval to generate an enhanced path; finally, the enhanced path is integrated into the large-scale language model to generate a medical answer. This invention, through a confidence-aware dynamic routing mechanism, achieves adaptive switching between lightweight retrieval and deep reasoning, alleviating the illusion problem of large-scale language models while improving the overall performance of medical question answering.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to a large model reasoning framework based on knowledge graphs, and more particularly to a large model reasoning enhancement method that combines knowledge graph embedding to enhance semantic path matching mechanism, aiming to ensure semantic consistency and fault tolerance in medical reasoning and reduce illusions in the large model reasoning process. Background Technology

[0002] Large language models (LLMs) have demonstrated remarkable capabilities in complex reasoning and natural language understanding tasks, and have transformative potential in medical applications such as diagnosis, treatment planning, and clinical decision support [1-3]. Their proficiency in understanding and generating human-like text opens new avenues for assisting physicians and improving healthcare. However, these models have an inherent and critical weakness that severely hinders their application in high-risk medical scenarios: the tendency for “illusion”, where the model generates text that sounds plausible but is actually incorrect or fictitious [5].

[0003] In the medical field, decisions are directly related to patients’ lives and health, so the tolerance for such errors is almost zero. Recent studies have shown that the frequency of hallucinations in medical settings is worrying. A comprehensive survey found that 91.8% of medical professionals have encountered medical hallucinations in their clinical practice, and 84.7% believe that these errors may affect patients’ health[6]. Specific examples include language models fabricating non-existent symptoms such as “leg swelling” when summarizing electronic health records[7], recommending tetracycline, which is contraindicated for pregnant women, when amoxicillin, which should be safe, is recommended[8]. Even in advanced GPT-4o models, the hallucination rate in time-sequencing tasks is as high as 24.6%, and the hallucination rate in laboratory data interpretation is as high as 18.7%[6]. Such errors are reflected in various medical tasks such as patient diagnosis, treatment recommendations, medical image interpretation, and clinical records[9-12]. These hallucinations can lead to serious consequences, from misdiagnosis and incorrect diagnostic reasoning to inappropriate treatment recommendations, which may cause significant harm to patients and bring legal risks.

[0004] This hallucination phenomenon stems from a variety of factors, including biases in the training data, the autoregressive nature of text generation, and the model's inability to access real-time or specialized medical knowledge bases. To mitigate this issue, recent research has explored enhancing large language models with external knowledge.

[0005] Knowledge graphs (KGs), in particular, provide structured and verifiable repositories of facts [13-17]. In the biomedical field, comprehensive knowledge graphs such as the Unified Medical Language System (UMLS)

[18] , DrugBank

[19] , and PrimeKG

[20] integrate millions of entities (e.g., diseases, drugs) and relationships (e.g., “treatment”, “cause”), providing a factual framework for the output of LLMs and reducing illusions by anchoring reasoning to organized knowledge.

[0006] While knowledge graph augmentation methods hold great promise, existing ensemble paradigms reveal key limitations when applied to medical reasoning. Crucially, none of these methods simultaneously possess semantic robustness to handle variations in medical terminology and the fault tolerance mechanisms required to address incomplete or fuzzy queries. Both capabilities are critical for reliable clinical deployment, as accuracy and adaptability are essential in multi-hop reasoning involving drug interactions, contraindications, and comorbidities. Summary of the Invention

[0007] To address these issues, this invention proposes a novel framework for evidence-based reasoning in the medical field. This framework is specifically designed to enhance the reasoning and question-answering capabilities of large language models to meet the stringent requirements of the medical domain regarding illusion. It employs a robust two-channel architecture to ensure stable reasoning. At its core, the first channel implements a semantic verification mechanism that instantiates the abstract reasoning plans generated by the large language model into concrete paths within the medical knowledge graph. These paths are rigorously evaluated using a Knowledge Graph Embedding (KGE) model to assign confidence scores, ensuring that the generated answers are based on structurally sound and verifiable facts. To address situations where initial planning attempts may fail, such as those involving sparse knowledge or complex reasoning, the framework also incorporates a fault-tolerant second channel. This mechanism performs iterative dynamic path construction guided by the large language model, enabling the system to autonomously navigate the graph and synthesize effective reasoning chains when direct retrieval is blocked. Both channels generate explicit reasoning paths with quantitative confidence scores, directly meeting the interpretability and credibility requirements necessary for clinical applications. By combining the output of the LLM with curated knowledge graph evidence and providing transparent reasoning traces, this invention mitigates the risk of illusion in the large model reasoning process. The detailed construction steps of this invention are as follows:

[0008] 1) Perform entity recognition on medical queries and align them with the knowledge graph;

[0009] 2) Fine-tune the large model based on the optimal path and interpretive reasoning process of medical queries in the training set on the knowledge graph;

[0010] 3) Use a fine-tuned large model to search the knowledge graph and obtain evidence paths;

[0011] 4) Calculate semantic coherence S(w) h The filtered evidence set W is obtained from the scores of q).

[0012] 5) First channel output: The model uses the evidence set W to provide the prediction result;

[0013] 6) When channel 1 fails to generate satisfactory reasoning path facts (W is empty), the query entity enters the fault-tolerant channel.

[0014] 7) Starting with the entity, maximize the local utility function Select the next jump node.

[0015] 8) Repeat step 7 until the state termination function is satisfied, then stop iterating and form an evidence path.

[0016] 9) Integrate the evidence paths retrieved from the second channel into the large language model to generate the final medical answer.

[0017] The main contributions of the temporal knowledge graph prediction method of this invention are as follows:

[0018] 1. A dual-channel reasoning framework is introduced, which combines semantic verification based on knowledge graph embedding (KGE) with dynamic path construction. By anchoring the output of the Large Language Model (LLM) to verifiable structured knowledge, the illusion phenomenon is effectively reduced.

[0019] 2. An enhanced knowledge graph embedding (KGE) semantic path matching mechanism is proposed, which comprehensively considers the relevance of the question path and the rationality of the knowledge graph structure to retrieve, verify and evaluate the complete reasoning path of medical queries.

[0020] 3. A fault-tolerant, dynamic path-building channel guided by a large language model was developed to address initial planning failures and knowledge graph sparsity issues, providing alternative inference chains and calibrated confidence scores to support more reliable and transparent medical question answering.

[0021] 4. Comprehensive experimental verification shows that the present invention achieves competitive performance in medical question-answering benchmark tests and provides an interpretable, evidence-based reasoning process, while also demonstrating the stability and robustness of the framework. Attached Figure Description

[0022] Figure 1 This is an overall framework diagram of the method proposed in this invention.

[0023] Figure 2 This describes the performance of the invention when addressing issues of different lengths.

[0024] Figure 3 This is a graph showing the performance of the invention on various datasets, as well as a statistical graph showing the problem length of each dataset.

[0025] Figure 4 This is a visualization of the performance and efficiency of the present invention when using different base models.

[0026] Figures 5(a) and 5(b) show the accuracy performance of the present invention on two datasets for problems with different inference hop counts. Detailed Implementation

[0027] The following section further explains this scheme with reference to the accompanying drawings and specific implementation methods. The first part summarizes the entire framework method; the second part details the two channels and explains the training process in detail; the third part covers the experimental details, including the benchmark dataset used, the initialization of experimental parameters, the presentation of results, and ablation experiments; and the fourth part provides summary comments, summarizing the findings and contributions of this work.

[0028] 1. Overview of the Scheme

[0029] This invention addresses the problems of unreliability and opacity in medical question answering by explicitly combining a large language model with a carefully curated medical knowledge graph to generate answers that are both fact-based and interpretable. Instead of relying solely on the latent knowledge of the large language model (which is susceptible to illusion and fragile in multi-hop reasoning), this invention is designed to operate on structured medical knowledge while maintaining the flexibility of natural language reasoning. Its core idea is a dual-path architecture that uses knowledge graph embedding to connect the semantic plan of the large language model with graph-based evidence, providing a plausible reasoning chain when direct retrieval fails. This design enables the framework to provide explicit reasoning paths and quantitative confidence scores for each clinical query to support the final answer.

[0030] Figure 1 A schematic diagram of the present invention is shown. To achieve reliable medical question answering based on knowledge graphs, the framework proposed in this invention is specifically designed with the following six steps: (1) Entity recognition and alignment: For a given medical query, the biomedical named entity recognition model is used to perform entity recognition on the medical query question to obtain the original entity set E. raw =f NER (Q) = {e1, e2, ..., e} k The original entity set is matched with concepts in the medical knowledge graph to obtain an aligned entity set. (2) First-channel path retrieval: Based on the entities in the aligned entity set, a fine-tuned LLM is used to perform path planning in the medical knowledge graph to obtain relational paths h = (r1, r2, ..., r...). l This allows for an initial retrieval and the acquisition of several reasoning paths. (3) A comprehensive semantic coherence score is introduced to identify the path that is aligned with the query and has the highest joint probability in terms of the rationality of the knowledge graph structure. Based on this semantic coherence score S(w h , q), measure and filter out evidence set W greater than the threshold; (4) First channel output: provide the filtered evidence set W as a prompt to the fine-tuned LLM, and finally generate a natural language answer, which is explicitly based on the provided evidence; (5) Second channel iterative retrieval: when channel 1 cannot generate satisfactory reasoning path facts (W is empty), the fault-tolerant channel will be activated. Starting from the entities in the aligned entity set, perform iterative multi-hop graph retrieval on the medical knowledge graph to obtain enhanced reasoning paths; (6) integrate the enhanced paths retrieved by the second channel into the large language model to generate the final medical answer.

[0031] 2. System Model

[0032] Medical knowledge graphs are formally represented as directed multigraphs. Where ε is a finite set of entities, It is a set of relation types. It is a fact triple set, denoted as (h, r, t), representing the relationship r between the head entity h and the tail entity t.

[0033] Given a natural language question q and its true answer The main objective of this invention's framework is to learn a large-scale model reasoning strategy based on knowledge graphs to maximize the probability of generating the correct answer. This strategy first identifies the optimal evidence subgraph containing the facts needed to answer the question. To operate. The overall learning problem can be formulated as operating on a training dataset for medical question answering. Maximize the expected value of the log-likelihood of the answer:

[0034]

[0035] The system of this invention consists of four core components: entity recognition and alignment, first channel (path planning and preliminary retrieval), second channel (fault-tolerant iterative retrieval), and fusion generation. It achieves more reliable answers to medical questions by identifying, verifying, and reasoning high-quality and semantically coherent reasoning paths in the knowledge graph.

[0036] 2.1 Entity Recognition and Alignment

[0037] For a given natural language medical query, the first step is to anchor it to a structured medical knowledge graph. The BioNER (Biomedical Named Entity Recognition) model is then used to identify entities within the given medical query, obtaining the original entity set E. raw =f NER (Q) = {e1, e2, ..., e} k Subsequently, each entity in the original entity set is matched with a concept in the medical knowledge graph to obtain an aligned entity set:

[0038]

[0039] Where θ link This is the similarity threshold.

[0040] 2.2 First Channel: Path Planning and Retrieval Verification

[0041] This channel implements a semantic verification mechanism that instantiates abstract reasoning plans generated by large language models into specific paths in a medical knowledge graph.

[0042] First, path planning and preliminary retrieval are performed. For a given medical query, based on the entities in the aligned entity set, a fine-tuned Large Language Model (LLM) is used to plan paths in the medical knowledge graph to obtain several inference paths. Each of the planned paths is h = (r1, r2, ..., r...). l () represents a strategy assumption, which guides the retrieved inference path to form the initial retrieval results of the first channel. The target loss for matching the optimal path is specifically defined as:

[0043]

[0044] Here, P θ (r i |r <i ,q) represents the large model based on problem q and relation sequence r <i Generate relation r i The conditional probability. For each path planning h = (r1, r2, ..., r... l The framework of this invention retrieves all corresponding reasoning path instances from the knowledge graph G, denoted as... Path Instance It is a sequence of connected triples starting from the subject entity e0 identified in the problem.

[0045] To identify the path that aligns with the query and has the highest joint probability in terms of knowledge graph structural rationality, this invention introduces a comprehensive semantic coherence score S(w hThe scoring mechanism invokes a pre-trained Knowledge Graph Embedding Model (KGE), allowing the plausibility of any triple (h, r, t) to be evaluated using an evaluation function Φ(·). The confidence score of a triple (h, r, t) is specifically defined as:

[0046]

[0047] Comprehensive scoring function S(w) h q) consists of two parts, and its specific formula is as follows:

[0048]

[0049] The first part is the Query-Path Relevance, which is calculated by the average vector cosine similarity between the question's vector embedding g(q) and each relation f(r′) in the path. A higher score indicates that the relations in the path are topically relevant to the query, thus ensuring that the reasoning process is closely related to the topic and highlights the key points. The second part is the Structural Plausibility, which is the geometric mean of the reasonableness scores derived from the triples on the pre-trained KGE model. A higher score indicates that the entire fact chain is internally consistent and strongly supported by reliable factual knowledge in the knowledge graph.

[0050] The system uses the semantic coherence score as a metric to select evidence sets W that score above a threshold. If the evidence set is valid, the selected evidence set W is provided as a prompt to the fine-tuned LLM, ultimately generating a natural language answer that is explicitly based on the provided evidence.

[0051] 2.3 Second Channel: Fault-Tolerant Iterative Retrieval

[0052] To address situations with sparse knowledge or complex reasoning, the system activates this fault-tolerant channel when channel 1 fails to generate satisfactory reasoning path facts (i.e., the evidence set W is empty). This channel aligns with the entity set E. q The entities in the search are seed nodes as the starting point to limit the search space and make the exploration more efficient.

[0053] In each layer of expansion on the graph, the model must decide which neighboring node to visit to expand the current path. The system selects the next jump node (r) by maximizing the local utility function Z. k+1 e k+1 The exploration strategy is specifically defined as follows:

[0054]

[0055] Here, N(e) k) represents the current entity e k The set of all outgoing triples. This invention introduces a utility function Z to guide the search towards the most promising expansion direction. The utility function Z combines the transition probabilities of LLM with the aforementioned semantic coherence, and is specifically defined as:

[0056]

[0057] in Is it LLM in the current inference path? The following are possible new paths that can be generated. First item. This is the LLM's relevance assessment of the potential next node given a currently constructed path. (Second item) The comprehensive semantic coherence score from Equation 4 is then reused to evaluate the quality of the entire potential path. The hyperparameter β∈[0,1] is used to balance the immediate suggestions of LLM with the overall semantic and structural quality of the path.

[0058] After each jump, the working subgraph expands to include the neighborhood of the newly arrived entity. This process is repeated until the preset maximum number of jumps L is reached. max (For example, set to 3) or the maximum utility Z when selecting the next iteration during the model iterative retrieval process. max Less than the minimum threshold τ. The termination state function U of this process is defined as:

[0059]

[0060] 2.4 Fusion Generation Output

[0061] Both the evidence set directly generated from the first channel and the enhanced paths retrieved from the second channel will ultimately be fused as enhanced reasoning paths and passed to the fine-tuned large language model generator. The model generates medical answers based on the retrieved reasoning paths and structured facts, along with explanations based on the knowledge graph-based reasoning paths. The specific goals of this generation process are:

[0062]

[0063] in, The correct answer A represents the given candidate reasoning path W. q The conditional probability, t i A represents q The labeled sequences. By jointly training the LLM on the planning task L1 and the reasoning task L2, we create a unified model that can both generate and explain knowledge graph-based reasoning paths.

[0064] Algorithm 1: A Large-Scale Dual-Channel Reliable Medical Inference Algorithm Based on Knowledge Graph

[0065] Data: Medical Inquiry Q

[0066] Medical knowledge graph

[0067] Biomedical named entity recognition model f NER Large Language Model (LLM) fine-tuned with entity similarity function Sim(·,·) θ The evaluation function Φ(·) of the pre-trained knowledge graph embedding model is based on the preset alignment threshold θ. link Preset confidence threshold $\tau$ Preset minimum utility threshold T min Preset maximum number of jumps L max Balance parameters α, β Result: Final medical answer A *

[0068]

[0069]

[0070] 3 Experimental Design

[0071] 3.1 Dataset

[0072] In our experiment on a dual-channel reliable reasoning framework for large-scale models that integrates knowledge graphs, we introduced the Unified Medical Language System (UMLS) as an external knowledge graph, which integrates millions of entities (e.g., diseases, drugs) and relationships (e.g., “treatment”, “cause”). To evaluate the performance of medical question answering tasks, we used five challenging public medical benchmark datasets, including MedQA, MedQA(Chinese)

[23] , MMLU-Pro

[24] , MedMCQA

[25] , and PubMedQA

[26] . We selected a subset of medical questions from these datasets that required detailed reasoning. These datasets not only test the model’s semantic robustness to changes in medical terminology, but also require the model to handle incomplete or fuzzy queries and complex multi-entity scenarios in medical queries.

[0073] 3.2 Baseline Model

[0074] To verify the effectiveness of this invention, we selected several representative large-scale language models and knowledge graph-based augmented reasoning models for comparison. The benchmark models cover standard generative models without external retrieval, as well as existing knowledge graph-based augmented question-answering paradigms.

[0075] 1. KGR

[27] (Knowledge Graph Reasoning): A graph multi-hop reasoning framework that combines large models. It alleviates the illusion of large models through rigorous structured retrieval. KGR aims to derive new knowledge based on existing facts in the graph. When combined with LLM, this type of model usually uses KG as a source of background knowledge, extracts local subgraphs and knowledge paths, and aligns and integrates them with text queries.

[0076] 2. KG-SFT

[22] (Knowledge Graph-Driven Supervised Fine-Tuning): KG-SFT is a knowledge graph-driven supervised fine-tuning framework, mainly used to solve the problem of insufficient knowledge understanding and operation capabilities of LLM when fine-tuning in specific vertical fields (such as medical and other low-data, knowledge-intensive scenarios).

[0077] 3. KAPING

[28] (Knowledge-Augmented language model PromptING): KAPING is a zero-shot knowledge enhancement prompting framework. It does not require any parameter training or fine-tuning of the LLM. Its workflow is very straightforward: extract entities from the question and link them to the KG, then convert the retrieved symbolic triples (head entity, relation, tail entity) into natural language text strings, calculate vector similarity, and retain the Top-K facts. Finally, inject the facts as prompts into the LLM to generate the answer.

[0078] 4. StructGPT

[29] : A general framework designed to improve the zero-shot reasoning ability of LLM on structured data (including knowledge graphs, tables, and databases). It provides LLM with dedicated interfaces to collect evidence from structured data (i.e., the Reading stage). After the collection is completed, LLM focuses on logical reasoning based on this information.

[0079] 5. ROG

[21] (Reasoning-on-Graph): Proposes a theoretical planning-retrieval-reasoning framework, enabling large models to perform better graph retrieval and reason based on the retrieved information. This method improves the ability of large models to answer questions based on knowledge graphs.

[0080] 6. ToG

[15] (Think-on-Graph): It is a plug-and-play, training-free deep reasoning framework that couples the reasoning process of LLM with the graph retrieval process of knowledge graph to handle multi-hop reasoning problems.

[0081] The experiment aimed to test the effectiveness of these methods in alleviating medical hallucinations and improving the accuracy of responses.

[0082] 3.3 Model Evaluation and Training Parameter Settings

[0083] In the semantic coherence scoring module of the first channel, the hyperparameter α, which balances the relevance of the problem path and the rationality of the structure, is set to 0.4. In the fine-tuning of the first channel, and in the utility function evaluation of the second channel, the parameter β, which balances the prior probability of the large model and semantic coherence, is set to 0.3. The minimum utility threshold τ in the termination state function is set to 0.25, and the maximum number of hops L is... max The limit is set to 3 jumps to prevent an exponential explosion within the vast medical landscape.

[0084] The primary evaluation metric is accuracy, which indicates whether the answer provided by the large model precisely matches the answers provided in the question-answering dataset. This is assessed using an automated fact-checking protocol.

[0085] 3.4 Results

[0086] Table 1 presents the experimental results, showing the accuracy comparison between the present invention and six baseline models for knowledge graph enhancement (TOG, RoG, KG-SFT, KGR, KAPING, and StructGPT) on five medical question-answering datasets (including MedQA, MedQA (Chinese), MMLU-Pro Medical Subset, MedMCQA, and PubMedQA).

[0087] As shown in Table 1, it demonstrates strong generalization ability across different languages, question formats, and clinical reasoning needs.

[0088] The performance parameters in Table 1 are presented as percentages, with bold text indicating the optimal result.

[0089] Table 1: Experimental results using different pedestal models on five medical datasets

[0090]

[0091] In the English version of the MedQA benchmark, REMEDY achieved an accuracy of 71.52%, outperforming the top-performing baseline RoG by 5.1 percentage points and more than 10 percentage points above ToG. This significant performance advantage on a dataset containing USMLE-style questions demonstrates that our semantic path matching mechanism effectively identifies clinically relevant reasoning chains, even in complex diagnostic scenarios. In the Chinese version of MedQA, REMEDY achieved an accuracy of 82.61%, outperforming ToG by 2.5 percentage points and nearly 7 percentage points above RoG. The continuous improvement achieved in both language versions indicates that the framework's knowledge graph alignment strategy can generalize beyond English medical terminology, a crucial feature for clinical applications in multilingual healthcare environments.

[0092] For the MMLU-Pro medical dataset, which contains expert-level professional exam questions, REMEDY achieved an accuracy of 86.54%, while ToG and RoG achieved 84.34% and 83.73%, respectively. Although the absolute difference is small in this challenging benchmark, our method still performs best among all evaluated methods. In PubMedQA, an evidence-based biomedical reasoning task requiring the integration of scientific summaries, our method achieved an accuracy of 93.25%, 2.5 percentage points higher than RoG and nearly 15 percentage points higher than ToG. On the MedMCQA dataset, which contains medical graduate entrance exam questions, our method achieved an accuracy of 72.66%. RoG scored higher on this particular benchmark at 75.96%, making it the only dataset where our framework did not achieve the best results. However, our method still significantly outperforms ToG and the remaining benchmark models. RoG's relatively strong performance on MedMCQA may reflect specific characteristics of this dataset that favor its relation path generation strategy. Future research is needed to further explore this pattern.

[0093] These results demonstrate that the performance improvement of our method stems from the collaborative design of its dual-channel architecture. KGE-enhanced semantic path matching in Channel 1 ensures that retrieved inference paths are semantically aligned with clinical questions and structurally plausible within the knowledge graph, thus filtering out spurious or illusory connections. When direct path retrieval fails due to sparsity of the knowledge graph or query complexity, fault-tolerant dynamic path building in Channel 2 enables the framework to iteratively construct alternative inference chains rather than reverting to unfounded LLM generation. The quantitative confidence score attached to each inference path further provides transparency, contributing to building clinical trust. Notably, the robust improvements observed on exam-style benchmarks (MedQA, MedMCQA), professional medical knowledge assessment (MMLU-Pro), and evidence-based reasoning tasks (PubMedQA) indicate that REMEDY's mechanism is beneficial for various forms of medical question answering.

[0094] 3.5 Ablation Experiment

[0095] To better illustrate the contribution of the dual-channel design in this invention, we conducted ablation experiments. Table 2 shows the ablation experimental results on the MedQA dataset for a single-channel module (using only channel 1 or only channel 2) and the complete dual-channel framework.

[0096] To further verify the independent contributions and synergistic effects of each core component in the "dual-channel knowledge graph reliable medical reasoning framework" proposed in this invention, we conducted detailed ablation experiments on four mainstream open-source large language models (Llama3-8B

[31] , Glm4-9B

[32] , DeepSeek-R1-8B

[33] , and Qwen3-8B

[30] ). The experiments mainly examined the accuracy performance of this framework after removing a certain channel:

[0097] Table 2: The impact of the two channels of the knowledge graph on model performance

[0098]

[0099] The experimental results in Table 2 clearly show that the complete dual-channel framework (Ours) achieved the best prediction accuracy across all base large models. Compared to the pure large model baseline (LLM-only), the introduction of the complete framework resulted in a 6.36% improvement for Llama3-8B, a 10.11% improvement for DeepSeek-R1-8B, an 11.16% improvement for Qwen3-8B, and a significant leap of 17.30% for Glm4-9B. This strongly demonstrates that the introduction of an external structured medical knowledge graph can greatly compensate for the knowledge sparsity in the LLM domain, significantly alleviate medical illusions, and improve the objectivity and correctness of generated answers.

[0100] When using only the first channel, all four models showed significant accuracy improvements compared to LLM-only (e.g., Qwen3-8B improved from 60.36% to 66.46%). This indicates that utilizing the prior knowledge of the large model for directed planning on the graph, supplemented by KGE semantic coherence verification, can effectively filter irrelevant noise and provide high-quality direct inference evidence for the model. Using only the second channel also outperformed the baseline model. Notably, on Llama3-8B, Channel-2 (58.20%) even slightly outperformed Channel-1 (57.81%) in single-node performance. This suggests that when the basic model's instruction following or initial path planning capabilities are weak, graph-based iterative multi-hop retrieval (Channel-2), which does not rely on LLM deep planning and starts directly from entities, can serve as an extremely robust backup knowledge supplement.

[0101] The core finding of the ablation experiments is the complementary synergistic effect of the two channels. Experimental results show that activating both channels simultaneously in this framework can outperform a single channel. For example, on Glm4-9B, performance bottlenecks were encountered when using Channel-1 (58.24%) or Channel-2 (55.46%) alone, but improved to 67.24% under the complete framework. This demonstrates the "dynamic routing and fault tolerance mechanism" of this invention: when the medical query is relatively direct or the model has strong planning capabilities, the first channel can accurately hit the target and output a high-quality answer; when the first channel cannot generate a coherent evidence path (confidence W is empty) due to sparse graph knowledge, ambiguous entity alignment, or the need for extremely complex multi-hop reasoning, the fault tolerance mechanism of the second channel is seamlessly activated. This dynamic adaptive switching ensures that the model achieves the best balance between "lightweight accurate retrieval" and "deep exploratory reasoning," preventing situations such as "direct rejection of answer" or "forced generation of illusions" caused by the failure of a single retrieval path.

[0102] 4. Conclusion

[0103] This invention proposes a dual-channel reliable reasoning framework for a large-scale model that integrates knowledge graphs in the medical field. This framework employs a strategy combining semantic verification based on knowledge graph embedding (KGE) with dynamic path construction. By anchoring the output of the large language model (LLM) to verifiable structured knowledge, it effectively reduces the illusion phenomenon. This method not only proposes an enhanced knowledge graph embedding semantic path matching mechanism to verify and evaluate medical query reasoning paths, but also develops a fault-tolerant, LLM-guided dynamic multi-hop retrieval channel to address the knowledge sparsity problem. Comprehensive experimental validation shows that this method not only achieves superior accuracy in medical question-answering benchmarks, but also provides an interpretable, evidence-based reasoning process, greatly improving the transparency, stability, and robustness of medical intelligent question-answering systems.

[0104] References:

[0105] [1] Thirunavukarasu, A.J., Ting, D.S.J., Elangovan, K., Gutierrez, L., Tan, T.F., Ting, D.S.W.: Large language models in medicine. Nature medicine 29(8), 1930 - 1940(2023)

[0106] [2] Lucas, H.C., Upperman, J.S., Robinson, J.R.: A systematic review of large lan - guage models and their implications in medical education. Medical education 58(11), 1276 - 1285(2024)

[0107] [3] Liu, F., Zhou, H., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S.S., Hua, Y., Zhou, P., et al.: Application of large language models in medicine. Nature Reviews Bioengineering, 1 - 20(2025)

[0108] [4] Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al.: Siren’s song in the ai ocean: A survey on hallucination in large language models. Computational Linguistics, 1 - 45(2025)

[0109] [5]Huang,L.,Yu,W.,Ma,W.,Zhong,W.,Feng,Z.,Wang,H.,Chen,Q.,Peng,W.,Feng,X.,Qin,B.,et al.:A survey on hallucination in large language mod-els:Principles,taxonomy,challenges,and open questions.ACM Transactions onInformation Systems 43(2),1-55(2025)

[0110] [6]Kim,Y.,Jeong,H.,Chen,S.,Li,S.S.,Lu,M.,Alhamoud,K.,Mun,J.,Grau,C.,Jung,M.,Gameiro,R.,et al.:Medical hallucinations in foundation models andtheir impact on healthcare.arXiv preprint arXiv:2503.05777(2025)

[0111] [7]Vishwanath,P.R.,Tiwari,S.,Naik,T.G.,Gupta,S.,Thai,D.N.,Zhao,W.,KWON,S.,Ardulov,V.,Tarabishy,K.,McCallum,A.,et al.:Faithfulness hal-lucination detection in healthcare ai.In:Artificial Intelligence and DataScience for Healthcare:Bridging Data-Centric AI and People-Centric Healthcare(2024)

[0112] [8]Pal,A.,Umapathi,L.K.,Sankarasubbu,M.:Med-halt:Medical domainhalluci-nation test for large language models.In:Proceedings of the 27thConference on Computational Natural Language Learning,pp.314-334(2023)

[0113] [9]Goodman,K.E.,Paul,H.Y.,Morgan,D.J.:Ai-generated clinical summariesrequire more than accuracy.Jama 331(8),637-638(2024)

[0114]

[10] Chen,J.,Yang,D.,Wu,T.,Jiang,Y.,Hou,X.,Li,M.,Wang,S.,Xiao,D.,Li,K.,Zhang,L.:Detecting and evaluating medical hallucinations in large visionlanguage models.arXiv preprint arXiv:2406.10185(2024)

[0115]

[11] Zhang,S.,Sambara,S.,Banerjee,O.,Acosta,J.N.,Fahrner,L.J.,Rajpurkar,P.:Radflag:A black-box hallucination detection method for medicalvision language models.In:Machine Learning for Health(ML4H),pp.1087-1103(2025)

[0116]

[12] Asgari,E.,Monta~na-Brown,N.,Dubois,M.,Khalil,S.,Balloch,J.,Yeung,J.A.,Pimenta,D.:A framework to assess clinical safety and hallucinationrates of llms for medical text summarisation.npj Digital Medicine 8(1),274(2025)

[0117]

[13] Pan,S.,Luo,L.,Wang,Y.,Chen,C.,Wang,J.,Wu,X.:Unifying largelanguage models and knowledge graphs:A roadmap.IEEE Transactions on Knowledgeand Data Engineering 36(7),3580-3599(2024)

[0118]

[14] Yang,L.,Chen,H.,Li,Z.,Ding,X.,Wu,X.:Give us the facts:Enhancinglarge language models with knowledge graphs for fact-aware languagemodeling.IEEE Transactions on Knowledge and Data Engineering 36(7),3091-3110(2024)

[0119]

[15] Sun,J.,Xu,C.,Tang,L.,Wang,S.,Lin,C.,Gong,Y.,Ni,L.,Shum,H.-Y.,Guo,J.:Think-on-graph:Deep and responsible reasoning of large language model onknowledge graph.In:Proceedings of the 12th International Conference onLearning Representations(2024)

[0120]

[16] Wen,Y.,Wang,Z.,Sun,J.:Mindmap:Knowledge graph prompting sparksgraph of thoughts in large language models.In:Proceedings of the 62nd AnnualMeeting of the Association for Computational Linguistics,pp.10370-10388(2024)

[0121]

[17] Tan,X.,Wang,X.,Liu,Q.,Xu,X.,Yuan,X.,Zhang,W.:Paths-over-graph:Knowledge graph empowered large language model reasoning.In:Proceedings ofthe ACM on Web Conference 2025,pp.3505-3522(2025)

[0122]

[18] Bodenreider,O.:The unified medical language system(umls):integrating biomed-ical terminology.Nucleic acids research 32,267-270(2004)

[0123]

[19] Wishart,D.S.,Feunang,Y.D.,Guo,A.C.,Lo,E.J.,Marcu,A.,Grant,J.R.,Sajed,T.,Johnson,D.,Li,C.,Sayeeda,Z.,et al.:Drugbank 5.0:a major update tothe drugbank database for 2018.Nucleic acids research 46,1074-1082(2018)

[0124]

[20] Chandak,P.,Huang,K.,Zitnik,M.:Building a knowledge graph toenable precision medicine.Scientific Data 10(1),67(2023)

[0125]

[21] Luo,L.,Li,Y.,Haffari,G.,Pan,S.:Reasoning on graphs:Faithful andinter-pretable large language model reasoning.In:Proceedings of the 12thInternational Conference on Learning Representations(2024)

[0126]

[22] Chen,H.,Shen,X.,Wang,J.,Wang,Z.,Lv,Q.,He,J.,Wu,R.,Wu,F.,Ye,J.:Knowledge graph finetuning enhances knowledge manipulation in large lan-guagemodels.In:Proceedings of the 13th International Conference on LearningRepresentations(2025)

[0127]

[23] Jin,D.,Pan,E.,Oufattole,N.,Weng,W.-H.,Fang,H.,Szolovits,P.:Whatdisease does this patient have?a large-scale open domain question answeringdataset from medical exams.Applied Sciences 11(14),6421(2021)

[0128]

[24] Wang,Y.,Ma,X.,Zhang,G.,Ni,Y.,Chandra,A.,Guo,S.,Ren,W.,Arulraj,A.,He,X.,Jiang,Z.,et al.:Mmlu-pro:A more robust and challenging multi-tasklanguage understanding benchmark.Advances in Neural Information ProcessingSystems 37,95266-95290(2024)

[0129]

[25] Pal,A.,Umapathi,L.K.,Sankarasubbu,M.:Medmcqa:A large-scale multi-subject multi-choice dataset for medical domain question answering.In:Confer-ence on Health,Inference,and Learning,pp.248-260(2022)

[0130]

[26] Jin,Q.,Dhingra,B.,Liu,Z.,Cohen,W.,Lu,X.:Pubmedqa:A dataset forbiomedical research question answering.In:Proceedings of the 2019 Conferenceon Empirical Methods in Natural Language Processing and the 9th InternationalJoint Conference on Natural Language Processing,pp.2567-2577(2019)

[0131]

[27] Guan,X.,Liu,Y.,Lin,H.,Lu,Y.,He,B.,Han,X.,Sun,L.:Mitigating largelan-guage model hallucinations via autonomous knowledge graph-basedretrofitting.In:Proceedings of the AAAI Conference on ArtificialIntelligence,vol.38,pp.18126-18134(2024)

[0132]

[28] Baek,J.,Aji,AF,Saffari,A.:Knowledge-augmented language model prompting for zero-shot knowledge graph question answering.

[0133]

[29] Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, WX, Wen, J.-R.:Structgpt:A general framework for large language model to reason over structured data.In:Proceedings of the 2023 Conference on Empirical Methods in Natural LanguageProcessing,pp.9237-9251(2023)

[0134]

[30] Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.

[0135]

[31] Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al.

[0136]

[32] GLM,T.,Zeng,A.,Xu,B.,Wang,B.,Zhang,C.,Yin,D.,Zhang,D.,Rojas,D.,Feng,G.,Zhao,H.,et al.:Chatglm:A family of large language models from glm-130b to glm-4 all tools.arXiv preprint arXiv:2406.12793(2024)

[0137]

[33] Guo,D.,Yang,D.,Zhang,H.,Song,J.,Wang,P.,Zhu,Q.,Xu,R.,Zhang,R.,Ma,S.,Bi,X.,et al.:Deepseek-r1 incentivizes reasoning in llms throughreinforcement learning.Nature 645(8081),633-638(2025)。

Claims

1. A large model double-channel trusted inference method of fusing a knowledge graph, characterized in that, Includes the following steps: 1) Entity Recognition and Alignment: For a given medical query, the biomedical named entity recognition model is used to identify entities in the medical query question to obtain an original entity set. Each entity in the original entity set is matched with the concept in the medical knowledge graph to obtain an aligned entity set. 2) First-channel path retrieval: Based on the entities in the aligned entity set, the fine-tuned LLM is used to perform path planning and preliminary retrieval in the medical knowledge graph to obtain several reasoning paths, which constitute the preliminary retrieval results; 3) Introduce a comprehensive semantic coherence score to identify the path that aligns with the query and has the highest joint probability in terms of the rationality of the knowledge graph structure. Based on this semantic coherence score, filter out the evidence set W that exceeds a threshold; 4) First channel output: The selected evidence set is provided as a prompt to the fine-tuned LLM, which ultimately generates a natural language answer that is explicitly based on the provided evidence; 5) Second Channel Iterative Retrieval: When Channel 1 fails to generate satisfactory inference path facts (W is empty), this fault-tolerant channel is activated. Starting from entities in the aligned entity set, iterative multi-hop graph retrieval is performed on the medical knowledge graph to obtain enhanced inference paths; 6) The enhanced paths retrieved from the second channel are fused into the large language model to generate the final medical answer. In step 2), based on the entities in the aligned entity set, path planning and preliminary retrieval are performed in the medical knowledge graph to obtain several reasoning paths. Each of the planned paths is h = (r1, r2, ..., r l () represents a strategy assumption, which is used to plan the inference path retrieved to form the first channel retrieval result; the target loss for matching the optimal path is specifically defined as: In step 3, a pre-trained knowledge graph embedding model (KGE) is used so that the validity of any triple can be evaluated by an evaluation function Φ(·). The confidence level of a triple (h, r, t) is specifically defined as follows: The semantic coherence score S(Wh, q) is composed of different parts, and its specific scoring function is as follows: The first part measures the relevance between the path and the initial query, calculated by the average vector cosine similarity between the question's vector embedding g(q) and each relation f(r') in the path. The second part is the geometric mean of the reasonableness scores derived from the triples on a pre-trained KGE model. The iterative multi-hop graph search in step 5 specifically includes the following sub-steps: Step 5-1: Take entities in the aligned entity set E q as seed nodes to limit the search space and make the exploration more efficient: Step 5-2: In each layer of expansion, the fine-tuned model must decide which neighboring node to visit to expand the current path. The next jump node (r) is selected by maximizing the local utility function Z. k+1 e k+1 The exploration strategy is specifically defined as: Utility function Specifically defined as: Step 5-3: Repeat Step 5-2 until a preset maximum hop count 3 or any possible next maximum utility Z is reached max Less than the minimum threshold. The abort state function U is defined as: Step 5-4: Pass all dynamically constructed path sets as enhanced reasoning paths to the fine-tuned generator LLM to generate the final answer and an explanation based on the knowledge graph-based reasoning path. This answer is explicitly based on the provided evidence and follows the specific objectives:

Images