An auxiliary decision-making and precise diagnosis and treatment system based on interpretable multi-modal reasoning

Abstract

This invention discloses an auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning, belonging to the field of medical informatics and clinical decision support technology. It includes: a data acquisition module for acquiring DKD clinical medical knowledge and multi-source patient data; a feature extraction module for processing the multi-source patient data to obtain a time-series data view of the patient; wherein the time-series data view includes: a fact table and a wide index table; multiple patient time-series data views constitute a structured feature warehouse; a knowledge graph construction module for constructing a medical knowledge graph based on DKD clinical medical knowledge; and a local reasoning module for constructing a local reasoning model and performing retrieval reasoning based on the structured feature warehouse and the medical knowledge graph to obtain the decision result.

Description

Technical Field

[0001] This invention belongs to the field of medical information technology and clinical decision support technology, specifically relating to an auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning. Background Technology

[0002] Diabetic nephropathy, one of the most serious chronic complications of diabetes, is closely related to the persistent damage to the microvessels of the kidneys caused by long-term hyperglycemia. Clinical manifestations include proteinuria and edema, and in advanced stages, it can lead to kidney failure, posing a serious threat to patients' lives and health. With the continuous increase in the global prevalence of diabetes, the number of people suffering from diabetic nephropathy is also showing a year-on-year upward trend, placing a heavy burden on the social healthcare system. Therefore, precise diagnosis and efficient management of diabetic nephropathy have become crucial for improving the quality of life of diabetic patients and reducing medical costs, possessing strong clinical urgency and social importance.

[0003] Currently, while some technical methods have been developed for the diagnosis and management of diabetic nephropathy in clinical practice, significant shortcomings remain in practical applications. At the data management level, the integration of data from different sources, such as hospital information systems (HIS / EMR), varies considerably. Diabetic nephropathy management data is fragmented and heterogeneous, preventing the full utilization of cross-source temporal characteristics and causal factors. At the clinical decision support level, general medical models or online clinical decision support systems (CDSS) can provide some treatment suggestions, but their adaptability to local data is poor, and the evidence and suggestions lack interpretability and traceability, with insufficient clinical safety safeguards. Search enhancement (RAG) technology is gradually being applied in the medical question-and-answer field, but it lacks structured constraints and clinical safety safeguards, easily leading to inconsistent or non-compliant suggestions. At the system deployment and efficacy evaluation level, existing AI tools are highly dependent on external systems, making it difficult to guarantee patient privacy and data compliance, and resulting in high costs for local deployment and maintenance.

[0004] In summary, the current field of decision support and treatment management for diabetic nephropathy faces challenges such as difficulties in data integration, insufficient process standardization, and a lack of reliable decision support. Therefore, there is an urgent need for a decision support and precision treatment system based on interpretable multimodal reasoning to address the problems existing in current technologies. Summary of the Invention

[0005] In view of this, the present invention provides an auxiliary decision-making and precision diagnosis and treatment system based on interpretable multimodal reasoning, which is used to solve the problems of data integration difficulties, insufficient process standardization, and lack of reliability of decision support in the field of auxiliary decision-making and diagnosis and treatment management of diabetic nephropathy.

[0006] To achieve the above objectives, the present invention provides an auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning, comprising: The data acquisition module is used to acquire multi-source data on DKD clinical medical knowledge and patients; The feature extraction module is used to process multi-source patient data to obtain a time-series data view of the patient; the time-series data view includes: a fact table and a wide index table; multiple patient time-series data views constitute a structured feature warehouse; The knowledge graph construction module is used to build a medical knowledge graph based on DKD clinical medical knowledge. The local reasoning module is used to build a local reasoning model and perform retrieval reasoning based on the structured feature warehouse and medical knowledge graph to obtain decision results.

[0007] As an embodiment of the present invention, the feature extraction module performs the following operations: Based on the preset data processing method, the patient's multi-source data is processed to obtain a time-series data view; the patient's multi-source data includes: electronic medical record text, laboratory test data, imaging and bedside equipment data, and follow-up and medication records.

[0008] As an embodiment of the present invention, the knowledge graph construction module performs the following operations: Based on the knowledge graph construction method, a medical knowledge graph is constructed according to DKD clinical medical knowledge. When constructing the medical knowledge graph, biomarkers, clinical status, drugs and contraindications, examinations and follow-up nodes are used as entities, and indications, contraindications, interactions, levels of evidence and follow-up strategies are used as relationships. Authoritative KDIGO or ADA guidelines are used as graph constraints.

[0009] As an embodiment of the present invention, the local inference module performs the following operations: Based on the domain distillation and model merging strategy, a local inference model is constructed by fine-tuning the model based on a general inference model and a medical domain corpus. The process involves acquiring the reasoning task, determining the task type, and then defining the decision prompt template and JSON output pattern based on the task type. The task types include: diagnosis, efficacy evaluation, and prognosis prediction. Each task type corresponds to a specific decision prompt template and JSON output pattern. Based on the local reasoning model, reasoning tasks are performed using structured feature binaries and medical knowledge graphs to obtain decision results.

[0010] As an embodiment of the present invention, the local inference model also performs the following operations: When the local inference model obtains the decision result, it performs temperature scaling and confidence estimation on the decision result to obtain a confidence score; Output the confidence score and decision result to the corresponding output terminal.

[0011] As an embodiment of the present invention, the output terminal performs the following operations: Obtain decision results and confidence scores; The decision results are subjected to semantic consistency and numerical verification to obtain verification data.

[0012] As one embodiment of the present invention, it also includes: The risk stratification and efficacy assessment module is used to evaluate intervention recommendations for patients to obtain risk stratification and efficacy assessment results, and to automatically intercept or prompt for review any intervention recommendations that do not meet the medical knowledge graph; among them, intervention recommendations are given by physicians or are the result of their decisions.

[0013] The beneficial effects of this invention are as follows: by integrating time-series data and knowledge graphs, the collaborative model makes diagnosis / stratification / prognosis more robust, providing consistent recommendations across departments and physicians; it can also provide individualized strategies (i.e., decision results) based on the patient's physiological and treatment trajectory (time-series data view corresponding to individual patients); each decision result provides evidence anchors, constraint sources, and feature contributions, significantly reducing the risk of a "black box" and facilitating clinical review; the local inference module performs local inference to meet privacy and regulatory requirements, and offline / edge deployment reduces external dependence and data out-of-domain risks; it solves the problems of data integration difficulties, insufficient process standardization, and lack of reliability in decision support in the field of diabetic nephropathy auxiliary decision-making and treatment management.

[0014] Other advantages, objectives, and features of the invention will be set forth in the following description and will be apparent to those skilled in the art in some respects, or may be learned by practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0015] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a schematic diagram of the modules of the present invention. Detailed Implementation

[0016] like Figure 1 As shown, this invention provides an auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning, comprising: The data acquisition module is used to acquire multi-source data on DKD clinical medical knowledge and patients; The feature extraction module is used to process multi-source patient data to obtain a time-series data view of the patient; the time-series data view includes: a fact table and a wide index table; multiple patient time-series data views constitute a structured feature warehouse; The knowledge graph construction module is used to build a medical knowledge graph based on DKD clinical medical knowledge. The local reasoning module is used to build a local reasoning model and perform retrieval reasoning based on the structured feature warehouse and medical knowledge graph to obtain decision results.

[0017] The working principle of the above technical solution is as follows: In actual use, the data acquisition module acquires multi-source data of DKD clinical medical knowledge and all DKD patients corresponding to the hospital, and uses this data to construct a medical knowledge graph through the knowledge graph construction module, forming a safety barrier for strategy recommendations. Simultaneously, the feature extraction module processes the multi-source data of several patients in a unified manner to form a time-series data view. This time-series data view includes: a fact table (a table in the data warehouse used to record specific business events, typically containing detailed event-related information such as patient ID, consultation time, and event type) and a wide indicator table (a wide-format table generated based on the fact table through summarization and calculation, used to present analytical indicators such as total number of patient visits and average consultation cost). Finally, the local reasoning module constructs a local reasoning model to perform retrieval and reasoning on the local structured feature warehouse and the medical knowledge graph to obtain the decision result. It is worth noting that the system architecture in the technical solution of this invention is applicable to multiple diseases such as cardiovascular and metabolic diseases, and the knowledge base and rules can be iteratively expanded to form a sustainable intelligent diagnosis and treatment platform within the hospital. The beneficial effects of the above technical solution are as follows: By integrating time-series data and knowledge graphs, the collaborative model makes diagnosis / stratification / prognosis more robust, and provides consistent recommendations across departments and physicians; it can also provide individualized strategies (i.e., decision results) based on the patient's physiological and treatment trajectory (time-series data view corresponding to individual patients); each decision result provides evidence anchors, constraint sources, and feature contributions, significantly reducing the risk of a "black box" and facilitating clinical review; the local inference module performs local inference to meet privacy and regulatory requirements, and offline / edge deployment reduces external dependence and data out-of-domain risks; it solves the problems of data integration difficulties, insufficient process standardization, and lack of reliability in decision support in the field of diabetic nephropathy auxiliary decision-making and treatment management.

[0018] In one embodiment, the feature extraction module performs the following operations: Based on the preset data processing method, the patient's multi-source data is processed to obtain a time-series data view; the patient's multi-source data includes: electronic medical record text, laboratory test data, imaging and bedside equipment data, and follow-up and medication records.

[0019] The working principle and beneficial effects of the above technical solution are as follows: The feature extraction module and data acquisition module collect and process multi-source patient data, supporting batch and drag-and-drop import of electronic medical record text, laboratory test (LIS) data, imaging and bedside equipment data, follow-up and medication records, as well as unstructured attachments such as PDF / Excel; at the same time, the data from each source is timestamped and deduplicated by identity primary key, and structured mapping is performed using the HL7 / FHIR resource model, and LOINC / SNOMED is used. The CT / ICD-10 / RxNorm platform performs terminology normalization and semantic standardization, unifies units and reference intervals, handles outliers and missing values, and forms an event-based time-series data view centered around the patient. To adapt to the data quality requirements of medical scenarios, the data processing employs NLP cleaning and entity extraction enhanced with a medical dictionary for text, establishes metadata indexes and external link references for image and equipment data, and performs table recognition and domain field mapping for PDF / Excel files. Finally, it constructs a fact table and indicator wide table with "patient-visit-event" as the primary key, accumulates a structured feature warehouse that can be used for model training and inference, and provides a standardized REST API for front-end and back-end calls to ensure high consistency and reusability under heterogeneous data sources.

[0020] In one embodiment, the knowledge graph construction module performs the following operations: Based on the knowledge graph construction method, a medical knowledge graph is constructed according to DKD clinical medical knowledge. When constructing the medical knowledge graph, biomarkers, clinical status, drugs and contraindications, examinations and follow-up nodes are used as entities, and indications, contraindications, interactions, levels of evidence and follow-up strategies are used as relationships. Authoritative KDIGO or ADA guidelines are used as graph constraints.

[0021] The working principle and beneficial effects of the above technical solution are as follows: A computable medical knowledge graph is constructed based on DKD clinical medical knowledge. Biomarkers (such as eGFR, UACR, HbA1c, and blood pressure), clinical status, drugs and contraindications, and examination and follow-up nodes are used as the main entities. Relationships such as "indications," "contraindications," "interactions," "levels of evidence," and "follow-up strategies" are encoded, and authoritative guidelines such as KDIGO / ADA are introduced as graph constraints. Guideline content is expressed in a structured manner through event-condition-action (ECA) and executable logical rules, forming a safety barrier for model generation and strategy recommendations. During inference, the model output is constrained by the graph for consistency verification and conflict resolution, triggering evidence supplementation or recommendation downgrading when necessary. The graph is implemented using a hybrid approach of attribute graph storage and triplet indexing, and provides a retrieval interface based on concept paths and constraint queries. It provides semantic support and traceable evidence sources in three types of tasks: diagnosis, efficacy evaluation, and prognosis prediction, enabling the system to establish a robust semantic bridge between multi-source data and clinical knowledge.

[0022] In one embodiment, the local inference module performs the following operations: Based on the domain distillation and model merging strategy, a local inference model is constructed by fine-tuning the model based on a general inference model and a medical domain corpus. The process involves acquiring the reasoning task, determining the task type, and then defining the decision prompt template and JSON output pattern based on the task type. The task types include: diagnosis, efficacy evaluation, and prognosis prediction. Each task type corresponds to a specific decision prompt template and JSON output pattern. Based on the local reasoning model, reasoning tasks are performed using structured feature binaries and medical knowledge graphs to obtain decision results.

[0023] The local inference model also performs the following operations: When the local inference model obtains the decision result, it performs temperature scaling and confidence estimation on the decision result to obtain a confidence score; Output the confidence score and decision result to the corresponding output terminal.

[0024] At the output end, the following operations are performed: Obtain decision results and confidence scores; The decision results are subjected to semantic consistency and numerical verification to obtain verification data.

[0025] The working principle and beneficial effects of the above technical solution are as follows: The module adopts a domain distillation and model merging strategy to integrate a general inference model with a medical domain corpus fine-tuning model into a local inference model, and combines LoRA / Adapter and prompting engineering to achieve task-specific adaptation; in resource-constrained scenarios, 4 / 8-bit quantization and layer clipping optimization are performed, supporting CPU / GPU hybrid acceleration and batch inference; during model inference, a unified API is provided through the Flask backend, with built-in request queues and session management, supporting multi-task routing and policy-level fallback for diagnosis, efficacy evaluation, and prognosis prediction; temperature scaling and confidence estimation are integrated on the model side, and the output includes both structured results and confidence scores; to ensure in-hospital deployment and privacy compliance, the model is loaded and run entirely locally, providing model repository management, hot switching, and version locking mechanisms, combined with caching and feature reuse to reduce latency, ensuring stable and reliable inference in edge and offline environments; Meanwhile, the system employs a Retrieval Enhanced Generation (RAG) framework, jointly indexing structured feature binaries and text evidence bases, and comprehensively utilizing semantic vector retrieval and sparse retrieval for evidence recall. Standardized decision suggestion templates and JSON output patterns are defined for diagnosis, efficacy evaluation, and prognostic prediction, respectively. The reasoning process is executed step-by-step: "evidence retrieval—constraint injection—structured generation—consistency verification—confidence calibration." In the step-by-step decision-making, the diagnostic stage focuses on DKD staging and complication identification; the efficacy stage revolves around risk-benefit assessment of past medication responses and alternative interventions; and the prognostic stage predicts and suggests progression risk and follow-up rhythm. All three share unified evidence anchors and guardrail rules. To suppress inappropriate or inconsistent suggestions, the generation stage employs constraint decoding and template filling, triggering re-retrieval or multi-candidate merging when necessary. Semantic consistency and numerical verification are performed at the output end to ensure clinically usable structured decision results.

[0026] In one embodiment, it also includes: The risk stratification and efficacy assessment module is used to evaluate intervention recommendations for patients to obtain risk stratification and efficacy assessment results, and to automatically intercept or prompt for review any intervention recommendations that do not meet the medical knowledge graph; among them, intervention recommendations are given by physicians or are the result of their decisions.

[0027] The working principle and beneficial effects of the above technical solution are as follows: The system jointly models time-series data views with clinical characteristics, calculates core indicators such as the eGFR decline slope, UACR classification and stability, blood pressure and blood glucose control trajectory, and comorbidity burden, and combines survival analysis and risk regression (Cox / GBM) to predict the progression risk in multiple time windows (such as 2 years / 5 years), and uses calibration curves and confidence intervals to ensure the interpretability and reliability of risk scores. In terms of efficacy evaluation, the system establishes a counterfactual and response scoring framework for major drugs (SGLT2i, MRA, etc.), uses baseline characteristics and previous follow-up responses to quantify potential benefits and adverse event risks, and provides individualized intervention priorities and follow-up rhythm suggestions. The output risk stratification and efficacy evaluation results are returned in a structured template and linked with the contraindications / interaction constraints of the knowledge graph to automatically intercept or prompt review of inappropriate interventions, forming a closed-loop management of "suggestion-implementation-follow-up-reassessment".

[0028] In one embodiment, the system provides interpretability throughout the entire reasoning and decision-making process: it links specific test values, time-series trajectories, and guideline clause sources using evidence anchors; it presents key influencing factors using feature contribution summaries (SHAP-style approximation or attention-weighted visualization); it assists in clinical risk perception with confidence and calibration information; and it records the input, retrieval, constraint, and output processes of each reasoning step with audit logs, achieving traceability and verifiability. At the compliance level, the system implements fine-grained RBAC permissions and multi-role isolation, providing data anonymization, static and transmission encryption, minimum necessary access control, compliance level configuration, and log auditing, supporting in-hospital local storage and offline deployment to reduce out-of-domain risks. The front end integrates AI assistant security prompts and a "?" help entry, displaying compliance statements and source links for key forms and suggestions. The back end automatically intercepts incomplete data, conflicting suggestions, and high-risk operations, and integrates manual review processes to ensure a privacy-preserving, compliant, auditable, and clinically controllable interpretable decision support experience.

[0029] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A decision-making assistance and precision diagnosis system based on interpretable multimodal reasoning, characterized in that, include: The data acquisition module is used to acquire multi-source data on DKD clinical medical knowledge and patients; The feature extraction module is used to process multi-source patient data to obtain a time-series data view of the patient; the time-series data view includes: a fact table and a wide index table; multiple patient time-series data views constitute a structured feature warehouse; The knowledge graph construction module is used to build a medical knowledge graph based on DKD clinical medical knowledge. The local reasoning module is used to build a local reasoning model and perform retrieval reasoning based on the structured feature warehouse and medical knowledge graph to obtain decision results.

2. The auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning according to claim 1, characterized in that, The feature extraction module performs the following operations: Based on the preset data processing method, the patient's multi-source data is processed to obtain a time-series data view; the patient's multi-source data includes: electronic medical record text, laboratory test data, imaging and bedside equipment data, and follow-up and medication records.

3. The auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning according to claim 1, characterized in that, The knowledge graph construction module performs the following operations: Based on the knowledge graph construction method, a medical knowledge graph is constructed according to DKD clinical medical knowledge. When constructing the medical knowledge graph, biomarkers, clinical status, drugs and contraindications, examinations and follow-up nodes are used as entities, and indications, contraindications, interactions, levels of evidence and follow-up strategies are used as relationships. Authoritative KDIGO or ADA guidelines are used as graph constraints.

4. The auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning according to claim 1, characterized in that, The local inference module performs the following operations: Based on the domain distillation and model merging strategy, a local inference model is constructed by fine-tuning the model based on a general inference model and a medical domain corpus. The process involves acquiring the reasoning task, determining the task type, and then defining the decision prompt template and JSON output pattern based on the task type. The task types include: diagnosis, efficacy evaluation, and prognosis prediction. Each task type corresponds to a specific decision prompt template and JSON output pattern. Based on the local reasoning model, reasoning tasks are performed using structured feature binaries and medical knowledge graphs to obtain decision results.

5. The auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning according to claim 4, characterized in that, The local inference model also performs the following operations: When the local inference model obtains the decision result, it performs temperature scaling and confidence estimation on the decision result to obtain a confidence score; Output the confidence score and decision result to the corresponding output terminal.

6. The auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning according to claim 5, characterized in that, At the output end, the following operations are performed: Obtain decision results and confidence scores; The decision results are subjected to semantic consistency and numerical verification to obtain verification data.

7. The auxiliary decision-making and precision diagnosis system based on interpretable multimodal reasoning according to claim 1, characterized in that, Also includes: The risk stratification and efficacy assessment module is used to evaluate intervention recommendations for patients to obtain risk stratification and efficacy assessment results, and to automatically intercept or prompt for review any intervention recommendations that do not meet the medical knowledge graph; among them, intervention recommendations are given by physicians or are the result of their decisions.

Images