Man-machine collaborative inquiry visual interaction system and method based on large medical model

Abstract

The invention discloses a human-machine collaborative inquiry visual interaction system and method based on a medical large model. The system comprises a real-time knowledge base module, a data fusion module, a large model reasoning module and a visual interaction module. The real-time knowledge base module is used for acquiring medical literature data in real time and dynamically updating a basic medical knowledge graph; the data fusion module is used for acquiring multi-source medical data of a target patient and constructing a personalized knowledge graph; the large model reasoning module is used for receiving the personalized knowledge graph to perform preliminary diagnosis reasoning, responding to an intervention instruction to perform adjustment and then performing reasoning again to generate a final diagnosis conclusion; the visual interaction module is used for visually displaying the diagnosis information and receiving an intervention operation instruction input by a user; according to the method, the personalized knowledge graph is constructed, the reasoning logic chain is visually displayed, and real-time intervention of doctors is supported, so that transparent man-machine collaborative diagnosis is realized, and the interpretability of diagnostic reasoning and the accuracy of clinical decision are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and smart healthcare technology, and in particular to a human-computer collaborative consultation visualization and interaction system and method based on a large medical model. Background Technology

[0002] With the explosive growth of medical data and the rapid development of artificial intelligence technology, assisted diagnostic systems have become an important research direction in the field of smart healthcare. Currently, the contradiction between supply and demand of clinical medical resources is becoming increasingly prominent. Traditional clinical decision-making relies too much on doctors' personal experience, is easily influenced by subjective factors, and has low efficiency in diagnosing complex cases, making it difficult to meet the growing demand for quality medical services. Therefore, how to use artificial intelligence technology to assist doctors in making accurate and efficient clinical diagnostic decisions has become a technical problem that urgently needs to be solved in this field.

[0003] In existing assisted diagnostic systems, traditional medical question-answering systems are mainly based on rule engines or simple retrieval and matching mechanisms. Although the logic is clear, their semantic understanding and complex reasoning capabilities are limited, making it difficult to cope with real cases with diverse clinical manifestations. On the other hand, systems based on deep learning models usually lack interpretability in their diagnostic decision-making process, and generally suffer from the "black box" problem, making it difficult for clinicians to understand and trust their output results. In addition, the human-computer interaction mode of existing systems is mostly a simple one-way question-and-answer mode, lacking real-time visualization and interaction mechanisms for the diagnostic reasoning process. Doctors cannot intervene, verify, or guide the reasoning process, making it difficult to achieve effective collaboration between humans and machines.

[0004] In terms of data processing, medical data is characterized by its multi-source heterogeneity, including electronic medical records, medical images, physiological monitoring signals, and medical literature. Existing systems often struggle to deeply integrate and uniformly represent this heterogeneous data, resulting in prominent information silos. This leads to one-sided and outdated diagnostic criteria, affecting the comprehensiveness and timeliness of diagnoses.

[0005] To address the aforementioned issues, some highly integrated solutions already exist in the prior art. For example, Chinese patent application CN120452837A discloses an intelligent decision-making system for clinical diagnostic decision analysis, which includes modules such as data acquisition, knowledge graph, and multimodal analysis, aiming to integrate data and provide decision support. However, this solution and similar technologies still have shortcomings in achieving true "human-machine collaboration": First, the reasoning logic of this solution is not transparent to the user, and the internal reasoning chain cannot be presented in an interactive and visual way; second, human-machine interaction is mostly limited to the confirmation or correction of the final result, failing to support doctors to intervene and provide dynamic guidance in real time during the system's reasoning process; third, the coupling between real-time updated medical knowledge and the immediate diagnostic reasoning process is insufficient, limiting the system's adaptive evolution capability. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the present invention provides a human-computer collaborative consultation visualization interaction system and method based on a large medical model to solve the problems of opaque reasoning process, insufficient fusion of multi-source data, one-way human-computer interaction and lagging knowledge updates in the prior art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a human-computer collaborative consultation visualization and interaction system based on a medical big model, including a real-time knowledge base module, a data fusion module, a big model reasoning module and a visualization and interaction module; It includes a real-time knowledge base module, a data fusion module, a large model reasoning module, and a visualization and interaction module; The real-time knowledge base module is used to acquire medical knowledge data in real time and dynamically update the pre-constructed basic medical knowledge graph. The data fusion module is connected to the real-time knowledge base module and is used to acquire multi-source medical data of the target patient, preprocess the medical data, and construct a personalized knowledge graph of the target patient based on the preprocessed medical data and the basic medical knowledge graph. The large model reasoning module has a pre-trained medical large model built in, which is connected to the data fusion module and the real-time knowledge base module respectively. It is used to perform preliminary diagnostic reasoning on the personalized knowledge graph based on the medical large model to obtain preliminary diagnostic information. The preliminary diagnostic information includes the preliminary diagnostic results, the logical chain of reasoning behind the preliminary diagnostic results, and the confidence level of the preliminary diagnostic results; The visualization and interaction module is connected to the large model inference module. It is used to visualize the preliminary diagnostic information and receive intervention operation instructions input by the user to adjust the personalized knowledge graph. The large model reasoning module is also used to input the intervention operation instructions and the adjusted personalized knowledge graph into the medical large model for re-reasoning to generate the final diagnostic conclusion; the visualization and interaction module is also used to visualize the final diagnostic conclusion.

[0008] In an optional implementation, a feedback optimization module is further included. The feedback optimization module is connected to the large model inference module and the real-time knowledge base module, respectively. It is used to encode the user's intervention operation instructions into reward signals in the reinforcement learning algorithm, and use the reward signals to adjust the parameters of the medical large model to optimize the diagnostic performance of the medical large model.

[0009] In one optional implementation, the real-time knowledge base module includes a knowledge base unit and a knowledge update unit; The knowledge base unit is connected to the data fusion module and is used to store the personalized knowledge graph of the target patient; The knowledge update unit is connected to the knowledge base unit and is used to dynamically update the basic medical knowledge graph and personalized knowledge graph in the knowledge base unit. The knowledge update unit is configured to perform the following steps: Real-time monitoring of designated medical knowledge data publishing channels to obtain updated medical knowledge data; The updated medical knowledge data and the aforementioned basic medical knowledge graph will be subjected to a consistency check. The verified medical knowledge data will be integrated into the basic medical knowledge graph in the knowledge base unit using an incremental update mechanism. The medical knowledge data includes medical literature data, clinical guideline data, and pharmacopoeia data.

[0010] In an optional implementation, the data fusion module includes: The data acquisition unit is used to acquire multi-source medical data of the target patient; The multi-source medical data includes electronic medical record data, physiological monitoring data, medical literature retrieval data, and medical imaging data; A data preprocessing unit, connected to the data acquisition unit, is used to preprocess the medical data to obtain preprocessed medical data. The data preprocessing includes data cleaning, data denoising, data completion, data anonymization, and standardization. The feature extraction unit, connected to the data preprocessing unit, is used to extract features from the preprocessed medical data to obtain medical data features. The medical data features include text features, physiological time-series features, and image features; A standardization processing unit, connected to the feature extraction unit, is used to standardize the features of the medical data. The knowledge graph construction module is connected to the real-time knowledge base module and the standardization processing unit, respectively, and is used to fuse the standardized medical data features with the basic medical knowledge graph to obtain a personalized knowledge graph for the target patient. The personalized knowledge graph includes symptom nodes, examination result nodes, drug nodes, disease nodes, and edges representing the logical relationships between the nodes.

[0011] In one optional implementation, the large model inference module includes: The reasoning execution unit is used to perform preliminary diagnostic reasoning on symptom nodes and examination result nodes in the personalized knowledge graph based on the pre-trained medical big model to obtain a preliminary diagnostic result; it is also used to input the adjusted personalized knowledge graph and the prompt words corresponding to the intervention operation instructions into the medical big model for re-reasoning to generate a final diagnostic conclusion. An interpretability component unit, connected to the inference execution unit, is used to extract the attention weights and saliency attribution of input features within the medical big model to generate an inference logic chain for preliminary diagnostic results. An uncertainty quantification unit, connected to the inference execution unit, is used to quantify and evaluate the uncertainty of the preliminary diagnosis result according to a preset Monte Carlo random inactivation algorithm, so as to generate the confidence level of the preliminary diagnosis result.

[0012] In an optional implementation, the preliminary diagnostic reasoning based on the symptom nodes and examination result nodes in the personalized knowledge graph according to the pre-trained medical large model to obtain a preliminary diagnostic result specifically includes the following steps: A1. Based on the symptom nodes and examination result nodes in the personalized knowledge graph, perform an initial analysis to generate a preliminary diagnostic hypothesis; A2. Based on multiple preset inference branches, perform multi-dimensional parallel verification of the preliminary diagnostic hypothesis to calculate the conditional probability score of the preliminary diagnostic hypothesis under the current inference branch. The reasoning branches include laboratory test analysis, imaging feature analysis, and medical history correlation analysis. A3. Weight and fuse all the conditional probability scores to obtain the overall confidence level of the preliminary diagnostic hypothesis; A4. Calculate the difference between the conditional probability score of each inference branch and the overall confidence level, and compare the difference with a preset threshold to perform conflict detection: If the difference between the conditional probability score of any inference branch and the overall confidence level exceeds the preset threshold, the inference branch is determined to be in conflict, triggering the reassessment mechanism and executing step A5. If the difference between the conditional probability score of all inference branches and the overall confidence level does not exceed the preset threshold, then it is determined that there is no conflict, the preliminary diagnostic hypothesis is maintained, and the preliminary diagnostic hypothesis is used as the preliminary diagnostic result and output. A5. In response to the reassessment mechanism, reduce the confidence of the initial diagnostic hypothesis, extract the feature nodes that cause the conflict, and generate alternative diagnostic hypotheses in combination with the feature nodes; A6. Integrate the preliminary diagnostic hypothesis with the alternative diagnostic hypothesis to obtain and output the preliminary diagnostic results.

[0013] In one optional implementation, the visual interaction module includes: The reasoning path display unit is used to dynamically display the reasoning logic chain in a graphical path. The user intervention interface unit is used to receive intervention operation instructions from the user to adjust the preliminary diagnostic information. The decision comparison unit is used to display the final diagnostic conclusion and the reference diagnostic conclusion stored in the real-time knowledge base module side by side.

[0014] In an optional implementation, the intervention command includes at least one of the following operations: Correction instructions are used to modify the initial diagnostic hypothesis or the confidence level of the initial diagnostic hypothesis; Supplementary operation instructions are used to add new evidence nodes to the reasoning logic chain of the initial diagnostic hypothesis; The delete operation command is used to delete existing nodes and / or connections between nodes in the reasoning logic chain of the preliminary diagnostic hypothesis.

[0015] Secondly, this invention provides a human-computer collaborative consultation visualization interaction method based on a large medical model, including: Acquire multi-source medical data of the target patient, preprocess the medical data, and construct a personalized knowledge graph of the target patient based on the preprocessed medical data and a pre-constructed basic medical knowledge graph; The personalized knowledge graph is used to perform preliminary diagnostic reasoning based on a pre-trained medical big data model to obtain at least one preliminary diagnostic information. The preliminary diagnostic information includes the preliminary diagnostic results, the logical chain of reasoning behind the preliminary diagnostic results, and the confidence level of the preliminary diagnostic results; The preliminary diagnostic information is presented to the user in a visual format, and intervention operation instructions input by the user based on the visual presentation are received. Based on the intervention operation instructions, the personalized knowledge graph is adjusted, and the intervention operation instructions and the adjusted personalized knowledge graph are input into the medical big model to generate a final diagnostic conclusion.

[0016] In an optional implementation, the method further includes: The user's intervention instructions are encoded into a reward signal in a reinforcement learning algorithm, and the parameters of the medical big data model are adjusted using the reward signal to optimize the diagnostic performance of the medical big data model.

[0017] The beneficial effects of the embodiments provided by the present invention include: This invention achieves a closed-loop process for human-machine collaborative diagnosis by constructing a collaborative architecture that includes a real-time knowledge base module, a data fusion module, a large-scale model inference module, and a visualization interaction module. The real-time knowledge base module dynamically updates the basic medical knowledge graph to ensure the timeliness of diagnostic evidence. The data fusion module constructs a personalized knowledge graph based on multi-source medical data to achieve semantic-level fusion of heterogeneous data. The large-scale model inference module dynamically adjusts the preliminary diagnostic information and re-infers it in response to intervention operation commands. Compared with the one-way output system in the prior art, this invention supports doctors to intervene and guide the inference path in real time, which significantly improves the efficiency of human-machine collaboration and enhances doctors' trust in AI diagnostic results.

[0018] This invention achieves transparency and interpretability in the diagnostic reasoning process by integrating a reasoning logic chain and confidence level into preliminary diagnostic information and combining it with the graphical display mechanism of a visualization interaction module. The large-model reasoning module generates a complete reasoning logic chain from symptoms to diagnosis, the uncertainty quantification unit generates a confidence score, and the visualization interaction module dynamically displays the reasoning process through the reasoning path display unit. Compared with the black-box mode of existing technologies that only output diagnostic conclusions, this invention enables doctors to clearly understand the diagnostic basis and reasoning process of the system, facilitating clinical verification and correction, and effectively solving the technical problem of opaque reasoning process.

[0019] This invention, by setting up a feedback optimization module, encodes the user's intervention commands into reward signals for a reinforcement learning algorithm and adjusts the parameters of a large medical model, thus achieving continuous learning and adaptive optimization of the system. The feedback optimization module records the doctor's corrective actions and transforms them into supervisory signals for model optimization. Through an incremental learning mechanism, it dynamically updates the basic medical knowledge graph and model parameters. Compared with existing systems where knowledge updates are lagging, this invention achieves deep coupling between real-time knowledge updates and diagnostic reasoning, enabling the system to continuously evolve with the development of medical knowledge and continuously improve diagnostic accuracy and clinical adaptability. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0021] Figure 1 This document shows a schematic diagram of the human-computer collaborative consultation visualization and interaction system based on a large medical model, as illustrated in an embodiment of this specification. Figure 2 The flowchart of the human-computer collaborative consultation visualization interaction method based on a large medical model in the embodiments of this specification is shown. Detailed Implementation

[0022] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention.

[0023] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be performed in the order they appear herein, or they may be performed in parallel. The operation numbers, such as S1, S2, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel.

[0024] Example 1

[0025] like Figure 1 As shown, this embodiment provides a human-computer collaborative consultation visualization interaction system based on a medical big model, including a real-time knowledge base module 100, a data fusion module 200, a big model reasoning module 300, and a visualization interaction module 400. The real-time knowledge base module 100 is used to acquire medical knowledge data in real time and dynamically update the pre-built basic medical knowledge graph. The data fusion module 200, connected to the real-time knowledge base module 100, is used to acquire multi-source medical data of the target patient, preprocess the medical data, and construct a personalized knowledge graph of the target patient based on the preprocessed medical data and the basic medical knowledge graph. The large-scale model reasoning module 300 is connected to the data fusion module 200 and the real-time knowledge base module 100, respectively. It is used to receive the personalized knowledge graph to perform preliminary diagnostic reasoning in order to obtain at least one preliminary diagnostic information. It is also used to receive the intervention operation instructions and the adjusted personalized knowledge graph sent by the visualization interaction module 400, and input the intervention operation instructions and the adjusted personalized knowledge graph into the medical large-scale model for re-reasoning to generate the final diagnostic result. The preliminary diagnostic information includes the preliminary diagnostic hypothesis, the logical chain of reasoning for the preliminary diagnostic hypothesis, and the confidence level of the preliminary diagnostic hypothesis. The visualization interaction module 400, connected to the large model reasoning module 300, is used to receive and visualize the preliminary diagnostic information and the final diagnostic conclusion; it is also used to receive intervention operation instructions input by the user to adjust the personalized knowledge graph.

[0026] For example, the real-time knowledge base module 100 includes a knowledge base unit 101 and a knowledge update unit 102; Specifically, the knowledge base unit 101 is connected to the data fusion module 200 and is used to store the personalized knowledge graph of the target patient; The knowledge update unit 102 is connected to the knowledge base unit 101 and is used to dynamically update the basic medical knowledge graph and personalized knowledge graph in the knowledge base unit 101. In some embodiments, the knowledge update unit 102 is configured to perform the following steps: Real-time monitoring of designated medical knowledge data publishing channels to obtain updated medical knowledge data; The updated medical knowledge data and the aforementioned basic medical knowledge graph will be subjected to a consistency check. The verified medical knowledge data will be integrated into the basic medical knowledge graph in the knowledge base unit using an incremental update mechanism. In this embodiment, medical knowledge data includes medical literature data, clinical guideline data, and pharmacopoeia data.

[0027] For example, the data fusion module 200 includes a data acquisition unit 201, a data preprocessing unit 202, a feature extraction unit 203, a standardization processing unit 204, and a map construction module 205; Specifically, the data acquisition unit 201 is used to acquire multi-source medical data of the target patient; Multi-source medical data includes electronic medical record data, physiological monitoring data, medical literature retrieval data, and medical imaging data; electronic medical record data includes structured data and unstructured data. Structured medical data includes, but is not limited to, patient basic information, medical history, examination results, and diagnostic records; Unstructured medical data includes, but is not limited to, patient complaints, doctor's notes, and examination reports; In this embodiment, the data acquisition unit 201 is equipped with an electronic medical record interface, a real-time physiological signal acquisition component, and a medical literature retrieval engine; The electronic medical record interface connects to hospital information systems, laboratory information systems, or image archiving systems to obtain electronic medical record data and medical image data. The real-time physiological signal acquisition component is used to acquire physiological parameters including electrocardiogram, blood oxygen and blood pressure in real time, with a sampling frequency of 100-500Hz; The medical literature search engine connects to medical literature databases to retrieve the latest research progress and clinical guidelines in relevant medical literature, ensuring the timeliness of diagnostic criteria.

[0028] It should be noted that the medical literature retrieval data in this embodiment is clearly distinguished from the medical literature data in the aforementioned medical knowledge data: Medical literature retrieval data refers to the real-time precise retrieval of medical literature databases for the specific rare symptoms or complex disease characteristics of the current target patient, extracting reference materials related to the patient's current condition. In other words, it serves the current specific patient and acts as an additional supplementary node in the construction of the personalized knowledge graph to enrich the patient's relevant personalized medical knowledge. Medical literature data refers to the continuous monitoring and capture of the latest authoritative medical knowledge resources such as top academic journals, papers and related literature, which are then integrated into the basic medical knowledge graph through an incremental update mechanism. This type of data serves the system as a whole, is not targeted at any specific patient, and is used to maintain and expand the system's underlying medical knowledge base. Through the aforementioned medical literature retrieval data and medical literature data, this system achieves the separation and synergy of global static knowledge and patient dynamic knowledge: the former provides precise literature support for individual patient conditions, while the latter ensures the timeliness and authority of the system's knowledge base. Together, they enhance the comprehensiveness and accuracy of diagnostic reasoning.

[0029] The data preprocessing unit 202 is connected to the data acquisition unit 201 and is used to preprocess the medical data to obtain preprocessed medical data. Data preprocessing includes data cleaning, data denoising, data completion, and data anonymization. For example, irrelevant symbols and garbled characters are removed from text data, image data is normalized and enhanced, time-series data is interpolated and smoothed, and privacy entity recognition models are used to mask or generalize sensitive information such as names and ID numbers to ensure data quality and security compliance.

[0030] The feature extraction unit 203 is connected to the data preprocessing unit 202 and is used to extract features from the preprocessed medical data to obtain medical data features. The medical data features include text features, physiological time-series features, and image features; Specifically, feature extraction includes the following steps: Natural language processing is performed on unstructured text data to extract structured information, and the structured information is converted into text features through text embedding technology. Physiological monitoring data were filtered and denoised to extract clinically significant temporal features; Image analysis is performed on medical imaging data to extract imaging features and convert them into image features using image coding techniques.

[0031] The standardization processing unit 204, connected to the feature extraction unit 203, is used to standardize the medical data features; the standardization processing includes medical terminology standardization, data format unification, and time series alignment. The medical terminology standardization is based on mapping medical data features to a predefined medical ontology in order to eliminate terminology differences between different data sources; the medical ontology includes SNOMED CT (Systematic Clinical Medical Terminology Set) or UMLS (Unified Medical Language System). The knowledge graph construction module 205 is connected to the knowledge base unit 101 and the standardization processing unit 204 respectively, and is used to fuse the standardized medical data features with the medical knowledge graph to obtain a personalized knowledge graph for the target patient. Specifically, knowledge integration includes the following steps: Using a pre-trained medical language model, named entity recognition is performed on the standardized medical data features to extract core medical entities. Among them, the core medical entities include disease entities, symptom entities, drug entities, and examination and testing item entities; Based on dependency parsing and preset semantic pattern matching rules, the semantic logical relationships between the core medical entities are extracted; The extracted core medical entities are mapped to the standardized concept nodes in the basic medical ontology library to eliminate synonym ambiguity, and the time series data of the target patients are used as attribute values ​​and attached to the corresponding entity nodes. The basic medical ontology is constructed based on the basic medical knowledge graph. The core medical entities and their semantic logical relationships are transformed into triplet data with a head node-edge-tail node structure. The triplet data is imported into a graph database to form a connected network structure, thereby constructing a personalized knowledge graph for the target patient. The graph database is a database system used to store and query graph structure data; in this embodiment, the Neo4j system can be used. Perform circular dependency checks on the personalized knowledge graph to ensure the consistency and integrity of the logical relationships between entity nodes in the personalized knowledge graph.

[0032] The personalized knowledge graph includes symptom nodes, examination result nodes, drug nodes, disease nodes, and edges representing the logical relationships between the nodes.

[0033] For example, the large model reasoning module 300 includes a reasoning execution unit 301, an interpretability component unit 302, and an uncertainty quantification unit 303; Specifically, the reasoning execution unit 301 is used to perform preliminary diagnostic reasoning on symptom nodes and examination result nodes in the personalized knowledge graph based on the pre-trained medical big model, so as to obtain a preliminary diagnostic result; it is also used to input the adjusted personalized knowledge graph and the prompt words corresponding to the intervention operation instructions into the medical big model for re-reasoning, so as to generate a final diagnostic conclusion. In some embodiments, the inference execution unit 301 is configured to perform the following steps: A1. Based on the symptom nodes and examination result nodes in the personalized knowledge graph, perform an initial analysis to generate a preliminary diagnostic hypothesis; A2. Based on multiple preset inference branches, perform multi-dimensional parallel verification of the preliminary diagnostic hypothesis to calculate the conditional probability score of the preliminary diagnostic hypothesis under the current inference branch. The reasoning branches include laboratory test analysis, imaging feature analysis, and medical history correlation analysis. Optionally, step A2 specifically includes the following steps: The laboratory examination analysis branch calculates the conditional probability score of the preliminary diagnostic hypothesis based on the attributes of the laboratory examination nodes in the personalized knowledge graph. The imaging feature analysis branch calculates the conditional probability score of the preliminary diagnostic hypothesis based on the imaging feature node attributes in the personalized knowledge graph. The medical history association analysis branch calculates the conditional probability score of the preliminary diagnostic hypothesis based on the attributes of the medical history feature nodes in the personalized knowledge graph.

[0034] A3. Weight and fuse all the conditional probability scores to obtain the overall confidence level of the preliminary diagnostic hypothesis; A4. Calculate the difference between the conditional probability score of each inference branch and the overall confidence level, and compare the difference with a preset threshold to perform conflict detection: If the difference between the conditional probability score of any inference branch and the overall confidence level exceeds the preset threshold, the inference branch is determined to be in conflict, triggering the reassessment mechanism and executing step A5. If the difference between the conditional probability score of all inference branches and the overall confidence level does not exceed the preset threshold, then it is determined that there is no conflict, the preliminary diagnostic hypothesis is maintained, and the preliminary diagnostic hypothesis is used as the preliminary diagnostic result and output. A5. In response to the reassessment mechanism, reduce the confidence of the initial diagnostic hypothesis, extract the feature nodes that cause the conflict, and generate alternative diagnostic hypotheses in combination with the feature nodes; Optionally, step A5 specifically includes the following steps: In response to the reassessment mechanism, the confidence level of the initial diagnostic hypothesis is reduced; The attention weights of each feature node under each conflict branch are calculated using a multimodal attention mechanism to extract the feature nodes that cause the conflict.

[0035] The feature nodes that caused the conflict are used as new prompts, and combined with the preliminary diagnostic hypothesis, they are input into the medical big model for re-reasoning to generate alternative diagnostic hypotheses.

[0036] The process involves calculating the attention weights of each feature node in each conflicting branch using a multimodal attention mechanism to extract the feature nodes that cause the conflict. Specifically, this includes the following steps: A501. Based on the medical data features corresponding to feature nodes in the personalized knowledge graph, the importance score of each modality feature relative to the current diagnostic state vector is calculated through the dot product attention mechanism. The formula for calculating the importance score is as follows: ; In the formula, The importance score for the i-th mode; Let T be the current diagnostic state vector, and T be the transpose sign; tanh(.) is the hyperbolic tangent activation function. Let be the feature vector of the i-th mode, where , For text features, Physiological temporal characteristics, Image features; The weight matrix is ​​a learnable matrix. For bias terms; Among them, the current diagnostic state vector is the hidden state vector used by the medical big model to generate diagnostic hypotheses during the decoding phase; A502. The importance score is converted into the contribution weight of each medical data feature using the Softmax function; The expression for the Softmax function is: ; In the formula, It is an exponential function; The contribution weight for the i-th mode; A503. The contribution weights are weighted and summed with the corresponding modal feature vectors to obtain the fused multimodal features: The functional expression for the fused multimodal features is as follows: ; In the formula, The fused multimodal features; A504. Feature nodes whose contribution weight is higher than a preset contribution weight threshold in the fused multimodal features are identified as feature nodes that cause conflicts. A505. The feature nodes that cause the conflict are used as new prompts, and combined with the preliminary diagnostic hypothesis, they are input into the medical big model for re-reasoning to generate at least one alternative diagnostic hypothesis.

[0037] A6. Integrate the preliminary diagnostic hypothesis with the alternative diagnostic hypothesis to obtain and output the preliminary diagnostic results.

[0038] Interpretability component unit 302, connected to inference execution unit 301, is used to extract attention weights and saliency attribution of input features within the medical big model to generate inference logic chain for preliminary diagnosis results. Optionally, the interpretability component unit 302 is configured to perform the following steps: In the decoding process of generating diagnostic hypotheses based on input nodes in a large medical model, the attention weight values ​​of the diagnostic output to each input node in the multi-head cross-attention layer of the model are extracted. The input nodes include symptom nodes and examination result nodes; For medical data characteristics, the integral gradient method or saliency mapping algorithm is used to calculate the gradient of the contribution of specific feature dimensions to the final diagnostic result. The attention weights and contribution gradients are normalized and mapped back to the nodes and edges of the personalized knowledge graph. Extract nodes and associated edges whose weights are higher than a preset weight threshold, and assemble them into a logical chain from patient symptoms to diagnosis in logical order. The contribution value is converted into visualization rendering parameters and sent to the visualization interaction module; The visualization rendering parameters include line thickness parameters to characterize the strength of the association and color depth parameters to characterize the level of contribution.

[0039] Uncertainty quantification unit 303, connected to inference execution unit 301, is used to quantify and evaluate the uncertainty of the preliminary diagnosis result according to a preset Monte Carlo random deactivation algorithm, so as to generate the confidence level of the preliminary diagnosis result.

[0040] Specifically, the uncertainty quantification unit 303 is configured to perform the following calculations: Extracting input graph features from personalized knowledge graphs; During the inference phase of the medical big data model, the Dropout mechanism of the medical big data model is enabled, and T random forward propagation calculations are performed on the same input map features. Wherein, the output prediction probability of the t-th forward propagation is: ; In the formula, t is the forward propagation index; T is the total number of samples; and x is the input map feature. For large-scale medical model functions; These are the network weight parameters at the t-th sampling time; The output prediction probability for the t-th forward propagation; Calculate the arithmetic mean of the prediction probabilities from T samplings to obtain the average prediction probability, and then calculate the variance of the average prediction probability to obtain the prediction variance. The formula for calculating the average predicted probability is as follows: ; In the formula, This represents the average predicted probability. The formula for calculating the prediction variance is as follows: ; In the formula, To predict variance; The prediction variance is converted into an uncertainty penalty factor and combined with the average prediction probability to generate the final confidence score displayed to the user.

[0041] For example, the visualization interaction module 400 includes a reasoning path display unit 401, a user intervention interface unit 402, a decision comparison unit 403, and a graph display unit 404; The reasoning path display unit 401 is used to dynamically display the reasoning logic chain in a graphical path. The graphical paths include tree diagram paths and flowchart paths; User intervention interface unit 402 is used to receive intervention operation instructions from the user to adjust the preliminary diagnostic information; The decision comparison unit 403 is used to display the final diagnostic conclusion and the reference diagnostic conclusion stored in the basic medical knowledge graph in parallel. The reference diagnostic conclusions include the doctor's initial proposed plan or the standard clinical pathway plan based on the traditional medical rule base.

[0042] Specifically, the initial treatment plan is prepared by the system from the hospital information system before the system starts deep reasoning of the large model. The initial diagnosis opinion that the doctor has entered based on the preliminary judgment is obtained by the doctor. The standard clinical pathway plan based on the traditional medical rule base is the routine standard clinical pathway guideline plan that the system directly retrieves from the basic medical knowledge graph based on the chief complaint information of the target patient.

[0043] In this embodiment, the final diagnostic conclusion and the reference diagnostic conclusion are displayed side by side through the decision comparison unit. This can intuitively present the difference between the personalized deep reasoning scheme based on the large model and the traditional scheme, highlighting the specificity of the large model scheme for the current patient, such as individual characteristics such as allergy history and gene mutation, and can provide doctors with multi-dimensional decision reference.

[0044] The graph display unit 404 is used to dynamically display the personalized knowledge graph in the form of a node-edge graph.

[0045] In some embodiments, the reasoning path display unit 401 is also used to provide a reasoning focus adjustment function, responding to the user's operation instructions to display detailed information of a specific reasoning step; and to display the credibility of the diagnostic results through color coding or transparency changes to achieve uncertainty visualization.

[0046] In some embodiments, the intervention instruction includes at least one of the following operations: Correction instructions are used to modify the initial diagnostic hypothesis or the confidence level of the initial diagnostic hypothesis; Supplementary operation instructions are used to add new evidence nodes to the reasoning logic chain of the initial diagnostic hypothesis; The delete operation command is used to delete existing nodes and / or connections between nodes in the reasoning logic chain of the preliminary diagnostic hypothesis.

[0047] In some embodiments, the system further includes a feedback optimization module; Specifically, the feedback optimization module is connected to the large model inference module 300 and the real-time knowledge base module 100, respectively. It is used to encode the user's intervention operation instructions into reward signals in the reinforcement learning algorithm, and use the reward signals to adjust the parameters of the medical large model to optimize the diagnostic performance of the medical large model. It is also used to store the diagnostic cases corresponding to the final diagnostic conclusion confirmed by the user to the knowledge base unit 101 to enrich the system's diagnostic experience and optimize subsequent inference performance.

[0048] In this embodiment, the user's intervention command is encoded into a reward signal in a reinforcement learning algorithm, and the parameters of the medical big data model are adjusted using the reward signal to optimize the diagnostic performance of the medical big data model. Specifically, this includes the following steps: B1. Based on the preset reward function mapping rules, the user's intervention operation instructions are encoded into reward signals in the reinforcement learning algorithm; The preset reward function mapping rules include: First mapping rule (positive strong reward): When it is detected that the user directly adopts and confirms the final diagnosis conclusion, or the user only polishes the wording of the diagnosis conclusion without changing the nodes or edges in the personalized knowledge graph, a positive strong reward signal is generated to give the highest reward to the current reasoning path of the medical big model. The second mapping rule (positive weak reward): When it is detected that a user executes a supplementary operation instruction to add a new evidence node to the personalized knowledge graph, and the supplementary operation does not change the main diagnosis result, a positive weak reward signal is generated to give a positive reward to the main reasoning path of the medical big model, while giving a slight penalty to the comprehensiveness of feature extraction in this reasoning. The third mapping rule (negative strong penalty): When it is detected that a user executes a modification operation command or a deletion operation command, and the operation causes the preliminary diagnostic hypothesis to be completely overturned or the core treatment plan to be changed, a negative strong penalty signal is generated to mark the original reasoning path of the medical big model as a negative sample. B2. In response to the triggering of the third mapping rule, the original inference path of the large model is marked as a negative sample, and the final correct inference logic chain generated after user modification is used as a positive sample and stored in the preset replay buffer. B3. Based on the reward signal, the medical large model is aligned and trained using a proximal strategy optimization algorithm.

[0049] In some embodiments, the final diagnostic conclusion includes the final disease diagnosis result, the core atlas chain of evidence supporting the diagnosis result, the recommended treatment plan for the patient, and the patient's prognostic assessment and precautions; The recommended treatment plan for the patient is generated by the large model reasoning module based on the diagnosis results, combined with the patient's individual characteristics in the personalized knowledge graph and the standard treatment plan in the basic medical knowledge graph. By matching and conflict detection between the standard plan and the individual characteristics, a personalized treatment plan suitable for the patient is selected. The patient's prognostic assessment and precautions are generated by the large model inference module based on the diagnostic results and the patient's individual characteristics in the personalized knowledge graph, combined with the historical case statistics in the basic medical knowledge graph, to obtain prognostic assessment information including the probability of treatment effect, risk of adverse reaction, follow-up recommendations, and personalized precautions.

[0050] To further illustrate the technical solution of this invention, several application examples are provided, as follows: In this embodiment, the system is deployed in a high-performance computing server cluster, which is equipped with multiple NVIDIA A100 graphics processors to support real-time computing of large-scale knowledge graph reasoning and deep neural network models.

[0051] Application Example 1: Emergency Diagnosis of Cardiovascular Diseases

[0052] The system was implemented in the cardiology department of a top-tier hospital. Taking virtual emergency patient data as an example, the specific process is as follows: Input the multi-source medical data of virtual patient A into the data fusion module 200, including: 58-year-old male, chief complaint of "chest pain for 3 hours"; electrocardiogram showing ST segment elevation; elevated troponin; 10-year history of hypertension; The data preprocessing unit 202 preprocesses the above-mentioned multi-source data, the feature extraction unit 203 extracts text features, time-series signal features and structured data features, the standardization processing unit 204 performs standardized mapping of medical terms, and the graph construction module 205 integrates the processed features with the basic medical knowledge graph to construct a personalized knowledge graph for the target patient.

[0053] The large model reasoning module 300 performs diagnostic reasoning based on the personalized knowledge graph: Preliminary analysis: Based on the chief complaint of "chest pain" and the electrocardiogram of "ST segment elevation", a preliminary diagnostic hypothesis of "acute myocardial infarction" is derived, with a confidence level of 85%.

[0054] In-depth reasoning: By integrating the elevated troponin levels with a 10-year history of hypertension, the diagnostic hypothesis of "acute ST-segment elevation myocardial infarction" was confirmed, with the confidence level increased to 92%. Treatment plan generation: Based on the disease node associations in the personalized knowledge graph, the recommended treatment plan "emergency PCI (percutaneous coronary intervention)" is generated and the surgical risk assessment results are output. During the reasoning process, the interpretable reasoning component extracts the reasoning logic chain, including the reasoning path from "chest pain" to "acute myocardial infarction" and the contribution of each evidence node.

[0055] The visual interaction module 400 displays the reasoning process to the doctor: The reasoning path display unit 401 displays the complete reasoning logic chain from symptoms to diagnosis in the form of a node-edge graph, and uses color coding to show the confidence level of each diagnostic hypothesis; The doctor reviews the reasoning logic chain in the reasoning path display unit 401 to confirm that the diagnostic basis is sufficient; The user intervention interface unit 402 supplements key information: the patient is allergic to iodine contrast agents (this information was missing during the initial data collection). In response to the intervention command, the large model reasoning module 300 adds "iodine contrast agent allergy" as a new evidence node to the reasoning logic chain and recalculates the reasoning; accordingly, the system adjusts the original recommended plan "emergency PCI" to "drug treatment first, while preparing an allergy treatment plan"; The system generates a detailed diagnostic report that includes the final diagnostic results, diagnostic basis, adjusted treatment plan, and precautions; The graph display unit 404 displays the updated personalized knowledge graph, with the addition of the "Iodine Contrast Agent Allergy" node and its related relationships; The feedback optimization module stores the complete case of this diagnosis in knowledge base unit 101 as a training sample for incremental learning, which is used to optimize the subsequent inference performance of the medical big model.

[0056] Application Example 2: Consultation of Difficult Cases in Oncology

[0057] To illustrate the implementation of this system for complex case consultations at a certain oncology center, using virtual patient B's data as an example, the specific workflow is as follows: Input the multi-source medical data of virtual patient B into the data fusion module 200, including: Pathological data: The pathology report, obtained through the electronic medical record interface, shows a diagnosis of lung adenocarcinoma; Genetic testing data: Genetic testing results were collected, showing EGFR gene mutations; Imaging data: Acquire CT imaging data to show multiple metastatic lesions; Medical literature search data: Retrieved the latest treatment guidelines and research progress for EGFR-mutant lung adenocarcinoma using a medical literature search engine; The data preprocessing unit 202 cleans, standardizes, and extracts features from the above-mentioned multi-source heterogeneous data. The atlas construction module 205 integrates the processed data with the basic medical knowledge graph to construct a personalized knowledge graph that includes tumor pathological classification, gene mutation characteristics, distribution of metastatic lesions, and the latest treatment evidence.

[0058] The large-scale model reasoning module 300 performs multi-dimensional reasoning based on personalized knowledge graphs. Generate a treatment reasoning tree: The reasoning path display unit 401 displays the complete reasoning path from diagnosis to treatment plan selection in the form of a tree diagram, including the selection branches of first-generation, second-generation, and third-generation targeted drugs based on EGFR mutations; Mechanism Visualization: The graph display unit 404 dynamically displays the mechanism of action, target sites, and known resistance mechanisms of targeted drugs in the form of a node-side graph, helping doctors understand the biological basis of drug selection; it is also used to display efficacy data, side effect profiles, expected resistance time, and medical insurance accessibility information of different targeted drugs; The visualization and interaction module 400 supports collaborative decision-making by multidisciplinary expert teams. Oncologists: View the system-recommended first-line treatment options through the reasoning path display unit, and confirm the selection through the user intervention interface unit 402; Radiologist: Assess the necessity and timing of radiation therapy; Pathologists: Confirm the reliability of pathological typing and genetic testing results; The large-scale model inference module 300 responds to intervention commands from multiple experts, dynamically integrates and supplements information, recalculates the global optimization, and generates the final comprehensive treatment decision, including: Personalized treatment plans: Generate comprehensive treatment plans that take into account the patient's age, physical condition, and personal treatment preferences; Patient prognostic assessment: Develop a treatment efficacy assessment plan and an adverse reaction monitoring plan; The feedback optimization module stores the complete case of this multidisciplinary consultation in knowledge base unit 101 to optimize the inference performance of the medical big model in complex tumor cases.

[0059] Example 2

[0060] like Figure 2 As shown, this invention provides a human-computer collaborative consultation visualization interaction method based on a large medical model, including: S1. Obtain multi-source medical data of the target patient, preprocess the medical data, and construct a personalized knowledge graph of the target patient based on the preprocessed medical data and the pre-constructed basic medical knowledge graph. S2. Based on the pre-trained medical big data model, perform preliminary diagnostic reasoning on the personalized knowledge graph to obtain at least one preliminary diagnostic information; The preliminary diagnostic information includes the preliminary diagnostic results, the logical chain of reasoning behind the preliminary diagnostic results, and the confidence level of the preliminary diagnostic results; S3. Display the preliminary diagnostic information to the user in a visual form, and receive the intervention operation instructions input by the user based on the visual display; S4. Based on the intervention operation instructions, adjust the personalized knowledge graph, and input the intervention operation instructions and the adjusted personalized knowledge graph into the medical big model to generate a final diagnostic conclusion.

[0061] In some embodiments, the method further includes: S1. The user's intervention operation instructions are encoded into a reward signal in a reinforcement learning algorithm, and the parameters of the medical big model are adjusted using the reward signal to optimize the diagnostic performance of the medical big model.

[0062] In some embodiments, before generating the final diagnostic conclusion in step S4, a step of doctor confirmation of the final diagnostic conclusion is further included, as follows: The final diagnostic conclusion is presented to the user in a visual format, and confirmation or correction commands are received from the user. If a confirmation command is received from the user, the final diagnostic conclusion will be output as a valid diagnostic result. If a correction instruction is received from the user, the process returns to step S4. The personalized knowledge graph is further adjusted according to the correction instruction, and the adjusted personalized knowledge graph and the prompt words corresponding to the correction instruction are input into the medical big data model for re-reasoning until a confirmation instruction is received from the user, at which point the final valid diagnostic conclusion is output.

[0063] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information all comply with relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals. In the technical solution disclosed herein, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0064] The above description is merely a preferred embodiment of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the present invention is defined by the appended claims rather than the foregoing description, and thus all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention.

Claims

1. A human-computer collaborative consultation visualization and interactive system based on a large medical model, characterized in that, It includes a real-time knowledge base module, a data fusion module, a large model reasoning module, and a visualization and interaction module; The real-time knowledge base module is used to acquire medical knowledge data in real time and dynamically update the pre-constructed basic medical knowledge graph. The data fusion module is connected to the real-time knowledge base module and is used to acquire multi-source medical data of the target patient, preprocess the medical data, and construct a personalized knowledge graph of the target patient based on the preprocessed medical data and the basic medical knowledge graph. The large model reasoning module has a pre-trained medical large model built in, which is connected to the data fusion module and the real-time knowledge base module respectively. It is used to perform preliminary diagnostic reasoning on the personalized knowledge graph based on the medical large model to obtain preliminary diagnostic information. The preliminary diagnostic information includes the preliminary diagnostic results, the logical chain of reasoning behind the preliminary diagnostic results, and the confidence level of the preliminary diagnostic results; The visualization and interaction module is connected to the large model inference module. It is used to visualize the preliminary diagnostic information and receive intervention operation instructions input by the user to adjust the personalized knowledge graph. The large model reasoning module is also used to input the intervention operation instructions and the adjusted personalized knowledge graph into the medical large model for re-reasoning to generate the final diagnostic conclusion; the visualization and interaction module is also used to visualize the final diagnostic conclusion.

2. The system according to claim 1, characterized in that, It also includes a feedback optimization module, which is connected to the large model inference module and the real-time knowledge base module, respectively. The feedback optimization module is used to encode the user's intervention operation instructions into reward signals in the reinforcement learning algorithm, and use the reward signals to adjust the parameters of the medical large model in order to optimize the diagnostic performance of the medical large model.

3. The system according to claim 1, characterized in that, The real-time knowledge base module includes a knowledge base unit and a knowledge update unit; The knowledge base unit is connected to the data fusion module and is used to store the personalized knowledge graph of the target patient; The knowledge update unit is connected to the knowledge base unit and is used to dynamically update the basic medical knowledge graph and personalized knowledge graph in the knowledge base unit. The knowledge update unit is configured to perform the following steps: Real-time monitoring of designated medical knowledge data publishing channels to obtain updated medical knowledge data; The updated medical knowledge data and the aforementioned basic medical knowledge graph will be subjected to a consistency check. The verified medical knowledge data will be integrated into the basic medical knowledge graph in the knowledge base unit using an incremental update mechanism. The medical knowledge data includes medical literature data, clinical guideline data, and pharmacopoeia data.

4. The system according to claim 1, characterized in that, The data fusion module includes: The data acquisition unit is used to acquire multi-source medical data of the target patient; The multi-source medical data includes electronic medical record data, physiological monitoring data, medical literature retrieval data, and medical imaging data; A data preprocessing unit, connected to the data acquisition unit, is used to preprocess the medical data to obtain preprocessed medical data. The data preprocessing includes data cleaning, data denoising, data completion, data anonymization, and standardization. The feature extraction unit, connected to the data preprocessing unit, is used to extract features from the preprocessed medical data to obtain medical data features. The medical data features include text features, physiological time-series features, and image features; A standardization processing unit, connected to the feature extraction unit, is used to standardize the features of the medical data. The knowledge graph construction module is connected to the real-time knowledge base module and the standardization processing unit, respectively, and is used to fuse the standardized medical data features with the basic medical knowledge graph to obtain a personalized knowledge graph for the target patient. The personalized knowledge graph includes symptom nodes, examination result nodes, drug nodes, disease nodes, and edges representing the logical relationships between the nodes.

5. The system according to claim 1, characterized in that, The large model inference module includes: The reasoning execution unit is used to perform preliminary diagnostic reasoning on symptom nodes and examination result nodes in the personalized knowledge graph based on the pre-trained medical big model to obtain a preliminary diagnostic result; it is also used to input the adjusted personalized knowledge graph and the prompt words corresponding to the intervention operation instructions into the medical big model for re-reasoning to generate a final diagnostic conclusion. An interpretability component unit, connected to the inference execution unit, is used to extract the attention weights and saliency attribution of input features within the medical big model to generate an inference logic chain for preliminary diagnostic results. An uncertainty quantification unit, connected to the inference execution unit, is used to quantify and evaluate the uncertainty of the preliminary diagnosis result according to a preset Monte Carlo random inactivation algorithm, so as to generate the confidence level of the preliminary diagnosis result.

6. The system according to claim 5, characterized in that, The process of performing preliminary diagnostic reasoning on symptom nodes and examination result nodes in a personalized knowledge graph based on a pre-trained medical model to obtain a preliminary diagnostic result includes the following steps: A1. Based on the symptom nodes and examination result nodes in the personalized knowledge graph, perform an initial analysis to generate a preliminary diagnostic hypothesis; A2. Based on multiple preset inference branches, perform multi-dimensional parallel verification of the preliminary diagnostic hypothesis to calculate the conditional probability score of the preliminary diagnostic hypothesis under the current inference branch. The reasoning branches include laboratory test analysis, imaging feature analysis, and medical history correlation analysis. A3. Weight and fuse all the conditional probability scores to obtain the overall confidence level of the preliminary diagnostic hypothesis; A4. Calculate the difference between the conditional probability score of each inference branch and the overall confidence level, and compare the difference with a preset threshold to perform conflict detection: If the difference between the conditional probability score of any inference branch and the overall confidence level exceeds the preset threshold, the inference branch is determined to be in conflict, triggering the reassessment mechanism and executing step A5. If the difference between the conditional probability score of all inference branches and the overall confidence level does not exceed the preset threshold, then it is determined that there is no conflict, the preliminary diagnostic hypothesis is maintained, and the preliminary diagnostic hypothesis is used as the preliminary diagnostic result and output. A5. In response to the reassessment mechanism, reduce the confidence of the initial diagnostic hypothesis, extract the feature nodes that cause the conflict, and generate alternative diagnostic hypotheses in combination with the feature nodes; A6. Integrate the preliminary diagnostic hypothesis with the alternative diagnostic hypothesis to obtain and output the preliminary diagnostic results.

7. The system according to claim 1, characterized in that, The visual interaction module includes: The reasoning path display unit is used to dynamically display the reasoning logic chain in a graphical path. The user intervention interface unit is used to receive intervention operation instructions from the user to adjust the preliminary diagnostic information. The decision comparison unit is used to display the final diagnostic conclusion and the reference diagnostic conclusion stored in the real-time knowledge base module side by side.

8. The system according to claim 1, 2, 6 or 7, characterized in that, The intervention operation instruction includes at least one of the following operations: Correction instructions are used to modify the initial diagnostic hypothesis or the confidence level of the initial diagnostic hypothesis; Supplementary operation instructions are used to add new evidence nodes to the reasoning logic chain of the initial diagnostic hypothesis; The delete operation command is used to delete existing nodes and / or connections between nodes in the reasoning logic chain of the preliminary diagnostic hypothesis.

9. A human-computer collaborative consultation visualization and interaction method based on a large medical model, characterized in that, include: Acquire multi-source medical data of the target patient, preprocess the medical data, and construct a personalized knowledge graph of the target patient based on the preprocessed medical data and a pre-constructed basic medical knowledge graph; The personalized knowledge graph is used to perform preliminary diagnostic reasoning based on a pre-trained medical big data model in order to obtain preliminary diagnostic information. The preliminary diagnostic information includes the preliminary diagnostic results, the logical chain of reasoning behind the preliminary diagnostic results, and the confidence level of the preliminary diagnostic results; The preliminary diagnostic information is presented to the user in a visual format, and intervention operation instructions input by the user based on the visual presentation are received. Based on the intervention operation instructions, the personalized knowledge graph is adjusted, and the intervention operation instructions and the adjusted personalized knowledge graph are input into the medical big model to generate a final diagnostic conclusion.

10. The method according to claim 9, characterized in that, The method further includes: The user's intervention instructions are encoded into a reward signal in a reinforcement learning algorithm, and the parameters of the medical big data model are adjusted using the reward signal to optimize the diagnostic performance of the medical big data model.

Images