Intelligent severe illness diagnosis and treatment auxiliary method, system, equipment and medium

Abstract

The invention belongs to the technical field of diagnosis and treatment assistance, and discloses an intelligent severe illness diagnosis and treatment assistance method, system and device and a medium, and the method comprises the steps: obtaining multi-modal patient data, and constructing a knowledge graph based on the multi-modal patient data; inputting the knowledge graph into a diagnosis suggestion model for classification prediction, and outputting the state of the patient and a corresponding diagnosis suggestion; wherein the diagnosis suggestion model is constructed based on a graph neural network and a convolutional neural network; the knowledge graph, the current state and the diagnosis suggestion of the patient are visually displayed, the displayed information is checked and modified through a doctor, and feedback information is generated; optimizing the diagnosis suggestion model based on the feedback information; and executing a diagnosis and treatment auxiliary task based on the optimized diagnosis suggestion model. According to the invention, fusion of multi-modal data is realized, and the system has self-optimization capability through a man-machine feedback mechanism, so that the accuracy and practicability of severe diagnosis and treatment assistance are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of diagnostic and treatment assistance technology, and in particular relates to an intelligent critical care diagnostic and treatment assistance method, system, device and medium. Background Technology

[0002] The Intensive Care Unit (ICU) is one of the most data-intensive and decision-making departments in a hospital. Patient vital sign monitoring data, medical records, medical images, and other modalities of data are continuously generated, forming a massive but heterogeneous information source. Currently, clinical decision-making mainly relies on doctors' comprehensive human judgment, resulting in an extremely heavy workload and a high risk of overlooking crucial details due to information overload.

[0003] Existing intelligent diagnostic and treatment support systems have the following limitations: First, most systems only process single-type data and lack mechanisms for effectively integrating and correlating multimodal information. Second, once deployed, their diagnostic models have fixed performance, making it difficult to adapt to the specific practical modes of different ICUs and emerging medical knowledge.

[0004] Therefore, there is an urgent need for an intelligent assistance method that can integrate multimodal data and achieve performance iteration through interaction with doctors. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent critical care diagnosis and treatment assistance method, system, device, and medium to solve the problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides an intelligent critical care diagnosis and treatment assistance method, comprising: Acquire multimodal patient data, which includes text record data, image data, and real-time sensor monitoring data; Knowledge graphs are constructed based on multimodal patient data; The knowledge graph is input into the diagnostic suggestion model for classification and prediction, and the patient's status and corresponding diagnostic suggestions are output; wherein, the diagnostic suggestion model is built based on graph neural networks and convolutional neural networks; The patient's knowledge graph, current status, and diagnostic suggestions are visualized, and feedback information is generated by doctors who check and modify the displayed information. The diagnostic suggestion model is optimized based on feedback information; Perform diagnostic and treatment assistance tasks based on the optimized diagnostic suggestion model.

[0007] Optionally, the knowledge graph construction process specifically includes: Medical named entity recognition based on large language models in text data, extracting entities and their corresponding relationships; Target detection and semantic segmentation based on CNN convolutional neural network, and entity extraction from image data; Assign a timestamp to each entity and relationship; Using the patient as the central node, the extracted entities are used as nodes, and relationships are used to connect the nodes to construct a knowledge graph.

[0008] Optionally, the training process of the diagnostic suggestion model specifically includes: Offline training: Acquire training data, which includes a knowledge graph constructed based on patient data and corresponding patient status and diagnostic suggestions; The training data is input into the diagnostic suggestion model for classification and prediction, and trained according to the target loss function to obtain the trained diagnostic suggestion model. Online learning: The parameters of the diagnostic suggestion model are fine-tuned and optimized based on real-time feedback from doctors, resulting in an optimized diagnostic suggestion model.

[0009] Optionally, the processing procedure for the diagnostic recommendation model specifically includes: The knowledge graph is input into the graph neural network to learn the relationships between entities and output the semantic state of the illness. Real-time sensor monitoring data is input into a convolutional neural network for classification and prediction, and the physiological time-series status is output. The patient's state vector representation is obtained by fusing the semantic state of the illness and the temporal state of the physiological sequence. Based on the patient state vector representation, a prediction is made and a diagnostic suggestion is output.

[0010] Optionally, the process of obtaining the feedback information specifically includes: The patient’s knowledge graph, current status and diagnostic suggestions are visualized. The doctor’s diagnostic suggestions are used to make judgments and generate feedback information and corresponding reasons. The feedback information includes confirmation, modification or rejection. The doctor's feedback is used as a reinforcement learning signal to fine-tune and optimize the parameters of the diagnostic suggestion model.

[0011] An intelligent critical care diagnosis and treatment assistance system includes: The data acquisition module is used to acquire multimodal patient data, including text records, image data, and real-time sensor monitoring data; and to construct a knowledge graph based on the multimodal patient data. A diagnostic suggestion generation module is used to input the knowledge graph into the diagnostic suggestion model for classification and prediction, and output the patient's status and corresponding diagnostic suggestions; wherein, the diagnostic suggestion model is built based on graph neural networks and convolutional neural networks; The human-computer interaction module is used to visualize the patient's knowledge graph, current status, and diagnostic suggestions, and to generate feedback information by allowing doctors to check and modify the displayed information. The feedback optimization module is used to optimize the diagnostic suggestion model based on feedback information; and to perform diagnostic and treatment assistance tasks based on the optimized diagnostic suggestion model.

[0012] An electronic device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the aforementioned intelligent critical care diagnosis and treatment assistance method.

[0013] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned intelligent critical care diagnosis and treatment assistance method.

[0014] The technical effects of this invention are as follows: This invention constructs a spatiotemporal dynamic knowledge graph, integrating data from different sources and formats into a unified, structured representation framework, providing a data foundation for comprehensive analysis of patient conditions. Simultaneously, by assigning timestamps to all entities and relationships, the knowledge graph can depict the evolution of the disease. Secondly, by utilizing a hybrid model of GNN (Graph Neural Network) and CNN (Convolutional Neural Network) for diagnostic suggestion inference, the cognitive load on doctors can be effectively reduced, shortening diagnosis time. Furthermore, this invention introduces an online learning mechanism based on human-computer interaction, allowing the model to continuously optimize through feedback, thereby improving accuracy. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating the implementation of this invention. Figure 2 This is a schematic diagram of the system structure in an embodiment of the present invention. Detailed Implementation

[0017] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0018] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0019] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.

[0020] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] like Figure 1 - Figure 2 As shown in the figure, this embodiment provides an intelligent critical care diagnosis and treatment assistance method, including: acquiring multimodal patient data, which includes text record data, image data, and real-time sensor monitoring data; constructing a knowledge graph based on the multimodal patient data; inputting the knowledge graph into a diagnostic suggestion model for classification and prediction, and outputting the patient's status and corresponding diagnostic suggestions; wherein, the diagnostic suggestion model is constructed based on graph neural networks and convolutional neural networks; visualizing the patient's knowledge graph, current status, and diagnostic suggestions, and having a doctor verify and modify the displayed information to generate feedback information; optimizing the diagnostic suggestion model based on the feedback information; and performing diagnosis and treatment assistance tasks based on the optimized diagnostic suggestion model.

[0023] Existing intelligent diagnostic and treatment assistance systems only process single-type data and lack mechanisms for effectively integrating and associating multimodal information. Furthermore, once deployed, their diagnostic models have fixed performance, cannot be iterated upon, and gradually become outdated. To address these issues, this embodiment provides an intelligent critical care diagnostic and treatment assistance method, including: acquiring multimodal data such as patient text records, image data, and real-time sensor monitoring; extracting medical entities and relationships from this data based on a large language model and a CNN model to construct a patient-centered dynamic knowledge graph with spatiotemporal information; then using a diagnostic suggestion model built on graph neural networks and convolutional neural networks to fuse and analyze the knowledge graph and real-time data, outputting patient status and diagnostic suggestions; after the diagnostic suggestions are generated, they are presented to doctors for verification through a visual interface, and the diagnostic suggestion model is continuously optimized using the doctor's feedback. This embodiment achieves deep fusion of multimodal data and, through a human-machine collaborative feedback mechanism, enables the system to have continuous self-optimization capabilities, significantly improving the accuracy and practicality of critical care diagnostic and treatment assistance.

[0024] The specific implementation process of this embodiment includes: (1) Data acquisition: Acquire various data of the target patient, including text record data (medical records, nursing records), image data (X-ray, CT, etc.) and real-time sensor monitoring data (ECG, blood pressure waveform, etc.). The data is preprocessed. For text records such as medical records, de-identification and medical terminology standardization are performed. For image data, format parsing and normalization are performed. For monitoring data, signal filtering and outlier correction are performed, and the data is converted into a regular time-series data stream.

[0025] (2) Knowledge graph construction: Entity and Relation Extraction: Using a pre-trained medical big language model, natural language processing is performed on text records to perform named entity recognition and relation extraction, identifying entities and relations such as disease, symptoms, drugs, and surgery (medication, cause, contraindications, etc.).

[0026] Meanwhile, a pre-trained CNN model is used to perform target detection and semantic segmentation on the image data, identifying radiological entities such as pulmonary effusion, pleural effusion, and fractures.

[0027] For each extracted entity and relation, mark the time point when it occurred or became effective. For example, the timestamp of the drug entity is the time of medication, and the timestamp of the hypotension event entity includes the start and end times.

[0028] Constructing a knowledge graph: Using the current patient as the central node, all extracted entities are used as other nodes, and the extracted relationships are used as edges to connect the nodes, thereby constructing a dynamic knowledge graph with temporal attributes.

[0029] This embodiment constructs a spatiotemporal dynamic knowledge graph, integrating data from different sources and formats into a unified, structured representation framework, providing a data foundation for comprehensive analysis of patient conditions. Furthermore, by assigning timestamps to all entities and relationships, the knowledge graph can depict the evolution of the disease.

[0030] (3) Generation of diagnostic suggestions: The model selection includes Graph Neural Network (GNN) and Convolutional Neural Network (CNN). GNN and CNN serve as two branches to process different data, and the outputs of the two branches are finally fused to provide diagnostic suggestions.

[0031] Before the initial deployment of the model, offline training is required. A large amount of historical case data (including knowledge graphs and final diagnostic labels) is used to conduct supervised offline training on the diagnostic suggestion model to obtain a basic model.

[0032] Model input includes static and dynamic information. Static information is the current subgraph of the patient's individualized knowledge graph. Dynamic information is real-time monitoring data, namely the patient's vital signs data.

[0033] Processing steps: Static information is input into a GNN (Geometric Logic Array), which processes the structured knowledge graph and learns the complex relationships between entities. The GNN aggregates node and neighborhood information through message passing, learning a fixed-dimensional semantic state vector that integrates all structured information from the knowledge graph. Real-time monitoring time-series data is input into a CNN (Neural Network Array) model. This network captures short-term and long-term dependencies in the data and outputs a vector representing the current physiological time-series state. The outputs of the two branches are fused to form a comprehensive patient state vector representation. This vector is then input into a Softmax classifier, which outputs the probability distribution of all possible diagnoses, generating diagnostic suggestions.

[0034] By using a hybrid model of GNN (Graph Neural Network) and CNN (Convolutional Neural Network) to make diagnostic suggestions, the cognitive load on doctors can be effectively reduced and the diagnosis time can be shortened. (4) Doctor's verification: The patient's knowledge graph and diagnostic suggestions output by the model are presented side by side on a single interface. Doctors review the diagnostic suggestions based on their clinical experience and provide feedback through the interface: confirm, modify, or reject. They can also input the reason for modification while providing feedback.

[0035] Feedback learning and model optimization: During model application, physician feedback serves as reinforcement learning signals, used to fine-tune the diagnostic model to adapt it to physician preferences or specific situations. Through continuous optimization, the diagnostic suggestion model's accuracy and reliability improve when processing subsequent patient data, thus enabling it to consistently and efficiently perform intelligent critical care assistance tasks.

[0036] Alternatively, this embodiment also provides an intelligent critical care diagnosis and treatment assistance system, including: The data acquisition module is used to acquire multimodal patient data, including text records, image data, and real-time sensor monitoring data; and to construct a knowledge graph based on the multimodal patient data. A diagnostic suggestion generation module is used to input the knowledge graph into the diagnostic suggestion model for classification and prediction, and output the patient's status and corresponding diagnostic suggestions; wherein, the diagnostic suggestion model is built based on graph neural networks and convolutional neural networks; The human-computer interaction module is used to visualize the patient's knowledge graph, current status, and diagnostic suggestions, and to generate feedback information by allowing doctors to check and modify the displayed information. The feedback optimization module is used to optimize the diagnostic suggestion model based on feedback information; and to perform diagnostic and treatment assistance tasks based on the optimized diagnostic suggestion model.

[0037] In practice, this embodiment also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the intelligent critical care diagnosis and treatment assistance method described above.

[0038] In practice, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned intelligent critical care diagnosis and treatment assistance method.

[0039] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An intelligent critical care diagnosis and treatment assistance method, characterized in that, include: Acquire multimodal patient data, which includes text record data, image data, and real-time sensor monitoring data; Knowledge graphs are constructed based on multimodal patient data; The knowledge graph is input into the diagnostic suggestion model for classification and prediction, and the patient's status and corresponding diagnostic suggestions are output; wherein, the diagnostic suggestion model is built based on graph neural networks and convolutional neural networks; The patient's knowledge graph, current status, and diagnostic suggestions are visualized, and feedback information is generated by doctors who check and modify the displayed information. The diagnostic suggestion model is optimized based on feedback information; Perform diagnostic and treatment assistance tasks based on the optimized diagnostic suggestion model.

2. The method according to claim 1, characterized in that, The construction process of the knowledge graph specifically includes: Medical named entity recognition based on large language models in text data, extracting entities and their corresponding relationships; Target detection and semantic segmentation based on CNN convolutional neural network, and entity extraction from image data; Assign a timestamp to each entity and relationship; Using the patient as the central node, the extracted entities are used as nodes, and relationships are used to connect the nodes to construct a knowledge graph.

3. The method according to claim 1, characterized in that, The training process of the diagnostic suggestion model specifically includes: Offline training: Acquire training data, which includes a knowledge graph constructed based on patient data and corresponding patient status and diagnostic suggestions; The training data is input into the diagnostic suggestion model for classification and prediction, and trained according to the target loss function to obtain the trained diagnostic suggestion model. Online learning: The parameters of the diagnostic suggestion model are fine-tuned and optimized based on real-time feedback from doctors, resulting in an optimized diagnostic suggestion model.

4. The method according to claim 1, characterized in that, The processing procedure of the diagnostic recommendation model specifically includes: The knowledge graph is input into the graph neural network to learn the relationships between entities and output the semantic state of the illness. Real-time sensor monitoring data is input into a convolutional neural network for classification and prediction, and the physiological time-series status is output. The patient's state vector representation is obtained by fusing the semantic state of the illness and the temporal state of the physiological sequence. Based on the patient state vector representation, a prediction is made and a diagnostic suggestion is output.

5. The method according to claim 1, characterized in that, The process of obtaining the feedback information specifically includes: The patient’s knowledge graph, current status and diagnostic suggestions are visualized. The doctor’s diagnostic suggestions are used to make judgments and generate feedback information and corresponding reasons. The feedback information includes confirmation, modification or rejection. The doctor's feedback is used as a reinforcement learning signal to fine-tune and optimize the parameters of the diagnostic suggestion model.

6. An intelligent critical care diagnosis and treatment auxiliary system, characterized in that, include: The data acquisition module is used to acquire multimodal patient data, which includes text record data, image data, and real-time sensor monitoring data. Knowledge graphs are constructed based on multimodal patient data; A diagnostic suggestion generation module is used to input the knowledge graph into the diagnostic suggestion model for classification and prediction, and output the patient's status and corresponding diagnostic suggestions; wherein, the diagnostic suggestion model is built based on graph neural networks and convolutional neural networks; The human-computer interaction module is used to visualize the patient's knowledge graph, current status, and diagnostic suggestions, and to generate feedback information by allowing doctors to check and modify the displayed information. The feedback optimization module is used to optimize the diagnostic suggestion model based on feedback information; and to perform diagnostic and treatment assistance tasks based on the optimized diagnostic suggestion model.

7. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program and the processor runs the computer program to enable the electronic device to perform an intelligent critical care diagnosis and treatment assistance method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements an intelligent critical care diagnosis and treatment assistance method as described in any one of claims 1-5.

Images