Assistant decision-making system based on multi-modal data

Abstract

The invention discloses a multi-modal data-based auxiliary decision-making system suitable for a perioperative period and an intensive care period, which can provide multi-dimensional reference contents such as real-time monitoring, attribution analysis and decision-making recommendation of physiological parameters for a user, can provide auxiliary information for decision making of medical personnel, and can improve decision-making efficiency. Meanwhile, the system greatly reduces the requirements for medical knowledge and personal professional literacy of medical personnel, not only reduces the requirements for personnel ability, but also weakens the dependence on human experience, and can realize dynamic adjustment of physiological parameter calibration data, so that compared with the traditional technology, the use reliability is improved; therefore, the method is very suitable for large-scale application and popularization.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical data processing, and particularly relates to an auxiliary decision-making system based on multi-modal data. Background Art

[0002] Currently, the main functions of the information systems commonly used in hospitals are automatic data collection and integration as well as process management, that is, recording key clinical parameters such as drug use, vital signs, monitoring results, etc., covering the entire process from surgical application, confirmation, anesthesia consultation, surgical arrangement, preoperative medical orders, surgical medical orders, postoperative instrument inventory, postoperative accounting, postoperative medical orders, and surgical medical record writing; however, the hospital information support systems on the market have the following deficiencies:

[0003] (1) It only has the function of data collection and recording, and cannot provide individual crisis event assessment and auxiliary decision-making functions. The assessment and decision-making of crisis events mainly rely on the medical knowledge and personal professional qualities of medical staff, which require high personnel capabilities and are highly dependent on manual experience; (3) Although there are corresponding clinical decision support systems on the market that can be used in conjunction with information systems, however, the existing artificial intelligence-based clinical decision support systems are more suitable for medical consultations and medical record management, and are not suitable for the perioperative period and the intensive care period, lacking tools for monitoring the physiological state of patients during the perioperative period and the intensive care period and recommending auxiliary decision-making plans; (3) The existing artificial intelligence-based perioperative risk assessment and clinical decision-making technologies cannot be adjusted according to the real-time situation of patients, that is, the risk threshold data tables of various physiological parameters are fixed values, without considering the dynamic changes in the physiological state of patients, and their reliability is poor; thus, based on the above deficiencies, how to provide an auxiliary decision-making system based on multi-modal data that does not rely on manual experience, is suitable for the perioperative period and the intensive care period, and has high reliability has become an urgent problem to be solved. Summary of the Invention

[0004] The purpose of the present invention is to provide an auxiliary decision-making system based on multi-modal data to solve the problems of high dependence on manual experience, inapplicability to the perioperative period and the intensive care period, and poor reliability existing in the prior art.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] In the first aspect, an auxiliary decision-making system based on multi-modal data is provided, including:

[0007] A data processing subsystem for obtaining human-machine medical interaction data, real-time medical event information, and medical assistance information within a first time period during a monitoring period, where the medical assistance information includes patient historical medical information sent by a medical information management system, patient physiological parameter information sent by a medical monitoring device, and medical data sent by a medical treatment device, and the first time period is a time period with a preset duration before the current moment, and the monitoring period includes the perioperative period and / or the intensive care period;

[0008] A status evaluation subsystem, communicatively connected to the data processing subsystem, for generating a calibration value interval of each physiological parameter within a second time period based on the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, so as to obtain the first status information of each physiological parameter of the patient at the current moment according to the calibration value interval of each physiological parameter and the patient physiological parameter information, and for predicting the second status information of each physiological parameter of the patient within the second time period based on the patient physiological parameter information, and visually displaying the first status information and the second status information of each physiological parameter, where the second time period is a time period with a preset duration after the current moment;

[0009] An auxiliary decision-making subsystem for obtaining attribution analysis support data and decision analysis support data, where the attribution analysis support data includes a first medical knowledge graph and first medical knowledge vector data, and the decision analysis support data includes a second medical knowledge graph and second medical knowledge vector data;

[0010] The auxiliary decision-making subsystem, communicatively connected to the data processing subsystem and the status evaluation subsystem, for performing data attribution analysis based on the attribution analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the first status information of each physiological parameter, and the second status information of each physiological parameter, to obtain a data attribution analysis result at the current moment, and visually displaying the data attribution analysis result, where the data attribution analysis result is the medical knowledge in the first medical knowledge graph and the first medical knowledge vector data that affects each physiological parameter;

[0011] The auxiliary decision-making subsystem is further configured to generate auxiliary decision-making advice information for the patient at the current moment based on the decision analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the data attribution analysis result, the first status information of each physiological parameter, and the second status information of each physiological parameter, and visually display the auxiliary decision-making advice information;

[0012] Wherein, when the current moment is the last moment within the second time period, the data processing subsystem updates the current moment to the last moment, and re-obtains the human-machine medical interaction data, real-time medical event information, and medical assistance information within the first time period until the end of the monitoring period.

[0013] Based on the above-disclosed content, the auxiliary decision-making system provided by the present invention includes a data processing subsystem, a state evaluation subsystem, and an auxiliary decision-making subsystem. Among them, the data processing subsystem is used to obtain the human-machine medical interaction data, real-time medical event information, and medical assistance information within a preset time period before the current moment during the perioperative period and / or intensive care period. And the medical assistance information includes the patient's historical medical information sent by the medical information management system, the patient's physiological parameter information sent by the medical monitoring device, and the medical data sent by the medical treatment device. In this way, it is equivalent to obtaining multi-modal data of the patient during the perioperative period and / or intensive care period. Then, the state evaluation subsystem obtains the calibration value intervals of each physiological parameter within the second time period (i.e., the time period of the preset time length after the current moment) according to the data obtained by the aforementioned data processing subsystem within the first time period. Then, according to the obtained calibration value intervals of each physiological parameter, the first state information of each physiological parameter of the patient (i.e., whether the physiological parameter is abnormal) is determined, and the second state information of each physiological parameter within the second time period is predicted and visually displayed. Based on this, the real-time monitoring and visual display of the patient's physiological state during the perioperative period and / or intensive care period can be realized.

[0014] After the real-time monitoring and display of the physiological parameters are completed, the auxiliary decision-making subsystem can obtain the medical knowledge graph and medical knowledge vector data for attribution analysis and decision-making analysis. Then, based on the medical knowledge graph and medical knowledge vector data for attribution analysis, and combined with the data output by the aforementioned data processing subsystem and state evaluation subsystem, data attribution analysis is carried out, so as to determine the medical knowledge that affects each physiological parameter from the medical knowledge graph and medical knowledge vector data for attribution analysis, so as to obtain the data attribution analysis result. Then, according to the data attribution analysis result, the knowledge graph and medical knowledge vector for decision-making analysis, and the data output by the aforementioned data processing subsystem and state evaluation subsystem, the auxiliary decision-making advice information of the patient at the current moment is generated. In this way, the auxiliary decision-making advice information at the current moment can be visually displayed, so as to assist medical staff in making decisions.

[0015] Finally, if the current moment is the last moment in the second time period, data needs to be updated. That is, taking the aforementioned last moment as the current moment, re-obtain the human-machine medical interaction data, real-time medical event information, and medical assistance information within the preset duration before it, and re-execute the above process until the end of the monitoring period. In this way, the dynamic adjustment of the calibration value intervals corresponding to each physiological parameter during the monitoring period and the dynamic generation of auxiliary decision-making suggestions can be achieved.

[0016] Through the above design, the present invention provides an auxiliary decision-making system applicable to the perioperative period and the intensive care period, which can provide users with multi-dimensional reference contents such as real-time monitoring, attribution analysis, and decision recommendation of physiological parameters, can provide auxiliary information for the decision-making of medical staff, and can improve the decision-making efficiency. At the same time, this system greatly reduces the requirements for the medical knowledge and personal professional qualities of medical staff, not only reduces the requirements for personnel capabilities, but also weakens the dependence on human experience. Moreover, the present invention can achieve the dynamic adjustment of the calibration data of physiological parameters. Therefore, compared with the traditional technology, the reliability of use is improved. Thus, the present invention is very suitable for large-scale application and promotion.

[0017] In a possible design, the human-machine medical interaction data includes physiological parameter constraint conditions.

[0018] The state evaluation subsystem includes: an optimization goal generation device and a parameter current state management and crisis warning device.

[0019] The optimization goal generation device is used to generate the calibration value intervals of each physiological parameter in the second time period according to the physiological parameter constraint conditions, the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, send the calibration value intervals of each physiological parameter to the parameter current state management and crisis warning device, and visually display the physiological parameter constraint conditions.

[0020] The parameter current state management and crisis warning device is used to generate the first state information of each physiological parameter of the patient at the current moment according to the calibration value intervals of each physiological parameter and the patient's physiological parameter information, and predict the second state information of each physiological parameter of the patient in the second time period according to the patient's physiological parameter information, the human-machine medical interaction data, and the real-time medical event information, and visually display the first state information and the second state information of each physiological parameter.

[0021] In a possible design, the optimization goal generation device includes: a first data call module, an optimization goal generation module, a first data output module, and a first display module.

[0022] The first data calling module, communicatively connected to the data processing subsystem, is configured to retrieve the human-machine medical interaction data, real-time medical event information, and medical assistance information in the data processing subsystem, and send the retrieved human-machine medical interaction data, real-time medical event information, and medical assistance information to the optimization target generation module, and send the physiological parameter constraint conditions in the human-machine medical interaction data to the first display module;

[0023] The optimization target generation module is configured to generate a calibration value range of each physiological parameter within a second time period according to the physiological parameter constraint conditions, the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, and send the calibration value range of each physiological parameter to the first data output module;

[0024] The first data output module is configured to receive the calibration value range of each physiological parameter, and send the calibration value range of each physiological parameter to the first display module and the parameter current state management and crisis warning device, and send it to the data processing subsystem for storage;

[0025] The first display module is configured to visually display the calibration value range of each physiological parameter and the physiological parameter constraint conditions.

[0026] In a possible design, the physiological parameter constraint conditions include a number of constraint conditions, and the optimization target generation module includes a first calculation and analysis unit, wherein the first calculation and analysis unit includes a number of parameter calculation units, and each parameter calculation unit corresponds to a constraint condition respectively;

[0027] Among them, each parameter calculation unit is configured to determine the corresponding physiological parameter from the patient physiological parameter information, and determine the corresponding constraint condition from the physiological parameter constraint conditions, so as to generate a calibration value range of the corresponding physiological parameter within a second time period based on the human-machine medical interaction data, the real-time medical event information, the patient historical medical information, the medical data, and the corresponding physiological parameter and the corresponding constraint condition, and adopt a first machine learning model;

[0028] The first calculation and analysis unit is further configured to send the calibration value range of the physiological parameter corresponding to each parameter calculation unit within a second time period to the first display module and the first data output module.

[0029] In a possible design, the optimization target generation module further includes: a first data calling unit and a first optimization unit;

[0030] The first data calling unit is configured to obtain the historical calibration value intervals of each physiological parameter within the first time period, the first state information of each physiological parameter at the current moment, at a preset interval, and retrieve the real-time medical event information, medical assistance information, and human-machine medical interaction data in the data processing subsystem, and send the historical calibration value intervals of each physiological parameter, the first state information of each physiological parameter, the real-time medical event information, the medical assistance information, and the human-machine medical interaction data to the first optimization unit;

[0031] The first optimization unit is configured to optimize the first machine learning models adopted by each parameter calculation unit according to the historical calibration value intervals of each physiological parameter, the first state information of each physiological parameter, the real-time medical event information, the medical assistance information, and the human-machine medical interaction data, to obtain each optimized first machine learning model, so as to use each optimized first machine learning model to generate the calibration value intervals corresponding to each physiological parameter.

[0032] In a possible design, the parameter current state management and crisis warning device includes: a second data calling module, a current state analysis module, a crisis warning module, a second data output module, and a second display module;

[0033] The second data calling module is communicatively connected to the data processing subsystem and the optimization target generating device, and is configured to retrieve the human-machine medical interaction data, real-time medical event information, and medical assistance information obtained by the data processing subsystem, and the calibration value intervals of each physiological parameter generated by the optimization target generating device at the current moment, and send the retrieved calibration value intervals of each physiological parameter and the patient physiological parameter information in the medical assistance information to the current state analysis module;

[0034] The second data calling module is further configured to send the human-machine medical interaction data, real-time medical event information, and medical assistance information to the crisis warning module;

[0035] The current state analysis module is configured to generate the first state information of each physiological parameter of the patient at the current moment according to the calibration value intervals of each physiological parameter and the patient physiological parameter information, and send the first state information of each physiological parameter to the second data output module;

[0036] The crisis warning module is configured to predict the second state information of each physiological parameter of the patient within the second time period according to the patient physiological parameter information in the medical assistance information, the human-machine medical interaction data, and the real-time medical event information, and send the second state information of each physiological parameter to the second data output module;

[0037] The second data output module is configured to receive the first status information and the second status information of each physiological parameter, and send the first status information and the second status information of each physiological parameter to the second display module;

[0038] The second display module is configured to visually display the first status information and the second status information of each physiological parameter. Among them, the first status information and the second status information of any physiological parameter include a crisis state or a normal state, and the second display module is configured to give an alarm prompt for the physiological parameter in the crisis state.

[0039] In a possible design, the crisis warning module includes: a second calculation and analysis unit, a second data call unit, and a second optimization unit;

[0040] The second calculation and analysis unit is configured to predict the second status information of each physiological parameter of the patient within a second time period according to the patient's physiological parameter information, the human-machine medical interaction data, and the real-time medical event information, and by using a second machine learning model. Among them, the human-machine medical interaction data includes historical auxiliary decision-making suggestions;

[0041] The second data call unit is configured to retrieve the human-machine interaction medical data, the real-time medical event information, and the medical assistance information in the data processing subsystem at a preset interval, and obtain the historical second status information of each physiological parameter within a first time period, and send the human-machine interaction medical data, the real-time medical event information, the medical assistance information, and the historical second status information of each physiological parameter to the second optimization unit;

[0042] The second optimization unit is configured to optimize the second machine learning model in the second calculation and analysis unit according to the human-machine interaction medical data, the real-time medical event information, the medical assistance information, and the historical second status information of each physiological parameter, so as to obtain an optimized second machine learning model, and use the optimized second machine learning model to generate the second status information corresponding to each physiological parameter.

[0043] In a possible design, the auxiliary decision-making subsystem includes: an attribution analysis device, a decision recommendation device, and a technical solution database;

[0044] Among them, the technical solution database stores the attribution analysis support data and the decision analysis support data;

[0045] An attribution analysis device is used to perform data attribution analysis based on the attribution analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the first state information of each physiological parameter, and the second state information of each physiological parameter, obtain the data attribution analysis result at the current moment, send the data attribution analysis result to the decision recommendation device, and visually display the data attribution analysis result;

[0046] A decision recommendation device is used to generate auxiliary decision-making advice information for the patient at the current moment based on the decision analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the data attribution analysis result, the first state information of each physiological parameter, and the second state information of each physiological parameter, and visually display the auxiliary decision-making advice information.

[0047] In a possible design, the attribution analysis device includes a third data calling module, an attribution analysis module, a third data output module, and a third display module;

[0048] The third data calling module is used to obtain the human-machine medical interaction data, the real-time medical event information, and the medical assistance information in the data processing subsystem, the first state information and the second state information of each physiological parameter in the state evaluation subsystem, and the attribution analysis support data in the technical solution database, and send the obtained human-machine medical interaction data, real-time medical event information, medical assistance information, the first state information and the second state information of each physiological parameter, and the attribution analysis support data to the attribution analysis module;

[0049] The attribution analysis module is used to perform data attribution analysis based on the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the first state information and the second state information of each physiological parameter, and the attribution analysis support data, and adopt an artificial intelligence algorithm to perform data attribution analysis to obtain a number of initial data attribution analysis results, where each initial data attribution analysis result includes a classification probability value corresponding to medical knowledge;

[0050] The attribution analysis module is further used to perform filtering processing on a number of initial data attribution analysis results according to the classification probability value, and after the filtering processing, obtain the data attribution analysis result and send it to the third data output module;

[0051] The third data output module is used to receive the data attribution analysis result and send the data attribution analysis result to the third display module;

[0052] The third display module is used to visually display the data attribution analysis result.

[0053] In a possible design, the data processing subsystem includes: a first data receiving device, a second data receiving device, a third data receiving device, a data storage device, and a data output device;

[0054] The first data receiving device is communicatively connected to the medical monitoring device, and is configured to obtain in real time the physiological parameter information of the patient sent by the medical monitoring device, and send the physiological parameter information of the patient to the data storage device;

[0055] The second data receiving device is communicatively connected to the medical treatment device, and is configured to obtain in real time the medical data sent by the medical treatment device, and is configured to obtain in real time the human-machine medical interaction data and real-time medical event information, and send the obtained medical data, human-machine medical interaction data, and real-time medical event information in real time to the data storage device;

[0056] The third data receiving device is communicatively connected to the medical information management system, and is configured to obtain in real time the historical medical information of the patient sent by the medical information management system, and send the obtained historical medical information of the patient in real time to the data storage device;

[0057] The data storage device is configured to store the received physiological parameter information of the patient, medical data, human-machine medical interaction data, real-time medical event information, and historical medical information of the patient;

[0058] The data output device is configured to read the physiological parameter information of the patient, medical data, human-machine medical interaction data, real-time medical event information, and historical medical information of the patient stored in the data storage device and within a first time period, and send the read physiological parameter information of the patient, medical data, human-machine medical interaction data, real-time medical event information, and historical medical information of the patient to the status evaluation subsystem and the auxiliary decision-making subsystem;

[0059] Wherein, the data processing subsystem further includes: a data preprocessing device, and the obtained medical data, human-machine medical interaction data, real-time medical event information, and historical medical information of the patient are text data;

[0060] The data preprocessing device is configured to perform missing value filling and outlier data clearing processing on the obtained physiological parameter information of the patient to obtain the processed physiological parameter information of the patient; and

[0061] perform event and entity extraction on the target data to obtain event and entity relationships, and generate structured target data based on the event and entity relationships, and store the processed physiological parameter information of the patient and the structured target data in the data storage device, wherein the target data includes the obtained medical data, human-machine medical interaction data, real-time medical event information, and historical medical information of the patient;

[0062] The data output device is further configured to send the processed physiological parameter information and structured target data of the patient within the first time period in the data storage device to the status evaluation subsystem and the auxiliary decision-making subsystem.

[0063] Beneficial effects:

[0064] (1) The present invention provides an auxiliary decision-making system applicable to the perioperative period and the intensive care period, which can provide users with multi-dimensional reference contents such as real-time monitoring, attribution analysis, and decision recommendation of physiological parameters, can provide auxiliary information for the decision-making of medical staff, and can improve the decision-making efficiency; at the same time, this system greatly reduces the requirements for the medical knowledge and personal professional qualities of medical staff, not only reduces the requirements for personnel capabilities, but also weakens the dependence on human experience, and the present invention can realize the dynamic adjustment of the calibration data of physiological parameters. Therefore, compared with the traditional technology, the reliability of use is improved; thus, the present invention is very suitable for large-scale application and promotion. Description of the Drawings

[0065] Figure 1 It is a schematic diagram of the architecture of the auxiliary decision-making system based on multi-modal data provided by an embodiment of the present invention;

[0066] Figure 2 It is a working flow chart of the auxiliary decision-making system based on multi-modal data provided by an embodiment of the present invention. Detailed Embodiments

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the present invention in combination with the drawings and the descriptions of the embodiments or the prior art. Obviously, the following descriptions of the structures of the drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. It should be noted here that the descriptions of these embodiment modes are used to help understand the present invention, but do not constitute a limitation to the present invention.

[0068] It should be understood that although terms such as first and second may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, the first unit can be called the second unit, and similarly, the second unit can be called the first unit, without departing from the scope of the exemplary embodiments of the present invention.

[0069] It should be understood that for the term "and / or" that may appear in this text, it is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, B exists alone, and both A and B exist simultaneously; for the term " / and" that may appear in this text, it is a description of another association object relationship, indicating that there can be two relationships. For example, A / and B can represent: A exists alone, and both A and B exist alone; in addition, for the character " / " that may appear in this text, generally it represents that the front and back associated objects are in an "or" relationship.

[0070] Embodiment:

[0071] Refer to Figure 1 As shown, the auxiliary decision-making system based on multimodal data provided in this embodiment may, but is not limited to, include: a data processing subsystem, a status evaluation subsystem, and an auxiliary decision-making subsystem. Among them, the data processing subsystem, as the data acquisition part, is used to obtain data in different medical devices and systems, and when interacting with humans, obtain external data, so as to provide a data basis for subsequent physiological parameter monitoring and generation of auxiliary decision-making suggestions; at the same time, the status evaluation subsystem is used to monitor and display the real-time status of the patient's physiological parameters, and the auxiliary decision-making subsystem provides auxiliary decision-making suggestion information according to the data output by the foregoing data processing subsystem and status evaluation subsystem.

[0072] Furthermore, the specific working processes of the foregoing three subsystems are disclosed as follows:

[0073] In this embodiment, the data processing subsystem is used to obtain the human-machine medical interaction data, real-time medical event information, and medical assistance information during the first time period in the guardianship period; among them, for example, the foregoing medical assistance information may, but is not limited to, include the patient's historical medical information sent by the medical information management system, the patient's physiological parameter information sent by the medical monitoring device, and the medical data sent by the medical treatment device; optionally, for example, the foregoing first time period is a time period preset duration before the current moment. For example, if the current moment is 3 o'clock on December 1, 2022, then the data of the foregoing various devices and management systems within the preset duration before 3 o'clock is obtained. Of course, the preset duration can be specifically set according to actual use and is not limited here.

[0074] Among them, the data processing subsystem performs real-time acquisition and storage of data, that is, it acquires and stores human-machine medical interaction data, real-time medical event information, and medical assistance information in real time during the monitoring period; then, when the status evaluation subsystem and the auxiliary decision-making subsystem send data call requests to the data processing subsystem, the data processing subsystem filters out the human-machine medical interaction data, real-time medical event information, and medical assistance information within the first time period from the stored data and sends them to the status evaluation subsystem and the auxiliary decision-making subsystem.

[0075] Meanwhile, for example, the monitoring period may include, but is not limited to, the perioperative period and / or the intensive care period; in this way, multimodal data of the patient during the monitoring period is obtained, and subsequent physiological parameter monitoring and generation of auxiliary decision-making suggestions are carried out based on the obtained multimodal data.

[0076] In specific applications, for example, the aforementioned human-machine medical interaction data may include, but is not limited to, physiological parameter constraint conditions, historical auxiliary decision-making suggestions (i.e., the auxiliary decision-making suggestions generated at the previous moment), and historical attribution analysis results (i.e., the results selected by medical staff in the historical data attribution analysis results); meanwhile, for example, the physiological parameter constraint conditions are used to formulate optimization goals, that is, the calibration value intervals corresponding to physiological parameters, which are preset information and mainly include respiratory constraints (such as respiratory depression, hyper / hypocapnia, acute respiratory distress, etc. constraint conditions), circulatory constraints (such as mortality constraint conditions), brain constraints (such as cerebral autoregulation curve constraint conditions), and other constraints (such as body temperature management constraints with mortality as the goal). Currently, the aforementioned examples are only illustrative and not limited thereto.

[0077] Similarly, for example, the real-time medical event information may include, but is not limited to, the data input by medical staff during human-machine interaction with the data processing subsystem, such as blood loss, infusion pump output data, drug use data, surgical data, operation data, adverse event data, intraoperative test and inspection result data, etc.; among them, the surgical data may include, but is not limited to, the name of the ongoing surgical procedure, etc.; the operation data includes operation names such as tracheal intubation, skin incision, tissue dissection, ligation, suture, opening / closing the chest, opening / closing the abdomen, etc.; the adverse event data may include, but is not limited to, event names such as massive hemorrhage, hypotension, cardiac arrest, etc.

[0078] Furthermore, for example, the patient's historical medical information may include, but is not limited to, the patient's basic information (such as height, weight, age, gender, current medical history, personal history, past medical history, family history, reproductive history, allergy history, planned surgical procedures and anesthesia methods, etc.), patient medical imaging examination data and examination reports obtained from the picture archiving and communication system, and various non-imaging test index data obtained from the laboratory information system, etc.

[0079] Meanwhile, for example, medical monitoring devices can include, but are not limited to, monitors, anesthesia machines, ventilators, and anesthesia information systems, etc. Correspondingly, patient physiological parameter information can be classified into four categories, namely respiration, circulation, brain, and others; among them, the physiological parameters corresponding to respiration can include, but are not limited to: tidal volume, airway wind pressure, airway plateau pressure, driving pressure, SpO2 (oxygen saturation), EtCO2 (end-tidal carbon dioxide), PEEP (positive end-expiratory pressure), respiratory rate, minute ventilation volume, etc.; the physiological parameters corresponding to circulation are: heart rate, systolic blood pressure, diastolic blood pressure, mean pressure, PPV, cardiac output (CO), cardiac index (CI), etc.; the physiological parameters corresponding to the brain are: left cerebral oxygen, right cerebral oxygen, intracranial pressure, cerebral perfusion pressure, bispectral index (BIS), etc.; and the physiological parameters corresponding to others are parameters such as temperature.

[0080] Furthermore, for example, medical treatment devices can include, but are not limited to: infusion pumps, urine volume monitoring devices, etc., and the corresponding medical data are the infusion volume corresponding to the infusion pump, urine volume, and other information; of course, the above examples are only for illustration and are not limited thereto.

[0081] In this way, after the data processing subsystem completes the acquisition of multi-modal data during the patient's monitoring period (that is, retrieves the multi-modal data within the first time period from its data storage device), it can send it to the state evaluation subsystem and the auxiliary decision-making subsystem, so that subsequently, based on this, real-time monitoring of the physiological parameter status and the generation of auxiliary decision-making suggestions can be carried out.

[0082] Among them, the state evaluation subsystem is communicatively connected to the data processing subsystem and is used to generate a calibration value range of each physiological parameter within the second time period according to the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, so as to obtain the first state information of each physiological parameter of the patient at the current moment according to the calibration value range of each physiological parameter and the patient physiological parameter information, and is used to predict the second state information of each physiological parameter of the patient within the second time period according to the patient physiological parameter information, and visually display the first state information and the second state information of each physiological parameter, where the second time period is a time period with a preset duration after the current moment.

[0083] In this embodiment, the calibration value range of any physiological parameter may but is not limited to include: the baseline range (i.e., the normal value range), the upper limit range, and the lower limit range of the any physiological parameter. Among them, the upper limit range may include several upper limit sub-ranges that decrease in sequence, and the lower limit range may also include several lower limit sub-ranges that decrease in sequence. For example, for the heart rate, the baseline range is the normal heart rate value range (i.e., 60 - 100 beats per minute), each upper limit sub-range may be the first upper limit range (such as 120 - 150 beats per minute), the second upper limit range (101 - 119 beats per minute), each lower limit sub-range may be the first lower limit sub-range (such as 50 - 59 beats per minute) and the second lower limit sub-range (such as 49 - 49 beats per minute), etc. Therefore, it can correspond to tachycardia, slightly fast heart rate, slightly slow heart rate, bradycardia, etc.; of course, the calibration value range can also be set with an upper limit range and a lower limit range, or can be set to the corresponding upper and lower limit values, and can be specifically set according to actual use, and is not limited to the foregoing examples here.

[0084] Thus, according to the physiological parameters of the patient collected in real time by the data processing subsystem, and combined with the calibration value ranges of the foregoing various physiological parameters, the first state information of each physiological parameter at the current moment can be obtained. Among them, the first state information may but is not limited to include a critical state or a normal state, that is, when the real-time data of each physiological parameter is not within the corresponding baseline range, it is determined that each physiological parameter is in a critical state, and vice versa, it is determined that it is in a normal state.

[0085] Furthermore, in this embodiment, predictions of various physiological parameters are also made, that is, based on the physiological parameter information of the patient, the second state information of each physiological parameter within the second time period is predicted. For example, the current moment is 3 o'clock on December 1, 2022, and the preset duration is 15 minutes. Then, the second time period is the time period between 3 o'clock and 3:15. That is to say, this embodiment can obtain the state information of each physiological parameter between 3 o'clock and 3:15. Thus, visualizing the first state information and the second state information of each physiological parameter can enable medical staff to timely grasp the physiological state of the patient; at the same time, this embodiment can also alarm each physiological parameter in a critical state, so as to realize the alarm prompt of the patient's physiological state.

[0086] After completing the real-time state monitoring of the patient's physiological parameters, this embodiment then performs data attribution analysis so as to subsequently generate auxiliary decision-making suggestion information of the patient at the current moment based on this; the working process of the auxiliary decision-making subsystem is as follows:

[0087] An auxiliary decision-making subsystem is used to obtain attribution analysis support data and decision analysis support data. Among them, the attribution analysis support data includes a first medical knowledge graph and first medical knowledge vector data, and the decision analysis support data includes a second medical knowledge graph and second medical knowledge vector data. Among them, for example, the aforementioned two medical knowledge graphs and medical knowledge vector data can be constructed based on, but not limited to, clinical practice guidelines, expert consensus, and / or medical standard terminology sets. Optionally, the specific construction process will be elaborated below.

[0088] In this embodiment, the auxiliary decision-making subsystem is communicatively connected to the data processing subsystem and the status evaluation subsystem. After obtaining the aforementioned support data, it is used to perform data attribution analysis based on the attribution analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the first status information of each physiological parameter, and the second status information of each physiological parameter, to obtain the data attribution analysis result at the current moment, and visualize the data attribution analysis result. Among them, in specific implementation, for example, the data attribution analysis result is the medical knowledge in the first medical knowledge graph and the first medical knowledge vector data that affects each physiological parameter. For example, the medical knowledge that affects blood pressure can be blood viscosity, human metabolism, etc. Of course, the aforementioned examples are only illustrative and are not limited to this here.

[0089] Furthermore, the auxiliary decision-making subsystem is also used to generate auxiliary decision-making suggestion information for the patient at the current moment based on the decision analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the data attribution analysis result, the first status information of each physiological parameter, and the second status information of each physiological parameter, and visualize the auxiliary decision-making suggestion information. In specific implementation, the auxiliary decision-making suggestion is the medical knowledge for adjusting each physiological parameter, and it is also selected from the second medical knowledge graph and the second medical knowledge vector data.

[0090] In this way, by displaying the status of the patient's physiological parameters at the current moment and generating the auxiliary decision-making suggestion information at the current moment, it can help medical staff timely understand the patient's physiological status and provide auxiliary information for the decision-making of medical staff, thereby improving the decision-making efficiency.

[0091] Further, in this embodiment, when the current moment is the last moment within the second time period, the data processing subsystem updates the current moment to the last moment, and re-obtains the human-machine medical interaction data, real-time medical event information, and medical assistance information within the first time period until the end of the monitoring period; for example, on the basis of the previous example, when the current moment is 3:14:59 s, then 3:14:59 s is used as the current moment, and then, the human-machine medical interaction data, real-time medical event information, and medical assistance information between 3 o'clock and 3:14:59 s can be obtained from the data stored in the data storage device in real time, and the previous process is re-executed until the end of the monitoring period; in this way, the dynamic adjustment of the calibration value intervals corresponding to each physiological parameter during the monitoring period and the dynamic generation of auxiliary decision-making suggestions can be realized, thereby improving the reliability of system monitoring.

[0092] Based on the above description, this embodiment provides an auxiliary decision-making system applicable to the perioperative period and the intensive care period, which can provide users with multi-dimensional reference content such as real-time monitoring, attribution analysis, and decision recommendation of physiological parameters. In this way, it can provide auxiliary information for the decision-making of medical staff, thereby improving the decision-making efficiency.

[0093] In a possible design, the second aspect of this embodiment provides a detailed system architecture of each subsystem in the first aspect of the embodiment, and its system architecture is as shown below.

[0094] First, a detailed system architecture of the data processing subsystem is provided:

[0095] In this embodiment, for example, the data processing subsystem may include, but is not limited to: a first data receiving device, a second data receiving device, a third data receiving device, a data storage device, and a data output device.

[0096] See Figure 1 As shown, the first data receiving device is communicatively connected to the medical monitoring device, and is used to obtain the patient's physiological parameter information sent by the medical monitoring device in real time, and send the patient's physiological parameter information to the data storage device; similarly, the second data receiving device is communicatively connected to the medical treatment device, and is used to obtain the medical data sent by the medical treatment device in real time, and is used to obtain the human-machine medical interaction data and real-time medical event information in real time, and send the medical data, human-machine medical interaction data, and real-time medical event information obtained in real time to the data storage device; and the third data receiving device is communicatively connected to the medical information management system, and is used to obtain the patient's historical medical information sent by the medical information management system in real time, and send the patient's historical medical information obtained in real time to the data storage device.

[0097] In this way, through three data receiving devices, multi-modal data including the patient's historical medical information, the patient's physiological parameter information, medical data, human-machine medical interaction data, and real-time medical event information can be obtained in real time, thereby providing a data basis for subsequent monitoring of the physiological parameter status and generation of auxiliary decision-making information.

[0098] Meanwhile, the data storage device is used to store the received patient's physiological parameter information, medical data, human-machine medical interaction data, real-time medical event information, and the patient's historical medical information. And the data output device is used to read the patient's physiological parameter information, medical data, human-machine medical interaction data, real-time medical event information, and the patient's historical medical information stored in the data storage device and within the first time period, and send the read patient's physiological parameter information, medical data, human-machine medical interaction data, real-time medical event information, and the patient's historical medical information to the status evaluation subsystem and the auxiliary decision-making subsystem, thereby realizing the monitoring of the physiological parameter status of the patient and the generation of auxiliary decision-making information.

[0099] Furthermore, in this embodiment, since the collected medical auxiliary information includes text information and numerical information, that is, the real-time obtained medical data, human-machine medical interaction data, real-time medical event information, and the patient's historical medical information are text data, while the patient's physiological parameter information is digital information. Therefore, to ensure the accuracy of the collected data, a data preprocessing device is also provided in the data processing subsystem to realize the data preprocessing of the foregoing data.

[0100] Specifically, the data preprocessing device is used to perform missing value filling and outlier data clearing processing on the real-time obtained patient's physiological parameter information to obtain the processed patient's physiological parameter information; and perform event and entity extraction on the target data to obtain the event and entity relationship, and generate structured target data based on the event and entity relationship, and store the processed patient's physiological parameter information and the structured target data in the data storage device; in this embodiment, for example, the target data includes the aforementioned real-time obtained medical data, human-machine medical interaction data, real-time medical event information, and the patient's historical medical information.

[0101] Furthermore, for example, the data preprocessing device mainly includes a text data processing module and a time series data processing module. See Figure 1As shown, the text data processing module mainly extracts events and entities from the aforementioned target data to generate structured target data, while the time series data processing module performs processing such as missing value filling and outlier removal on real-time continuous waveform data and numerical data (i.e., the aforementioned patient physiological parameter information, such as respiratory rate, heart rate, blood pressure waveform, airway pressure waveform), so as to obtain the processed patient physiological parameter information; then, both of them store the data obtained from their respective processing into the data storage device; finally, when receiving a data call request from the rest of the subsystems, the data output device sends the processed patient physiological parameter information and structured target data within the first time period in the data storage device to the status evaluation subsystem and the auxiliary decision-making subsystem for subsequent physiological parameter status monitoring and generation of auxiliary decision-making information.

[0102] Optionally, for example, NLP (Natural Language Processing) information extraction technology can be used to perform event and entity extraction; of course, the NLP technology is a commonly used technology for text processing, and its principle will not be elaborated here.

[0103] In addition, for example, the data storage device can also store the data output by the status evaluation subsystem and the auxiliary decision-making subsystem, and the data output device can also read the output data corresponding to the status evaluation subsystem and the auxiliary decision-making subsystem stored in the data storage device for visual display.

[0104] In this way, through the detailed architecture description of the aforementioned data processing subsystem, the acquisition of multi-modal medical data of the patient during the monitoring period can be completed, thus providing an accurate data basis for subsequent status monitoring and generation of auxiliary decision-making information.

[0105] Secondly, the following discloses one system architecture of the status evaluation subsystem:

[0106] In this embodiment, refer to Figure 1As shown, the example state evaluation subsystem may but is not limited to include: an optimization target generation device and a parameter current state management and crisis warning device; among them, the optimization target generation device is used to generate a calibration value interval of each physiological parameter within a second time period according to the physiological parameter constraint conditions, the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, and send the calibration value intervals of each physiological parameter to the parameter current state management and crisis warning device, and visually display the physiological parameter constraint conditions; in this way, the generated calibration value intervals of each physiological parameter can be used as the target reference intervals during the patient state process, so that based on this, the state monitoring of the patient's physiological parameters can be carried out; at the same time, the example can also send the calibration value intervals of the foregoing physiological parameters within the second time period to the data storage device for storage.

[0107] Furthermore, one specific architecture of the disclosed optimization target generation device is as follows.

[0108] In specific applications, refer to Figure 1 As shown, the example optimization target generation device may but is not limited to include: a first data calling module, an optimization target generation module, a first data output module, and a first display module.

[0109] Among them, the first data calling module is communicatively connected to the data processing subsystem, and is used to retrieve the human-machine medical interaction data, the real-time medical event information, and the medical assistance information in the data processing subsystem, and send the retrieved human-machine medical interaction data, real-time medical event information, and medical assistance information to the optimization target generation module, and send the physiological parameter constraint conditions in the human-machine medical interaction data to the first display module; specifically, it retrieves the human-machine medical interaction data, real-time medical event information, and medical assistance information that have been processed by the data preprocessing device and are within the first time period from the data storage device.

[0110] Then, the optimization target generation module is used to generate a calibration value interval of each physiological parameter within a second time period according to the physiological parameter constraint conditions, the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, and send the calibration value intervals of each physiological parameter to the first data output module.

[0111] In specific implementation, the physiological parameter constraint conditions include several constraint conditions, that is, the four major types of constraint conditions exemplified above; in this way, the optimization target generation module generates the calibration value intervals of each physiological parameter according to the foregoing several constraint conditions and the data obtained by the three data receiving devices.

[0112] Optionally, refer to Figure 1As shown, the optimization target generation module mainly includes a first calculation and analysis unit. Among them, the first calculation and analysis unit includes several parameter calculation units, and each parameter calculation unit corresponds to a constraint condition. In this way, each parameter calculation unit is used to determine the corresponding physiological parameter from the patient's physiological parameter information, and determine the corresponding constraint condition from the physiological parameter constraint condition, so as to generate the calibration value interval of the corresponding physiological parameter within the second time period based on the human-machine medical interaction data, the real-time medical event information, the patient's historical medical information, the medical data, as well as the corresponding physiological parameter and the corresponding constraint condition, and by using the first machine learning model.

[0113] At the same time, the first calculation and analysis unit is also used to send the calibration value intervals of the physiological parameters corresponding to each parameter calculation unit within the second time period to the first display module and the first data output module for visual display, and output them to the parameter current state management and crisis warning device through the first data output module for parameter state monitoring.

[0114] In specific applications, for example, several parameter calculation units may include, but are not limited to, a heart rate and blood pressure optimization target calculation unit, a cerebral oxygen optimization target calculation unit, and a respiratory parameter optimization target calculation unit.

[0115] Among them, in a specific implementation, for example, a machine model based on the BP (blood pressure) waveform and with the mortality rate as a constraint is set in the heart rate and blood pressure optimization target calculation unit. The model outputs the calibration value intervals of a series of parameters such as HR (heart rate), SBP (systolic blood pressure), MBP (mean blood pressure), and DBP (diastolic blood pressure). For example, when the optimization management target is set to a mortality rate less than 0.0093, the upper and lower limits of the SBP optimization management threshold (i.e., the calibration value interval) are obtained as 130 - 175, and the upper and lower limits of the HR optimization management are 58 - 90.

[0116] Similarly, in a specific implementation, a machine learning model based on the EEG (electroencephalogram) waveform, cerebral oxygen saturation, and with the depth of anesthesia as a constraint is set in the cerebral oxygen optimization target calculation unit. The model output is the calibration value interval of the cerebral oxygen saturation.

[0117] For another example, in a specific implementation, a machine learning model based on respiratory parameters, tissue oxygen saturation, and with the probability of acute respiratory distress as a constraint can also be set in the cerebral oxygen optimization target calculation unit. The model output is the calibration value intervals of tissue oxygen saturation, pulse oxygen saturation, PaO2 (evaluating arterial oxygen partial pressure), PaCO2 (carbon dioxide partial pressure), respiratory rate, PEEP (positive end-expiratory pressure), and FiO2 (fraction of inspired oxygen).

[0118] Of course, the foregoing examples are for illustration purposes. For different target computing units, machine learning models with different constraint conditions can be set, which are not limited to the foregoing examples. At the same time, the first machine learning module can use common neural network models, taking the historical physiological parameters of patients and different constraint conditions as inputs and the calibration value intervals of different patients' physiological parameters as outputs for training. In this way, the trained model can be directly used to generate the calibration value intervals corresponding to each physiological parameter.

[0119] Furthermore, referring to Figure 1 as shown, for example, the optimization target generation module may further include: a first data calling unit and a first optimization unit. The first data calling unit is configured to obtain the historical calibration value intervals of each physiological parameter within the first time period, the first state information of each physiological parameter at the current moment, and retrieve the real-time medical event information, medical assistance information, and human-machine medical interaction data in the data processing subsystem (of course, it refers to the preprocessed data in the foregoing data storage device within the first time period), and send the historical calibration value intervals of each physiological parameter, the first state information of each physiological parameter, the real-time medical event information, the medical assistance information, and the human-machine medical interaction data to the first optimization unit.

[0120] The first optimization unit is configured to optimize the first machine learning model adopted by each parameter computing unit according to the historical calibration value intervals of each physiological parameter, the first state information of each physiological parameter, the real-time medical event information, the medical assistance information, and the human-machine medical interaction data, to obtain each optimized first machine learning model, so as to use each optimized first machine learning model to generate the calibration value intervals corresponding to each physiological parameter.

[0121] In this embodiment, the first data calling unit is equivalent to obtaining historical data and the data obtained by the data processing subsystem to perform the feedback optimization process of the machine learning model in each of the foregoing parameter computing units, that is, continuously optimizing the algorithm models in a series of parameter computing units used in the computing and analysis unit.

[0122] In this embodiment, the preset interval can be, but is not limited to, set as an integer multiple of the first time period. For example, if the first time period is 30 minutes, it can be set to update the machine learning model every 150 minutes. For example, when starting to time, if the current time is exactly 3 o'clock, then after 150 minutes, that is, when the current time is 5:30, obtain the historical calibration value interval of each physiological parameter between 5 o'clock and 5:30, as well as the first state information at the current moment and the data obtained by the data processing subsystem within the first time period. Then, based on this, feedback to optimize the algorithm model in the parameter calculation unit. Of course, feedback optimization is a common optimization method for machine learning, and its principle will not be elaborated here.

[0123] Based on this, during the use process, the machine generation model corresponding to the calibration value interval can be continuously optimized, thereby continuously improving the accuracy of generating the calibration value interval corresponding to each physiological parameter.

[0124] At the same time, the optimization target generation module can also dynamically adjust the calibration value interval of each physiological parameter in real time throughout the monitoring period, thereby ensuring the reliability of parameter monitoring. The dynamic adjustment can refer to the first aspect of the foregoing embodiment and will not be elaborated here.

[0125] Thus, after the optimization target generation module generates the calibration value interval of each physiological parameter within the second time period, it can be output to the first data output module, so that the first data output module receives the calibration value interval of each physiological parameter and sends the calibration value interval of each physiological parameter to the first display module and the parameter current state management and crisis warning device, and sends it to the data processing subsystem for storage. Finally, the first display module is used to visually display the calibration value interval of each physiological parameter and the physiological parameter constraint conditions.

[0126] In this embodiment, after obtaining the calibration value interval of each physiological parameter, the state monitoring of the physiological parameter can be carried out based on the parameter current state management and crisis warning device. Specifically, its working process is as follows.

[0127] The parameter current state management and crisis warning device is used to generate the first state information of each physiological parameter of the patient at the current moment according to the calibration value interval of each physiological parameter and the patient's physiological parameter information, and predict the second state information of each physiological parameter of the patient within the second time period according to the patient's physiological parameter information, the human-machine medical interaction data, and the real-time medical event information, and visually display the first state information and the second state information of each physiological parameter.

[0128] In this embodiment, refer to Figure 1As shown, the example parameter current state management and crisis warning device may include, but is not limited to: a second data calling module, a current state analysis module, a crisis warning module, a second data output module, and a second display module.

[0129] Among them, the second data calling module is communicatively connected to the data processing subsystem and the optimization target generation device, and is used to retrieve the human-machine medical interaction data, real-time medical event information, and medical assistance information obtained by the data processing subsystem (the foregoing data is the data processed by the data preprocessing device), and the calibration value intervals of each physiological parameter generated by the optimization target generation device at the current moment, and send the retrieved calibration value intervals of each physiological parameter and the patient physiological parameter information in the medical assistance information to the current state analysis module.

[0130] The current state analysis module is used to generate the first state information of each physiological parameter of the patient at the current moment according to the calibration value intervals of each physiological parameter and the patient physiological parameter information, and send the first state information of each physiological parameter to the second data output module; in this embodiment, the current state analysis module compares the real-time data of each physiological parameter with the corresponding calibration value interval to obtain the first state information of each physiological parameter. That is, assuming that the real-time data of any physiological parameter is lower than the second lower sub-interval in the calibration value interval, it can be explained that there is a problem of low parameter, that is, in a crisis state; in this embodiment, the low parameter has different prompt names according to different physiological parameters, such as bradycardia, severe hypotension, etc. At the same time, when the real-time data of the any physiological parameter is higher than the first upper sub-interval in the calibration value interval, the parameter can be severely too high, such as severe hypertension, etc.; of course, the names of the crisis states corresponding to different physiological parameters are different, such as large tidal volume, high airway pressure, high oxygenation, low oxygenation, hypocapnia, severe hypoxemia, low body temperature, high body temperature, bradycardia, deep anesthesia, light anesthesia, hypotension, hypertension, etc., which will not be elaborated here one by one.

[0131] In this way, the current state analysis module can be used to monitor the crisis or optimal state of each physiological parameter of the patient.

[0132] At the same time, this embodiment also has a state prediction function, that is, the second data calling module is also used to send the human-machine medical interaction data, real-time medical event information, and medical assistance information to the crisis warning module; the crisis warning module is used to predict the second state information of each physiological parameter of the patient within the second time period according to the patient physiological parameter information, the human-machine medical interaction data, and the real-time medical event information in the medical assistance information, and send the second state information of each physiological parameter to the second data output module.

[0133] Furthermore, referring to Figure 1 as shown, for example, the aforementioned crisis warning module may include, but is not limited to: a second calculation and analysis unit, a second data calling unit, and a second optimization unit.

[0134] In specific implementation, the second calculation and analysis unit is used to predict the second state information of each physiological parameter of the patient within the second time period according to the patient's physiological parameter information, the human-machine medical interaction data, and the real-time medical event information, and by using a second machine learning model. Among them, the human-machine medical interaction data includes historical auxiliary decision-making suggestions. Thus, the second calculation and analysis unit performs the state prediction of each physiological parameter based on the historical auxiliary decision-making suggestions, the aforementioned patient's physiological parameter information, and the real-time medical event information.

[0135] Optionally, the aforementioned second machine learning model may include, but is not limited to, a one-dimensional convolutional neural network (1-Dimensional Convolutional Neural Networks, 1-DCNN), a logistic regression model, a multivariate regression analysis model, a GRU (gated recurrent unit), a random forest, an ARIMA (autoregressive integrated moving average model), etc.; at the same time, what the aforementioned second machine learning model outputs is the probability of a crisis occurring or a classification result; of course, the aforementioned second machine learning model is trained with historical auxiliary decision-making suggestions, patient physiological parameter information, and historical real-time medical event information as inputs and the state prediction results of different physiological parameters as outputs; therefore, when in use, the trained second machine learning model can be directly called to perform the state prediction of physiological parameters.

[0136] At the same time, to ensure the accuracy of the prediction of the second machine learning model, this embodiment also sets up a model optimization process, that is:

[0137] The second data calling unit is used to retrieve the human-machine interaction medical data, real-time medical event information, and medical assistance information in the data processing subsystem at a preset interval, and obtain the historical second state information of each physiological parameter within the first time period, and send the human-machine interaction medical data, real-time medical event information, medical assistance information, and the historical second state information of each physiological parameter to the second optimization unit; and the second optimization unit is used to optimize the second machine learning model in the second calculation and analysis unit according to the human-machine interaction medical data, real-time medical event information, medical assistance information, and the historical second state information of each physiological parameter, to obtain an optimized second machine learning model, so as to use the optimized second machine learning model to generate the second state information corresponding to each physiological parameter.

[0138] In this embodiment, it is equivalent to continuously accessing the real-time data of the patient (i.e., the aforementioned machine-interactive medical data, real-time medical event information, and medical assistance information) and the results of historical predictions (i.e., the historical second state information of the aforementioned various physiological parameters) to the second optimization unit, and then running a feedback optimization algorithm to continuously optimize and adjust the crisis warning model (i.e., the second machine learning model).

[0139] In this way, continuous optimization of the second machine learning model in the crisis warning model can be achieved, thus ensuring the accuracy of the prediction.

[0140] Thus, after obtaining the predicted states of the various physiological parameters through the crisis warning module, the real-time states of the various physiological parameters at the current moment can be combined for output display, that is: the second data output module is used to receive the first state information and the second state information of the various physiological parameters, and send the first state information and the second state information of the various physiological parameters to the second display module; and the second display module is used to visually display the first state information and the second state information of the various physiological parameters, wherein the first state information and the second state information of any physiological parameter include a crisis state or a normal state, and the second display module is used to give an alarm prompt for the physiological parameters in the crisis state.

[0141] In addition, in this embodiment, the second data output module is further used to send the first state information and the second state information of the various physiological parameters to the aforementioned data storage device for storage.

[0142] Through the above detailed architecture description of the state evaluation subsystem, real-time state monitoring and alarm of the various physiological parameters of the patient during the monitoring period can be achieved, and at the same time, state prediction of the various physiological parameters can be realized, thus providing a data basis for subsequent attribution analysis and generation of auxiliary decision-making recommendation information.

[0143] Finally, the detailed system architecture of the following disclosed auxiliary decision-making subsystem is as follows:

[0144] See Figure 1 As shown, the auxiliary decision-making subsystem may include, but is not limited to: an attribution analysis device, a decision recommendation device, and a technical solution database; wherein, the technical solution database stores the attribution analysis support data and the decision analysis support data; in this embodiment, the first medical knowledge graph in the aforementioned attribution analysis support data is a causal knowledge graph.

[0145] Optionally, the construction process of the following disclosed first medical knowledge graph is as follows:

[0146] Construct a medical knowledge graph medical knowledge base using clinical medical guidelines, the OMOP (Observational Medical Outcomes Partnership) terminology library (i.e., the medical standard terminology library), electronic medical records, doctor experience, and medical literature as knowledge sources. Then, use OMOP terminology coding as medical concepts to uniformly identify the data in the knowledge base, and construct the top-level framework of the medical knowledge structure based on the OMOP common data model to establish the top-level semantic classes and semantic relationships. Then, for the information of specific diseases, add subclass, sub-relationship, instance, and attribute information to finally form a causal medical knowledge graph.

[0147] Similarly, the construction process of the first medical vector data is as follows: Construct a knowledge base using clinical guidelines, the OMOP (Observational Medical Outcomes Partnership) terminology library, electronic medical records, doctor experience, and medical literature as knowledge sources. Then, create keywords for the knowledge base, split the documents in the knowledge base into segments, extract summaries for each document segment, and then use the extracted key information as nodes. Next, create a tree-like index for each document in the knowledge base in a bottom-up manner. Finally, use the fine-tuned Chinese semantic vector model to extract features for each document segment and node in the knowledge base, and then construct the first medical knowledge vector data.

[0148] Furthermore, the construction process of the second medical knowledge graph in the decision analysis support data is as follows:

[0149] Extract causal knowledge related to diagnosis and treatment from knowledge sources such as clinical medical guidelines, the OMOP (Observational Medical Outcomes Partnership) terminology library, electronic medical records, doctor experience, and medical literature, and construct a causal knowledge graph medical knowledge base containing key information such as medical history, symptoms, signs, laboratory test results, medications, and demographic information. Then, use OMOP terminology coding as medical concepts for unified identification, and construct the top-level framework of the medical knowledge structure based on the OMOP common data model to establish the top-level semantic classes and semantic relationships. Finally, for the information of specific diseases, add subclass, sub-relationship, instance, and attribute information to form the second medical knowledge graph. Based on this, the aforementioned knowledge graph uses the OWL restrict language and the Apache Jena Rules language to construct a clinical decision support inference rule library, and the node elements in the rules conform to the medical knowledge structure framework of the knowledge graph and the OMOP terminology coding system.

[0150] The construction process of the second medical knowledge vector data is as follows:

[0151] By extracting causality knowledge related to diagnosis and treatment from knowledge sources such as clinical medicine guidelines, the OMOP (Observational Medical Outcomes Partnership) terminology library, electronic medical records, doctor experience, and medical literature, a causality - containing knowledge base is constructed, which includes key diagnostic information such as medical history, symptoms, signs, laboratory test results, medications, demographic information, etc. Keywords are created for the knowledge base, the documents in the knowledge base are segmented into fragments, abstracts are extracted from each document fragment, and then the extracted key information is used as nodes to create a tree - like index for each document in the knowledge base in a bottom - up manner; then, through the fine - tuned Chinese semantic vector model, feature extraction is performed on each document fragment and node in the knowledge base, and based on this, the second medical knowledge vector data is constructed.

[0152] Of course, the foregoing is only one way to construct the knowledge graph and medical knowledge vector data, and other methods can also be used for construction, and it is not limited to the foregoing examples here.

[0153] Thus, when the attribution analysis support data and decision - making analysis support data are obtained, based on this, attribution analysis can be performed, and auxiliary decision - making recommendation information can be generated.

[0154] Specifically, the attribution analysis device is used to perform data attribution analysis according to the attribution analysis support data, the human - machine medical interaction data, the real - time medical event information, the medical assistance information, the first - state information of each physiological parameter, and the second - state information of each physiological parameter, obtain the data attribution analysis result at the current moment, send the data attribution analysis result to the decision - making recommendation device, and visually display the data attribution analysis result.

[0155] In this embodiment, as shown in Figure 1 For example, the attribution analysis device may include, but is not limited to: a third data - calling module, an attribution analysis module, a third data - output module, and a third display module.

[0156] Among them, the third data - calling module is used to obtain the human - machine medical interaction data, the real - time medical event information, and the medical assistance information in the data processing subsystem (the foregoing data are the data processed by the data pre - processing device, and the data acquisition time is within the first time period), the first - state information and the second - state information of each physiological parameter in the state evaluation subsystem, and the attribution analysis support data in the technical solution database, and send the obtained human - machine medical interaction data, real - time medical event information, medical assistance information, the first - state information and the second - state information of each physiological parameter, and the attribution analysis support data to the attribution analysis module.

[0157] In this embodiment, the attribution analysis module extracts keywords from the first status information and the second status information by using the aforementioned first medical knowledge graph and the first medical knowledge vector data, then performs similarity matching with the aforementioned vector data, and recalls relevant knowledge data in combination with the knowledge graph, so as to obtain corresponding medical knowledge, that is:

[0158] The attribution analysis module is used to perform data attribution analysis on the human-machine medical interaction data, real-time medical event information, medical assistance information, the first status information and the second status information of each physiological parameter, and the attribution analysis support data by using an artificial intelligence algorithm to obtain a number of initial data attribution analysis results; in this embodiment, each initial data attribution analysis result includes the classification probability value of the corresponding medical knowledge. Then, the attribution analysis module selects the final medical knowledge according to the classification probability value, that is, the attribution analysis module is also used to filter a number of initial data attribution analysis results according to the classification probability value, so as to obtain the data attribution analysis result after filtering and send it to the third data output module.

[0159] In specific implementation, refer to Figure 1 As shown, the attribution analysis module includes a third calculation and analysis unit and a first filtering unit. Among them, the third calculation and analysis unit is used to perform attribution analysis to obtain a number of initial data attribution analysis results; and the first filtering unit is used to filter a number of initial data attribution analysis results according to the classification probability value, so as to obtain the data attribution analysis result; in this embodiment, for example, the first filtering unit filters out the initial data attribution analysis results with a classification probability value lower than the probability threshold, and then uses the remaining initial data attribution analysis results as the final data attribution result; among them, for example, the probability threshold can be but is not limited to 80%; of course, the probability threshold can be specifically set according to actual use and is not limited to the aforementioned example here.

[0160] After completing the attribution analysis, the third data output module is used to receive the data attribution analysis result and send the data attribution analysis result to the third display module; and the third display module can visually display the data attribution analysis result so that medical staff can view the medical knowledge affecting each physiological parameter; in addition, in this embodiment, the third data output module can also send the aforementioned data attribution analysis result to the data storage device for storage.

[0161] In specific implementation, when the third calculation and analysis unit performs attribution analysis, the artificial intelligence algorithms used can include but are not limited to decision trees, correlation analysis, Cot model (Chain of Thought, which allows the large model to reason step by step rather than directly giving the answer at once), RAG (Retrieval-Augmented Generation), Graph RAG (Retrieval-Augmented Generation combined with knowledge graphs), or KG-RAG (Knowledge Graph Enhanced Prompt Generation Framework), etc. Of course, the above methods are common methods for attribution analysis, and their principles will not be elaborated here.

[0162] After completing the data attribution analysis, based on the data attribution analysis results, as well as the data output by the aforementioned data processing subsystem and status evaluation subsystem, auxiliary decision-making recommendation information can be generated. The process is as follows:

[0163] The decision recommendation device is used to generate auxiliary decision-making recommendation information for the patient at the current moment according to the decision analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the data attribution analysis results, the first state information of each physiological parameter, and the second state information of each physiological parameter, and visually display the auxiliary decision-making recommendation information.

[0164] In specific implementation, as shown in Figure 1 For example, the decision recommendation device can include but is not limited to: a fourth data calling module, a decision recommendation module, a fourth data output module, and a fourth display module.

[0165] Among them, the fourth data calling module is used to obtain the human-machine medical interaction data, real-time medical event information, and medical assistance information in the data processing subsystem, the first state information and second state information of each physiological parameter in the status evaluation subsystem, the data attribution analysis results, and the decision analysis support data in the technical solution database, and send the obtained data to the decision recommendation module.

[0166] Among them, the decision recommendation module combines the second medical knowledge graph and the second medical knowledge vector data to extract keywords for the current crisis state (i.e., each first state information) and the predicted crisis state (i.e., each second state information), then performs similarity matching with the second medical knowledge vector data, and recalls relevant knowledge data in combination with the second medical knowledge graph, so as to obtain the corresponding auxiliary decision-making recommendation information.

[0167] Specifically, the decision recommendation module is used to generate auxiliary decision-making recommendation information for the patient at the current moment based on the decision analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the data attribution analysis result, the first state information of each physiological parameter, and the second state information of each physiological parameter.

[0168] Optionally, for example, the decision recommendation module may include a fourth calculation and analysis unit and a second filtering unit. Among them, the fourth calculation and analysis unit uses the second medical knowledge graph and the second medical knowledge vector data to generate a number of initial auxiliary decision-making recommendation information. Each initial auxiliary decision-making recommendation information corresponds to a probability value, that is, the probability value of the recalled associated knowledge data. Then, the second filtering unit filters the number of initial auxiliary decision-making recommendation information to obtain the auxiliary decision-making recommendation information; of course, the second filtering unit also filters out the initial auxiliary decision-making recommendation information with a probability value lower than the probability threshold; at the same time, the initial auxiliary decision-making recommendation information is actually also the associated knowledge in the second medical knowledge graph and the second medical knowledge vector data, and its essence is a data matching and recall process.

[0169] Optionally, for example, the fourth calculation and analysis unit can use, but is not limited to, decision trees, correlation analysis, etc. to perform similarity matching and recall of associated knowledge data.

[0170] After obtaining the auxiliary decision-making recommendation information, it can be sent to the fourth data output module, and the fourth data output module sends it to the fourth display module for visual display based on the fourth display module; of course, when obtaining the data attribution analysis result and the auxiliary decision-making recommendation information, medical staff can manually select and display the data attribution analysis result and the auxiliary decision-making recommendation information selected by the medical staff.

[0171] Thus, the working process of the auxiliary decision-making system based on multi-modal data provided in this embodiment is as follows:

[0172] See Figure 2 As shown, the multi-modal data is collected through the data processing subsystem and then preprocessed; then, the preprocessed multi-modal data is transmitted to the optimization target generation module to formulate the target (that is, generate the calibration value interval corresponding to each physiological parameter) and display it; then, the current state management of the parameters and the crisis warning device monitor the current state of the physiological parameters and predict the state in the future time period, and also perform visual display.

[0173] Next, the preprocessed multimodal data, the current status and predicted status of each physiological parameter are transmitted to the auxiliary decision-making subsystem for attribution analysis, and the data attribution analysis results are visually displayed; finally, based on the data attribution analysis results, auxiliary decision-making suggestion information is generated, and decision recommendation and visual display are carried out; thus, it can assist medical staff in making decisions, thereby improving the decision-making efficiency.

[0174] Through the above description, the present invention provides an auxiliary decision-making system applicable to the perioperative period and the intensive care period, which can provide multi-dimensional reference content such as real-time monitoring, attribution analysis, and decision recommendation of physiological parameters for users, can provide auxiliary information for the decision-making of medical staff, and can improve the decision-making efficiency; at the same time, this system greatly reduces the requirements for the medical knowledge and personal professional qualities of medical staff, not only reduces the requirements for personnel capabilities, but also weakens the dependence on human experience, and the present invention can realize the dynamic adjustment of the calibration data of physiological parameters. Therefore, compared with the traditional technology, the reliability of use is improved; thus, the present invention is very suitable for large-scale application and promotion.

[0175] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An auxiliary decision-making system based on multimodal data, characterized in that Comprising: A data processing subsystem for obtaining human-machine medical interaction data, real-time medical event information, and medical assistance information within a first time period during the guardianship period, wherein the medical assistance information includes patient historical medical information sent by a medical information management system, patient physiological parameter information sent by a medical monitoring device, and medical data sent by a medical treatment device, and the first time period is a time period with a preset duration before the current moment, and the guardianship period includes the perioperative period and / or the intensive care period; A status evaluation subsystem communicatively connected to the data processing subsystem for generating a calibrated value range of each physiological parameter within a second time period according to the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, so as to obtain the first status information of each physiological parameter of the patient at the current moment according to the calibrated value range of each physiological parameter and the patient physiological parameter information, and for predicting the second status information of each physiological parameter of the patient within the second time period according to the patient physiological parameter information, and visually displaying the first status information and the second status information of each physiological parameter, wherein the second time period is a time period with a preset duration after the current moment; An auxiliary decision-making subsystem for obtaining attribution analysis support data and decision analysis support data, wherein the attribution analysis support data includes a first medical knowledge graph and first medical knowledge vector data, and the decision analysis support data includes a second medical knowledge graph and second medical knowledge vector data; The auxiliary decision-making subsystem communicatively connected to the data processing subsystem and the status evaluation subsystem for performing data attribution analysis according to the attribution analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the first status information of each physiological parameter, and the second status information of each physiological parameter, obtaining a data attribution analysis result at the current moment, and visually displaying the data attribution analysis result, wherein the data attribution analysis result is the medical knowledge in the first medical knowledge graph and the first medical knowledge vector data that affects each physiological parameter; The auxiliary decision-making subsystem is further configured to generate auxiliary decision-making advice information for the patient at the current moment according to the decision analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the data attribution analysis result, the first status information of each physiological parameter, and the second status information of each physiological parameter, and visually display the auxiliary decision-making advice information; Wherein, when the current moment is the last moment within the second time period, the data processing subsystem updates the current moment to the last moment, and re-obtains the human-machine medical interaction data, the real-time medical event information, and the medical assistance information within the first time period until the end of the guardianship period.

2. The auxiliary decision-making system based on multimodal data according to claim 1, wherein, The human-machine medical interaction data includes physiological parameter constraint conditions; The state evaluation subsystem includes: an optimization target generation device and a parameter current state management and crisis warning device; The optimization target generation device is configured to generate a calibration value range of each physiological parameter within a second time period according to the physiological parameter constraint conditions, the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, send the calibration value range of each physiological parameter to the parameter current state management and crisis warning device, and visually display the physiological parameter constraint conditions; The parameter current state management and crisis warning device is configured to generate first state information of each physiological parameter of the patient at the current moment according to the calibration value range of each physiological parameter and the patient's physiological parameter information, and predict second state information of each physiological parameter of the patient within a second time period according to the patient's physiological parameter information, the human-machine medical interaction data, and the real-time medical event information, and visually display the first state information and the second state information of each physiological parameter.

3. An auxiliary decision-making system based on multi-modal data according to claim 2, characterized in that, The optimization target generation device includes: a first data calling module, an optimization target generation module, a first data output module, and a first display module; The first data calling module is communicatively connected to the data processing subsystem, and is configured to retrieve the human-machine medical interaction data, the real-time medical event information, and the medical assistance information in the data processing subsystem, send the retrieved human-machine medical interaction data, real-time medical event information, and medical assistance information to the optimization target generation module, and send the physiological parameter constraint conditions in the human-machine medical interaction data to the first display module; The optimization target generation module is configured to generate a calibration value range of each physiological parameter within a second time period according to the physiological parameter constraint conditions, the human-machine medical interaction data, the real-time medical event information, and the medical assistance information, and send the calibration value range of each physiological parameter to the first data output module; The first data output module is configured to receive the calibration value range of each physiological parameter, send the calibration value range of each physiological parameter to the first display module and the parameter current state management and crisis warning device, and send it to the data processing subsystem for storage; The first display module is configured to visually display the calibration value range of each physiological parameter and the physiological parameter constraint conditions.

4. An auxiliary decision-making system based on multimodal data according to claim 3, characterized in that, The physiological parameter constraint conditions include a plurality of constraint conditions, and the optimization target generation module includes a first calculation and analysis unit, wherein the first calculation and analysis unit includes a plurality of parameter calculation units, and each parameter calculation unit corresponds to a constraint condition respectively; Among them, each parameter calculation unit is used to determine the corresponding physiological parameter from the patient physiological parameter information, and determine the corresponding constraint condition from the physiological parameter constraint conditions, so as to generate the calibrated value interval of the corresponding physiological parameter within the second time period based on the human-machine medical interaction data, the real-time medical event information, the patient historical medical information, the medical data, as well as the corresponding physiological parameter and the corresponding constraint condition, and use the first machine learning model; The first calculation and analysis unit is further used to send the calibrated value interval of the physiological parameter corresponding to each parameter calculation unit within the second time period to the first display module and the first data output module.

5. An auxiliary decision-making system based on multimodal data according to claim 4, characterized in that The optimization target generation module further includes: a first data calling unit and a first optimization unit; The first data calling unit is used to obtain the historical calibrated value interval of each physiological parameter within the first time period, the first state information of each physiological parameter at the current moment at a preset interval, and retrieve the real-time medical event information, medical assistance information and human-machine medical interaction data in the data processing subsystem, and send the historical calibrated value interval of each physiological parameter, the first state information of each physiological parameter, the real-time medical event information, the medical assistance information and the human-machine medical interaction data to the first optimization unit; The first optimization unit is used to optimize the first machine learning model adopted by each parameter calculation unit according to the historical calibrated value interval of each physiological parameter, the first state information of each physiological parameter, the real-time medical event information, the medical assistance information and the human-machine medical interaction data, and obtain each optimized first machine learning model, so as to use each optimized first machine learning model to generate the calibrated value interval corresponding to each physiological parameter.

6. The auxiliary decision-making system based on multimodal data according to claim 2, characterized in that, The parameter current state management and crisis warning device includes: a second data calling module, a current state analysis module, a crisis warning module, a second data output module and a second display module; The second data calling module is communicatively connected to the data processing subsystem and the optimization target generation device, and is used to retrieve the human-machine medical interaction data, real-time medical event information and medical assistance information obtained by the data processing subsystem, and the calibrated value interval of each physiological parameter generated by the optimization target generation device at the current moment, and send the retrieved calibrated value interval of each physiological parameter and the patient physiological parameter information in the medical assistance information to the current state analysis module; The second data calling module is further used to send the human-machine medical interaction data, real-time medical event information and medical assistance information to the crisis warning module; The current state analysis module is used to generate the first state information of each physiological parameter of the patient at the current moment according to the calibrated value interval of each physiological parameter and the patient physiological parameter information, and send the first state information of each physiological parameter to the second data output module; A crisis warning module, configured to predict, based on the patient's physiological parameter information in the medical assistance information, the human-machine medical interaction data, and the real-time medical event information, the second status information of each physiological parameter of the patient within a second time period, and send the second status information of each physiological parameter to a second data output module; The second data output module is configured to receive the first status information and the second status information of each physiological parameter, and send the first status information and the second status information of each physiological parameter to a second display module; The second display module is configured to visually display the first status information and the second status information of each physiological parameter, wherein the first status information and the second status information of any physiological parameter include a crisis status or a normal status, and the second display module is configured to give an alarm prompt for the physiological parameter in the crisis status.

7. An auxiliary decision-making system based on multi-modal data according to claim 6, characterized in that, The crisis warning module includes: a second calculation and analysis unit, a second data calling unit, and a second optimization unit; The second calculation and analysis unit is configured to predict, based on the patient's physiological parameter information, the human-machine medical interaction data, and the real-time medical event information, and by using a second machine learning model, the second status information of each physiological parameter of the patient within a second time period, wherein the human-machine medical interaction data includes historical auxiliary decision-making suggestions; The second data calling unit is configured to retrieve the human-machine interaction medical data, the real-time medical event information, and the medical assistance information in the data processing subsystem at a preset interval, and obtain the historical second status information of each physiological parameter within a first time period, and send the human-machine interaction medical data, the real-time medical event information, the medical assistance information, and the historical second status information of each physiological parameter to the second optimization unit; The second optimization unit is configured to optimize the second machine learning model in the second calculation and analysis unit according to the human-machine interaction medical data, the real-time medical event information, the medical assistance information, and the historical second status information of each physiological parameter, to obtain an optimized second machine learning model, so as to use the optimized second machine learning model to generate the second status information corresponding to each physiological parameter.

8. An auxiliary decision-making system based on multimodal data according to claim 1, characterized in that The auxiliary decision-making subsystem includes: an attribution analysis device, a decision recommendation device, and a technical solution database; Wherein, the technical solution database stores the attribution analysis support data and the decision analysis support data; The attribution analysis device is configured to perform data attribution analysis based on the attribution analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the first status information of each physiological parameter, and the second status information of each physiological parameter, obtain a data attribution analysis result at the current moment, send the data attribution analysis result to the decision recommendation device, and visually display the data attribution analysis result; A decision recommendation device, which is used to generate auxiliary decision-making recommendation information for a patient at the current moment according to the decision analysis support data, the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the data attribution analysis result, the first state information of each physiological parameter, and the second state information of each physiological parameter, and visually display the auxiliary decision-making recommendation information.

9. An auxiliary decision-making system based on multi-modal data according to claim 8, characterized in that, The attribution analysis device includes a third data call module, an attribution analysis module, a third data output module, and a third display module; The third data call module is used to obtain the human-machine medical interaction data, the real-time medical event information, and the medical assistance information in the data processing subsystem, the first state information and the second state information of each physiological parameter in the state evaluation subsystem, and the attribution analysis support data in the technical solution database, and send the obtained human-machine medical interaction data, real-time medical event information, medical assistance information, the first state information and the second state information of each physiological parameter, and the attribution analysis support data to the attribution analysis module; The attribution analysis module is used to perform data attribution analysis on the human-machine medical interaction data, the real-time medical event information, the medical assistance information, the first state information and the second state information of each physiological parameter, and the attribution analysis support data by using an artificial intelligence algorithm to obtain a number of initial data attribution analysis results, where each initial data attribution analysis result includes a classification probability value of the corresponding medical knowledge; The attribution analysis module is further used to perform filtering processing on a number of initial data attribution analysis results according to the classification probability value, so as to obtain the data attribution analysis result after the filtering processing and send it to the third data output module; The third data output module is used to receive the data attribution analysis result and send the data attribution analysis result to the third display module; The third display module is used to visually display the data attribution analysis result.

10. An auxiliary decision-making system based on multimodal data according to claim 1, characterized in that, The data processing subsystem includes: a first data receiving device, a second data receiving device, a third data receiving device, a data storage device, and a data output device; The first data receiving device is communicatively connected to the medical monitoring device and is used to obtain the physiological parameter information of the patient sent by the medical monitoring device in real time and send the physiological parameter information of the patient to the data storage device; The second data receiving device is communicatively connected to the medical treatment device and is used to obtain the medical data sent by the medical treatment device in real time, and is used to obtain the human-machine medical interaction data and the real-time medical event information in real time, and send the medical data, the human-machine medical interaction data, and the real-time medical event information obtained in real time to the data storage device; The third data receiving device is communicatively connected to the medical information management system and is used to obtain the historical medical information of the patient sent by the medical information management system in real time and send the historical medical information of the patient obtained in real time to the data storage device; A data storage device for storing received patient physiological parameter information, medical data, human-machine medical interaction data, real-time medical event information, and patient historical medical information; A data output device for reading the patient physiological parameter information, medical data, human-machine medical interaction data, real-time medical event information, and patient historical medical information stored in the data storage device and within a first time period, and sending the read patient physiological parameter information, medical data, human-machine medical interaction data, real-time medical event information, and patient historical medical information to a status evaluation subsystem and an auxiliary decision-making subsystem; Wherein, the data processing subsystem further includes: a data preprocessing device, and the real-time acquired medical data, human-machine medical interaction data, real-time medical event information, and patient historical medical information are text data; A data preprocessing device for performing missing value filling and outlier data clearing processing on the real-time acquired patient physiological parameter information to obtain processed patient physiological parameter information; and Performing event and entity extraction on the target data to obtain event and entity relationships, generating structured target data based on the event and entity relationships, and storing the processed patient physiological parameter information and the structured target data in the data storage device, wherein the target data includes real-time acquired medical data, human-machine medical interaction data, real-time medical event information, and patient historical medical information; The data output device is further configured to send the processed patient physiological parameter information and the structured target data within a first time period in the data storage device to the status evaluation subsystem and the auxiliary decision-making subsystem.

Images