Multi-modal large model-based sepsis interpretable early warning system

Abstract

The invention relates to a sepsis interpretable early warning system and method based on a multi-mode large model, and belongs to the field of medical big data and artificial intelligence. The method is realized by a sepsis interpretability early warning system based on a multi-modal large model. The sepsis interpretability early warning system based on the multi-modal large model is composed of a large language model module, a representation module, an adjustment and optimization module, an evaluation and verification module and a risk early warning module. The method comprises the following steps: S1, data acquisition and preprocessing; s2, data characterization; s3, establishing a large language model; s4, finely adjusting the system; s5, evaluating the verification system; s6, performing sepsis risk analysis, and outputting an interpretable prediction text; and S7, carrying out early warning on the sepsis risk. According to the method, medical workers can be efficiently assisted in timely and accurate early warning of sepsis of a patient, and interpretable diagnosis and treatment reference schemes are output.

Description

Technical Field

[0001] The present invention relates to an interpretable early warning system and method for sepsis based on a multimodal large model, belongs to the field of medical big data and artificial intelligence, and is particularly suitable for an interpretable early warning system for sepsis based on a multimodal large model. Background Art

[0002] Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection. Due to the uncertainty of the pathogenesis, the diversity of symptoms, and the complexity of treatment, accurate prediction and decision support have always been difficult. It is also one of the most challenging problems in the field of disease prediction and one of the global health priorities. Sepsis is acute and serious, and must be treated immediately when it occurs. Every hour of delay in treatment will increase the mortality rate by about 6%. However, in addition to general guidelines, there is no intelligent decision support tool for the best treatment of sepsis that can be explained. If the occurrence of sepsis can be predicted in advance, accurately and explainably, and the optimal and explainable treatment strategy based on patient-level intelligence is given, the risk of sepsis can be effectively reduced. Therefore, it is urgent to carry out key technical research on the best sepsis treatment decision support tool.

[0003] At present, the use of machine learning algorithms to predict and intervene in sepsis has achieved good results, but there are still two key bottlenecks that need to be overcome; first: the "black box" nature of existing sepsis prediction and intervention models leads to insufficient decision support capabilities, and the provided treatment intervention strategies are difficult to gain full trust from doctors and patients. Second: the lack of capture and modeling of intraoperative time series data makes it impossible to effectively extract nonlinear relationships and high-order features of sepsis monitoring data, so that the model cannot learn the time dependency in the data. With the development of large language models, their application potential in the medical field is huge. They can be used for diagnosis of critical illnesses (sepsis), risk prediction, personalized treatment, etc., and are very worthy of in-depth study.

[0004] Representation learning technology is a technology that maps data to a low-dimensional vector space, which can effectively retain the key information of the data. Representation learning is an important research direction in machine learning, and its goal is to learn high-quality representations of data. According to the goal of representation learning, the methods of representation learning can be divided into shallow representation learning and deep representation learning. Shallow representation learning uses simple models or rules to learn and transform data; deep representation learning is mainly based on deep learning to learn and represent data. However, the representation of intraoperative monitoring data based on traditional representation methods cannot effectively extract the nonlinearity and time dependency in the data. Therefore, it is urgent to explore a new representation method to generate patient texts to make interpretable sepsis predictions through large language model capabilities, which is expected to significantly improve the medical staff's credibility in the prediction results. Summary of the invention

[0005] In view of this, the present invention provides an interpretable early warning method for sepsis based on a multimodal large model, which aims to use the powerful ability of a large language model to analyze and process monitoring data, accurately warn patients of sepsis risks, and provide interpretable results. The patient's intraoperative monitoring data is converted into text form through a semantic symbolization method, and then represented in a word-embedded manner to facilitate the understanding and analysis of the large language model, improve the model's robustness, and enhance the model's interpretability. Finally, the large language model is fine-tuned and optimized accordingly to achieve an interpretable early warning of sepsis.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] The method for interpretable early warning of sepsis based on a multimodal large model is characterized in that it is implemented by an interpretable early warning system for sepsis based on a multimodal large model; the interpretable early warning system for sepsis based on a multimodal large model is composed of a large language model module, a representation module, a tuning module, an evaluation and verification module, and a risk warning module; the representation module is respectively connected to the input of the system and the large language model module; the large language model module is respectively connected to the tuning module and the risk warning module; the tuning module is connected to the evaluation and verification module; the risk warning module is connected to the system output; the input data types of the system are time series data, text data and image data.

[0008] Furthermore, the large language model module is a Med-PaLM or M4-Transformer large language model; the representation module includes: a piecewise approximate aggregation (PAA) module, a symbolic aggregate approximation (SAX) module, a semantic abstract rule engine, and an image recognition module; the tuning module includes a LoRA (Low-Rank Adaptation) technology module and a human feedback reinforcement learning (RLHF) module; the evaluation and verification module calculates the accuracy, recall rate, and F1 value of the large language model, and verifies whether the large language model meets the performance requirements in terms of accuracy, timeliness, and practicality through comparative analysis; the risk warning module is a Transformer classifier.

[0009] Furthermore, the segmented aggregation approximation module is connected in series with the symbolic aggregation approximation module and then connected in parallel with the image recognition module, and then connected in series with the semantic abstraction rule engine; the inputs of the segmented aggregation approximation module and the image recognition module are the time series data and image data input by the system; the output readable text of the semantic abstraction rule engine is spliced ​​with the text data input by the system as the input of the large language model module.

[0010] Furthermore, the segmented aggregation approximation module is used to perform segmented averaging on time series data, simplify data representation, and retain important features; the symbolic aggregation approximation module is used to perform symbolization on the basis of PAA processing, further simplify data representation, and facilitate semantic abstraction; the semantic abstraction rule engine realizes the conversion from symbols to semantics based on the experience and medical knowledge of medical staff, converts symbolized data into readable text, and enhances the interpretability of the data; the image recognition module uses the YOLO network to perform target detection on image data and obtain the corresponding target label symbol.

[0011] Preferably, the time series data input into the multimodal large model-based sepsis interpretable early warning system can be drawn into an image as image data, and can be directly input into the large language model module for processing without symbolization, so as to improve the recognition accuracy of the time series data.

[0012] The method for interpretable early warning of sepsis based on a multimodal large model is characterized in that the specific steps of the method are as follows:

[0013] S1: Collect and preprocess patient data;

[0014] S2: The characterization module characterizes the processed patient data;

[0015] S3: Establish a large language model module for the interpretable early warning system of sepsis based on a multimodal large model;

[0016] S4: Use the tuning module to fine-tune the interpretable early warning system for sepsis based on the multimodal large model;

[0017] S5: The evaluation and verification module evaluates and verifies the interpretable early warning system for sepsis based on the multimodal large model;

[0018] S6: Use the tuned large language model module to perform sepsis risk analysis on patients and output interpretable prediction text;

[0019] S7: Use the risk warning module to classify the predicted text into two categories and issue warnings to patients at risk of sepsis.

[0020] Furthermore, the patient data described in step S1 includes: time series data, text data and image data; the preprocessing includes: data cleaning, data enhancement and normalization processing; the data enhancement is to perform image enhancement processing on the image data; and the normalization processing is to perform normalization processing on the time series data.

[0021] Preferably, the preprocessing described in step S1 also includes scoring the patient data using the SOFA scoring method to generate partial text data; the SOFA score ranges from 0 to 24 points and consists of 6 parts, each of which ranges from 0 to 4 points. The 6 parts are respiratory system (Respiratory), coagulation system (Coagulation), liver function (Liver), cardiovascular system (Cardiovascular), central nervous system (Central Nervous System), and renal function (Renal), and each part requires a number of individual sign monitoring data to calculate a score.

[0022] Further, step S2 is specifically as follows:

[0023] S201: For the normalized patient time series data, assume that the time series data at N moments is C = {x1, x2, ...x N}, the w-dimensional segmented time series data is obtained by using the segmented aggregation approximation module: in,

[0024] S202: Using the symbol aggregation approximation module to process the segmented time series data to obtain symbol time series data

[0025] S203: For the symbolic time series data, a one-to-one mapping relationship between symbols and medical-specific vocabulary is established according to the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine, and the semantic abstract rule engine is used to output a readable text of the time series data;

[0026] S204: performing target detection on the image data using an image recognition module to obtain coordinates of a corresponding detection frame, frame confidence, category confidence, and category label;

[0027] S205: For the output of S204, a one-to-one mapping relationship between the output and medical-specific vocabulary is established according to the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine, and the semantic abstract rule engine is used to output a readable text of the image data.

[0028] Preferably, the semantic abstraction rule engine is a Transformer network.

[0029] Furthermore, the step S202 is specifically as follows:

[0030] S2021: Initialize the corresponding α symbols {l1,…,l α};

[0031] S2022: Using α-1 points {z1, ..., z α-1} Divide the area under the standard normal distribution curve into α segments;

[0032] S2023: If but if but if but Among them, 2≤j≤α.

[0033] Further, step S4 is specifically as follows:

[0034] S401: Fine-tune the large language model using the LoRA technology module;

[0035] S402: Medical staff manually answer and rank the prediction results, intervention methods, and interpretability generated by the large language model;

[0036] S403: normalizing different ranking results as scalar reward values ​​for training;

[0037] S404: Combine the scalar reward value of the feedback and use the human feedback reinforcement learning module to tune the large language model.

[0038] Then, the present invention also includes an electronic device, the electronic device comprising:

[0039] at least one processor; and

[0040] a memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned multimodal large model-based interpretable early warning method for sepsis.

[0042] Finally, the present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the above-mentioned multimodal large model-based interpretable early warning method for sepsis when executed.

[0043] The beneficial effects of the present invention are as follows: the present invention provides an interpretable early warning method for sepsis based on a multimodal large model, which processes and characterizes diversified patient data, combines a large language model for analysis and prediction, and optimizes the large language model using tuning technology. It can efficiently assist medical workers in achieving timely and accurate early warning of sepsis in patients, and output interpretable diagnosis and treatment reference plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to make the purpose and technical solution of the present invention, the present invention provides the following drawings for explanation:

[0045] Figure 1 The figure is an architecture diagram of the interpretable early warning system for sepsis based on a multimodal large model. The solid arrows represent data transmission, and the dot-dash arrows represent training.

[0046] Figure 2 is a flow chart of the method of the present invention;

[0047] Figure 3 This is a flow chart of Embodiment 2 of the present invention;

[0048] Figure 4 The electrocardiogram PAA processing result in Example 2 of the present invention; original data is the collected patient heart rate data; PAA is the data after PAA processing;

[0049] Figure 5 The SAX processing result of the electrocardiogram in Example 2 of the present invention; original data is the collected heart rate data of the patient; SAX is the data after PAA processing; a, b, c, d are symbols;

[0050] Figure 6 The output readable text annotation diagram of the ECG semantic abstraction rule engine in Example 2 of the present invention; wherein original data is the collected patient heart rate data; SAX is the data processed by the SAX-based semantic abstraction rule engine;

[0051] Figure 7 This is a schematic diagram of fine-tuning a large language model in Embodiment 2 of the present invention;

[0052] Figure 8 It is a schematic diagram of the structure of an electronic device in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0053] Example 1: The data of existing sepsis-related patients are divided into three types: time series data, text data and image data. Specifically, the data includes:

[0054] [1]. Patient electronic medical record (EMR): includes the patient’s basic information, medical history, medication status, etc.

[0055] [2]. Monitor data: real-time collection of patients’ vital signs data such as heart rate, blood pressure, and blood oxygen saturation (SPO2).

[0056] [3]. Test results: laboratory test reports of blood, urine, etc.

[0057] [4]. Imaging data: such as X-rays, CT, MRI and other imaging data.

[0058] In order to solve the "black box" nature of sepsis prediction and intervention mechanisms and improve the accuracy and interpretability of decision-making, the present invention provides an "interpretable sepsis early warning system based on a multimodal large model" combined with Figure 1 The multimodal large model-based sepsis interpretable early warning system is composed of a large language model module (1), a representation module (2), a tuning module (3), an evaluation and verification module (4), and a risk early warning module (5); the representation module (2) is connected to the input of the system and the large language model module (1) respectively; the large language model module (1) is connected to the tuning module (3) and the risk early warning module (5) respectively; the tuning module (3) is connected to the evaluation and verification module (4); the risk early warning module (5) is connected to the system output; the input data types of the system are: time series data - monitor data, text data - patient electronic medical records (EMR) and test results, image data - imaging data.

[0059] Furthermore, the large language model module (1) is a Med-PaLM or M4-Transformer large language model; the representation module (2) includes: a piecewise approximate aggregation (PAA) module (21), a symbolic aggregate approximation (SAX) module (22), a semantic abstract rule engine (23), and an image recognition module (24); the tuning module (3) includes a LoRA (Low-Rank Adaptation) technology module (31) and a human feedback reinforcement learning (RLHF) module (32); the evaluation and verification module (4) is used to calculate the accuracy, recall rate, and F1 value of the large language model, and to verify whether the large language model meets the performance requirements in terms of accuracy, timeliness, and practicality through comparative analysis; the risk warning module (5) is a Transformer classifier.

[0060] Furthermore, the segmented aggregation approximation module (21) is connected in series with the symbolic aggregation approximation module (22), and then connected in parallel with the image recognition module (24), and then connected in series with the semantic abstraction rule engine (23); the inputs of the segmented aggregation approximation module (21) and the image recognition module (24) are the time series data and image data input by the system; the output readable text of the semantic abstraction rule engine (23) is spliced ​​with the text data input by the system as the input of the large language model module (1).

[0061] Furthermore, the segmented aggregation approximation module (21) is used to perform segmented averaging on the time series data, simplify the data representation, and retain important features; the symbolic aggregation approximation module (22) is used to perform symbolization on the basis of PAA processing, further simplify the data representation, and facilitate semantic abstraction; the semantic abstraction rule engine (23) realizes the conversion from symbols to semantics based on the experience and medical knowledge of medical personnel, converts the symbolized data into readable text, and enhances the interpretability of the data; the image recognition module (24) uses the YOLO network to perform target detection on the image data and obtain the corresponding target label symbol.

[0062] The interaction process corresponding to each module in the above system is as follows: Figure 2 As shown, specifically including:

[0063] S1: Collect and preprocess patient data;

[0064] S2: the characterization module (2) characterizes the processed patient data;

[0065] S3: Establish a large language model module for the interpretable early warning system of sepsis based on a multimodal large model (1);

[0066] S4: Use the tuning module (3) to fine-tune the interpretable early warning system for sepsis based on the multimodal large model;

[0067] S5: Evaluation and verification module (4) evaluates and verifies the interpretable early warning system for sepsis based on the multimodal large model;

[0068] S6: Use the tuned large language model module (1) to perform sepsis risk analysis on patients and output interpretable prediction text;

[0069] S7: Use the risk warning module (5) to classify the predicted text into two categories and issue warnings to patients at risk of sepsis.

[0070] The patient data described in step S1 includes: time series data, text data and image data; the preprocessing includes: data cleaning, data enhancement and normalization processing; the data enhancement is to perform image enhancement processing on the image data; the normalization processing is to perform normalization processing on the time series data.

[0071] Step S2 is specifically as follows:

[0072] S201: For the normalized patient time series data, assume that the time series data at N moments is C = {x1, x2, ...x N}, the w-dimensional segmented time series data is processed by the segmented aggregation approximation module (21) as follows: in,

[0073] S202: Using the symbol aggregation approximation module to process the segmented time series data 22 to obtain symbol time series data

[0074] S203: For the symbolic time series data, a one-to-one mapping relationship between the symbol and the medical vocabulary is established according to the experience and medical knowledge of the medical staff to establish a semantic abstract rule engine (23), and the semantic abstract rule engine (23) is used to output a readable text of the time series data;

[0075] S204: using the image recognition module (24) to perform target detection on the image data, and obtain the coordinates of the corresponding detection frame, frame confidence, category confidence and category label;

[0076] S205: For the output of S204, a one-to-one mapping relationship between the output and medical-specific vocabulary is established based on the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine (23), and the semantic abstract rule engine (23) is used to output a readable text of the image data.

[0077] The step S202 is specifically as follows:

[0078] S2021: Initialize the corresponding α symbols {l1,…,l α};

[0079] S2022: Using α-1 points {z1, ..., z α-1} Divide the area under the standard normal distribution curve into α segments;

[0080] S2023: If but if but if but Among them, 2≤j≤α.

[0081] Further, step S4 is specifically as follows:

[0082] S401: fine-tune the large language model using the LoRA technology module (31);

[0083] S402: Medical staff manually answer and rank the prediction results, intervention methods, and interpretability generated by the large language model;

[0084] S403: normalizing different ranking results as scalar reward values ​​for training;

[0085] S404: Combine the scalar reward value of the feedback and use the human feedback reinforcement learning module (32) to tune the large language model.

[0086] Example 2: In response to the scenario of Example 1, the present invention provides an “explainable early warning method for sepsis based on a multimodal large model”.

[0087] Combination Figure 3 The specific steps of this method are as follows:

[0088] S1: Collect and preprocess patient data;

[0089] S2: Characterize the processed patient data;

[0090] S3: Build and fine-tune a large language model of a sepsis interpretable early warning system based on a multimodal large model for prediction, intervention, and interpretability;

[0091] S4: Use the tuned large language model to perform sepsis risk analysis and validation in perioperative patients.

[0092] Further, the patient data described in step S1 includes: any combination of time series data, text data and image data; the preprocessing includes: data cleaning, data enhancement, missing data processing, feature extraction and standardization processing; the data enhancement is to perform image enhancement processing on the image data; data cleaning specifically includes: removing duplicate and erroneous data and filling missing values; standardization specifically includes: standardizing data from different sources for unified analysis; missing data processing specifically includes: processing missing data using interpolation, mean filling and other methods; feature extraction specifically includes: extracting features meaningful to sepsis monitoring, such as key physiological indicators, change trends, etc.

[0093] Further, step S2 is specifically as follows:

[0094] S201: For the normalized patient time series data, assume that the time series data at N moments is C = {x1, x2, ...x N}, perform segmented aggregation approximation processing, and obtain the w-dimensional segmented time series data as in,

[0095] S202: Perform symbol aggregation approximation processing on the segmented time series data to obtain symbol time series data

[0096] S203: For the symbolic time series data, a one-to-one mapping relationship between symbols and medical-specific vocabulary is established according to the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine, and the semantic abstract rule engine is used to output a readable text of the time series data;

[0097] S204: performing target detection on the image data using image recognition technology to obtain the coordinates of the corresponding detection box, box confidence, category confidence, and category label;

[0098] S205: For the output of S204, a one-to-one mapping relationship between the output and medical-specific vocabulary is established according to the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine, and the semantic abstract rule engine is used to output a readable text of the image data.

[0099] Here, taking the heart rate data of patients in the monitoring period as an example, the results of segmented aggregation approximation, symbolic aggregation approximation and semantic abstraction are respectively as follows: Figure 4 , Figure 5 , Figure 6 shown.

[0100] Specifically, for heart rate, this embodiment provides a general form of the rule, and the specific code is as follows, where: <pattern>is the SAX mode, <l>It's the level. <f>It is a feature, <mod>is a modifier for mode movement, <pm>It is a mode shift.

[0101] { <pattern>}＝{ <l> <f> <mod> <pm>}

[0102] <l>=[‘low','medium',’high’]

[0103] <f>=The values are shown in Input.

[0104] <mod>=[‘slowly','rapidly','upward','downward']

[0105] <pm>=['decreasing','increasing','steady','peak','varying']

[0106] Furthermore, the step S202 is specifically as follows:

[0107] S2021: Initialize the corresponding α symbols {l1,…,l α };

[0108] S2022: Using α-1 points {z1, ..., z α-1 } Divide the area under the standard normal distribution curve into α segments;

[0109] S2023: If but if but if but Among them, 2≤j≤α.

[0110] Further, combined with Figure 7 , step S3 is specifically as follows:

[0111] S301: Fine-tune the large language model using LoRA technology;

[0112] S302: Medical staff manually answer and rank the prediction results, intervention methods, and interpretability generated by the large language model;

[0113] S303: normalizing different ranking results as scalar reward values ​​for training;

[0114] S304: Combine the scalar reward value of the feedback and use human feedback reinforcement learning to tune the large language model.

[0115] Specifically, the analysis and verification of step S4 is achieved by outputting interpretable prediction text, and the specific functions that can be achieved are: perioperative sepsis early warning, perioperative sepsis intervention, disease prediction explanatory description, intelligent decision-making explanatory description and intervention counterfactual prediction.

[0116] Example 3: Based on the system of Example 1, the present invention provides an "improved interpretable early warning method for sepsis based on a multimodal large model". Figure 2 The specific steps of this method are as follows:

[0117] S1: Collect and preprocess patient data;

[0118] S2: The characterization module characterizes the processed patient data;

[0119] S3: Establish a large language model module for the interpretable early warning system of sepsis based on a multimodal large model;

[0120] S4: Use the tuning module to fine-tune the interpretable early warning system for sepsis based on the multimodal large model;

[0121] S5: The evaluation and verification module evaluates and verifies the interpretable early warning system for sepsis based on the multimodal large model;

[0122] S6: Use the tuned large language model module to perform sepsis risk analysis on patients and output interpretable prediction text;

[0123] S7: Use the risk warning module to classify the predicted text into two categories and issue warnings to patients at risk of sepsis.

[0124] The patient data described in step S1 includes: time series data, text data and image data; the preprocessing includes: data cleaning, data enhancement and normalization processing; the data enhancement is to perform image enhancement processing on the image data; the normalization processing is to perform normalization processing on the time series data.

[0125] The preprocessing described in step S1 also includes scoring the patient data using the SOFA scoring method to generate some text data; the SOFA score ranges from 0 to 24 points and consists of 6 parts, each of which ranges from 0 to 4 points. The 6 parts are respiratory system (Respiratory), coagulation system (Coagulation), liver function (Liver), cardiovascular system (Cardiovascular), central nervous system (Central Nervous System), and renal function (Renal), and each part requires a number of individual sign monitoring data to calculate a score.

[0126] Step S2 is specifically as follows:

[0127] S201: For the normalized patient time series data, assume that the time series data at N moments is C = {x1, x2, ...x N }, the w-dimensional segmented time series data is obtained by using the segmented aggregation approximation module: in,

[0128] S202: Using the symbol aggregation approximation module to process the segmented time series data to obtain symbol time series data

[0129] S203: For the symbolic time series data, a one-to-one mapping relationship between symbols and medical-specific vocabulary is established according to the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine, and the semantic abstract rule engine is used to output a readable text of the time series data;

[0130] S204: performing target detection on the image data using an image recognition module to obtain coordinates of a corresponding detection frame, frame confidence, category confidence, and category label;

[0131] S205: For the output of S204, a one-to-one mapping relationship between the output and medical-specific vocabulary is established according to the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine, and the semantic abstract rule engine is used to output a readable text of the image data.

[0132] The semantic abstract rule engine is a Transformer network.

[0133] The step S202 is specifically as follows:

[0134] S2021: Initialize the corresponding α symbols {l1,…,l α };

[0135] S2022: Using α-1 points {z1, ..., z α-1 } Divide the area under the standard normal distribution curve into α segments;

[0136] S2023: If but if but if but Among them, 2≤j≤α.

[0137] Step S4 is specifically as follows:

[0138] S401: Fine-tune the large language model using the LoRA technology module;

[0139] S402: Medical staff manually answer and rank the prediction results, intervention methods, and interpretability generated by the large language model;

[0140] S403: normalizing different ranking results as scalar reward values ​​for training;

[0141] S404: Combine the scalar reward value of the feedback and use the human feedback reinforcement learning module to tune the large language model.

[0142] Embodiment 4: For the scenario of embodiment 1, Figure 8 A schematic diagram of an electronic device (60) that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0143] Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0144] like Figure 8 As shown, the electronic device (60) includes at least one processor (61), and a memory connected to the at least one processor (61) in communication, such as a read-only memory (ROM) (62), a random access memory (RAM) (63), etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor (61) can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) (62) or the computer program loaded from the storage unit (68) to the random access memory (RAM) (63). In RAM 43, various programs and data required for the operation of the electronic device (60) can also be stored. The processor (61), ROM 42 and RAM 43 are connected to each other via a bus (64). An input / output (I / O) interface (65) is also connected to the bus (64).

[0145] A number of components in the electronic device (60) are connected to the I / O interface (65), including: an input unit (66), such as a keyboard, a mouse, etc.; an output unit (67), such as various types of displays, speakers, etc.; a storage unit (68), such as a disk, an optical disk, etc.; and a communication unit (69), such as a network card, a modem, a wireless communication transceiver, etc. The communication unit (69) allows the electronic device (60) to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0146] The processor (61) may be any general and / or dedicated processing component with processing and computing capabilities. Some examples of the processor (61) include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor (61) executes the various methods and processes described above, such as a method for processing the harmfulness of a charging device.

[0147] In some embodiments, the hazardousness handling method of the charging device can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit (68). In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device (60) via the ROM (62) and / or the communication unit (69). When the computer program is loaded into the RAM (63) and executed by the processor (61), one or more steps of the hazardousness handling method of the charging device described above can be performed. Alternatively, in other embodiments, the processor (61) can be configured to execute the hazardousness handling method of the charging device in any other appropriate manner (for example, by means of firmware).

[0148] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0150] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0151] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0152] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0153] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0154] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.< / pm> < / mod> < / f> < / l> < / pm> < / mod> < / f> < / l> < / pattern> < / pm> < / mod> < / f> < / l> < / pattern>

Claims

1. An explainable early warning system for sepsis based on a multimodal large model, characterized by: It consists of a large language model module, a representation module, a tuning module, an evaluation and verification module, and a risk warning module; the representation module is respectively connected to the system input and the large language model module; the large language model module is respectively connected to the tuning module and the risk warning module; the tuning module is connected to the evaluation and verification module; the risk warning module is connected to the system output; the input data type of the system is time series data, text data and image data; the large language model module is a Med-PaLM or M4-Transformer large language model.

2. The interpretable early warning system for sepsis based on a multimodal large model according to claim 1, characterized in that: The representation module includes: Piecewise Approximate Aggregation (PAA) module, Symbolic Aggregate Approximation (SAX) module, semantic abstract rule engine, and image recognition module; the tuning module includes LoRA (Low-Rank Adaptation) technology module and human feedback reinforcement learning (RLHF) module; the evaluation and verification module calculates the accuracy, recall rate, and F1 value of the large language model, and verifies whether the large language model meets the performance requirements in terms of accuracy, timeliness, and practicality through comparative analysis; the risk warning module is a Transformer classifier.

3. The interpretable early warning system for sepsis based on a multimodal large model according to claim 2, characterized in that: The segmented aggregation approximation module is connected in series with the symbolic aggregation approximation module, and then connected in parallel with the image recognition module, and then connected in series with the semantic abstraction rule engine; the inputs of the segmented aggregation approximation module and the image recognition module are the time series data and image data input by the system; the output readable text of the semantic abstraction rule engine is spliced ​​with the text data input by the system as the input of the large language model module.

4. The interpretable early warning system for sepsis based on a multimodal large model according to claim 2, characterized in that: The segmented aggregation approximation module is used to perform segmented averaging on time series data, simplify data representation, and retain important features; the symbolic aggregation approximation module is used to perform symbolization on the basis of PAA processing, further simplify data representation, and facilitate semantic abstraction; the semantic abstraction rule engine realizes the conversion from symbols to semantics based on the experience and medical knowledge of medical staff, converts symbolized data into readable text, and enhances the interpretability of data; the image recognition module uses the YOLO network to perform target detection on image data and obtain the corresponding target label symbol.

5. The interpretable early warning system for sepsis based on a multimodal large model according to claim 1, characterized in that: The time series data input into the multimodal large model-based sepsis interpretable early warning system can be drawn into an image as image data, and can be directly input into the large language model module for processing without symbolization, so as to improve the recognition accuracy of the time series data.

6. The method for interpretable early warning of sepsis based on a multimodal large model as described in any one of claims 1 to 5, characterized in that: The specific steps of this method are as follows: S1: Collect and preprocess patient data; S2: The characterization module characterizes the processed patient data; S3: Establish a large language model module for the interpretable early warning system of sepsis based on a multimodal large model; S4: Use the tuning module to fine-tune the interpretable early warning system for sepsis based on the multimodal large model; S5: The evaluation and verification module evaluates and verifies the interpretable early warning system for sepsis based on the multimodal large model; S6: Use the tuned large language model module to perform sepsis risk analysis on patients and output interpretable prediction text; S7: Use the risk warning module to classify the predicted text into two categories and warn patients at risk of sepsis; The patient data described in step S1 includes: time series data, text data and image data; the preprocessing includes: data cleaning, data enhancement and normalization processing; the data enhancement includes performing image enhancement processing on the image data; the normalization processing includes performing normalization processing on the time series data; Step S2 is specifically as follows: S201: For the normalized patient time series data, assume that the time series data at N moments is C = {x1, x2, ...x N }, the w-dimensional segmented time series data is obtained by using the segmented aggregation approximation module: in, S202: Using the symbol aggregation approximation module to process the segmented time series data to obtain symbol time series data S203: For the symbolic time series data, a one-to-one mapping relationship between symbols and medical-specific vocabulary is established according to the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine, and the semantic abstract rule engine is used to output a readable text of the time series data; S204: performing target detection on the image data using an image recognition module to obtain coordinates of a corresponding detection frame, frame confidence, category confidence, and category label; S205: For the output of S204, a one-to-one mapping relationship between the output and medical-specific vocabulary is established according to the experience and medical knowledge of medical personnel to establish a semantic abstract rule engine, and the semantic abstract rule engine is used to output a readable text of the image data; The step S202 is specifically as follows: S2021: Initialize the corresponding ɑ symbols {l1,…,l α }; S2022: Using ɑ-1 points {z1,…,z α-1 } Divide the area under the standard normal distribution curve into α segments; S2023: If but if but if but Among them, 2≤j≤ɑ; The step S4 is specifically as follows: S401: Fine-tune the large language model using the LoRA technology module; S402: Medical staff manually answer and rank the prediction results, intervention methods, and interpretability generated by the large language model; S403: normalizing different ranking results as scalar reward values ​​for training; S404: Combine the scalar reward value of the feedback and use the human feedback reinforcement learning module to tune the large language model.

7. The method for interpretable early warning of sepsis based on a multimodal large model according to claim 6, characterized in that: The preprocessing described in step S1 also includes scoring the patient data using the SOFA scoring method to generate some text data; the SOFA score ranges from 0 to 24 points and consists of 6 parts, each of which ranges from 0 to 4 points. The 6 parts are respiratory system (Respiratory), coagulation system (Coagulation), liver function (Liver), cardiovascular system (Cardiovascular), central nervous system (Central Nervous System), and renal function (Renal), and each part requires a number of individual sign monitoring data to calculate a score.

8. The method for interpretable early warning of sepsis based on a multimodal large model according to claim 6, characterized in that: The semantic abstract rule engine is a Transformer network.

9. An electronic device, characterized in that: The electronic device comprises: At least one processor: and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the interpretable early warning method for sepsis based on a multimodal large model according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the interpretable early warning method for sepsis based on a multimodal large model according to any one of claims 6 to 8 when executed.

Images