Sepsis explainability early warning system based on multi-modal large model

Abstract

The application is a sepsis explainable early warning system and method based on a multi-modal large model, belonging to the field of medical big data and artificial intelligence. The sepsis explainable early warning system based on a multi-modal large model is realized by the system, which is composed of a large language model module, a representation module, a tuning module, an evaluation and verification module, and a risk early warning module. The method steps are as follows: S1: data acquisition and preprocessing; S2: data representation; S3: establishing a large language model; S4: fine-tuning the system; S5: evaluating and verifying the system; S6: analyzing the risk of sepsis and outputting explainable prediction text; and S7: early warning of the risk of sepsis. The method can efficiently assist medical workers to realize timely and accurate early warning of sepsis in patients, and output explainable diagnosis and treatment reference schemes.

Description

Technical Field

[0001] This invention relates to a sepsis interpretability early warning system and method based on a multimodal large model, belonging to the fields of medical big data and artificial intelligence, and is particularly applicable to sepsis interpretability early warning based on a multimodal large model. Background Technology

[0002] Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection. Due to the uncertainty of its pathogenesis, the diversity of its symptoms, and the complexity of its treatment, accurate prediction and decision support for sepsis have always been challenging issues and among the most difficult problems in disease prediction, making it a global health priority. Sepsis has a rapid onset and severe symptoms, requiring immediate treatment; every hour of delay increases the mortality rate by approximately 6%. However, apart from general guidelines, there are currently no interpretable intelligent decision support tools for optimal sepsis treatment. If sepsis can be predicted in advance with accurate and interpretable results, and optimal, interpretable treatment strategies can be provided intelligently at the patient level, the risk of sepsis can be effectively reduced. Therefore, research into key technologies for optimal sepsis treatment decision support tools is urgently needed.

[0003] Currently, machine learning algorithms have achieved considerable success in predicting and intervening in sepsis. However, two key bottlenecks remain to be overcome: First, the "black box" nature of existing sepsis prediction and intervention models results in insufficient decision support capabilities, making it difficult for the provided treatment intervention strategies to gain the full trust of doctors and patients. Second, the lack of capture and modeling of intraoperative time-series data prevents the effective extraction of nonlinear relationships and high-order features from sepsis monitoring data, thus hindering the model's ability to learn temporal dependencies within the data. With the development of large language models, their application potential in the medical field is enormous, applicable to the diagnosis, risk prediction, and personalized treatment of critical illnesses (such as sepsis), making them worthy of in-depth research.

[0004] Representation learning is a technique that maps data to a low-dimensional vector space, effectively preserving key information. It's an important research direction in machine learning, aiming to learn high-quality representations of data. Based on its goals and methods, representation learning can be divided into shallow representation learning and deep representation learning. Shallow representation learning uses simple models or rules to learn and transform data; deep representation learning primarily relies on deep learning to learn data representations. However, traditional representation methods for intraoperative monitoring data cannot effectively extract nonlinearities and temporal dependencies. Therefore, there is an urgent need to explore new representation methods, such as generating patient text and using large language models for interpretable sepsis prediction, which is expected to significantly improve the credibility of prediction results for medical personnel. Summary of the Invention

[0005] In view of this, this invention provides an interpretable early warning method for sepsis based on a multimodal large model. It aims to leverage the powerful capabilities of large language models to analyze and process monitoring data, accurately predicting the risk of sepsis in patients and providing interpretable results. The method transforms intraoperative monitoring data into text form using semantic symbolization, then represents it using word embedding, facilitating understanding and analysis by the large language model, improving model robustness, and enhancing interpretability. Finally, corresponding fine-tuning and optimization are performed on the large language model to achieve interpretable early warning for sepsis.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The sepsis interpretability early warning method based on a multimodal large model is characterized by being implemented by a sepsis interpretability early warning system based on a multimodal large model. This system comprises a large language model module, a representation module, an optimization module, an evaluation and verification module, and a risk early warning module. The representation module is connected to the system input and the large language model module. The large language model module is connected to the optimization module and the risk early warning module. The optimization module is connected to the evaluation and verification module. The risk early warning module is connected to the system output. The system input data types include time-series data, text data, and image data.

[0008] Furthermore, the large language model module is either Med-PaLM or M4-Transformer large language model; the representation module includes: Piecewise Approximate Aggregation (PAA) module, Symbolic Aggregate Approximation (SAX) module, semantic abstraction rule engine, and image recognition module; the optimization module includes LoRA (Low-Rank Adaptation) technology module and Reinforcement Learning from Human Feedback (RLHF) module; the evaluation and verification module calculates the accuracy, recall, and F1 score of the large language model, and verifies whether the large language model meets the performance requirements in terms of accuracy, timeliness, and practicality through comparative analysis; the risk warning module is a Transformer classifier.

[0009] Furthermore, the segmented aggregation approximation module and the symbolic aggregation approximation module are connected in series and then in parallel with the image recognition module, and then in series with the semantic abstraction rule engine; the inputs of the segmented aggregation approximation module and the image recognition module are the time-series data and image data input by the system; the output readable text of the semantic abstraction rule engine is concatenated with the text data input by the system and then used as the input of the large language model module.

[0010] Furthermore, the segmented aggregation approximation module is used to perform segmented averaging on time-series data, simplifying data representation and retaining important features; the symbolic aggregation approximation module performs symbolization based on PAA processing, further simplifying data representation and facilitating semantic abstraction; the semantic abstraction rule engine realizes the conversion from symbols to semantics based on medical staff experience and medical knowledge, converting symbolized data into readable text and enhancing data interpretability; the image recognition module uses the YOLO network to perform target detection on image data and obtain corresponding target label symbols.

[0011] Preferably, the time-series data input to the sepsis interpretability early warning system based on a multimodal large model can be plotted as an image and used as image data. Without symbolization, it can be directly input into the large language model module for processing to improve the recognition accuracy of the time-series data.

[0012] A sepsis interpretability early warning method based on a multimodal large model is characterized by the following specific steps:

[0013] S1: Collect and preprocess patient data;

[0014] S2: The characterization module characterizes the processed patient data;

[0015] S3: Establish a large language model module for a sepsis interpretability early warning system based on a multimodal large model;

[0016] S4: Fine-tune the sepsis interpretability early warning system based on a multimodal large model using the tuning module;

[0017] S5: The evaluation and validation module evaluates and validates the sepsis interpretability early warning system based on a multimodal large model;

[0018] S6: Use the optimized large language model module to perform sepsis risk analysis on patients and output interpretable prediction text;

[0019] S7: Use the risk warning module to perform binary classification on the predicted text and issue warnings to patients at risk of sepsis.

[0020] Furthermore, the patient data mentioned in step S1 includes: time-series data, text data, and image data; the preprocessing includes: data cleaning, data augmentation, and normalization; the data augmentation is image augmentation processing of the image data; and the normalization processing is normalization processing of the time-series data.

[0021] Preferably, the preprocessing in step S1 further includes scoring the patient data using the SOFA scoring method to generate partial text data; the SOFA score ranges from 0 to 24 points and consists of 6 parts, each ranging from 0 to 4 points. The 6 parts are respiratory system, coagulation system, liver function, cardiovascular system, central nervous system, and renal function. Each part requires several signs and symptoms monitoring data to calculate a score.

[0022] Furthermore, step S2 specifically includes:

[0023] S201: For normalized patient time series data, assume the time series data for N time points is C = {x1, x2, ... x...} N The piecewise aggregation approximation module is used to process the w-dimensional piecewise time series data to obtain the following: in,

[0024] S202: Obtain symbolic time-series data by processing segmented time-series data using a symbolic aggregation approximation module.

[0025] S203: For symbolic time-series data, establish a one-to-one mapping relationship between symbols and medical-specific terms based on the experience and medical knowledge of medical personnel, build a semantic abstraction rule engine, and use the semantic abstraction rule engine to output readable text of the time-series data;

[0026] S204: For image data, use the image recognition module to perform target detection on the image data, and obtain the coordinates of the corresponding detection box, box confidence, class confidence and class label;

[0027] S205: Based on the output of S204, establish a semantic abstraction rule engine by creating a one-to-one mapping relationship between medical personnel's experience and medical knowledge and medical-specific terms, and use the semantic abstraction rule engine to output readable text of the image data.

[0028] Preferably, the semantic abstraction rule engine is a Transformer network.

[0029] Furthermore, step S202 specifically includes:

[0030] S2021: Initialize and set the corresponding α symbols {l1, ..., l α};

[0031] S2022: Using α-1 points {z1, ..., z2} α-1 The area under the standard normal distribution curve is divided into equal segments α.

[0032] S2023: If but if but if but Where 2≤j≤α.

[0033] Furthermore, step S4 specifically includes:

[0034] S401: Fine-tuning of large language models using the LoRA technology module;

[0035] S402: Medical staff manually rank the prediction results, intervention methods, and interpretability generated by the large language model;

[0036] S403: Normalize the different ranking results and use them as the scalar reward value for training;

[0037] S404: Optimize the large language model by combining the scalar reward value of the feedback with the human feedback reinforcement learning module.

[0038] The invention further includes an electronic device comprising:

[0039] At least one processor; and

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the above-described sepsis interpretability early warning method based on a multimodal large model.

[0042] Finally, the present invention also discloses a computer-readable storage medium storing computer instructions for causing a processor to execute the above-described sepsis interpretability early warning method based on a multimodal large model.

[0043] The beneficial effects of this invention are as follows: This invention provides an interpretable early warning method for sepsis based on a multimodal large model. By processing and representing diverse patient data, combining analysis and prediction with a large language model, and optimizing the large language model using tuning techniques, it can efficiently assist medical workers in achieving timely and accurate early warning of sepsis in patients and output interpretable diagnostic and treatment reference plans. Attached Figure Description

[0044] To illustrate the objectives and technical solutions of this invention, the following figures are provided:

[0045] Figure 1 This is an architecture diagram of a sepsis interpretability early warning system based on a multimodal large model; solid arrows represent transmitted data, and dotted-dash arrows represent training.

[0046] Figure 2 This is a flowchart of the method of the present invention;

[0047] Figure 3 This is a flowchart of Embodiment 2 of the present invention;

[0048] Figure 4 This is the ECG PAA processing result of Embodiment 2 of the present invention; wherein, original data is the collected patient heart rate data; PAA is the data after PAA processing;

[0049] Figure 5 This is the SAX processing result of the electrocardiogram in Embodiment 2 of the present invention; where original data is the collected patient heart rate data; SAX is the data after PAA processing; a, b, c, and d are symbols;

[0050] Figure 6 This is a readable text annotation diagram of the output of the electrocardiogram semantic abstraction rule engine in Embodiment 2 of the present invention; wherein, original data is the collected patient heart rate data; SAX is the data processed by the semantic abstraction rule engine based on SAX;

[0051] Figure 7 This is a schematic diagram of the fine-tuning of the large language model in Embodiment 2 of the present invention;

[0052] Figure 8 This is a schematic diagram of the electronic device in Embodiment 4 of the present invention. Detailed Implementation

[0053] Example 1: Existing sepsis-related patient data, categorized into three types: time-series data, text data, and image data. Specifically, the data includes:

[0054] [1]. Patient electronic medical record (EMR): includes the patient's basic information, medical history, medication information, etc.

[0055] [2]. Monitor data: Real-time collection of vital signs data such as heart rate, blood pressure, and blood oxygen saturation (SpO2) of patients.

[0056] [3]. Test results: Laboratory test reports for blood, urine, etc.

[0057] [4]. Imaging data: such as X-ray, CT, MRI and other imaging data.

[0058] To address the "black box" nature of sepsis prediction and intervention mechanisms and improve the accuracy and interpretability of decision-making, this invention provides a "Sepsis Interpretability Early Warning System Based on a Multimodal Large Model," combining... Figure 1 The sepsis interpretability early warning system based on a multimodal large model consists of a large language model module (1), a representation module (2), an optimization module (3), an evaluation and verification module (4), and a risk warning module (5). The representation module (2) is connected to the system input and the large language model module (1). The large language model module (1) is connected to the optimization module (3) and the risk warning module (5). The optimization module (3) is connected to the evaluation and verification module (4). The risk warning module (5) is connected to the system output. The data types of the system input are: time series data - monitor data, text data - patient electronic medical records (EMR) and test results, and image data - imaging data.

[0059] Furthermore, the large language model module (1) is a Med-PaLM or M4-Transformer large language model; the representation module (2) includes: a Piecewise Approximate Aggregation (PAA) module (21), a Symbolic Aggregate Approximation (SAX) module (22), a semantic abstraction rule engine (23), and an image recognition module (24); the optimization module (3) includes a LoRA (Low-Rank Adaptation) technology module (31) and a Reinforcement Learning from Human Feedback (RLHF) module (32); the evaluation and verification module (4) calculates the accuracy, recall, and F1 score of the large language model, and verifies whether the large language model meets the performance requirements in terms of accuracy, timeliness, and practicality through comparative analysis; the risk warning module (5) is a Transformer classifier.

[0060] Furthermore, the segmented aggregation approximation module (21) and the symbolic aggregation approximation module (22) are connected in series and then connected in parallel with the image recognition module (24), and then connected in series with the semantic abstraction rule engine (23); the inputs of the segmented aggregation approximation module (21) and the image recognition module (24) are the time-series data and image data input by the system; the output readable text of the semantic abstraction rule engine (23) is concatenated with the text data input by the system and then used as the input of the large language model module (1).

[0061] Furthermore, the segmented aggregation approximation module (21) is used to perform segmented averaging on time-series data, simplifying data representation and retaining important features; the symbolic aggregation approximation module (22) is used to symbolize data based on PAA processing, further simplifying data representation and facilitating semantic abstraction; the semantic abstraction rule engine (23) realizes the conversion from symbol to semantic based on the experience and medical knowledge of medical personnel, converting symbolic data into readable text and enhancing the interpretability of data; the image recognition module (24) uses the YOLO network to perform target detection on image data and obtain the corresponding target label symbol.

[0062] The interaction flow corresponding to each module in the above system is as follows: Figure 2 As shown, the specific items include:

[0063] S1: Collect and preprocess patient data;

[0064] S2: The characterization module (2) characterizes the processed patient data;

[0065] S3: Establish a large language model module for a sepsis interpretability early warning system based on a multimodal large model (1);

[0066] S4: Fine-tune the sepsis interpretability early warning system based on a multimodal large model using the tuning module (3);

[0067] S5: Evaluation and verification module (4) evaluates and verifies the sepsis interpretability early warning system based on a multimodal large model;

[0068] S6: Use the optimized large language model module (1) to perform sepsis risk analysis on patients and output interpretable prediction text;

[0069] S7: Use the risk warning module (5) to perform binary classification on the predicted text and issue warnings to patients at risk of sepsis.

[0070] The patient data mentioned in step S1 includes: time-series data, text data, and image data; the preprocessing includes: data cleaning, data augmentation, and normalization; the data augmentation involves image enhancement processing of the image data; and the normalization processing involves normalizing the time-series data.

[0071] Step S2 is as follows:

[0072] S201: For normalized patient time series data, assume the time series data for N time points is C = {x1, x2, ... x...} N The piecewise aggregation approximation module (21) is used to process the w-dimensional piecewise time series data to obtain the following: in,

[0073] S202: The symbolic time-series data is processed using the symbolic aggregation approximation module to obtain symbolic time-series data.

[0074] S203: For symbolic time series data, establish a one-to-one mapping relationship between symbols and medical terms based on the experience and medical knowledge of medical personnel to establish a semantic abstraction rule engine (23), and use the semantic abstraction rule engine (23) to output readable text of time series data;

[0075] S204: For image data, the image recognition module (24) is used to perform target detection on the image data to obtain the coordinates of the corresponding detection box, the box confidence, the category confidence and the category label;

[0076] S205: Based on the output of S204, establish a one-to-one mapping relationship between medical personnel's experience and medical knowledge and medical-specific terms to build a semantic abstraction rule engine (23), and use the semantic abstraction rule engine (23) to output readable text of image data.

[0077] The specific steps of S202 are as follows:

[0078] S2021: Initialize and set the corresponding α symbols {l1, ..., l α};

[0079] S2022: Using α-1 points {z1, ..., z2} α-1 The area under the standard normal distribution curve is divided into equal segments α.

[0080] S2023: If but if but if but Where 2≤j≤α.

[0081] Furthermore, step S4 specifically includes:

[0082] S401: Fine-tune the large language model using the LoRA technology module (31);

[0083] S402: Medical staff manually rank the prediction results, intervention methods, and interpretability generated by the large language model;

[0084] S403: Normalize the different ranking results and use them as the scalar reward value for training;

[0085] S404: Combine the scalar reward value of the feedback and use the human feedback reinforcement learning module (32) to optimize the large language model.

[0086] Example 2: For the scenario in Example 1, the present invention provides a "sepsis interpretability early warning method based on a multimodal large model".

[0087] Combination Figure 3 The specific steps of this method are as follows:

[0088] S1: Collect and preprocess patient data;

[0089] S2: Characterize the processed patient data;

[0090] S3: Establish large language models for a sepsis interpretability early warning system based on a multimodal large model for prediction, intervention, and interpretability, and fine-tune them;

[0091] S4: Use the optimized large language model to conduct sepsis risk analysis and validation for perioperative patients.

[0092] Furthermore, the patient data mentioned in step S1 includes any combination of time-series data, text data, and image data; the preprocessing includes data cleaning, data augmentation, missing data handling, feature extraction, and standardization; the data augmentation involves image enhancement processing of the image data; data cleaning specifically involves removing duplicate and erroneous data and filling in missing values; standardization specifically involves standardizing data from different sources to facilitate unified analysis; missing data handling specifically involves using interpolation, mean imputation, and other methods to process missing data; feature extraction specifically involves extracting features meaningful for sepsis monitoring, such as key physiological indicators and trends.

[0093] Furthermore, step S2 specifically includes:

[0094] S201: For normalized patient time series data, assume the time series data for N time points is C = {x1, x2, ... x...} NAfter performing segmented aggregation approximation processing, we obtain w-dimensional segmented time series data. in,

[0095] S202: Perform symbol aggregation approximation processing on the segmented time series data to obtain symbolic time series data.

[0096] S203: For symbolic time-series data, establish a one-to-one mapping relationship between symbols and medical-specific terms based on the experience and medical knowledge of medical personnel, build a semantic abstraction rule engine, and use the semantic abstraction rule engine to output readable text of the time-series data;

[0097] S204: For image data, use image recognition technology to perform target detection on the image data, and obtain the coordinates of the corresponding detection box, the box confidence, the class confidence, and the class label;

[0098] S205: Based on the output of S204, establish a semantic abstraction rule engine by creating a one-to-one mapping relationship between medical personnel's experience and medical knowledge and medical-specific terms, and use the semantic abstraction rule engine to output readable text of the image data.

[0099] Here, taking the patient's heart rate data during the monitoring period as an example, the results obtained after performing segmented aggregation approximation, symbolic aggregation approximation, and semantic abstraction are as follows: Figure 4 , Figure 5 , Figure 6 As shown.

[0100] Specifically, regarding heart rate, this embodiment provides the general form of the rule, and the specific code is as follows: <pattern>It is SAX mode. <l>It's a level. <f>It is a characteristic. <mod>It is a modifier for pattern shifting. <pm>It's a mode shift.

[0101] { <pattern>}＝{ <l> <f> <mod> <pm>}

[0102] <l>=[‘low','medium',’high’]

[0103] <f>=The values are shown in Input.

[0104] <mod>=[‘slowly','rapidly','upward','downward']

[0105] <pm>=['decreasing','increasing','steady','peak','varying']

[0106] Furthermore, step S202 specifically includes:

[0107] S2021: Initialize and set the corresponding α symbols {l1, ..., l α };

[0108] S2022: Using α-1 points {z1, ..., z2} α-1 The area under the standard normal distribution curve is divided into equal segments α.

[0109] S2023: If but if but if but Where 2≤j≤α.

[0110] Furthermore, combined with Figure 7 Step S3 is as follows:

[0111] S301: Fine-tuning large language models using LoRA technology;

[0112] S302: Medical staff manually rank the prediction results, intervention methods, and interpretability generated by the large language model;

[0113] S303: Normalize the different ranking results and use them as the scalar reward value for training;

[0114] S304: Optimize large language models by combining scalar reward values ​​from feedback with reinforcement learning based on human feedback.

[0115] Specifically, the analysis and verification in step S4 are achieved by outputting interpretable predictive text, which can specifically achieve the following functions: perioperative sepsis early warning, perioperative sepsis intervention, disease prediction explanatory explanation, intelligent decision-making explanatory explanation, and intervention counterfactual prediction.

[0116] Example 3: Based on the system of Example 1, this invention provides an "improved method for interpretable early warning of sepsis based on a multimodal large model." Combined with... Figure 2 The specific steps of this method are as follows:

[0117] S1: Collect and preprocess patient data;

[0118] S2: The characterization module characterizes the processed patient data;

[0119] S3: Establish a large language model module for a sepsis interpretability early warning system based on a multimodal large model;

[0120] S4: Fine-tune the sepsis interpretability early warning system based on a multimodal large model using the tuning module;

[0121] S5: The evaluation and validation module evaluates and validates the sepsis interpretability early warning system based on a multimodal large model;

[0122] S6: Use the optimized large language model module to perform sepsis risk analysis on patients and output interpretable prediction text;

[0123] S7: Use the risk warning module to perform binary classification on the predicted text and issue warnings to patients at risk of sepsis.

[0124] The patient data mentioned in step S1 includes: time-series data, text data, and image data; the preprocessing includes: data cleaning, data augmentation, and normalization; the data augmentation involves image enhancement processing of the image data; and the normalization processing involves normalizing the time-series data.

[0125] The preprocessing described in step S1 also includes scoring patient data using the SOFA scoring method to generate partial text data; the SOFA score ranges from 0 to 24 points and consists of 6 parts, each ranging from 0 to 4 points. The 6 parts are respiratory system, coagulation system, liver function, cardiovascular system, central nervous system, and renal function. Each part requires several signs and symptoms monitoring data to calculate a score.

[0126] Step S2 is as follows:

[0127] S201: For normalized patient time series data, assume the time series data for N time points is C = {x1, x2, ... x...} N The piecewise aggregation approximation module is used to process the w-dimensional piecewise time series data to obtain the following: in,

[0128] S202: Obtain symbolic time-series data by processing segmented time-series data using a symbolic aggregation approximation module.

[0129] S203: For symbolic time-series data, establish a one-to-one mapping relationship between symbols and medical-specific terms based on the experience and medical knowledge of medical personnel, build a semantic abstraction rule engine, and use the semantic abstraction rule engine to output readable text of the time-series data;

[0130] S204: For image data, use the image recognition module to perform target detection on the image data, and obtain the coordinates of the corresponding detection box, box confidence, class confidence and class label;

[0131] S205: Based on the output of S204, establish a semantic abstraction rule engine by creating a one-to-one mapping relationship between medical personnel's experience and medical knowledge and medical-specific terms, and use the semantic abstraction rule engine to output readable text of the image data.

[0132] The semantic abstraction rule engine is the Transformer network.

[0133] The specific steps of S202 are as follows:

[0134] S2021: Initialize and set the corresponding α symbols {l1, ..., l α };

[0135] S2022: Using α-1 points {z1, ..., z2} α-1 The area under the standard normal distribution curve is divided into equal segments α.

[0136] S2023: If but if but if but Where 2≤j≤α.

[0137] Step S4 is as follows:

[0138] S401: Fine-tuning of large language models using the LoRA technology module;

[0139] S402: Medical staff manually rank the prediction results, intervention methods, and interpretability generated by the large language model;

[0140] S403: Normalize the different ranking results and use them as the scalar reward value for training;

[0141] S404: Optimize the large language model by combining the scalar reward value of the feedback with the human feedback reinforcement learning module.

[0142] Example 4: For the scenario in Example 1, Figure 8 A schematic diagram of an electronic device (60) that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0143] Electronic devices can also refer to various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0144] like Figure 8 As shown, the electronic device (60) includes at least one processor (61) and a memory, such as a read-only memory (ROM) (62) or a random access memory (RAM) (63), which is communicatively connected to the at least one processor (61). The memory stores computer programs executable by the at least one processor. The processor (61) can perform various appropriate actions and processes based on the computer programs stored in the ROM (62) or loaded from storage unit (68) into the RAM (63). The RAM (43) can also store various programs and data required for the operation of the electronic device (60). The processor (61), ROM (42), and RAM (43) are interconnected via a bus (64). An input / output (I / O) interface (65) is also connected to the bus (64).

[0145] Multiple components in the electronic device (60) are connected to an I / O interface (65), including: input units (66), such as a keyboard, mouse, etc.; output units (67), such as various types of displays, speakers, etc.; storage units (68), such as disks, optical disks, etc.; and communication units (69), such as network cards, modems, wireless transceivers, etc. The communication unit (69) allows the electronic device (60) to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0146] The processor (61) can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processors (61) include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor (61) performs the various methods and processes described above, such as hazardous handling methods for charging devices.

[0147] In some embodiments, the hazardous handling method of the charging device may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit (68). In some embodiments, part or all of the computer program may be loaded into and / or installed on an electronic device (60) via a ROM (62) and / or a communication unit (69). When the computer program is loaded into RAM (63) and executed by a processor (61), one or more steps of the hazardous handling method of the charging device described above may be performed. Alternatively, in other embodiments, the processor (61) may be configured to perform the hazardous handling method of the charging device by any other suitable means (e.g., by means of firmware).

[0148] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0150] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0151] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0152] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0153] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0154] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.< / pm> < / mod> < / f> < / l> < / pm> < / mod> < / f> < / l> < / pattern> < / pm> < / mod> < / f> < / l> < / pattern>

Claims

1. A sepsis interpretability early warning system based on a multimodal large model, characterized in that, It consists of a large language model module, a representation module, an optimization module, an evaluation and verification module, and a risk warning module; the representation module is connected to the system's input and the large language model module, respectively. The large language model module is connected to the tuning module and the risk warning module respectively; the tuning module is connected to the evaluation and verification module; the risk warning module is connected to the system output; the data types of the system input are time series data, text data and image data; the large language model module is a Med-PaLM or M4-Transformer large language model. The representation module includes: a Piecewise Approximate Aggregation (PAA) module, a Symbolic Aggregate Approximation (SAX) module, a semantic abstraction rule engine, and an image recognition module; the optimization module includes a LoRA (Low-Rank Adaptation) technology module and a Reinforcement Learning from Human Feedback (RLHF) module; the evaluation and verification module calculates the accuracy, recall, and F1 score of the large language model, and verifies whether the large language model meets the performance requirements in terms of accuracy, timeliness, and practicality through comparative analysis; the risk warning module uses a Transformer classifier; The segmented aggregation approximation module and the symbolic aggregation approximation module are connected in series and then in parallel with the image recognition module, and then in series with the semantic abstraction rule engine; the inputs of the segmented aggregation approximation module and the image recognition module are the time-series data and image data input by the system; the output readable text of the semantic abstraction rule engine is concatenated with the text data input by the system and then used as the input of the large language model module; The segmented aggregation approximation module is used to perform segmented averaging on time-series data, simplifying data representation and retaining important features; the symbolic aggregation approximation module symbolizes data based on PAA processing, further simplifying data representation and facilitating semantic abstraction; the semantic abstraction rule engine converts symbols into semantics based on medical staff experience and medical knowledge, transforming symbolic data into readable text and enhancing data interpretability; the image recognition module uses the YOLO network to perform target detection on image data and obtain corresponding target label symbols. The time-series data input to the sepsis interpretability early warning system based on a multimodal large model is plotted as an image. As image data, it does not need to be symbolized and is directly input into the large language model module for processing to improve the recognition accuracy of time-series data.

2. A sepsis interpretability early warning method based on a multimodal large model, applied to the sepsis interpretability early warning system based on a multimodal large model as described in claim 1, characterized in that, The specific steps of this method are as follows: S1: Collect and preprocess patient data; S2: The characterization module characterizes the processed patient data; S3: Establish a large language model module for a sepsis interpretability early warning system based on a multimodal large model; S4: Fine-tune the sepsis interpretability early warning system based on a multimodal large model using the tuning module; S5: The evaluation and validation module evaluates and validates the sepsis interpretability early warning system based on a multimodal large model; S6: Use the optimized large language model module to perform sepsis risk analysis on patients and output interpretable prediction text; S7: Use the risk warning module to perform binary classification on the predicted text and issue warnings to patients at risk of sepsis; The patient data mentioned in step S1 includes: time-series data, text data, and image data; the preprocessing includes: data cleaning, data augmentation, and normalization; the data augmentation involves image augmentation of the image data; the normalization involves normalization of the time-series data. Step S2 is as follows: S201: For normalized patient time series data, assume the time series data at N time points is C = {x1, x2, ... x...} N The piecewise aggregation approximation module is used to process the w-dimensional piecewise time series data to obtain the following: in, S202: Obtain symbolic time-series data by processing segmented time-series data using a symbolic aggregation approximation module. S203: For symbolic time-series data, establish a one-to-one mapping relationship between symbols and medical-specific terms based on the experience and medical knowledge of medical personnel, build a semantic abstraction rule engine, and use the semantic abstraction rule engine to output readable text of the time-series data; S204: For image data, use the image recognition module to perform target detection on the image data, and obtain the coordinates of the corresponding detection box, box confidence, class confidence and class label; S205: Based on the output of S204, establish a one-to-one mapping relationship between medical personnel's experience and medical knowledge and medical-specific terms to build a semantic abstraction rule engine, and use the semantic abstraction rule engine to output readable text of image data. The specific steps of S202 are as follows: S2021: Initialize and set the corresponding α symbols {l1,…,lα}; S2022: Using α-1 points {z1,…,z…} α-1 The area under the standard normal distribution curve is divided into equal segments α. S2023: If but if but if but Where 2≤j≤α; The specific steps of S4 are as follows: S401: Fine-tuning of large language models using the LoRA technology module; S402: Medical staff manually rank the prediction results, intervention methods, and interpretability generated by the large language model; S403: Normalize the different ranking results and use them as the scalar reward value for training; S404: Optimize the large language model by combining the scalar reward value of the feedback with the human feedback reinforcement learning module.

3. The sepsis interpretability early warning method based on a multimodal large model according to claim 2, characterized in that, The preprocessing described in step S1 also includes scoring patient data using the SOFA scoring method to generate partial text data; the SOFA score ranges from 0 to 24 points and consists of 6 parts, each ranging from 0 to 4 points. The 6 parts are respiratory system, coagulation system, liver function, cardiovascular system, central nervous system, and renal function. Each part requires several signs and symptoms monitoring data to calculate a score.

4. The sepsis interpretability early warning method based on a multimodal large model according to claim 2, characterized in that, The semantic abstraction rule engine is the Transformer network.

5. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that is executed by the at least one processor to enable the at least one processor to perform the sepsis interpretability early warning method based on a multimodal large model as described in any one of claims 2-4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the sepsis interpretability early warning method based on a multimodal large model as described in any one of claims 2-4.

Images