ICU critical patient multi-mode risk early warning system based on deep learning

Abstract

The invention belongs to the technical field of artificial intelligence and medical health information processing, particularly relates to an ICU critical patient multi-mode risk early warning system based on deep learning, and aims to solve the problems that in intensive care, illness state prediction lags behind, multi-source data fusion is difficult, and single-index early warning precision is low. According to the system, high-frequency physiological signals, clinical observation values, inspection images and unstructured texts are collected through the data sensing layer, cross-modal embedding and comparative learning alignment are carried out through the heterogeneous fusion layer, and a shared semantic space is constructed; the time sequence reasoning layer utilizes an improved space-time diagram neural network to model a dynamic knowledge graph, and causal and time sequence dependence among clinical events is captured; and the risk decision-making layer outputs multi-dimensional risk probabilities of sepsis, respiratory failure and sudden cardiac arrest in parallel. The system supports online incremental learning and individualized time decay modeling, the timeliness, accuracy and interpretability of early warning are improved, and the method is obviously superior to a traditional method in clinical verification.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and medical health information processing technology, specifically relating to a deep learning-based multimodal risk early warning system for critically ill ICU patients. Background Technology

[0002] The deep integration of artificial intelligence (AI) technology into the healthcare field is driving the rapid development of clinical decision support systems towards intelligence and precision. As a crucial branch of AI, deep learning, with its powerful nonlinear mapping capabilities and multi-level feature extraction advantages, has been widely applied in various scenarios such as medical image analysis, electronic medical record mining, and physiological signal processing, significantly improving the efficiency and accuracy of disease prediction, diagnostic assistance, and treatment plan recommendations. Among these, multimodal data fusion, as a key pathway to achieving comprehensive patient status representation, aims to integrate heterogeneous information from different sources, such as vital sign monitoring data, laboratory test results, imaging reports, and nursing records, to construct a more complete and dynamic model of disease progression.

[0003] Among them, the Intensive Care Unit (ICU), as the most critically ill and rapidly changing clinical unit in a hospital, has extremely high requirements for the timeliness of early risk identification and intervention. In this environment, patients' physiological states exhibit highly nonlinear and complex coupling characteristics, making it difficult for single-modal data to accurately capture early warning signals of potential deterioration. Therefore, risk warning methods based on multimodal data have become a core research direction for improving the quality of ICU management. Its basic goal is to achieve early prediction of major clinical events such as sepsis, acute respiratory failure, and arrhythmias by modeling the spatiotemporal correlations between multiple sources of information.

[0004] While existing technologies have attempted to introduce deep neural networks into ICU risk warning tasks and have initially achieved trend prediction and anomaly detection for some physiological parameters, they still face a series of key technical bottlenecks: multimodal data generally suffers from inconsistent time scales, large differences in sampling frequencies, and uneven distribution of missing values, making it difficult for traditional fusion strategies to effectively align semantic information; most models use static weight allocation mechanisms for modality fusion, failing to dynamically adjust the contribution of each modality according to the patient's real-time status, thus weakening the model's adaptability in personalized warnings; simultaneously, deep network structures are susceptible to small sample sizes and class imbalance during training, resulting in limited generalization performance and a high false alarm rate; furthermore, existing systems lack interpretable output of warning results, making it difficult to provide clinicians with reliable decision-making support. These shortcomings are particularly prominent in the high-intensity, fast-paced clinical practice of ICUs, severely restricting the practical application of intelligent warning systems. Therefore, there is an urgent need for a new architecture that can achieve efficient, robust, and interpretable multimodal risk warning. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning-based multimodal risk early warning system for critically ill ICU patients, addressing the technical challenges of untimely prediction of patient deterioration, weak multi-source heterogeneous data fusion capabilities, and insufficient sensitivity and specificity of single-modal monitoring indicators in the current intensive care environment. In modern ICU clinical practice, although vital sign monitoring equipment is highly advanced, capable of real-time acquisition of physiological parameters such as heart rate, blood pressure, and blood oxygen saturation, and electronic medical record systems continuously record laboratory test results, imaging reports, and nursing documentation, existing early warning mechanisms still generally rely on single-parameter alarms triggered by fixed thresholds or early warning scoring systems based on simple scoring rules. Such methods struggle to capture the nonlinear dynamic characteristics of complex pathological evolution processes, easily generating numerous false positive alarms, leading to staff fatigue; more importantly, they lack the ability to perform deep semantic association and cross-domain collaborative modeling of multimodal medical information, failing to achieve early and accurate prediction of critical events such as sepsis, acute respiratory failure, and cardiac arrest.

[0006] The technical solution of this invention is to construct an end-to-end intelligent early warning system consisting of a data perception layer, a heterogeneous fusion layer, a temporal reasoning layer, and a risk decision-making layer. The data perception layer is responsible for parallel access to four types of core data streams from the hospital information system: the first type is high-frequency physiological signal streams, including ECG waveforms sampled per second, invasive arterial pressure curves, respiratory flow-volume loops, and bispectral index sequences of EEG; the second type is structured clinical observation values, covering hourly recorded body temperature, consciousness status scores, urine output, and mechanical ventilation parameters; the third type is semi-structured test and imaging reports, including textual descriptions and key conclusion labels of blood routine tests, coagulation function tests, procalcitonin test values, and chest X-rays; the fourth type is unstructured medical texts, involving physician ward round records, surgical records, and nursing handover notes. These four types of data streams exhibit significant misalignment on the time axis, and their sampling frequencies differ by three orders of magnitude, posing a risk of dimensionality curse and information distortion if directly fused.

[0007] The heterogeneous fusion layer employs a staged embedding strategy to achieve multimodal alignment. For high-frequency physiological signal streams, a set of parallel one-dimensional convolutional neural networks extracts local temporal features and utilizes a self-attention mechanism to capture long-range dependencies, generating a fixed-length temporal feature vector sequence. For structured clinical observations, after standardization, they are input into a fully connected network for nonlinear mapping, forming a low-dimensional dense representation. For semi-structured test and imaging reports, a pre-trained medical language model is used for semantic encoding, extracting key feature vectors containing pathological indicators. For unstructured medical texts, a domain-adapted bidirectional long short-term memory network combined with a named entity recognition module is used to extract key events such as symptoms, diagnoses, and treatment interventions, along with their timestamps. The preliminary representations of all modalities are uniformly projected into a shared high-dimensional semantic space, and their relative position distribution is optimized through cross-modal contrastive learning, allowing semantically similar heterogeneous information to cluster in the vector space, thereby establishing implicit correspondences between modalities.

[0008] The temporal inference layer performs dynamic evolutionary modeling based on an improved spatiotemporal graph neural network architecture. The timeline of each patient's ICU stay is divided into consecutive 2-hour analysis windows. All modal representations collected within each window serve as node inputs, constructing an individualized dynamic knowledge graph. Nodes in the graph represent different types of clinical events or physiological states, and the edge weights are determined by the temporal co-occurrence probability between two nodes and the causal strength in the medical prior knowledge base. At each time step, the graph neural network performs message passing operations, aggregating information from neighboring nodes to update its own state, and introducing a gating mechanism to control the rate of forgetting historical memories and accepting new information. Furthermore, a differentiable time decay function is integrated internally into the network, assigning exponentially decreasing weights to long-term observations to ensure the model remains highly sensitive to recent changes in the patient's condition. After multiple rounds of iterative propagation, a comprehensive health status embedding vector corresponding to each analysis window is output.

[0009] The risk decision layer receives state embedding vectors from the temporal inference layer and drives three dedicated prediction branches in parallel. The first branch targets the risk of sepsis onset within 48 hours, using a binary classifier with class imbalance compensation. The second branch addresses the risk of respiratory failure requiring intubation within 24 hours, estimating the cumulative risk function using a deep partial logistic regression model within a survival analysis framework. The third branch focuses on extreme events such as cardiac arrest within 12 hours, constructing a reconstruction error estimator based on anomaly detection principles to quantify the probability of a sudden crisis by comparing the deviation between the original input and the autoencoder reconstruction output. The risk probability values ​​output from each branch are integrated by a learnable weighted fusion unit to generate a final multi-dimensional early warning index. This index is divided into four levels: green (low risk), yellow (moderate concern), orange (high alert), and red (emergency intervention), and is pushed to the healthcare workstation in real time via a visual interface.

[0010] Preferably, in the cross-modal contrastive learning process, positive sample pairs are constructed as follows: different modal data items collected from the same patient within the same time window are considered positive examples; negative samples are selected from data combinations of different patients at the same absolute time point or with the same number of hospital stays but separated by more than 6 hours. The loss function uses normalized temperature-scaled cross-entropy to forcibly narrow the distance between positive sample pairs and widen the distance between negative sample pairs.

[0011] Preferably, the edge weights in the dynamic knowledge graph are initialized based on a causal relationship matrix constructed according to authoritative medical guidelines, including but not limited to 287 deterministic or probabilistic medical rules such as "elevated lactate → insufficient tissue perfusion", "PaO2 / FiO2 < 200 → increased likelihood of ARDS", and "new-onset atrial fibrillation + BNP > 500 → cardiac decompensation". During the model training phase, the edge weights are allowed to be fine-tuned through the backpropagation algorithm to adapt to the clinical practice patterns of specific medical institutions.

[0012] Preferably, the differentiable time decay function is in the form of an exponential decay kernel, and its decay coefficient is not a fixed constant, but is jointly regulated by the patient's age, underlying disease burden index and admission APACHE II score, so that the retention period of historical information of patients with chronic diseases is longer than that of younger patients, reflecting individualized differences in memory persistence.

[0013] Preferably, the deep partial logistic regression model under the survival analysis framework calculates the partial likelihood function gradient within the risk set at each time step and stabilizes the training process through gradient pruning and momentum optimization strategies; the model output not only includes the risk score, but also provides the importance ranking of key covariates to assist in clinical interpretation.

[0014] Preferably, the learnable weighted fusion unit employs a gated attention mechanism to automatically adjust the contribution ratio of the three prediction branches based on the current patient's chief diagnosis: for pneumonia patients, the respiratory failure branch has an increased weight; for patients with abdominal infection, the sepsis branch receives a higher priority; and for post-cardiac surgery patients, the influence of the cardiac arrest branch is strengthened.

[0015] Preferably, the system is deployed on a hospital private cloud platform, adopts a containerized microservice architecture, and each functional module communicates through well-defined API interfaces; data transmission uses the national cryptographic SM4 encryption algorithm throughout, and static data storage adopts transparent database encryption technology to ensure that patient privacy complies with the requirements of the Personal Information Protection Law and the Measures for the Administration of Network Security of Medical and Health Institutions.

[0016] Preferably, in addition to displaying multi-dimensional warning indices, the visualization interface simultaneously presents key evidence chains supporting the judgment, including the most discriminative physiological waveform segments, abnormal laboratory numerical trend graphs, and danger signal keywords in text records, helping doctors quickly verify the rationality of the system's inference.

[0017] Preferably, the system has online incremental learning capabilities. When the complete diagnosis and treatment path of a newly confirmed case is backfilled into the database, a small-batch retraining process is automatically triggered to periodically update the model parameters, so that the early warning performance can continuously evolve over time.

[0018] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0019] This system fundamentally changes the limitations of traditional ICU early warning systems that rely on static thresholds and isolated indicators. For the first time, it achieves deep fusion and joint reasoning of four types of heterogeneous data: high-frequency physiological signals, structured observations, laboratory images, and free text. By constructing a dynamic knowledge graph and a temporal graph neural network, it can explicitly model complex causal and temporal dependencies between clinical events, significantly improving the ability to identify latent disease deterioration. A cross-modal contrastive learning strategy effectively solves the semantic gap problem of multi-source data, giving the system stronger contextual understanding. An individualized, adjustable time decay mechanism reflects a scientific simulation of human physiological memory characteristics, enhancing the model's biological interpretability. A multi-task collaborative prediction architecture combined with a gating fusion strategy… The system can dynamically focus on key risk dimensions based on patient type, improving the targeting and practicality of early warnings. The interpretable design of the system outputs key evidence chains, reducing the technical barriers to clinical adoption. The closed-loop online incremental learning mechanism ensures the long-term effectiveness of the model and avoids performance degradation due to changes in medical practice. The overall solution has completed a prospective controlled trial in the ICU of a tertiary hospital. The results show that compared with the traditional EWS system, this invention extends the average early warning time for sepsis from 3.2 hours to 7.8 hours, increases the sensitivity of respiratory failure warning from 61% to 89%, reduces the false alarm rate of cardiac arrest to 0.7 times / 1,000 person-days, and reduces the invalid alarm rate to 22% of the original level, significantly reducing the burden on medical staff and improving patient prognosis. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall technical solution architecture proposed in this invention;

[0021] Figure 2 This is a schematic diagram of the core principle framework of heterogeneous multimodal data fusion and dynamic temporal reasoning collaborative modeling in this invention. Detailed Implementation

[0022] Please refer to Figure 1 and Figure 2This invention provides a deep learning-based multimodal risk early warning system for critically ill ICU patients. Its technical architecture consists of a data perception layer, a heterogeneous fusion layer, a temporal reasoning layer, and a risk decision-making layer. Deployed on a hospital's private cloud platform, the system adopts a containerized microservice architecture. Each functional module communicates through well-defined application programming interfaces (APIs), achieving highly cohesive, loosely coupled, distributed operation. The system's overall workflow begins with the real-time access of multi-source medical data, followed by phased embedding and semantic alignment to form a unified representation. Then, an individualized dynamic knowledge graph is constructed, and the evolution of the patient's condition is modeled in a spatiotemporal graph neural network. Finally, an interpretable, tiered early warning instruction is generated through a multi-task collaborative prediction mechanism and pushed to the medical staff's terminal interface.

[0023] The data perception layer, as the system's input front end, is responsible for parallel acquisition of four core data streams from the hospital information system, ensuring their precise alignment and integrity verification along the timeline. The first category is high-frequency physiological signal streams, including ECG waveforms sampled per second, invasive arterial pressure curves, respiratory flow-volume loops, and bispectral exponential EEG sequences. These signals are output from the monitors in raw analog form, then converted to digital signals and transmitted to edge computing nodes. The system is configured with a dedicated data acquisition agent program that polls each bedside device at a fixed frequency of 200 Hz, packaging the raw waveform data into standardized message queue telemetry transmission protocol data packets and adding nanosecond-level timestamps to ensure that the time synchronization error between devices is controlled within ±5 milliseconds. All waveform data is cached locally for 30 seconds to handle brief network interruptions, and forward error correction coding ensures transmission reliability. The second category is structured clinical observations, covering hourly recorded body temperature, consciousness level scores, urine output, and mechanical ventilation parameters. This type of data originates from the timed snapshot interface of the electronic medical record system. The system automatically generates observation templates for all inpatients each day at midnight and retrieves the latest entries through a lightweight polling mechanism. For missing values, the system performs imputation based on medical rationality rules: if there are no temperature records for two consecutive hours, linear interpolation is used to estimate the intermediate value; if the mechanical ventilation mode changes, the system mandates the completion of the complete parameter set at the time of the change, otherwise a data quality alarm is triggered. The third category is semi-structured test and imaging reports, including textual descriptions and key conclusion labels for blood routine tests, coagulation function tests, procalcitonin test values, and chest X-rays. This information is retrieved on demand through query interfaces provided by the hospital laboratory information system and the imaging archiving and communication system. The system establishes a report metadata index, recording the sample collection time, test completion time, reviewing physician number, and report release status for each report. When a new report enters the review queue, the system initiates a preprocessing process, extracting key fields such as white blood cell count, neutrophil percentage, D-dimer concentration, and PaO2 / FiO2 ratio, and mapping them to conceptual identifiers in a standard terminology set. The fourth category is unstructured medical text, including doctor's ward round records, surgical records, and nursing handover notes. The system subscribes to text update events in electronic medical records through a natural language processing gateway. Whenever a new document is submitted, the text parsing service is immediately invoked for preliminary processing. All four types of data items are assigned globally unique resource locators upon entering the system and stored in a distributed database cluster that supports time-series indexing, facilitating efficient retrieval and correlation analysis by time window.

[0024] The heterogeneous fusion layer undertakes the tasks of semantic alignment and joint representation learning for multimodal data. Its core objective is to transform four types of information with significant differences in sampling frequency and data morphology into a unified vector representation in a shared semantic space. This layer adopts a staged embedding strategy, designs dedicated feature extractors for different modalities, and optimizes their relative distribution relationships through a cross-modal contrastive learning mechanism. For high-frequency physiological signal streams, the system configures a set of parallel one-dimensional convolutional neural networks, each dedicated to processing a specific type of waveform signal. Taking an electrocardiogram waveform as an example, its input is a voltage sequence of length 12000 (corresponding to 60 seconds × 200 Hz). The first convolutional layer has a kernel size of 16, a stride of 2, and 32 channels, using a modified linear unit as the activation function. The second convolutional layer expands the kernel size to 32 and increases the number of channels to 64. The third layer further expands to 64, while keeping the number of channels unchanged at 64. Each convolutional layer is followed by a batch normalization layer and a max pooling layer, with a pooling window size of 2. After three layers of convolutional downsampling, the original sequence is compressed into a high-order feature sequence of length 750. Subsequently, a self-attention mechanism module is introduced, comprising eight parallel attention heads. Each head projects the input sequence into query, key, and value vectors, each with a dimension of 64. An attention weight matrix is ​​generated by calculating the dot product similarity between the query and key and applying a scaling factor of 1 / √64. This matrix is ​​then weighted and summed with the value vector to obtain a context-aware feature representation. The multi-head outputs are concatenated and linearly transformed to restore the original dimensions, followed by residual connections and layer normalization. This attention module is repeated four times, ultimately outputting a fixed-length feature vector with a dimension of 512 that includes long-range dependencies. This vector is considered a comprehensive representation of the intraocular electrocardiographic activity over the current 60-second timeframe. Similarly, the invasive arterial pressure curve, respiratory flow-volume loop, and EEG bispectral index sequence are processed by independent but structurally identical convolutional-attention networks, each generating a corresponding 512-dimensional temporal feature vector. The feature vectors of all physiological signals are aggregated according to the acquisition time to form a dynamically updated feature sequence.

[0025] For structured clinical observations, the system first performs standardized preprocessing: body temperature is subtracted from 37.0 and then divided by 1.0; consciousness level scores are directly mapped to integers from 0 to 15; urine output is logarithmically transformed to log(1+urine output); and mechanical ventilation parameters such as positive end-expiratory pressure, tidal volume, and inspired oxygen concentration are normalized to zero mean and unit variance. The processed numerical vectors are input into a fully connected network containing three hidden layers with 128, 64, and 32 neurons respectively, using a modified linear unit activation function. The last layer outputs a 128-dimensional dense representation. An L2 regularization term with a coefficient of 0.001 is introduced during network training to prevent overfitting. This representation captures the nonlinear interaction effects between various observation indicators; for example, hypothermia accompanied by oliguria may indicate early shock, while the combination of high positive end-expiratory pressure and low tidal volume reflects the application of lung-protective ventilation strategies.

[0026] For semi-structured examinations and imaging reports, the system employs a pre-trained medical language model for semantic encoding. The model used is a BERT variant pre-trained on a large-scale Chinese electronic medical record corpus, with a vocabulary expanded to include over 20,000 medical terms. The input text is segmented, and start and end markers are added before being fed into the encoder. The vector corresponding to the [CLS] marker in the last hidden state of the model is extracted as the semantic summary of the entire report, with a dimension of 768. To further enhance the ability to identify pathological signs, a binary classification adapter module is added above the [CLS] vector to determine whether the report contains specific danger signals, such as "septic shock," "acute kidney injury," and "high probability of pulmonary embolism," among 47 preset keywords. The adapter consists of two fully connected layers, with the middle layer having a dimension of 256, and the output layer using the sigmoid activation function. During training, a weighted binary cross-entropy loss function is used, with a positive sample weight set to 5 to compensate for the low proportion of rare diseases in the dataset. The final 768-dimensional output vector not only retains the original semantic information but also enhances the expression strength of key diagnostic clues.

[0027] For unstructured medical text, the system employs a domain-adapted bidirectional long short-term memory (LSTM) network combined with a named entity recognition (NIR) module for key information extraction. The LSM network consists of two layers, each with 256 hidden units. The forward and backward state vectors are concatenated to form a 512-dimensional context-aware representation. A conditional random field (CRF) decoding layer is then deployed to annotate medical entity categories in the text, including symptoms (e.g., fever, chest pain), signs (e.g., tachycardia, hypotension), diagnoses (e.g., pneumonia, heart failure), treatments (e.g., antibiotic use, hemodialysis), examinations (e.g., CT scan, echocardiography), and time expressions (e.g., "yesterday afternoon," "3 days post-surgery"). The entity recognition model is trained on a clinical text corpus containing 100,000 annotated sentences, achieving an F1 score of 92.4%. The system further constructs an event extraction rule engine, organizing identified entities into subject-verb-object triples and attaching timestamps. For example, the statement "The patient experienced shortness of breath at 9:00 AM, and SpO2 dropped to 88%, so the oxygen concentration was increased to 5 L / min" is parsed into three events: {shortness of breath, occurrence time, 09:00}, {SpO2, value, 88%, measurement time, 09:02}, and {oxygen concentration, adjustment, 5 L / min, execution time, 09:05}. All events are sorted by time and input into a time-aware graph attention network to generate a 512-dimensional document-level representation integrating semantic and temporal information.

[0028] The initial representations of all modalities are uniformly projected onto a shared high-dimensional semantic space. The system sets up a common projection matrix to linearly map the output dimensions of physiological signals, structured observations, test reports, and text records from 512, 128, 768, and 512, respectively, to a unified 1024-dimensional space. The projected vectors are then processed by layer normalization to eliminate scale differences. To establish implicit correspondences between modalities, the system implements a cross-modal contrastive learning strategy. The construction rule for positive sample pairs is as follows: different modal data items collected from the same patient within the same 2-hour analysis window are considered positive examples; negative samples are selected from data combinations of different patients at the same absolute time point or with the same number of hospital stays but separated by more than 6 hours. The system uses a normalized temperature-scaled cross-entropy loss function to optimize the model parameters, the mathematical expression of which is as follows:

[0029]

[0030] in, and Let be the vector representation of a pair of positive samples in the projection space. For cosine similarity, The temperature parameter is set to a learnable value, with an initial value of 0.07. For indicator functions, when The value is 1 if positive and 0 otherwise. This loss function forces positive sample pairs to be closer together and negative sample pairs to be further apart, allowing semantically similar heterogeneous information to cluster in the vector space. For example, the distance between an ECG image showing ST-segment elevation and its corresponding diagnostic text "acute myocardial infarction" in the embedding space is significantly smaller than the distance to other irrelevant text. During training, a momentum update strategy is used to maintain a large memory queue to store the embedding vectors of the past 10,000 negative samples, enhancing the contrast density. After end-to-end training, the system obtains a set of encoders capable of generating cross-modal aligned representations, laying the foundation for subsequent temporal inference.

[0031] The temporal reasoning layer performs dynamic evolutionary modeling based on an improved spatiotemporal graph neural network architecture, aiming to explicitly characterize the complex causal and temporal dependencies between clinical events. The system divides each patient's timeline during ICU hospitalization into consecutive 2-hour analysis windows, each covering the period from the start time T to T+2 hours. For each window, the system retrieves all modal representations generated within that period from the database and constructs an individualized dynamic knowledge graph based on these representations. Nodes in the graph represent different types of clinical events or physiological states, including but not limited to 136 predefined node types such as "hyperlactatemia," "hypotension," "oliguria," "increased respiratory rate," "confusion," "new-onset atrial fibrillation," and "antibiotic initiation." Node attributes include their occurrence time, duration, severity level, and source modality. Edge connections are not fixed but determined by the temporal co-occurrence probability between two nodes and the causal strength in the medical prior knowledge base. The system incorporates a causal matrix containing 287 deterministic or probabilistic medical rules, such as "elevated lactate → insufficient tissue perfusion," "PaO2 / FiO2 < 200 → increased likelihood of ARDS," and "new-onset atrial fibrillation + BNP > 500 → cardiac decompensation." Each rule is assigned an initial causal weight, ranging from 0.3 to 0.9, reflecting its recognized strength in clinical practice. When two nodes are temporally adjacent and meet the preconditions of a rule, the system automatically establishes a directed edge, using the initial weight as the starting point.

[0032] This spatiotemporal graph neural network performs message passing operations at each time step. Let the current graph be G=(V,E), where V is the set of nodes and E is the set of edges. For each node v∈V, its state vector... exist The update process for each moment is as follows: First, collect all its neighboring nodes. exist state of time And based on edge weights Multiply by attention coefficient to form aggregated messages :

[0033]

[0034] Attention coefficient The calculation is performed using a single-layer feedforward network, with the input being [ , The concatenated vectors are then output and normalized using softmax. This mechanism allows the model to dynamically focus on neighboring events that have the greatest impact on the current node. (Aggregate messages) With the node's own state The concatenated data is input into a gating loop unit, which contains an update gate. Reset door With candidate state Calculation:

[0035]

[0036]

[0037]

[0038] in For the sigmoid function, For element-wise multiplication, , , This is a learnable weight matrix. The gating mechanism effectively controls the rate of forgetting historical memories and the rate of accepting new information, preventing the disappearance of long-term dependencies. Specifically, the network integrates a differentiable time decay function, assigning exponentially decreasing weights to long-term observations. This function takes the form of an exponential decay kernel, with a decay coefficient... It is not a fixed constant, but rather depends on the patient's age, underlying disease burden index, and hospital admission. Co-adjustment of scores:

[0039]

[0040] in The patient's age ranges from 18 to 98 years. The Charlson Comorbidity Index ranges from 0 to 37. The Acute Physiology and Chronic Health Status Score (AHSS) ranges from 0 to 71. The formula shows that younger patients have a smaller decay coefficient (rapid memory loss), while older patients or those with multiple co-existing conditions retain historical information for a longer period, reflecting individualized differences in memory persistence. This coefficient directly affects the bias term of the update gate in the gated recurrent unit, thereby regulating the retention rate of historical states. After multiple rounds of iterative propagation (typically 6 rounds), the state vector of each node fully absorbs relevant information from the graph. Finally, all node embeddings are averaged to generate a comprehensive health status embedding vector with a dimension of 1024 corresponding to the 2-hour window. This vector serves as a holistic representation of the patient's current condition and is input to the risk decision layer.

[0041] The risk decision layer receives state embedding vectors from the temporal inference layer and drives three dedicated prediction branches in parallel to achieve multi-task collaborative risk assessment. The first branch targets the risk of sepsis onset within 48 hours, employing a binary classifier with class imbalance compensation. Since sepsis cases account for a relatively low percentage (approximately 12%) of ICU patients, the system introduces class weights into the loss function, multiplying the loss for positive samples by a factor of 4.3 to make the model focus more on the minority class. The classifier consists of a three-layer fully connected network with hidden layer dimensions of 512, 256, and 128, using a modified linear unit activation function. The output layer uses a sigmoid activation function to generate a risk probability p_sepsis between 0 and 1. The annotation criteria in the training data follow the definition in the third edition of the "International Consensus on Sepsis and Septic Shock," including evidence of suspected infection and a SOFA score ≥2 points higher than baseline. The second branch addresses the risk of respiratory failure requiring intubation within 24 hours, using a deep partial logistic regression model within a survival analysis framework to estimate the cumulative risk function. The model discretizes the time axis into 12 intervals, each lasting 2 hours. At each time step Calculate the gradient of the partial likelihood function within the risk set:

[0042]

[0043] in, For time steps The partial likelihood loss, For the total risk of all patients at risk, In time Patients who experienced the event (intubation) gathered together. For groups of patients at risk, For patients In time The linear predictor output is obtained by affine transformation of the state embedding vector. The total loss is the sum of partial likelihoods at all time steps. The model output not only includes a risk score but also calculates the importance ranking of each covariate through gradient backpropagation to aid clinical interpretation. The third branch focuses on extreme events of cardiac arrest within 12 hours, constructing a reconstruction error estimator based on anomaly detection principles. The system trains a deep autoencoder, with the encoder part identical to the input network of the risk decision layer, and the decoder having a symmetric structure, attempting to reconstruct the original multimodal input from compressed representations. For normal states, the reconstruction error is low; when patients approach cardiac arrest, physiological signals exhibit highly atypical patterns, causing a sharp increase in reconstruction error. The system uses a moving average control chart to monitor this error, and when it exceeds the μ+3σ threshold (μ and σ are statistically derived from historical data), it is judged as a high-risk state. The risk probability values ​​output from each branch are integrated by a learnable weighted fusion unit to generate the final multidimensional warning index. This unit employs a gated attention mechanism, automatically adjusting the contribution ratios of the three prediction branches based on the patient's current chief diagnosis: for pneumonia patients, the respiratory failure branch's weight is increased to 0.6; for patients with abdominal infection, the sepsis branch receives a weight of 0.7; and for post-cardiac surgery patients, the cardiac arrest branch's influence increases to 0.55. The weights are normalized using softmax to ensure a sum of 1. The final warning index is divided into four levels according to the following rules: [0, 0.3) is green (low risk), [0.3, 0.5) is yellow (moderate concern), [0.5, 0.7) is orange (high alert), and [0.7, 1.0] is red (urgent intervention). The grading results, along with the key evidence chain, are pushed to the healthcare workstation in real time via a visual interface.

[0044] The visualization interface is designed according to clinical usability principles. The left side displays the patient's vital signs trend over the past 24 hours in a timeline format, highlighting the dynamic changes in heart rate, blood pressure, blood oxygen saturation, and respiratory rate. Abnormal values ​​are marked with flashing red. The upper right side displays a large, colored indicator of the current warning level, while the lower section lists three categories of evidence supporting this assessment. The first category is the most discriminative physiological waveform segment, such as an ECG showing a gradually decreasing R-wave amplitude, which the system automatically extracts and labels as "Progressive low voltage in the QRS complex, suggesting possible cardiac tamponade." The second category is abnormal laboratory value trend graphs, such as three consecutive blood lactate tests showing values ​​of 2.1 mmol / L, 3.8 mmol / L, and 6.5 mmol / L, plotted as a line graph with the upper limit of the normal range marked. The third category is warning signal keywords in text records, such as "patient complains of worsening chest tightness," "significant night sweats," and "cold extremities," presented as a tag cloud, with font size reflecting their frequency of occurrence in recent documents. A "One-Click Confirmation" button is located at the bottom of the interface. Clicking this button indicates that medical staff have reviewed the warning information, and this operation is simultaneously logged to the audit trail system. The system has online incremental learning capabilities. Once the complete treatment path of a newly confirmed case is backfilled into the database, a small-batch retraining process is automatically triggered. Every Sunday at 2:00 AM, the system samples 50 newly added positive cases from the past 7 days and their matched 150 negative controls to form a training set. The model is fine-tuned for 10 epochs at a learning rate of 0.0001, periodically updating the model parameters to continuously improve the warning performance over time. Data transmission uses the national standard SM4 encryption algorithm throughout, and static data storage employs transparent database encryption technology. All access requests require authentication through role-based access control policies to ensure that patient privacy complies with the requirements of the Personal Information Protection Law and the Measures for the Management of Network Security in Medical and Health Institutions.

[0045] This embodiment constructs a closed-loop intelligent early warning system through the synergistic operation of the aforementioned technical solutions. The system fundamentally changes the limitations of traditional ICU early warning systems that rely on static thresholds and isolated indicators, achieving for the first time deep fusion and joint reasoning of four types of heterogeneous data: high-frequency physiological signals, structured observations, laboratory images, and free text. By constructing a dynamic knowledge graph and a temporal graph neural network, it can explicitly model complex causal and temporal dependencies between clinical events, significantly improving the ability to identify latent disease deterioration. The cross-modal contrastive learning strategy effectively solves the semantic gap problem of multi-source data, giving the system stronger contextual understanding. The individualized, adjustable time decay mechanism reflects a scientific simulation of human physiological memory characteristics, enhancing the model's biological interpretability. The multi-task collaborative prediction architecture, combined with a gating fusion strategy, can dynamically focus on key risk dimensions based on patient type, improving the targeting and practicality of early warnings. The system's interpretability design outputs a key evidence chain, reducing the technical barriers to clinical adoption. The closed-loop online incremental learning mechanism ensures the long-term effectiveness of the model, avoiding performance degradation due to changes in medical practice. The overall solution has been completed in a prospective controlled trial in the ICU of a tertiary hospital. The results show that compared with the traditional early warning scoring system, the present invention extends the average early warning time for sepsis from 3.2 hours to 7.8 hours, increases the sensitivity of respiratory failure warning from 61% to 89%, reduces the false alarm rate of cardiac arrest to 0.7 times / 1,000 person-days, and reduces the invalid alarm rate to 22% of the original level, significantly reducing the burden on medical staff and improving patient prognosis.

Claims

1. A deep learning-based multimodal risk early warning system for critically ill ICU patients, characterized in that, include: The data perception layer is used to receive high-frequency physiological signal streams, structured clinical observations, semi-structured test and imaging reports, and unstructured medical texts from the hospital information system in parallel to obtain four types of heterogeneous data streams. The heterogeneous fusion layer is used to perform modality-specific encoding and cross-modal contrastive learning on the four types of heterogeneous data streams respectively, so as to obtain a unified representation vector sequence aligned in the shared semantic space. The temporal reasoning layer is used to construct an individualized dynamic knowledge graph based on the unified representation vector sequence, and to perform message passing and state updates through a spatiotemporal graph neural network that integrates medical prior knowledge, gating mechanisms and individualized time decay functions, so as to obtain a comprehensive embedding vector representing the patient's health status. The risk decision layer receives the comprehensive embedded vector, drives multiple prediction branches for different clinical events in parallel, and then integrates them through a learnable weighted fusion unit that can dynamically adjust the weight of each branch according to the patient's chief diagnosis to generate a graded early warning index and push it to the medical workstation.

2. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 1, characterized in that, The heterogeneous fusion layer includes: The physiological signal encoding subunit is used to process the high-frequency physiological signal stream through a parallel one-dimensional convolutional neural network and a self-attention mechanism to obtain a temporal feature vector sequence. The structured observation coding subunit is used to perform nonlinear mapping on the standardized structured clinical observations through a fully connected network to obtain a dense representation vector; The report semantic encoding subunit is used to perform semantic encoding on the semi-structured test and image reports through a pre-trained medical language model to obtain pathological indicator feature vectors; The text event extraction subunit is used to process the unstructured medical text by combining a domain-adapted bidirectional long short-term memory network with a named entity recognition module, in order to extract key event representation vectors with timestamps.

3. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 2, characterized in that, The heterogeneous fusion layer further includes: The common projection subunit is used to linearly map the temporal feature vector sequence, dense representation vector, pathological indicator feature vector, and key event representation vector to the same high-dimensional semantic space, respectively. The cross-modal alignment subunit is used to construct positive sample pairs based on different modal data items of the same patient within the same time window, and to construct negative sample pairs based on different time data items of different patients. The parameters of the common projection subunit are optimized by contrastive learning loss function to bring the positive sample pairs closer and push the negative sample pairs apart.

4. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 1, characterized in that, The temporal reasoning layer includes: The dynamic graph construction sub-unit is used to divide the patient's hospitalization timeline into continuous analysis windows, and use the unified representation vector in each window as a node. The directed edges and their weights are initialized according to the temporal co-occurrence probability between nodes and the preset medical causal rule matrix to construct the individualized dynamic knowledge graph. The spatiotemporal graph neural network subunit is used to aggregate neighbor node information at each time step to update the current node state. The aggregation process is controlled by messages modulated by edge weights and attention coefficients, and the fusion of historical memory and new information is managed by a gated recurrent unit.

5. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 4, characterized in that, The spatiotemporal graph neural network subunit further includes: The individualized time decay subunit is used to dynamically adjust the decay coefficient of the time decay function according to the patient's age, underlying disease burden index and admission score, so that the gated loop unit assigns an exponentially decreasing weight to long-term observations, and the historical information retention period for patients with chronic diseases is longer than that for younger patients.

6. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 1, characterized in that, The risk decision-making layer includes: The sepsis risk prediction branch is used to determine the probability of sepsis onset within a first preset time period in the future based on the comprehensive embedding vector using a binary classifier with a class imbalance compensation mechanism. The respiratory failure risk prediction branch is used to estimate the cumulative risk function of endotracheal intubation required within the next second preset time period based on the comprehensive embedded vector using a deep partial logistic regression model under the survival analysis framework. The cardiac arrest risk prediction branch is used to employ a reconstruction error evaluator based on the anomaly detection principle to quantify the probability of cardiac arrest occurring within a third preset time period by comparing the reconstruction deviation of the autoencoder on the original multimodal input.

7. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 6, characterized in that, The learnable weighted fusion unit is used to dynamically calculate and assign fusion weights to the sepsis risk prediction branch, respiratory failure risk prediction branch, and cardiac arrest risk prediction branch based on the current patient's chief complaint diagnosis label using a gated attention mechanism.

8. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 1, characterized in that, The risk decision-making layer also includes: The evidence chain generation subunit is used to synchronize with the generation of the graded early warning index, extract and organize key evidence supporting the early warning judgment, including the most discriminative physiological waveform segments, abnormal laboratory numerical trend graphs, and danger signal keywords in text records.

9. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 1, characterized in that, The system also includes: The online incremental learning module is used to automatically trigger a small-batch retraining process after the complete diagnosis and treatment path data of newly diagnosed cases is backfilled into the database, so as to periodically fine-tune the model parameters of the heterogeneous fusion layer, temporal inference layer and risk decision layer.

10. The deep learning-based multimodal risk early warning system for critically ill ICU patients according to claim 1, characterized in that, The system is deployed on the hospital's private cloud platform and adopts a containerized microservice architecture. Each functional module communicates through well-defined application programming interfaces. Data transmission uses the national standard SM4 encryption algorithm, and static data storage uses transparent database encryption technology.

Images