Emergency treatment high-risk patient real-time grading method based on improved multi-mode Transform algorithm

Abstract

The invention discloses an emergency treatment high-risk patient real-time grading method based on an improved multi-mode Transform algorithm. The method comprises the following steps: S1, generating a time synchronization multi-mode event sequence; s2, obtaining a corresponding single-mode feature representation tensor; s3, obtaining a risk weighted cross-modal attention matrix; s4, in the improved multi-modal Transform fusion network, generating a fusion feature representation tensor by using a time sequence sliding window cache and incremental updating mechanism, and outputting a risk grading label and a corresponding risk grading confidence coefficient based on the fusion feature representation tensor; s5, inputting the risk grading label and the risk grading confidence into the interpretive sub-network, and generating clinical causal chain prompt information; and S6, synchronizing the risk grading label, the risk grading confidence and the clinical causal chain prompt information. According to the method, the accuracy of high-risk patient identification and the adaptability of the model to a clinical complex scene are remarkably improved, and a test result shows that the real-time identification accuracy of a high-risk case is improved compared with that of a conventional multi-modal model.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm. Background Art

[0002] With the rapid development of emergency medicine informatization, emergency departments of various hospitals have gradually introduced digital and intelligent tools to assist in patient triage and critical illness identification, improve in-hospital treatment efficiency and the survival rate of critically ill patients. Real-time classification of emergency high-risk patients is a key link in emergency management. Its classification results are directly related to the triggering of green channels, bed resource allocation and the priority of multidisciplinary treatment processes. At present, emergency triage in most hospitals is still based on manual assessment and a single data stream, and usually relies on nurses to make preliminary judgments based on the subjective experience of the chief complaint text, some vital sign waveforms or simple medical images. This is not only inefficient and easily affected by the experience differences and workload of medical staff, but also difficult to identify potential high-risk cases in a timely and accurate manner, resulting in delayed treatment for some critically ill patients.

[0003] In recent years, with the development of multimodal deep learning technology, researchers have attempted to introduce multimodal neural networks and Transformer-structured artificial intelligence algorithms into the classification of high-risk emergency patients. Some existing systems can integrate multi-source information such as text, images, and vital signs to improve the objectivity and automation of classification. However, significant technical shortcomings still exist in practical applications:

[0004] In emergency settings, multimodal data streams are often collected asynchronously, and medical imaging results often lag behind the chief complaint text and real-time vital signs. Existing technologies often use simple early splicing or late result fusion, which cannot achieve alignment of asynchronous data at the second level, and can easily cause high-risk features to be missed or repeatedly calculated; secondly, the real-time fusion of multi-source data lacks a robust temporal dynamic mechanism. Simple interception and static window processing will reduce the model's sensitivity to changes in the condition, resulting in delayed response to high-risk events; finally, current multimodal Transformer-based inference algorithms generally have high requirements for computing power and storage resources, making it difficult to run efficiently on low-computing power terminals such as emergency triage desks and mobile carts. The inference speed is slow and lacks the interpretability required for clinical practice, and it is unable to output causal chain clinical explanations that medical staff can trust.

[0005] Therefore, there is an urgent need to propose systematic solutions to the shortcomings of existing technologies. Summary of the Invention

[0006] One purpose of the present invention is to propose a real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm. The present invention significantly improves the accuracy of high-risk patient identification and the model's adaptability to complex clinical scenarios. Test results show that the real-time identification accuracy of high-risk cases is improved compared with conventional multimodal models.

[0007] According to an embodiment of the present invention, a real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm includes the following steps:

[0008] S1. Collect heterogeneous medical data streams from emergency scenes and generate time-synchronized multimodal event sequences;

[0009] S2. Input the time-synchronized multimodal event sequence into a text unimodal encoder, an image unimodal encoder, and a vital sign unimodal encoder, respectively, to obtain corresponding unimodal feature representation tensors;

[0010] S3. Calculate the high-risk probability score at each moment based on the unimodal feature representation tensor and dynamically reconstruct the cross-modal attention weights to obtain the risk-weighted cross-modal attention matrix.

[0011] S4. In the improved multimodal Transformer fusion network, a fused feature representation tensor is generated using a temporal sliding window cache and incremental update mechanism. This fused feature representation tensor is then output to the hierarchical inference network, which then outputs the risk classification label and the corresponding risk classification confidence based on the fused feature representation tensor.

[0012] S5. Input the risk classification label and risk classification confidence into the explanatory subnetwork to generate clinical causal chain prompt information;

[0013] S6. Synchronize risk classification labels, risk classification confidence, and clinical causal chain prompt information.

[0014] Optionally, the S1 includes the following steps:

[0015] S11. Establish a unified time base T ref , perform clock calibration on the text complaint acquisition terminal, medical image acquisition terminal and vital sign waveform acquisition terminal respectively, and obtain the time offset Δ of each terminal t ext、Δ image and Δ w ave;

[0016] S12. Collect text complaint information stream to obtain a text complaint raw data set, collect medical image information stream to obtain a medical image raw data set, collect vital sign waveform information stream to obtain a vital sign raw data set;

[0017] S13. Perform nanosecond synchronization on the timestamps of the text complaint raw data set, the medical image raw data set, and the vital sign raw data set to obtain a synchronized timestamp. Where x∈{text,image,wave}, is the original timestamp, Δ x is the corresponding terminal time offset, δ x is the network transmission delay compensation amount;

[0018] S14. Set the calibrated timestamp t x Combined with the corresponding data content, they are merged in ascending time order to form a time-synchronized multimodal event sequence.

[0019] Optionally, the S2 includes the following steps:

[0020] S21. Based on the modality identifier of each event in the time-synchronized multimodal event sequence, perform triage processing according to modality type, dividing data with the modality identifier of text chief complaint information into a text modality input sequence, data with the modality identifier of medical imaging information into an imaging modality input sequence, and data with the modality identifier of vital sign waveform information into a vital sign waveform modality input sequence;

[0021] S22. Input the text modality input sequence into the text unimodal encoder. The text unimodal encoder performs embedding calculation on each text complaint information based on the medical semantic BERT structure to generate a text modality feature tensor.

[0022] S23. Input the image modality input sequence to the image single-modality encoder, which uses a lightweight residual visual Transformer network to extract spatial structural features of each frame of medical image data and generate an image modality feature tensor;

[0023] S24. Input the vital sign waveform modality input sequence into the vital sign single modality encoder. The vital sign single modality encoder is based on a sequential Transformer network structure and combines local time domain convolution and frequency domain Fourier projection operations to extract sequence features from each waveform sampling data segment to generate a vital sign waveform modality feature tensor.

[0024] S25. Unify and encapsulate the text modality feature tensor, image modality feature tensor, and vital sign waveform modality feature tensor to form a single modality feature representation tensor set.

[0025] Optionally, S3 includes the following steps:

[0026] S31. Based on the single modality feature representation tensor set, at each synchronization timestamp, the text modality feature vector, the imaging modality feature vector, and the vital sign waveform modality feature vector are spliced ​​according to time consistency to obtain a cross-modality feature vector;

[0027] S32. Input the cross-modal feature vector into a three-layer evidence-uncertainty network, and output the predicted mean, predicted variance, and Dirichlet hyperparameter vector through the three-layer evidence-uncertainty network respectively;

[0028] S33. Obtain a high-risk probability score by performing temperature scaling and normalization calculation on the predicted mean value after subtracting the uncertainty compensation term;

[0029] S34. Generate a modal risk weight coefficient vector based on the high-risk probability score and the Dirichlet hyperparameter vector. The modal risk weight coefficient vector includes the text modal risk weight, the imaging modal risk weight and the vital sign waveform modal risk weight. Multiply the text modal feature vector, the imaging modal feature vector and the vital sign waveform modal feature vector with the text modal risk weight, the imaging modal risk weight and the vital sign waveform modal risk weight respectively to obtain the risk-weighted single modal feature. The three risk-weighted single modal features are combined to form a risk-weighted feature matrix.

[0030] Optionally, the S4 includes the following steps:

[0031] S41. Based on the risk-weighted feature matrix, the risk-weighted cross-modal attention matrix is ​​calculated by combining the query linear transformation matrix and the key linear transformation matrix;

[0032] S42. The risk-weighted cross-modal attention matrix and the unimodal feature representation tensor set are combined to form a fusion input pair, which is written into the streaming cache of the improved multimodal Transformer fusion network in a synchronized time sequence. The fusion input pair is used to generate the fusion feature representation tensor

[0033] S43. In the improved multimodal Transformer fusion network, an adaptive time series sliding window is established for the real-time classification task of emergency high-risk patients. The window length of the adaptive time series sliding window is L adaptive According to the high-risk probability score r at the current time t t Dynamic adjustment;

[0034] S44. At each current time t, based on the fused input pairs cached in the adaptive temporal sliding window, an incremental risk-aware attention propagation mechanism is used to perform feature fusion operations to generate an incremental fused feature vector with continuous emergency risk perception capabilities;

[0035] S45. Input the incremental fusion feature vector into the fusion feature stability discriminator, and the fusion feature stability discriminator is based on the fusion feature change rate. Determine the stability of the patient's condition trend, when the fusion feature change rate When the threshold is exceeded, the current feature is marked as an unstable disease pattern;

[0036] S46. Implement a modality-level dynamic pruning strategy for unstable disease pattern features, prune the fusion feature dimensions corresponding to the modalities whose modality risk weights are lower than the pruning threshold, and obtain a disease-driven fusion feature vector;

[0037] S47. Calculate the dynamic quantization bit width value of the fusion feature according to the high-risk probability score corresponding to the fusion feature vector. The dynamic quantization bit width value is obtained by subtracting the difference between the maximum quantization bit width value and the minimum quantization bit width value from the maximum quantization bit width value and multiplying it by the high-risk probability score.

[0038] S48. Input the dynamically quantized fusion feature vector into the risk grading prediction module, calculate the predicted probability value of each emergency risk level, use the risk level with the maximum predicted probability value as the emergency risk grading label at the current moment, and use the maximum predicted probability value as the corresponding risk grading confidence.

[0039] Optionally, the output of the risk classification confidence is defined based on the fused feature vector, the dynamic quantization bit width value, and the predicted probability value:

[0040] High confidence output: the high risk probability score r at the current moment t ≥0.75, fusion feature change rate Δ If the value is less than the stability threshold and the predicted probability value of the corresponding risk level is ≥0.85, it means that the current disease trend is stable and the feature expression is clear. The improved multimodal Transformer fusion network has a consistent judgment on the emergency risk classification label.

[0041] Medium confidence output: 0.4≤high risk probability score r t <0.75, or the fusion feature change rate Close to the stability threshold, and the predicted probability value of the corresponding risk level is between 0.6 and 0.85, indicating that the disease characteristics are relatively vague or in a transitional state. The improved multimodal Transformer fusion network has medium credibility for the emergency risk classification label;

[0042] Low confidence output: high risk probability score r t <0.4, or the fusion feature change rate If the stability threshold is exceeded and the predicted probability value of the corresponding risk level is less than 0.6, it means that the current disease characteristics fluctuate violently, the information expression between modalities is inconsistent, or the reasoning is unstable. There is uncertainty in the output of emergency risk grading labels by the improved multimodal Transformer fusion network.

[0043] Optionally, the S4 includes the following steps:

[0044] S71. Calculate a modal causal factor score vector in the explanatory subnetwork, where the modal causal factor score vector is obtained by element-wise multiplying the modal risk weight coefficient vector with the confidence adjustment factor and the high-risk score adjustment factor;

[0045] S72. Sort the text modality, medical imaging modality, and vital sign waveform modality in descending order based on the modality causal factor score vector. Select the modality with the largest value in the modality causal factor score vector as the dominant modality. The dominant modality is used to identify the data type that plays a dominant role in the emergency risk classification decision at the current moment. Identify the key feature index set that leads to the current risk label determination in the feature representation of the dominant modality.

[0046] S73. Based on the key feature index set and the unimodal feature tensor of the dominant modality, a feature mapping network is used to generate an intermediate semantic interpretation representation tensor to form a structured medical factor expression reflecting the key pathologies, symptoms, or signs under the dominant modality;

[0047] S74. Input the intermediate semantic interpretation tensor into the language generation module to automatically convert the structured medical factor expression into a natural language interpretation description;

[0048] S75. Encapsulate the key feature index set, dominant modality type, natural language explanation description, risk grading label and risk grading confidence into clinical causal chain prompt information.

[0049] The beneficial effects of the present invention are:

[0050] (1) The present invention introduces a dynamic risk-aware attention mechanism based on temporal anchors into the multimodal Transformer architecture, and combines it with a modal weighting method driven by adaptive high-risk probability scores to achieve millisecond-level temporal alignment and risk-sensitive fusion of heterogeneous medical data such as text complaint information, medical imaging information, and vital sign waveform information. It can automatically perceive the temporal relationship of high-risk events in asynchronous data streams, enhance the expressive power of key features in cross-modal reasoning, and significantly improve the accuracy of high-risk patient identification and the model's adaptability to complex clinical scenarios. Test results show that the real-time recognition accuracy of high-risk cases is improved compared with conventional multimodal models.

[0051] (2) The present invention designs an innovative mechanism for adaptively adjusting the length of the temporal sliding window and the dynamic quantization bit width of the fusion features based on the high-risk score, thereby realizing the sensitive adjustment of computing power in the hierarchical reasoning process. By introducing the fusion feature stability judgment and modality-level dynamic pruning strategy, it can focus on the key modal features that are sensitive to changes when the condition fluctuates violently, and perform real-time pruning of irrelevant modalities with low risk weights. In combination with dynamic quantization, the model computational load is reduced, so that the hierarchical reasoning delay is significantly shortened while ensuring the classification accuracy and stability.

[0052] (3) The present invention establishes an explanatory subnetwork, maps risk grading labels, risk grading confidence and modal risk weight information into key feature indexes of the dominant modality, and outputs structured clinical causal chain prompt information through the feature mapping network and medical semantic generation module. The causal chain automatically describes the basis of the model grading decision in natural language, including high-weight features in the chief complaint text, medical images and vital sign waveforms, which can be directly pushed to the emergency information system for reference by medical staff, realizing a traceable explanation of each grading reasoning result, improving the medical staff's trust and adoption rate in the AI-assisted grading system, and providing higher-value intelligent support for clinical decision-making in emergency green channels and multidisciplinary treatment processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0054] Figure 1 This is a flowchart of a real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm proposed in the present invention. DETAILED DESCRIPTION

[0055] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0056] refer to Figure 1 A real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm includes the following steps:

[0057] S1. Collect heterogeneous medical data streams from the emergency room, including textual complaint information, medical imaging information, and vital sign waveform information. Perform communication protocol parsing and nanosecond-level timestamp synchronization on these heterogeneous medical data streams to generate time-synchronized multimodal event sequences.

[0058] S2. Input the time-synchronized multimodal event sequence into a text unimodal encoder, an image unimodal encoder, and a vital sign unimodal encoder, respectively, to obtain corresponding unimodal feature representation tensors;

[0059] S3. Calculate the high-risk probability score at each moment based on the unimodal feature representation tensor. In the risk-driven attention guidance module, dynamically reconstruct the cross-modal attention weights based on the high-risk probability scores to obtain a risk-weighted cross-modal attention matrix. This risk-weighted cross-modal attention matrix and the unimodal feature representation tensor are then input into the improved multimodal Transformer fusion network.

[0060] S4. In the improved multimodal Transformer fusion network, a fused feature representation tensor is generated using a temporal sliding window cache and incremental update mechanism. This fused feature representation tensor is then output to the hierarchical inference network, where pruning and dynamic quantization operations are performed. Based on the fused feature representation tensor, a risk classification label and corresponding risk classification confidence score are output.

[0061] S5. Input the risk classification label and risk classification confidence into the explanatory sub-network to generate clinical causal chain prompt information, which includes key modal feature indexes and natural language descriptions;

[0062] S6. Synchronize risk classification labels, risk classification confidence, and clinical causal chain prompt information.

[0063] In this embodiment, S1 includes the following steps:

[0064] S11. Establish a unified time base T ref , perform clock calibration on the text complaint acquisition terminal, medical image acquisition terminal and vital sign waveform acquisition terminal respectively, and obtain the time offset Δ of each terminal t ext、Δ image and Δ w ave;

[0065] S12. Collect text complaint information stream to obtain a text complaint raw data set, collect medical image information stream to obtain a medical image raw data set, collect vital sign waveform information stream to obtain a vital sign raw data set;

[0066] S13. Perform nanosecond synchronization on the timestamps of the text complaint raw data set, the medical image raw data set, and the vital sign raw data set to obtain a synchronized timestamp. Where x∈{text,image,wave}, is the original timestamp, Δ x is the corresponding terminal time offset, δ xis the network transmission delay compensation amount;

[0067] S14. Set the calibrated timestamp t x Combined with the corresponding data content, they are merged in ascending time order to form a time-synchronized multimodal event sequence.

[0068] In this embodiment, S2 includes the following steps:

[0069] S21. Based on the modality identifier of each event in the time-synchronized multimodal event sequence, perform triage processing according to modality type, dividing data with the modality identifier of text chief complaint information into a text modality input sequence, data with the modality identifier of medical imaging information into an imaging modality input sequence, and data with the modality identifier of vital sign waveform information into a vital sign waveform modality input sequence;

[0070] S22. Input the text modality input sequence into a text unimodal encoder. The text unimodal encoder performs embedding calculations on each text complaint based on the medical semantic BERT architecture to generate a text modality feature tensor. Each feature vector in the text modality feature tensor is used to represent the structural representation of the corresponding text complaint information in the semantic dimension.

[0071] S23. Input the image modality input sequence to an image unimodal encoder. The image unimodal encoder uses a lightweight residual visual Transformer network to extract spatial structural features from each frame of medical image data and generate an image modality feature tensor. Each feature vector in the image modality feature tensor is used to represent the spatial semantic representation of the corresponding image frame.

[0072] S24. Input the vital sign waveform modal input sequence into the vital sign single-modal encoder. The vital sign single-modal encoder, based on a sequential Transformer network structure, combines local time-domain convolution with frequency-domain Fourier projection operations to extract sequence features from each waveform sampling data segment, generating a vital sign waveform modal feature tensor. Each eigenvector in the vital sign waveform modal feature tensor is used to characterize the time-frequency joint distribution characteristics of the corresponding waveform segment.

[0073] S25. Unify and encapsulate the text modality feature tensor, image modality feature tensor, and vital sign waveform modality feature tensor to form a single modality feature representation tensor set.

[0074] In this embodiment, S3 includes the following steps:

[0075] S31. Based on the single modality feature representation tensor set, at each synchronization timestamp, the text modality feature vector, the imaging modality feature vector, and the vital sign waveform modality feature vector are spliced ​​according to time consistency to obtain a cross-modality feature vector;

[0076] S32. Input the cross-modal feature vector into a three-layer evidence-uncertainty network, which then outputs a predicted mean, predicted variance, and Dirichlet hyperparameter vector. The predicted mean is used to reflect the central tendency of the high-risk probability, the predicted variance is used to reflect the uncertainty of the high-risk probability, and the Dirichlet hyperparameter vector is used to subsequently generate the risk weight distribution.

[0077] Three-layer evidence-uncertainty network construction method:

[0078] The first layer takes the concatenated cross-modal feature vector as input. The input layer normalizes and performs preliminary linear mapping on the input features. The preliminary linear mapping transforms the dimension of the original multimodal fusion information through the fully connected layer to obtain the first layer of latent feature representation. The first layer of latent feature representation is used to retain the main correlation information of the original multimodal data.

[0079] In the second layer, the implicit feature representations from the first layer are fed into a fully connected second layer network. This second layer introduces a nonlinear activation function to enhance the modeling of nonlinear feature interactions in complex emergency scenarios. The second layer output further extracts high-order relationships between cross-modal features and, under the action of the activation function, strengthens the response to key medical risk features, capturing the spatiotemporal correlation characteristics and deep expression patterns of high-risk patients.

[0080] In the third layer, the output of the second layer is input into the third layer of evidence generation and uncertainty modeling. The third layer of fully connected structure outputs the central tendency (prediction mean) of the emergency high-risk probability score on the one hand, and calculates the uncertainty of risk prediction (prediction variance) through a separate output branch on the other hand. At the same time, the prior parameters (Dirichlet hyperparameter vector) of each modal risk distribution are calculated through another output branch. The three sets of outputs are normalized or activated in this layer, so that the prediction mean, prediction variance and Dirichlet hyperparameter vector correspond to the central tendency, uncertainty amplitude and modal risk distribution structure of the high-risk probability, respectively.

[0081] The three-layer evidence-uncertainty network adopts a progressive structural design, which not only enables the effective integration of the input emergency multimodal features, but also fully reflects the model's predictive confidence in high-risk, subjective uncertainty, and inter-modal risk attribution weights, meeting the multiple requirements of clinical graded reasoning and medical interpretation for reliability and traceability.

[0082] S33. Obtain a high-risk probability score by subtracting an uncertainty compensation term from the predicted mean and then performing temperature scaling and normalization. The uncertainty compensation term is determined by the compensation coefficient and the square root of the square of the predicted variance. The temperature scaling is controlled by the temperature coefficient and is normalized to the interval [0, 1] using a sigmoid function.

[0083] S34. Generate a modal risk weight coefficient vector based on the high-risk probability score and the Dirichlet hyperparameter vector. The modal risk weight coefficient vector includes the text modal risk weight, the imaging modal risk weight and the vital sign waveform modal risk weight. Multiply the text modal feature vector, the imaging modal feature vector and the vital sign waveform modal feature vector with the text modal risk weight, the imaging modal risk weight and the vital sign waveform modal risk weight respectively to obtain the risk-weighted single modal feature. The three risk-weighted single modal features are combined to form a risk-weighted feature matrix.

[0084] In this embodiment, S4 includes the following steps:

[0085] S41. Based on the risk-weighted feature matrix, the risk-weighted cross-modal attention matrix is ​​calculated by combining the query linear transformation matrix and the key linear transformation matrix;

[0086] S42. The risk-weighted cross-modal attention matrix and the unimodal feature representation tensor set are combined to form a fusion input pair, which is written into the streaming cache of the improved multimodal Transformer fusion network in a synchronized time sequence. The fusion input pair is used to generate the fusion feature representation tensor

[0087] S43. In the improved multimodal Transformer fusion network, an adaptive time series sliding window is established for the real-time classification task of emergency high-risk patients. The window length of the adaptive time series sliding window is L adaptive According to the high-risk probability score r at the current time t t Dynamic Adjustment:

[0088]

[0089] Among them, L base is the basic window length, λ is the risk sensitivity factor, r t Score the probability of high-risk risks at the current moment;

[0090] S44. At each current moment t, an incremental risk-aware attention propagation mechanism is used to perform feature fusion operations based on the fused input pairs cached within the adaptive temporal sliding window. The incremental risk-aware attention propagation mechanism uses the risk-weighted cross-modal attention matrix of the fused input pair at the current moment as the initial attention state, and gradually fuses the risk-weighted cross-modal attention matrices at historical moments tl to generate an incremental fused feature vector with continuous emergency risk perception capabilities.

[0091] S45. Input the incremental fusion feature vector into the fusion feature stability discriminator, and the fusion feature stability discriminator is based on the fusion feature change rate. Determine the stability of the patient's condition trend, when the fusion feature change rate When the threshold is exceeded, the current feature is marked as an unstable condition mode. The threshold is the feature change stability threshold, which reflects the significance of the real-time condition fluctuation of emergency high-risk patients.

[0092] S46. Implement a modality-level dynamic pruning strategy on the unstable disease pattern features marked by the fusion feature stability discriminator. The modality-level dynamic pruning strategy prunes the fusion feature dimensions corresponding to the modalities whose modality risk weights are lower than the pruning threshold based on the current modality risk weight coefficient vector to obtain a disease-driven fusion feature vector.

[0093] S47. Calculate the dynamic quantization bit width value of the fused feature based on the high-risk probability score corresponding to the fused feature vector. The dynamic quantization bit width value is used to control the representation accuracy of the fused feature in the hierarchical inference network. The dynamic quantization bit width value is obtained by subtracting the difference between the maximum quantization bit width value and the minimum quantization bit width value from the maximum quantization bit width value and multiplying it by the high-risk probability score. The higher the high-risk probability score, the lower the dynamic quantization bit width value, reducing the inference delay when the emergency condition is more critical. The maximum quantization bit width value is used to represent the upper limit of quantization accuracy when the condition is stable, and the minimum quantization bit width value is used to represent the lower limit of quantization accuracy when the condition is critical. The high-risk probability score is the disease severity score corresponding to the fused feature at the current moment;

[0094] S48. Input the dynamically quantized fusion feature vector into the risk grading prediction module, and calculate the predicted probability value of each emergency risk level. The predicted probability value is used to characterize the possibility that the fusion feature belongs to the risk level. The predicted probability value is obtained by performing a dot product operation on the risk level weight vector corresponding to each risk level and the dynamically quantized fusion feature vector, adding a bias term and taking the exponent, and normalizing the exponential results of all risk levels. Finally, the risk level with the maximum predicted probability value is used as the emergency risk grading label at the current moment, and the maximum predicted probability value is used as the corresponding risk grading confidence. The risk level weight vector is used to characterize the importance direction of each level feature, and the bias term is used to adjust the classification boundary of each risk level. The risk grading label is the final output decision label of the emergency system.

[0095] In this embodiment, the output of the risk classification confidence is defined based on the fused feature vector, the dynamic quantization bit width value, and the predicted probability value:

[0096] High confidence output: the high risk probability score r at the current moment t ≥0.75, fusion feature change rate If the value is less than the stability threshold and the predicted probability value of the corresponding risk level is ≥0.85, it means that the current disease trend is stable and the feature expression is clear. The improved multimodal Transformer fusion network has a consistent judgment on the emergency risk classification label.

[0097] Medium confidence output: 0.4≤high risk probability score r t <0.75, or the fusion feature change rate Close to the stability threshold, and the predicted probability value of the corresponding risk level is between 0.6 and 0.85, indicating that the disease characteristics are relatively vague or in a transitional state. The improved multimodal Transformer fusion network has medium credibility for the emergency risk classification label;

[0098] Low confidence output: high risk probability score r t <0.4, or the fusion feature change rate If the stability threshold is exceeded and the predicted probability value of the corresponding risk level is less than 0.6, it means that the current disease characteristics fluctuate violently, the information expression between modalities is inconsistent, or the reasoning is unstable. There is uncertainty in the output of emergency risk grading labels by the improved multimodal Transformer fusion network.

[0099] In this embodiment, S4 includes the following steps:

[0100] S71. Combine the risk grading confidence, the modal risk weight coefficient vector, and the high-risk probability score to calculate a modal causal factor score vector in the explanatory subnetwork. The modal causal factor score vector is used to measure the influence of the text modality, medical imaging modality, and vital sign waveform modality in the current emergency risk grading decision. The modal causal factor score vector is obtained by performing an element-by-element product of the modal risk weight coefficient vector with the confidence adjustment factor and the high-risk score adjustment factor. The confidence adjustment factor is set based on the risk grading confidence, and the high-risk score adjustment factor is set based on the high-risk probability score.

[0101] S72. Sort the text modality, medical imaging modality, and vital sign waveform modality in descending order based on the modality causal factor score vector. Select the modality with the largest value in the modality causal factor score vector as the dominant modality. The dominant modality is used to identify the data type that plays a dominant role in the emergency risk classification decision at the current moment. Identify the key feature index set that leads to the current risk label determination in the feature representation of the dominant modality.

[0102] S73. Based on the key feature index set and the unimodal feature tensor of the dominant modality, a feature mapping network is used to generate an intermediate semantic interpretation representation tensor. The intermediate semantic interpretation representation tensor is used to integrate key physiological indicators or imaging signs to form a structured medical factor expression that reflects the key pathologies, symptoms, or signs of the dominant modality.

[0103] S74. Input the intermediate semantic interpretation tensor into the language generation module, automatically converting the structured medical factor expression into a natural language interpretation description to clearly express the causal chain of the current risk grading decision;

[0104] S75. The key feature index set, dominant modality type, natural language explanation description, risk grading label and risk grading confidence are encapsulated together as clinical causal chain prompt information. The clinical causal chain prompt information is used to provide model judgment basis and assist clinical interpretation for medical staff.

[0105] Example 1: In this Example 1, the data of high-risk emergency patients actually admitted to the emergency department of a large tertiary hospital covers multiple scenarios such as pre-hospital 120 emergency system access, in-hospital triage desk, and mobile emergency carts. A total of 14,832 emergency patient samples were collected. In order to verify the effectiveness of the technology of the present invention, a full-process comparative experiment was conducted using the method of the present invention and the traditional static multimodal fusion method (an early fusion algorithm based on LSTM-MLP and an algorithm based on the risk-free dynamic mechanism of the traditional multimodal Transformer).

[0106] In Example 1, the emergency department of a Chinese hospital received a male patient who was sent to the hospital by the ambulance before the hospital. The patient was 59 years old and complained of sudden chest tightness and chest pain accompanied by dyspnea for about 2 hours. When he arrived at the hospital, his condition was critical. The pre-hospital ambulance had uploaded the patient's ECG waveform, heart rate changes, and pulse blood oxygen data in real time through the Internet of Things platform. The patient's chief complaint information was entered at the triage desk at the hospital, and the first bedside lung ultrasound and chest X-ray images were collected and uploaded simultaneously. At this time, the emergency triage desk terminal and the mobile terminal on the cart side simultaneously called the grading model of the present invention to perform real-time disease classification.

[0107] The patient data is first acquired through a multimodal data acquisition interface to obtain the main complaint text: sudden chest tightness and pain accompanied by dyspnea, inability to lie flat, and obvious sweating. The imaging information includes 5 lung ultrasound images and 1 chest X-ray. The vital signs waveform includes a 120-second electrocardiogram curve, real-time pulse oximetry and respiratory curves. Each data item is timestamped in nanoseconds, and clock calibration and synchronization are automatically completed on the triage equipment to generate a time-synchronized multimodal event sequence.

[0108] After data is input into the model, the textual complaint sequence is encoded using the medical semantic BERT unimodal encoder to generate a 768-dimensional embedding vector. The image information is encoded using a lightweight residual ViT model, outputting a 512-dimensional spatial feature vector for each frame. The ECG waveform data is processed using a temporal transformer to generate a 384-dimensional time-frequency feature tensor. The model automatically aligns the asynchronous millisecond-level streams of images, text, and waveforms, and fuses them using an adaptive risk-driven attention mechanism.

[0109] After receiving synchronized segments from all modalities, the model calculated high-risk probability scores in real time. The system detected abnormal ECG waveforms (ST-segment elevation, T-wave inversion), a drop in pulse oximetry to 89%, the presence of chest pain and dyspnea keywords in the chief complaint text, and B-lines and focal consolidation in the lung ultrasound image. Based on the risk perception mechanism, the model adaptively adjusted the sliding window to 16 frames, focusing on data segments where the condition changed dramatically within the current 120 seconds. It also dynamically quantized the fused features to 6-bit accuracy to improve inference speed.

[0110] The inference network quickly pruned, retaining only high-weight features related to respiration, ECG, and chief complaint. After dynamic quantification, the classification result was delivered within 1.7 seconds: High-risk Level II (indicating immediate access to the emergency department's green channel), with a risk classification confidence level of 0.92. The explanatory subnetwork automatically generated a prompt: The patient complained of sudden chest pain, dyspnea, ST-segment elevation on the ECG, decreased blood oxygen, and focal B-lines and consolidation on lung ultrasound. Combining the risk score and trend, the patient was classified as acute and high-risk. The classification label and causal chain explanation information were pushed to the hospital's HIS and ICU coordination systems in real time, initiating the bed reservation, green channel, and multidisciplinary consultation process within 5 seconds.

[0111] The research team selected 1,000 samples of high-risk emergency patients and conducted blind comparisons using the method of the present invention, the traditional static multimodal LSTM-MLP early fusion method, and the conventional multimodal Transformer method without a risk dynamic mechanism. The three groups of algorithms all used the same original samples and annotated labels, with a training data set of 12,000 cases, a validation set of 1,200 cases, and a test set of 1,600 cases. The labels were divided into three levels: high risk (grade I-II), medium risk (grade III), and low risk (grade IV-V). All tests were carried out in the actual deployment environment of the hospital emergency triage desk.

[0112] Table 1 Part of the training samples of the method of the present invention (sample display):

[0113]

[0114]

[0115] Table 2 Comparison of key performance data of test sets

[0116]

[0117] Comparison shows that the method of the present invention is significantly superior to traditional technologies in terms of high-risk emergency case identification accuracy, low missed detection rate and explanation chain output. It can still guarantee second-level reasoning and stable graded output in the low-computing power mobile terminal scenario. Taking ICU emergency bed warning as an example, the missed detection rate of traditional methods is as high as 7.6%-13.1%. The solution of the present invention controls the missed detection rate to within 1.5% and achieves a causal chain output availability rate of more than 98.5%, which greatly improves the trust and compliance of emergency doctors in AI grading recommendations.

[0118] The present invention introduces a dynamic risk-aware attention mechanism based on temporal anchors into the multimodal Transformer architecture, and combines it with a modal weighting method driven by adaptive high-risk probability scores to achieve millisecond-level temporal alignment and risk-sensitive fusion of heterogeneous medical data such as textual chief complaint information, medical imaging information, and vital sign waveform information. It can automatically perceive the temporal relationship of high-risk events in asynchronous data streams, enhance the expressiveness of key features in cross-modal reasoning, and significantly improve the accuracy of high-risk patient identification and the model's adaptability to complex clinical scenarios. Test results show that the real-time recognition accuracy of high-risk cases is improved compared with conventional multimodal models.

[0119] The present invention designs an innovative mechanism for adaptively adjusting the length of the temporal sliding window and the dynamic quantization bit width of the fusion features based on high-risk risk scores, thereby realizing sensitive adjustment of computing power in the hierarchical reasoning process. By introducing fusion feature stability judgment and modal-level dynamic pruning strategy, it can focus on key modal features that are sensitive to changes when the condition fluctuates violently, and perform real-time pruning of irrelevant modalities with low risk weights. In combination with dynamic quantization, it reduces the model computational load, significantly shortens the hierarchical reasoning delay, and ensures the classification accuracy and stability.

[0120] The present invention establishes an explanatory subnetwork, maps risk grading labels, risk grading confidence and modal risk weight information into key feature indexes of the dominant modality, and outputs structured clinical causal chain prompt information through the feature mapping network and medical semantic generation module. The causal chain automatically describes the basis of the model grading decision in natural language, including high-weight features in the chief complaint text, medical images and vital sign waveforms, which can be directly pushed to the emergency information system for reference by medical staff, realizing a traceable explanation of each grading reasoning result, improving the medical staff's trust and adoption rate in the AI-assisted grading system, and providing higher-value intelligent support for clinical decision-making in emergency green channels and multidisciplinary treatment processes.

[0121] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm, characterized by: The steps include: S1. Collect heterogeneous medical data streams from emergency scenes and generate time-synchronized multimodal event sequences; S2. Input the time-synchronized multimodal event sequence into a text unimodal encoder, an image unimodal encoder, and a vital sign unimodal encoder, respectively, to obtain corresponding unimodal feature representation tensors; S3. Calculate the high-risk probability score at each moment based on the unimodal feature representation tensor and dynamically reconstruct the cross-modal attention weights to obtain the risk-weighted cross-modal attention matrix. S4. In the improved multimodal Transformer fusion network, a fused feature representation tensor is generated using a temporal sliding window cache and incremental update mechanism. This fused feature representation tensor is then output to the hierarchical inference network, which then outputs the risk classification label and the corresponding risk classification confidence based on the fused feature representation tensor. S5. Input the risk classification label and risk classification confidence into the explanatory subnetwork to generate clinical causal chain prompt information; S6. Synchronize risk classification labels, risk classification confidence, and clinical causal chain prompt information.

2. A real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm according to claim 1, characterized in that: Said S1 comprises the following steps: S11. Establish a unified time base T ref , perform clock calibration on the text complaint acquisition terminal, medical image acquisition terminal and vital sign waveform acquisition terminal respectively, and obtain the time offset Δ of each terminal t ext、Δ image and Δ w ave; S12. Collect text complaint information stream to obtain a text complaint raw data set, collect medical image information stream to obtain a medical image raw data set, collect vital sign waveform information stream to obtain a vital sign raw data set; S13. Perform nanosecond synchronization on the timestamps of the text complaint raw data set, the medical image raw data set, and the vital sign raw data set to obtain a synchronized timestamp. Where x∈{text,image,wave}, is the original timestamp, Δ x is the corresponding terminal time offset, δ x is the network transmission delay compensation amount; S14. Set the time stamp after calibration Combined with the corresponding data content, they are merged in ascending time order to form a time-synchronized multimodal event sequence.

3. A real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm according to claim 2, characterized in that: The S2 comprises the following steps: S21. Based on the modality identifier of each event in the time-synchronized multimodal event sequence, perform triage processing according to modality type, dividing data with the modality identifier of text chief complaint information into a text modality input sequence, data with the modality identifier of medical imaging information into an imaging modality input sequence, and data with the modality identifier of vital sign waveform information into a vital sign waveform modality input sequence; S22. Input the text modality input sequence into the text unimodal encoder. The text unimodal encoder performs embedding calculation on each text complaint information based on the medical semantic BERT structure to generate a text modality feature tensor. S23. Input the image modality input sequence to the image single-modality encoder, which uses a lightweight residual visual Transformer network to extract spatial structural features of each frame of medical image data and generate an image modality feature tensor; S24. Input the vital sign waveform modality input sequence into the vital sign single modality encoder. The vital sign single modality encoder is based on a sequential Transformer network structure and combines local time domain convolution and frequency domain Fourier projection operations to extract sequence features from each waveform sampling data segment to generate a vital sign waveform modality feature tensor. S25. Unify and encapsulate the text modality feature tensor, image modality feature tensor, and vital sign waveform modality feature tensor to form a single modality feature representation tensor set.

4. A real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm according to claim 3, characterized in that: The S3 includes the following steps: S31. Based on the single modality feature representation tensor set, at each synchronization timestamp, the text modality feature vector, the imaging modality feature vector, and the vital sign waveform modality feature vector are spliced ​​according to time consistency to obtain a cross-modality feature vector; S32. Input the cross-modal feature vector into a three-layer evidence-uncertainty network, and output the predicted mean, predicted variance, and Dirichlet hyperparameter vector through the three-layer evidence-uncertainty network respectively; S33. Obtain a high-risk probability score by performing temperature scaling and normalization calculation on the predicted mean value after subtracting the uncertainty compensation term; S34. Generate a modal risk weight coefficient vector based on the high-risk probability score and the Dirichlet hyperparameter vector. The modal risk weight coefficient vector includes the text modal risk weight, the imaging modal risk weight and the vital sign waveform modal risk weight. Multiply the text modal feature vector, the imaging modal feature vector and the vital sign waveform modal feature vector with the text modal risk weight, the imaging modal risk weight and the vital sign waveform modal risk weight respectively to obtain the risk-weighted single modal feature. The three risk-weighted single modal features are combined to form a risk-weighted feature matrix.

5. A real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm according to claim 4, characterized in that: The S4 comprises the following steps: S41. Based on the risk-weighted feature matrix, the risk-weighted cross-modal attention matrix is ​​calculated by combining the query linear transformation matrix and the key linear transformation matrix; S42. The risk-weighted cross-modal attention matrix and the unimodal feature representation tensor set are combined to form a fusion input pair, which is written into the streaming cache of the improved multimodal Transformer fusion network in a synchronized time sequence. The fusion input pair is used to generate the fusion feature representation tensor S43. In the improved multimodal Transformer fusion network, an adaptive time series sliding window is established for the real-time classification task of emergency high-risk patients. The window length of the adaptive time series sliding window is L adaptive According to the high-risk probability score r at the current time t t Dynamic adjustment; S44. At each current time t, based on the fused input pairs cached in the adaptive temporal sliding window, an incremental risk-aware attention propagation mechanism is used to perform feature fusion operations to generate an incremental fused feature vector with continuous emergency risk perception capabilities; S45. Input the incremental fusion feature vector into the fusion feature stability discriminator, and the fusion feature stability discriminator is based on the fusion feature change rate. Determine the stability of the patient's condition trend, when the fusion feature change rate When the threshold is exceeded, the current feature is marked as an unstable disease pattern; S46. Implement a modality-level dynamic pruning strategy for unstable disease pattern features, prune the fusion feature dimensions corresponding to the modalities whose modality risk weights are lower than the pruning threshold, and obtain a disease-driven fusion feature vector; S47. Calculate the dynamic quantization bit width value of the fusion feature according to the high-risk probability score corresponding to the fusion feature vector; S48. Input the dynamically quantized fusion feature vector into the risk grading prediction module, calculate the predicted probability value of each emergency risk level, use the risk level with the maximum predicted probability value as the emergency risk grading label at the current moment, and use the maximum predicted probability value as the corresponding risk grading confidence.

6. A real-time classification method for emergency high-risk patients based on an improved multimodal Transformer algorithm according to claim 5, characterized in that: The output of the risk classification confidence is defined based on the fusion feature vector, the dynamic quantization bit width value and the predicted probability value: High confidence output: the high risk probability score r at the current moment t ≥0.75, fusion feature change rate Δ If the value is less than the stability threshold and the predicted probability value of the corresponding risk level is ≥0.85, it means that the current disease trend is stable and the feature expression is clear. The improved multimodal Transformer fusion network has a consistent judgment on the emergency risk classification label. Medium confidence output: 0.4≤high risk probability score r t <0.75, or the fusion feature change rate Close to the stability threshold, and the predicted probability value of the corresponding risk level is between 0.6 and 0.85, indicating that the disease characteristics are relatively vague or in a transitional state. The improved multimodal Transformer fusion network has medium credibility for the emergency risk classification label; Low confidence output: high risk probability score r t <0.4, or the fusion feature change rate If the stability threshold is exceeded and the predicted probability value of the corresponding risk level is less than 0.6, it means that the current disease characteristics fluctuate violently, the information expression between modalities is inconsistent, or the reasoning is unstable. There is uncertainty in the output of emergency risk grading labels by the improved multimodal Transformer fusion network.

7. The method for real-time classification of emergency high-risk patients based on an improved multimodal Transformer algorithm according to claim 5, characterized in that: The S4 comprises the following steps: S71. Calculate a modal causal factor score vector in the explanatory subnetwork, where the modal causal factor score vector is obtained by element-wise multiplying the modal risk weight coefficient vector with the confidence adjustment factor and the high-risk score adjustment factor; S72. Sort the text modality, medical imaging modality, and vital sign waveform modality in descending order based on the modality causal factor score vector. Select the modality with the largest value in the modality causal factor score vector as the dominant modality. The dominant modality is used to identify the data type that plays a dominant role in the emergency risk classification decision at the current moment. Identify the key feature index set that leads to the current risk label determination in the feature representation of the dominant modality. S73. Based on the key feature index set and the unimodal feature tensor of the dominant modality, a feature mapping network is used to generate an intermediate semantic interpretation representation tensor to form a structured medical factor expression reflecting the key pathologies, symptoms, or signs under the dominant modality; S74. Input the intermediate semantic interpretation tensor into the language generation module to automatically convert the structured medical factor expression into a natural language interpretation description; S75. Encapsulate the key feature index set, dominant modality type, natural language explanation description, risk grading label and risk grading confidence into clinical causal chain prompt information.

Images