An emergency patient condition grading and rapid triage method and system based on large models and multi-modal perception

Abstract

The application provides a method and system for emergency patient condition grading and rapid triage based on large models and multi-modal perception, applied to the field of data processing technology. Through the whole process of collecting multi-source heterogeneous medical data such as text, vital signs, images, medical history, etc., a complete threshold system is constructed according to clinical standards and treatment goals, including grading, critical value, MEWS score, response time, etc. After data cleaning, error calibration, time alignment and feature fusion, a standardized triage data set is formed. Combined with the complexity of the disease, the computing power and the characteristics of the equipment, a collaborative decision-making strategy is developed, the reasoning window and the collection frequency are set, and the data batches are split and transmitted in order according to the rules. Finally, multi-modal large models are used to realize the mapping learning of features and decisions, dynamically optimize the triage parameters through weighted fusion, automatically output the disease grading, treatment department and treatment priority instructions, and form an efficient, accurate and adaptable intelligent triage control process.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for grading and rapidly triaging emergency patients based on large models and multimodal perception. Background Technology

[0002] Current emergency triage mainly falls into two categories: traditional manual triage and existing intelligent triage systems. Both have significant shortcomings in practical applications. Traditional manual triage suffers from strong subjectivity and poor consistency and accuracy. It relies heavily on the personal experience of medical staff, and is significantly influenced by subjective judgment, experience level, and fatigue. This can lead to triage biases in patients with complex comorbidities, atypical symptoms, or unclear patient complaints, resulting in missed diagnoses of critical cases and misdiagnosis of mild cases, making it difficult to ensure objective and consistent triage results. In emergency situations, data sources are fragmented, including multimodal and heterogeneous information such as written patient complaints, vital signs, laboratory indicators, imaging data, and past medical history. Manual methods struggle to comprehensively integrate and assess this information quickly, easily missing key warning signs, especially regarding the identification of latent critical illnesses. Furthermore, manual triage is slow and lacks standardized procedures, leading to congestion and prolonged waiting times during peak emergency patient periods. The lack of unified standards between different hospitals and personnel hinders standardized, scalable, and replicable triage quality control, failing to meet the demands of efficient modern emergency care.

[0003] Existing intelligent triage systems suffer from weak data processing capabilities and insufficient multimodal fusion. Most existing systems can only process single-type data, failing to effectively integrate multi-source information such as text, physical signs, images, time series data, and medical history. Data standardization, normalization, and outlier handling mechanisms are inadequate, lacking robust handling methods for common emergency problems such as missing data, data noise, and time series misalignment, resulting in poor generalization ability. Most intelligent models operate as "black boxes," lacking strong binding to emergency clinical norms, critical value rules, MEWS scores, and disease grading standards, leading to outputs that do not conform to clinical logic. Furthermore, the lack of interpretable evidence results in low trust among medical staff, hindering practical application. Existing systems do not dynamically adjust the inference window, collection frequency, and compensation strategy based on disease complexity, system computing power, and device acquisition capabilities; they do not support adaptive compensation for missing modalities, exhibiting insufficient stability in scenarios with incomplete data, device asynchrony, or peak traffic, easily leading to missed alarms, delayed responses, and triage errors. Existing technologies generally lack a standardized system covering the entire process from data acquisition, preprocessing, feature governance, batch control, transmission scheduling, model inference, result verification to instruction output, and have not implemented key mechanisms such as hard constraints on critical values, priority scheduling for critical cases, time-series alignment, and knowledge enhancement.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0006] According to one aspect of this application, a method for emergency patient condition grading and rapid triage based on large models and multimodal perception is provided, comprising: collecting multi-source heterogeneous medical perception and clinical correlation data of the entire process of emergency patient triage; setting condition grading thresholds, vital sign critical value ranges, MEWS score thresholds, and triage response time thresholds based on emergency triage accuracy requirements, critical illness identification sensitivity standards, and treatment efficiency targets; processing the multi-source medical perception and triage correlation data, completing text signal cleaning and normalization, vital sign measurement data error calibration, multimodal time-series data alignment, and symptom-sign-image-medical history correlation feature extraction, creating a standardized triage management dataset, reading the text feature dimensions, vital sign parameter types, and time-series data volume information of the dataset, and sorting the data according to the degree of correlation between the data and triage accuracy and treatment efficiency; and combining the condition complexity of the dataset with the system... By considering computing power resources, the characteristics of multimodal large-scale models, and the response characteristics of medical sensing devices, a triage collaborative decision-making strategy is determined. The inference window size and dynamic adjustment frequency are set based on text feature dimensions and time-series data volume, matching the data acquisition frequency of medical sensing devices. The dataset is split according to the target triage decision-making strategy, forming control batches containing text feature data, vital sign accuracy data, image feature data, and triage execution label information. Data is transmitted in an orderly manner based on the inference window size and dynamic adjustment frequency. The mapping relationship between symptom features, vital sign parameters, image information, and triage decision-making strategies is learned through a multimodal feature fusion large-scale model. Triage execution parameters are optimized using a weighted fusion logic of text data, vital sign data, image information, and comorbidity parameters. Combined with patient type, hospital resource allocation, and emergency scenario requirements, triage level, recommended departments, and treatment priority instructions are dynamically adapted to generate triage control signals.

[0007] Another aspect of this application is a system for emergency patient condition grading and rapid triage based on large model and multimodal perception, used to execute executable instructions to perform the above-mentioned method for emergency patient condition grading and rapid triage based on large model and multimodal perception.

[0008] According to another aspect of this application, an electronic device includes: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the above-described method for emergency patient condition grading and rapid triage based on large model and multimodal perception by executing the executable instructions.

[0009] According to another aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a second processor, implements the above-described method for emergency patient condition grading and rapid triage based on large model and multimodal perception.

[0010] The beneficial technical effects of this application are as follows: By collecting heterogeneous medical data from multiple sources such as text, vital signs, images, and medical history, a multi-dimensional threshold system is constructed, including disease grading, critical vital sign values, MEWS scores, and triage timeliness. A standardized triage management dataset is generated through a five-stage closed-loop processing process: text governance, vital sign quality control, image enhancement, temporal alignment, and knowledge association, combined with three-level compliance verification and modality loss adaptive compensation. Triage collaborative decision-making strategies are dynamically formulated based on disease complexity, computing power, and equipment capabilities, setting inference windows, adjusting frequencies, and data collection frequencies. A multimodal large-scale model composed of BioBERT, XGBoost+CNN-LSTM, ResNet-50, and a knowledge-enhanced attention fusion model is used to achieve mapping learning between symptoms, vital signs, images, comorbidity data, and triage decisions, dynamically outputting disease grading, attending departments, and treatment priorities, forming a full-process, interpretable, and highly robust emergency intelligent triage system.

[0011] This application achieves deep fusion of multimodal data and knowledge-enhanced reasoning, significantly improving the accuracy of critical illness identification and triage consistency, and reducing missed and misdiagnosed rates. Rigid criticality values ​​and timeliness thresholds are established to ensure zero-delay alarms and priority treatment for critically ill patients, guaranteeing the golden window for intervention. Adaptive compensation for missing modalities is supported, ensuring stable operation and strong robustness even in complex scenarios such as incomplete data and peak traffic. The model has interpretable evidence and clinical rule validation, and its output aligns with emergency diagnosis and treatment logic, resulting in high clinical acceptance. Full-process automation and standardization significantly shorten triage time, reduce the workload of medical staff, and optimize the efficiency of emergency resource scheduling.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0013] Figure 1 This document illustrates a flowchart of a method for emergency patient condition grading and rapid triage based on large model and multimodal perception, provided in an embodiment of this application. Figure 2This illustration shows a schematic diagram of the structure of a system for emergency patient condition grading and rapid triage based on large model and multimodal perception, provided in an embodiment of this application. Detailed Implementation

[0014] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0015] In one implementation, Figure 1 The diagram illustrates a flowchart of a method for emergency patient condition grading and rapid triage based on large model and multimodal perception, according to an embodiment of this application.

[0016] S101 collects multi-source heterogeneous medical perception and clinical correlation data throughout the entire process of emergency patient triage.

[0017] In one implementation, the system seamlessly integrates with emergency room equipment and information systems across the entire process through standardized interfaces. It primarily uses automatic data collection, supplemented by manual data entry, to fully acquire structured and unstructured medical perception data and clinically relevant data throughout the triage process. After unifying the timing, format, and encapsulation, the data is output to downstream modules, providing complete, compliant, and directly usable raw data for subsequent data processing and model inference.

[0018] The system first establishes stable connections with the electronic medical record system, laboratory information system, medical image archiving and communication system, emergency monitoring equipment, and nurse workstations. Data collection is initiated synchronously according to a unified timestamp, ensuring no information is missed in any stage of the triage process. During data collection, structured data such as basic patient information, vital signs, laboratory indicators, and past medical history are automatically acquired first. Simultaneously, unstructured data such as chief complaint texts, nursing records, emergency images, and physician diagnoses are also acquired. Furthermore, clinically relevant data such as the time of visit, treatment status, equipment status, and triage nodes are collected concurrently, achieving simultaneous acquisition of multi-source data.

[0019] For patients with impaired consciousness or inability to express themselves independently, the system provides a manual entry point for medical staff to supplement missing information such as chief complaints and disease progression, ensuring data integrity and usability. After data collection, the system automatically performs integrity checks, alerting users to missing core indicators, removing duplicate data, and retaining the latest valid records. Finally, the system uniformly encapsulates all collected multi-source heterogeneous data, maintaining clinical correspondence and temporal consistency among different data types, forming a standardized raw data set. This set is then stably transmitted to the data preprocessing module, providing a high-quality data source for subsequent text normalization, error calibration, temporal alignment, and feature extraction.

[0020] S102 sets thresholds for disease severity classification, critical vital signs range, MEWS score threshold, and triage response time threshold based on the requirements for emergency triage accuracy, the sensitivity standard for critical illness identification, and the goal of treatment efficiency.

[0021] In one implementation, based on the expert consensus standards for emergency triage, the sensitivity requirements for critical illness identification, and the efficiency target for emergency golden treatment, a four-level (I-IV) disease grading threshold system is set, and the grading is locked according to any highest-level indicator. Based on the "Expert Consensus Standards for Emergency Triage," the sensitivity requirements for critical illness identification, and the efficiency target for emergency golden treatment, a four-level (I-IV) disease grading threshold system is systematically constructed, covering vital signs, laboratory indicators, level of consciousness, pain score, and MEWS score. The grading thresholds are directly linked to key clinical indicators such as body temperature, heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, blood oxygen saturation, blood potassium, and blood glucose. A mandatory rule is adopted to lock the final grading using any highest-level indicator; if any indicator reaches a higher risk level, the corresponding grading is immediately locked, and downgrading is never allowed. This grading rule forms a strong correlation with critical vital signs and MEWS scores, directly converting quantitative scores into clinical grading. At the same time, it uses weighted enhancement judgment for patients with hidden critical illnesses, complex comorbidities, and atypical symptoms to ensure that critical illnesses are not missed or misdiagnosed, providing a unified, rigid, and traceable basis for the allocation of treatment priorities and the scheduling of medical resources.

[0022] Based on emergency clinical treatment guidelines, critical illness early warning indicators, and patient safety requirements, the system sets critical value ranges for vital signs such as body temperature, heart rate, respiratory rate, systolic / diastolic blood pressure, SpO2, serum potassium, and blood glucose. Following the "Expert Consensus on Emergency Triage" and emergency critical value management standards, the system sets upper and lower critical values ​​for eight core vital signs and laboratory indicators—body temperature, heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, serum oxygen saturation (SpO2), serum potassium, and blood glucose—forming rigid judgment ranges. Critical values ​​serve as the system's highest priority trigger condition, directly and strongly linked to the Level I emergency / critical illness classification. If any indicator reaches or exceeds the critical value range, the system immediately classifies the patient as Level I critical, automatically triggering the highest-level audible and visual alarm, prioritizing the allocation of medical resources, and bypassing the regular triage queue to enter the emergency treatment process. The critical value range is simultaneously connected to the data verification module and the rule correction module. Abnormal measurement values ​​are subject to statistical and clinical dual verification to distinguish between the actual critical state and the equipment measurement error, thus avoiding false triggering. For patients who have triggered the critical value, the system locks the classification throughout the process and does not allow automatic downgrading, ensuring that critically ill patients receive treatment without delay and completely eliminating waiting delays from the rule level.

[0023] Based on the improved early warning scoring rules, disease risk stratification standards, and emergency treatment priority determination requirements, the system sets MEWS score grading thresholds and establishes a strong correlation mapping with the four-level disease severity classification. The system divides MEWS scores into four threshold ranges: 0–2, 3–4, 5–6, and ≥7, corresponding to disease levels IV, III, II, and I, respectively. Higher scores indicate more severe conditions, and the classification level automatically increases, achieving a direct conversion between quantitative scores and clinical classification. This mapping relationship serves as the core judgment basis for the rule verification module. It is connected to the output results of the multimodal fusion triage model in real time to verify the compliance and correct the deviation of the disease classification inferred by the model, ensuring that the triage results meet the emergency clinical standards. At the same time, the MEWS score is directly integrated into the treatment priority ranking and department recommendation process, serving as a key reference indicator for priority allocation and resource scheduling, thereby comprehensively improving the consistency, accuracy and clinical credibility of the triage results.

[0024] Based on the emergency department's peak flow capacity, the time requirements for triage per patient, and the need for rapid response to critically ill patients, the system sets triage response time thresholds and automatic alarm delay thresholds for Level I-II critically ill patients. Following the "Expert Consensus on Emergency Triage" and emergency clinical quality control standards, the system sets full-process triage response time thresholds. Simultaneously, it sets automatic alarm delay thresholds for Level I and II critically ill patients, forming a tiered timeliness management system. The triage response time thresholds are dynamically adapted according to the emergency department's flow level, with differentiated time limits for regular, off-peak, and peak periods to ensure that the entire triage process for a single patient does not exceed the time limit. The automatic alarm delay thresholds for Level I-II critically ill patients employ a zero-delay / extremely short-delay mandatory strategy; once a patient is determined to be critically ill, the highest-level audible and visual alarm and interface pop-up are immediately triggered without waiting or delay. This threshold system, together with the system scheduling strategy, the data collection frequency of medical sensing devices, the inference window size, and the data transmission sequence, forms a comprehensive linkage mechanism. In scenarios of overloaded emergency patient traffic, limited computing power, and concurrent data processing, it automatically prioritizes critical care data, reduces unnecessary waiting, and locks inference resources. This ensures that patients at levels I-II can receive timely alarms, priority scheduling, and rapid treatment under any load conditions. From both rule and mechanism perspectives, it eliminates the risk of missed alarms and missed treatments for critical care cases due to system congestion and time delays, and comprehensively guarantees the implementation of the golden time for emergency treatment.

[0025] Based on multimodal data missing scenarios, complex comorbidity weighting rules, and model inference confidence requirements, the system sets three key parameters: modality missing threshold, comorbidity weighting coefficient, and inference confidence threshold. These parameters are determined according to the characteristics of emergency clinical data and the multimodal fusion inference mechanism. For real-world scenarios where emergency text, physical signs, and imaging data are prone to single or multiple missing items, the system categorizes missing modalities by quantity and criticality, setting a modality missing threshold to automatically identify text, physical sign, imaging, and combined missing states. When the threshold is reached, modality masking, learnable missing embedding, and dynamic attention compensation mechanisms are immediately activated to ensure stable inference even with incomplete data.

[0026] To address the challenges of complex comorbidities involving multiple pathological layers and strong feature interactions, the system establishes tiered comorbidity weighting coefficients. During the multimodal feature fusion stage, comorbidity-related features such as hypertension, diabetes, coronary heart disease, and stroke are weighted and enhanced to ensure that comorbidity information contributes appropriately to disease grading, preventing triage bias caused by overlooking comorbidities. To ensure that model inference results meet clinical reliability, interpretability, and verifiability requirements, the system sets inference confidence thresholds. When the confidence levels of the model's output grading, priority, and departmental recommendations fall below the threshold, the system automatically triggers clinical rule verification, feature re-verification, and manual review processes. Simultaneously, interpretable evidence and risk warnings are generated to ensure that every triage result is safe, compliant, and traceable.

[0027] S103 processes multi-source medical perception and triage-related data, completing text signal cleaning and normalization, vital sign measurement data error calibration, multimodal time series data alignment, symptom-sign-image-medical history association feature extraction, creating a standardized triage management dataset, reading the dataset's text feature dimensions, vital sign parameter types, and time series data volume information, and sorting them according to the degree of correlation between the data and triage accuracy and treatment efficiency.

[0028] In one implementation, based on the disease complexity distribution of the dataset, the system's computing power capacity, the adaptability of the multimodal large model, and the response time of medical sensing devices, the multimodal preprocessing engine and the triage feature configuration center collaboratively determine the end-to-end processing and knowledge enhancement control strategy for emergency data. The multimodal preprocessing engine is responsible for reading and parsing the disease complexity distribution, the system's real-time computing power capacity, and the modality adaptability and inference overhead characteristics of the multimodal large model in the dataset; the triage feature configuration center is responsible for reading the hardware parameters of the medical sensing devices, such as sampling frequency, transmission latency, and response time, and matching them with the clinical constraints of the golden timeframe for emergency treatment.

[0029] The two systems work together bidirectionally through a standardized interface, combining emergency clinical pathology analysis, early identification rules for critically ill patients, and the time-series requirements of the diagnosis and treatment process. This clarifies the direction of multimodal data governance, uniformly defining five core feature configuration dimensions: textual symptoms, vital signs, medical images, past medical history, and temporal changes. This establishes unified rules, processing granularity, and target paths for the entire data processing process, including subsequent data cleaning, normalization, error calibration, temporal alignment, feature extraction, and knowledge graph enhancement. The system can dynamically adapt processing strategies based on dataset composition, computing power level, and device capabilities. When the dataset mainly consists of complex comorbidities, elderly high-risk patients, the system has mid-range computing power, and the sensing device has a moderate acquisition frequency, it automatically establishes a data governance direction centered on "high robustness, lightweight inference, modality missing compatibility, and priority for critically ill features." This prioritizes ensuring the efficiency of critically ill feature extraction and the accuracy of triage, while balancing system computational load and triage real-time performance, ensuring stable output of high-quality standardized data even in complex scenarios such as peak emergency periods, incomplete data, and limited computing power.

[0030] A five-stage closed-loop processing mechanism—text governance, vital sign quality control, image enhancement, temporal alignment, and knowledge association—is adopted to standardize and process multi-source medical perception data collected throughout the entire emergency triage process. The system first classifies and governs the data according to six categories: chief complaint text, vital signs, laboratory indicators, emergency images, past medical history, and time series. Then, it performs adaptation and matching around three core objectives: triage accuracy, critical illness detection rate, and treatment response efficiency. Simultaneously, it addresses three objectives: data standardization, feature interpretability, and model generalization ability, ultimately generating basic triage data processing results enhanced with a medical knowledge graph.

[0031] In the text governance stage, unstructured texts such as chief complaints, present medical history, and nursing records undergo text cleaning, medical-specific word segmentation, emergency terminology normalization, and medical vector conversion to unify emergency symptom descriptions, eliminate colloquial and fragmented noise, and output standardized text features. In the vital signs quality control stage, missing value classification and imputation, outlier statistics and clinical double verification, numerical normalization, and unit standardization are performed on vital signs and laboratory indicators to ensure accurate, stable, and comparable data. In the image enhancement stage, CT, ECG, and X-ray images undergo format standardization, noise reduction, contrast enhancement, and ROI lesion region extraction to enhance key imaging features of critically ill patients. In the temporal alignment stage, text, vital signs, and image data collected from different devices and at different times undergo unified timestamp encoding and dynamic window normalization to achieve temporal synchronization of multimodal data. In the knowledge association stage, combined with a medical knowledge graph, symptoms, vital signs, images, and past medical history are clinically semantically associated and matched with pathological pathways to form interpretable and inferable clinical knowledge clusters. After processing, the system is adapted and optimized with the aim of triage accuracy, critical illness detection rate, and treatment response efficiency, while ensuring data standardization, feature interpretability, and model generalization ability. Finally, it outputs high-quality basic data processing results with knowledge graph enhancement.

[0032] The system performs three levels of validity and clinical compliance checks on the basic data processing results. It initiates medical semantic cleaning and terminology standardization optimization to address text noise, non-standard terminology, and fragmented chief complaints. For abnormal vital signs and measurement errors, it triggers statistical and clinical dual verification and error calibration correction. For multimodal temporal misalignment, it performs unified timestamp alignment and dynamic window normalization. Redundant features and low-value dimensions are screened using attention weights. This generates a standardized data processing solution that includes cleaning rules, quality control parameters, temporal standards, feature maps, and missing data compensation strategies. The system performs three levels of validity and clinical compliance checks on the basic data processing results, adhering to emergency clinical quality control standards and multi-center data standards throughout the process. It strictly implements closed-loop verification and optimization according to the four dimensions of text, vital signs, temporal sequence, and features to ensure data compliance, accuracy, temporal consistency, and feature validity, providing high-quality input for subsequent model inference. To address issues such as text noise, colloquial expressions, non-standard medical terminology, and fragmented chief complaints, the system initiates a medical semantic cleaning and emergency terminology standardization optimization process. It employs medical-specific word segmentation and an emergency symptom dictionary to uniformly convert non-standard expressions into standard clinical terms, eliminating text noise and semantic ambiguity, and ensuring that text features can be stably recognized by the model.

[0033] In response to abnormal vital signs, measurement errors, and numerical jumps, the system triggers a dual verification mechanism involving statistical detection and attending physicians. First, abnormal values ​​are identified through statistical methods, then clinical verification is performed by emergency department attending physicians to distinguish between true critical indicators and equipment measurement errors. Error data is calibrated and corrected, while true critical values ​​are retained and marked with high-risk characteristics. To address issues such as multimodal temporal misalignment and asynchronous data acquisition from different devices, the system performs unified timestamp alignment and dynamic window normalization. This aligns all modal data, including text, vital signs, images, and medical history, to the same consultation time reference, and completes temporal normalization according to the inference window, ensuring complete synchronization of multimodal data in the time dimension.

[0034] For redundant features, low-value dimensions, and features weakly correlated with triage, the system uses attention weights to filter key features. It ranks features based on their contribution to triage accuracy, critical illness detection rate, and treatment efficiency, retaining high-contribution core features and eliminating redundant and low-value dimensions, thus simplifying the feature space and improving model inference efficiency. After all three levels of verification are completed, the system automatically generates a standardized data processing solution including cleaning rules, quality control parameters, time-series standards, feature maps, and missing data compensation strategies. All processing steps are traceable, reproducible, and verifiable, ensuring that the entire data processing process is standardized, transparent, and stable.

[0035] The system integrates and executes standardized data processing schemes, combining multi-center emergency clinical quality control rules with a modality missing adaptive compensation mechanism to create high-quality datasets. Simultaneously, it assigns corresponding ranking priorities to different data items based on the correlation weights between features and triage accuracy, critical illness detection rate, and treatment efficiency, generating triage management datasets and feature weight ranking results adapted to complex emergency scenarios and supporting modality missing adaptation. The system integrates and executes standardized data processing schemes, adhering to multi-center emergency clinical quality control rules and the requirements of the "Expert Consensus on Emergency Triage," and simultaneously enables a modality missing adaptive compensation mechanism. For scenarios where any single modality or multiple modalities are missing simultaneously in text, physical signs, or images, the system automatically completes feature completion and information compensation through modality masking, learnable missing embeddings, and dynamic attention compensation techniques, ensuring the generation of stable, reliable, and high-quality standardized datasets even with incomplete data.

[0036] After the dataset is created, the system calculates and assigns weights to different data items based on the correlation contribution of each feature dimension with triage accuracy, critical illness detection rate, and treatment efficiency. Strongly correlated features such as vital signs, state of consciousness, MEWS score, and high-risk lesions on imaging are given high priority, while weakly correlated features such as age, gender, and medical history are given secondary priority.

[0037] Ultimately, a standardized triage management dataset is generated that is adapted to complex scenarios such as peak emergency room visits, complex comorbidities, and incomplete data, and supports modality missing adaptation. The feature weight ranking results are output simultaneously, so that the dataset structure, feature order and inference input format are fully matched with the input requirements of multimodal large models, ensuring that subsequent triage inference is efficient, accurate and interpretable.

[0038] S104. Combining the complexity of the disease in the dataset, the system's computing resources, the characteristics of the multimodal large model, and the response characteristics of the medical sensing devices, a triage and collaborative decision-making strategy is determined. Based on the text feature dimensions and the amount of time-series data, the inference window size and dynamic adjustment frequency are set to match the data collection frequency of the medical sensing devices.

[0039] In one implementation, based on the disease complexity distribution of the dataset, the system's computing power capacity, the adaptability of the multimodal large model, and the response time of medical sensing devices, the strategy formulation module, in collaboration with the data processing module, the multimodal fusion triage model module, and the medical sensing device layer, uniformly determines the triage collaborative decision-making and dynamic inference strategy. The strategy formulation module first reads the disease complexity distribution, disease risk stratification, and multi-center data quality control rules from the data processing module; then it reads the system's real-time computing power load, memory usage, inference latency, and other computing power capacity patterns; simultaneously, it acquires the multimodal large model's input adaptation characteristics for text, vital signs, images, and comorbidity data, its modality missing compatibility, and inference overhead; and it collects hardware operating parameters of the medical sensing devices, such as sampling frequency, transmission latency, response time, and data interface format.

[0040] Combining the requirements of the golden timeframe for emergency treatment, the sensitivity of early identification of critically ill patients, the classification rules for levels I-IV, the MEWS score mapping relationship, and clinical quality control standards, a comprehensive triage collaborative decision-making and dynamic inference strategy was formulated. This strategy clarifies the overall rules for inference window scheduling, multimodal data acquisition, adaptive compensation for missing modalities, abnormal data monitoring and readjustment, and priority protection for critically ill patients. The strategy can be dynamically adapted to different scenarios: when the dataset mainly consists of patients with complex comorbidities and elderly critically ill patients, the system has mid-range computing power, and the monitoring equipment has a moderate sampling frequency, a strategy of lightweight inference, high-frequency acquisition, rapid compensation, and priority protection for critically ill patients is automatically adopted. When there is peak emergency traffic, limited computing power, or incomplete data, a strategy of simplifying the inference path, compressing the inference window, and improving the compensation response speed is automatically activated, ensuring stable, efficient, and reliable operation under various complex scenarios.

[0041] The system sets the inference window size, dynamic adjustment frequency, and modality missing compensation interval based on text feature dimensions, image feature dimensions, temporal vital sign data volume, and complex comorbidity weighting rules. The system also incorporates complex comorbidity weighting rules and emergency clinical quality control requirements to uniformly quantify these settings. The inference window limits the temporal data range covered by a single model inference, ensuring disease assessment is completed within the golden treatment window. The dynamic adjustment frequency controls the refresh cycle of triage strategies, feature weights, and inference parameters, ensuring the system adapts to changes in patient condition in real time. The modality missing compensation interval specifies the automatic compensation and resampling time interval when any modality of text, vital signs, or image data is missing, ensuring stable inference in scenarios with incomplete data. These three elements work together to form a complete temporal control system for model inference, deeply integrated with the input specifications of the multimodal large model, the sampling frequency of medical sensing devices, and the emergency department's traffic load status.

[0042] When the text features are high-dimensional, the image dimensions are complete, and the amount of time-series vital signs data is large, the inference window is set to 15 minutes after the visit, the dynamic adjustment frequency is set to once every 3 minutes, and the modality missing compensation interval is set to 1 minute. When the patient has complex comorbidities, high disease complexity, or is in the peak of emergency traffic, the system automatically increases the compensation frequency, shortens the compensation interval, reduces the inference window, and improves the adjustment accuracy to ensure the real-time and accurate triage of critically ill and complex patients.

[0043] The system matches the inference window, adjustment frequency, and compensation interval with the acquisition capabilities of the multimodal sensing device, calibrating and determining the hierarchical data acquisition frequencies for text acquisition, vital sign sampling, and image acquisition. The system globally matches and calibrates the pre-set inference window, dynamic adjustment frequency, and modality loss compensation interval with the actual acquisition capabilities, interface specifications, and response latency of the multimodal sensing device, determining the text acquisition frequency, vital sign sampling frequency, and image acquisition frequency respectively, forming a hierarchical acquisition frequency system fully adapted to the emergency clinical workflow. The text acquisition frequency is consistent with the input rhythm and completion status of the patient's chief complaint, present medical history, and past medical history, achieving complete text information acquisition without omissions or delays. The vital sign sampling frequency is synchronized with the dynamic change rate and fluctuation characteristics of vital signs such as heart rate, blood pressure, respiration, and blood oxygen saturation, using a higher sampling frequency for rapidly changing key signs to ensure complete and traceable temporal characteristics. The image acquisition frequency is matched with the examination process, imaging time, and transmission rate of emergency CT, electrocardiogram, and X-ray images, adhering to the principles of acquisition upon completion of examination, priority transmission, and no blocking of inference. All three are strictly aligned with the inference window length, dynamic adjustment cycle, and missing data compensation interval, ensuring that the data acquisition rhythm is consistent with the model input sequence.

[0044] For example, rapidly fluctuating vital signs such as heart rate, blood pressure, and blood oxygen saturation are sampled continuously at high frequency and uploaded in real time to ensure sufficient temporal data and complete change trajectories. Textual data such as chief complaints and medical history are collected immediately upon completion of input and automatically captured to ensure the integrity of textual feature dimensions. Imaging data such as CT scans, electrocardiograms, and chest X-rays are acquired and transmitted in a single package after the examination to ensure the integrity of imaging feature dimensions and avoid duplicate collection. All collection frequencies are dynamically fine-tuned based on the complexity of the condition, weighting rules for complex comorbidities, and modality missing compensation requirements, ensuring that the collection behavior is fully coordinated with the model inference sequence, data compensation timing, and emergency treatment rhythm, and ensuring that all multimodal data entering the inference window are temporally consistent, dimensionally complete, and of the required quality. This tiered data acquisition frequency system achieves full alignment between the acquisition rhythm of text, vital signs, and images and the model inference sequence, clinical process, and compensation mechanism through precise matching of inference window, adjustment frequency, compensation interval, and equipment acquisition capabilities. It is fully transparent, has clear rules, and can be directly implemented, significantly improving the standardization of data acquisition, consistency of timing, and stability of triage inference.

[0045] The system drives devices to collect multi-source medical data in real time at a predetermined frequency, completing data reception, caching, and modal alignment according to the inference window timing rules. Following calibrated and determined text acquisition, vital sign sampling, and image acquisition frequencies, the system drives medical sensing devices, electronic medical record systems, laboratory information systems, and medical image archiving and communication systems to collect multi-source medical data in real time through standardized interfaces. The acquisition process strictly follows emergency clinical procedures: text data is captured immediately after the chief complaint, present illness, and past medical history are entered; vital sign data is continuously acquired at a dynamic timing frequency; and image data is acquired all at once after the examination, ensuring that the acquisition rhythm is perfectly matched with the inference window, dynamic adjustment frequency, and compensation interval.

[0046] The system uniformly receives, caches, and queues all collected data according to the inference window timing rules. It performs unified timestamp encoding on multimodal data such as text, vital signs, test indicators, images, past medical history, and time series, and completes modal alignment and dynamic window normalization. This ensures that data collected from different sources and at different times are arranged in an orderly manner and synchronized in time on the same consultation time benchmark, forming well-structured, dimensionally complete, and time-series consistent input data. This provides high-quality, directly inferable standardized input for the multimodal fusion triage model.

[0047] For example, the system continuously receives dynamic vital signs data such as heart rate, blood pressure, and blood oxygen saturation at a set high frequency, caches all temporal vital signs sequences within the inference window for nearly 15 minutes, simultaneously captures text data such as chief complaint and present medical history, immediately acquires image data after CT, electrocardiogram and other imaging examinations are completed, and uniformly aligns all modal data to the time of the patient's first visit, forming a standardized data structure that is temporally complete, modally complete and can be directly input into the model.

[0048] If modality loss, insufficient dimensions, temporal misalignment, or abnormal comorbidity features occur, the system reports the anomaly information to the upper-level module, triggering a window-frequency-compensation three-in-one adaptive readjustment mechanism to re-optimize the inference window, acquisition frequency, and missing feature compensation strategy until the data input meets the requirements of multimodal fusion inference. The system performs real-time full-domain monitoring of the data input status, covering the completeness of text, vital signs, and images, the completeness of feature dimensions, temporal consistency, and the rationality of comorbidity features. When modality loss, insufficient feature dimensions, multimodal temporal misalignment, abnormal complex comorbidity features, or indicator conflicts are detected, the system immediately reports the anomaly type, location, and severity to the upper-level strategy formulation module and data processing module, and simultaneously triggers the window-frequency-compensation three-in-one adaptive readjustment mechanism.

[0049] Once activated, the mechanism optimizes three core parameters—inference window size, triage data acquisition frequency, and modality missing compensation strategy—according to emergency clinical rules and model inference constraints. This involves dynamically shrinking or expanding the inference window to accommodate the current amount of effective data; increasing the frequency of vital sign sampling based on critical illness priority; adjusting text capture timing; and optimizing image acquisition rhythm. Simultaneously, modality masking, learnable missing embeddings, and dynamic attention compensation are enabled to complete features and enhance information for missing modalities. The system then enters a cyclical calibration process, repeatedly verifying data integrity, temporal alignment, and feature validity until the input quality fully meets the inference requirements of the multimodal fusion triage model.

[0050] For example, when a patient fails to complete imaging examinations, resulting in missing imaging modalities, the system automatically reduces the inference window, increases the sampling frequency of key vital signs such as heart rate, blood pressure, and blood oxygen, shortens the missing compensation interval, and initiates text plus vital sign dual-modal compensation logic; when asynchronous acquisition by multiple devices causes time sequence misalignment, the system re-executes unified timestamp alignment and dynamic window normalization; when complex comorbidity features are identified as abnormal or with insufficient weight, the system automatically increases the comorbidity weight coefficient to strengthen the contribution of comorbidity information in disease grading until all data indicators meet the standards.

[0051] S105 splits the dataset according to the target triage decision strategy to form a control batch containing text feature data, vital sign accuracy data, image feature data and triage execution label information, and transmits the data in an orderly manner based on the inference window size and dynamic adjustment frequency.

[0052] In one implementation, the data transmission module performs the operation, establishing a bidirectional data connection with the data processing module, strategy formulation module, and decision execution module. The module first reads the triage collaborative decision-making strategy output by the strategy formulation module, then reads the standardized triage management dataset output by the data processing module. It then structurally splits the dataset into five categories: textual symptom features, vital sign parameters, imaging features, comorbidity information, and triage execution labels. This results in a clear data category classification, a multimodal feature association table, and a triage label matching mapping relationship, ensuring a one-to-one correspondence between each data category and the triage decision. The system categorizes patient complaints and present medical history into the textual feature category; heart rate, blood pressure, and blood oxygen saturation into the vital sign accuracy category; CT scans and electrocardiograms into the imaging feature category; hypertension and diabetes into the comorbidity association category; and finally, the grade, department, and priority are categorized into the triage execution label category.

[0053] The system standardizes the categorized data based on the inference window size set by the strategy formulation module, determining the data length, feature count, and time span of each batch to form a unified splitting granularity standard. A larger inference window includes a longer time series of data; a smaller inference window results in finer splitting granularity and a higher update frequency. The splitting granularity directly determines the smallest unit for subsequent batch transmission, ensuring a perfect match with the model's inference input specifications. When the inference window is set to 15 minutes, the splitting granularity includes all vital sign sampling points, complete text, and image data from the past 15 minutes; when the inference window is set to 5 minutes, only the most recent 5 minutes of high-real-time data are retained, achieving lightweight and rapid splitting.

[0054] The system determines the update cycle of controlled batches based on the dynamic adjustment frequency output by the strategy formulation module. Simultaneously, it defines the refresh rules for batches when data is missing, in conjunction with the modal missing compensation interval. A higher dynamic adjustment frequency results in a shorter batch update cycle; for critically ill patients, the update cycle is automatically shortened to ensure data real-time performance. The batch update cycle is synchronized with the modal compensation interval to ensure rapid data recovery when data is missing. If the dynamic adjustment frequency is once every 3 minutes, the batch update cycle is set to 3 minutes; the modal missing compensation interval is set to 1 minute. If a batch of image data is missing, compensation is automatically initiated and the batch is regenerated within 1 minute.

[0055] The system groups and integrates text feature data, vital sign accuracy data, image feature data, comorbidity parameter data, and triage execution label data according to a predetermined splitting granularity standard and batch update cycle, forming standardized control batches with unified structure, complete fields, and temporal alignment. Each batch contains all the input fields required for model inference and can be directly fed into the multimodal fusion triage model without secondary processing. The system packages text, vital signs, images, comorbidities, and label data of the same patient and within the same inference window into a single control batch, ensuring temporal consistency and modal integrity, which can be directly used for model input.

[0056] The system prioritizes all controlled batches based on disease severity, risk of critical illness, and urgency of triage, with Level I and Level II critical illness batches transmitted first. Combining the synchronization rhythm of the inference window with dynamic timing adjustment requirements, the system develops an orderly transmission plan, clearly defining batch numbers, detailed data composition, transmission timing, synchronization verification rules, and modal missing data compensation rules, forming a complete dataset for transmitting control information. Level I critical illness batches have the highest priority and are transmitted first; ordinary emergency batches are sorted by visit time; during transmission, data integrity is verified synchronously, and missing data is immediately retransmitted according to the compensation rules.

[0057] Following an ordered transmission scheme, the system transmits control batches sequentially, stably, and in real-time to the decision execution module based on the inference window size and dynamic adjustment frequency. This provides continuous, standardized, and time-aligned input data for the multimodal feature fusion model. The transmission process maintains a time-series rule, ensuring first-come, first-served transmission and prioritizing critically ill patients. Upon completion of transmission, a confirmation signal is sent, completing the entire data transmission loop. The system pushes a batch of data every 3 minutes according to the inference window, with critically ill patient batches transmitted in real-time. After transmission, a success status is sent back to the strategy formulation module, ensuring uninterrupted inference flow.

[0058] S106 learns the mapping relationship between symptom features, vital sign parameters, imaging information and triage decision-making strategies through a large multimodal feature fusion model. It optimizes triage execution parameters according to the weighted fusion logic of text data, vital sign data, imaging information and comorbid parameters. Combined with patient type, hospital resource allocation and emergency scenario needs, it dynamically adapts triage level, recommended departments, and treatment priority instructions to generate triage control signals.

[0059] In one implementation, the standardized triage and management dataset is categorized and split, including textual symptom feature data, vital sign parameter data, imaging feature data, comorbidity association data, and triage execution label data, generating data category classification results, a multimodal feature association table, and triage label matching mapping relationships. A bidirectional data connection is established between the decision execution module and the data transmission module, the multimodal fusion triage model module, and the interpretability and post-processing module. The system reads the standardized triage and management dataset from the data transmission module and, based on the emergency clinical data structure and multimodal inference input specifications, structurally splits and categorizes the dataset into five major categories: textual symptom features, vital sign parameters, imaging features, comorbidity association information, and triage execution labels.

[0060] During the decomposition process, the system standardizes medical terminology and semantics for unstructured texts such as chief complaints, present medical history, and nursing records, forming unified textual symptom feature data; it performs numerical calibration and unit unification for vital signs and laboratory indicators such as body temperature, heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, SpO2, serum potassium, and blood glucose, forming standardized vital sign parameter data; it performs ROI extraction and feature enhancement processing on emergency images such as CT scans, electrocardiograms, and X-rays, forming standardized image feature data; it encodes and weights information on chronic disease histories and complex comorbidities such as hypertension, diabetes, coronary heart disease, and stroke, forming comorbidity association data; and it organizes clinical output items such as disease severity classification results, recommended departments, treatment priorities, and alarm levels into triage execution label data.

[0061] The system uses a medical knowledge graph to associate multimodal features, establishes clinical logical relationships between symptoms, signs, images, and comorbidities, and generates data classification results, multimodal feature association tables, and triage label matching mapping relationships. This ensures that each input feature forms a unique and traceable correspondence with the final triage decision, grading result, and department recommendation, providing a structured, standardized, and alignable input foundation for multimodal fusion reasoning and interpretable output.

[0062] Based on the target triage decision-making strategy, the system standardizes the data classification results, multimodal feature association table, and triage label matching mapping relationship. The data splitting granularity is set according to the inference window size, and the batch update cycle and modality compensation interval are determined based on the dynamic adjustment frequency. The system standardizes the data classification results, multimodal feature association table, and triage label matching mapping relationship according to the target triage decision-making strategy. The data splitting granularity is set according to the inference window size; the larger the inference window, the longer the time-series data included in the splitting granularity. The batch update cycle is determined based on the dynamic adjustment frequency, and the modality compensation interval is also determined to ensure that data updates and missing data compensation are performed synchronously.

[0063] For example, the inference window is set to 15 minutes, the splitting granularity includes all vital sign sampling points, complete text and images within 15 minutes; the dynamic adjustment frequency is once every 3 minutes, the batch update cycle is set to 3 minutes, and the modal compensation interval is set to 1 minute.

[0064] Based on the granularity of data splitting and the batch update cycle, the system groups and integrates various data categories to form controlled batches containing subsets of textual symptoms, vital signs, imaging features, comorbidity parameters, and triage execution labels. The system uniformly groups and integrates textual symptom feature data, vital sign parameter data, imaging feature data, comorbidity-related data, and triage execution label data according to the established granularity of data splitting and the batch update cycle. The integration process strictly follows multi-center emergency clinical quality control rules, using a unified timestamp as a benchmark to complete full-modal temporal alignment. Field standardization and dimensional completion are performed on textual, vital sign, imaging, medical history, and temporal data. Redundancy removal, anomaly verification, and missing data compensation are performed simultaneously, ultimately forming standardized controlled batches with unified structure, temporal alignment, complete fields, and compliant traceability. Each controlled batch fully contains all the input items required by the multimodal fusion triage model, supporting direct input into the model for inference without additional preprocessing, format conversion, or feature completion. The system packages all data from the same patient and the same inference window, including textual symptoms, vital signs, emergency images, complex comorbid information, and triage execution labels, into an independent control batch. This ensures modality integrity, temporal consistency, complete features, and corresponding labels, meeting the requirements for adaptive compensation for complex emergency scenarios and modality loss.

[0065] The system prioritizes the transmission of each controlled batch, and formulates an ordered transmission plan based on the synchronization rhythm of the inference window and the requirements for dynamic timing adjustment. This generates dataset transmission control information including batch number, data composition details, transmission timing plan, synchronization verification rules, and modality missing compensation rules. The data transmission module executes the priority ranking and transmission plan formulation for controlled batches, establishing bidirectional data interaction with the strategy formulation module, decision execution module, and data processing module. Based on the patient's I-IV disease severity classification, critical illness risk score, MEWS score, critical vital signs, and triage urgency, the system prioritizes all standardized controlled batches across multiple dimensions, setting the batches of Class I and II critically ill patients as the highest priority for priority transmission, scheduling, and inference, ensuring that critically ill data is not blocked by regular traffic and that there are no timing delays.

[0066] The system strictly matches the synchronization rhythm and dynamic adjustment timing requirements of the inference window, and formulates an orderly transmission plan based on the emergency room traffic load, system computing power capacity, and equipment acquisition response time. The transmission process follows the principles of prioritizing critical cases, time sequence alignment, batch independence, and immediate compensation. For batches of the same priority, transmission is carried out in the order of patients' arrival time to ensure that the data within the inference window is complete, the time sequence is not disordered, and the modalities are not missing.

[0067] The system automatically generates complete dataset transmission control information, including a unique batch number, detailed data composition of text / vital signs / images / comorbidities / labels, transmission timing plan, transmission synchronization verification rules, and modality missing compensation rules. This enables full-process traceability, monitoring, and reversibility of transmission. Real-time data integrity verification is performed during transmission. If any anomalies occur, such as missing text, incomplete vital sign acquisition, missing images, or timing misalignment, re-acquisition, interpolation completion, or batch retransmission is immediately initiated according to preset compensation rules, ensuring that every batch of data fed into the model meets the requirements for multimodal fusion inference.

[0068] For example, Level I critically ill patients are automatically placed at the top of the transmission queue and transmitted with the highest bandwidth and shortest latency; Level III and IV general emergency patients are queued in order of their appointment time; if a missing image modality is detected during transmission, the system immediately triggers re-import and re-transmission at the compensation interval to keep the inference window data continuously available.

[0069] The system learns the mapping relationship between symptom features, vital signs, imaging information, comorbidity data, and triage decision-making strategies through a large multimodal feature fusion model. It optimizes triage execution parameters based on a weighted fusion logic of text data, vital signs, imaging information, and comorbidity parameters. Combining patient type, hospital resource allocation, and emergency scenario requirements, it dynamically adapts triage levels, recommended departments, and treatment priority instructions, generating triage control signals with interpretable evidence. The system inputs standardized control batches into the multimodal fusion triage model module. This module establishes bidirectional data connections with the decision execution module, interpretability and post-processing module, and data transmission module. The system is composed of four hierarchically connected sub-models: a text parsing sub-model (BioBERT), a structured data inference sub-model (XGBoost+CNN-LSTM), an image feature extraction sub-model (ResNet-50), and a fusion decision sub-model (adaptive attention mechanism). The feature vectors output by each sub-model are mapped to a unified dimension and then fed into the fusion layer, achieving end-to-end multimodal feature learning.

[0070] The fusion decision sub-model employs an attention-based fusion model (AttentionFusionModel) to fuse the feature vectors output from the text parsing sub-model, structured data inference sub-model, and image feature extraction sub-model. It automatically assigns weights to each modality's features through an attention mechanism (e.g., image features and vital signs of critically ill patients have higher weights than the chief complaint text features), ultimately outputting the patient's condition level, triage priority, and recommended department, thus addressing the shortcomings of existing technologies in multimodal information fusion. This step is not simply about using attention, but incorporates five core innovations: ① Modal serialization: Convert heterogeneous multimodal data into a unified sequence, tokenize it, and map it to a unified vector space, solving the problem that traditional methods cannot handle unstructured text and image features.

[0071] ② Dynamic weighting mechanism: The weights are not preset rules, but are automatically calculated by the model based on the deep semantics of the input data (such as the critical value characteristics of the images), and are adaptive. For example, a fast heart rate alone may not be critical, but when accompanied by signs of aortic dissection on imaging, its weight is immediately adjusted to the highest.

[0072] ③ Introducing time axis semantics: Adding time stamp encoding during serialization not only focuses on the current blood oxygen saturation, but also pays attention to the decline curve over the past 15 minutes. The Time-AwareTransformer structure is used to better capture the dynamic evolution of vital signs.

[0073] ④ Knowledge Enhancement and Medical Atlas: A medical knowledge graph (KG) is introduced in the modal serialization stage. When “enlarged cardiac shadow” in the image and “dyspnea” in the text are identified, the clinical pathway knowledge of “acute left heart failure” is automatically associated to calibrate the dynamic weights. Knowledge enhancement and medical atlas inject the prior knowledge in the KG into the representation learning of the model through semantic retrieval or graph attention mechanism. (1) Atlas construction: Existing medical knowledge graphs (such as UMLS) or self-constructed emergency and cardiovascular specialty sub-atlases are used. The core is to define entities and relationships. Entities include enlarged cardiac shadow, dyspnea, acute left heart failure, chest pain, aortic widening, ST segment elevation, hypertension, diabetes, etc. Relationships include prompt, manifestation, cause, belong to, complication, etc. (2) Image modality: Chest X-ray, CT, and electrocardiogram are encoded using a pre-trained visual model (CNN or VisionTransformer). When visual features such as “enlarged cardiac shadow” are detected, they are mapped to the “enlarged cardiac shadow” entity node in the knowledge graph through the classification head. (3) Text Modality: The BioBERT language model is used to encode the chief complaint and present medical history. Entity linking technology is used to map text symptoms such as "dyspnea" and "chest pain" to corresponding entity nodes in the knowledge graph. (4) Knowledge Retrieval: The identified entities (such as cardiomegaly and dyspnea) are used as queries to retrieve relevant neighbor nodes and clinical pathways in the knowledge graph. The retrieved knowledge is integrated into the Transformer computation, and the standard attention mechanism is modified. When calculating attention weights, the original features are no longer relied upon, but knowledge similarity scores are introduced. Knowledge enhancement is achieved based on knowledge gating fusion or cross-modal semantic retrieval. ⑤ End-to-end multi-task output: A single forward propagation solves three clinical problems simultaneously: triage, staffing, and orientation, significantly reducing system latency and conforming to the fast-paced workflow of the emergency department.

[0074] The specific implementation process is as follows: 1. Input Feature Vector Preprocessing and Dimension Unification: Since the three sub-models output feature vectors with different dimensions, a feature mapping layer is first constructed to map them to the same semantic space. The text feature vector T is the output vector of the text parsing sub-model, the structured data feature vector S is the output vector after MLP encoding, and the image feature vector I is the feature map output by ResNet or ViT. Three fully connected layers are constructed to map T, S, and I to a unified embedding dimension, and after layer normalization, a unified dimension feature vector is output.

[0075] 2. Attention Mechanism Fusion Layer: To address the insufficient fusion of multimodal information, an adaptive intra-modal and inter-modal attention mechanism is adopted. First, a feature sequence is constructed, treating the three modal feature vectors as a sequence of length 3, with each element representing a modality. Then, multi-head self-attention calculation is performed, inputting the sequence into the Transformer encoder layer. Through self-attention, each modal feature references other modal features to complete the update. The query, key, and value matrix and attention weights between modalities are calculated. The weights are obtained through adaptive learning by the model. The weights of image features and vital signs features of critically ill patients are automatically increased, while the weights of textual complaints are correspondingly decreased. Finally, feature fusion output is performed, concatenating or pooling the cross-modal fused feature vectors to obtain the final fused feature vector.

[0076] 3. Implementation of time axis semantics: Time axis semantics is implemented by introducing timestamp encoding into the standard Transformer structure. A fixed time window is set, and time-series vital sign data within the window are extracted. A corresponding vector is generated for each time difference and discretized into buckets. The vectors of each bucket are learned through the embedding layer, enabling the model to perceive the time proximity of the occurrence of vital sign data, identify the danger decline curve, and improve the ability to identify critically ill patients sensitive to time-series changes.

[0077] The model learns the clinical mapping relationship between textual symptom features, vital sign parameters, medical imaging information, comorbidity data, and triage decision-making strategies in a supervised manner. It strictly follows the expert consensus rules for emergency triage, the MEWS scoring system, and critical vital sign thresholds. Following a dynamic weighted fusion logic of text, vital signs, imaging, and comorbidity parameters, it automatically optimizes triage execution parameters. For critically ill patients, the system automatically increases the weight of imaging features and vital sign features; for patients with complex comorbidities, it automatically increases the weight of comorbidity parameters and correspondingly decreases the weight of the chief complaint in the text, achieving adaptive weighted fusion that fully aligns with emergency clinical logic.

[0078] After completing the fusion reasoning, the system dynamically outputs the final disease classification, recommended departments, and treatment priority scheduling instructions by combining patient type, hospital resource allocation, emergency room traffic scenarios, and treatment priority rules. All results are simultaneously sent to the interpretability and post-processing module to generate reasoning basis in natural language form, modal feature importance ranking, and image lesion heat map annotations, forming a triage control signal with complete interpretable basis, which is automatically synchronized to the hospital EMR system, nurse workstations, and human-computer interaction interfaces.

[0079] In one implementation, such as Figure 2 As shown, this application also provides a system for emergency patient condition grading and rapid triage based on large models and multimodal perception, including: Data acquisition module 201 is used to collect multi-source heterogeneous medical perception and clinical correlation data throughout the entire process of emergency patient triage; The threshold setting module 202 is used to set the threshold for disease classification, the range of critical vital signs, the MEWS score threshold, and the triage response time threshold based on the requirements for emergency triage accuracy, the sensitivity standard for critical illness identification, and the goal of treatment efficiency. Data processing module 203 is used to process multi-source medical perception and triage related data. It uses text signal cleaning and normalization algorithm, vital sign measurement data error calibration model, multimodal time series data alignment logic, and symptom-sign-image-medical history association feature extraction method to create a standardized triage management dataset. It reads the text feature dimension, vital sign parameter type, and time series data volume information of the dataset and sorts them according to the degree of correlation between the data and triage accuracy and treatment efficiency. The strategy formulation module 204 is used to determine the triage and collaborative decision-making strategy by combining the complexity of the disease in the dataset, the computing power resources of the system, the characteristics of the multimodal large model and the response characteristics of the medical sensing device. It sets the inference window size and dynamic adjustment frequency based on the text feature dimension and the amount of time-series data, and matches the data collection frequency of the medical sensing device. Data transmission module 205 is used to split the dataset according to the target triage decision strategy, form a control batch containing text feature data, vital sign accuracy data, image feature data and triage execution label information, and transmit the data in an orderly manner based on the inference window size and dynamic adjustment frequency; The decision execution module 206 is used to learn the mapping relationship between symptom features, vital sign parameters, imaging information and triage decision strategies through a multimodal feature fusion model. It optimizes the triage execution parameters according to the weighted fusion logic of text data, vital sign data, imaging information and comorbid parameters. Combined with patient type, hospital resource allocation and emergency scenario requirements, it dynamically adapts triage level, recommended departments, and treatment priority instructions to generate triage control signals.

[0080] The computer-readable storage medium provided in the above embodiments of this application and the method for emergency patient condition grading and rapid triage based on large model and multimodal perception provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0081] The various embodiments in this application are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of the method, system, electronic device, and readable storage medium for evaluating emergency patient condition grading and rapid triage based on large model and multimodal perception are basically similar to the embodiments of the method for emergency patient condition grading and rapid triage based on large model and multimodal perception described above, and are therefore described simply. Relevant parts can be referred to in the description of the embodiments of the method for emergency patient condition grading and rapid triage based on large model and multimodal perception described above.

Claims

1. A method for emergency patient condition grading and rapid triage based on a large model and multimodal perception, characterized in that, include: Collect multi-source heterogeneous medical perception and clinical correlation data throughout the entire process of emergency patient triage; Based on the requirements for emergency triage accuracy, the standards for critical illness identification sensitivity, and the goals for treatment efficiency, thresholds for disease grading, critical vital signs ranges, MEWS score thresholds, and triage response time thresholds are set. The data related to multi-source medical perception and triage are processed to complete text signal cleaning and normalization, error calibration of vital sign measurement data, alignment of multimodal time series data, and extraction of symptom-sign-image-medical history association features. A standardized triage management dataset is created, and the text feature dimension, vital sign parameter type, and time series data volume information of the dataset are read. The dataset is then sorted according to the degree of correlation between the data and triage accuracy and treatment efficiency. By combining the complexity of the disease in the dataset, the computing power resources of the system, the characteristics of the multimodal large model and the response characteristics of the medical sensing device, a triage and collaborative decision-making strategy is determined. The inference window size and dynamic adjustment frequency are set according to the text feature dimension and the amount of time series data, and matched with the data collection frequency of the medical sensing device. The dataset is split according to the target triage decision-making strategy to form a control batch containing text feature data, vital sign accuracy data, image feature data and triage execution label information, and the data is transmitted in an orderly manner based on the inference window size and dynamic adjustment frequency; By learning the mapping relationship between symptom features, vital sign parameters, imaging information and triage decision-making strategies through a large multimodal feature fusion model, the triage execution parameters are optimized according to the weighted fusion logic of text data, vital sign data, imaging information and comorbid parameters. Combined with patient type, hospital resource allocation and emergency scenario needs, the triage level, recommended departments, and treatment priority instructions are dynamically adapted to generate triage control signals.

2. The method as described in claim 1, characterized in that, Based on the requirements for emergency triage accuracy, the sensitivity standards for identifying critically ill patients, and the goals for treatment efficiency, thresholds for disease severity determination, critical vital sign ranges, MEWS score thresholds, and triage response time thresholds are set, including: Based on the expert consensus standards for emergency triage, the sensitivity requirements for critical illness identification, and the efficiency target for emergency golden treatment, a four-level disease classification threshold is set for grades I–IV, and the classification is locked according to any highest-level indicator. Based on emergency clinical diagnosis and treatment guidelines, critical illness early warning indicators, and patient life safety requirements, critical value ranges for vital signs such as body temperature, heart rate, respiratory rate, systolic / diastolic blood pressure, SpO2, serum potassium, and blood glucose are set. Based on the improved early warning scoring rules, disease risk stratification standards and emergency treatment priority determination requirements, MEWS score grading thresholds are set and strongly correlated with the four-level disease grading. Based on the peak flow capacity of the emergency department, the time requirement for triage of a single patient, and the need for rapid response to critical illnesses, triage response time thresholds and automatic alarm delay thresholds for Level I-II critical illnesses are set. Based on the scenarios of missing multimodal data, complex comorbidity weighting rules, and model inference confidence requirements, we set thresholds for modality missing judgment, comorbidity weighting coefficients, and inference confidence thresholds.

3. The method as described in claim 1, characterized in that, The data related to multi-source medical perception and triage are processed, including text signal cleaning and normalization, error calibration of vital sign measurement data, alignment of multimodal time-series data, and extraction of symptom-sign-image-medical history correlation features. A standardized triage management dataset is created by reading the dataset's text feature dimensions, vital sign parameter types, and time-series data volume information, and sorting the data according to their correlation with triage accuracy and treatment efficiency. Based on the distribution of disease complexity in the dataset, the system's computing power carrying capacity, the adaptability of multimodal large models, and the response time of medical sensing devices, the multimodal preprocessing engine and the triage feature configuration center work together to determine the full-link processing and knowledge enhancement management strategy for emergency data. By analyzing clinical pathological characteristics and matching them with the golden treatment time requirements, the direction of multimodal data governance and the core feature configuration dimensions are clarified. A five-stage closed-loop processing mechanism—text governance, vital sign quality control, image enhancement, temporal alignment, and knowledge association—is adopted to process multi-source medical perception data. First, the data is classified and governed according to chief complaint text, vital signs, laboratory indicators, emergency images, past medical history, and time series. Then, based on the core objectives of triage accuracy, critical illness detection rate, and treatment response efficiency, adaptive matching is completed. The three core objectives of data standardization, feature interpretability, and model generalization ability are also addressed to generate basic results of triage data processing with knowledge graph enhancement. The basic results of data processing are verified for three levels of validity and clinical compliance. Medical semantic cleaning and terminology standardization optimization are initiated to address text noise, non-standard terminology, and fragmented chief complaints. Statistical and clinical dual verification and error calibration correction are triggered for abnormal signs and measurement errors. Unified timestamp alignment and dynamic window normalization are performed for multimodal temporal misalignment. Key feature screening based on attention weight is carried out for redundant features and low-value dimensions. A standardized data processing scheme including cleaning rules, quality control parameters, temporal standards, feature maps, and missing data compensation strategies is generated. The standardized data processing scheme is integrated and implemented. A high-quality dataset is produced by combining multi-center emergency clinical quality control rules and modality missing adaptive compensation mechanism. At the same time, different data items are assigned corresponding sorting priorities based on the correlation weights between features and triage accuracy, critical illness detection rate and treatment efficiency. This generates a triage management dataset and feature weight sorting results that are adapted to complex emergency scenarios and support modality missing adaptation.

4. The method as described in claim 1, characterized in that, Based on the complexity of the disease in the dataset, system computing resources, the characteristics of multimodal large models, and the response characteristics of medical sensing devices, a triage and collaborative decision-making strategy is determined. The inference window size and dynamic adjustment frequency are set according to the text feature dimensions and the amount of time-series data, matching the data acquisition frequency of the medical sensing devices, including: Based on the distribution of disease complexity in the dataset, the computing power carrying capacity of the system, the adaptability of multimodal large models, and the response time of medical sensing devices, a collaborative triage decision-making and dynamic reasoning strategy is determined. The system sets the inference window size, dynamic adjustment frequency, and modality missing compensation interval based on text feature dimensions, image feature dimensions, temporal vital sign data volume, and complex comorbidity weighting rules; The system matches the inference window, adjustment frequency, and compensation interval with the acquisition capabilities of the multimodal sensing device, calibrates and determines the hierarchical data acquisition frequency for text acquisition, vital sign sampling, and image acquisition; The system drives the device to collect multi-source medical data in real time at a determined frequency, and completes data reception, caching and modal alignment according to the inference window timing rules. If modality is missing, dimension is insufficient, timing is misaligned, or co-occurring features are abnormal, the system reports the abnormal information to the upper-level module, triggering the window-frequency-compensation three-in-one adaptive readjustment mechanism to re-optimize the inference window, acquisition frequency, and missing compensation strategy until the data input meets the requirements of multimodal fusion inference.

5. The method as described in claim 4, characterized in that, By learning the mapping relationship between symptom features, vital sign parameters, imaging information, and triage decision-making strategies through a large multimodal feature fusion model, triage execution parameters are optimized according to a weighted fusion logic of text data, vital sign data, imaging information, and comorbid parameters. Combined with patient type, hospital resource allocation, and emergency scenario requirements, triage level, recommended departments, and treatment priority instructions are dynamically adapted to generate triage control signals, including: The standardized triage and management dataset is classified and split, including text symptom feature data, vital sign parameter data, image feature data, comorbidity association data, and triage execution label data, generating data category classification results, multimodal feature association table, and triage label matching mapping relationship; Based on the target triage decision-making strategy, the data category classification results, multimodal feature association table, and triage label matching mapping relationship are standardized. The data splitting granularity is set in combination with the inference window size, and the batch update cycle and modality compensation interval are determined according to the dynamic adjustment frequency. Based on the splitting granularity standard and batch update cycle, the data categories are grouped and integrated to form a control batch containing text symptom subsets, vital sign subsets, image feature subsets, comorbidity parameter subsets, and triage execution label subsets. Prioritize the transmission of each controlled batch, and formulate an orderly transmission plan based on the synchronization rhythm of the inference window and the requirements for dynamic adjustment of timing. Generate dataset transmission control information that includes batch number, data composition details, transmission timing plan, synchronization verification rules, and modality missing compensation rules. By learning the mapping relationship between symptom features, vital sign parameters, imaging information, comorbidity data and triage decision-making strategies through a large multimodal feature fusion model, the triage execution parameters are optimized according to the weighted fusion logic of text data, vital sign data, imaging information and comorbidity parameters. Combined with patient type, hospital resource allocation and emergency scenario requirements, the triage level, recommended departments, and treatment priority instructions are dynamically adapted to generate triage control signals with interpretable evidence.

6. A system for emergency patient condition grading and rapid triage based on a large model and multimodal perception, characterized in that, The system includes: Data acquisition module 201 is used to collect multi-source heterogeneous medical perception and clinical correlation data throughout the entire process of emergency patient triage; The threshold setting module 202 is used to set the threshold for disease classification, the range of critical vital signs, the MEWS score threshold, and the triage response time threshold based on the requirements for emergency triage accuracy, the sensitivity standard for critical illness identification, and the goal of treatment efficiency. Data processing module 203 is used to process multi-source medical perception and triage related data. It uses text signal cleaning and normalization algorithm, vital sign measurement data error calibration model, multimodal time series data alignment logic, and symptom-sign-image-medical history association feature extraction method to create a standardized triage management dataset. It reads the text feature dimension, vital sign parameter type, and time series data volume information of the dataset and sorts them according to the degree of correlation between the data and triage accuracy and treatment efficiency. The strategy formulation module 204 is used to determine the triage and collaborative decision-making strategy by combining the complexity of the disease in the dataset, the computing power resources of the system, the characteristics of the multimodal large model and the response characteristics of the medical sensing device. It sets the inference window size and dynamic adjustment frequency based on the text feature dimension and the amount of time-series data, and matches the data collection frequency of the medical sensing device. Data transmission module 205 is used to split the dataset according to the target triage decision strategy, form a control batch containing text feature data, vital sign accuracy data, image feature data and triage execution label information, and transmit the data in an orderly manner based on the inference window size and dynamic adjustment frequency; The decision execution module 206 is used to learn the mapping relationship between symptom features, vital sign parameters, imaging information and triage decision strategies through a multimodal feature fusion model. It optimizes the triage execution parameters according to the weighted fusion logic of text data, vital sign data, imaging information and comorbid parameters. Combined with patient type, hospital resource allocation and emergency scenario requirements, it dynamically adapts triage level, recommended departments, and treatment priority instructions to generate triage control signals.

7. An electronic device, characterized in that, include: First processor; The processor also includes a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the method for emergency patient condition grading and rapid triage based on large model and multimodal perception as described in any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the second processor, it implements the method for emergency patient condition classification and rapid triage based on large model and multimodal perception as described in any one of claims 1 to 5.

Images