Intelligent emergency critical value early warning system and method based on multi-modal CT image

Abstract

The application discloses an emergency critical value intelligent early warning system and method based on multi-modal CT images, and relates to the technical field of medical artificial intelligence. The system comprises a unified multi-modal basic model layer, a critical value prompt engine layer and a clinical application interface layer connected in sequence. The unified multi-modal basic model layer adopts a visual-language dual-encoder architecture, and establishes deep semantic association between images and texts by contrast learning pre-training on massive CT images and corresponding report texts. The critical value prompt engine layer comprises a learnable prompt template library and a feature adapter, which are used to guide general features to specific critical value perception tasks. The clinical application interface layer is responsible for receiving standardized DICOM images, scheduling processing and generating a push structured early warning report. The application significantly improves the explainability and clinical practicability of early warning results by combining a single unified model with an extensible prompt mechanism.

Description

Technical Field

[0001] This invention relates to the field of medical artificial intelligence technology, specifically to an intelligent early warning system and method for emergency critical values ​​based on multimodal CT images. Background Technology

[0002] With the rapid development of medical imaging technology, computed tomography (CT) has become an indispensable diagnostic tool in emergency medicine, playing a crucial role, especially in assessing life-threatening emergencies such as acute trauma, stroke, and aortic dissection. Under high-pressure work conditions such as night shifts and holidays when medical resources are relatively strained, emergency department and radiologists face massive amounts of complex imaging data, requiring them to quickly and accurately identify critical values ​​in the images to initiate immediate clinical intervention. However, manual image interpretation heavily relies on the physician's experience and focus, and under fatigue, there is a high risk of missed or delayed diagnoses, directly impacting patient survival rates and prognosis. Therefore, developing an auxiliary system capable of automatically and in real-time analyzing CT images of multiple body parts and intelligently providing early warnings of critical values ​​is an urgent clinical need with significant social value.

[0003] Currently, some technical solutions have attempted to apply artificial intelligence to the early warning of critical values ​​in medical images. For example, the prior art patent for a patient emergency examination system and method, with publication number CN118058758A, typically integrates multiple single-function models to construct a system framework capable of handling various critical values. However, such solutions are essentially simple combinations and chains of multiple narrow artificial intelligence models.

[0004] The aforementioned technical approach, based on the integration of multiple independent and narrow AI models, has inherent technical bottlenecks when dealing with complex and dynamic emergency scenarios involving whole-body CT imaging. The problems include: poor system scalability, superficial understanding of complex or occult signs, and a lack of interpretability in warning results. First, each new anatomical location or new type of critical value requires the collection of new data, training, and integration of a completely new model, resulting in a bloated system architecture, high maintenance and update costs, and severely limited scalability. Second, independent classifier models are typically trained on limited, labeled disease data, resulting in a narrow "field of vision." They can only identify typical signs seen in the training set. For occult signs, rare signs, or complex critical situations involving multiple systems (such as shock-related multi-system CT complexes) where training data coverage is insufficient, their perception and understanding are superficial, their generalization performance is inadequate, and they pose a high risk of missed reports. Finally, such models typically only output a "yes / no" classification label and probability, failing to provide structured analysis like a radiologist's diagnostic report. They cannot clarify the "location, characterization, quantification, and grading" of lesions, resulting in poor interpretability of warning results. Clinicians find it difficult to quickly review and trust them, thus limiting their practical utility as a reliable and agile clinical decision support tool in high-pressure scenarios such as night shifts.

[0005] In summary, existing emergency CT critical value early warning technologies, limited by their inherent architecture relying on multiple independent and narrow AI models, struggle to achieve intelligent early warnings that cover multiple parts of the body, provide a deep understanding of complex signs, and offer interpretable results. Furthermore, significant differences in diagnostic experience and skill levels between different medical institutions and physicians make it difficult to guarantee the consistency and accuracy of critical value interpretation. During peak emergency periods, the surge in patient numbers and increased workload for physicians not only increase the risk of missed diagnoses due to fatigue but also potentially delay treatment due to inconsistent diagnostic capabilities and untimely report generation. This is particularly critical for time-sensitive emergencies such as stroke, where clinical requirements necessitate image diagnosis and report generation within half an hour; traditional manual image interpretation methods struggle to meet these timeliness requirements while maintaining quality. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an intelligent early warning system and method for emergency critical values ​​based on multimodal CT images. By adopting a unified multimodal basic model pre-trained on massive image-report pairs and combining it with learnable critical value prompt templates for dynamic guidance, it can achieve in-depth understanding and structured description of complex or hidden critical signs in CT images of multiple parts of the body. This provides interpretable and scalable real-time intelligent early warning for emergency clinics, assisting doctors in making rapid decisions.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, an emergency critical value intelligent early warning system based on multimodal CT images, the system comprising: a unified multimodal basic model layer, a critical value prompting engine layer, and a clinical application interface layer connected in sequence; The unified multimodal basic model layer includes a whole-body CT multimodal basic model with a vision-language dual encoder architecture, which is used to encode the input whole-body 3D CT volume images using general vision-language features. The vision encoder is a Transformer architecture model adapted to 3D medical images, and the language encoder is a language model pre-trained on biomedical text corpus. The whole-body CT multimodal basic model is pre-trained by comparative learning based on massive CT images and corresponding radiology report texts to establish cross-modal associations between image features and text concepts. The critical value prompting engine layer includes a learnable critical value prompting template library and a feature adapter. The critical value prompting template library stores structured text prompting templates corresponding to different body parts or critical value types. The feature adapter is used to receive multimodal features from the unified multimodal base model layer and combine them with specific prompting templates retrieved from the critical value prompting template library to generate guided critical value perception features. The clinical application interface layer is used to receive CT image data in DICOM format for medical digital imaging and communication, call the critical value prompting engine layer for processing, and convert the structured text output generated by the whole-body CT multimodal basic model based on the guided critical value perception features into a standardized critical value early warning report for push.

[0008] Furthermore, the visual encoder is specifically a three-dimensional SwinTransformer architecture model based on a window attention mechanism, which is used to segment the input three-dimensional CT volume data into non-overlapping three-dimensional image blocks and extract multi-scale three-dimensional visual features through a hierarchical moving window attention mechanism.

[0009] Furthermore, the language encoder is specifically a variant BERT model that has been continuously pre-trained on biomedical literature and electronic medical record texts, used to encode radiology report texts or the structured text prompt templates into text feature vectors.

[0010] Furthermore, each prompt template in the learnable critical value prompt template library consists of a set of trainable vector parameters. The feature adapter is a multilayer perceptron used to fuse the multimodal feature vectors output by the unified multimodal base model layer with the trainable vector parameters to generate guiding features containing prior knowledge of specific critical values.

[0011] Furthermore, the pre-training objective function for the contrastive learning employs an image-text contrastive loss function. Its mathematical expression is:

[0012] in, This represents the image-text contrast loss function. This represents the number of image-text pairs in a training batch. Indicates the number of calculations currently in progress. Image-text pairs, for the summation index. Indicates the batch used to match the first For other image-text pairs being compared, iterate through the summation in the denominator. to , Indicates the first Normalized visual feature vectors obtained by encoding a CT image using a visual encoder. Indicates the relationship with the first The normalized text feature vector of a radiology report text paired with a CT image, encoded by a language encoder. and These represent the first and second digits within the batch. Feature vectors of images and text The temperature coefficient is a learnable scalar parameter. Represents an exponential function. Represents the natural logarithm function. This represents the vector dot product operation; The image-text contrast loss function The goal is to bring the features of matched image-text pairs closer together, while pushing the features of mismatched pairs further apart.

[0013] Furthermore, the structured text output is a critical value-aware summary that includes location information, qualitative description, and severity rating; The location information includes the anatomical system and specific body parts; The qualitative description includes a textual description of key radiological features; The severity grading is based on predefined clinical critical value criteria.

[0014] Furthermore, the location information, qualitative descriptions, and severity grading in the structured text output are organized according to a predefined critical value classification system based on the human anatomy system. The critical value classification system divides critical value types into multiple independent categories, including the central nervous system, circulatory system, and respiratory system, based on the main anatomical structures of the human body covered by CT images. Each category has a corresponding set of standard anatomical site descriptions and a set of key imaging signs descriptions.

[0015] Furthermore, the clinical application interface layer specifically includes an image input submodule, an inference scheduling submodule, a report generation submodule, and an early warning push submodule; The image input submodule is used to parse and standardize DICOM format CT images; The inference scheduling submodule is used to automatically select the corresponding critical value prompt template based on the DICOM header information of the image and call the critical value prompt engine layer; The report generation submodule is used to convert structured text output into a fixed-format early warning report; The warning push submodule is used to forcibly push warning reports to the target terminal through the standard interface or message service interface of the clinical information system.

[0016] On the other hand, the intelligent early warning method for critical emergency values ​​based on multimodal CT images is applicable to intelligent early warning systems for critical emergency values ​​based on multimodal CT images. The specific steps of this method are as follows: Receives emergency whole-body CT image data in DICOM format for medical digital imaging and communication; The CT image data is encoded using a visual encoder based on a unified multimodal basic model layer to obtain general visual features; Based on the scanned area information contained in the CT image data, the corresponding structured text prompt template is retrieved from the learnable prompt template library of the critical value prompt engine layer; The structured text prompt template is encoded by the language encoder of the unified multimodal basic model layer to obtain the prompt text features; By comparing the cross-modal association capabilities obtained from pre-training, the general visual features and the prompt text features are fused to generate critical value perception features after guidance. Based on the critical value perception features, the unified multimodal basic model generates structured text output containing anatomical localization, description of key imaging features, and severity grading. The structured text output is converted into a standardized critical value warning report through the clinical application interface layer and pushed to the emergency clinical terminal in real time.

[0017] Furthermore, the training process of the unified multimodal base model layer includes a pre-training phase and a prompting engine fine-tuning phase; The pre-training phase uses massive CT images and corresponding radiology report text pairs as training data. The visual encoder and language encoder are trained with image-text contrast loss and masked language modeling loss as joint optimization objectives, so that the model can learn the deep semantic relationship between medical images and descriptive text. The fine-tuning phase of the prompt engine uses a dataset labeled with standard critical value sign description text. The parameters of the visual encoder and language encoder are fixed, and only the vector parameters in the learnable critical value prompt template library and the parameters of the feature adapter are trained. The optimization goal is to make the features generated by the model under the guidance of the structured text prompt templates accurately match the features of the standard critical value sign description text.

[0018] Compared with existing technologies, this intelligent early warning system and method for emergency critical values ​​based on multimodal CT images has the following beneficial effects: I. This invention constructs a unified multimodal basic model layer and introduces a learnable critical value prompt template library and feature adapter. It dynamically combines general image features with prior critical value knowledge to generate critical value perception features with clear guidance. This eliminates the simple splicing mode of multiple independent and narrow AI models in traditional solutions. When faced with new anatomical sites or critical value types, it is only necessary to expand the prompt template library and make light adjustments. There is no need to collect data repeatedly and train a brand new model. This effectively overcomes the inherent bottleneck of the bloated system architecture and poor scalability of existing technologies, and significantly reduces the complexity and cost of system maintenance and updates.

[0019] Second, this invention pre-trains a unified multimodal basic model through comparative learning based on massive image-report pairs, enabling the model to deeply establish cross-modal associations between image features and text semantics. Guided by prompt templates, the model can generate structured text outputs that include specific anatomical locations, descriptions of key imaging features, and severity grading. This output format mimics the diagnostic reports of professional physicians, providing a clear process from localization to qualitative to grading, thereby greatly enhancing the interpretability and clinical credibility of the warning results and facilitating rapid review and decision-making by doctors.

[0020] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0022] Fig. 1 This is a diagram of the overall system architecture of the present invention; Fig. 2 This is a flowchart of the multimodal feature processing of the present invention; Fig. 3 This is a flowchart illustrating the model training and application process of the present invention. Detailed Implementation

[0023] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0024] Example like Figs. 1 to 3 As shown, in related technologies, emergency CT critical value early warning schemes based on the integration of multiple independent narrowing AI models suffer from poor scalability, insufficient understanding of complex signs, and weak interpretability of results, making it difficult to meet actual clinical needs. To solve the above technical problems, this embodiment will elaborate on the specific implementation of an intelligent early warning system and method for emergency critical values ​​based on multimodal CT images.

[0025] The emergency critical value intelligent early warning system based on multimodal CT imaging in this embodiment mainly consists of a unified multimodal basic model layer, a critical value prompting engine layer, and a clinical application interface layer connected in sequence. Each layer works collaboratively to achieve fully automated processing from CT image data input to standardized critical value early warning report output. The overall system architecture follows a modular design principle, with data transmission between layers achieved through standardized feature interaction interfaces, ensuring system stability and maintainability.

[0026] In this embodiment, the unified multimodal basic model layer is the core computing unit of the entire system. It adopts a vision-language dual encoder architecture and establishes cross-modal associations between image features and text concepts through massive data pre-training, providing a general and efficient feature foundation for subsequent critical value perception.

[0027] The visual encoder employs a Transformer architecture model adapted to 3D medical images, specifically a 3D SwinTransformer architecture model based on a window attention mechanism. This architecture can effectively handle the high-dimensional data characteristics of whole-body 3D CT volumetric images and accurately extract multi-scale visual features.

[0028] Specifically, the input to the 3D SwinTransformer is the raw whole-body 3D CT volume data. First, data preprocessing is performed. Considering the significant differences in grayscale values ​​between CT images, grayscale normalization is first applied to map the grayscale values ​​to a fixed range, eliminating the influence of differences in scanning parameters from different devices. Then, according to preset block division rules, the 3D CT volume data is divided into non-overlapping 3D image blocks. The size of the 3D image blocks needs to balance the fine granularity of feature extraction with computational efficiency, typically determined based on the average size of human anatomical structures and the minimum identifiable scale of common clinical lesions, ensuring that each image block contains meaningful anatomical information or lesion signs.

[0029] In the feature extraction stage, the 3D SwinTransformer employs a hierarchical moving window attention mechanism. This mechanism divides the entire 3D image patch set into multiple non-overlapping windows, calculating local attention within each window, effectively reducing computational complexity. The hierarchical structure consists of multiple stages, each containing two sub-steps: window attention calculation and moving window attention calculation. In the first stage, preliminary feature encoding is performed on the input 3D image patches to obtain low-scale local features. As the hierarchy deepens, the receptive field is expanded by merging adjacent 3D image patches, while simultaneously increasing the level of feature abstraction, gradually extracting mesoscale and high-scale global features. These multi-scale features correspond to anatomical structural information at different levels: low-scale features focus on subtle tissue textures, mesoscale features reflect the local morphology of organs, and high-scale features encompass the overall relationships between multiple organs, providing comprehensive feature support for subsequent identification of multiple sites and types of critical values.

[0030] The language encoder employs a variant of BERT pre-trained on biomedical text corpora. Its core function is to encode radiology report texts or structured text prompt templates into text feature vectors with semantic expressive capabilities.

[0031] In addition, the language encoder can also receive and encode clinical complaint text information (such as descriptions of patient symptoms and signs), using it as additional text input to perform multimodal fusion with imaging features and report text, thereby introducing clinical context information into critical value judgment and improving the pertinence and accuracy of early warning.

[0032] Specifically, the biomedical text corpus used for pre-training covers multiple types, including biomedical literature published in core medical journals and electronic medical record texts accumulated by hospitals. The electronic medical record texts include progress notes, examination reports, and discharge summaries in inpatient medical records, as well as visit records and diagnostic opinions in outpatient medical records, ensuring that the corpus can cover core semantic information such as common anatomical terms, descriptions of imaging signs, and disease diagnosis names in emergency scenarios.

[0033] The pre-training process employs a continuous learning approach. Initial pre-training is performed on a general biomedical corpus to learn fundamental semantic knowledge and linguistic expression rules in the biomedical field. Subsequently, a secondary pre-training process is conducted using a corpus specifically designed for radiology reports, specifically enhancing the model's encoding capabilities for imaging-related texts. During pre-training, the model gradually develops a deep understanding of biomedical texts by learning contextual dependencies, semantic similarity, and terminology association rules. In the encoding process, the text data is first segmented, converting continuous text sequences into discrete lexical units. These units are then mapped to low-dimensional vectors through an embedding layer. Finally, a multi-layer Transformer encoder calculates and outputs a fixed-dimensional text feature vector. The dimensionality of the text feature vector is consistent with that of the visual feature vector output by the visual encoder, ensuring the feasibility of subsequent cross-modal feature fusion.

[0034] The core of the pre-training of the unified multimodal base model layer is to establish cross-modal associations between image features and text features through contrastive learning. The pre-training objective function adopts the image-text contrastive loss function. Its mathematical expression is as follows:

[0035] in, This is the image-text contrast loss function, used to measure the difference in feature distance between matched image-text pairs and unmatched image-text pairs; This represents the number of image-text pairs in a training batch. Its value needs to take into account both hardware computing power and data distribution characteristics. It is usually set reasonably based on the video memory capacity of the training device and the total number of image-text pairs in the dataset to ensure the stability and efficiency of the training process. For the currently computed image-text pair index, Index other image-text pairs used for comparison in the batch; For the first Normalized visual feature vectors obtained by encoding a CT image using a visual encoder. In order to be with the first The normalized text feature vector of a radiology report text paired with a CT image, encoded by a language encoder. and The first in each batch Feature vectors of images and text; The temperature coefficient is a learnable scalar parameter, typically ranging from 0.01 to 0.5, used to adjust the smoothness of the feature distribution and avoid gradient vanishing or training instability. Represents an exponential function. Represents the natural logarithm function. This represents the vector dot product operation, used to calculate the similarity between two feature vectors.

[0036] The pre-training process for contrastive learning is as follows: First, a large-scale training dataset is constructed, with each data sample being an image-text pair, i.e., a single whole-body CT image and its corresponding radiology report text. The radiology report text needs to be preprocessed to remove irrelevant formatting information and redundant descriptions, retaining core information directly related to the image content, such as anatomical location, sign descriptions, and diagnostic conclusions.

[0037] During training, each batch is randomly selected For each image-text pair, , and its corresponding visual feature vector With text feature vectors Consider them as a matching pair, Compared with other text feature vectors within the batch , Compared with other visual feature vectors within the batch These are considered mismatched pairs. The image-text contrast loss function is then applied. The optimization enables the model to learn to narrow the feature distance between matching pairs while widening the feature distance between unmatched pairs.

[0038] During training, an adaptive optimizer is used to update the model parameters, and the learning rate is dynamically adjusted according to the training epochs. The training effect of the model is judged by monitoring the feature matching accuracy on the validation set. When the validation set accuracy tends to stabilize and no longer improves, pre-training is stopped. Through this pre-training process, the unified multimodal base model layer can grasp the deep semantic relationship between image features and text concepts, laying the foundation for subsequent critical value perception combined with text prompt templates.

[0039] In this embodiment, the critical value prompting engine layer is a key module connecting the unified multimodal basic model layer with clinical application needs. Through a learnable critical value prompting template library and feature adapter, it guides general multimodal features into perceptual features with clear critical value orientation, thereby improving the pertinence and accuracy of critical value identification.

[0040] The learnable critical value prompt template library stores structured text prompt templates corresponding to different body parts or critical value types. Each prompt template consists of a set of trainable vector parameters. The dimension of the vector parameters is consistent with the dimension of the multimodal feature vector output by the unified multimodal base model layer, ensuring that the two can be effectively fused.

[0041] Specifically, the critical value prompt template library is constructed based on a predefined critical value classification system of the human anatomical system. This classification system categorizes critical value types into multiple independent categories according to the major anatomical structures covered by CT images, including the central nervous system, circulatory system, respiratory system, abdominal system, and skeletal system. Each category has a corresponding set of standard anatomical site descriptions and a set of key imaging features descriptions, providing a basis for the design of structured text prompt templates.

[0042] Taking critical values ​​related to the central nervous system as an example, the corresponding structured text prompt template includes three core parts: anatomical location guidance, sign description guidance, and severity guidance. The anatomical location guidance section includes descriptions of standard anatomical locations such as the cerebral hemispheres, cerebellum, brainstem, and ventricles; the sign description guidance section includes descriptions of key imaging signs such as high-density shadows corresponding to cerebral hemorrhage, low-density shadows corresponding to cerebral infarction, and mass effect corresponding to intracranial hematoma; the severity guidance section includes reference descriptions of mild, moderate, and severe grading based on clinical critical value standards. Similarly, the prompt template for critical values ​​related to the circulatory system includes descriptions of anatomical locations such as the coronary arteries, aorta, and pericardium, descriptions of signs such as intimal flap shadows corresponding to aortic dissection and low-density fluid shadows corresponding to pericardial effusion, as well as corresponding severity grading guidance.

[0043] Each structured text prompt template is initialized with random vector parameters before model training. It is then iteratively updated via backpropagation during the prompt engine fine-tuning phase, gradually optimizing it into a feature vector that accurately guides the model to focus on specific critical value types. The template library is organized using a categorized index structure, with each template corresponding to a unique index identifier. These index identifiers correspond one-to-one with body parts and critical value types, facilitating rapid retrieval of the corresponding prompt template based on the scanned area in the CT image. The templates in the prompt template library can be dynamically adapted based on clinical complaint information. For example, upon receiving a complaint of 'abdominal pain,' the library automatically enhances the detection and description of critical abdominal signs.

[0044] The feature adapter is a multilayer perceptron whose core function is to receive multimodal features from the unified multimodal base model layer and combine them with specific prompt templates retrieved from the critical value prompt template library to generate guiding features containing prior knowledge of specific critical values.

[0045] Specifically, the number of layers in a multilayer perceptron is set to 2 to 4, with the choice of layer number balancing feature fusion performance and computational efficiency. Each layer is implemented using a fully connected layer, and the number of neurons in the fully connected layer is determined based on the input feature dimension and output feature requirements, typically matching or being an integer multiple of the input feature dimension. Except for the output layer, all other layers use non-linear activation functions, commonly including ReLU or GELU, to enhance the model's ability to fit complex features; the output layer uses a linear activation function to ensure that the output guiding features remain within a suitable numerical range, facilitating further processing by the subsequent model.

[0046] The feature fusion process is as follows: First, the multimodal feature vector output from the unified multimodal base model layer is summed element-wise with the retrieved cue template vector parameters. The weight coefficients are learned through training and are used to adjust the contribution ratio of multimodal features and cue template features. Then, the weighted feature vector is input into a multilayer perceptron, passing through fully connected layers and activation functions for nonlinear transformation and dimensionality mapping. Finally, the guided critical value perception feature is output. This critical value perception feature retains the general feature information extracted by the multimodal base model while incorporating prior knowledge of specific critical value types. This allows subsequent models to accurately focus on the relevant signs of the target critical value, improving the sensitivity and specificity of critical value recognition.

[0047] In this embodiment, the clinical application interface layer serves as the interaction unit between the system and actual clinical application scenarios. It is responsible for receiving, processing, scheduling, generating reports, and pushing early warnings for CT image data, ensuring that the system can be seamlessly integrated into existing clinical workflows and providing clinicians with real-time and reliable critical value early warning support.

[0048] The core function of the image input submodule is to parse and standardize DICOM format CT images. DICOM format is a standard format for medical images, containing the image data itself as well as head information such as patient information, examination parameters, and scan site.

[0049] Specifically, the image input submodule supports reading various DICOM files, including DICOM files generated by different examination types such as conventional CT scans and enhanced CT scans. During the reading process, the head information and image pixel data are first extracted from the file using a DICOM parsing library. The extracted head information includes key parameters such as the patient's examination number, scan time, scan location, slice thickness, window width, and window level. After the image pixel data is extracted, it undergoes format conversion, transforming the original pixel matrix into a tensor format that the model can process.

[0050] Standardization processing mainly includes two aspects: First, pixel value standardization. Based on the grayscale distribution characteristics of CT images, abnormal grayscale value interference is removed, and pixel values ​​are mapped to a unified numerical range to ensure that CT images from different devices and with different examination parameters have a consistent grayscale value range. Second, image size standardization. According to the input requirements of a unified multimodal basic model layer, the images are scaled or cropped to ensure that the three-dimensional dimensions of all input images are consistent, avoiding abnormal model feature extraction due to size differences. After standardization processing is completed, the image data is transmitted to the inference scheduling submodule for further processing.

[0051] The inference scheduling submodule is used to automatically select the corresponding critical value prompt template based on the DICOM header information of the image and call the critical value prompt engine layer.

[0052] Specifically, the system first receives standardized CT image data and its corresponding DICOM head information from the image input submodule. Scan site information is then extracted from the head information, typically represented by standardized anatomical terms such as head, chest, abdomen, and spine. Subsequently, the inference scheduling submodule matches the scan site information with indexes in the critical value prompt template library according to preset mapping rules. For example, when the scan site is the head, the prompt template index corresponding to critical values ​​related to the central nervous system is matched; when the scan site is the chest, the prompt template indices corresponding to critical values ​​related to the respiratory and circulatory systems are matched.

[0053] After matching is complete, the inference scheduling submodule sends a template retrieval request to the critical value prompt template library, retrieving the corresponding structured text prompt template based on the index identifier. Simultaneously, the standardized CT image data is transmitted to the unified multimodal base model layer, triggering the model to encode visual features. Once the unified multimodal base model layer outputs multimodal features, the inference scheduling submodule calls the critical value prompt engine layer, inputting the multimodal features and the retrieved prompt template into the feature adapter to initiate the critical value perception feature generation process. The inference scheduling submodule coordinates the working sequence of each module through a task scheduling mechanism, ensuring smooth data transmission and processing, and avoiding data backlog or processing delays.

[0054] The report generation submodule converts structured text output into a fixed-format warning report. The structured text output is a critical value-aware summary containing location information, qualitative descriptions, and severity ratings. It is the core result generated by the unified multimodal basic model based on guided critical value-aware features.

[0055] Specifically, the location information includes the anatomical system and the specific location. The anatomical system is determined according to the critical value classification system, such as the central nervous system and the circulatory system. The specific location is a subdivision within the anatomical system, such as the cerebral hemisphere or the coronary artery. The qualitative description is a textual description of key imaging signs, such as high-density shadows or intimal flaps, accurately reflecting the imaging characteristics of the lesion. The severity grading is based on predefined clinical critical value standards and is divided into three levels: mild, moderate, and severe. Each level corresponds to a clear range of imaging indicators to ensure the objectivity and consistency of the grading results.

[0056] The report generation submodule organizes the aforementioned structured information into a standardized early warning report according to a fixed report format. The report format follows clinical work practices and mainly includes the report title, basic patient information, examination information, critical value warning summary, and clinical recommendations. The report title is clearly identified as "Emergency CT Critical Value Intelligent Early Warning Report"; the basic patient information includes key information such as the patient's examination number, name, gender, and age, derived from the DICOM header information; the examination information includes the scan site, examination type, and scan time; the critical value warning summary details the location information, qualitative description, and severity classification; the clinical recommendations section provides corresponding treatment suggestions based on the severity classification, such as immediate re-examination, emergency consultation, and initiation of emergency procedures. During the report generation process, the text is ensured to be standardized, concise, and accurate, avoiding ambiguity and facilitating quick reading and understanding by clinicians.

[0057] The early warning push submodule is used to forcibly push early warning reports to target terminals through the standard interface or message service interface of the clinical information system. Target terminals include emergency room doctor workstations, radiologist workstations, nurse station terminals, and hospital emergency command center terminals, ensuring that relevant medical staff can receive critical value early warning information in a timely manner.

[0058] Specifically, the standard interface of the clinical information system adopts the HL7FHIR interface protocol, which is widely used in the medical industry. This protocol supports standardized transmission of medical data and can seamlessly integrate with the existing clinical information systems of most hospitals. The message service interface uses WebSocket or HTTP push protocol to ensure real-time transmission of warning reports. The push process adopts a forced push mechanism. When a warning report is generated, the warning push submodule immediately sends a push request to the target terminal. After receiving the request, the target terminal notifies medical staff through pop-up reminders, sound alarms, etc., and displays the core content of the warning report on the terminal interface.

[0059] To ensure the reliability of push notifications, the early warning push module has a message retransmission mechanism. When the initial push fails, the system automatically attempts to retransmit multiple times; if multiple retransmissions still fail, a push failure log will be recorded, and a notification will be sent through a backup communication channel. Simultaneously, the system will record the push status, including push time, receiving terminal, recipient, and confirmation time, for easy tracking and management later.

[0060] In this embodiment, the intelligent early warning method for emergency critical values ​​based on multimodal CT images is applicable to the aforementioned intelligent early warning system. This method achieves fully automated early warning from CT image data to standardized early warning reports through multi-step collaborative processing. The specific steps are as follows: First, the system receives emergency whole-body CT image data in DICOM format from the clinical application interface layer via the image input submodule. Simultaneously, it receives and parses clinical complaint text information (such as descriptions of patient symptoms and signs). This image data originates from the hospital's CT scanning equipment and is transmitted to the system server via the hospital's internal network. During the reception process, the system verifies the integrity of the data transmission to ensure that the received DICOM files are complete and undamaged. For batch transmissions of image data, the system employs an asynchronous reception method to avoid system congestion due to excessive data volume, ensuring a highly efficient and stable reception process.

[0061] After receiving CT image data, the image input submodule parses and standardizes it to obtain standardized 3D CT volume data. Subsequently, the standardized 3D CT volume data is input into the visual encoder of the unified multimodal base model layer. The visual encoder adopts a 3D SwingTransformer architecture, performing 3D image block segmentation and hierarchical moving window attention calculation on the 3D CT volume data, progressively extracting low-scale, mid-scale, and high-scale visual features, and finally outputting general visual features with global semantic information. These general visual features comprehensively reflect the anatomical structures and potential lesion signs in CT images, providing a visual foundation for subsequent cross-modal fusion.

[0062] While the visual encoder performs feature encoding, the inference scheduling submodule extracts scan site information from the DICOM header information. Based on the scan site information, the inference scheduling submodule matches the corresponding structured text prompt template from the learnable prompt template library of the critical value prompt engine layer using preset mapping rules. For example, when the scan site is the abdomen, the prompt template corresponding to the abdominal system class critical value is matched. This template includes abdominal-related anatomical location guidance, sign description guidance, and severity guidance information. After the template is retrieved, it is transmitted to the language encoder of the unified multimodal base model layer.

[0063] After receiving a structured text prompt template, the language encoder performs word segmentation and embedding layer mapping to convert the text template into a low-dimensional vector. Subsequently, through multi-layer Transformer encoder computation, the semantic information of the text template is learned, ultimately outputting fixed-dimensional prompt text features. These prompt text features contain prior knowledge of specific urgency value types, providing semantic guidance for subsequent feature fusion.

[0064] The unified multimodal base model layer fuses general visual features and cue text features through cross-modal association capabilities acquired during pre-training. The fusion process first calculates the semantic similarity between the two features using vector dot product operations, adjusts the fusion weights based on the similarity, and then inputs the weighted feature vector into a feature adapter. The feature adapter uses a non-linear transformation of a multilayer perceptron to convert the fused features into guided crisis value perception features. These features focus on information relevant to specific crisis value types, effectively improving the accuracy of subsequent structured text output.

[0065] The unified multimodal base model layer generates structured text output containing anatomical localization, key imaging features, and severity grading based on guided critical value perception features. During generation, the model semantically analyzes the critical value perception features to determine the corresponding anatomical system and specific location of the lesion, extracts key imaging features and describes them in text, and grades the severity of the lesion according to predefined clinical critical value criteria. The structured text output follows a predefined critical value classification system based on human anatomy, ensuring the standardization and logical consistency of the information.

[0066] After receiving structured text output, the report generation submodule of the clinical application interface layer converts it into a standardized critical value warning report according to a fixed format. The warning report includes core content such as basic patient information, examination information, a critical value warning summary, and clinical recommendations. Subsequently, the warning push submodule forcibly pushes the warning report to target terminals such as emergency room doctor workstations and radiologist workstations through the standard interface or message service interface of the clinical information system, ensuring that medical staff receive and process critical value information in a timely manner.

[0067] In this embodiment, the training process of the unified multimodal base model layer includes a pre-training stage and a prompt engine fine-tuning stage. Through the two-stage training, it is ensured that the model has both general cross-modal correlation capabilities and can accurately adapt to the specific needs of emergency critical value early warning.

[0068] During the pre-training phase, the core objective is to enable the model to learn the deep semantic relationships between medical images and descriptive text, laying the foundation for subsequent critical value perception.

[0069] Specifically, the pre-training phase uses a massive amount of CT images paired with corresponding radiology report texts as training data. The training data comes from emergency CT examination cases from multiple tertiary hospitals, covering patients of different ages and disease types, ensuring data diversity and representativeness. The CT images include scans of multiple parts of the body, such as the head, chest, abdomen, and spine, while the radiology reports are official reports reviewed by professional physicians, containing detailed anatomical locations, descriptions of signs, and diagnostic conclusions.

[0070] The joint optimization objective of the pre-training is the image-text contrast loss and the masked language modeling loss. The image-text contrast loss is as described above. The first method is used to optimize the cross-modal matching ability of image features and text features; the second method is to enhance the language encoder's ability to understand text semantics. The method is to randomly mask some words in the radiology report text, let the model predict the masked words according to the context, and optimize the model parameters through the prediction error.

[0071] During training, batch gradient descent is used to update the parameters of the visual encoder and language encoder. The AdamW optimizer is selected, and a cosine annealing strategy is employed for the learning rate. The initial learning rate is determined based on the model parameter size and data volume, and is gradually reduced as the training epochs increase to avoid parameter oscillations in the later stages of training. The batch size is determined based on hardware computing power to ensure efficient training. During training, performance is evaluated on the validation set at regular intervals. Validation metrics include image-text matching accuracy and masked language prediction accuracy. When the validation metrics stabilize, the pre-training phase ends, and the parameters of the visual encoder and language encoder at this point are saved.

[0072] During the prompt engine fine-tuning phase, the core objective is to optimize the parameters of the learnable critical value prompt template library and feature adapter, so that the features generated by the model under the guidance of structured text prompt templates can accurately match the features of the standard critical value sign description text.

[0073] Specifically, the fine-tuning phase uses a dataset labeled with standard critical value sign descriptions. The dataset was constructed as follows: CT images and radiology reports corresponding to clinically diagnosed critical value cases were selected. Senior radiologists and emergency room physicians jointly labeled the reports, extracting standard critical value sign descriptions, including anatomical location, key signs, and severity information. After labeling, the dataset was divided into training and validation sets for model fine-tuning and performance evaluation.

[0074] During fine-tuning, the parameters of the visual encoder and language encoder obtained in the pre-training phase are fixed, preventing them from participating in gradient updates. Only the vector parameters in the learnable crisis value cue template library and the parameters of the feature adapter are trained. The optimization objective is to minimize the distance between the crisis value-aware features generated by the model and the features encoded by the language encoder from the standard crisis value sign description text. Specifically, the mean squared error loss function is used to measure the difference between the two, and the parameters of the cue template library and feature adapter are updated through backpropagation.

[0075] The learning rate for fine-tuning is set to one-tenth to one-hundredth of the learning rate used in the pre-training stage to avoid excessive learning rate leading to model parameter oscillations. The batch size for training is kept consistent with that used in the pre-training stage to ensure the stability of the training process. During training, model performance is monitored by the feature matching error and critical value recognition accuracy on the validation set. When the recognition accuracy on the validation set reaches a preset threshold and no longer improves, fine-tuning training is stopped, the final model parameters are saved, and the entire model training process is completed.

[0076] In this embodiment, the system and method effectively solve the problems of poor system scalability, insufficient understanding of complex signs, and weak interpretability of results in the prior art through the collaborative design of a unified multimodal basic model layer, a critical value prompting engine layer, and a clinical application interface layer.

[0077] In terms of scalability, the system is based on a unified multimodal base model. When adding new critical value types, there is no need to train a completely new model; only the corresponding structured text prompt template needs to be added to the prompt template library and fine-tuned, significantly reducing the system's maintenance and update costs. Regarding the understanding of complex signs, the unified multimodal base model establishes deep cross-modal associations through massive data pre-training. Combined with the guidance of prompt templates, it can better identify occult signs, rare signs, and multi-system complex critical situations, improving the system's generalization performance. In terms of result interpretability, the system's output structured text prompts include detailed information such as anatomical location, qualitative description, and severity grading, forming structured analysis results similar to professional physician reports. This facilitates review and trust by clinicians, enhancing the system's practical application value in clinical scenarios. Furthermore, the system can be seamlessly integrated into existing clinical workflows, outputting standardized critical value warning reports in real time. This provides clinicians with rapid and reliable decision support, helping to reduce the missed diagnosis rate and delayed diagnosis rate of emergency critical values, improving patient treatment prognosis, and possessing significant clinical application significance and practical value.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An intelligent early warning system for critical values ​​in emergency departments based on multimodal CT imaging, characterized in that, The system consists of: a unified multimodal basic model layer, a critical value prompting engine layer, and a clinical application interface layer, which are connected in sequence. The unified multimodal basic model layer includes a whole-body CT multimodal basic model with a vision-language dual encoder architecture, which is used to encode the input whole-body 3D CT volume images using general vision-language features. The vision encoder is a Transformer architecture model adapted to 3D medical images, and the language encoder is a language model pre-trained on biomedical text corpus. The whole-body CT multimodal basic model is pre-trained by comparative learning based on massive CT images and corresponding radiology report texts to establish cross-modal associations between image features and text concepts. The critical value prompting engine layer includes a learnable critical value prompting template library and a feature adapter. The critical value prompting template library stores structured text prompting templates corresponding to different body parts or critical value types. The feature adapter is used to receive multimodal features from the unified multimodal base model layer and combine them with specific prompting templates retrieved from the critical value prompting template library to generate guided critical value perception features. The clinical application interface layer is used to receive CT image data in DICOM format for medical digital imaging and communication, call the critical value prompting engine layer for processing, and convert the structured text output generated by the whole-body CT multimodal basic model based on the guided critical value perception features into a standardized critical value early warning report for push.

2. The intelligent early warning system for critical emergency values ​​based on multimodal CT images according to claim 1, characterized in that, The visual encoder is specifically a 3D SwinTransformer architecture model based on a window attention mechanism, which is used to segment the input 3D CT volume data into non-overlapping 3D image blocks and extract multi-scale 3D visual features through a hierarchical moving window attention mechanism.

3. The intelligent early warning system for critical emergency values ​​based on multimodal CT images according to claim 1, characterized in that, The language encoder is specifically a variant of BERT model that has been continuously pre-trained on biomedical literature and electronic medical record texts, used to encode radiology report texts or the structured text prompt templates into text feature vectors.

4. The intelligent early warning system for critical emergency values ​​based on multimodal CT images according to claim 1, characterized in that, Each prompt template in the learnable critical value prompt template library consists of a set of trainable vector parameters. The feature adapter is a multilayer perceptron used to fuse the multimodal feature vectors output by the unified multimodal base model layer with the trainable vector parameters to generate guiding features containing prior knowledge of specific critical values.

5. The intelligent early warning system for critical emergency values ​​based on multimodal CT images according to claim 1, characterized in that, The pre-training objective function for the contrastive learning employs an image-text contrast loss function. Its mathematical expression is: ； in, This represents the image-text contrast loss function. This represents the number of image-text pairs in a training batch. Indicates the number of calculations currently in progress. Image-text pairs, for summation index. Indicates the batch used to match the first For other image-text pairs being compared, iterate through the summation in the denominator. to , Indicates the first Normalized visual feature vectors obtained by encoding a CT image using a visual encoder. Indicates the relationship with the first The normalized text feature vector of a radiology report text paired with a CT image, encoded by a language encoder. and These represent the first and second digits within the batch. Feature vectors of images and text The temperature coefficient is a learnable scalar parameter. Represents an exponential function. Represents the natural logarithm function. This represents the vector dot product operation; The image-text contrast loss function The goal is to bring the features of matched image-text pairs closer together, while pushing the features of mismatched pairs further apart.

6. The intelligent early warning system for critical emergency values ​​based on multimodal CT images according to claim 1, characterized in that, The structured text output is a critical value-aware summary that includes location information, qualitative description, and severity rating. The location information includes the anatomical system and specific body parts; The qualitative description includes a textual description of key radiological features; The severity grading is based on predefined clinical critical value criteria.

7. The emergency critical value intelligent early warning system based on multimodal CT images according to claim 6, characterized in that, The location information, qualitative descriptions, and severity grading in the structured text output are organized according to a predefined critical value classification system based on the human anatomy system. The critical value classification system divides critical value types into multiple independent categories, including the central nervous system, circulatory system, and respiratory system, based on the main anatomical structures of the human body covered by CT images. Each category has a corresponding set of standard anatomical site descriptions and a set of key imaging signs descriptions.

8. The intelligent early warning system for critical emergency values ​​based on multimodal CT images according to claim 1, characterized in that, The clinical application interface layer specifically includes an image input submodule, an inference scheduling submodule, a report generation submodule, and an early warning push submodule; The image input submodule is used to parse and standardize DICOM format CT images; The inference scheduling submodule is used to automatically select the corresponding critical value prompt template based on the DICOM header information of the image and call the critical value prompt engine layer; The report generation submodule is used to convert structured text output into a fixed-format early warning report; The warning push submodule is used to forcibly push warning reports to the target terminal through the standard interface or message service interface of the clinical information system.

9. An intelligent early warning method for critical emergency values ​​based on multimodal CT images, applicable to the intelligent early warning system for critical emergency values ​​based on multimodal CT images as described in any one of claims 1 to 8, characterized in that, The specific steps of this method are as follows: Receives emergency whole-body CT image data in DICOM format for medical digital imaging and communication; The CT image data is encoded using a visual encoder of a unified multimodal basic model layer to obtain general visual features; Based on the scanned area information contained in the CT image data, the corresponding structured text prompt template is retrieved from the learnable prompt template library of the critical value prompt engine layer; The structured text prompt template is encoded by the language encoder of the unified multimodal basic model layer to obtain the prompt text features; By comparing the cross-modal association capabilities obtained from pre-training, the general visual features and the prompt text features are fused to generate critical value perception features after guidance. Based on the critical value perception features, the unified multimodal basic model generates structured text output containing anatomical localization, description of key imaging features, and severity grading. The structured text output is converted into a standardized critical value warning report through the clinical application interface layer and pushed to the emergency clinical terminal in real time.

10. The intelligent early warning method for emergency critical values ​​based on multimodal CT images according to claim 9, characterized in that, The training process of the unified multimodal base model layer includes a pre-training phase and a prompting engine fine-tuning phase; The pre-training phase uses massive CT images and corresponding radiology report text pairs as training data. The visual encoder and language encoder are trained with image-text contrast loss and masked language modeling loss as joint optimization objectives, so that the model can learn the deep semantic relationship between medical images and descriptive text. The fine-tuning phase of the prompt engine uses a dataset labeled with standard critical value sign description text. The parameters of the visual encoder and language encoder are fixed, and only the vector parameters in the learnable critical value prompt template library and the parameters of the feature adapter are trained. The optimization goal is to make the features generated by the model under the guidance of the structured text prompt templates accurately match the features of the standard critical value sign description text.

Images