Method for predicting spontaneous cerebral hemorrhage progress by explaining multi-mode deep learning model
By constructing a multimodal deep learning model that combines CT images and clinical data, the problem of predicting the risk of ICH hematoma expansion was solved, and the interpretability and accuracy of the model were achieved, supporting early clinical intervention and reducing patient mortality and disability rates.
Patent Information
- Application Number
- CN202511405997.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-29
- Publication Date
- 2026-01-09
AI Technical Summary
The lack of interpretable and reliable models in the current technology for predicting the risk of hematoma expansion in acute spontaneous intracerebral hemorrhage (ICH) makes it difficult for physicians to rely on transparent decision-making processes.
An interpretable multimodal deep learning model was constructed, which combined CT image features and clinical data. The SHAP algorithm and gradient-weighted class activation mapping technique were used to rank and visualize the importance of features. ResNet and Transformer networks were used to extract image features, and multilayer perceptron was used to predict the probability of hematoma expansion.
It provides a multimodal model for rapidly and reliably predicting hematoma expansion in the acute phase, supporting early clinical identification of high-risk patients and individualized treatment, thereby reducing mortality and disability rates in ICH patients.
Smart Images
Figure CN121306535A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model, belonging to the technical field of predicting intensive care needs for patients with intracerebral hemorrhage. Background Technology
[0002] Spontaneous intracerebral hemorrhage (ICH) is a serious neurological disorder with a high mortality rate and often leads to long-term disability. Hematoma expansion (HE) is a key predictor of poor prognosis in ICH patients. ICH accounts for 10%–15% of all strokes, but its mortality and disability rates are far higher than other stroke types: approximately 40% of patients die within one month of onset, and over 75% of survivors suffer from long-term neurological dysfunction. In the acute phase, 20%–30% of patients experience hematoma expansion (HE) within 24 hours, a key modifiable factor for poor prognosis; for every 1 ml increase in hematoma volume, the risk of death increases by approximately 5%. Therefore, early identification and timely intervention of HE risk are crucial for improving the prognosis of ICH patients.8 However, there is currently no widely accepted and reliable model for acute-phase HE risk stratification.
[0003] Non-contrast computed tomography (NCCT) is the preferred imaging modality for the clinical diagnosis of acute intracerebral hemorrhage (ICH). A range of signs identified on NCCT, such as the black hole sign, fluid level sign, irregular shape sign, mixed sign, island sign, and vortex sign, have been shown to help predict hematoma expansion (HE) and patient prognosis.
[0004] In recent years, deep learning, as an emerging tool, has demonstrated great potential in predicting ICH outcomes. Integrating imaging features, deep learning-extracted features, and clinical data into a multimodal framework can capture complementary information, thereby improving prediction accuracy. However, despite its enormous potential, artificial intelligence (AI) models in the medical field often face interpretability challenges due to their "black box" nature, while doctors rely on transparent decision-making processes. Therefore, providing an interpretable and reliably accurate model for predicting ICH is an urgent problem to be solved. Summary of the Invention
[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model, which can be used to accurately and reliably predict the risk of hematoma expansion in the acute phase of acute intracerebral hemorrhage.
[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:
[0007] A method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model includes the following steps:
[0008] Step S1: Collect medical data of patients with spontaneous intracerebral hemorrhage, including CT images and clinical data;
[0009] Step S2: After sorting and filtering the medical data of all patients with spontaneous cerebral hemorrhage, the data is divided into training set, test set and external validation set.
[0010] Step S3: After preprocessing the CT image, the entire hematoma region is segmented from the image and the hematoma region is taken as the volume of interest; and the imaging signs related to hematoma expansion in the CT image are evaluated to construct a structured feature matrix containing clinical variables and imaging signs.
[0011] Step S4: Construct a multimodal prediction model and output the probability of hematoma expansion from the multimodal prediction model;
[0012] Step S5: Evaluate the performance and clinical applicability of the multimodal prediction model using an external validation set;
[0013] Step S6: Use the SHAP algorithm to rank the importance of structured features and interpret the model output results; at the same time, use gradient-weighted class activation mapping technology to visualize the image learning process of the prediction model.
[0014] Furthermore, in step S3, the image is preprocessed, specifically including the following steps:
[0015] Step S301: Read the original CT images and brain parenchymal hematoma images of each patient and sort them according to the CT slice location;
[0016] Step S302: Use the OpenCV library to scale each CT slice to the target size;
[0017] Step S303: Linearly map the pixel values of the brain parenchymal hematoma image to the [0,1] interval; for DICOM images with fewer than 28 layers, use one-dimensional linear interpolation; while for images with more than 28 layers, use equidistant sampling to unify the number of layers to 28.
[0018] Furthermore, in step S3, the entire hematoma region is segmented from the image, and the hematoma region is taken as the volume of interest; specifically, this includes the following steps:
[0019] Step S311: Two radiologists with more than five years of experience use 3D-Slicer to delineate and segment the volume of interest along the hematoma contour in baseline CT scans and follow-up CT scans. The size of the volume of interest is calculated using the volume module of 3D-Slicer.
[0020] Step S312: If the volume of interest increases by more than 6 cm 3If the hematoma volume exceeds 33% of the baseline volume, it is defined as hematoma enlargement. Based on the definition of hematoma enlargement, all patients with spontaneous intracerebral hemorrhage are divided into two categories: hematoma enlargement and hematoma non-enlargement.
[0021] Step S313: Assign a value of 1 to the region of interest segmented from the baseline CT scan, and assign a value of 0 to the region outside the region of interest; then export the assigned binary image as a mask image in DICOM image format.
[0022] Furthermore, in step S3, the imaging features related to hematoma expansion in the CT images are evaluated, and a structured feature matrix containing clinical variables and imaging features is constructed, specifically including the following steps:
[0023] Step S321: Imaging signs closely related to hematoma expansion are evaluated by two experienced radiologists, who assess the location of the hematoma, the regularity of its shape, and whether it has ruptured into the ventricles.
[0024] Step S322: Preprocess numerical clinical variables and imaging features;
[0025] Step S323: Use SPSS data analysis software to perform univariate analysis on the structured data. The structured features include clinical variables and CT signs assessed by radiologists.
[0026] Step S324: Compare the F-values of each image feature through analysis of variance. The F-value is a statistical measure that reflects the significance of differences between groups. The F-value is used as the ranking criterion for feature selection. The size of the F-value is used as the score. The 20 features with the highest scores are selected from all candidate features. The structured features are constructed using these 20 features with the highest scores and used as the training input for the subsequent multimodal prediction model.
[0027] Furthermore, in step S322, the numerical clinical variables and imaging features are preprocessed, specifically including the following steps:
[0028] After obtaining numerical clinical variables and imaging features, a feature matrix was constructed using median interpolation;
[0029] Then, the numerical features are standardized using sklearn.preprocessing.StandardScaler, specifically including:
[0030] Calculate the mean μ and standard deviation σ for each feature on the training set;
[0031] Each feature is standardized, and the standardization calculation formula is as follows:
[0032]
[0033] Where x represents the original feature value, μ represents the mean of the feature in the training set, σ represents the standard deviation of the feature in the training set, and z represents the standardized feature value.
[0034] Furthermore, in step S4, a multimodal prediction model is constructed, and the probability of hematoma expansion is output by the multimodal prediction model. This specifically includes the following steps:
[0035] Step S41: Use a deep neural network ResNet50 with 50 layers of residual connections to perform residual learning, extract the high-order spatial features of the original CT image and reduce its dimensionality;
[0036] Step S42: Use a deep neural network ResNet18 with 18 layers of residual connections to perform feature extraction and dimensionality reduction on brain parenchymal hematoma images to supplement the original CT images.
[0037] Step S43: The features of the original CT image and the brain parenchymal hematoma image are fused by the Transformer Encoder neural network component. The Transformer Encoder neural network component uses a self-attention mechanism to capture long-range dependencies and context, thereby achieving spatial global feature fusion across CT slices.
[0038] Step S44: Use a multilayer perceptron to map clinical variables and imaging features into a shared latent space and learn the most predictive indicators of hematoma expansion.
[0039] Step S45: The global features of the CT image extracted by the deep neural network ResNet and the Transformer neural network components are concatenated with the numerical structured data features after standardization and feature selection to obtain a joint feature vector; the joint feature vector is input into the fully connected layer and mapped to the output probability value in the interval [0,1] through the sigmoid function; the closer the output probability value is to 1, the higher the risk of hematoma expansion judged by the model; the closer the value is to 0, the lower the risk.
[0040] Furthermore, in step S5, the performance and clinical applicability of the multimodal prediction model are evaluated using an external validation set, specifically including the following steps:
[0041] Based on data from an external validation set, the performance and clinical applicability of the multimodal prediction model were evaluated using receiver operating characteristic (ROC) area under the curve, calibration curve, and decision curve analysis.
[0042] By adopting the above technical solution, the present invention has the following beneficial effects:
[0043] 1. This invention develops and validates a multimodal prediction model for predicting hematoma expansion (HE) in the acute phase of intracerebral hemorrhage (ICH). The model integrates imaging features, deep learning features, and clinical data, and then uses SHAP analysis to elucidate the importance of structured features. Furthermore, gradient-weighted class activation mapping (Grad-CAM) is employed to generate local visual interpretations for image-based predictions.
[0044] 2. The clinical significance of this invention lies in providing a reliable tool for early hematoma expansion risk stratification. The multimodal prediction model of this invention can rapidly estimate the risk of hematoma expansion based on initial NCCT within 24 hours of admission, enabling clinicians to identify high-risk patients and promptly initiate individualized intensive treatment (such as adjusting antithrombotic therapy and strengthening blood pressure management) to improve prognosis. Furthermore, this model can serve as a stratification tool for clinical trials, facilitating the screening of patients with high hematoma expansion risk for targeted interventions (such as hemostatic drug trials), thereby improving trial efficiency and helping to reduce mortality and disability rates in patients with acute cerebral hemorrhage.
[0045] 3. The multimodal prediction model of this invention integrates clinical data, CT imaging features, and deep learning features to predict the risk of hematoma. Its advantage stems from the complementarity of multimodal data: clinical features (such as history of hypertension and antiplatelet therapy) reflect the patient's baseline condition; manually assessed CT features (such as vortex sign and mixed sign) indicate the dynamic changes of hematoma; and imaging features extracted by deep learning capture subtle structural information that is often ignored by traditional methods. These data together comprehensively characterize the potential mechanism of hematoma expansion.
[0046] 4. The multimodal prediction model of this invention has been externally validated, demonstrating superior performance compared to standalone CT-based deep learning models and machine learning methods based on structured data. Quantitative and visual interpretation of model decisions is achieved through SHAP and Grad-CAM. This model can rapidly predict the risk of hematoma expansion in the acute phase of acute intracerebral hemorrhage, supporting early risk stratification and individualized treatment adjustments in clinical practice. It also helps identify high-risk patients in clinical trials, providing a promising strategy for reducing mortality and disability rates in ICH patients. Attached Figure Description
[0047] Figure 1 This is a flowchart of the method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model according to the present invention;
[0048] Figure 2 This is a flowchart illustrating the enrollment of patients with spontaneous intracerebral hemorrhage according to the present invention.
[0049] Figure 3 This is a schematic diagram of the interpretable multimodal deep learning model architecture of the present invention;
[0050] Figure 4The receiver operating characteristic curves of the machine learning prediction model, deep learning prediction model, and multimodal deep learning prediction model of the present invention are shown on the training set and external validation set, respectively.
[0051] Figure 5 The calibration curves and decision curve analysis (DCA) curves of the machine learning prediction model, deep learning prediction model, and multimodal deep learning prediction model of this invention are shown.
[0052] Figure 6 This is the SHAP analysis cell diagram of the present invention;
[0053] Figure 7 This is a summary diagram of the SHAP analysis illustrating the overall impact of each feature on the prediction results, as presented in this invention. Detailed Implementation
[0054] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0055] like Figure 1 As shown in the figure, this embodiment provides a method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model, including the following steps:
[0056] Step S1: Collect medical data from patients with spontaneous intracerebral hemorrhage at three medical centers. This medical data includes CT images and clinical data. Specifically:
[0057] This embodiment collects medical data from all adult inpatients with spontaneous intracerebral hemorrhage (ICH) from hospitals A, B, and C.
[0058] Inclusion criteria included: 1. Diagnosis of cerebral hemorrhage by non-contrast CT (NCCT); 2. Baseline CT scan slice thickness ≥5mm, completed within 24 hours of symptom onset; 3. Follow-up CT scan completed within 30 hours of baseline CT scan.
[0059] Exclusion criteria include: 1. Baseline CT scan completed more than 24 hours after symptom onset; 2. Intracerebral hemorrhage secondary to tumors, aneurysms, arteriovenous malformations, or trauma; 3. Surgical intervention received before the follow-up CT scan; 4. Isolated intraventricular hemorrhage; 5. Incomplete CT images or clinical data.
[0060] Demographic and clinical data were extracted from each patient's medical record. Recorded clinical variables included age, sex, and past medical history (hypertension and hypertension duration >10 years, hyperlipidemia, respiratory diseases, atrial fibrillation, coronary artery disease, diabetes, liver disease, kidney disease, smoking, alcohol consumption, and stroke history). Blood pressure classification, systolic and diastolic blood pressure, Glasgow Coma Scale (GCS), triglycerides, total cholesterol, and blood glucose were also recorded at admission. Clinical laboratory tests included activated partial thromboplastin time (APTT) and prothrombin time (PT). Treatment-related variables, including history of anticoagulant use, history of antiplatelet use, hemostatic therapy, intracranial pressure-lowering therapy, antihypertensive therapy, analgesia and sedation therapy, and antiemetic and gastric-protective therapy, were all recorded.
[0061] The CT scan was performed in the supine position, with a tube voltage of 120kVp, a slice thickness of 5mm, and an image resolution of 512×512 pixels. The CT images were exported in DICOM image format.
[0062] Step S2: Divide the collected medical data of all patients with spontaneous intracerebral hemorrhage into a training set, a test set, and an external validation set. Specifically:
[0063] like Figure 2 As shown, in this embodiment, 80% of the 319 patients from Hospital A are assigned to the training set and 20% to the test set; the 402 patients from Hospitals B and C are combined to form an independent external validation set.
[0064] The training set is used for parameter learning and fitting of the multimodal prediction model; the test set, independent of the training set, is used to evaluate the model's performance and guide parameter tuning during the development phase, thereby avoiding overfitting; the external validation set, derived from data from different centers, is used to test the model's generalization ability and robustness in real-world clinical applications. This division ensures that the constructed model possesses both high predictive accuracy and good clinical generalizability.
[0065] Step S3: After preprocessing the CT image, the entire hematoma region is segmented from the image, and the hematoma region is taken as the volume of interest. After evaluating the imaging features related to hematoma expansion in the CT image, a structured feature matrix containing clinical variables and imaging features is constructed. Specifically:
[0066] In step S3, the image is preprocessed, specifically as follows:
[0067] Step S301: Read the original CT images and brain parenchymal hematoma images of each patient, and sort them according to the CT slice position to ensure that the CT slice order is consistent. The CT slice is an extremely thin cross-sectional image reconstructed by computer after X-ray scanning.
[0068] Step S302: Use the OpenCV library to scale each CT slice to the target size (e.g., 256×256 pixels, which is specified during initialization);
[0069] Step S303: Linearly map the pixel values of the brain parenchymal hematoma image to the [0,1] interval; for DICOM images with fewer than 28 layers, use one-dimensional linear interpolation; while for images with more than 28 layers, use equidistant sampling to unify the number of layers to 28.
[0070] The processed image is used as the model input for subsequent multimodal prediction.
[0071] In step S3, the entire hematoma region is segmented from the image, and this hematoma region is taken as the volume of interest. Specifically:
[0072] Step S311: Two radiologists with more than five years of experience use 3D-Slicer (version 5.7.0 in this embodiment) to delineate and segment the volume of interest along the hematoma contour in the baseline CT scan and follow-up CT scan. The size of the volume of interest is calculated using the volume module of 3D-Slicer.
[0073] Step S312: If the volume of interest increases by more than 6 cm 3 If the hematoma volume exceeds 33% of the baseline volume, it is defined as hematoma enlargement. Based on the definition of hematoma enlargement, all patients with spontaneous intracerebral hemorrhage are divided into two categories: hematoma enlargement and hematoma non-enlargement.
[0074] Step S313: Assign a value of 1 to the segmented volume of interest in the baseline CT scan, and assign a value of 0 to the remaining regions; then export the assigned binary image as a mask image in DICOM image format for subsequent deep learning analysis, such as... Figure 2 As shown.
[0075] In step S3, imaging features related to hematoma expansion in CT images are evaluated, and a structured feature matrix containing clinical variables and imaging features is constructed. Specifically:
[0076] Step S321: Imaging signs closely related to hematoma expansion are evaluated by two experienced radiologists. The imaging signs evaluated include mixed signs, vortex signs, black hole signs, island signs, satellite signs, fluid level signs, and irregular shape signs. In addition, the location of the hematoma, the regularity of the hematoma shape, and whether it has ruptured into the ventricle are also evaluated.
[0077] Step S322: Before constructing structured features, perform the following preprocessing:
[0078] All analyses were performed using PyCharm (version 2024.1.7) and Python (version 3.10). PyCharm is a Python integrated development environment (IDE) that includes tools to improve development efficiency;
[0079] After obtaining numerical clinical variables and imaging features, a feature matrix was constructed using median interpolation. Subsequently, the numerical features were standardized using sklearn.preprocessing.StandardScaler (scikit-learn 1.5.1), a method that improves model performance by removing the mean and scaling to unit variance.
[0080] (1) Fitting: Calculate the mean (μ) and standard deviation (σ) of each feature on the training set;
[0081] (2) Standardize each feature as shown below:
[0082]
[0083] Where x represents the original feature value, μ represents the mean of the feature in the training set, σ represents the standard deviation of the feature in the training set, and z represents the standardized feature value.
[0084] On the test and validation sets, only the StandardScaler parameters fitted on the training set are used to prevent information leakage during model evaluation.
[0085] Step S323: In this embodiment, SPSS data analysis software is used to perform univariate analysis on structured data. The structured features include clinical variables and CT signs evaluated by radiologists.
[0086] Step S324: Compare the F-scores of each imaging feature using analysis of variance (ANOVA). The F-score is a statistical measure reflecting the significance of differences between groups; the larger the F-score, the more significant the difference in distinguishing the hematoma expansion group from the non-expansion group. In this embodiment, the F-score is used as the ranking criterion for feature selection. The size of the F-score is used as the score, and the 20 features with the highest scores are selected from all candidate features. These 20 highest-scoring features are used to construct structured features and serve as the training input for the subsequent multimodal prediction model.
[0087] This embodiment uses one-way ANOVA to screen 20 features significantly associated with hematoma expansion (HE) from structured features. These features include: clinical signs (such as island sign, vortex sign, black hole sign, regular and irregular hematoma morphology), treatment measures (such as intracranial pressure-lowering therapy, hemostasis, antihypertensive therapy, analgesia and sedation), and clinical variables (such as Glasgow Coma Scale (GCS) score, systolic blood pressure, hematoma volume, APTT, PT, history of stroke, history of antiplatelet therapy, and hypertension duration >10 years). For example, "intracranial pressure-lowering therapy" is a key feature affecting prognosis; patients receiving this treatment often have more severe conditions and a higher risk of hematoma expansion, suggesting that clinicians should be cautious when lowering intracranial pressure.33 Conversely, "analgesia and sedation" has an inhibitory effect on hematoma expansion; when used appropriately, it can stabilize the patient's condition and reduce risk, creating favorable conditions for subsequent treatment.
[0088] Step S4: Construct a multimodal prediction model. Use CT images, imaging features, and clinical variables as inputs to the multimodal prediction model, and output the probability of hematoma expansion. Specifically:
[0089] like Figure 3 As shown, the architecture of the multimodal deep learning model can be divided into the following five parts: 1. Original image feature extraction; 2. Brain parenchymal hematoma image feature extraction; 3. Sequence temporal feature fusion; 4. Structured data processing; 5. Multimodal fusion and outcome classification.
[0090] Step S41: Use a deep neural network ResNet50 with 50 layers of residual connections to perform residual learning, extract the high-order spatial features of the original CT image and reduce its dimensionality to ensure alignment and computational control during subsequent fusion.
[0091] Step S42: A lighter ResNet18 deep neural network with 18 layers of residual connections is used to extract features and reduce dimensions on brain parenchymal hematoma images. The hematoma region features extracted by this network are complementary to the global features of the original CT image, which helps the model capture both the overall anatomical structure and the local information of the lesion, thereby improving the prediction performance.
[0092] Step S43: The Transformer Encoder neural network component fuses features from the original CT image and the brain parenchymal hematoma image. The Transformer Encoder neural network component utilizes a self-attention mechanism to capture long-range dependencies and context, achieving cross-slice spatial global feature fusion. The original CT image provides overall structural information of the brain tissue, reflecting the state of the brain tissue surrounding the hematoma; while the brain parenchymal hematoma image emphasizes the lesion area, highlighting the morphology and expansion of the hematoma. By fusing these two complementary types of information, the model can simultaneously utilize global structural and local lesion features, thereby improving the accuracy of hematoma expansion prediction.
[0093] Step S44: Use a multilayer perceptron (MLP) to map clinical variables and imaging features into a shared latent space and learn the most predictive indicators of hematoma expansion.
[0094] Step S45: The global features of the CT image extracted by the deep neural network ResNet and the Transformer neural network components are concatenated with the numerical structured data features after standardization and feature selection to obtain a joint feature vector; the joint feature vector is input into the fully connected layer and mapped to the output probability value in the interval [0,1] through the sigmoid function; the closer the output probability value is to 1, the higher the risk of hematoma expansion judged by the model; the closer the value is to 0, the lower the risk.
[0095] Step S5: Evaluate the performance and clinical applicability of the multimodal prediction model using an external validation set. Specifically:
[0096] This embodiment is based on data from an external validation set, such as... Figure 4 , 5 As shown, the performance and clinical applicability of the multimodal prediction model were evaluated using receiver operating characteristic (AUC), calibration curve, and decision curve analysis (DCA).
[0097] Step S6: The SHAP (SHapley Additive exPlanations) algorithm is used to rank the importance of structured features and interpret the model output. Simultaneously, Gradient Weighted Class Activation Mapping (Grad-CAM) is used to visualize the image learning process of the prediction model. This enhances the interpretability of the model prediction, quantifies and visualizes the contribution of structured data to the prediction results through SHAP, and Gradient Weighted Class Activation Mapping (Grad-CAM) can visually highlight regions in the image that play a key role in the model's prediction. Specifically:
[0098] To enhance the interpretability of multimodal prediction models, SHAP analysis is used to quantify the contribution of structured features to the prediction results. Based on game theory, the Shapley value, such as... Figure 6 , 7 As shown, SHAP visualizes the impact of each feature and is commonly used in regression and classification tasks. Figure 6 The SHAP swarm plot showed that "intracranial pressure reduction therapy", "island sign", and "mixed sign" were positively correlated with hematoma expansion, while "sedation and analgesia therapy" and "hypertensive therapy" had inhibitory effects. Figure 6 "Intracranial pressure reduction therapy," "sedation and analgesia therapy," and "hypertensive therapy" were listed as the top three most influential features. These findings emphasize the need for close monitoring, cautious use of sedatives, and timely blood pressure control during intracranial pressure management; imaging features can also provide early clinical warnings. Overall, the SHAP analysis supports early risk assessment, individualized treatment planning, and outcome evaluation, contributing to improved prognosis for patients with intracerebral hemorrhage.
[0099] To interpret the focus of the multimodal prediction model on CT images, we use Grad-CAM to visualize the attention regions. Grad-CAM generates heatmaps based on the gradient of the target class relative to the feature map of the last convolutional layer, thereby highlighting important regions in the image and demonstrating the spatial basis of the model's predictions.
[0100] The specific embodiments described above further illustrate the technical problems, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model, characterized in that, Includes the following steps: Step S1: Collect medical data of patients with spontaneous intracerebral hemorrhage, including CT images and clinical data; Step S2: After sorting and filtering the medical data of all patients with spontaneous cerebral hemorrhage, the data is divided into training set, test set and external validation set. Step S3: After preprocessing the CT image, the entire hematoma region is segmented from the image and the hematoma region is taken as the volume of interest; and the imaging signs related to hematoma expansion in the CT image are evaluated to construct a structured feature matrix containing clinical variables and imaging signs. Step S4: Construct a multimodal prediction model and output the probability of hematoma expansion from the multimodal prediction model; Step S5: Evaluate the performance and clinical applicability of the multimodal prediction model using an external validation set; Step S6: Use the SHAP algorithm to rank the importance of structured features and interpret the model output results; at the same time, use gradient-weighted class activation mapping technology to visualize the image learning process of the prediction model.
2. The method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model according to claim 1, characterized in that, In step S3, the image is preprocessed, specifically including the following steps: Step S301: Read the original CT images and brain parenchymal hematoma images of each patient and sort them according to the CT slice location; Step S302: Use the OpenCV library to scale each CT slice to the target size; Step S303: Linearly map the pixel values of the brain parenchymal hematoma image to the [0,1] interval; for DICOM images with fewer than 28 layers, use one-dimensional linear interpolation; while for images with more than 28 layers, use equidistant sampling to unify the number of layers to 28.
3. The method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model according to claim 1, characterized in that, In step S3, the entire hematoma region is segmented from the image, and the hematoma region is taken as the volume of interest; specifically, this includes the following steps: Step S311: Two radiologists with more than five years of experience use 3D-Slicer to delineate and segment the volume of interest along the hematoma contour in baseline CT scans and follow-up CT scans. The size of the volume of interest is calculated using the volume module of 3D-Slicer. Step S312: If the volume of interest increases by more than 6 cm 3 If the hematoma volume exceeds 33% of the baseline volume, it is defined as hematoma enlargement. Based on the definition of hematoma enlargement, all patients with spontaneous intracerebral hemorrhage are divided into two categories: hematoma enlargement and hematoma non-enlargement. Step S313: Assign a value of 1 to the region of interest segmented from the baseline CT scan, and assign a value of 0 to the region outside the region of interest; then export the assigned binary image as a mask image in DICOM image format.
4. The method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model according to claim 1, characterized in that, In step S3, the imaging features related to hematoma expansion in the CT images are evaluated, and a structured feature matrix containing clinical variables and imaging features is constructed. This specifically includes the following steps: Step S321: Imaging signs closely related to hematoma expansion are evaluated by two experienced radiologists, who assess the location of the hematoma, the regularity of its shape, and whether it has ruptured into the ventricles. Step S322: Preprocess numerical clinical variables and imaging features; Step S323: Use SPSS data analysis software to perform univariate analysis on the structured data. The structured features include clinical variables and CT signs assessed by radiologists. Step S324: Compare the F-values of each image feature through analysis of variance. The F-value is a statistical measure that reflects the significance of differences between groups. The F-value is used as the ranking criterion for feature selection. The size of the F-value is used as the score. The 20 features with the highest scores are selected from all candidate features. The structured features are constructed using these 20 features with the highest scores and used as the training input for the subsequent multimodal prediction model.
5. The method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model according to claim 4, characterized in that, In step S322, the numerical clinical variables and imaging features are preprocessed, specifically including the following steps: After obtaining numerical clinical variables and imaging features, a feature matrix was constructed using median interpolation; Then, the numerical features are standardized using sklearn.preprocessing.StandardScaler, specifically including: Calculate the mean μ and standard deviation σ for each feature on the training set; Each feature is standardized, and the standardization calculation formula is as follows: Where x represents the original feature value, μ represents the mean of the feature in the training set, σ represents the standard deviation of the feature in the training set, and z represents the standardized feature value.
6. The method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model according to claim 1, characterized in that, In step S4, a multimodal prediction model is constructed, and the probability of hematoma expansion is output by the multimodal prediction model; specifically, it includes the following steps: Step S41: Use a deep neural network ResNet50 with 50 layers of residual connections to perform residual learning, extract the high-order spatial features of the original CT image and reduce its dimensionality; Step S42: Use a deep neural network ResNet18 with 18 layers of residual connections to perform feature extraction and dimensionality reduction on brain parenchymal hematoma images to supplement the original CT images. Step S43: The features of the original CT image and the brain parenchymal hematoma image are fused by the Transformer Encoder neural network component. The Transformer Encoder neural network component uses a self-attention mechanism to capture long-range dependencies and context, thereby achieving spatial global feature fusion across CT slices. Step S44: Use a multilayer perceptron to map clinical variables and imaging features into a shared latent space and learn the most predictive indicators of hematoma expansion. Step S45: The global features of the CT image extracted by the deep neural network ResNet and the Transformer neural network components are concatenated with the numerical structured data features after standardization and feature selection to obtain a joint feature vector; the joint feature vector is input into the fully connected layer and mapped to the output probability value in the interval [0,1] by the sigmoid function; The closer the output probability value is to 1, the higher the risk of hematoma expansion as determined by the model; the closer the value is to 0, the lower the risk.
7. The method for predicting the progression of spontaneous intracerebral hemorrhage using an interpretable multimodal deep learning model according to claim 1, characterized in that, In step S5, the performance and clinical applicability of the multimodal prediction model are evaluated using an external validation set, specifically including the following steps: Based on data from an external validation set, the performance and clinical applicability of the multimodal prediction model were evaluated using receiver operating characteristic (ROC) area under the curve, calibration curve, and decision curve analysis.