A Prediction System for the Efficacy of Rehabilitation Medication Based on Multimodal Feature Fusion for Cerebral Hemorrhage

By using a multimodal feature fusion system, multidimensional data from patients with cerebral hemorrhage are collected and processed. Deep learning networks are used to extract and fuse features to generate individualized medication plans. This solves the problem that existing cerebral hemorrhage rehabilitation medication plans rely on experience-based judgment, and achieves accurate efficacy prediction and individualized treatment.

CN122907362BActive Publication Date: 2026-11-17SICHUAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611365583.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-09-04
Publication Date
2026-11-17
Estimated Expiration
2046-09-04

AI Technical Summary

Technical Problem

Current rehabilitation medication regimens for cerebral hemorrhage rely on the experience and judgment of clinicians and fail to systematically integrate multi-dimensional heterogeneous modal information, resulting in insufficient feature representation capabilities, limited prediction accuracy and generalization ability, and deep learning models cannot quantify the contribution of modal features and drug components.

Method used

The system uses a multimodal feature fusion system to collect various data such as head CT images, time-series clinical biochemical tests, and rehabilitation function assessments. It performs modal preprocessing, time-series alignment, and heterogeneous data conflict detection. High-dimensional features are extracted using a deep learning network, and personalized medication plans are generated through cross-modal deep fusion and interpretable attribution analysis.

Benefits of technology

It enables accurate prediction of the efficacy of rehabilitation medication for cerebral hemorrhage, quantifies the contribution of modal features and drug components, generates individualized medication plans, supports clinical physician interaction and real-time iterative optimization, and improves prediction accuracy and interpretability of individualized treatment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122907362B_ABST
    Figure CN122907362B_ABST
Patent Text Reader

Abstract

This invention belongs to the field of medical information processing technology and discloses a system for predicting the efficacy of rehabilitation medication for cerebral hemorrhage based on multimodal feature fusion. The system connects to various diagnostic and treatment systems such as hospital electronic medical records, imaging, and laboratory tests, as well as wearable vital sign monitoring devices, to collect seven core modalities of data, including cranial imaging and biochemical tests. It combines the pathological time characteristics of the edema peak period 72 hours after cerebral hemorrhage, the hematoma absorption period 1-3 weeks, and the neurological function recovery period 3-6 months. The system completes modal-specific preprocessing, temporal alignment, and data logic conflict detection. The system adapts deep learning models such as 3D residual convolutional networks, bidirectional long short-term memory networks, and medical-specific BERT models to different modalities to extract high-dimensional features strongly correlated with efficacy and remove redundant information. The system also models the semantic association between modalities through a cross-attention mechanism specific to cerebral hemorrhage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of medical information processing technology, specifically a system for predicting the efficacy of rehabilitation medication for cerebral hemorrhage based on multimodal feature fusion. Background Technology

[0002] Because there are individual differences among patients with cerebral hemorrhage in terms of bleeding site, hematoma volume, underlying diseases, body drug metabolism level, and compliance with rehabilitation training, the current clinical formulation of medication regimens and efficacy evaluation rely heavily on the experience judgment of clinicians, making it difficult to achieve individualized medication during the rehabilitation period of cerebral hemorrhage.

[0003] In the field of medical informatics, medical data processing and intelligent prediction technologies have been gradually applied to the diagnosis and treatment of cerebrovascular diseases. However, existing technologies for predicting the efficacy of medications for cerebral hemorrhage still have the following technical problems: Most existing solutions only use head CT imaging data or clinical biochemical test data at a single time point to build predictive models. They fail to systematically integrate multi-dimensional heterogeneous modal information throughout the entire rehabilitation cycle of cerebral hemorrhage, and cannot fully characterize the individual differences of patients and the intrinsic mechanisms of drug response. This results in insufficient feature representation ability of the model and limited predictive accuracy and generalization ability.

[0004] Existing multimodal correlation solutions mostly adopt shallow fusion methods such as simple splicing of feature dimensions and fixed weight weighted fusion. These methods fail to effectively solve core problems such as heterogeneity, dimension mismatch, temporal asynchrony, noise interference, and modal redundancy between different modal data. They are prone to problems such as feature conflict, drowning of effective information, and model overfitting, and cannot explore the potential complementary correlations and synergistic effects between different modal features.

[0005] Existing deep learning-based prediction models are mostly black box models, which can only output a single efficacy prediction result and cannot quantify the contribution of different modal features and different drug components to the efficacy. Summary of the Invention

[0006] The purpose of this invention is to provide a system for predicting the efficacy of rehabilitation medication for cerebral hemorrhage based on multimodal feature fusion, so as to solve one or more problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a system for predicting the efficacy of rehabilitation medication for cerebral hemorrhage based on multimodal feature fusion, comprising the following modules: Furthermore, the multimodal data acquisition module interfaces with the hospital's electronic medical record system, image archiving and communication system, clinical testing system, rehabilitation assessment system, and wearable vital sign monitoring device to complete the standardized acquisition of multimodal medical data throughout the rehabilitation cycle of patients with cerebral hemorrhage. The core modalities acquired include cranial CT image data, time-series clinical biochemical test data, rehabilitation function assessment data, dynamic data of medication regimens, electronic medical record text data, continuous vital sign time-series data, and real-time feedback data of rehabilitation training specifically for cerebral hemorrhage. A dynamic data quality assessment unit is set up to detect the integrity, accuracy, and temporal consistency of data in each modality in real time, automatically mark abnormal data, and trigger data re-collection or correction processes.

[0008] Furthermore, the heterogeneous data preprocessing module receives multi-dimensional multimodal data acquired by the multimodal data acquisition module. Based on the heterogeneous characteristics of different modal data and combined with the pathophysiological laws of cerebral hemorrhage rehabilitation, the peak period of edema 72 hours after hemorrhage, the hematoma absorption period of 1-3 weeks, and the neurological function recovery period of 3-6 months are the core time features. Dedicated preprocessing units for each modality are set up to complete the standardized preprocessing operations of the corresponding data for each modality. For the aforementioned image data, an algorithm adapted to the morphological characteristics of cerebral hemorrhage hematomas is used to automatically segment the hematoma lesion area. A dynamic calculation subunit for hematoma volume is set up. While completing grayscale normalization, noise reduction, and resampling, temporal features such as hematoma shrinkage rate and edema regression trend are extracted. For the aforementioned time-series clinical biochemical test data and continuous vital sign time-series data, an adaptive missing value imputation algorithm designed in combination with the clinical data distribution characteristics of cerebral hemorrhage patients is used to complete outlier removal, temporal alignment, and Z-score standardization. Subunits for extracting blood pressure fluctuation rhythm and dynamic trends of blood glucose and blood lipids are set up. For the electronic medical record text data, an optimized medical entity recognition algorithm is used to extract entities related to cerebral hemorrhage, and semantic encoding and structure conversion are completed. The optimized medical entity recognition algorithm is specifically designed for clinical text processing of cerebral hemorrhage. It pre-constructs a dedicated entity dictionary covering the bleeding site, hematoma volume grading, neurological function score, complication type, rehabilitation drug name, dosage, and route of administration. Using a combination of rule matching and semantic association, it locates and extracts core diagnostic and treatment entities from the medical record, automatically filters out irrelevant text information such as chief complaint and past medical history, and converts scattered unstructured free text into structured data with complete fields and uniform format.

[0009] The module is set up with a time alignment unit, based on three core time nodes in the entire rehabilitation cycle of cerebral hemorrhage: 72 hours after hemorrhage, 1-3 weeks, and 3-6 months after hemorrhage. The alignment nodes are dynamically adjusted in combination with the individual patient's rehabilitation progress to achieve time dimension synchronization of all multimodal data. At the same time, a heterogeneous data conflict pre-detection unit is set up to identify logical conflicts between different modal data in advance and trigger the manual verification process.

[0010] Furthermore, the domain feature extraction module sets up dedicated feature extraction units corresponding to each modality based on the data output by the heterogeneous data preprocessing module. Based on the data characteristics of each modality and the pathological mechanism of cerebral hemorrhage rehabilitation, a deep learning network is used to complete the adaptive extraction of high-dimensional features. For the aforementioned image data, an optimized 3D residual convolutional neural network is used, with sub-layers for extracting hematoma edge features and edema range correlation features, focusing on extracting radiomics deep features related to the rehabilitation efficacy of cerebral hemorrhage; for the aforementioned time-series clinical biochemical test data and continuous vital sign time-series data, an optimized bidirectional long short-term memory network is used, incorporating an attention mechanism, focusing on extracting time-series dynamic change features related to blood pressure fluctuations and hematoma absorption; for the aforementioned electronic medical record text data, an optimized medical-domain-specific BERT model is used, with a built-in semantic dictionary for cerebral hemorrhage, focusing on extracting clinical semantic features including clinical symptoms, complications, and medication responses; for the aforementioned rehabilitation function assessment data, an optimized fully connected neural network is used to extract high-dimensional representation features related to neurological function recovery scores; The optimized 3D residual convolutional neural network is designed for head CT images of cerebral hemorrhage. The network structure includes an input layer, five sets of cascaded 3D residual modules, a batch normalization layer, a global average pooling layer, a fully connected mapping layer, and an output layer. Identical residual connections are used between layers. The input layer receives a 512×512×48 three-dimensional head image tensor and performs lesion region normalization and resampling. Each set of 3D residual modules includes two 3×3×3 convolutional kernels, a batch normalization operation, a ReLU activation function, and residual skip connections. The convolution stride is uniformly 1, and the padding method is SAME padding to preserve the detailed features of the lesion edge. At the end of the network, the spatial dimension is compressed through a global average pooling layer, and then mapped to a 256-dimensional standardized image feature vector through a fully connected layer. This vector represents high-dimensional radiomics features that are strongly correlated with drug efficacy, such as hematoma volume, edema range, hemorrhage site, and lesion density. L2 regularization and random data pruning are added during network training to improve the robustness of feature extraction.

[0011] Each feature extraction unit operates in parallel and independently, outputting normalized feature vectors for each modality. A feature validity screening unit is set up to automatically remove redundant features based on the correlation between the rehabilitation efficacy of cerebral hemorrhage and the treatment efficacy.

[0012] The system adopts a distributed intelligent scheduling mechanism for multimodal data of cerebral hemorrhage, which uniformly manages the operation status of various special feature extraction units such as images, time series, and text. It dynamically allocates computing resources according to the processing complexity, data volume and clinical priority of different modal data. After all units complete feature extraction, the feature aggregation process is automatically triggered. The system uses a collaborative verification mechanism to check the temporal correspondence, dimensional matching degree and numerical rationality of each modality feature.

[0013] Furthermore, the cross-modal deep fusion module, based on the modal feature vectors output by the domain feature extraction module, adopts a cross-modal attention mechanism for cerebral hemorrhage combined with an adaptive weight allocation strategy to achieve deep nonlinear fusion of multimodal features. The cross-modal deep fusion module, through the cross-modal attention unit for cerebral hemorrhage, models the semantic association and complementary relationship between different modal features based on the characteristics of the seven modal data. The association between imaging hematoma features and blood pressure time series features, and between blood pressure time series features and rehabilitation training feedback features are the core associations. It focuses on strengthening the modal interaction related to the efficacy of medication for cerebral hemorrhage and generates intermodal interaction features. Through the modality adaptive weight allocation unit, an optimized weight calculation algorithm is adopted, which combines the contribution of each modality feature to the efficacy prediction and the clinical diagnosis and treatment priority of cerebral hemorrhage. Blood pressure data and hematoma imaging data are given higher priority than routine test data. The fusion weights are dynamically allocated to suppress the weights of noisy and redundant modalities and increase the weights of core effective modalities. Through the feature fusion unit, the interactive features and the weighted original features of each modality are concatenated and nonlinearly mapped to generate the final multimodal fusion feature vector. The nonlinear mapping stage employs a multi-layer fully connected network structure to perform layer-by-layer high-order abstraction and feature transformation on the spliced ​​hybrid features. The network learns the intrinsic correlations and interaction rules between different features layer by layer, automatically filtering redundant information and noise interference in the fused features, and focusing on retaining the core feature information that is strongly correlated with the efficacy of medication for cerebral hemorrhage rehabilitation. The mapping process dynamically adapts to the dimensional changes and data distribution characteristics of the input features, and finally outputs a standardized multimodal fusion feature vector with fixed dimensions, strong representation ability, and low noise content.

[0014] The cross-modal deep fusion module includes a modality missing fault tolerance unit. It automatically adjusts the fusion strategy based on the missing characteristics of cerebral hemorrhage rehabilitation data, and uses the same-modal temporal completion and different-modal feature substitution to deal with the modality missing problem. It also sets a real-time evaluation unit for fusion effect and dynamically adjusts the fusion parameters.

[0015] The system sets quantitative and clinically relevant criteria for judging the fusion effect. It comprehensively scores the results from four dimensions: feature discrimination ability, modal correlation degree, noise redundancy ratio, and prediction contribution improvement. Only when all four indicators reach the preset qualified thresholds is the fusion effect judged to be qualified and the fusion feature is output. If the criteria are not met, the system automatically records the current parameters and effect score, calls the optimal fusion parameter combination of similar cases, and gradually adjusts the core parameters such as cross-attention intensity and modal weight ratio. The fusion calculation is then repeated until the effect is qualified.

[0016] The fusion parameter adjustment adopts a combination of historical best reference and small-step iterative optimization. The system automatically stores the historical best fusion parameter configuration for similar patient data. When the fusion effect does not meet the standard, the best parameters in the same scenario are called first for preliminary calibration. Then, the core parameters such as cross-attention intensity, modality weight ratio, and nonlinear mapping coefficient are fine-tuned in a small and gradual manner. The fusion effect is re-evaluated immediately after each adjustment to gradually approach the optimal fusion state.

[0017] Furthermore, the efficacy prediction and attribution module constructs a cerebral hemorrhage rehabilitation medication efficacy prediction model based on the multimodal fusion feature vector output by the cross-modal deep fusion module. It also integrates an interpretable attribution analysis unit. The prediction model adopts an optimized multilayer perceptron framework and incorporates a dedicated loss function for cerebral hemorrhage rehabilitation efficacy prediction. It focuses on improving the accuracy of the neurological function improvement efficacy score and hematoma absorption efficiency prediction. It sets up an adverse reaction risk prediction subunit and outputs three core indicators of the patient's medication regimen within the preset rehabilitation period: neurological function improvement efficacy score, hematoma absorption efficiency prediction value, and adverse reaction probability. The system sets clinically acceptable error control standards for various core indicators. The deviation between the predicted value of the neurological function improvement efficacy score and the actual clinical assessment result is controlled within the reasonable range allowed by the diagnosis and treatment reference. The predicted deviation of hematoma absorption efficiency does not exceed the preset clinical reference threshold. The matching degree between the predicted result of the adverse reaction probability and the actual clinical situation meets the clinical decision reference requirements. Predictions that exceed the error range will automatically trigger the model to recalculate or prompt manual review.

[0018] The interpretability attribution analysis unit is based on the SHAP interpretability framework and uses an optimized attribution algorithm. It sets up a clinical terminology mapping unit for cerebral hemorrhage, which translates the contribution of each modality feature, each drug component, and the drug dosage to the efficacy prediction results into language that clinicians can directly understand. It generates a feature contribution ranking table, an attribution heatmap, and a clinical interpretation report, and clarifies the core influencing factors that affect drug efficacy. The core influencing factors include the bleeding site, hematoma volume, blood pressure control level, and drug dosage. The attribution visualization results are presented in a graphical format that is easy for clinicians to understand, including bar charts comparing feature contributions, heatmaps of core influencing factors, and waterfall charts of efficacy contributions. These charts show the positive promoting and negative inhibiting effects of various modal features, drug components, and drug dosages on efficacy. All graphs are labeled with common clinical terms and their level of influence. The visualization results can be viewed in full screen, exported locally, or printed on paper.

[0019] An attribution result validation unit was set up to verify the accuracy of the attribution analysis by combining historical clinical case data.

[0020] The attribution result verification unit constructs a three-dimensional clinical verification system. The first dimension verifies the consistency between the ranking of core influencing factors and clinical experience. The second dimension verifies the matching degree between the feature contribution value and the actual efficacy of historical cases. The third dimension verifies the conformity between the attribution interpretation content and clinical diagnosis and treatment guidelines. During the verification process, real diagnosis and treatment data of similar cases are retrieved for cross-comparison, and attribution results that do not conform to clinical logic are automatically recalculated.

[0021] Furthermore, the medication regimen optimization module takes maximizing the efficacy of medication, minimizing the risk of adverse reactions, and shortening the rehabilitation cycle as its three-dimensional core optimization objectives. It combines the core influencing factors output by the interpretable attribution analysis unit, incorporates the clinical guidelines for rehabilitation medication for cerebral hemorrhage, adopts a deep reinforcement learning algorithm specifically for cerebral hemorrhage, and designs a reward function specifically for cerebral hemorrhage. The neurological function score, hematoma absorption efficiency, and adverse reaction incidence rate are all core parameters of the reward function. It automatically optimizes the type of drug, dosage, frequency of administration, and duration of medication, generates multiple alternative individualized medication regimens, and outputs the efficacy prediction value, adverse reaction risk value, core influencing factors, and clinical adaptation suggestions for each regimen. The system transforms the treatment guidelines for rehabilitation medication in cerebral hemorrhage into rigid constraint rules that can be executed by the model. It embeds deep reinforcement learning into the strategy generation and reward calculation process throughout the entire process. In all decision-making steps, such as drug selection, dosage setting, frequency adjustment, and cycle planning, the reward function design incorporates clinical standard indicators.

[0022] The medication regimen optimization module has a clinician interaction function, which supports clinicians to input constraint adjustment instructions. The constraints include the patient's history of drug allergies, abnormal liver and kidney function, and clinical medication preferences. It also sets up a regimen iteration optimization unit to feed back the efficacy data and rehabilitation assessment data after clinical medication to the model in real time, and dynamically adjust the algorithm parameters of the medication regimen optimization model.

[0023] The system sets clear trigger conditions for iterative updates of the treatment plan. When the actual rate of hematoma absorption, the degree of improvement in neurological function deviates from the predicted value beyond the clinically permissible range, or when unforeseen adverse reactions occur, or when the rehabilitation stage is significantly delayed or advanced, the system automatically triggers the treatment plan re-optimization process. The system combines the latest clinical data to recalculate feature weights and medication parameters to generate a new treatment plan that is adapted to the current rehabilitation status.

[0024] The iterative optimization of medication regimens employs a mechanism of real-time data feedback and incremental updates. After clinicians apply the system-recommended regimens, they input clinical data such as the patient's actual hematoma absorption data, neurological function recovery scores, adverse reaction occurrences, and actual recovery cycle into the system in real time. The system automatically labels the efficacy data as high-quality incremental training samples and periodically feeds them back to the optimization model to incrementally update the reward function, policy network, and decision rules.

[0025] Furthermore, the clinical interactive visualization module adopts a layout specifically designed for the rehabilitation and treatment of cerebral hemorrhage, displaying the patient's multimodal data overview, efficacy prediction results, characteristic attribution heatmap, details of alternative drug regimens, and adverse reaction risk warnings. It has a data comparison function, enabling comparison of the patient's multimodal data at different time points, efficacy change trends, and medication regimen adjustment records. It also includes a data interaction unit, supporting clinicians in entering new clinical data for patients, adjusting medication regimen constraints, triggering efficacy re-prediction and regimen re-optimization, and providing a quick query function to quickly retrieve medication regimens and efficacy information for similar cases, thus assisting clinical decision-making. A hospital information system interface unit is set up, and an optimized interface protocol is adopted to automatically write the efficacy prediction report, medication plan suggestion and attribution analysis report generated by the system into the patient's electronic medical record, and support two-way data synchronization. The clinical interactive visualization module includes an operation log recording unit to record all operational behaviors. It has log retrieval and statistical functions, supporting log retrieval by patient ID, operation type, and time range. It also includes a clinical feedback collection unit, allowing doctors to input the actual application effects of medication plans and suggestions for improvement.

[0026] The beneficial effects of this invention are as follows: 1. This invention integrates with various diagnostic and treatment systems in hospitals, such as electronic medical records, imaging, and laboratory tests, as well as wearable vital sign monitoring devices, to collect seven core modal data, including cranial imaging and biochemical tests. It combines the pathological time characteristics of the 72-hour peak edema period of cerebral hemorrhage, the 1-3 week hematoma absorption period, and the 3-6 month neurological function recovery period to complete modal-specific preprocessing, time sequence alignment, and data logic conflict detection.

[0027] 2. This invention adapts deep learning models such as 3D residual convolutional networks, bidirectional long short-term memory networks, and medical-specific BERT models to different modal data, extracts high-dimensional features that are strongly correlated with therapeutic effects, and removes redundant information; it models the semantic association between modalities through a cross-attention mechanism specifically for cerebral hemorrhage, dynamically improves the weight of core modalities by combining an adaptive weight allocation strategy, and is equipped with a modality missing fault tolerance unit.

[0028] 3. This invention generates clinically readable attribution reports based on the SHAP framework, quantifying the contribution of each modality feature and drug component to the therapeutic effect; with the goals of maximizing efficacy, minimizing adverse reactions, and shortening the recovery period, it automatically generates individualized medication plans in conjunction with clinical diagnosis and treatment guidelines, supporting physician interaction constraints and real-time iterative optimization; at the same time, it connects to the hospital information system through a dedicated visualization module to achieve data synchronization and decision support. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating the overall workflow of the system of the present invention; Figure 2 This is a flowchart of the heterogeneous data preprocessing sub-process of the present invention; Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] like Figures 1 to 2 As shown, this embodiment of the invention provides a system for predicting the efficacy of rehabilitation medication for cerebral hemorrhage based on multimodal feature fusion, comprising the following modules: In this embodiment of the invention, the multimodal data acquisition module interfaces with the hospital's electronic medical record system, image archiving and communication system, clinical testing system, rehabilitation assessment system, and wearable vital sign monitoring device to complete the standardized acquisition of multimodal medical data throughout the rehabilitation cycle of patients with cerebral hemorrhage. The core modalities acquired include cranial CT image data, time-series clinical biochemical test data, rehabilitation function assessment data, dynamic data of medication regimens, electronic medical record text data, continuous vital sign time-series data, and real-time feedback data of rehabilitation training specifically for cerebral hemorrhage. The image data includes time-series sub-data of hematoma dynamic evolution, the rehabilitation function assessment data includes sub-data of dynamic scoring of neurological function deficits, and the electronic medical record text data includes sub-data of text related to cerebral hemorrhage complications. The system addresses the practical clinical needs of cerebral hemorrhage rehabilitation and treatment by establishing standardized acquisition guidelines and execution principles for seven core modalities. Head CT imaging data is collected at key time points: the peak edema period 72 hours after hemorrhage, the hematoma absorption period 1-3 weeks, and the neurological function recovery period 3-6 months. Time-series clinical biochemical test data are continuously collected at fixed time intervals, including core indicators such as complete blood count, liver and kidney function, electrolytes, and coagulation function. Dynamic data on medication regimens are updated in real-time as clinical medication is adjusted, recording all changes in drug type, dosage, frequency, and cycle. Continuous vital sign data is collected uninterruptedly through wearable monitoring devices. All modal data are stored in a standardized format uniformly used in the medical industry, accompanied by complete acquisition time, device information, and personnel identification.

[0032] The artificial intelligence model system adopts a training mode combining offline pre-training and online incremental fine-tuning. The training set, validation set, and test set are randomly divided in a 7:2:1 ratio, with a batch size of 32. The basic optimizer is AdamW, with an initial learning rate of 1e-4. A cosine annealing learning rate scheduling strategy is used to dynamically adjust the learning rate, and the weight decay coefficient is 1e-5. The model iteration training termination condition is set as follows: the validation set loss function decreases by less than 0.001 for 15 consecutive iterations, or the maximum number of iterations (200) is reached. The model evaluation metrics include mean absolute error (MAE), area under the curve (AUC), classification accuracy, and regression coefficient of determination (R²). 2 After the model completes offline pre-training, it supports online fine-tuning based on real-time data from a single center when deployed to a clinical environment.

[0033] The training data employs a multi-dimensional stratified sampling design to ensure that the dataset covers various types of cerebral hemorrhage rehabilitation patients with different bleeding sites, hematoma volume grades, degrees of neurological deficits, types of comorbidities, age groups, and underlying disease conditions. Clinical data from different treatment centers and with different treatment plans are also included.

[0034] A dynamic data quality assessment unit is set up to detect the integrity, accuracy, and temporal consistency of data in each modality in real time, automatically mark abnormal data, and trigger data re-collection or correction processes.

[0035] The data re-collection and anomaly correction process adopts an execution mode that combines automated push and manual confirmation. The system automatically generates a standardized re-collection task sheet containing abnormal data types, missing fields, standard collection requirements, and time limits for completion. This sheet is then pushed to the clinical work terminals of the attending medical staff. After completing the data re-collection or anomaly correction, the medical staff can send the data back to the system for secondary quality verification with one click. Qualified data that passes the verification is automatically included in the dataset. If the verification fails, a re-collection reminder is issued with the reason for failure noted. The entire process retains complete records of the number of re-collections, processing time, and operators.

[0036] The data quality dynamic assessment unit establishes a comprehensive quantitative assessment system for the seven modalities of data throughout the entire rehabilitation cycle of cerebral hemorrhage. The completeness assessment sets mandatory data items, minimum collection frequency, and effective data coverage thresholds for each modality of data according to clinical diagnosis and treatment guidelines, and automatically marks data that does not meet the coverage requirements as missing data. The accuracy assessment uses three rules to automatically identify abnormal data that exceeds physiological limits, does not conform to diagnosis and treatment logic, or has incorrect formatting, through verification of reasonable numerical range, verification of clinical logic, and verification of data format standardization. The temporal consistency assessment uses the onset time of the patient's cerebral hemorrhage as the absolute benchmark, verifies the timestamp order, collection time interval and matching degree of the rehabilitation stage of all modal data, and classifies the detected abnormal data into mild, moderate and severe abnormalities, triggering three processing processes: automatic correction, manual review and data re-collection, and automatically generating detailed data quality assessment reports and abnormality handling logs.

[0037] In this embodiment of the invention, the heterogeneous data preprocessing module receives multi-dimensional multimodal data acquired by the multimodal data acquisition module. Based on the heterogeneous characteristics of different modal data and combined with the pathophysiological laws of cerebral hemorrhage rehabilitation, the peak period of edema 72 hours after hemorrhage, the hematoma absorption period of 1-3 weeks, and the neurological function recovery period of 3-6 months are the core time features. A dedicated preprocessing unit for each modality is set up to complete the standardized preprocessing operation of the data corresponding to each modality. Each modality-specific preprocessing unit adopts a time-coordinated and data-interoperable execution logic. Using the onset time of cerebral hemorrhage as a unified benchmark, the preprocessing process of images, time series, and text data is started synchronously. The preprocessing progress and key data are shared between units in real time to ensure that all modality data are transmitted synchronously to the domain feature extraction module after standardization, time alignment, and conflict detection.

[0038] For the aforementioned image data, an algorithm adapted to the morphological characteristics of cerebral hemorrhage hematomas is used to automatically segment the hematoma lesion area. A dynamic calculation subunit for hematoma volume is set up. While completing grayscale normalization, noise reduction, and resampling, temporal features such as hematoma shrinkage rate and edema regression trend are extracted. For the aforementioned time-series clinical biochemical test data and continuous vital sign time-series data, an adaptive missing value imputation algorithm designed in combination with the clinical data distribution characteristics of cerebral hemorrhage patients is used to complete outlier removal, temporal alignment, and Z-score standardization. Subunits for extracting blood pressure fluctuation rhythm and dynamic trends of blood glucose and blood lipids are set up. The automatic hematoma lesion segmentation algorithm is deeply adapted to the imaging features of cerebral hemorrhage lesions. It adopts a two-stage processing flow of coarse segmentation and fine segmentation. First, the brain region localization algorithm removes background noise and irrelevant tissues in the image to complete the coarse localization of the lesion and locate the hematoma area. Then, combined with the density difference, edge contour and morphological features of the hematoma, a fine segmentation method that links edge detection and region growth is used to separate the hematoma tissue from the normal brain tissue, skull, blood vessels and other interfering areas, and completely delineate the true boundary and range of the hematoma. After the segmentation is completed, the hematoma volume is calculated in real time by a three-dimensional volume calculation method. At the same time, the automatic segmentation results are compared and calibrated with the data labeled by the clinician and the segmentation deviation is automatically corrected.

[0039] The adaptive missing value imputation algorithm fully considers the physiological indicator changes of patients with cerebral hemorrhage at different rehabilitation stages and formulates differentiated processing strategies for different types of missing data in time series data. For short-term missing data within three consecutive time steps, linear smooth interpolation of adjacent time point data is used to fill the missing data, preserving the trend of the original data. For long-term missing data exceeding three time steps, the algorithm matches the mean data of similar patients from a standardized case database based on individual characteristics such as patient age, gender, hemorrhage site, and rehabilitation stage. For discrete random missing data, a combination of pattern matching and clinical rules is used to supplement reasonable values. During the imputation process, all manually imputed data are specially marked, and clinical rationality is simultaneously verified after imputation is completed.

[0040] For the electronic medical record text data, an optimized medical entity recognition algorithm is used to extract entities related to cerebral hemorrhage, with the bleeding site, hematoma size, and complication type being the core entities, and semantic encoding and structure conversion are completed. A time-series alignment unit is set up, based on three core time nodes in the entire rehabilitation cycle of cerebral hemorrhage: 72 hours after hemorrhage, 1-3 weeks, and 3-6 months after hemorrhage are key nodes. The alignment nodes are dynamically adjusted in combination with the individual patient's rehabilitation progress to achieve synchronization of the time dimension of all multimodal data. At the same time, a heterogeneous data conflict pre-detection unit is set up to identify logical conflicts between different modal data in advance. The discrepancy between the imaging hematoma volume and the clinical assessment results is a typical data conflict scenario, which triggers the manual verification process.

[0041] The heterogeneous data conflict pre-detection unit covers the data logic verification rules for the entire clinical scenario of cerebral hemorrhage. It focuses on monitoring conflicts such as those between imaging data and clinical laboratory data, electronic medical record text data and time-series monitoring data, and rehabilitation assessment data and vital sign data. It automatically compares the hematoma volume calculated by the imaging system with the clinical description results in the electronic medical record, the blood pressure monitoring data with the blood pressure value recorded in the medical record, and the neurological function rehabilitation score with the real-time physiological indicator status. It marks conflicting data with obvious numerical contradictions and logical inconsistencies. The unit automatically generates a detailed report containing the conflict type, conflict data source, and specific conflict content, suspends the subsequent feature extraction process of the conflict data, and promptly pushes it to the clinical medical staff terminal for manual verification and correction. Only after the data is corrected and passes the second verification can it be reintegrated into the data processing flow.

[0042] The time-series alignment unit uses the onset time of the patient's cerebral hemorrhage as the baseline anchor point. It first constructs three standardized rehabilitation time windows: the peak edema period 72 hours after hemorrhage, the hematoma absorption period 1-3 weeks, and the neurological function recovery period 3-6 months. Then, it makes personalized dynamic adjustments according to the individual rehabilitation status of the patients. For patients with rapid hematoma absorption, good neurological function recovery, and no complications, the boundaries of each stage time window are appropriately narrowed earlier. For patients with slow hematoma absorption, complications such as lung infection or abnormal renal function, or delayed rehabilitation, the time window range is reasonably extended later. The unit dynamically offsets the alignment nodes to map all modal data, such as cranial imaging, biochemical tests, vital signs, and medication data, to the same rehabilitation time node.

[0043] In this embodiment of the invention, the domain feature extraction module sets up a dedicated feature extraction unit corresponding to each modality based on the data output by the heterogeneous data preprocessing module. Based on the data characteristics of each modality and the pathological mechanism of cerebral hemorrhage rehabilitation, a deep learning network is used to complete the adaptive extraction of high-dimensional features. For the aforementioned image data, an optimized 3D residual convolutional neural network is employed, with sub-layers for extracting hematoma edge features and edema extent correlation features, focusing on extracting radiomics deep features related to the rehabilitation efficacy of cerebral hemorrhage. For the aforementioned time-series clinical biochemical test data and continuous vital sign time-series data, an optimized bidirectional long short-term memory (BiLSTM) network is employed, incorporating an attention mechanism, focusing on extracting time-series dynamic change features related to blood pressure fluctuations and hematoma absorption, adapting to the core needs of blood pressure regulation and hematoma absorption monitoring during the cerebral hemorrhage rehabilitation period. For the aforementioned electronic medical record text data, an optimized medical-specific BERT model is employed, with a built-in semantic dictionary specifically for cerebral hemorrhage, focusing on extracting clinical semantic features including clinical symptoms, complications, and medication responses; a medication efficacy feature extraction sub-unit is also included. The optimized medical-domain-specific BERT model is based on the native BERT-base architecture. Its core structure consists of a 12-layer Transformer encoder, 768-dimensional hidden layers, a 12-head self-attention mechanism, and a maximum sequence length of 512. The model is first pre-trained using medical text corpora, then fine-tuned using cerebral hemorrhage medical records with a learning rate of 5e-5 and 10 rounds of fine-tuning. The input layer converts electronic medical record text into a sequence of tokens labeled with medical entities, capturing semantic associations of clinical entities such as symptoms, signs, complications, and medication history through the self-attention mechanism. The model incorporates a dedicated semantic dictionary for cerebral hemorrhage, including professional terms such as hemorrhage location, hematoma grade, neurological function score, and commonly used rehabilitation drugs, improving entity recognition and semantic encoding accuracy. Finally, it outputs a 768-dimensional textual semantic feature vector, transforming unstructured medical record text into computable high-dimensional semantic features.

[0044] The bidirectional long short-term memory (BiLSTM) network focuses on the dynamic changes of temporal physiological indicators during the recovery period of cerebral hemorrhage. The network structure consists of an input layer, a bidirectional LSTM hidden layer, a temporal attention weighting layer, a feature normalization layer, and an output layer. The input layer receives 32-dimensional clinical temporal features with a time step length of 128, covering the entire time-cycle of temporal data during the peak period of cerebral hemorrhage edema, the hematoma absorption period, and the neurological function recovery period. The bidirectional LSTM hidden layer has a dimension of 256, and the forward and backward networks simultaneously capture the historical dependence and future trends of temporal data, focusing on mining the fluctuation rhythm and changing trends of indicators such as blood pressure, heart rate, blood glucose, and blood lipids. The temporal attention layer strengthens the feature weights of key time nodes and key physiological indicators through a learnable weight matrix. Finally, a 128-dimensional temporal feature vector is output. The network training adopts an early stopping strategy to retain the temporal correlation features between physiological indicators and drug efficacy.

[0045] Each feature extraction unit operates in parallel and independently, outputting normalized feature vectors for each modality. A feature validity screening unit is set up to automatically remove redundant features based on the correlation between the rehabilitation efficacy of cerebral hemorrhage and the treatment efficacy.

[0046] The system employs a standardized normalization process that is consistent with the distribution of medical data for the original high-dimensional features extracted from all modalities. Based on the overall distribution characteristics of the full training dataset, the system uniformly maps feature data from different modalities, magnitudes, and units of measurement to a reasonable numerical range. The normalized data exhibits good horizontal comparability, and features extracted from different patients, different rehabilitation stages, and different modalities can be directly used for cross-modal fusion calculations.

[0047] The feature validity screening unit constructs a dual feature screening mechanism. The first dimension is based on clinical relevance screening, prioritizing the retention of core features directly related to hematoma absorption rate, neurological function recovery, adverse drug reactions, and blood pressure control effects. The second dimension is based on model contribution screening, eliminating redundant and invalid features with variances approaching zero, no correlation with efficacy, or multicollinearity through feature variance analysis, correlation analysis, and importance ranking. After screening, the unit performs unified standardization and normalization on all retained valid features, mapping feature data of different modalities and magnitudes to a unified numerical range, eliminating calculation bias caused by differences in feature dimensions.

[0048] In this embodiment of the invention, the cross-modal deep fusion module, based on the modal feature vectors output by the domain feature extraction module, adopts a cross-modal attention mechanism for cerebral hemorrhage combined with an adaptive weight allocation strategy to achieve deep nonlinear fusion of multimodal features. The cross-modal deep fusion module, through the cross-modal attention unit for cerebral hemorrhage, models the semantic association and complementary relationship between different modal features based on the characteristics of the seven modal data. The association between image hematoma features and blood pressure time series features, and blood pressure time series features and rehabilitation training feedback features are the core association relationships. The module focuses on strengthening the modal interaction related to the efficacy of medication for cerebral hemorrhage and generating intermodal interaction features. The proposed intermodal cross-attention mechanism aligns with the correlation between the pathological mechanism of cerebral hemorrhage and the efficacy of medication, focusing on two core clinical correlations: the relationship between blood pressure time-series characteristics and rehabilitation training feedback characteristics, and the relationship between biochemical test characteristics and medication regimen characteristics. Guided by the clinical diagnosis and treatment priorities of cerebral hemorrhage, the intermodal cross-attention mechanism strengthens the feature interaction weights of core modalities, explores the potential complementary patterns between multimodal data, and transforms scattered single-modal features into fusion features with clinical synergistic significance.

[0049] The intermodal cross-attention mechanism is customized for multimodal fusion in cerebral hemorrhage. It is constructed based on the scaling dot product attention principle. The input is the standardized feature vector extracted from the seven modalities, which are used to construct the query vector Q, key vector K, and value vector V, respectively. The core calculation formula is Attention(Q, K, V) = softmax((QK)). T ) / √d k )×V, where the feature dimension d k =64, using scaling factors to avoid softmax gradient vanishing caused by high-dimensional features; focusing on two core modal interactions of cerebral hemorrhage: blood pressure time series and rehabilitation training feedback, and biochemical tests and medication regimens, suppressing noise interference from irrelevant modalities, and using multi-head cross-attention parallel mining to uncover potential complementary associations between different modalities, generating a 512-dimensional interaction feature vector containing modal interaction information.

[0050] Through the modality adaptive weight allocation unit, an optimized weight calculation algorithm is adopted, which combines the contribution of each modality feature to the efficacy prediction and the clinical diagnosis and treatment priority of cerebral hemorrhage. Blood pressure data and hematoma imaging data are given higher priority than routine test data. The fusion weights are dynamically allocated to suppress the weights of noisy and redundant modalities and increase the weights of core effective modalities. Through the feature fusion unit, the interactive features and the weighted original features of each modality are concatenated and nonlinearly mapped to generate the final multimodal fusion feature vector. The dynamic adjustment of modal priorities is based on the patient's rehabilitation stage as the core trigger condition. During the peak edema period 72 hours after the onset of cerebral hemorrhage, the weight priority of cranial imaging and continuous blood pressure data is automatically increased. During the hematoma absorption period of 1-3 weeks, the weight priority of imaging data and biochemical test data is simultaneously increased. During the neurological function recovery period of 3-6 months, the weight priority of rehabilitation function assessment and medication feedback data is increased.

[0051] The modality adaptive weight allocation unit adopts an adaptive calculation strategy that combines clinical prior weights and model-learned weights. The prior weights for hematoma images and continuous blood pressure data are 0.2, for rehabilitation assessment data 0.15, for biochemical tests and medical record text data 0.12, and for medication regimens and rehabilitation feedback data 0.10. The unit dynamically learns the actual contribution of each modality to efficacy prediction through backpropagation and automatically adjusts the weight allocation results. All weights are normalized and summed to 1 to suppress the weight ratio of noisy and redundant modalities and maximize the feature contribution of core effective modalities. It also supports adaptive weight compensation in modality missing scenarios.

[0052] The cross-modal deep fusion module includes a modality missing fault-tolerant unit. Based on the missing characteristics of cerebral hemorrhage rehabilitation data, it automatically adjusts the fusion strategy and uses the same-modal temporal completion and different-modal feature replacement to handle the modality missing problem, ensuring the stability of the fused features and the robustness of the model. A real-time evaluation unit for fusion effect is set up to dynamically adjust the fusion parameters and ensure the effectiveness of the fused features.

[0053] The system, combined with the actual scenario of clinical data collection for cerebral hemorrhage, has formulated clear, quantifiable, and executable criteria for determining modal missing data. The effective coverage rate of single-modal data is the core criterion. When the effective coverage rate of single-modal data is lower than a set threshold, it is judged as local missing data. When there is no effective data collection for a single modality, it is judged as global missing data. When two or more core modalities have local or global missing data at the same time, it is judged as multimodal missing data. The criteria are fully adapted to the uncertainty of data collection during the rehabilitation period of cerebral hemorrhage. Different missing data levels correspond to dedicated error-tolerant processing strategies to ensure that the missing data determination is reasonable and the processing methods are appropriate and effective.

[0054] The real-time evaluation unit for fusion effect establishes a multi-dimensional fusion effect evaluation system, comprehensively evaluating the fusion results from four aspects: feature discrimination, modal correlation, noise ratio, and prediction contribution. The feature discrimination is used to measure the ability of fusion features to distinguish patients with different therapeutic responses. The modal correlation is used to verify the mining effect of inter-modal interaction features. The noise ratio is used to detect the proportion of redundant information and interference features. The prediction contribution is used to evaluate the improvement effect of fusion features on subsequent therapeutic prediction. The real-time evaluation unit for fusion effect sets a clear threshold for achieving the fusion effect. If the threshold is not reached, the core configurations such as cross-attention intensity, modal weight allocation ratio, and nonlinear mapping parameters are automatically adjusted, and multimodal feature fusion is performed again until the fusion effect meets the standard. The unit completely retains the parameter configuration, effect score, and optimization record of each fusion. Through continuous iteration and optimization of the fusion strategy, it always outputs high-quality multimodal fusion feature vectors.

[0055] The modality loss tolerance unit formulates a graded tolerance strategy for common modality loss scenarios in the clinical data collection process of cerebral hemorrhage. For the case of local data loss in a single modality, the feature is repaired by smoothing the historical time-series data of the modality loss tolerance unit to retain the core information of the modality. For the case of overall data loss in a single modality, the feature of the heteromodality with the highest correlation is automatically matched for replacement compensation. When rehabilitation assessment data is missing, neurofunctional imaging and vital signs are used as a joint replacement. In the extreme case of simultaneous loss of multiple modalities, a simplified fusion process is initiated, prioritizing the retention of three types of clinical core modal data: hematoma imaging, continuous blood pressure, and medication regimen, while abandoning the input of non-critical modal features.

[0056] In this embodiment of the invention, the efficacy prediction and attribution module is based on the multimodal fusion feature vector output by the cross-modal deep fusion module to construct a efficacy prediction model for cerebral hemorrhage rehabilitation medication. At the same time, it integrates an interpretable attribution analysis unit. The efficacy prediction model uses a dedicated loss function to optimize the fitting accuracy and improve the prediction accuracy of neurological function improvement score and hematoma absorption efficiency. Simultaneously, an adverse reaction risk prediction subunit is set up to output three core indicators of the patient's medication regimen within a preset rehabilitation period: neurological function improvement efficacy score, hematoma absorption efficiency prediction value, and adverse reaction probability, which are adapted to the needs of clinical medication safety assessment. The system sets standardized grading thresholds that align with clinical practice for three core efficacy indicators. The neurological function improvement efficacy score is divided into four levels based on the actual improvement: significant improvement, moderate improvement, mild improvement, and no improvement. The hematoma absorption efficiency prediction value is divided into three levels based on the absorption rate per unit time: rapid absorption, normal absorption, and slow absorption. The probability of adverse reactions is divided into four levels based on the risk value: high risk, medium risk, low risk, and no risk. The indicator output is accompanied by clear clinical grading results.

[0057] The system constructs a clinical linkage prediction logic based on three core indicators. Based on the pathological development of cerebral hemorrhage, the hematoma absorption efficiency is taken as the core correlation factor for the improvement of neurological function, and the drug dosage and liver and kidney function indicators are taken as the core correlation factors for the occurrence of adverse reactions. The prediction process considers the clinical causal relationship and mutual influence between the indicators.

[0058] The efficacy prediction model is a multi-task learning multilayer perceptron, specifically designed for evaluating the efficacy of medications used in the rehabilitation of patients with cerebral hemorrhage. The network structure consists of an input layer, four hidden fully connected layers, a multi-task output head, and activation layers. The input layer receives a 512-dimensional feature vector after cross-modal fusion. The hidden layers have dimensions of 1024, 512, 256, and 128 respectively. Batch normalization and GELU activation functions are added to each layer to improve feature fitting ability. The model adopts a multi-task output architecture, corresponding to the efficacy score for improving neurological function, the predicted value of hematoma absorption efficiency, and the probability of adverse reactions. The output layer activation functions are Linear and Sigmoid, respectively. The dedicated loss function is a multi-task weighted loss, calculated as L = 1.0 × L_MAE (efficacy score) + 1.0 × L_MAE (hematoma absorption) + 1.0 × L_BCE (adverse reactions), balancing the prediction accuracy of the three core indicators. Gradient pruning is used during model training to prevent gradient explosion.

[0059] The interpretability attribution analysis unit is based on the SHAP interpretability framework and uses an optimized attribution algorithm. It sets up a clinical terminology mapping unit for cerebral hemorrhage to translate the contribution of each modality feature, each drug component, and the drug dosage to the efficacy prediction results into language that clinicians can directly understand. For example, "For every 5mg increase in amlodipine dosage, the blood pressure control target achievement rate increases by 12% and the neurological function score increases by 8%" and "For every 10% increase in the rate of hematoma volume reduction, the efficacy score increases by 15%" are typical interpretation examples. It generates a feature contribution ranking table, an attribution heatmap, and a clinical interpretation report to identify the core influencing factors affecting drug efficacy. The core influencing factors include the bleeding site, hematoma volume, blood pressure control level, and drug dosage. The clinical terminology mapping unit for cerebral hemorrhage constructs a dedicated mapping rule library covering the entire diagnosis and treatment scenario of cerebral hemorrhage, establishes a one-to-one correspondence between model-calculated values ​​and clinical medical language, and directly converts abstract values ​​such as feature contribution and drug influence values ​​output by the model into clinically familiar diagnostic and treatment expressions such as bleeding site, hematoma volume, blood pressure control level, and drug dosage; at the same time, it classifies influencing factors into three levels according to their contribution: key influencing factors, important influencing factors, and general influencing factors.

[0060] The attribution analysis unit employs the KernelSHAP interpretability algorithm, optimized specifically for clinical medical scenarios. Inputs include multimodal fusion features and model prediction results. It quantifies the impact of each feature, drug component, and dosage on the final therapeutic effect through marginal contribution calculation. The KernelSHAP interpretability algorithm, based on game theory principles, fairly allocates the predicted contribution of each feature, generating a feature contribution ranking table, attribution heatmap, and waterfall plot. An integrated clinical terminology mapping unit for cerebral hemorrhage automatically converts numerical contribution values ​​into clinically readable expressions. Features with an absolute contribution value > 0.1 are identified as core influencing factors, with key factors such as hemorrhage site, blood pressure control level, and medication dosage highlighted. The unit also includes an attribution result verification module, which verifies the accuracy and clinical consistency of the attribution analysis by comparing historical clinical case data with actual therapeutic effects.

[0061] An attribution result verification unit is set up to verify the accuracy of attribution analysis by combining clinical historical case data, providing clinicians with interpretable, traceable, and verifiable basis for medication adjustment.

[0062] The attribution result verification unit constructs a standardized attribution verification process. It accurately matches control cases from the historical case database that are highly similar to the current patient in terms of bleeding site, hematoma volume, recovery stage, and underlying diseases. It retrieves complete clinical information such as feature contribution data, actual drug efficacy, and adverse reaction occurrence from similar cases and performs multi-dimensional comparison and verification with the attribution analysis results of the current patient. The verification content includes three key indicators: consistency of feature contribution ranking, matching degree of core influencing factors, and deviation value between efficacy prediction and actual results. If the verification is successful, a clinical attribution report is directly generated. If there is a deviation in the verification, the contribution values ​​of each feature are recalculated, and the parameters of the attribution analysis model are optimized.

[0063] The efficacy prediction model establishes a normalized incremental update mechanism, regularly summarizing newly added clinical data on cerebral hemorrhage cases, medication efficacy data, rehabilitation assessment data, and adverse reaction occurrence data, and selecting high-quality labeled data as incremental training samples. The efficacy prediction model is updated using a small-range incremental fine-tuning method, without changing the basic architecture and core algorithm of the original model, only optimizing feature weights, prediction logic, and decision thresholds. After incremental updates, the prediction accuracy is automatically verified through a validation set.

[0064] In this embodiment of the invention, the medication regimen optimization module takes maximizing the efficacy of medication, minimizing the risk of adverse reactions, and shortening the rehabilitation period as its three-dimensional core optimization objectives. It incorporates the core influencing factors output by the interpretable attribution analysis unit and integrates the clinical norms for cerebral hemorrhage rehabilitation medication, such as blood pressure control targets, principles for the use of nerve repair drugs, and medication norms for the prevention of complications. It adopts a deep reinforcement learning algorithm specifically for cerebral hemorrhage and designs a reward function specifically for cerebral hemorrhage. The neurological function score, hematoma absorption efficiency, and incidence of adverse reactions are all core parameters of the reward function. It automatically optimizes the types of drugs, dosages, dosing frequencies, and medication periods, generating multiple alternative individualized medication regimens. At the same time, it outputs the efficacy prediction value, adverse reaction risk value, core influencing factors, and clinical adaptation suggestions corresponding to each regimen. The system comprehensively and weights multiple alternative drug regimens based on three core dimensions: efficacy prediction score, adverse reaction risk value, and estimated recovery period duration. It prioritizes and recommends regimens with better efficacy, lower risk, and shorter recovery period. Doctors can directly select the recommended regimens or make minor adjustments to parameters such as drug type and dosage based on the regimens.

[0065] The medication regimen optimization module employs the DDPG deep deterministic strategy gradient reinforcement learning algorithm. The algorithm architecture includes an Actor network, a Critic network, a target network, and an experience replay pool. The state space comprises the patient's multimodal fusion features and rehabilitation stage information, while the action space includes continuous medication parameters such as drug type, dosage, dosing frequency, and treatment cycle. The reward function is a three-dimensional weighted core objective, calculated as R = 0.6 × NIHSS neurological function score improvement rate + 0.3 × hematoma absorption rate - 0.1 × adverse reaction risk, maximizing efficacy, minimizing risk, and shortening the rehabilitation cycle. Algorithm parameters are set as a discount factor γ = 0.95, a soft update coefficient τ = 0.005, and an experience replay pool capacity of 100,000. Gaussian noise is added during training to enhance exploration capabilities, and a clinical treatment guideline constraint layer is embedded to generate 3-5 alternative medication regimens labeled with efficacy prediction and risk level.

[0066] The medication regimen optimization module has a clinician interaction function, supporting clinicians to input constraint adjustment commands. The constraints include the patient's history of drug allergies, abnormal liver and kidney function, and clinical medication preferences, triggering regimen re-optimization and efficacy re-prediction. A regimen iteration optimization unit is set up to feed back clinical efficacy data and rehabilitation assessment data to the model in real time, dynamically adjust the optimization algorithm parameters, and set up a regimen compliance verification unit to ensure that the optimized medication regimen complies with the clinical diagnosis and treatment guidelines for cerebral hemorrhage rehabilitation.

[0067] The system intelligently analyzes and transforms various constraints input by clinicians into rules, automatically converting personalized constraints such as drug allergies, abnormal liver and kidney function, and medication preferences described in text into computational constraint rules that the model can recognize and execute. During the medication regimen optimization process, the system locks in contraindications such as allergic drugs and prohibited drugs, and limits the dosage and frequency of medication within a safe range, following the medication preferences and clinical habits set by doctors throughout the process. Constraints participate in the entire process of regimen generation and optimization calculation.

[0068] The medication regimen compliance verification unit establishes a comprehensive verification rule base based on the clinical diagnosis and treatment guidelines for cerebral hemorrhage rehabilitation, drug instructions for use, and clinical drug contraindication guidelines. This includes key verification items such as drug incompatibilities, maximum dosage limits, frequency restrictions, contraindications for special populations, suitability for patients with hepatic or renal insufficiency, and drug allergies. The unit verifies each candidate medication regimen generated by the system item by item, strictly screening for compliance risks such as overdosing, incompatible drug combinations, and unreasonable dosing frequencies. Regimens that pass verification are marked as qualified and allowed for output. Regimens with compliance risks are automatically marked with specific risk points and violation criteria, and returned to the medication regimen optimization module for parameter readjustment.

[0069] Based on general compliance verification, the protocol compliance verification unit also sets up exclusive secondary safety verification rules for the special pathological conditions of patients with cerebral hemorrhage. It focuses on verifying the potential impact of drugs on intracranial pressure, coagulation function, and the progression of cerebral edema, and investigates medication regimens that may aggravate intracranial hypertension, induce hemorrhage risk, or exacerbate cerebral edema.

[0070] In this embodiment of the invention, the clinical interactive visualization module adopts a layout specifically designed for the rehabilitation and treatment of cerebral hemorrhage. It displays the patient's multimodal data overview, efficacy prediction results, feature attribution heatmap, details of alternative medication regimens, and adverse reaction risk warnings. It has a data comparison function, enabling comparison of the patient's multimodal data at different time points, efficacy trends, and medication regimen adjustment records. It also includes a data interaction unit, supporting clinicians in entering new clinical data for patients, adjusting medication regimen constraints, triggering efficacy re-prediction and regimen re-optimization, and providing a quick query function to quickly retrieve medication regimens and efficacy information for similar cases, thus assisting clinical decision-making. The multi-time point comparison function supports the custom selection of key rehabilitation nodes such as 72 hours, 1 week, 2 weeks, 1 month, 3 months, and 6 months after the onset of the disease. It simultaneously displays core information such as hematoma volume, continuous blood pressure, neurological function score, medication regimen, efficacy prediction value, and adverse reaction risk at each node, and presents the trend of change in various forms such as line graphs, bar charts, and comparison tables, showing the patient's rehabilitation progress and the effect of medication adjustment.

[0071] The similar case retrieval unit uses bleeding site, hematoma volume, rehabilitation stage, age, and underlying disease type as core matching dimensions. Employing a weighted similarity matching algorithm, it quickly filters the top five similar cases from a standardized cerebral hemorrhage rehabilitation case database. The search results comprehensively display basic information, medication regimen details, efficacy prediction data, actual clinical efficacy, adverse reactions, rehabilitation period duration, and treatment precautions for similar cases. These are sorted from highest to lowest similarity, and the matching score and matching basis are also labeled. Doctors can view the clinical application effects of similar cases to gain mature medication reference experience.

[0072] A hospital information system interface unit is set up, and an optimized interface protocol is adopted to automatically write the efficacy prediction report, medication plan suggestion and attribution analysis report generated by the system into the patient's electronic medical record, and support two-way data synchronization. When outputting efficacy prediction results and medication regimen suggestions, the system simultaneously marks the clinical applicability boundaries of the model, clearly indicating that for patients with special conditions such as severe coagulation dysfunction, advanced brain herniation, end-stage malignant tumors, and severe liver and kidney failure, the model prediction results are for reference only and should be comprehensively judged by clinicians in combination with the patient's actual condition and clinical experience.

[0073] The medical data security transmission protocol adopts a high-strength encryption standard specific to the medical industry, encrypting all diagnostic and treatment data, prediction results, and medication plans transmitted between the system and the hospital information system. All sensitive information involving patient identity is de-identified before transmission. The protocol has the characteristics of data anti-tampering, anti-loss, and traceability. Data integrity is verified in real time during transmission. The system achieves bidirectional real-time data synchronization with the hospital's electronic medical record system, imaging system, and laboratory system through standardized interfaces. It automatically archives efficacy prediction reports, individualized medication plans, and attribution analysis reports to the patient's electronic medical record, while simultaneously acquiring the latest diagnostic and treatment data from the hospital system and retaining all data transmission and operation records.

[0074] The clinical interaction visualization module includes an operation log recording unit to record all operations such as data collection, parameter adjustment, protocol optimization, and clinical feedback. It has log retrieval and statistical functions and supports searching logs by patient ID, operation type, and time range. It also includes a clinical feedback collection unit, which allows doctors to enter the actual application effect of medication protocols and improvement suggestions. Feedback data can be back-synchronized to each upstream processing module.

[0075] The clinical feedback collection unit automatically categorizes the content submitted by doctors into types such as efficacy deviation, risk misjudgment, unreasonable treatment plan, interface optimization, and function suggestions. It regularly summarizes frequently reported issues and makes targeted adjustments to the model feature weights, prediction logic, treatment plan generation rules, and interface interaction methods for issues that are frequently reported in clinical practice.

[0076] The operation log recording unit records all system operations, including data acquisition, preprocessing, feature extraction, fusion calculation, efficacy prediction, treatment plan optimization, visualization query, and data synchronization. It records detailed information on all fields, including operator identity, operation time, patient ID, operation content, operation terminal, and execution result. Log retrieval supports multi-dimensional combined queries by patient ID, operation type, time range, operator, etc. The retrieval results can generate statistical reports and be exported.

[0077] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0078] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A system for predicting the efficacy of rehabilitation medication for cerebral hemorrhage based on multimodal feature fusion, characterized in that, Includes the following modules: The multimodal data acquisition module is used to collect multimodal medical data throughout the rehabilitation cycle of patients with cerebral hemorrhage, and to perform real-time quality control and abnormal data correction on the collected data. The heterogeneous data preprocessing module is used to perform standardized preprocessing, temporal alignment, and data logic conflict detection on heterogeneous data of different modalities by combining the pathological characteristics of cerebral hemorrhage rehabilitation. The domain feature extraction module is used to extract high-dimensional features related to the efficacy of rehabilitation medication for cerebral hemorrhage in each modality by adopting a feature extraction mechanism adapted to each modality, and to remove redundant features. The cross-modal deep fusion module is used to model the semantic associations between modalities, dynamically adjust the fusion weights of each modality, generate multimodal fusion feature vectors, and dynamically tolerate and control the fusion process. The efficacy prediction and attribution module is used to build a medication efficacy prediction model based on multimodal fusion feature vectors, output core efficacy indicators, integrate interpretable attribution mechanisms, quantify the contribution of each influencing factor to efficacy, and transform it into standardized clinical expressions. The medication regimen optimization module is used to generate and iteratively optimize individualized cerebral hemorrhage rehabilitation medication regimens by combining clinical diagnosis and treatment guidelines with interactive constraints, while also completing compliance verification of the regimens. The clinical interactive visualization module is used to visualize and interact with diagnosis and treatment-related information, complete the connection with the hospital information system, and record the logs of the entire diagnosis and treatment process; The heterogeneous data preprocessing module, taking into account the heterogeneous characteristics of different modal data and the pathophysiological laws of cerebral hemorrhage rehabilitation, sets up a dedicated preprocessing unit for each modality, which is one-to-one with each modality, to complete the standardized preprocessing operation of the corresponding data for each modality. For image data, the system performs automatic segmentation of lesion areas, grayscale normalization, denoising and resampling, and extracts temporal variation features of lesions. For time-series clinical biochemical test data and continuous vital sign time-series data, outlier removal, time-series alignment and standardization were performed to extract the dynamic trend characteristics of key physiological indicators; For electronic medical record text data, complete medical entity recognition, semantic encoding and structure conversion; The heterogeneous data preprocessing module has a built-in time alignment unit and a data conflict pre-detection unit. The time alignment unit uses the core time nodes of the entire rehabilitation cycle of cerebral hemorrhage as a benchmark and dynamically adjusts the alignment nodes in combination with the individual rehabilitation progress of the patient to achieve time dimension synchronization of all multimodal data. The data conflict pre-detection unit is used to identify logical conflicts between different modalities of data in advance and trigger the manual verification process. The cross-modal deep fusion module employs a cross-attention mechanism specifically designed for cerebral hemorrhage, combined with an adaptive weight allocation strategy, to achieve deep nonlinear fusion of multimodal features. The cross-modal deep fusion module has a built-in inter-modal cross-attention unit, which is used to model the semantic association and complementary relationship between different modal features, strengthen the modal interaction related to the efficacy of drugs for cerebral hemorrhage, and generate inter-modal interaction features. The cross-modal deep fusion module has a built-in modal adaptive weight allocation unit, which is used to dynamically allocate the fusion weight of each modality by combining the contribution of each modality feature to the efficacy prediction and the clinical diagnosis and treatment priority of cerebral hemorrhage, and suppress the weight ratio of noisy and redundant modes. The built-in feature fusion unit is used to concatenate and nonlinearly map the interactive features with the weighted original features of each modality to generate the final multimodal fusion feature vector. The cross-modal deep fusion module also includes a modality missing fault tolerance unit and a fusion effect real-time evaluation unit. The modality missing fault tolerance unit is used to automatically adjust the fusion strategy based on the missing characteristics of cerebral hemorrhage rehabilitation data; the fusion effect real-time evaluation unit is used to dynamically optimize the fusion parameters.

2. The multi-modal feature fusion based drug efficacy prediction system for brain hemorrhage rehabilitation according to claim 1, characterized in that, The multimodal data acquisition module interfaces with the hospital's electronic medical record system, image archiving and communication system, clinical testing system, rehabilitation assessment system, and wearable vital sign monitoring device to complete the standardized acquisition of multimodal medical data throughout the rehabilitation cycle of patients with cerebral hemorrhage. The core modal data acquired includes cranial imaging data, time-series clinical biochemical test data, rehabilitation function assessment data, dynamic data of medication regimens, electronic medical record text data, continuous vital sign time-series data, and real-time feedback data of rehabilitation training specifically for cerebral hemorrhage. The multimodal data acquisition module has a built-in dynamic data quality assessment unit, which is used to detect the integrity, accuracy and temporal consistency of each modality data in real time, automatically mark abnormal data and trigger data supplementation or correction processes.

3. The multi-modal feature fusion based drug efficacy prediction system for brain hemorrhage rehabilitation according to claim 2, characterized in that, The domain-specific feature extraction module is equipped with dedicated feature extraction units corresponding to each modality. Based on the data characteristics of each modality and the pathological mechanism of cerebral hemorrhage rehabilitation, a deep learning network adapted to each modality is used to adaptively extract high-dimensional features related to the efficacy of medication. Each dedicated feature extraction unit runs in parallel and independently, and outputs the normalized feature vector of the corresponding modality. The domain-specific feature extraction module has a built-in feature validity screening unit, which is used to automatically remove redundant and invalid features based on the correlation between each feature and the efficacy of cerebral hemorrhage rehabilitation.

4. The multi-modal feature fusion based drug efficacy prediction system for brain hemorrhage rehabilitation according to claim 3, characterized in that, The efficacy prediction and attribution module constructs a predictive model for the efficacy of medication for cerebral hemorrhage rehabilitation based on the multimodal fusion feature vector output by the cross-modal deep fusion module. The prediction model adopts a deep learning framework adapted to the cerebral hemorrhage rehabilitation scenario. The prediction model has a built-in dedicated loss function to optimize the fitting accuracy and is used to output three core efficacy indicators: the neurological function improvement efficacy score, the predicted value of hematoma absorption efficiency, and the probability of adverse reactions within the preset rehabilitation period.

5. The multi-modal feature fusion based drug efficacy prediction system for cerebral hemorrhage rehabilitation according to claim 4, characterized in that, The efficacy prediction and attribution module integrates an interpretable attribution analysis unit. This unit is based on an interpretability framework adapted to medical scenarios and includes a built-in clinical terminology mapping unit for cerebral hemorrhage. This unit transforms the contribution of each modality feature, drug component, and dosage to the efficacy prediction results into standardized clinical expressions, generating feature contribution ranking, attribution visualization results, and clinical interpretation reports to clarify the core influencing factors affecting drug efficacy. The interpretable attribution analysis unit also includes an attribution result verification unit, which verifies the accuracy and clinical consistency of the attribution analysis results by combining historical clinical case data.

6. The multi-modal feature fusion based drug efficacy prediction system for brain hemorrhage rehabilitation according to claim 5, characterized in that, The medication regimen optimization module takes maximizing the efficacy of medication, minimizing the risk of adverse reactions, and shortening the rehabilitation cycle as its three-dimensional core optimization objectives. It combines the core influencing factors output by the interpretable attribution analysis unit, incorporates the clinical diagnosis and treatment guidelines for cerebral hemorrhage rehabilitation medication, adopts a deep reinforcement learning algorithm specifically for cerebral hemorrhage scenarios, designs an appropriate reward function, automatically optimizes drug types, drug dosages, dosing frequency and medication cycle, generates multiple sets of alternative individualized medication regimens, and outputs the efficacy prediction value, adverse reaction risk value, core influencing factors and clinical adaptation suggestions for each regimen. The medication regimen optimization module supports clinicians in inputting constraints, which include the patient's history of drug allergies, abnormal liver and kidney function, and clinical medication preferences. The built-in scheme iteration and optimization unit is used to feed back the actual efficacy data and rehabilitation assessment data after clinical medication to the model in real time and dynamically adjust the algorithm parameters.

7. The multi-modal feature fusion based drug efficacy prediction system for cerebral hemorrhage rehabilitation according to claim 6, characterized in that, The clinical interactive visualization module adopts a dedicated interface layout for cerebral hemorrhage rehabilitation diagnosis and treatment, which is used to display the patient's multimodal data overview, efficacy prediction results, attribution visualization results, details of alternative drug regimens, and adverse reaction risk warning information; it has a multi-time point data comparison function, which can compare the multimodal data of patients at different rehabilitation stages, efficacy change trends, and medication regimen adjustment records; The clinical interaction visualization module has a built-in data interaction unit that allows clinicians to enter new clinical data for patients, adjust medication regimen constraints, and trigger efficacy re-prediction and regimen re-optimization. It also has a built-in similar case retrieval unit for searching and matching medication regimens and clinical efficacy data for similar cases to assist in clinical decision-making.

8. The multi-modal feature fusion based drug efficacy prediction system for brain hemorrhage rehabilitation according to claim 7, characterized in that, The clinical interactive visualization module has a built-in hospital information system interface unit and adopts a medical data security transmission protocol. It can automatically write the efficacy prediction report, medication plan suggestion, and attribution analysis report generated by the system into the patient's electronic medical record and support two-way synchronization of diagnosis and treatment data. It also has a built-in operation log recording unit and a clinical feedback collection unit. The operation log recording unit is used to record all operation behaviors of the system throughout the entire process and has log retrieval and statistical functions. The clinical feedback collection unit allows doctors to enter the actual clinical application effect of the medication plan and improvement suggestions for continuous iterative optimization of the model.

Citation Information

Patent Citations

  • Cerebral hemorrhage scalp acupuncture curative effect prediction method and system based on machine learning

    CN121281811A

  • Cerebral hemorrhage intelligent decision support system based on deep adaptive feature fusion

    CN121601212A