Multimodal time sequence fusion lung adenocarcinoma meningeal metastasis prognosis risk generation method

Abstract

The application discloses a kind of lung adenocarcinoma meningeal metastasis prognosis risk generation methods of multimodal time sequence fusion, comprising: collecting the longitudinal multimodal data of lung adenocarcinoma meningeal metastasis patient, including clinical and tumor characteristics, treatment information, cerebrospinal fluid index, serum test result, brain MRI image and cerebrospinal fluid cytology image, and add time stamp for each data point;The longitudinal multimodal data is preprocessed;Construct time-aware multimodal fusion model TEMPO;TEMPO model is trained, so that it simultaneously outputs two risk indicators: central nervous system progression risk within 8 weeks and all-cause mortality risk within 6 months;Based on continuous assessment data, the comprehensive risk score of patient is dynamically updated using exponential weighted average method, and risk stratification is carried out according to the double output results.The application can process irregular follow-up and missing data, integrate multimodal longitudinal clinical data, and simultaneously predict 8-week central nervous system progression risk and 6-month mortality.

Description

Technical Field

[0001] This invention belongs to the field of computer data processing technology, specifically relating to a multimodal temporal fusion method for generating prognostic risks of lung adenocarcinoma meningeal metastasis. Background Technology

[0002] Leptomeningeal metastasis (LM) refers to a disease state in which malignant cells spread within the spaces between the pia mater and cerebrospinal fluid (CSF). It is most commonly seen in patients with lung adenocarcinoma, breast cancer, and melanoma. Historically, patients with leptomeningeal metastasis have had short survival rates. The treatment landscape has changed with the advent of targeted therapies capable of penetrating the central nervous system. However, significant heterogeneity in patient prognosis remains: some patients maintain intracranial control for months, while others deteriorate rapidly after diagnosis. This uncertainty increases the difficulty of deciding on the intensity and timing of treatment.

[0003] In contemporary neuro-oncology practice, the management of leptomeningeal metastases relies on repeated assessments of neurological function, brain MRI, and cerebrospinal fluid examinations throughout the disease course. These continuous observations are typically used to determine whether to intensify central nervous system-targeted therapy, maintain the current regimen, or adjust treatment goals, but they do not themselves translate into individualized prospective risk assessments.

[0004] Existing LM prognostic tools mainly suffer from the following technical deficiencies: 1. Limitations of static prediction: Traditional prognostic scoring systems (such as molGPA) are based only on baseline characteristics at the time of diagnosis and cannot capture the dynamic evolution of the disease. In the validation cohort, the discrimination is extremely low (C index is only about 0.51).

[0005] 2. Insufficient data processing capabilities: Routine clinical follow-up data is inherently irregular and incomplete. Traditional methods require data imputation or time binning, which can obscure important prognostic information carried in the follow-up model.

[0006] 3. Lack of multimodal integration: LM management relies on a comprehensive assessment of neurological examination, brain MRI, cerebrospinal fluid cytology and laboratory tests, but existing models cannot effectively integrate these heterogeneous data.

[0007] 4. Single outcome prediction: Existing models can only predict overall survival and cannot provide a joint estimate of short-term progression risk and intermediate mortality risk at the same time, making it difficult to guide clinical decision-making. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides a multimodal time-series fusion method for generating prognostic risks of lung adenocarcinoma meningeal metastasis. By constructing a time-aware multimodal fusion model TEMPO, it can utilize irregular multimodal longitudinal data collected in routine clinical practice to dynamically update the patient's 8-week central nervous system progression risk and 6-month all-cause mortality risk at each follow-up. The risk information is generated through a four-quadrant risk stratification framework for users to view, providing risk reference for users.

[0009] This invention discloses a multimodal temporal fusion method for generating prognostic risk of lung adenocarcinoma meningeal metastasis, comprising: Step 1: Collect longitudinal multimodal data of patients with lung adenocarcinoma and meningeal metastasis. The longitudinal multimodal data includes clinical and tumor characteristics, treatment information, cerebrospinal fluid indicators, serum test results, brain MRI images and cerebrospinal fluid cytology images, and add a timestamp to each data point. Step 2: Construct longitudinal evaluation units. Group multimodal data with an interval of no more than N days into the same evaluation unit. Use the latest date within the evaluation unit as the evaluation timestamp, and the prediction of each evaluation unit only uses the data before this timestamp. Step 3: Preprocess the longitudinal multimodal data to extract clinical data features, brain MRI image features, and cerebrospinal fluid cytology image features, respectively; Step 4: Construct the time-aware multimodal fusion model TEMPO, introduce time position encoding and modality mask, and fuse multimodal features with time gap features and missing indicators; Step 5: Train the TEMPO model to output two risk indicators simultaneously: a risk score for central nervous system progression within 8 weeks after each assessment unit and a risk score for all-cause mortality within 6 months. Step 6: Based on the patient's continuous assessment data, the patient's comprehensive risk score is dynamically updated using the index-weighted average method, and a four-quadrant risk stratification label and risk trajectory visualization result are generated according to the dual output results.

[0010] As a further improvement of the present invention, in step 2, multimodal data with a time interval of 3 to 14 days (preferably 5 days) are grouped into the same evaluation unit; the derivation queue and the validation queue are divided by time segmentation: patients diagnosed with meningeal metastasis before a specified time (e.g., July 1, 2024) are in the derivation queue, and patients diagnosed on or after the specified time are in the validation queue.

[0011] As a further improvement of the present invention, in step 3, the brain MRI image feature extraction adopts a convolutional neural network (CNN) encoder or a visual Transformer encoder, preferably a ResNet-18 encoder, and preferentially uses axial T1 weighted sequences, and uses axial T2 weighted sequences when missing; the cerebrospinal fluid cytology image feature extraction adopts a convolutional neural network (CNN) encoder or a visual Transformer encoder, and average pooling is used to obtain evaluation unit-level features when there are multiple images.

[0012] As a further improvement of the present invention, in step 4, the missing data processing method is as follows: instead of data interpolation, a modal mask is constructed to indicate whether each modality exists, and gap indicators are extracted, including whether brain MRI or cerebrospinal fluid examination has been performed within a preset time period (e.g., 28 to 120 days, preferably 84 days), and the missing patterns are incorporated into the model as part of the record.

[0013] As a further improvement of the present invention, in step 5, the model training adopts a joint loss function: the risk of central nervous system progression at 8 weeks adopts binary cross-entropy loss, and the risk of all-cause mortality at 6 months adopts Cox partial likelihood loss; during training, each mini-batch contains at most one assessment of each patient to avoid information leakage caused by patient-specific dependence.

[0014] As a further improvement of the present invention, in step 6, the formula for the exponentially weighted average method is: in, For the first k The patient-level comprehensive risk score at the time of the second assessment r j For the first j A single risk score in this assessment. For the first j Next and first k The time interval between assessments (days) γ The decay parameter is set to 0.0248 / day, with a half-life of 14~60 days (preferably 28 days). The preferred half-life is 28 days, which means that earlier evaluation results are given lower weight.

[0015] As a further improvement of the present invention, in step 6, the four-quadrant risk stratification adopts a preset risk threshold to divide patients into four quadrants: low progression and low mortality (quadrant I), high progression and low mortality (quadrant II), low progression and high mortality (quadrant III), and high progression and high mortality (quadrant IV), and generates corresponding risk stratification labels; wherein, the preset risk threshold includes one of the following: quantile threshold, ROC determination threshold, or user-configured threshold, for example, the preset risk threshold is 0.5.

[0016] As a further improvement of the present invention, it also includes a model interpretability analysis step: using the integrated gradient method to calculate the global feature importance, generating spatial attribution maps of brain MRI and cerebrospinal fluid cytology images, identifying the clinical variables and imaging regions that contribute most to risk prediction, and generating interpretable information for users to view.

[0017] As a further improvement of the present invention, the model performance was validated using time-dependent evaluation metrics: the 6-month all-cause mortality rate was evaluated using the Harrell C-index, and the 8-week central nervous system progression rate was evaluated using the time-dependent AUC. In the validation cohort, the model's 6-month mortality rate C-index was 0.69 (95% CI: 0.64-0.74), and the 8-week progression AUC was 0.70 (95% CI: 0.61-0.80), which were significantly better than the baseline Cox model (C-index 0.51, 95% CI: 0.42-0.57).

[0018] This invention also discloses a multimodal temporal fusion prognostic risk generation device for lung adenocarcinoma meningeal metastasis, used to implement the above method, comprising: The data acquisition module is used to collect longitudinal multimodal data of patients with lung adenocarcinoma and meningeal metastases and add timestamps. The evaluation unit construction module is used to group multimodal data with a time interval not exceeding a preset number of days into the same evaluation unit; The data preprocessing module is used to preprocess longitudinal multimodal data and extract features; The model building and training module is used to build and train the time-aware multimodal fusion model TEMPO. The risk information generation module is used to generate patients’ 8-week central nervous system progression risk score, 6-month all-cause mortality risk score, comprehensive risk score, four-quadrant risk stratification label and risk trajectory visualization results based on the trained TEMPO model. The output module is used to output the generated risk information to the user.

[0019] As a further improvement of the present invention, the TEMPO model includes: The structured data encoding module is used to encode clinical and tumor characteristics, treatment information, cerebrospinal fluid indicators, and serum test results; MRI image encoding module, used to extract feature vectors from brain MRI images; The cerebrospinal fluid cytology image encoding module is used to extract feature vectors from cerebrospinal fluid cytology images; The time encoding module is used to generate time location codes and time gap features; The modality availability mask module is used to construct a modality mask vector indicating the presence or absence of each modality; The gap indicator generation module is used to generate gap indicators that reflect the intensity of clinical follow-up. The multimodal fusion module is used to fuse multimodal features with temporal location codes, modal masks, and gap indicators; The study included two predictive branches: an 8-week branch predicting the risk of central nervous system progression and a 6-month branch predicting the risk of all-cause mortality.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention employs a time-aware temporal modeling method, which can handle irregular follow-up intervals commonly encountered in clinical practice, improve the temporal alignment stability of cross-modal asynchronous medical data, and make full use of the patient's longitudinal disease trajectory information, rather than relying solely on baseline data at the time of diagnosis. This invention incorporates data missing patterns as part of clinical information into the model, and replaces traditional data imputation with modality masks and gap indicators to reduce information bias caused by traditional imputation and preserve the clinical significance of "not having performed a certain examination". This invention enables joint prediction of two outcomes, simultaneously outputting the risk of central nervous system progression in the short term (8 weeks) and the risk of all-cause mortality in the medium term (6 months), improving the comprehensiveness and relevance of risk information and meeting the risk assessment needs of different time dimensions; The multimodal fusion architecture of this invention integrates clinical data, brain MRI images, and cerebrospinal fluid cytology images. Ablation experiments have confirmed that each modality makes a significant contribution to risk prediction, improving the model's prediction accuracy and significantly outperforming the traditional baseline Cox model. This invention proposes a four-quadrant risk stratification framework based on dual outputs, which can intuitively display the patient's risk status and its longitudinal changes, realize the continuous generation and visualization of patient-level dynamic risk trajectories, and provide users with clear and actionable risk information. This invention improves the robustness of the model under modality missing conditions by introducing modality availability mask and gap indicator, and can adapt to the differences in data integrity in different hospitals and different clinical scenarios; This invention uses an exponentially weighted average method to dynamically update patients' comprehensive risk scores, which improves the consistency of risk information generation in irregular follow-up scenarios and can accurately reflect the dynamic changes in patients' risk status. Attached Figure Description

[0021] Figure 1 This is a flowchart of the multimodal temporal fusion method for generating prognostic risks of lung adenocarcinoma meningeal metastasis disclosed in this invention; Figure 2 This is a framework diagram of the TEMPO time-aware multimodal prediction of 8-week CNS progression and 6-month mortality in this invention; Figure 3The discriminant performance of the present invention in terms of 6-month mortality and 8-week CNS progression includes time-dependent performance and ablation comparison plot; Figure 4 This is a representative attribution diagram of the interpretability of the integrated gradient-based model of this invention and brain MRI and cerebrospinal fluid cytology. Figure 5 This invention presents a combined risk stratification approach for clinical pathways: a four-quadrant diagram, a longitudinal migration diagram, and a residual inconsistency diagram. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The present invention will now be described in further detail with reference to the accompanying drawings: like Figure 1 As shown, the present invention provides a method for generating risk of lung adenocarcinoma meningeal metastasis patients based on multimodal temporal fusion. The method is constructed based on routine clinical data of lung adenocarcinoma meningeal metastasis patients admitted to Beijing Tiantan Hospital from January 1, 2020 to December 31, 2025. All data come from electronic health record system, laboratory information system, pathology archive system and image archiving and communication system.

[0024] The evaluation method specifically includes: S1, Research Cohort and Data Acquisition This study included 516 patients with lung adenocarcinoma and leptomeningeal metastases who met the inclusion criteria. Six patients who did not meet the criteria were excluded (4 with unclear leptomeningeal metastasis diagnosis, 1 with missing diagnosis date, and 1 with other active malignancies). The baseline characteristics of the included patients are shown in Table 1: median age 59 years (IQR: 52-65 years), 302 females (58.5%), and 338 non-smokers (65.5%). The median interval from diagnosis of primary lung cancer to diagnosis of leptomeningeal metastases was 32 months (IQR: 17-51 months). 97.1% of patients had brain metastases at the time of diagnosis of leptomeningeal metastases, of which 86.4% had ≥3 brain metastases.

[0025] A total of 2,603 ​​longitudinal follow-up assessments were conducted by all patients. A strict time-segmentation method was used to divide the population into the inferred cohort and the validation cohort: 345 patients diagnosed with leptomeningeal metastases before July 1, 2024, were in the inferred cohort, and 171 patients diagnosed on or after July 1, 2024, were in the validation cohort. The median follow-up time was 8 months (IQR: 3.0–15.0 months).

[0026] The collected multimodal data includes: Clinical and oncological characteristics: age, sex, smoking history, time of lung cancer diagnosis, time of diagnosis of meningeal metastasis, number of brain metastases, previous treatment history, etc. Treatment information: number of lines of systemic therapy, targeted therapy, chemotherapy, radiotherapy, etc.; Cerebrospinal fluid indicators: cerebrospinal fluid biochemistry, percentage of atypical cells in cerebrospinal fluid, etc. Serum laboratory tests: complete blood count, liver and kidney function tests, etc. Imaging data: Brain MRI images; Pathological data: Cerebrospinal fluid cytology images.

[0027] Table 1 S2. Construction of Vertical Evaluation Units and Timestamp Rules This invention uses the longitudinal assessment unit as the basic unit of analysis, defined as the time point after the diagnosis of meningeal metastasis where at least one relevant clinical assessment (clinical visit, cerebrospinal fluid examination, brain MRI or cerebrospinal fluid cytology examination) is recorded.

[0028] To address the issue of asynchronous multimodal data, the following rules are used to construct the evaluation unit: When there is a clear clinical consultation date for assessing dural metastasis, that date is used as the assessment timestamp; Otherwise, multimodal data with an interval of 3 to 14 days (preferably 5 days) are grouped into the same evaluation unit, and the latest record date within the unit is used as the evaluation timestamp. Strictly adhering to the chronological order, each evaluation unit's prediction uses only the timestamp and previously generated data to avoid information leakage.

[0029] The intervals between consecutive assessments were significantly irregular, with an overall median interval of 27 days (IQR: 14-39 days). The median intervals for the derivation cohort and the validation cohort were 28 days and 27 days, respectively, which is consistent with the actual clinical follow-up model.

[0030] S3. Multimodal data preprocessing and feature extraction S3.1 Clinical and Laboratory Data Preprocessing For continuous laboratory indicators, units were standardized and outliers were handled; values ​​exceeding the physiologically reasonable range were marked as missing and a missing indicator was added. Categorical variables were coded according to clinical significance; for example, molecular status was divided into five categories: EGFR exon19 deletion, EGFR L858R, rare EGFR mutations, other operable driver mutations (ALK / ROS1 / MET, etc.), and undetected driver mutations. All continuous variables were z-score standardized using the mean and standard deviation of the derived cohort.

[0031] S3.2 Brain MRI Image Preprocessing and Feature Extraction Brain MRI images were converted from DICOM format to a standardized format, and spatial resampling and sequence-specific intensity normalization were performed. Axial T1-weighted sequences were preferentially selected for feature extraction, and axial T2-weighted sequences were used as substitutes when missing sequences were available. A fixed number of axial slices uniformly covering the intracranial volume were selected from each MRI examination and input into a pre-trained ResNet-18 encoder to extract image features.

[0032] To address the issue of insufficient labeled data for rare diseases, the MRI encoder employs self-supervised pre-training initialization: brain MRI images from 500 patients with lung adenocarcinoma without intracranial metastases are used for pre-training. No risk labels are used during the pre-training process, and the pre-training weights are fine-tuned in the derivation queue.

[0033] S3.3 Cerebrospinal Fluid Cytology Image Preprocessing and Feature Extraction The resolution of digitized cerebrospinal fluid cytology images was normalized and the staining intensity was normalized. A ResNet-18 encoder was used to extract features from a single image. When multiple images were obtained for the same cytology examination, average pooling was used to obtain cytology feature vectors at the evaluation unit level.

[0034] S4. Missing Data Handling The lack of multimodal data is a common phenomenon in clinical practice. This invention does not use traditional data imputation methods, but instead incorporates the missing patterns as part of the clinical information into the model: Construct a modality mask vector to indicate the presence of each modality (clinical, MRI, cytology) in each assessment unit; Extraction gap indicators, including whether a brain MRI or cerebrospinal fluid examination was performed in the past 84 days, reflect the intensity of clinical follow-up; Add a binary missing indicator for missing structured variables.

[0035] The modal loss at the assessment unit level is shown in Table 2: the overall cerebrospinal fluid biochemical loss rate was 3.2%, the percentage loss rate of atypical cells in cerebrospinal fluid was 7.6%, the loss rate of cerebrospinal fluid cytology images was 35.1%, and the loss rate of brain MRI was 14.3%.

[0036] Table 2 S5 and TEMPO model architecture and training The overall architecture of the TEMPO model is as follows: Figure 2 As shown, this is a time-updated multimodal deep learning model. This model integrates structured clinical data, brain MRI images, and cytological / pathological images through a modality-specific encoder; then, it combines temporal location encoding and a fusion layer to fuse multimodal features; finally, it outputs two core predictions in pairs: the risk of central nervous system (CNS) progression at 8 weeks and the risk of death at 6 months. The rightmost panel in the figure shows a conceptual risk path diagram constructed based on the model's joint output. Figure 2 The multimodal deep learning model includes three parallel feature extraction branches, a temporal fusion module, and two prediction heads.

[0037] S5.1 Feature Extraction Branch Clinical data branch: The preprocessed clinical and laboratory features are mapped into 16-dimensional feature vectors through a fully connected layer; MRI image branch: 8-dimensional image feature vectors are extracted using a pre-trained ResNet-18 encoder; Cellular image branch: 8-dimensional cellular feature vectors are extracted using a ResNet-18 encoder.

[0038] S5.2 Timing Fusion Module The feature vectors from the three branches are concatenated with temporal location encoding, modality mask, and gap indicator, and then input into the fusion layer for feature fusion. The fusion layer consists of multiple fully connected layers and batch normalization layers, ultimately outputting a 64-dimensional fused feature vector.

[0039] S5.3 Dual Predictor Head 8-week central nervous system progression prediction head: Outputs the probability of a patient developing central nervous system progression as determined by the RANO-LM criteria within 56 days after this assessment, trained using binary cross-entropy loss; 6-month all-cause mortality prediction head: Output the patient's risk of death within 6 months after the assessment, trained with Cox partial likelihood loss.

[0040] The model's total loss function is a weighted sum of the losses from the two predictor heads, with L2 regularization added to prevent overfitting. During training, patient-level splitting is used for internal validation, with each mini-batch containing at most one evaluation per patient to avoid information leakage due to intra-patient dependencies. Hyperparameters are determined in the derivation queue via grid search. The final model is retrained on the complete derivation queue and then evaluated in the time-validation queue.

[0041] S6. Model Performance Evaluation The model's performance in the time-validation queue is as follows: Figure 3 As shown, A represents the receiver operating characteristic (ROC) curves of the model's discrimination against all-cause mortality risk at preset time points (30 days and 180 days) in the validation cohort; the diagonal line in the figure represents the model's lack of discrimination (equivalent to random guessing). B represents the area under the time-dependent ROC curve (AUC) at different time points during the follow-up process; the dashed line represents the average AUC value at all time points. C represents the ROC curve of the central nervous system (CNS) progression risk score at 8 weeks (56-day time window) in the validation cohort; the diagonal line in the figure represents the model's lack of discrimination. D compares the performance of the TEMPO complete model with the preset component ablation model and the baseline control model in predicting mortality risk at 6 months; the dots in the figure represent the model's performance estimates, and the horizontal line represents the 95% confidence interval (95% CIs).

[0042] in conclusion: The Harrell C index for 6-month all-cause mortality was 0.69 (95% CI: 0.64–0.74), which was significantly better than the Cox model using only baseline characteristics (C index 0.51, 95% CI: 0.42–0.57). The time-dependent AUC for central nervous system progression at 8 weeks was 0.70 (95% CI: 0.61–0.80). Time-dependent AUC analysis showed that the model performed better within a shorter prediction time window, with an AUC of 0.98 for 30-day mortality, which gradually decreased to 0.76 for 180 days as the prediction time lengthened.

[0043] Ablation test results ( Figure 3 The results of D and Table 3 confirm the importance of timing integration and multimodal input: After removing the static model of the time series history, the 6-month mortality C-index decreased to 0.55, and the 8-week progression AUC decreased to 0.52; Removing deep pathomic features significantly reduced the 8-week progression AUC to 0.37; After removing deep radiomics features, the AUC decreased to 0.44 at 8 weeks. Using only the baseline model based on clinical data, the 6-month mortality C-index was 0.60, and the 8-week progression AUC was only 0.39.

[0044] Table 3 Subgroup analysis showed that the model maintained stable performance across different sex, age, and EGFR status subgroups (Table 4).

[0045] Table 4 S7. Model Interpretability Analysis The global feature importance was calculated using the ensemble gradient method, and the results are as follows: Figure 4 As shown in Figure A, positive values ​​indicate that the feature increases the risk predicted by the model, while negative values ​​indicate that the feature decreases the risk predicted by the model. The horizontal axis represents the "integral gradient attribution value," with red bars representing features that increase risk and blue bars representing features that decrease risk. Features that contribute most to risk prediction include time-related features (time after diagnosis of meningeal metastasis, time from lung cancer diagnosis to meningeal metastasis), the number of brain metastases, cerebrospinal fluid cytology results, and routine cerebrospinal fluid laboratory indicators.

[0046] To further explain the model's imaging decision-making basis, spatial attribution maps of brain MRI and cerebrospinal fluid cytology images were generated. Figure 4 (B) Figure 4 Representative T1-weighted magnetic resonance imaging (MRI) images (axial, coronal, and sagittal) and cerebrospinal fluid cytology images of the brain, along with corresponding attribution heatmaps, are used to illustrate key regions contributing to the model's predictive results. The attribution heatmap is a qualitative visualization of the model's behavior: warm-colored areas represent regions the model classifies as "higher risk," while cool-colored areas represent regions the model classifies as "lower risk." The attribution map shows that the model's attention is focused on imaging regions associated with meningeal metastasis and tumor cell regions in cytological images; these visualizations help increase clinicians' confidence in the model's predictions.

[0047] S8. Four-Quadrant Risk Stratification and Clinical Application Based on the model's dual-output results, a four-quadrant risk stratification framework was constructed. Figure 5 A plots the 6-month mortality risk and 8-week central nervous system (CNS) progression risk for each time-stamped follow-up anchor point. For visualization, the 8-week CNS progression risk score is mapped to the 0–1 interval using an ordinal-preserving monotonic transformation, with the dashed line representing the preset risk cutoff point. Using 0.5 as the cutoff value, each assessment unit is divided into four quadrants: Quadrant I: Low risk of progression / low risk of death (54.6%); Quadrant II: High risk of progression / Low risk of death (29.7%); Quadrant III: Low risk of progression / high risk of death (0%) Quadrant IV: High risk of progression / high risk of death (15.7%).

[0048] Analyze the quadrant transition probabilities between consecutive assessments. Figure 5B represents the row-normalized transition probability matrix of quadrant status between consecutive follow-up anchor points within a patient; it can be observed that the patient's risk status has a certain degree of stability, but also exhibits dynamic changes. For example, a patient currently in quadrant I has a 75% probability of remaining in quadrant I at the next assessment, while there is a 21% probability of progressing to quadrant II.

[0049] Figure 5 C represents the residual risk of CNS progression at 8 weeks after adjusting for mortality risk severity: stratified by quartiles of predicted 6-month mortality risk. Residual inconsistency is defined as: in, This represents the 8-week CNS progression risk score mapped to the 0-1 interval after order-preserving transformation; the upper tail anchor point is marked. The top 15% of data points.

[0050] Figure 5 The D-case illustrates the risk trajectory of a representative patient: this patient had EGFR L858R-mutant lung adenocarcinoma with meningeal metastases. Prior to the first RANO-LM progression on day 175, the model-predicted 8-week progression risk had already increased; the risk of death gradually increased over the following 6 months, with a second progression occurring on day 388, ultimately leading to death on day 441. This case demonstrates that the model can capture signals of disease progression early, providing an opportunity for clinical intervention.

[0051] Example This embodiment demonstrates the specific application of the method of the present invention based on clinical data from a 56-year-old female patient with EGFR L858R-mutant lung adenocarcinoma: The patient was diagnosed with lung adenocarcinoma in March 2023 and developed headache and vomiting symptoms in August 2024. Brain MRI and cerebrospinal fluid cytology examination confirmed meningeal metastasis, and the patient was included in the validation cohort. All clinical data, three brain MRI images, and five cerebrospinal fluid cytology examination results of the patient since the diagnosis of leptomeningeal metastasis were collected to construct six longitudinal assessment units. The multimodal data is preprocessed and features are extracted, and then input into the trained TEMPO model; The model outputs the 8-week progression risk and 6-month mortality risk for each assessment, and calculates the patient-level comprehensive risk score using an exponentially weighted average method. Based on the dual-output results, the patient's risk status is stratified into four quadrants: the first three assessments are all in quadrant I (low risk), and the fourth assessment progresses to quadrant II (high progression / low mortality), generating a risk trajectory visualization result for users to view and as an auxiliary information output.

[0052] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating prognostic risk of lung adenocarcinoma meningeal metastasis through multimodal temporal fusion, characterized in that, include: Step 1: Collect longitudinal multimodal data of patients with lung adenocarcinoma and meningeal metastasis. The longitudinal multimodal data includes clinical and tumor characteristics, treatment information, cerebrospinal fluid indicators, serum test results, brain MRI images and cerebrospinal fluid cytology images, and add a timestamp to each data point. Step 2: Construct longitudinal evaluation units. Group multimodal data with an interval of no more than N days into the same evaluation unit. Use the latest date within the evaluation unit as the evaluation timestamp, and the prediction of each evaluation unit only uses the data before this timestamp. Step 3: Preprocess the longitudinal multimodal data to extract clinical data features, brain MRI image features, and cerebrospinal fluid cytology image features, respectively; Step 4: Construct the time-aware multimodal fusion model TEMPO, introducing time location encoding and modality masking, and fusing multimodal features with time location encoding, modality masking, and gap indicators; the missing data processing method is as follows: instead of data imputation, a modality mask is constructed to indicate the existence of each modality, and gap indicators are extracted, including whether brain MRI or cerebrospinal fluid examination was performed within a preset time period in the past, and the missing patterns are incorporated into the model as part of the clinical record; Step 5: Train the TEMPO model to output two risk indicators simultaneously: a risk score for central nervous system progression within 8 weeks after each assessment unit and a risk score for all-cause mortality within 6 months. Step 6: Based on the patient's continuous assessment data, the patient's comprehensive risk score is dynamically updated using the index-weighted average method, and a four-quadrant risk stratification label and risk trajectory visualization result are generated according to the dual output results.

2. The method according to claim 1, characterized in that, In step 2, multimodal data at preset time intervals are grouped into the same evaluation unit; the derivation queue and the validation queue are divided using the time segmentation method: patients diagnosed with meningeal metastasis before the specified time are in the derivation queue, and patients diagnosed on or after the specified time are in the validation queue.

3. The method according to claim 1, characterized in that, In step 3, brain MRI image feature extraction uses a convolutional neural network encoder or a visual Transformer encoder, preferentially using axial T1-weighted sequences, and using axial T2-weighted sequences when missing; cerebrospinal fluid cytology image feature extraction uses a convolutional neural network encoder or a visual Transformer encoder, and average pooling is used to obtain evaluation unit-level features when there are multiple images.

4. The method according to claim 1, characterized in that, In step 5, the model training uses a joint loss function: the risk of central nervous system progression at 8 weeks is given by binary cross-entropy loss, and the risk of all-cause mortality at 6 months is given by Cox partial likelihood loss; each mini-batch during training contains at most one assessment per patient.

5. The method according to claim 1, characterized in that, In step 6, the formula for the exponentially weighted average method is: ； ； in, For the first k The patient-level comprehensive risk score at the time of the second assessment r j For the first j A single risk score in this assessment. For the first j Next and first k The time interval between assessments is in days. γ This is the attenuation parameter.

6. The method according to claim 1, characterized in that, In step 6, the four-quadrant risk stratification uses a preset risk threshold to divide patients into four quadrants: low progression and low mortality, high progression and low mortality, low progression and high mortality, and generates corresponding risk stratification labels; wherein, the preset risk threshold includes one of the following: quantile threshold, ROC determination threshold, or user-configured threshold.

7. The method according to claim 1, characterized in that, It also includes a model interpretability analysis step: using the integrated gradient method to calculate the global feature importance, and generating spatial attribution maps of brain MRI and cerebrospinal fluid cytology images, identifying the clinical variables and imaging regions that contribute the most to risk prediction, and generating interpretable information for users to view.

Images