Bladder cancer vessel tumor embolus assessment method and system based on multi-modal feature fusion

Abstract

The invention belongs to the technical field of medical data processing, and particularly relates to a bladder cancer vessel tumor embolus assessment method and system based on multi-modal feature fusion, and the optimization method comprises the steps: obtaining and preprocessing the pathological information and image data of a bladder cancer patient, extracting a CT image corresponding to a tumor, and determining a region of interest; image omics features are extracted from the region of interest of the CT image, and deep learning features are extracted from the maximum tumor cross section; carrying out feature selection processing on the obtained radiomics features and deep learning features, and inputting the processed features into a fusion model; and outputting a probability value of'lymphatic vessel invasion 'by the fusion model to obtain an evaluation result. A multi-modal machine learning model is constructed by fusing radiomics features and deep learning features extracted from a CT image, and the objective of the invention is to noninvasively and accurately evaluate the lymphatic vessel infringement state of bladder cancer before an operation, so that a key basis is provided for clinical individualized treatment decision-making.

Description

Technical Field

[0001] This invention belongs to the field of medical data processing technology, specifically a method and system for assessing bladder cancer vascular tumor thrombi based on multimodal feature fusion. Background Technology

[0002] The statements in this section merely refer to the background art related to this invention and do not necessarily constitute prior art.

[0003] Bladder cancer is one of the most common malignant tumors of the urinary system. Based on the depth of invasion, it can be divided into non-muscle-invasive bladder cancer and muscle-invasive bladder cancer. Among them, lymphovascular invasion (LVI) is a pathological feature that refers to tumor cells invading the walls of lymphatic vessels or blood vessels. It is an independent adverse prognostic factor that reflects the tumor's strong invasiveness and high metastatic potential.

[0004] In clinical decision-making, LVI (Large Invasive Ligament) can be used to screen patients. For example, patients with non-muscle-invasive bladder cancer whose LVI is found in transurethral resection of bladder tumor specimens should be classified as "very high-risk" and early radical cystectomy should be considered.

[0005] However, transurethral resection of bladder tumor (TURP) specimens suffer from issues such as tissue fragmentation, cauterization artifacts, and limited sampling area, leading to poor accuracy and reproducibility in pathologists' assessment of LVI. There is also a risk of inconsistency between the LVI status of TURP specimens and ultimately radical specimens, further reducing the reliability of preoperative risk assessment.

[0006] For muscle-invasive bladder cancer, the standard treatment is neoadjuvant chemotherapy combined with radical cystectomy. However, statistics show that approximately 50% of patients cannot tolerate the toxicity of neoadjuvant chemotherapy due to their physical condition; on the other hand, the efficacy of the "one-size-fits-all" chemotherapy approach is questionable, necessitating the use of biomarkers to screen for those who truly benefit. Recent studies have shown that LVI status can serve as an effective indicator for predicting whether patients will benefit from perioperative chemotherapy.

[0007] In current clinical practice, the diagnosis of LVI relies entirely on the pathological evaluation of postoperative tissue specimens (such as those from transurethral resection or radical cystectomy). Because LVI status can only be determined postoperatively, treatment decisions are delayed. For high-risk non-muscle-invasive bladder cancer patients who should have received more aggressive surgery (such as early radical cystectomy), or muscle-invasive bladder cancer patients who should have been given neoadjuvant chemotherapy as a priority, the inability to obtain LVI information at this crucial preoperative decision-making point may lead to missed opportunities for optimal treatment and negatively impact patient prognosis.

[0008] With the development of artificial intelligence technology, radiomics and deep learning offer potential solutions to the aforementioned problems. Radiomics can extract quantitative features from medical images in high throughput, while deep learning can automatically learn deep and complex patterns in images. However, existing technologies mostly focus on bladder cancer staging or prognosis, and have not proposed relevant solutions for the core clinical need of "precise LVI stratification" (such as screening for extremely high-risk individuals with non-muscle-invasive bladder cancer). Summary of the Invention

[0009] This invention provides a method and system for assessing bladder cancer vascular tumor thrombus based on multimodal feature fusion, which can predict LVI status non-invasively and accurately before surgery, thus overcoming the shortcomings of existing technologies.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention discloses a method for assessing bladder cancer vascular tumor thrombi based on multimodal feature fusion, comprising the following steps: Acquire and preprocess pathological information and imaging data of bladder cancer patients, extract CT images corresponding to the tumor, and determine the region of interest. Radiomics features were extracted from regions of interest in CT images, and deep learning features were extracted from the largest tumor cross section, respectively. The obtained image omics features and deep learning features are processed by feature selection and then input into the fusion model; The fusion model outputs the probability value of "lymphovascular invasion" to obtain the evaluation result; Specifically, the optimal radiomics model and deep learning model are trained using the radiomics features and deep learning features after feature selection. The prediction outputs of the optimal radiomics model and deep learning model are used as new input features. A fusion model is constructed by combining the new input features with a stacking ensemble strategy.

[0011] Furthermore, the image omics features include at least first-order statistics, shape descriptors, and texture features, with texture features including features calculated based on at least one of the following: gray-level co-occurrence matrix, gray-level run-length matrix, gray-level size region matrix, gray-level dependency matrix, and neighborhood gray-level tone difference matrix.

[0012] Furthermore, deep learning features are extracted from the largest tumor cross-section. Specifically, the largest two-dimensional tumor cross-section in the z-axis direction is used as the input image, and deep learning features are extracted based on a pre-trained neural network model.

[0013] Further feature selection processing specifically includes: Calculate the intra-group correlation coefficient and retain features whose intra-group correlation coefficient is greater than a set value; Normalize the radiomics features and deep learning features, and perform Student's t test or Mann-Whitney U test, retaining features with a significance level P value less than the set value; Calculate the Pearson correlation coefficient among the remaining features and remove feature pairs whose absolute correlation coefficient value is greater than a set value; The remaining features are selected based on the LASSO logistic regression algorithm, and features with non-zero coefficients are retained as the final feature set.

[0014] Furthermore, the fusion model is built based on a stacked integration strategy, including: Pathological information and imaging data of bladder cancer patients were acquired and preprocessed. Based on the data source, the data were divided into a training queue, an internal validation queue, and an external validation queue. The training queue and the internal validation queue belonged to the same data source. The optimal radiomics model and deep learning model are selected by comparing the area under the receiver operating characteristic curves of the training cohort, internal validation cohort, and external validation cohort. The prediction results of the radiomics model and the deep learning model are combined as new features to form a training dataset, which is then input into a fusion model based on logistic regression to output the probability value of "lymphovascular invasion" and obtain the evaluation result.

[0015] Furthermore, the optimal radiomics model is the multilayer perceptron, and the optimal deep learning model is the logistic regression model.

[0016] Furthermore, it also includes prognostic assessment, specifically: using the median of the predicted probabilities output by the fusion model as the cutoff point, the clinicopathological information is divided into a high-risk group and a low-risk group for vascular tumor thrombus, and the differences in progression-free survival and overall survival between the two groups are compared using Kaplan-Meier, and the predictive efficacy of the fusion model for progression-free survival and overall survival at a set time period is evaluated by plotting receiver operating characteristic curves.

[0017] A second aspect of the present invention discloses a bladder cancer vascular tumor thrombus assessment system based on multimodal feature fusion, comprising: The data input and preprocessing module is configured to: acquire and preprocess pathological information and imaging data of bladder cancer patients, extract CT images corresponding to the tumor, and determine the region of interest. The feature extraction module is configured to: extract image features from the region of interest and extract deep learning features on the largest tumor cross section in the region of interest; The feature selection and evaluation module is configured to: process image features and deep learning features through feature selection and input them into the fusion model; The output module is configured to output the probability value of "lymphovascular invasion" from the fusion model to obtain the evaluation result.

[0018] A third aspect of the present invention discloses a computer program product comprising computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the aforementioned method for assessing bladder cancer vascular thrombi based on multimodal feature fusion.

[0019] A fourth aspect of the present invention discloses an electronic device, including at least one processor and a memory connected to the processor, the memory being used to store a computer program; the processor being used to execute the computer program, enabling the electronic device to implement the above-described method for assessing bladder cancer vascular thrombi based on multimodal feature fusion.

[0020] Compared with existing technologies, one or more of the above technical solutions have the following beneficial effects: 1. By fusing radiomics features extracted from CT images with deep learning features, a multimodal machine learning model is constructed, aiming to non-invasively and accurately assess the lymphovascular invasion status of bladder cancer before surgery, thereby providing key evidence for clinical decision-making.

[0021] 2. Radiomics features are typically hand-designed, exhibiting strong interpretability but weak generalization, while deep learning features are automatically learned, possessing strong generalization but weak interpretability. These two types of features exhibit heterogeneity. This approach first constructs optimal sub-models separately to avoid direct conflicts between cross-modal features and reduce redundant information interference in the fusion process. Then, a stacked fusion strategy is employed, using the sub-model outputs as input to the fusion model instead of directly fusing the original features, thus reducing fusion complexity.

[0022] 3. Deep learning models have a "black box" property, for example, they cannot explain why a certain CT image is judged as positive for "lymphovascular invasion". This solution targets the region of interest when "lymphovascular invasion" is judged, so that the prediction results have clear imaging evidence and improve clinical acceptance.

[0023] 4. For the core clinical decision-making need of "precise preoperative stratification of lymphovascular invasion" (such as screening of extremely high-risk groups for non-muscle-invasive bladder cancer), the clinical net benefit was confirmed by constructing a visual nomogram and decision curve analysis, and its predictive ability for progression-free survival and overall survival was verified. The technological achievement was transformed into a tool that can directly guide clinical decision-making, realizing the leap from technical model to clinical solution. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0025] Figure 1A schematic diagram illustrating the assessment process of bladder cancer vascular tumor thrombus based on multimodal feature fusion, provided for one or more embodiments of the present invention; Figure 2 A schematic diagram of the evaluation process (first part) for bladder cancer vascular tumor thrombus provided for one or more embodiments of the present invention; Figure 3 A schematic diagram of the evaluation process (part two) for bladder cancer vascular tumor thrombus provided for one or more embodiments of the present invention; Figure 4 A schematic diagram of the evaluation process for bladder cancer vascular tumor thrombus provided in one or more embodiments of the present invention (Part 3); Figure 5 A schematic diagram illustrating the use of several common machine learning algorithms to screen radiomics models, provided for one or more embodiments of the present invention; Figure 6 A schematic diagram illustrating the selection of deep learning models using several common machine learning algorithms, provided for one or more embodiments of the present invention; Figure 7 A performance diagram illustrating prediction using three queues—LRDs, Rads, and DLs—provided for one or more embodiments of the present invention; Figure 8 A schematic diagram illustrating the performance distribution of predictions using three models—LRDs, Rads, and DLs—provided for one or more embodiments of the present invention. Figure 9 Clinical nomograms based on LRDs provided for one or more embodiments of the present invention; Figure 10 A schematic diagram illustrating the prognostic value (survival rate) of LRDs based on one or more embodiments of the present invention; Figure 11 A schematic diagram illustrating the prognostic value (sensitivity and specificity) of LRDs based on one or more embodiments of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0028] As described in the background section, lymphovascular invasion (LVI), as a key prognostic and treatment guidance indicator, can only be diagnosed through postoperative pathology. This results in a significant diagnostic lag and specimen limitations, making it impossible for doctors to develop the most accurate individualized plans before surgery (such as timely surgery for high-risk patients or screening for patients who will benefit from chemotherapy).

[0029] With the development of artificial intelligence technology, radiomics and deep learning offer potential solutions to the aforementioned problems. Radiomics can extract quantitative features from medical images in high throughput, while deep learning can automatically learn deep and complex patterns in images. Although some studies have explored the application of single technologies in bladder cancer staging or prognostic prediction, existing technologies have not yet publicly provided a specific technical solution that can effectively integrate radiomics and deep learning and accurately predict the lymphovascular invasion status of bladder cancer based on preoperative CT images.

[0030] Therefore, this solution provides a method and system for assessing bladder cancer vascular thrombus based on multimodal feature fusion. By fusing radiomics features extracted from CT images with deep learning features, a multimodal machine learning model is constructed. This aims to non-invasively and accurately assess the lymphovascular invasion status of bladder cancer before surgery, thereby providing key evidence for clinical decision-making.

[0031] like Figure 1 As shown, the method for assessing bladder cancer vascular tumor thrombi based on multimodal feature fusion includes the following steps: Obtain the patient's clinicopathological information and extract the region of interest from the CT images within the clinicopathological information; Image features are extracted from the region of interest, and deep learning features are extracted from the largest tumor cross section in the region of interest. Image features and deep learning features are processed by feature selection and then input into the fusion model; The fusion model outputs the probability value of "lymphovascular invasion" to obtain the evaluation result.

[0032] This plan follows the TRIPOD+AI statement and the Helsinki Declaration reporting guidelines.

[0033] The following is combined Figures 2-4 This document details the evaluation process provided in this solution.

[0034] 1. Patient data acquisition.

[0035] We retrospectively collected data from 249 patients who underwent radical cystectomy (RC) at a certain hospital between January 2019 and January 2025 (patients were informed before data collection that the collected data would only be used for research and data training). Patients were randomly assigned to the training set and the internal validation set at a ratio of 7:3. An additional 53 patients meeting the criteria were selected from publicly available data, such as The Cancer Image Archive (TCIA cohort), as an external validation set. Clinicopathological data, including age, sex, tumor TNM stage, HER2 status, and Ki-67 expression, were collected through electronic medical records and databases.

[0036] 2. CT image acquisition and region of interest delineation.

[0037] CT images were saved in DICOM format, and the renal venous phase (the most commonly used phase for bladder cancer identification) was selected for subsequent analysis. Three-dimensional regions of interest (ROIs) were delineated using software, and all ROIs were reviewed and confirmed by technicians with several years of experience in bladder imaging (e.g., radiologists).

[0038] The 3D segmentation results were saved as Nifti format mask files. To assess intra-observer reproducibility, the physicians re-delineated the ROIs of 30 randomly selected patients one month later and calculated the intraclass correlation coefficient (ICC).

[0039] 3. Radiomics feature extraction.

[0040] Image omics feature extraction was performed using software such as the PyRadiomics package, following the Image Biomarker Standardization Initiative (IBSI) guidelines. Extracted features included first-order statistics, shape descriptors, and texture features based on GLCM (Gray-Level Co-occurrence Matrix), GLRLM (Gray-Level Run-Length Matrix), GLSZM (Gray-Level Region Size Matrix), GLDM (Gray-Level Dependency Matrix), and NGTDM (Neighborhood Gray-Level Difference Matrix).

[0041] 4. Deep learning feature extraction.

[0042] The largest two-dimensional tumor cross section in the z-axis direction is selected as the input image.

[0043] Before inputting the grayscale image into the model, the grayscale image is standardized with a window level of 50 and a window width of 350; all cropped images are resized to 224×224 and the pixel values ​​are normalized to [0,1].

[0044] Pre-trained ResNet-152 was fine-tuned on tumor images in the training set to enhance the ability to identify lymphovascular invasion (LVI).

[0045] Gradient-weighted class activation mapping (Grad-CAM) is used to improve the interpretability of classification results. The deep learning features extracted by the fine-tuned ResNet-152 are reduced to 64 principal component features.

[0046] 5. Feature selection.

[0047] A four-step feature selection strategy is adopted on the training set: 1) Repeatability: Feature retention of ICC > 0.8; 2) Standardization: After Z-score normalization, perform Studentt test or Mann-Whitney U test, and retain those with P < 0.05; 3) Collinearity: Features with a Pearson correlation coefficient > 0.9 are removed; 4) LASSO logistic regression was used to screen features with non-zero coefficients as the final modeling features.

[0048] 6. Integration of model and nodal plot construction.

[0049] The filtered features are input into eight commonly used machine learning algorithms (SVM, LR, KNN, RandomForest, ExtraTrees, AdaBoost, LightGBM, MLP), and the optimal algorithm is selected based on AUC.

[0050] The best-performing radiomics model and deep learning model were named Rads (radiomics model) and DLs (deep learning model), respectively.

[0051] Subsequently, a stacked integration strategy was adopted to construct a comprehensive prediction model - bladder cancer lymphovascular invasion radiomics and deep learning model (LRDs).

[0052] Stacking ensemble strategy refers to training one model to combine other models. The specific process in this scheme is as follows: 1) The collected data is randomly divided into a training queue and an internal validation queue in a 7:3 ratio, while the TCIA queue serves as an independent external validation queue.

[0053] 2) The optimal model was selected by comparing the area under the receiver operating characteristic curves (AUC) of the training cohort, internal validation cohort, and external validation cohort.

[0054] Multilayer perceptron (MLP) was chosen to build Rads because of its superior performance and minimal variation in AUC values ​​between the internal and external validation queues. Specifically, the MLP achieved an AUC of 0.859 (95% confidence interval: 0.774–0.945) in the internal validation queue and an AUC of 0.832 (95% confidence interval: 0.721–0.943) in the external validation queue. Logistic regression (LR) was chosen to construct the deep learning algorithms because its AUC values ​​showed minimal variation in the validation cohort. LR achieved an AUC of 0.746 (95% confidence interval: 0.636–0.856) in the internal validation cohort and an AUC of 0.773 (95% confidence interval: 0.640–0.906) in the external validation cohort.

[0055] 3) Combine the prediction results of the Rads model and the prediction results of the DLs model as new features to form a new training dataset.

[0056] 4) For this new training dataset, a comprehensive fusion model called LRDs was constructed using logistic regression.

[0057] 5) The model achieved an AUC of 0.859 in the training group and demonstrated strong predictive performance in both the internal and external validation groups (AUC values ​​were both 0.872).

[0058] Finally, the trained meta-model is used to make predictions on the test data.

[0059] The following details the specific process of "3" in the above process, which involves combining the prediction results of the Rads model and the DLs model as new features to form a new training dataset.

[0060] (1) The prediction results of the generated sub-model.

[0061] For any input sample, the two sub-models output their predicted probabilities of the presence of vascular tumor thrombi, denoted as P0. MLP and P LR The probability values ​​range from [0,1], representing the model's confidence that the sample is positive. These two probability values ​​will serve as new feature inputs for subsequent fusion training.

[0062] (2) Ten-fold cross-validation.

[0063] To improve the model's generalization ability, this embodiment uses 10-fold cross-validation to generate out-of-fold predictions for the sub-models in the training queue. The specific process is as follows: The training queue is randomly divided into 10 non-overlapping subsets; In each iteration, nine subsets are selected as training subsets to train the MLP model and the LR model respectively. Using the trained model described above, predict the remaining subset to obtain the out-of-division prediction probability value P for each sample. MLP and P LR ; Repeat the above iterative steps 10 times, and for each sample, obtain the prediction result of the model output that has not seen the sample before.

[0064] The above steps yield two columns of predicted features—P—with the same length as the training set. MLP_train (Outlier probability prediction in MLP) and P LR_train (LR's out-of-bounds probability prediction) constitutes the core part of the new training feature set.

[0065] (3) Generate the corresponding prediction results on the validation set.

[0066] After cross-validation, the MLP and LR models retrained on the entire training queue are used to predict the internal and external validation queues respectively, obtaining the corresponding probability outputs, denoted as: Internal validation set: P MLP_internal and P LR_internal ; External validation set: P MLP_external and P LR_externa l; The above prediction results are used for subsequent validation of the integrated model and independent performance evaluation.

[0067] (4) Feature fusion.

[0068] For any sample, the prediction probability results of the MLP model and the LR model are combined at the feature level to form a new feature matrix.

[0069] For the training queue, the new training dataset can be represented as: X stack_train = [P MLP_train, P LR_train ]; The same logic applies to internal and external verification queues, and can be represented as follows: X stack_internal = [P MLP_internal , P LR_internal ]; X stack_external = [P MLP_external , P LR_external ].

[0070] (6) Generate a fusion training dataset.

[0071] The new training dataset obtained after the above steps will be used as the input data for the subsequent integrated fusion model (logistic regression), as shown in Table 1. Table 1 Input data for the fusion model

[0072] The performance and robustness of LRDs were evaluated using AUC (95% CI), accuracy, sensitivity, specificity, PPV, NPV, F1 score, calibration curve, and decision curve analysis (DCA), and inter-model comparisons were performed using the DeLong test and the net reclassification improvement index (NRI).

[0073] Univariate and multivariate logistic regression were performed to evaluate the independent predictive value of pathological variables and LRDs for LVI (in multivariate regression, P < 0.05 was considered an independent predictor). Nonoplots were constructed on the training set based on the independent predictors, and their predictive performance was validated on an internal validation set.

[0074] 7. Prognostic assessment potential.

[0075] Patients were divided into high-risk and low-risk groups based on the median LRDs score. The Kaplan-Meier method was used to compare the differences in progression-free survival (PFS) and overall survival (OS) between the two groups, and Cox proportional hazards regression was used to assess the prognostic significance. Furthermore, ROC curves were used to evaluate the predictive efficacy of LRDs for 1-year, 3-year, and 5-year PFS and OS.

[0076] 8. Statistical analysis.

[0077] All data analysis and visualization were performed using software (such as Python 3.7.12 and R4.4.0).

[0078] Continuous variables are expressed as mean ± standard deviation or median (interquartile range), while categorical variables are expressed as number of cases (percentage).

[0079] Machine learning algorithms (SVM, RandomForest, ExtraTrees, etc.) are based on scikit-learn 1.0.2, LightGBM uses lightgbm 3.3.2; statistical modeling uses statsmodels 0.13.2.

[0080] The radar chart was plotted using the R package ggplot2.

[0081] Single / multivariate logistic regression, nomograms, and Kaplan-Meier survival analysis were performed using the rms, survminer, and survival packages, respectively.

[0082] A two-sided P < 0.05 is considered statistically significant.

[0083] In this scheme, multiple machine learning algorithms are used to screen radiomics models and deep learning models, respectively, as follows: Figure 5 and Figure 6 As shown in the figure. The prediction performance of LRDs, Rads, and DLs in the three queues are respectively as follows. Figure 7 and Figure 8 As shown, the clinical nomogram based on LRDs is as follows: Figure 9 As shown, the prognostic value of LRDs is as follows: Figure 10 and Figure 11 As shown.

[0084] result.

[0085] 1. Patient Baseline Characteristics. A total of 302 patients were included, with 249 from a specific hospital and 53 from the TCIA public cohort. The cohorts were randomly assigned in a 7:3 ratio to the training set (n=174, 69.9%) and the internal validation set (n=75, 30.1%); the TCIA cohort served as the external validation set. There were no statistically significant differences among the three groups in terms of sex, age, TNM stage, median overall survival (OS), and median progression-free survival (PFS). The number of LVI-positive cases were: 74 in the training set, 31 in the internal validation set, and 29 in the external validation set.

[0086] 2. Feature Extraction and Screening. A total of 1834 radiomics features and 64 deep learning principal component features were extracted from enhanced CT scans. After a four-step screening process, 13 radiomics features and 5 deep learning features were retained. Pearson correlation analysis showed that the pairwise correlation coefficients of all selected features were <0.75, indicating low multicollinearity. Among them, log_sigma_3_0mm_3D_glszm_SmallAreaEmphasis and DL_3 had the highest importance scores.

[0087] 3. Rads and DLs Sub-model Construction. Among the eight machine learning algorithms, the Multilayer Perceptron (MLP) was selected as the Rads factor model due to its high AUC and small cross-center variability: internal validation AUC 0.859 (95% CI: 0.774–0.945), external validation AUC 0.832 (95% CI: 0.721–0.943). Among the deep learning sub-models (DLs), Logistic Regression (LR) showed the most robust performance: internal validation AUC 0.746 (95% CI: 0.636–0.856), external validation AUC 0.773 (95% CI: 0.640–0.906).

[0088] 4. Construction and validation of integrated model LRDs.

[0089] By fusing Rads and Deep Learning (DLs), an integrated model, LRDs, was obtained. The training set AUC was 0.859; both the internal and external validation sets achieved an AUC of 0.872. Subgroup analysis showed that LRDs remained stable in both NMIBC and MIBC. NRI tests demonstrated that LRDs significantly outperformed any single sub-model in all three groups. Calibration curves showed a high degree of consistency between predicted probabilities and actual results; DCA curves confirmed that LRDs provided significant net clinical benefit over a wide threshold range.

[0090] This approach provides a fusion model for the decision-making process of two typical patients. Based on the patients' CT images, the predictive model of this approach calculates the probability of "LVI presence". Patient 1 was pathologically confirmed to be LVI negative. Rads made predictions by analyzing the texture, shape, and grayscale features of the 3D-CT images, while DLs made predictions by acquiring deep learning features automatically analyzed by ResNet152. Finally, LRDs fused the opinions of the two sub-models, determining that Patient 1 had a probability of LVI of 0.28, tending to believe that there was no vascular tumor thrombus, and its postoperative pathology confirmed that no LVI was found. Similarly, Patient 2 was pathologically confirmed to be LVI positive, and LRDs predicted that this patient had a probability of LVI of 0.65, tending to believe that there was a vascular tumor thrombus (illustrated diagram).

[0091] 5. Pathology - LRDs nomogram.

[0092] Univariate and multivariate logistic regression analysis showed that LRDs score, tumor invasion depth, lymph node metastasis, and HER2 expression were independent predictors of LVI (P<0.05). Based on this, a nomogram was constructed with an AUC of 0.912 on the training set and 0.886 on the internal validation set, facilitating individualized estimation of LVI risk.

[0093] 6. Prognostic value of LRDs.

[0094] Kaplan–Meier analysis showed that high LRDs scores were associated with significantly worse PFS (P<0.01, HR=2.35) and OS (P<0.01, HR=3.1). Time-dependent ROC curves showed that the AUCs for 1-year, 3-year, and 5-year PFS and OS were all >0.65. Sankey plots further visualized the flow of patients in the high-risk / low-risk groups among relapse, survival, and LVI status, visually demonstrating the prognostic significance of LRDs.

[0095] This solution follows the IBSI and TRIPOD+AI specifications, with clearly defined tools and parameters, and reproducible results.

[0096] By merging radiomics models (Rads, which capture quantitative features) and deep learning models (DLs, which capture abstract features) using a stacking ensemble strategy, an LRDs model was constructed. The AUC of 0.872 was achieved in both internal and external validation, outperforming any single sub-model and achieving "1+1>2".

[0097] The four-step screening process in this scheme (ICC > 0.8 to ensure repeatability, P < 0.05 to ensure correlation, decollinearity, and LASSO dimensionality reduction) ultimately retains 13 radiomics features and 5 deep learning features to ensure model reliability.

[0098] The nomogram in this solution enables visual estimation of LVI risk and can also differentiate between patient PFS / OS (HR 2.35 and 3.1 respectively). Grad-CAM improves the interpretability of the model and directly guides treatment decisions.

[0099] This solution achieves "integration" and "prediction of LVI in bladder cancer using preoperative enhanced CT". First, optimal sub-models are established separately (MLP for Rads, LR for Drags), then stacked and fused. NRI confirms that LRDs are superior to the sub-models, thus achieving model "integration". Using preoperative enhanced CT venous phase images and postoperative pathology as the gold standard for modeling, this solution addresses the pain point of inaccurate preoperative LVI assessment, achieving "preoperative enhanced CT prediction of LVI".

[0100] This solution overcomes the following technical difficulties.

[0101] 1. Cross-modal fusion is challenging: The heterogeneity of radiomics features (hand-designed, highly interpretable but with weak generalization) and deep learning features (automatically learned, highly generalizable but with weak interpretability) makes it difficult for traditional methods to achieve effective fusion (e.g., direct feature splicing can easily lead to the curse of dimensionality or feature conflicts). This approach first constructs optimal sub-models separately to avoid direct conflicts between cross-modal features; the deep learning features are first reduced to 64 principal components to reduce the interference of redundant information on fusion; and a "stacked ensemble" strategy is adopted, using the output of the sub-model as the input of the upper-level model instead of directly fusing the original features, thus reducing the fusion complexity. 2. Limited high-quality data (key point): LVI annotation relies on the gold standard of postoperative pathology, requiring simultaneous acquisition of "preoperative enhanced CT + postoperative RC specimen pathology results", which makes sample collection difficult (most centers have insufficient case volume and require long-term follow-up). 3D ROI delineation requires professional physicians (urologists + radiologists with 15 years of experience for review), which is costly and makes it difficult to ensure consistent labeling. This solution uses multi-center collaboration, dual physician delineation, ICC testing, and senior physician review to ensure data quality.

[0102] 3. Overcoming the inherent limitation of poor reproducibility of radiomics features: Radiomics features are highly sensitive to image preprocessing (such as window width and level, slice thickness) and ROI segmentation accuracy. Differences in operation between different centers / physicians can lead to feature instability, making it difficult to use in cross-center models. This approach overcomes this technical difficulty by standardizing image preprocessing, rigorously verifying reproducibility, and extracting features in accordance with IBSI guidelines. 4. The "black box" nature of deep learning: The "black box" nature of deep learning models (they cannot explain why they judge LVI positivity) leads to low trust in their predictions by clinicians, making them difficult to implement. This solution introduces Grad-CAM technology to visualize the "region of interest" (ROI) when the model judges LVI, giving the prediction results clear imaging evidence and improving clinical acceptance. 5. Overcoming the design bias of disconnect between clinical needs and technology (key point): Existing studies mostly focus on bladder cancer staging or prognosis, failing to address the core clinical need of "precise LVI stratification" (such as screening for very high-risk NMIBC patients and screening for NAC beneficiaries in MIBC patients), and do not provide practical tools (such as nomograms). This approach focuses on LVI as an "independent adverse prognostic factor," directly linking it to clinical decision-making. It constructs nomograms to achieve visualized risk estimation and uses DCA to confirm clinical net benefit, ensuring that the model can directly guide treatment. It also extends the prognostic assessment function by linking LVI prediction with PFS / OS, providing a more comprehensive basis for the development of individualized treatment plans.

[0103] Correspondingly, a bladder cancer vascular tumor thrombus assessment system based on multimodal feature fusion includes: The data input and preprocessing module is configured to: acquire the patient's clinicopathological information and extract the region of interest from the CT image in the clinicopathological information; The feature extraction module is configured to: extract image features from the region of interest and extract deep learning features on the largest tumor cross section in the region of interest; The feature selection and evaluation module is configured to: process image features and deep learning features through feature selection and input them into the fusion model; The output module is configured to output the probability value of "lymphovascular invasion" from the fusion model to obtain the evaluation result.

[0104] Correspondingly, a computer program product includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the aforementioned method for assessing bladder cancer vascular thrombi based on multimodal feature fusion.

[0105] Accordingly, an electronic device includes at least one processor and a memory connected to the processor, the memory being used to store computer programs; the processor is used to execute the computer programs, enabling the electronic device to implement the above-mentioned method for assessing bladder cancer vascular thrombi based on multimodal feature fusion.

[0106] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for evaluating bladder cancer vessel embolus based on multi-modal feature fusion, characterized in that, The method comprises the following steps: Obtaining pathological information and image data of a bladder cancer patient and preprocessing, extracting a CT image corresponding to a tumor and determining a region of interest; Extracting image features from the region of interest of the CT image, and extracting deep learning features from the maximum tumor section; The obtained image features and deep learning features are processed by feature selection and input into a fusion model; The fusion model outputs a probability value of "lymphatic vessel invasion", and obtains an evaluation result; The optimal image feature model and the optimal deep learning model are obtained by training the image features and the deep learning features processed by feature selection, respectively, and the prediction outputs of the optimal image feature model and the optimal deep learning model are used as new input features, and the new input features are combined with a stacking ensemble strategy to construct a fusion model. 2.The method for evaluating bladder cancer vessel embolus based on multi-modal feature fusion according to claim 1, wherein, The image features at least include first-order statistics, shape descriptors and texture features, and the texture features include at least one of features calculated based on a gray level co-occurrence matrix, a gray level run-length matrix, a gray level size zone matrix, a gray level dependence matrix and a neighborhood gray level tone difference matrix. 3.The method for evaluating bladder cancer vessel embolus based on multi-modal feature fusion according to claim 1, wherein, The deep learning features are extracted from the maximum tumor section, specifically: taking the maximum two-dimensional tumor section in the z-axis direction as the input image, and extracting the deep learning features based on a pre-trained neural network model. 4.The method for evaluating bladder cancer vessel embolus based on multi-modal feature fusion according to claim 1, wherein, The feature selection processing is specifically: Calculating the intraclass correlation coefficient, and retaining features with an intraclass correlation coefficient greater than a set value; Performing normalization processing on the image features and the deep learning features, and performing Student's t test or Mann-Whitney U test, and retaining features with a significance level P value less than a set value; Calculating the Pearson correlation coefficient between the remaining features, and eliminating feature pairs with an absolute value of the correlation coefficient greater than a set value; Based on the LASSO logistic regression algorithm, the remaining features are screened, and features with non-zero coefficients are retained as the final feature set. 5.The method for evaluating bladder cancer vessel embolus based on multi-modal feature fusion according to claim 1, wherein, The fusion model is constructed based on the stacking ensemble strategy, including: Obtaining pathological information and image data of a bladder cancer patient and preprocessing, dividing the data into a training queue, an internal validation queue and an external validation queue according to the data source, and the training queue and the internal validation queue belong to the same data source; Selecting the optimal image feature model and the optimal deep learning model by comparing the areas under the receiver operating characteristic curves of the training queue, the internal validation queue and the external validation queue; The prediction results of the image feature model and the deep learning model are combined to form a training data set, which is input into a fusion model constructed based on a logistic regression, and a probability value of "lymphatic vessel invasion" is output to obtain an evaluation result. 6.The method for evaluating bladder cancer vessel embolus based on multi-modal feature fusion according to claim 1, wherein, The optimal image feature model is a multilayer perception machine, and the optimal deep learning model is a logistic regression model. 7.The method for assessing bladder cancer vessel embolus based on multi-modal feature fusion according to claim 1, wherein, Further comprising prognosis evaluation, specifically: taking the median of the prediction probability output by the fusion model as the cut-off point, dividing the clinical pathological information into a high-risk group and a low-risk group, comparing the progression-free survival and overall survival of the two groups by Kaplan-Meier, and evaluating the prediction performance of the fusion model for the progression-free survival and overall survival of the set time period by drawing a receiver operating characteristic curve.

8. A bladder cancer vessel embolus assessment system based on multi-modal feature fusion for implementing the method of any one of claims 1-7, characterized in that, It comprises: The data input and preprocessing module is configured to: acquire clinical pathology information of a patient, and extract a region of interest of a CT image in the clinical pathology information; The feature extraction module is configured to: extract image features from the region of interest, and extract deep learning features on a maximum tumor section in the region of interest; The feature selection and evaluation module is configured to: perform feature selection processing on the image features and the deep learning features, and input the image features and the deep learning features into a fusion model; The result output module is configured to: output a probability value of "lymphatic vessel invasion" by the fusion model, and obtain an evaluation result.

9. A computer program product, characterised in that, The computer readable instructions, when executed on an electronic device, cause the electronic device to implement the steps in the bladder cancer vessel embolus evaluation method based on multi-modal feature fusion according to any one of claims 1-7.

10. An electronic device, comprising: The computer readable instructions, when executed on an electronic device, cause the electronic device to implement the steps in the bladder cancer vessel embolus evaluation method based on multi-modal feature fusion according to any one of claims 1-7.

Images