Artificial intelligence and physics-based models for evaluating magnetic resonance imaging data of bladders
Patent Information
- Application Number
- PCT/US2026/020703
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-03-26
- Filing Date
- 2026-03-25
- Publication Date
- 2026-10-01
Smart Images

Figure US2026020703_01102026_PF_FP_ABST
Abstract
Description
Atty. Dkt. No.: 115872-3450ARTIFICIAL INTELLIGENCE AND PHYSICS-BASED MODELS FOR EVALUATING MAGNETIC RESONANCE IMAGING DATA OF BLADDERS CROSS REFERENCES TO RELATED APPLICATIONS[00011 The present application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 778,082, filed March 26, 2025, which is incorporated by reference in its entirety.STATEMENT OF GOVERNMENT SUPPORT
[0002] This invention was made with government support under CA008748 awarded by the National Institutes of Health. The government has certain rights in the invention.BACKGROUND
[0003] A computing device may apply computer vision techniques on a digital image to generate an output.SUMMARY
[0004] Aspects of the present disclosure are directed to systems, methods, devices, apparatus, and non-transitory computer-readable media for classifying tumors in bladders using magnetic resonance imaging (MRI) data. One or more processors can receive for a subject at risk of or diagnosed with bladder cancer, a plurality of MRI images of a bladder having at least one tumor associated with the bladder cancer in the subject, the plurality of MRI images comprising (i) a first MRI image of the bladder acquired in accordance with a first imaging sequence of a plurality imaging sequences and (ii) a second MRI image of the bladder acquired in accordance with a second imaging sequence of the plurality of imaging sequences. The one or more processors can identify (i) a first region of interest (ROI) defining the at least one tumor in the first MRI image and (ii) a second ROI defining the at least one tumor in the second MRI image. The one or more processors can determine a plurality of features defining one or more visual characteristics of the at least one tumor based on applying-1- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450a first model to the first MRI image and the first ROI. The one or more processors can determine a plurality of metrics identifying one or more imaging biomarkers associated with the at least one tumor based on applying a second model to the second MRI and the second ROI. The one or more processors can generate, using the plurality of features and the plurality of metrics, a classification corresponding to the at least one tumor associated with the bladder cancer in the subject. The one or more processors can store, using one or more data structures, an association between the subject and the classification.
[0005] In some embodiments, the one or more processors can determine, for each of the plurality of features and the plurality of metrics, a corresponding significance metric as a function of at least one of the plurality of features or the plurality of metrics. In some embodiments, the one or more processors can select a first portion of the plurality of features and a second portion of the plurality of metrics based on the significance metric for each of the plurality of features and the plurality of metrics. In some embodiments, the one or more processors can generate the classification using the first portion of the plurality of features and the second portion of the plurality of metrics.
[0006] In some embodiments, the one or more processors can receive, via a user interface, a selection of the first model from a plurality of first models. In some embodiments, the one or more processors can apply the first model selected from the plurality of first models on the first MRI image. In some embodiments, the one or more processors can receive, via the user interface, a selection of a third model from a plurality of third models. The one or more processors can apply the third model selected from the plurality of third models to the plurality of features and the plurality of metrics. In some embodiments, the one or more processors can generate a parametric map identifying the plurality of metrics for a plurality of voxels within the second ROI in the second MRI image. The plurality of metrics for the second imaging sequence corresponding to a diffusion-weighted imaging (DWI) sequence comprises at least one of an apparent diffusion coefficient, true diffusion coefficient, a pseudo diffusion coefficient, a perfusion fraction, or a kurtosis coefficient. The plurality of metrics for the second imaging sequence corresponding to a dynamic contrast enhanced (DCE) sequence comprises at least one of volume transfer constant (Ktrans), a blood plasma volume fraction (vP), or a volume fraction of extravascular extracellular space (ve).-2- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450
[0007] In some embodiments, the one or more processors can receive, via a user interface, one or more inputs corresponding to the first ROI in the first MRI image and the second ROI in the second MRI image. In some embodiments, the one or more processors can apply one or more image segmentation models to the first MRI image to identify the first ROI and the second MRI image to identify the second ROI. In some embodiments, the one or more processors can receive a score indicating a severity of the bladder cancer in the subject. In some embodiments, the one or more processors can determine the plurality of features based on applying the first model to the score. In some embodiments, the one or more processors can determine the plurality of metrics based on the applying the second model to the score.
[0008] In some embodiments, the one or more processors can generate the classification identifying one of presence or absence of the bladder cancer in the subject based on applying a third model to at least one of the plurality of features or the plurality of metrics. In some embodiments, the one or more processors can generate the classification identifying a type of a plurality of types for the bladder cancer, based on applying a third model to at least one of the plurality of features or the plurality of metrics. The plurality of types of the bladder cancer may include at least one of urothelial carcinoma, squamous cell carcinoma, adenocarcinoma, or small cell carcinoma.
[0009] In some embodiments, the one or more processors can generate the classification indicating a survival metric of the subject based on applying third model to at least one of the plurality of features or the plurality of metrics. The survival metric can include at least one of an overall survival (OS) metric, a progression free survival (PFS) metric, or a time to progression (TTP) metric. In some embodiments the one or more processors can generate the classification identifying the subject as one of a plurality of severity levels for the bladder cancer, based on applying a third model to at least one of the plurality of features or the plurality of metrics.
[0010] In some embodiments, the one or more processors can generate the classification identifying the subject as a candidate or non-candidate for the administration of therapy for the bladder cancer. In some embodiments, the one or more processors can generate the classification identifying the subject as one of a predicted responder or a predicted non-responder to the therapy for the bladder cancer based on applying a third -3- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450model to at least one of the plurality of features or the plurality of metrics. In some embodiments, the therapy for the bladder cancer can include at least one of neoadjuvant chemotherapy, surgical resection, adjuvant chemotherapy, or radiotherapy. The subject may be administered with a therapeutically effective amount of therapy when the classification identifies the subject as the candidate.
[0011] In some embodiments, the one or more processors can receive the plurality of MRI images of the bladder having the at least one tumor acquired at a first time prior to or subsequent to administration of a therapy for bladder cancer. The one or more processors can generate the classification indicating an effect of the therapy associated with the first time, based on (i) applying a third model to at least one of the plurality of features or the plurality of metrics and (ii) a second classification at a second time. In some embodiments, the one or more processors can determine a score indicating a reliability of the classification as a function of the classification, the plurality of features, and the plurality of metrics. The one or more processors can store the association of the subject with the classification and the score.
[0012] In some embodiments, the one or more processors can provide, for presentation via a user interface, an output including information based on one or more of the score, the classification, or at least one of the plurality of MRI images. In some embodiments, the plurality of imaging sequences for the plurality of MRI images further comprises at least one of T1 -weighted, T2-weighted, diffusion-weighted imaging (DWI), and dynamic contrast enhanced (DCE) MRI.BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The foregoing and other objects, aspects, features, and advantages of the disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:
[0014] FIGs. 1A-1D: The workflow of the AI-BLADE toolbox includes preprocessing steps for mpMRI, MRI-Deep Feature analysis (DFA) (Blocks A-C) and MRI-Model Based Analysis (MBA) (Blocks A-B). MRI-MBA (Block B): Representative parametric ADC (xl0'3mm2 / s) and D (xl0'3mm2 / s) maps from mono-exponential and non--4- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450Gaussian intravoxel incoherent motion modeling of DW data, and Ktrans(min-1) map from Patlak and extended Tofts models for DCE-MRI data.
[0015] FIGs. 2A and 2B screens of the user interface for AI-BLADE toolbox.
[0016] FIG. 3: ROC curve for Al models with different feature selection algorithms and classifiers. (A) ROC curves for ResNetl52, (B) ROC curves for EfficientNet-B7, (C) ROC curves for VGG19, (D) ROC curves for AlexNet. Abbreviations: NFS: No Feature Selection, FC: Fully Connected Layer, DFT: Decision Fine Tree.
[0017] FIG. 4A: MRI-MBA provides fitting algorithms to analyze diffusion weighted (DW)-MRI data using mono-exponential and non-Gaussian intravoxel incoherent motion (NG-IVIM) models. The model’s derived parametric maps are overlaid on the diffusion weighted images (b=0 s / mm2).
[0018] FIG. 4B: MRI-MBA provides fitting algorithms to analyze diffusion weighted (DW) data using monoexponential and non-Gaussian intravoxel incoherent motion (NG-IVIM) models. The model’s derived representative parametric maps are overlaid on the diffusion weighted images (b=0 s / mm2).
[0019] FIG. 5A: MRI-MBA includes algorithms to analyze DCE-MRI data using Patlak and extended Tofts models. The derived parametric maps with values are overlaid on the images.
[0020] FIG. 5B: MRI-MBA includes algorithms to analyze DCE-MRI data using Patlak and extended Toft models. The model’s derived representative parametric maps are overlaid on the post-contrast T1 weighted images.
[0021] FIG. 6: Proposed architecture for web-based cloud implementation of AI-BLADE.
[0022] FIGs. 7A-7J depict ROC curves for Al models using the MRI-DFA toolkit with different feature selection algorithms and classifiers.
[0023] FIG. 8 is a block diagram of a system of classifying bladder risk using magnetic resonance imaging (MRI) data, in accordance with an illustrative embodiment.-5- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450
[0024] FIG. 9 is a block diagram of a process for determining regions of interest (ROI) within a bladder, in accordance with an illustrative embodiment.
[0025] FIG. 10A is a block diagram of a process for generating features using one or more feature extraction models, in accordance with an illustrative embodiment.
[0026] FIG. 10B is a block diagram of a process for generating a parametric map using one or more metric evaluation models, in accordance with an illustrative embodiment.
[0027] FIG. 11 is a block diagram of a process for generating a classification of a tumor within the bladder using one or more classification models, in accordance with an illustrative embodiment.
[0028] FIG. 12 is a flow diagram of a method of classifying bladder risk using magnetic resonance imaging (MRI) data, in accordance with an illustrative embodiment.
[0029] FIG. 13 is a block diagram of a computing environment according to an example implementation of the present disclosure.DETAILED DESCRIPTION
[0030] Following below are more detailed descriptions of various concepts related to, and embodiments of, systems and methods for classifying tumors in bladder using magnetic resonance imaging (MRI) data. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in any of numerous ways, as the disclosed concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided primarily for illustrative purposes.
[0031] Section A describes AI-BLADE toolbox: Al-powered BLADdEr multiparametric MRI Analysis for Clinical Application
[0032] Section B describes systems and methods for classifying tumors in bladder using magnetic resonance imaging (MRI) data.
[0033] Section C describes a network and computing environment.-6- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450A. AI-BLADE Toolbox: Al-Powered Bladder Multiparametric MRI Analysis for Clinical Application
[0034] Objectives: There is a growing need to develop user-friendly, bladder-specific image analysis tools that can produce reliable artificial intelligence (Al)-quantitative imaging biomarkers (QIBs) derived from multiparametric (mp)MRI data for clinical applications. To address this need, disclosed is an Al-powered BLADdEr multiparametric MRI Analysis for Clinical Application, a vendor-agnostic, flexible, and user-friendly toolbox designed for estimating mpMRI derived quantitative metrics.
[0035] Methods: AI-BLADE is an advanced tool for bladder-specific mpMRI analysis with two core functionalities: (i) Deep Feature Analysis (MRI-DFA toolkit) and (ii) Data-Driven Model-Based Analysis (MRI-MBA toolkit). AI-BLADE offers customizable parameters and acts as a one-stop-shop solution for bladder cancer (BCa) clinical applications. The models within DFA and MBA were tested separately on two patient cohorts. DFA was used to classify BCa histology subtypes (n=104) with T2-weighted images, while MBA was used to interrogate tumor physiology by deriving mpMRI QIBs, including apparent diffusion coefficient (ADC), and volume transfer constant (Ktrans) obtained from 34 BCa patients.
[0036] Results: AI-BLADE can extract deep features from the MRI-DFA toolkit for the classification of BCa histology subtypes, and the Al model, VGG19, achieved the highest area under the curve of ROC (AUC) of 0.79, demonstrating strong performance. The MRI-MBA toolkit was seamlessly used to process mpMRI data, providing parametric maps and QIBs numerical values for use toward clinical endpoints. The mean ADC and Ktransvalues were 1.22xl0'3(mm2 / s) and 0.27 (min-1), respectively, reflecting underlying tumor physiology.
[0037] Conclusion: The AI-BLADE toolbox can improve clinical imaging practices and BCa research. Its capabilities are especially valuable in busy hospitals or practices, where timely decision-making is critical for optimizing patient outcomes.
[0038] Advances in knowledge: This is the first study to design, develop, and implement a novel bladder-specific Al toolbox for analyzing mpMRI data. AI-BLADE-7- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450enables advanced image analysis workflow, facilitating AI-QIB-based clinical decisionmaking for patients with BCa.Introduction
[0039] The model-based analysis of quantitative multiparametric (mp) magnetic resonance imaging (MRI) data has been accelerated by the availability of software, either by the vendor, in-house developed, or by open-source tools for clinical applications. For example, bladder cancer (BCa) detection and characterization have been improved due to developments in mpMRI, including diffusion-weighted (DW)- and dynamic contrast-enhanced (DCE)-MRI data acquisition and analysis. BCa typically originates in the urothelial cells lining the bladder’s interior and is classified as either non-muscle invasive or muscle-invasive, depending on whether the cancer has invaded the bladder wall. An accurate assessment of tumor stage, histological subtype, and risk of progression is essential for guiding optimal BCa treatment, including bladder-sparing approaches, neoadjuvant chemotherapy, or radical cystectomy. Therefore, a standardized approach for acquisition, interpretation, and reporting, termed VI-RADS (Vesical Imaging-Reporting and Data System), was developed through consensus from existing literature on bladder mpMRI. However, the VI-RADS scoring system is qualitative in nature and could be enhanced by incorporating quantitative values of the biomarkers derived from mpMRI data.
[0040] Recent advancements in artificial intelligence (Al) for BCa present unparalleled opportunities to improve diagnosis and outcome prediction using the mpMRI data in BCa. Al architectures, particularly convolutional neural networks (CNNs), have demonstrated remarkable success in enhancing image processing capabilities, with pretrained architectures significantly reducing computation time. The deep learning models, trained on large datasets, can capture texture and shape details of underlying imaging patterns. These models have gained a lot of attention in medical imaging by enhancing diagnostic accuracy and enabling Al-powered advancements in healthcare.
[0041] In BCa, model-based mpMRI data analysis is still essential as it helps to derive biomarkers that capture insight into tumor physiology and have shown promise in their clinical application. In DW-MRI, the image contrast is generated based on Brownian motion of water molecules within tumor tissues. The apparent diffusion coefficient (ADC) derived -8- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450from mono-exponential modeling of the DW-MRI data, reflecting the interstitial water mobility and cell membrane integrity, shows improved evaluation tumor staging, characteristics, and treatment response in BCa. DW-MRI data modeling using an extended non-gaussian intravoxel incoherent motion model (NG-IVIM) enables the assessment of the deviation of Brownian motion of water molecules in tumor tissue extracellular space from the Gaussian assumption, as well as the diffusion of water molecules within the capillary network. On the other hand, the time series of contrast agent (CA) kinetic curves obtained from T1 -weighted DCE-MRI data allows for the evaluation of tumor microvascular integrity and vessel permeability using a gadolinium-based CA. The Patlak model is a simplified form of the extended Tofts model, and both models are commonly used for pharmacokinetic analysis of the DCE-MRI data that provides an estimate of tumor vessel perfusion and permeability.
[0042] In general, quantitative imaging biomarkers derived from these model-based methods are estimated using either linear or nonlinear least squares fitting approaches by minimizing the cost function. Initial results suggest that fitting DW- and DCE-MRI data, which were analyzed using a conventional quantitative or semi-quantitative approach, are ineffective for bladder tumor staging and assessing the pathological response of neoadjuvant chemotherapy (NAC) or radiotherapy in BCa patients. For BCa clinical applications, the mpMRI data analyses have been done either using dedicated in-house tools, freely available software, or vendor-provided solutions on standalone advanced workstations.
[0043] There is an unmet need for a comprehensive bladder-specific Al toolbox that can process both qualitative and quantitative mpMRI images, derive robust biomarkers for clinical applications, and offer a one-stop-shop solution for BCa clinical applications. The system and methods described herein are a novel, vendor-agnostic, flexible, and user-friendly Al toolbox to process bladder mpMRI data, including T2W, DW, and DCE-MRI, for clinical use seamlessly.Materials and Methods:AI-BLADE Overall Workflow and Core Functionalities
[0044] The Al-powered BLADdEr multiparametric MRI Analysis for Clinical Application (AI-BLADE) is an intuitive, vendor-neutral software suite designed for -9- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450comprehensive quantitative analysis of mpMRI data. As illustrated in FIGs. 1 A-D, the AI-BLADE workflow provides a streamlined and efficient approach to analyzing mpMRI bladder data. FIGs. 2A and 2B show the graphical user interface (GUI), which offers an intuitive, step-by-step process to streamline analysis. The core functionality of AI-BLADE VI. 0 comprises two key components: (i) Deep Features Analysis (MRI-DFA) which enables the extraction and analysis of deep features from standard anatomical images such as T2-weighted(w) MRI images, and (ii) Data-driven Model-Based Analysis (MRI-MBA) allows quantitative analysis of DW- and DCE-MRI data. At present, the MRI-DFA is available as a Python-based toolkit, offering flexibility and ease of integration into a variety of imaging data. Meanwhile, the MRI-MBA is a MATLAB-based toolkit. Both toolkits are designed to provide researchers with advanced, automated tools for accurate, reproducible, and scalable MRI data analysis and interpretation.Preprocessing Steps
[0045] AI-BLADE natively supports NIfTI image files and includes an interface for converting DICOM images. The current toolbox facilitates the integration of mpMRI data, as illustrated in FIG. 1A (Preprocessing Block), including standard T2W, DW-, and DCE-MRI, which are essential for prospective clinical imaging. Tumor segmentation was performed on T2W images by an experienced genitourinary (GU) radiologist. The segmented Regions of Interest (ROIs) were stored as labeled masks in NIfTI format, preserving the spatial resolution and alignment with the original images for further processing. To ensure compatibility with the MRI-DFA / MBA toolkits, ITK-SNAP was used for tumor contouring and saving the NIfTI files. The delineated ROIs from T2W, DW-, and DCE-MRI images were used as input and integrated into the AI-BLADE platform for subsequent analysis, as described in the sections below.MRI-DFA:
[0046] The MRI-DFA is a comprehensive, deep learning-based toolkit designed to facilitate the MRI data analysis and classification of various clinical endpoints, such as BCa histology subtypes. This toolkit leverages open-source Al, image informatics libraries, and statistical toolbox in a modular fashion, providing a streamlined and efficient workflow for medical imaging analysis. There are three blocks A-C (FIG. IB; MRI-DFA), and the -10- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450rationale behind their design with functionalities is as follows: MRI-DFA Block-A is dedicated to further preprocessing of T2W images, facilitating the conversion of T2W images in NIfTI format into a NumPy format compatible with deep learning models. N4 bias field correction is also performed to mitigate low-frequency intensity non-uniformity on T2W images, as this can distort overall signal intensity. The block also includes functionality for partitioning data into training and testing sets, ensuring that deep learning models are provided with appropriately formatted and divided datasets. Other key preprocessing steps encompass normalization and resizing of T2W images. Normalization ensures that pixel intensity values are scaled to a consistent range, which is critical for stabilizing the training process of the deep learning models.
[0047] MRI-DFA Block-B provides access to a variety of robust deep-learning models, each with unique architecture and strengths. The models included in this block are ResNet-18, AlexNet, ResNet-152, VGG-16, ResNet-101, DenseNet-201, EfficientNet-B7, GoogLeNet, EfficientNet-V2, EfficientNet-V3, DenseNet-121, SqueezeNet, MobileNet_V2, MobileNet_V3, Wide_ResNetl01_2, RegNet_X_32, and ShuffleNet-V2. All models in Block-B are used with pretrained weights, eliminating the need for retraining and allowing for immediate application to new datasets. The inclusion of multiple models provides flexibility, enabling users to select the most appropriate model based on their specific data characteristics and analysis needs. Pretrained models are advantageous because they have been trained on large-scale datasets, learning intricate features and patterns that are transferable to new tasks.
[0048] MRI-DFA Block-C is responsible for extracting features from the fully connected (FC) layers of the deep learning models. The feature extraction process can be tailored to meet the specific requirements of the user’ s dataset, ensuring that the most relevant and discriminative features are captured. The FC layers in deep learning models are particularly valuable because they integrate high-level abstract representations learned from earlier convolutional layers. These representations capture complex patterns and dependencies in the data, which are essential for accurate classification or further analysis. Extracting features from the FC layers allows the representations to be used in a compact, fixed-size format that is more suitable for downstream tasks, such as classification or regression. This approach has been widely used in deep learning models to improve the-11- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450performance of image classification tasks, as the FC layers are designed to map the learned features to decision boundaries.MRI-MBA
[0049] MRI-MBA is a toolkit designed to facilitate the quantitative analysis of DW-and DCE-MRI data, enabling the capture of deeper insights into physiology, such as tumor cellularity, tissue microstructure, and tumor permeability / perfusion characteristics. This toolkit is seamlessly integrated to derive quantitative imaging biomarker (QIB) metrics from bladder mpMRI data. MRI-MBA comprises two blocks (FIG. 1C; MRI-DF A): MRI-MBA Block-A is dedicated to further preprocessing of image data, similar to MRI-DFA Block-A.MRI-MBA Block B is dedicated to data-driven model-fitting (mono-exponential model and NG-IVIM for DW-MRI, and the Patlak model and the extended Tofts model for DCE-MRI) to derive QIBs from bladder mpMRI data.
[0050] DW MRI: The amount of DW signal intensity attenuation due to diffusion weighting (b-value) modeled using a mono-exponential approach allows the calculation of ADC values (Equation [1]). The NG-IVIM model extends the diffusion term of the IVIM model by incorporating the diffusion kurtosis coefficient (Equation [2]), which describes diffusion phenomena in the extracellular space and capillary network and also characterizes tissue microstructure. In typical diffusion models, water molecules are assumed to undergo Gaussian diffusion. However, the NG-IVIM model accounts for deviations from this assumption, considering the more complex nature of tissue diffusion processes.sb _ -bxADC [J]O— = f~bD* + (1 - p]where Sb and So are the signals with and without b (s / mm2) values, ADC (mm2 / s), D (mm2 / s), and D*(mm2 / s) are the apparent, true, and pseudo-diffusion coefficients (mm2 / s), respectively, f is the perfusion fraction, and K is the kurtosis coefficient (unitless). These quantitative imaging biomarkers (QIBs) are surrogates for tumor cellularity, capillary perfusion, and tissue microstructure.-12- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450
[0051] DCE-MRI: In DCE-MRI, the time course of signal intensity data acquired using a known data acquisition method, such as spoiled gradient recalled echo, is related to the longitudinal relaxation rate R1 (R1 = 1 / T1) and can be expressed as follows under the fast exchange limit approximation:= Rio + G x Ct(t) x Ct(t) [3]where Rio is the pre-contrast relaxation rate of the tissue, n is the longitudinal relativity of Gadolinium-based contrast agent (CA), and Ct(t) is the CA concentration in tissue, respectively. In DCE-MRI, pharmacokinetic models are commonly used to analyze the enhancement of tissue signal data by calculating the tissue CA concentration. Models such as the Patlak and extended Tofts are frequently employed to analyze DCE-MRI data, providing valuable insights into tumor tissue perfusion / permeability, and other physiological parameters. The Patlak plot, a graphical analysis technique, is based on a unidirectional model for the transfer of CA from the vascular space to the extravascular extracellular space (EES). This technique uses a linear fit of multiple-time tissue uptake data of the CA to estimate key metrics, including the volume transfer constant (Ktrans) and the blood plasma volume fraction (vP).
[0052] The two parameters of the Patlak model are expressed asGW= Ktransx4 Gtodr[4]Cp(t) Cp(t) P
[0053] The three parameters extended Tofts model is given by equation [5]:Ct(t) = Ktransx f° ekeP(-t T^T) x Cp(r)dT + Cp(t) x vp[5]where kep= Ktrans / verepresents the rate constant of CA transport from the EES to vascular space, veis the volume fraction of the EES, and CP(t) is the time course of plasma CA concentration (called arterial input function [AIF]). Ktrans, ve, and vPare the QIBs that reflect tumor perfusion and permeability, leakage space of CA, and vascular integrity.
[0054] The final analysis for clinical endpoints is achieved by using three BLOCKS D-F, which are briefly detailed here:-13- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450Feature Selection
[0055] Once deep features from MRI-DFA or parametric maps from MRI-MBA are extracted from the respective models, they may be used for feature selection (FIG. ID; Block-D). This block includes several algorithms for selecting the most relevant features, including (Minimum Redundancy Maximum Relevance) MRMR, Chi-squared (Chi-2), ReliefF, ANOVA (Analysis of Variance), and Kruskal Wallis algorithms. Feature selection helps reduce the dimensionality of the data and improves the efficiency and performance of the classification models by focusing on the most informative features.Classification
[0056] Block-E (FIG. ID) has multiple classifier options to perform the final classification, from bladder mpMRI for tumor detection, identifying histology subtypes, and risk stratification to prediction of response and outcome. By offering various classifiers, the toolbox allows users to choose the best-suited method for their specific analysis, enhancing the robustness and accuracy of the classification results. The classifiers include traditional machine learning algorithms such as Support Vector Machines (SVM), Decision Tree, and k-Nearest Neighbors (k-NN).Statistical Analysis
[0057] Block-F (FIG. ID), the final block, plays a pivotal role in providing a statistical evaluation of the classification model’s performance. It calculates key metrics, including the Receiver Operating Characteristic (ROC) curve, accuracy, sensitivity, specificity, precision, and the Fl score. These metrics collectively offer a thorough and nuanced assessment of the model, addressing its ability to differentiate between classes, handle imbalances, and maintain reliability in its predictions. The inclusion of the ROC curve, which plots the true positive rate against the false positive rate, provides a visual and quantitative measure of the model’s discriminatory power across various thresholds. By reporting these statistical parameters, Block-F ensures that the classification results are reliable and interpretable, aiding researchers and practitioners in making informed decisions based on the model's output. Presented herein are the classification results for BCa histology subtypes using the ROC curve to showcase the model's robustness and overall effectiveness.-14- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450MRI-DFA
[0058] For this study as a clinical test case, developed bladder-specific Al toolbox AI-BLADE was developed to extract deep features (MRI-DFA) for classifying BCa histology subtypes. MRIs were acquired using an imaging protocol, and data from 104 BCa patients (median age: 66 years; 66 M, 38 F) was included. The pathology evaluation classified 65 patients as pure urothelial carcinoma (UC) and 39 patients as non-predominant variant histology (npVH). Deep features were extracted from four Al models for this analysis.Final Analysis (Blocks D-F)
[0059] The performance of these models was evaluated using the Area Under the Curve (AUC) metric. Higher AUC values, closer to 1, indicate stronger discriminatory power, meaning the model is more effective at differentiating between PureUC and npVH. The final results for the four Al models are as follows:
[0060] ResNetl52: The ResNetl52 model in the MRI-DFA toolkit with the KNN classification algorithm was evaluated using two feature selection techniques: ANOVA and MRMR. The model utilized a Fully Connected (FC) layer for classification. Regardless of whether ANOVA or MRMR feature selection was applied, the ResNetl 52 model consistently yielded an AUC value of 0.61, as shown in FIG. 3, panel A. This result suggests that the choice of feature selection method did not significantly affect the model’s performance. EfficientNet-B7: Using multiple settings from the MRI-DFA toolkit, deep features were extracted from the EfficientNet-B7 model, and the KNN classifier was employed with Kruskal-Wallis feature selection. This configuration resulted in an AUC of 0.65, which is lower compared to the AUC of 0.73 achieved with the Decision Tree classifier and no feature selection, as shown in FIG. 3, panel B.
[0061] VGG19: Using the MRI-DFA toolkit settings with the VGG19 model, the Decision Tree classifier, no feature selection, and (FC7), an AUC of 0.79 was achieved, as shown in FIG. 3, panel C. This was the highest AUC among all models tested, indicating strong performance in distinguishing between the npVH and Pure UC classes in the dataset.
[0062] AlexNet from the MRI-DFA toolkit settings with the AlexNet model, the SVM classifier, and a Fully Connected (FC7) layer without feature selection achieved an -15- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450AUC of 0.69. When the KNN classifier was used with the FC7 layer and ReliefF feature selection, the AUC decreased to 0.60, as shown in FIG. 3, panel D. This suggests that while the feature selection method was applied, it did not enhance the model’s performance in this case and, in fact, resulted in a lower AUC. The MRI-DFA toolkit allowed comprehensive evaluation of these models across multiple settings and algorithms, including SVM, KNN, and decision tree classifiers. This one-stop-shop solution provided flexibility in testing various configurations, ensuring a thorough assessment of model performance on the given dataset.
[0063] A detailed comparative analysis of all configurations using the additional pretrained models integrated within the MRI-DFA framework is provided in the Appendix and FIGs. 7A-J. This table serves as a reference for model benchmarking and highlights the relative contribution of each network to the MRI-DFA toolkit’s classification capabilityMRI-MBA Derived QIBs
[0064] MRI-MBA facilitates the quantitative analysis of DW- and DCE-MRI data, enabling deeper insights into tumor cellularity, tissue microstructure, and tumor permeability / perfusion characteristics. This tool is seamlessly integrated to derive quantitative imaging biomarkers from mpMRI data. The MRI-MBA workflows for DW-MRI utilized both the mono-exponential (metric: ADC) and NG-IVIM (metrics: D, D*, f, and K) models, which effectively quantified tumor diffusion and microvascular perfusion fraction as well as tissue microstructure parameters. FIGs. 4A and 4B show the representative parametric maps generated from the MRI-MBA toolkit for DW data, which utilized both the monoexponential and NG-IVIM models. The mean (+SD) values of ADC, D, D*, f, and K values were 1.22+0.32* 10'3(mm2 / s), 1.21+ 0.37*10'3(mm2 / s), 25.57+8.57*10'3(mm2 / s), 0.24+ 0.06, and K= 0.41+0.16, respectively, were obtained from patients with pure UC of NMIBC (n=22) prior to treatment.
[0065] FIG. 5A presents the MRI-MBA workflows for DCE-MRI, where the Patlak (Ktransand vP) and extended Tofts models (Ktrans, ve, and vP) were applied to the bladder dataset. These models enabled the extraction of perfusion-related parameters, which are essential for evaluating tumor permeability and perfusion. The results highlight the utility of MRI-MBA in enhancing quantitative mpMRI analysis, providing valuable quantitative -16- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450imaging biomarkers for clinical applications in bladder cancer, ranging from histology subtypes classification to predicting treatment response and outcomes.
[0066] FIG. 5B shows the representative parametric maps of the parameters, which were generated from the Patlak model (Ktransand vP) and the extended Tofts model (Ktrans, ve, and vP), respectively. The mean (± SD) values of the extended Tofts model-derived Ktrans, ve, and vPwere 0.27±0.22 (min-1), ve=038±0.19, and vP=0.03 ±0.02, respectively. For the Patlak model, Ktransand vPvalues were 0.076±0.075 (min-1) and 0.079±0.073, respectively. These QIBs were extracted from NMIBC patients (n=34) prior to treatment. The mean (±SD) Ktrans, Ve, and vPvalues derived from the extended Tofts model for pure UC (n=31) and npVH (n=3) were as follows: 0.28±0.23 vs. 0.19±0.09, 0.40±0.19 vs 0.20±0.16, and 0.0243±0.0205 vs.0.020±0.008, respectively. The results highlight the utility of MRI-MBA, which provides valuable QIBs for clinical applications in bladder cancer.Discussion
[0067] In clinical imaging, there is a growing demand for advanced quantitative imaging tools with Al-driven models for specific clinical endpoints. Currently, available tools for quantitative analysis of mpMRI data from maj or MRI vendors are still closed-source, limiting their broader adoption and utilization. Unlike these existing solutions, the BLADE-AI toolkit is the first bladder-specific vendor-agnostic tool for analyzing mpMRI data. It has capabilities to perform such as histological pattern recognition, predictive analytics, and realtime monitoring that can be used easily by both clinicians and imaging researchers. The toolbox enabled the exploration of different Al and machine learning algorithms and configurations, allowing for a thorough evaluation of model performance across a variety of settings. The results in BCa highlight the effectiveness of the proposed MRI-DFA toolkit in tackling complex classification tasks, demonstrating that deep learning models can be highly effective when appropriately configured. This toolkit proved to be valuable for model evaluation, offering a one-stop-shop solution for testing multiple options, adjusting settings, and optimizing configurations to enhance performance. By incorporating pretrained deep learning architectures and feature selection, the MRI-DFA approach reduces computational demands and improves model performance by focusing on the most relevant features.-17- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450
[0068] Additionally, MRI-MBA provides estimates of QIBs that can capture tumor cellularity, tissue microstructure, and tumor permeability / perfusion, which are crucial for clinical applications in BCa patient management. The parametric maps derived from DW-and DCE-MRI reflect tumor heterogeneity and provide a non-invasive window to tumor physiology. The derived QIBs have shown promise in clinical endpoints in BCa.
[0069] The blocks D-F in this toolbox allow users to perform step-wise feature selection and classification towards various clinical endpoints followed by statistical analysis. This adaptability in feature selection implies that the AI-BLADE toolbox can be effectively utilized across a broad spectrum of clinical applications. The combination of different biomarkers provides an understanding of tumor biology, facilitating more personalized treatment options. AI-BLADE toolbox has the potential to impact both imaging research and clinical practice in BCa significantly. AI-BLADE may be accessible via web-based cloud architectures for broader availability and use (FIG. 6).
[0070] The deep features extracted from FC layers of CNNs can capture hierarchical and abstract patterns in imaging data beyond what handcrafted features can achieve. In the present disclosure, deep feature extraction from T2W imaging data was paired with classical classifiers to form a robust and interpretable framework. VGG19 was combined with a Decision Tree classifier, and no feature selection achieved the highest performance (AUC=0.79). The EfficientNet-B7 demonstrated strong performance (AUC=0.73) without feature selection. Similarly, ResNetl52 and Al exNet consistently yielded lower AUCs, regardless of feature selection or classifier choice. Extracted features were assessed both with and without feature selection prior to classifier training.
[0071] The strong performance observed in the BCa classification with VGG19 can be attributed to its sequential convolutional design using small 3^3 filters and deeper hierarchical layers, which enhance feature abstraction and spatial detail capture within bladder lesions. Unlike architectures such as DenseNet or EfficientNet that rely on dense connectivity or compound scaling, VGG19 provides a more stable feature extraction process, reducing the risk of overfitting in small and heterogeneous datasets. These findings suggest that moderately deep, uniform convolutional architectures may offer a favorable balance between generalization and discriminative power for bladder mpMRI analysis. Emerging as the most effective feature extractor, VGG19 underscores the potential of the MRI-DFA -18- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450toolkit as a generalizable framework for broader clinical applications by leveraging pretrained deep learning architectures and integrating feature selection strategies.
[0072] Quantitative DW- and DCE-MRI, through their derived QIBs, together interrogate tumor physiology by evaluating cellularity and cell membrane integrity via water diffusion and vascular permeability and integrity using CA, respectively. The degree of DW-MRI signal attenuation caused by restricted or hindered water diffusion depends on the changes in extracellular extravascular space (EES) volume and increased tissue tortuosity. Areas of restricted diffusion appear as high signal intensity on DW images and correspond to low apparent diffusion coefficient (ADC) values on ADC maps. These imaging characteristics provide diagnostic and prognostic insights, such as tumor stage, grade, and size, correlated with histopathological findings.
[0073] Bladder tumors typically exhibit higher cellular density, leading to lower ADC values than normal bladder wall (1.05 ± 0.22* 1 O'3mm2 / s vs. 1.83 ± 0.18* 10'3mm2 / s). It was reported that muscle-invasive bladder cancer (MIBC) exhibited significantly lower ADC values than the non-muscle-invasive bladder cancer (NMIBC) (0.759* 10'3mm2 / s vs.1.120* 10'3mm2 / s). Similarly, lower ADC values for MIBC of 0.964* 10'3mm2 / s were reported and compared to NMIBC 1.205*1 O'3mm2 / s (the highest b-value of 2000 s / mm2). In the present disclosure, the ADC value of 1.22* 10'3mm2 / s for the NMIBC cohort is within the range of reported values.
[0074] ADC is a composite metric that accounts for both diffusion within the EES and the capillary network, which follows a Gaussian distribution for water diffusion. The deviation of diffusivity from a non-Gaussian distribution arises from tissue microstructural complexity. The curvature observed in signal decay at higher b-values is effectively captured by the diffusion kurtosis imaging (DKI) model, characterized by QIB, DaPP, and KaPP. One approach used DKI-derived parameters to stratify MIBC and NMIBC; for MIBC, DaPPvalues were 1.635* 10'3(mm2 / s) and KaPP=0.78, respectively, whereas for NMIBC these values were 2.038* 10'3(mm2 / s) and 0.610, respectively. NG-IVIM estimates ‘K’ in addition to the true molecular diffusion in the EES and perfusion-related QIBs in the capillary network (i.e., f and D*). The K-value (derived from NG-IVIM rather than the DKI model) was 0.41, slightly lower than in another report. This could be due to different data modeling approaches, as well as the use of different multiple b-value DW data acquisition protocols.-19- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450
[0075] The DCE-MRI signal enhancement pattern exhibited is closely reflected by tumor vascular integrity. For example, NMIBC tumors, which are typically smaller and more superficial, with small areas of necrosis, often exhibit functionality of the tumor vasculature, contributing to higher contrast enhancement. The semi-quantitative analysis of DCE data can extract parameters such as time-intensity curves, wash-in / wash-out rates, and AUC. These quantities have demonstrated potential for tumor staging and grading but lack physiological specificity and do not directly quantify tumor physiological QIBs, such as Ktrans, which reflects both blood flow and vascular permeability, with its value depending on the tissue characteristics. The Patlak model may be applied in BCa, yielding Ktransand vpvalues of 0.076 min'1and 0.075, respectively.
[0076] The Patlak model is simpler, linear, computationally efficient, and less sensitive to noise compared to nonlinear models like the Tofts model. It also accounts for irreversible contrast agent extravasation from plasma into the EES without backflux, suitable for quantifying low levels of CA extravasation. However, for significant backflux, a full reversible two-compartment model, the extended Tofts model, may be necessary to quantify and maintain the quantitative accuracy of QIBs. One approach used the extended Tofts model in patients with histologically confirmed BCa (stage T2-T4) and reported the values of Ktrans= 0.085 min'1, ve= 0.34, and vp= 0.0095. In the present disclosure, these corresponding values fortheNMIBC cohort derived from the extended Tofts model were 0.27 min'1, 0.38, and 0.03, respectively. The mean Ktrans, ve, and vpvalues showed a trend toward differences of 33%, 49%, and 17%, respectively, between with pure UC (n= 31) and those with npVH (n= 3) among with NMIBC patients. The mean Ktransvalue is within the range of the reported value; however, these QIBs numerical value may vary, depending on the data acquisition protocol, signal quality, the specific models used for data fitting, and the tumor tissue under study.Conclusion
[0077] AI-BLADE is a flexible, vendor-agnostic, and user-friendly software suite for the analysis of mpMRI data for clinical endpoints. AI-BLADE shows promise to impact both clinical imaging practice and bladder cancer research. This is especially valuable in busy hospitals or practices, where timely decision-making is critical for optimizing patient outcomes. The MRI-DFA toolkit integrates novel deep learning models with advanced feature selection methods to improve diagnostic accuracy, speed, and interpretability in -20- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450clinical research. In parallel, the MRI-MBA toolkit derives physiological QIBs through data-driven modeling of DWand DCE-data, enabling characterization of the tumor microenvironment. Together, deep features and QIBs offer complementary insights into the tumor microenvironment that can enhance the diagnosis, monitoring, and treatment of BCa, particularly in clinical settings where timely decision-making is critical for effective patient management.Appendix
[0078] Table 1. Bladder MRI protocolParameter Tiw Tiw DW DCEField Strength 1.5 T / 3T 1.5 T / 3T 1.5 T / 3T 1.5 T / 3T Sequence FRFSE FRFSE SS-EPI FSPGR Plane Orientation Multiplanar Multiplanar Axial Axial Field of view (FOV) 250-350 250-350 200-250 200-250 (mm)Repetition time (TR) 400-700 4000-6000 3500-5000 3.5-4.5 (ms)Echo time (TE) (ms) 1.2-2.4 82-120 60-80 1.2-2.24. 256-320 x 192- 256-320 xAcquisition Matrix 128 x 128 256 x 192256 192-256Slice thickness / gap3-4 / i_03-4 / 1-0 4-0 / 1-0 4-0 / 1-0 (mm)Number of excitations 2 2 2-8 1 Flip Angles (FAs)1(deg)0 and 800- # b-values (s / mm2) N / A N / A 1000, up to N / A 2000 optional
[0079] Tlw: T1 weighted imaging; T?w: T2 weighted imaging; DW: Diffusion weighted; DCE: Dynamic contrast enhanced.
[0080] Tio mapping acquisition: FAs: 5°, 15°, and 30°; Other MR parameters are the same as DCE acquisition.4906-1384-1819.1Atty. Dkt. No.: 115872-3450B. Systems and Methods for Classifying Tumors in Bladders Using Magnetic Resonance Imaging (MRI) Data
[0081] Bladder cancer diagnosis faces several technical challenges, particularly in the areas of early detection, tumor staging, and differentiation between non-muscle invasive and muscle-invasive tumors. Other diagnostic tools such as cystoscopy and urine cytology are limited such that cystoscopy is invasive and can miss flat lesions such as carcinoma in situ and cytology has low sensitivity for low-grade tumors. Imaging techniques such as magnetic resonance imaging (MRI) scanning provide structural information but often struggle with clearly delineating tumor boundaries, differentiating tumor from inflammation, or accurately staging small or subtle lesions. Moreover, the heterogeneity of bladder cancer, such as varying shapes, sizes, and growth patterns, makes it difficult to develop protocols to perform across all patient cases.
[0082] While machine learning (ML) models can be used to analyze medical images or cytology slides, they still face significant challenges. For one, there may be the lack of well annotated dataset specific to bladder cancer, which limits the generalizability of ML models. For another, bladder imaging data are often noisy and variable due to differences in scanners, patient movement, and imaging protocols, which challenges model robustness. ML models also struggle with interpretability as physicians need transparent outputs to support the diagnosis, yet many models act as black boxes. As a result, ML systems for bladder cancer diagnosis may consume significant computing resources to interpret the varying imaging data with lower accuracy or applicability. Furthermore, these systems may lack ability to effectively process multi-parametric MRI data.
[0083] To address these challenges, presented herein are systems and methods for classifying tumors in bladders using MRI data. A data processing system may have a machine learning architecture that integrates a wide set of heterogeneous models to support deep feature analysis and data-driven model-based analysis forbladder cancer clinical applications. The data processing system may receive MRI images in different sequences, such as T2-weighted imaging, diffusion-weighted imaging (DWI), and dynamic contrast-enhanced (DCE) MRI, among others. The data processing system may identify a region of interest (ROI) within each MRI image, for example, via clinician input or automated segmentation. The data processing system can use a set of feature extraction models to extract deep features -22- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450from the MRI image data (e.g., about the indicated ROI). The data processing system may use a set of metric evaluation models to derive physics-based metrics from the MRI image data. These may include diffusion-related metrics (e.g., apparent diffusion coefficient, true diffusion coefficient, perfusion fraction, kurtosis) and perfusion-related metrics (e.g., Ktrans, Ve, Vp), among others. Using the features and the metrics, the data processing system may execute classification models to generate various classes and measures to characterize the MRI image data. The output from the architecture may be used to facilitate diagnosis and clinical decisions.
[0084] In this manner, by combining feature extraction models with physics-based metric models, the data processing system may avoid retraining large networks while providing features and metrics, thereby avoiding computational overhead from repetitive retraining. In addition, the feature and metric models may reduce high-dimensional data into compact, informative representations, improving execution speed and memory efficiency for large imaging datasets. The feature extraction and quantitative metric evaluation can operate on different MRI sequences, enabling efficient parallel computation and improved throughput. From a clinician perspective, the creation of physics-based imaging biomarkers and parametric maps can improve transparency of Al outputs compared to black-box image-only models. Furthermore, the integration of extracted features with quantitative imaging biomarkers can improve tumor detection, subtype classification, and risk stratification, beyond manual qualitative MRI interpretation alone.
[0085] Referring now to FIG. 8, depicted is a block diagram of a system 100 of classifying bladder risk using magnetic resonance imaging (MRI) data. In a brief overview, the system 100 can include at least one computing device 110 including a user interface 130, at least magnetic resonance imaging (MRI) scanner 115, at least one data processing system 120, and at least one database 125, among others, communicatively coupled via at least one network 101. The data processing system 120 can include at least one image indexer 135, at least one label generator 140, at least one model selector 145, at least one feature extractor 150, at least one metric evaluator 155, at least one classification generator 160, a plurality of feature extraction models 165, a plurality of metric evaluation models 170, a plurality of classification models 175, and at least one output handler 180, among others. The system 100 may be used to implement the functions detailed herein in Section A. Each of the-23- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450components of the system 100 can be implemented using the computing system as described in Section C.
[0086] In further detail, the computing device 110 can be any device comprising one or more processors coupled with memory and software and capable of providing an output classification. The computing device 110 can be associated with an entity (e.g., clinician, physician, doctor) examining the subject or biomedical images from the subject. The computing device 110 can be in communication with the data processing system 120 and the imaging device 110 to exchange data. The computing device 110 can display images acquired from the imaging device 110 on a display.
[0087] The MRI scanner 115 can be any device capable of acquiring MRI images of a subject. The MRI scanner 115 can obtain, generate or otherwise acquire MRI images of at least one organ of a subject, such as the bladder. The MRI scanner 115 can capture the image by using magnetic fields and radio waves to align hydrogen atoms within the body. The acquisition of images may be in accordance with any number of imaging sequences, such as T1 -weighted imaging, T2-weighted imaging, diffusion-weighted imaging (DWI), and dynamic contrast-enhanced (DCE)-MRI, among others. The image data provided by the MRI scanner 115 may be multiparametric, with images in multiple imaging sequences. The MRI scanner 115 can be in communication with the data processing system 120 and the computing device 110 to provide acquired images. The MRI scanner 115 can be any non-invasive imaging device (e.g., medical imaging device) to capture, detect, or otherwise identify images of soft tissue within a subject 210 (e.g., bladder, brain, stomach, liver). Although the context of the following description is described primarily for bladders, the aspects of the present disclosure can be used for other soft tissues within a subject. The MRI scanner 115 can include hardware and software to capture images such as a main magnet, a computing system, radiofrequency coils, gradient coils, among other components. The MRI scanner 115 can include a user interface to receive inputs from a clinician 205 (e.g., doctor, physician, or technician examining a subject, etc.).
[0088] The data processing system 120 can be any computing device comprising one or more processors coupled with memory and software capable of performing the various processes and tasks described herein. The data processing system 120 can be housed within a computing system (e.g., laptop, PC, smart device) or within a server group (e.g., a data -24- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450center, a branch office, or a server site), and include instructions to manage the identifying of images, generating a classification, and storing an association. The data processing system 120 can be in communication with the MRI scanner 115, the computing device 110, and the database 125, among others.
[0089] The image indexer 135 can receive, retrieve, or otherwise obtain a plurality of MRI images from the MRI scanner 115. The label generator 140 can generate, determine, or otherwise indicate regions of interest (ROIs) from the plurality of MRI images. The model selector 145 can select, identify, or otherwise determine at least one feature extraction model 165. The feature extractor 150 can extract, determine, or otherwise indicate at least one feature from the ROIs. The metric evaluator 155 can use the metric evaluation models 170 to determine, generate, or otherwise identify a plurality of metrics. The classification generator 160 can generate, determine, or otherwise identify a classification based on the MRI image.
[0090] The feature extraction models 165 can be any type of ML algorithm or model to determine a plurality of features defining characteristics of a tumor within the bladder. The feature extraction models 165 can be maintained on the data processing system 120. The architecture for the feature extraction models 165 may be in accordance with any number of artificial intelligence or machine learning models, such as a deep learning artificial neural network (ANN) (e.g., convolutional neural network (CNN), residual neural network (RNN)), a long short-term memory model (LSTM model, or transformer model), a clustering algorithm, a support vector machine (SVM), a decision tree, a Bayesian model, a regression model, among others. The feature extraction models 165 can be, for example, a deep learning convolutional neural network (CNN) such as an ResNet-18, AlexNet, ResNet-152, VGG-16, ResNet-101, DenseNet-201, EfficientNet-B7, GoogLeNet, EfficientNet-V2, EfficientNet-V3, DenseNet-121, SqueezeNet, MobileNet_V2, MobileNet_V3, Wide_ResNetl01_2, RegNet_X_32, and ShuffleNet-V2, among other architectures. In general, the feature extraction models 165 can have a ROI within a bladder as an input and a plurality of features as an output. The feature extraction modes 165 may have been initialized, trained, and established using training data in accordance with learning techniques (e.g., supervised).
[0091] The metric evaluation models 170 can be any type of ML algorithm or model to determine a parametric map defining the tumor within the bladder. The metric evaluation -25- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450models 170 can be maintained on the data processing system 120. The architecture for the metric evaluation models 170 may be in accordance with any number of artificial intelligence or machine learning models, such as a deep learning artificial neural network (ANN) (e.g., convolutional neural network (CNN), a residual neural network (RNN)), a long short-term memory model (LSTM model, or transformer model), a clustering algorithm, a support vector machine (SVM), a decision tree, a Bayesian model, a regression model, among others. The metric evaluation models 170 can have a ROI within a bladder as an input and a parametric map as an output. The metric evaluation models 170 may have been initialized, trained, and established using training data in accordance with learning techniques (e.g., supervised).
[0092] The classification models 175 can be any type of ML algorithm or model to generate a classification corresponding to a tumor within the bladder. The classification models 175 can be maintained on the data processing system 120. The classification models 175 can be, for example, deep learning deep learning neural networks (e.g., convolutional neural network (CNN), residual neural network (RNN)), a long short-term memory model (LSTM model, or transformer model), regression model (e.g., linear or logistic regression), clustering model (e.g., k-nearest neighbor, hierarchical clustering, Gaussian mixture model, or density-based spatial clustering of applications with noise (DBSCAN)), random forest, support vector machine (SVM), among others. The classification models 175 can have a parametric map and a plurality of features as an input a classification as an output. The classification models 175 may have been initialized, trained, and established using training data in accordance with learning techniques (e.g., supervised).
[0093] FIG. 9 is a block diagram of a process 200 of determining regions of interest (ROI) within a bladder. The process 200 can include or correspond to operations performed in the system 100. The image indexer 135 can receive, obtain, or otherwise retrieve a set of MRI images 215A-N (sometimes referred to as MRI images 215) from the MRI scanner 115. The clinician 205 can control the MRI scanner 115 to capture, identify, or otherwise obtain the set of MRI images 215 of the bladder 211 of the subject 210. The MRI images 215 can include various orientations or views of the bladder 211, such as a top view, a side view, a bottom view, among other views. The bladder 211 can include lesions, tumor 212, blood vessels, abnormal cells, necrotic tissues, obstructions, calcifications, among others. The MRI scanner 115 can include a user interface to receive inputs from a clinician 205 (e.g., doctor,-26- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450physician, technician, or intern examining a subject 210, etc.). For example, the clinician 205 can use one or more components of the MRI scanner 115 to capture the MRI images 215 of the bladder 211. The MRI images 215 can indicate (e.g., via depiction) that the bladder 211 includes one or more tumors 212. The tumors 212 may be associated with the bladder cancer in the subject 210.
[0094] The acquisition of the MRI images 215 may be in accordance with any number of imaging sequences, such as T1 -weighted imaging, T2-weighted imaging, diffusion-weighted imaging (DWI), and dynamic contrast-enhanced (DCE)-MRI, among others. For instance, the MRI image 215 A may be in accordance with a first imaging sequence (e.g., T1 -weighted or T2-weighted) and the MRI image 215B may be in accordance with a second imaging sequence (e.g., DWI or DCE). In some embodiments, the set of MRI images 215 may be acquired prior to or subsequent to administration of therapy to the subj ect 210 for the bladder cancer. In some embodiments, the set of MRI images 215 may be acquired subsequent to the administration of the therapy for the bladder cancer. The therapy may include, for example, neoadjuvant chemotherapy (e.g., cisplatin-based therapy, or antibodyconjugates (such as enfortumab vedotin (EV) with and without immunotherapy such as pembrolizumab), surgical resection (e.g., radical cystectomy), adjuvant chemotherapy, or radiotherapy, among others. The acquisition time may be prior to or subsequent to the administration of the therapy to the subject 210. The MRI images 215 may be used to facilitate a longitudinal study of the subject 210 over time to track the progress of the bladder cancer or the effects of the therapy.
[0095] The subject 210 may be diagnosed with or at risk of developing bladder cancer. The bladder cancer can be at least one of transitional cell carcinoma papillary urothelial carcinoma, flat carcinoma in situ, invasive urothelial carcinoma, squamous cell carcinoma, adenocarcinoma of the bladder, small cell carcinoma, large cell carcinoma, among other cancers within the bladder. Upon acquisition of the MRI images 215 by the MRI scanner 115, the MRI scanner 115 can transmit the MRI images 215 to the image indexer 135.
[0096] The image indexer 135 can receive, obtain, or otherwise retrieve a plurality of MRI images 215 from the MRI scanner 115. In some embodiments, the image indexer 135 can label or highlight at least one tumor 212 within the bladder 211 of the subject 210.-27- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450The tumor 212 can be associated with the bladder cancer impacting the subject 210. The image indexer 135 can generate an image sequence for the MRI images 215 for display on the user interface 130 of the computing device 110. The image sequence can include MRI images 215 that provide a view of the tumor 212 of the bladder 211. For example, a first image sequence captured at a first time period can be of a first view of the bladder 211. A second image sequence captured at a second time period can be of a second view of the bladder 211. The first view can be different from the second view. Each image sequence can correspond to a different view of the bladder. Once each image sequence is sorted or indexed, the image indexer 135 can provide, transmit, or otherwise send the image sequences or the MRI images 215 to the label generator 140. The image sequences can include at least one of T1 -weighted, T2-weighted, diffusion-weighted (DWI), and dynamic contrast enhanced (DCE) MRI.
[0097] The label generator 140 can identify, generate, or otherwise render a plurality of regions of interest (ROI) 220A-N (sometimes referred to as ROI 220) from within the MRI images 215. The ROI 220 can be a region, area, or portion of the MRI image 215 that defines at least one tumor 212 within the bladder 211 of the subject 210. The ROI 220 can be shown in various forms, such as an irregular shape, a contour based shape, an elliptical shape, pixelwise mask, voxel-wise mask, bounding boxes, label maps, among other representations. The ROI 220 can be annotated manually (e.g., by the clinician 205), semi automatically (e.g., by the clinician 205 and the label generator 140), or automatically (e.g., using image segmentation models).
[0098] In some implementations, the label generator 140 can delineate, define, or otherwise generate the ROIs 220 using image segmentation model. The image segmentation models can be, for example, U-Net, DeepLabV3+, Fully Convolutional Network, or SegNet, among other image segmentation models. The image segmentation model may have been trained using training data. The training data may include a set of example MRI images and manually created annotations defining ROIs (e.g., corresponding to tumors) in the example MRI images. The label generator 140 may apply the image segmentation model to the MRI image 215. Based on the application of the image segmentation model, the label generator 140 may generate a definition of the ROIs 220 within each MRI image 215.-28- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450
[0099] In some embodiments, the label generator 140 may obtain or receive definitions for the ROIs 220 via user input. For instance, the image indexer 135 can include machine readable instructions to generate a user interface 130 when providing the MRI images 215 to the computing device 110. The user interface 130 can display the MRI images 215 on the computing device 110. The user interface 130 can include one or more tools to edit, modify, or otherwise annotate the MRI images 215. The one or more tools can include one or more image segmentation models. The clinician 205 can interact with the MRI images 215 or a mask of the images via the user interface 130. The label generator 140 can detect or receive one or more inputs via the user interface 130 of the computing device 110. The one or more inputs can correspond to a labeling, annotating, or otherwise a tracing of the ROI 220 within each MRI image 215. The label generator can apply the inputs to the one or more image segmentation models to identify the ROI 220 for each MRI image 215.
[0100] The computing device 110 can transmit one or more inputs via the user interface 130. The computing device 110 can include a user interface 130 to mark or label the MRI images 215 within the label generator 140. The image indexer 135 can transmit, send, or otherwise provide a mask, a copy, or a duplication of the MRI images 215 to the computing device 110. Upon reception of the MRI images 215, the user interface 130 of the computing device 110 can receive one or more inputs from the clinician 205 to label or identify the ROI 220. The computing device 110 can transmit the ROIs 220 to the label generator 140 for application to the MRI images 215.
[0101] The label generator 140 can identify or receive at least one score 225 indicating a severity of the bladder cancer in the subject 210. The severity of the bladder cancer can include stage 0 (non-invasive), stage 1 (no muscle involvement), stage 2 (equivocal muscle involvement), stage 3 (high suspicion of muscle involvement), and stage 4 (clears signs of muscle invasion). As the stages increase, the severity can increase. The score 225 can indicate or correspond to a vesical imaging-reporting and data system (VI-RADS) score to access the risk of muscle invasion within the bladder 211. For instance, the score 225 can correspond or include VI-RADS for bladder cancer. The clinician 205 can assign the score 225 based on the MRI images 215 from the MRI scanner 115. For example, the user interface 130 can display the MRI images 215 at the computing device 110. The-29- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450label generator 140 can receive an input that indicates the score 225 for the sequence of images.
[0102] The label generator 140 can apply, input, or otherwise feed the one or more inputs to the one or more image segmentation models. As described above, the image segmentation models can include at least one of U-Net, DeepLabV3+, Fully Convolutional Network, or SegNet, or Swin-Unet, among other image segmentation models. Each of the one or more segmentation models can be pretrained on one or more large datasets. In feeding, the image segmentation models can ingest the one or more inputs on the mask of the MRI image 215 and process the features associated with the inputs. The one or more image segmentation models can output, provide, or otherwise identify ROIs 220 corresponding to the tumor 212 of the bladder 211 of the MRI image 215. The label generator 140 can provide, transmit, or otherwise send the at least one ROI 220 within the MRI images 215 to the model selector 145. Concurrently, the label generator 140 can provide, send, or transmit the at least one ROI 220 within the MRI images 215 to the metric evaluator 155.
[0103] FIG. 10A is a block diagram of a process 300 of generating features using one or more feature extraction models 165. The process 300 can include or correspond to operations performed in the system 100. The feature extractor 150 can apply, feed, or otherwise input at least one feature extraction model 165 to the MRI image 215A (e.g., Tl-weighted or T2-weighted images) and the ROI 220A. The feature extraction models 165 can include at least one of ResNet-18, AlexNet, DenseNet, Movilenet_V2,V3, W-resnetlOl, shuffleNetV2, SqueezeNet, EfficinetN-V2,B7, among others. Each of the feature extraction model 165 can be pretrained to determine a plurality of features 305.
[0104] Prior to applying the at least one feature extraction model 165, the model selector 145 can provide or display each of the feature extraction models 165 to the computing device 110. The model selector 145 can generate a user interface 130 to display each of the feature extraction models 165. The user interface 130 can provide information or configurations for each of the feature extraction models 165. In some instances, the user interface 130 can provide at least one actionable object for a clinician 205 to select the feature extraction model 165 to extract features 305 of the ROI 220A. Using the user interface 130, the clinician 205 can select a subset of the feature extraction models 165 to generate the plurality of features 305. The at least one actionable object can be configured to receive -30- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450inputs from the clinician 205. From interactions with the user interface 130, the model selector 145 can receive or identify the selection of the feature extraction model 165. In some embodiments, the model selector 145 may receive the selection of the feature extraction model 165 in addition with an indication of a task. The model selector 145 may train or finetune the selected feature extraction model 165 using the training data for the task. For instance, the model selector 145 may identify training data for the task of cancer subtyping, and may train the selected feature extraction model 165 in accordance with supervised or unsupervised learning.
[0105] For example, the selected feature extraction model 165 can be configured to operate in accordance with the selected deep learning model. The selected feature extraction model 165 can be configured to operate in accordance with the selected deep learning model. The selected deep learning model can be used as a foundation, backbone, source architecture, or feature-generating network to define the feature extraction model 165. The model selector 145 and the feature extractor 150 can map, align, translate, or otherwise associate the output of the selected deep learning model with the selected feature extraction model 165 using one or more conversion operations, adaptation operations, resizing operations, normalization operations, projection operations, or interface layers. For example, the system 100 can transform an output deep features representation generated into a format compatible with a selected feature extraction model 165. In some implementations, the combination of different architectures can be performed using shared latent representations or one or more intermediate data structures.
[0106] In some implementations, the feature extraction model 165 can be selected from a same deep learning model selected by the model selector 145. For example, a fully connected layer, penultimate layer, final pooling layer, convolution block output, or classifier input of VGG-16 can be selected for feature extraction when VGG-16 is the selected deep learning model. The combination of the feature extraction model 165 and the deep learning model can be predefined, user selected, automatically generated, recommended by the model selector 145, or otherwise determined based on one or more configurations, processing constraints, image characteristics, the ROI 220A, the score 225, or one or more target clinical endpoints. In this manner, the system 100 can support combinations in which the selected deep learning model and the selected feature extraction model 165 originate from a same-31- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450architecture, as well as heterogeneous combinations in which the selected deep learning model and the selected feature extraction layer originate from different architectures.
[0107] The selected deep learning model and the selected feature extraction model 165 can be derived from the same architecture or from different architectures to determine the plurality of features 305 for the ROI 220 A. For example, the model selector 145 can select VGG-16 as the selected deep learning model. The model selector 145 can select an FC layer of VGG-16 as the feature extraction model 165 to generate the features 305. In another example, the model selector 145 can select ResNet-18 as the selected deep learning model. The model selector 145 can select a fully connected layer of ResNet-18 as the feature extraction model 165. In another example, the model selector 145 can select EfficientNet-B7 as the selected deep learning model and can select a feature extraction layer associated with DenseNet-121 or MobileNet_V2 as the feature extraction model 165. Accordingly, the feature extractor 150 can generate the features 305 using combinations of deep learning models and feature extraction layers that are matched or unmatched with respect to architecture.
[0108] In some instances, the model selector 145 can automatically select or identify the at least one feature extraction model 165 based on an indication from the feature extractor 150. The indication can include a configuration for a recommended feature extraction model 165. The feature extractor 150 can generate, determine, or otherwise indicate the indication based on the one or more factors associated with the ROI 220, the computing device 110, or the score 225. For example, the indication can include a configuration for ResNet-18 as the feature extraction model, responsive to detecting reduced computing resources available for the data processing system 120. In another example, the indication can include a configuration for an VGG16 in response to the number of MRI images 215 exceeding a threshold. Based on the indication, the model selector 145 can select, determine, or otherwise identify the feature extraction model 165.
[0109] Each of the plurality of feature extraction models 165 can include a plurality of feature extraction layers. The feature extraction layers can transform or modify the MRI images 215 into features 305. Some layers can be configured to detect, indicate, or otherwise identify a plurality of patterns within the MRI image 215 A or the ROI 220 A. The patterns can indicate edges, lines, corners, among others features of factors associated with the ROI -32- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450220A. Some layers can add, combine, or otherwise include the patterns into a plurality of shapes. Some layers can use the plurality of shapes to detect, identify, or otherwise determine the features 305 of the ROI 220 A.
[0110] With the selection, the feature extractor 150 may apply the selected feature extraction model 165 to the MRI image 215 A. In some embodiments, the feature extractor 150 may apply the feature extraction model 165 to the MRI image 215 A and the ROI 220A. In applying, the feature extractor 150 may process the input (e.g., the MRI image 215A and the ROI 220A). Based on applying, the feature extractor 150 can determine or otherwise generate the plurality of features 305. The features 305 of the ROI 220 A can be or define one or more visual characteristics of the tumor 112. The one or more visual characteristics can include patterns, smoothness, roughness, mean intensity, color histograms, color variance, area, perimeter, compactness, solidity, eccentricity, strength of edges, boundary irregularities, among others. The features 305 can further define morphological features (e.g., size of the tumor 212, a ratio of width to height, bounding box dimensions), spatial features (e.g., coordinates of the tumor 212, spatial distribution of intensity values), biological features (e.g., blood flow, diffusion coefficients), contextual features (e.g., tumor 212 in relation to surrounding tissues, neighboring abnormalities), among other features.
[0111] FIG. 10B is a block diagram of a process 350 of generating a parametric map 355 using one or more metric evaluation models 170A-N (sometimes referred to as metric evaluation models 170). The process 350 can include or correspond to operations performed in the system 100. Under the process 350, the metric evaluator 155 can apply, feed, or otherwise input the MRI image 215B (e.g., diffusion-weighted imaging (DWI) or dynamic contrast enhanced (DCE) MRI) or the ROI 220B to the one or more metric evaluation models 170. In some embodiments, the metric evaluator 155 can apply the MRI image 215B, the ROI 220B and the score 225 to one or more metric evaluation models 170. The metric evaluation models 170 can include one or more models associated with diffusion weighted imaging, such as mono-exponential models, non-gaussian intravoxel incoherent motion models, Patlak model, extended Tofts model, bi-exponential models, stretched exponential models, kurtosis model, anomalous diffusion model, combined diffusion kurtosis and IVIM model, tri -exponential model, among other metric models. The metric evaluation models 170 can be pretrained using parameter estimation or model fitting. For example, the metric-33- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450evaluator 155 can be trained using parameter estimation, Bayesian inference, or regularization to train the metric evaluation models 170.
[0112] Prior to applying the at least one metric evaluation model 170, the model selector 145 can provide or display each of the metric evaluation models 170 to the computing device 110. The model selector 145 can generate a user interface 130 to display each of the metric evaluation models 170. The user interface 130 can provide information or configurations for each of the metric evaluation models 170. In some instances, the user interface 130 can provide at least one actionable object for a clinician 205 to select the metric evaluation model 170 to apply to one or more of the MRI image 215B, the ROI 220B, and the score 225. Using the user interface 130, the clinician 205 can select a subset of the metric evaluation models 170 to generate the plurality of features 305. The at least one actionable object can be configured to receive inputs from the clinician 205. From interactions with the user interface 130, the model selector 145 can receive or identify the selection of the metric evaluation model 170.
[0113] The metric evaluation models 170 can generate, determine, or otherwise identify a plurality of metrics 360A-N (referred to as metrics 360) for the tumor 212 within the ROI 220B using the MRI image 215B. The metrics 360 can correspond or refer to one or more imaging biomarkers associated with the at least one tumor 212. The one or more imaging biomarkers can be indicators within tumor tissue. The imaging biomarkers can be at least one of diagnostic, prognostic, predictive, monitoring, risk, predisposition. Examples of the imaging biomarkers can include functional biomarkers such as apparent diffusion coefficient (ADC), true diffusion coefficient (D), a pseudo diffusion coefficient (D*), a perfusion fraction (f), or a kurtosis coefficient (K), among others.[0H4] The metrics 360 for the MRI images 215 can correspond to or refer to a diffusion-weighted imaging (DWI) sequence. The metrics 360 for the DWI sequence can include at least one of apparent diffusion coefficient, true diffusion coefficient, a pseudo diffusion coefficient, a perfusion fraction, or a kurtosis coefficient. In this manner, the one or more metric evaluation models 170 can correspond to mono-exponential models and non-gaussian intravoxel incoherent motion models. In some instances, the metrics 360 for the images can correspond or refer to dynamic contrast enhanced (DCE) MRI sequence. The metrics 360 for the DCE-MRI sequence can include at least one of volume transfer constant,-34- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450a blood plasma volume fraction (vP), or a volume fraction of extracellular extravascular space (ve). In this manner, the one or more metric evaluation models 170 can correspond to Patlak models and extended Tofts models.
[0115] The metric evaluation models 170 can be represented by one or more equations. For example, a first metric evaluation model 170A can be represented as:Sb>e-bxADCs0or_ _ f-bD* | _ f'^e-bxD+^KtbxD')2SQIn these equations, Sb and So are the signals with and without b (s / mm2) values, ADC (mm2 / s), D (mm2 / s), and D*(mm2 / s) are the apparent, true, and pseudo-diffusion coefficients (mm2 / s), respectively, f is the perfusion fraction, and K is the kurtosis coefficient (unitless). These quantitative imaging biomarkers (QIBs) are surrogates for tumor cellularity, capillary perfusion fraction, and tissue microstructure.
[0116] The one or more metric evaluation models 170 can be represented by a Patlak model or an extended Tofts model as described in Section A. The metric evaluator 155 can apply or input the ROI 220B to the metric evaluation models 170 to generate a parametric map 355. The parametric map 355 can be an image or a matrix representation of the metrics 360. As a matrix representation, the parametric map 355 may identify for each voxel, within the ROI 220B or the tumor 212, a corresponding metric 360. The overall set of voxels may correspond to the ROI 220B or the tumor 212. The voxels can include an associated intensity that represents, for example, signal strength, parameter values, and a size, among others. Each voxel can indicate a value of a respective parameter corresponding to metric 360. The parameter can be a diffusion coefficient, perfusion rate, among other parameters. The parametric map 355 can include one or more colors indication a variance or variation of parameters within the tumor 212. Each voxel within the ROI 220B or the tumor 212 can be assigned a value that is associated with the metric.-35- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450
[0117] For example, the clinician 205 can use the MRI scanner 115 to acquire the MRI images 215 of the bladder 211. The label generator 140 can generate, assign, or define the ROI 220 within the MRI images 215. The metric evaluator 155 can apply the metric evaluation model 170 to the ROI 220B voxel -wise. The metric evaluation model 170 can generate metrics 360 for each voxel based on the parameter Apparent Diffusion Coefficient (ADC). Using the metrics 360, the metric evaluator 155 can generate a parametric map 355 for using the values of the parameter across the voxels. The metric evaluator 155 can provide, transmit, or send the metrics 360 and the parametric map 355 to the classification generator 160.
[0118] FIG. 11 is a block diagram of a process 400 of generating a classification 405 of a tumor 212 within the bladder 211 using one or more classification models 175. The process 400 can include or correspond to operations performed in the system 100. Upon receipt of the features 305, the metrics 360 and the parametric map 355, the classification generator 160 can determine, generate or otherwise establish a significance metric as a function of the features 305 and the metrics 360. The classification generator 160 can execute one or more or models to select targeted features (e.g., a first portion of the features 305 and a second portion of the metrics 360) based on the significance metric to fine tune the classification models 175. The targeted features can include Minimum Redundancy Maximum Relevance (MRMR), Chi-squared (Chi-2), Relief, ANOVA (Analysis of Variance), and Kruskal Wallis algorithms, among others. For example, the classification generator 160 can select MRMR based on a portion of the features 305 and a portion of the metrics 360 or the parametric map 355 to reduce a dimensionality of the MRI images 215. By reducing the dimensionality, the systems and methods described herein, can improve efficiency and execution of the classification models 175 by focusing on the targeted features.
[0119] Prior to applying the at least one classification model 175, the model selector 145 can select, identify, or otherwise determine at least one classification model 175 to generate the classification 405. The model selector 145 can provide or display each of the classification models 175 to the computing device 110. The model selector 145 can generate a user interface 130 to display each of the classification models 175. The user interface 130 can provide information or configurations for each of the classification models 175. In some instances, the user interface 130 can provide at least one actionable object for a clinician 205-36- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450to select the classification model 175. The model selector 145 can select the classification model 175 based on an input received at the user interface 130. For example, a clinician 205 can interact with the user interface 130. The user interface 130 can receive the input indicating the use of SVM for the classification model 175A. Using the user interface 130, the clinician 205 can select a subset of the classification models 175 to generate the plurality of features 305. The at least one actionable object can be configured to receive inputs from the clinician 205. From interactions with the user interface 130, the model selector 145 can receive or identify the selection of the classification model 175. In some embodiments, the model selector 145 may receive the selection of the classification model 175 in addition with an indication of a task. The model selector 145 may train or fine-tune the selected classification model 175 using the training data for the task. For example, the model selector 145 may identify training data for the task of response prediction and may train the selected classification model 175 using the training data in accordance with supervised or unsupervised learning.
[0120] For example, the classification model 175 can be used in accordance with any combination of the selected deep learning model and the selected feature extraction model 165. In some implementations, the model selector 145 can select a deep learning model and select a feature extraction model 165 associated with the selected deep learning model. The model selector 145 can select a classification model 175 to classify the features 305 generated from the ROI 220A. The combination of the selected deep learning model, the selected feature extraction model 165, and the classification model 175 can be predefined, user selected or automatically generated. For example, the model selector 145 can select VGG-16 as the selected deep learning model, select a fully connected layer of VGG-16, and select an SVM as the classification model 175. In another example, the model selector 145 can select EfficientNet-B7 as the selected deep learning model, can select a feature extraction layer associated with DenseNet-121 as the feature extraction model 165, and can select k-nearest neighbors as the classification model 175. Accordingly, the system 100 can support combinations in which the selected deep learning model, the selected feature extraction model 165, and the classification model 175 are matched or unmatched with respect to model.
[0121] The classification generator 160 can apply, input, or otherwise feed one or more of the features 305, the metrics 360 and the parametric map 355 to one or more-37- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450classification models 175. In some embodiments, the classification generator 160 can apply the features 305 and the metrics 360 to the classification model 175. In some embodiments, the classification generator 160 can apply the features 305 and the parameter map 355 to the classification model 175. In some instances, the classification generator 160 can apply, input, or otherwise feed the targeted features to the one or more classification models 175. The one or more classification models 175 can be any classification models, such as support vector machines (SVM), decision tree, k-nearest neighbors, among other classification models 175. Based on the application of the classification model 175, the classification generator 160 can determine or generate one or more classifications 405.
[0122] Based on the application of one or more of the features 305, the metrics 360, or the parametric map 355 to the one or more classification models 175, the classification generator 160 can create, determine, or otherwise generate the classification 405 for the at least one tumor associated with the bladder cancer in the subject 210. The classification 405 can be a binary classification or a multiclass classification. The classification 405 can be based on the respective classification model 175. For example, if a SVM is used, the classification 405 can be generated based on a decision function. In another example, if a decision tree is used, the classification 405 can be a leaf node with a specific label. In another example, if k-nearest neighbor used, the classification 405 can be based on a class of the nearest neighbors to a respective node.
[0123] In some embodiments, the classification generator 160 can generate, determine, or otherwise identify the classification 405 to indicate or identify a presence or an absence of the bladder cancer based on applying the classification model 175. The classification 405 can include an intermediate value to indicate the presence of bladder cancer or the absence of bladder cancer. The intermediate value can be based on the features 305 or the metrics 360. For example, the intermediate value can increase based on the detection of one or more visual characteristics associated with the features 305 that indicate the presence of bladder cancer. The intermediate value can increase based on the detection of one or more biomarkers that indicate the presence of bladder cancer. The intermediate value can increase and satisfy a threshold, thereby the classification 405 can indicate that the subject 210 is suffering or succumb to bladder cancer. Furthermore, the satisfaction of the threshold can indicate that the tumor 212 within the bladder 211 is associated with bladder cancer. If the-38- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450intermediate value satisfies (e.g., greater than or equal to), the classification generator 160 can generate the classification 405 to indicate the presence of bladder cancer. On the other hand, if the intermediate value does not satisfy (e.g., less than) the threshold, the classification generator 160 can generate the classification 405 to indicate the absence of bladder cancer.
[0124] In some embodiments, the classification generator 160 can generate, determine, or otherwise identify the classification 405 to indicate or identify a type for the bladder cancer based on applying the classification model 175. The type for the bladder cancer can be of a plurality of types of bladder cancer. The types of bladder cancer can include, for example, urothelial carcinoma, squamous cell carcinoma, adenocarcinoma, small cell carcinoma, or other rare subtypes such as sarcomas. For instance, the classification generator 160 can apply the features 305, the parametric map 355, and the metrics 360 to at least one classification model 175 A. The classification model 175 A can use the biomarkers associated with the metrics 360 to identify the type of bladder cancer. In some examples, the classification model 175A can use the visual characteristics defined in the features 305 to identify the type of bladder cancer.
[0125] In some embodiments, the classification generator 160 can determine or generate the classification 405 to indicate or identify a severity level for the bladder cancer based on applying the classification model 175. The severity levels can include stage 0 (in situ), stage 1 (localized), stage 2 (larger localized tumor), stage 3 (locally advanced), and stage 4 (metastatic). As the stages increase, the severity can increase. For instance, the classification generator 160 can apply the features 305, the parametric map 355, and the metrics 360 to at least one classification model 175A. The classification model 175A can use the biomarkers associated with the metrics 360 indicating a stage 0 severity level for the bladder cancer. Based on applying the classification model 175, the classification generator 160 can calculate, generate, or otherwise determine a value indicating the severity for the bladder cancer in the subject 210. With the determination, the classification generator 160 can compare the value with a range of values for the severity levels. In accordance with the ranges, the classification generator 160 can assign or identify the severity level for the bladder cancer in the subject 210.
[0126] In some embodiments, the classification generator 160 can determine or generate the classification 405 to indicate or identify the subject 210 as a candidate or non- -39- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450candidate for therapy for bladder cancer based on applying the classification model 175. The therapy can include at least one of neoadjuvant chemotherapy, surgical resection, adjuvant chemotherapy, or radiotherapy, among others. The subject 210 is administered with the therapy if the classification 405 identifies the subject as a candidate or a predicted responder to the therapy. The intermediary value of the classification 405 based on the features 305 and the metrics 360 can satisfy the threshold to indicate that the subject 210 qualifies as a predicted responder to the therapy. Based on applying the classification model 175, the classification generator 160 can calculate, generate, or otherwise determine a value indicating a likelihood of response to the therapy for the subject 210. The classification generator 160 can compare the value with a threshold defining whether the subject 210 qualifies for additional therapy or not. If the value satisfies (e.g., is greater than or equal to) the threshold, the classification generator 160 can generate the classification 405 to identify the subject 210 as the candidate who is predicted responder to the therapy. In some embodiments, the classification generator 160 can generate the classification 405 to identify the subject 210 as the predicted responder to the therapy. On the other hand, if the value does not satisfy (e.g., is less than) the threshold, the classification generator 160 can generate the classification 405 to identify the subject 210 as the non-candidate for the therapy. In some embodiments, the classification generator 160 can generate the classification 405 to identify the subject 210 as the non-predicted responder to the therapy.
[0127] In some embodiments, the classification generator 160 can create, determine, or otherwise generate the classification 405 to indicate or identify the subject 210 as a predicted responder or a predicted non-responder to the therapy for bladder cancer based on applying the classification model 175. The therapy can be or include at least one of neoadjuvant chemotherapy, surgical resection, adjuvant chemotherapy, or radiotherapy, among others. The subject 210 can be administered with the therapy if the classification 405 indicates the subject 210 as a predicted responder. Based on applying the classification model 175, the classification generator 160 can calculate, generate, or otherwise determine a value indicating a probability of response to therapy for the subject 210. The classification generator 160 can compare the value with a threshold defining whether the subject 210 is to be categorized as a predicted responder (or predicted non-responder). If the value satisfies (e.g., is greater than or equal to) the threshold, the classification generator 160 can generate the classification 405 to identify the subject 210 as the predicted responder. On the other -40- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450hand, if the value does not satisfy (e.g., is less than) the threshold, the classification generator 160 can generate the classification 405 to identify the subject 210 as the predicted nonresponder.
[0128] In some embodiments, the classification generator 160 can determine or generate the classification 405 to identify or indicate at least one survival metric of the subject 210. The survival metric may include, for example, at least one of an overall survival (OS) metric, a progression free survival (PFS) metric, or a time to progression (TTP) metric. The OS metric may indicate a length of time from a reference time (e.g., acquisition of MRI images 215) to a time of death of the subject 210. The PFS metric may indicate a length of time from the reference time to a time of disease progression or death. The TTP metric may indicate a length of time from the reference time to an objective measure of disease progression. In some embodiments, the classification 405 may indicate a risk stratification category (e.g., low, intermediate, and high categories) for the subject 210 based on the survival metric. For example, based on applying the classification model 175, the classification generator 160 can calculate, generate, or otherwise determine an intermediary value for the survival metric in the subject 210. With the determination, the classification generator 160 can compare the value with ranges of values for the survival -based risk stratification categories. In accordance with the ranges, the classification generator 160 can assign or identify the risk stratification category for the subject 210.
[0129] In some embodiments, the classification generator 160 can determine or generate the classification 405 to identify or indicate a longitudinal measure of the therapy associated with an acquisition time for the MRI image 215. The longitudinal measure may include, for example, an effect of the administration of the therapy to the subject 210 for the bladder cancer. The generation of the classification 405 may be based on applying the classification model 175 to one or more of the features 305, the metrics 360, or the parametric map 355 derived from the MRI images 215 at a first acquisition time and a classification from a second acquisition time. The first acquisition time may be subsequent to the administration of the therapy and the second acquisition time may be prior to the administration of the therapy. The classification from the second acquisition time may include, for example, one or more of the presence or absence of the bladder cancer, severity level for the bladder cancer, and survival metric, among others. In some embodiments, the generation of the classification-41- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450405 may be based on applying the classification model 175 to one or more of the features 305, the metrics 360, or the parametric map 355 derived from the MRI images 215 at the first acquisition time and one or more second features, second metrics, or second parameter metric derived from the MRI images at the second acquisition time.
[0130] From applying, the classification generator 160 can process the input using the classification model 175 to determine an intermediary value indicating a degree of effect of the therapy. In accordance with the intermediary value, the classification generator 160 may generate the classification 405. In some embodiments, the classification 405 may indicate a stratification category (e.g., low, intermediate, and high effect categories) for the subject 210 based on the intermediary value indicating the degree of effect. With the determination, the classification generator 160 can compare the intermediary value with ranges of values for the stratification categories. In accordance with the ranges, the classification generator 160 can assign or identify the effect stratification category for the subject 210.
[0131] The output handler 180 can determine, generate, or otherwise indicate a score for the classification 405. The score can be a metric, a flag, an indication, a marker, a grade, or a value indicating a performance or reliability of the classification 405. The score can correspond or refer to statistical evaluation (e.g., reliability or confidence) for the performance of the classification models 175. The one or more metrics associated with the score can include Receiver Operating Characteristic (ROC) curve, accuracy, sensitivity, specificity, precision, and an Fl score. The output handler 180 can determine the reliability of the classification 405 as a function of the classification 405, the features 305, and the metrics 360. In some instances, the output handler 180 can determine the reliability of the classification 405 as a function of the classification 405, the features 305, the metrics 360, the parametric maps 355, and the score 225. The score can be provided to the computing device 110 for review by the clinician 205. The score can indicate reliability when compared against a threshold. For example, if the score exceeds the threshold, the score can indicate that the classification 405 is reliable. The output handler 180 can provide the classification to the computing device 110 and store as an association between the subject 210, the classification 405 and the score within the database 125. However, if the score does not exceed the-42- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450threshold, the output handler 180 can trigger the classification generator 160 to generate a subsequent classification 405.
[0132] The output handler 180 can store, house, or otherwise maintain an association between the subject 210 and the classification 405 within the database 125. The association can be a link, a map, or a connection between the subject 210 and the classification 405. To store the association, the output handler 180 can generate one or more data structures. The one or more data structures can include an array, a linked list, a stack, a tree, a hash table, among others. For example, the data structure can be a hash table where the subject 210 is the key to the hash table and the classification 405 is the value of the hash table. The hash table can include a plurality of keys (i.e., for each subject 210) mapped to a plurality of values (i.e., classification 405 of the subject 210). In another example, the data structure can be a plurality of linked lists. A first linked list can correspond to a first subject 210. Each node in the first linked list corresponding to the classification 405 of the first subject 210. Concurrently, a second linked list can correspond to a second subject 210. Each node in the second linked list corresponding to the classification 405 of the second subject 210. In some implementations, the output handler 180 can store an association between the ROI 220 and the classification 405.
[0133] The output handler 180 can generate, determine, or otherwise provide at least one output 410 using the classification 405 to the computing device 110. The output 410 can include, for example, the classification 405 for presentation to the clinician 205 or the subject 210 through the computing device 110. The output 410 can be, for example, a notification for a clinician to examine the subject 210 based on the presence or absence of bladder cancer, a severity of the bladder cancer, a type of the bladder cancer, a notification to a therapy for the subject 210, or a notification for the subject 210 to request for medical attention. Accordingly, the output handler 180 can provide the output 410 as a notification to the computing device 110 of the clinician 205. In some embodiments, the output 410 can be a notification that includes information associated with the score 225, the classification 405, or at least one of the plurality of MRI images 215. The information can be associated with the features 305 and the metrics (e.g., imaging biomarkers).
[0134] Upon receipt of the output 410, the computing device 110 may present, render, or otherwise display the output 410 for interpretation by the clinician 205. Once the -43- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450output 410 is presented, the computing device 110 can obtain, retrieve, or otherwise access the MRI images 215 associated with the subject 210 and the association from within the database 125. The computing device 110 can extract the ROI 220 from the masks on the MRI images 215 and update each of the MRI images 215 associated with the subject 210 to include the ROI 220. The computing device 110 may display information based on the output 410 received from the data processing system 120. The information may be used to facilitate clinical decision on the part of the clinician 205 examining the subject 210. When the classification 405 in the output 410 indicates the subject 210 as a candidate who is a predicted responder to the therapy, the subject 210 may be considered for no further treatment (e.g., by the clinician 205). For example, enfortumab vedotin plus pembrolizumab (EVP) has produced high pathologic complete response (pCR) rates and improved event-free survival (EFS), making response-adapted strategies, including bladder preservation for selected patients with muscle invasive bladder cancer, plausible. In another example, the subject 210 with muscle-invasive bladder cancer may be administered with a neoadjuvant therapy followed by radical cystectomy. When the classification 405 in the output 410 indicates that the subject 210 underwent EVP therapy and was classified as a complete responder prior to radical cystectomy (i.e., complete surgical removal of the urinary bladder), the clinician may determine not to proceed with surgery for subject 210. Conversely, when the classification 405 in the output 410 indicates the subject 210 is a predicted non-responder to the therapy, the subject 210 may undergo additional aggressive therapy (e.g., by the clinician 205).
[0135] In this manner, the data processing system 120 described herein can improve on the identification of tumors and risk associated with tumors within the bladder while reducing wasted computing resources and reducing improper classifications for the risk associated with the bladder cancer. The data processing system 120 may leverage pretrained models for immediate inference, eliminates retraining overhead. The dual architecture may allow for parallel processing of MRI images to extract features 305 and derive metrics 360. By extracting the features 305 and determining metrics 360 (e.g., imaging biomarkers), the data processing system 120 can accurately classify the presence of bladder cancer, a type of the bladder cancer, a severity of the bladder cancer, and a severity level of the bladder cancer. Furthermore, the data processing system 120 may integrate a wide set of heterogenous models (e.g., feature extraction models 165, metric evaluation models 170, and classification models -44- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450175) that can be selected. From a clinical perspective, the data processing system 120 may enhance diagnostic accuracy and extracting information that would otherwise be unavailable through other approaches, such as with the quantitative imaging biomarkers (QIBs) and parameter maps. The classifications 405 generated using the classification models 175 from these features and QIBs may be used to facilitate diagnosis, risk stratification, individualized treatment decisions.
[0136] FIG. 12 is a flow diagram of a method 500 of classifying bladder risk using magnetic resonance imaging (MRI) data. The method 500 can be implemented or performed by any components detailed herein, such as system 100 or system 600. Under the method 500, a computing system can receive a plurality of MRI images from an MRI scanner (505). The computing system can identify a first ROI and a second ROI (510). The computing system can determine a plurality of features based on the first ROI (515). The computing system can determine a plurality of metrics based on the second ROI (520). The computing system can generate a classification of the first subject based on the plurality of features and the plurality of metrics (525). The computing system can provide an output based on the classification (530).C. Network and Computing Environment
[0137] Various operations described herein can be implemented on computer systems. FIG. 13 shows a simplified block diagram of a representative server system 600, computing system 614, and network 626 usable to implement certain embodiments of the present disclosure. In various embodiments, server system 600 or similar systems can implement services or servers described herein or portions thereof. Computing system 614 or similar systems can implement clients described herein. The system 100 described herein can be similar to the server system 600. Server system 600 can have a modular design that incorporates a number of modules 602 (e.g., blades in a blade server embodiment); while two modules 602 are shown, any number can be provided. Each module 602 can include processing unit(s) 604 and local storage 606.
[0138] Processing unit(s) 604 can include a single processor, which can have one or more cores, or multiple processors. In some embodiments, processing unit(s) 604 can include a general-purpose primary processor as well as one or more special-purpose co-processors -45- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450such as graphics processors, digital signal processors, or the like. In some embodiments, some, or all processing units 604 can be implemented using customized circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. In other embodiments, processing unit(s) 604 can execute instructions stored in local storage 606. Any type of processors in any combination can be included in processing unit(s) 604.
[0139] Local storage 606 can include volatile storage media (e.g., DRAM, SRAM, SDRAM, or the like) and / or non-volatile storage media (e.g., magnetic, or optical disk, flash memory, or the like). Storage media incorporated in local storage 606 can be fixed, removable, or upgradeable as desired. Local storage 606 can be physically or logically divided into various subunits such as a system memory, a read-only memory (ROM), and a permanent storage device. The system memory can be a read-and-write memory device or a volatile read-and-write memory, such as dynamic random-access memory. The system memory can store some or all of the instructions and data that processing unit(s) 604 need at runtime. The ROM can store static data and instructions that are needed by processing unit(s) 604. The permanent storage device can be a non-volatile read-and-write memory device that can store instructions and data even when module 602 is powered down. The term “storage medium” as used herein includes any medium in which data can be stored indefinitely (subject to overwriting, electrical disturbance, power loss, or the like) and does not include carrier waves and transitory electronic signals propagating wirelessly or over wired connections.
[0140] In some embodiments, local storage 606 can store one or more software programs to be executed by processing unit(s) 604, such as an operating system and / or programs implementing various server functions such as functions of the system 100 or any other system described herein, or any other server(s) associated with system 100 or any other system described herein.
[0141] Software” refers generally to sequences of instructions that, when executed by processing unit(s) 604, cause server system 600 (or portions thereof) to perform various operations, thus defining one or more specific machine embodiments that execute and perform the operations of the software programs. The instructions can be stored as firmware residing in read-only memory and / or program code stored in non-volatile storage media that -46- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450can be read into volatile working memory for execution by processing unit(s) 604. Software can be implemented as a single program or a collection of separate programs or program modules that interact as desired. From local storage 606 (or non-local storage described below), processing unit(s) 604 can retrieve program instructions to execute and data to process in order to execute various operations described above.
[0142] In some server systems 600, multiple modules 602 can be interconnected via a bus or other interconnect 608, forming a local area network that supports communication between modules 602 and other components of server system 600. Interconnect 608 can be implemented using various technologies, including server racks, hubs, routers, etc.
[0143] A wide area network (WAN) interface 610 can provide data communication capability between the local area network (e.g., through the interconnect 608) and the network 626, such as the Internet. Other technologies can be used to communicatively couple the server system 600 with the network 626, including wired (e.g., Ethernet, IEEE 602.3 standards) and / or wireless technologies (e.g., Wi-Fi, IEEE 602.11 standards).
[0144] In some embodiments, local storage 606 is intended to provide working memory for processing unit(s) 604, providing fast access to programs and / or data to be processed while reducing traffic on interconnect 608. Storage for larger quantities of data can be provided on the local area network by one or more mass storage 612 that can be connected to interconnect 608. Mass storage 612 can be based on magnetic, optical, semiconductor, or other data storage media. Direct attached storage, storage area networks, network-attached storage, and the like can be used. Any data stores or other collections of data described herein as being produced, consumed, or maintained by a service or server can be stored in mass storage 612. In some embodiments, additional data storage resources may be accessible via WAN interface 610 (potentially with increased latency).
[0145] Server system 600 can operate in response to requests received via WAN interface 610. For example, one of modules 602 can implement a supervisory function and assign discrete tasks to other modules 602 in response to received requests. Work allocation techniques can be used. As requests are processed, results can be returned to the requester via WAN interface 610. Such operation can generally be automated. Further, in some embodiments, WAN interface 610 can connect multiple server systems 600 to each other,-47- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450providing scalable systems capable of managing high volumes of activity. Other techniques for managing server systems and server farms (collections of server systems that cooperate) can be used, including dynamic resource allocation and reallocation.
[0146] Server system 600 can interact with various user-owned or user-operated devices via a wide-area network such as the Internet. An example of a user-operated device is computing system 614. Computing system 614 can be implemented, for example, as a consumer device such as a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smartwatch, eyeglasses), desktop computer, laptop computer, and so on.
[0147] For example, computing system 614 can communicate via WAN interface 610. Computing system 614 can include computer components such as processing unit(s) 616, storage device 618, network interface 620, user input 622, and user output 624. Computing system 614 can be a computing device implemented in a variety of form factors, such as a desktop computer, laptop computer, tablet computer, smartphone, other mobile computing device, wearable computing device, or the like.
[0148] Processing unit 616 and storage device 618 can be similar to processing unit(s) 604 and local storage 606 described above. Suitable devices can be selected based on the demands to be placed on computing system 614. For example, computing system 614 can be implemented as a “thin” client with limited processing capability or as a high-powered computing device. Computing system 614 can be provisioned with program code executable by processing unit(s) 616 to enable various interactions with server system 600.
[0149] Network interface 620 can provide a connection to the network 626, such as a wide area network (e.g., the Internet) to which WAN interface 610 of server system 600 is also connected. In various embodiments, network interface 620 can include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various RF data communication standards such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, LTE, etc.).
[0150] User input device 622 can include any device (or devices) via which a user can provide signals to computing system 614; computing system 614 can interpret the signals as indicative of particular user requests or information. In various embodiments, user input -48- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450device 622 can include any or all of a keyboard, touch pad, touch screen, mouse, or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, and so on.
[0151] User output device 624 can include any device via which computing system 614 can provide information to a user. For example, user output device 624 can include display-to-display images generated by or delivered to computing system 614. The display can incorporate various image generation technologies, e.g., a liquid crystal display (LCD), light-emitting diode (LED) display including organic light-emitting diodes (OLED), projection system, cathode ray tube (CRT), or the like, together with supporting electronics (e.g., digital-to-analog or analog-to-digital converters, signal processors, or the like). Some embodiments can include a device such as a touchscreen that functions as both input and output device. In some embodiments, other user output devices 624 can be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile “display” devices, printers, and so on.
[0152] Some embodiments include electronic components, such as microprocessors, storage, and memory that store computer program instructions in a computer readable storage medium. Many of the features described in this specification can be implemented as processes that are specified as a set of program instructions encoded on a computer readable storage medium. When one or more processing units execute these program instructions, they cause the processing unit(s) to perform various operations indicated in the program instructions. Examples of program instructions or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter. Through suitable programming, processing unit(s) 604 and 616 can provide various functionality for server system 600 and computing system 614, including any of the functionality described herein as being performed by a server or client, or other functionality.
[0153] It will be appreciated that server system 600 and client computing system 614 are illustrative and that variations and modifications are possible. Computer systems used in connection with embodiments of the present disclosure can have other capabilities not specifically described here. Further, while server system 600 and client computing system 614 are described with reference to particular blocks, it is to be understood that these blocks -49- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450are defined for convenience of description and are not intended to imply a particular physical arrangement of component parts. For instance, different blocks can be, but need not be, located in the same facility, in the same server rack, or on the same motherboard. Further, the blocks need not correspond to physically distinct components. Blocks can be configured to perform various operations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how the initial configuration is obtained. Embodiments of the present disclosure can be realized in a variety of apparatus including electronic devices implemented using any combination of circuitry and software.
[0154] While the disclosure has been described with respect to specific embodiments, one skilled in the art will recognize that numerous modifications are possible. Embodiments of the disclosure can be realized using a variety of computer systems and communication technologies, including but not limited to specific examples described herein. Embodiments of the present disclosure can be realized using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein can be implemented on the same processor or different processors in any combination. Where components are described as being configured to perform certain operations, such configuration can be accomplished, e.g., by designing electronic circuits to perform the operation, by programming programmable electronic circuits (such as microprocessors) to perform the operation, or any combination thereof. Further, while the embodiments described above may make reference to specific hardware and software components, those skilled in the art will appreciate that different combinations of hardware and / or software components may also be used and that particular operations described as being implemented in hardware might also be implemented in software or vice versa.
[0155] Computer programs incorporating various features of the present disclosure may be encoded and stored on various computer readable storage media; suitable media include magnetic disk or tape, optical storage media such as compact disk (CD) or digital versatile disk (DVD), flash memory, and other non-transitory media. Computer readable media encoded with the program code may be packaged with a compatible electronic device,-50- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450or the program code may be provided separately from electronic devices (e.g., via Internet download or as a separately packaged computer-readable storage medium).
[0156] Thus, although the disclosure has been described with respect to specific embodiments, it will be appreciated that the disclosure is intended to cover all modifications and equivalents within the scope of the following claims.-51- 4906-1384-1819.1
Claims
Atty. Dkt. No.: 115872-3450WHAT IS CLAIMED IS1. A method of classifying tumors in bladder using magnetic resonance imaging (MRI) data, comprising:receiving, by one or more processors, for a subject at risk of or diagnosed with bladder cancer, a plurality of MRI images of a bladder having at least one tumor associated with the bladder cancer in the subject, the plurality of MRI images comprising (i) a first MRI image of the bladder acquired in accordance with a first imaging sequence of a plurality imaging sequences and (ii) a second MRI image of the bladder acquired in accordance with a second imaging sequence of the plurality of imaging sequences;identifying, by the one or more processors, (i) a first region of interest (ROI) defining the at least one tumor in the first MRI image and (ii) a second ROI defining the at least one tumor in the second MRI image;determining, by the one or more processors, a plurality of features defining one or more visual characteristics of the at least one tumor based on applying a first model to the first MRI image and the first ROI;determining, by the one or more processors, a plurality of metrics identifying one or more imaging biomarkers associated with the at least one tumor based on applying a second model to the second MRI and the second ROI;generating, by the one or more processors, using the plurality of features and the plurality of metrics, a classification corresponding to the at least one tumor associated with the bladder cancer in the subject; andstoring, by the one or more processors, using one or more data structures, an association between the subject and the classification.
2. The method of claim 1, further comprising:determining, by the one or more processors, for each of the plurality of features and the plurality of metrics, a corresponding significance metric as a function of at least one of the plurality of features or the plurality of metrics; andselecting, by the one or more processors, a first portion of the plurality of features and a second portion of the plurality of metrics based on the significance metric for each of the plurality of features and the plurality of metrics,-52- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450wherein generating the classification further comprises generating the classification using the first portion of the plurality of features and the second portion of the plurality of metrics.
3. The method of any one of claims 1 or 2, further comprising receiving, by the one or more processors, via a user interface, a selection of the first model from a plurality of first models, wherein determining the plurality of features further comprises applying the first model selected from the plurality of first models on the first MRI image.
4. The method of any one of claims 1-3, further comprising receiving, by the one or more processors, via the user interface, a selection of a third model from a plurality of third models, wherein generating the classification further comprises applying the third model selected from the plurality of third models to the plurality of features and the plurality of metric.
5. The method of any one of claims 1-4, wherein determining the plurality of metrics further comprises generating a parametric map identifying the plurality of metrics for a plurality of voxels within the second ROI in the second MRI image.
6. The method of any one of claims 1-5, wherein the plurality of metrics for the second imaging sequence corresponding to a diffusion-weighted imaging (DWI) sequence comprises at least one of an apparent diffusion coefficient, true diffusion coefficient, a pseudo diffusion coefficient, a perfusion fraction, or a kurtosis coefficient.
7. The method of any one of claims 1-6, wherein the plurality of metrics for the second imaging sequence corresponding to a dynamic contrast enhanced (DCE) MRI sequence comprises at least one of volume transfer constant (Ktrans), a blood plasma volume fraction (vP), or a volume fraction of extravascular extracellular space (ve).
8. The method of any one of claims 1-7, wherein identifying the first ROI and the second ROI further comprises at least one of-53- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450receiving, via a user interface, one or more inputs corresponding to the first ROI in the first MRI image and the second ROI in the second MRI image; orapplying one or more image segmentation models to the first MRI image to identify the first ROI and the second MRI image to identify the second ROI.
9. The method of any one of claims 1-8, further comprising receiving, by the one or more processors, a score indicating a severity of the bladder cancer in the subject; and wherein determining the plurality of features further comprises determining the plurality of features based on applying the first model to the score,wherein determining the plurality of metrics further comprises determining the plurality of metrics based on the applying the second model to the score.
10. The method of any one of claims 1-9, wherein generating the classification further comprises generating the classification identifying one of presence or absence of the bladder cancer in the subject based on applying a third model to at least one of the plurality of features or the plurality of metrics.
11. The method of any one of claims 1-10, wherein generating the classification further comprises generating the classification identifying a type of a plurality of types for the bladder cancer, based on applying a third model to at least one of the plurality of features or the plurality of metrics,wherein the plurality of types of the bladder cancer comprises at least one of urothelial carcinoma, squamous cell carcinoma, adenocarcinoma, or small cell carcinoma.
12. The method of any one of claims 1-11, wherein generating the classification further comprises generating the classification indicating a survival metric of the subject based on applying third model to at least one of the plurality of features or the plurality of metrics, wherein the survival metric comprises at least one of an overall survival (OS) metric, a progression free survival (PFS) metric, or a time to progression (TTP) metric.
13. The method of any one of claims 1-12, wherein generating the classification further comprises generating the classification identifying the subject as one of a plurality of severity-54- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450levels for the bladder cancer, based on applying a third model to at least one of the plurality of features or the plurality of metrics.
14. The method of any one of claims 1-13, wherein generating the classification further comprises generating the classification identifying the subject as one of a candidate or noncandidate for a therapy for the bladder cancer.
15. The method of any one of claims 1-14, wherein generating the classification further comprises generating the classification identifying the subject as one of a predicted responder or a predicted non-responder to the therapy for the bladder cancer based on applying a third model to at least one of the plurality of features or the plurality of metrics.
16. The method of any one of claims 14 or 15, wherein the therapy for the bladder cancer comprises at least one of neoadjuvant chemotherapy, surgical resection, adjuvant chemotherapy, or radiotherapy, and wherein the subject is administered with the therapy when the classification identifies the subject as the candidate or the predicted responder.
17. The method of any one of claims 1-16, wherein receiving the plurality of MRI images further comprises receiving the plurality of MRI images of the bladder having the at least one tumor acquired at a first time prior to or subsequent to administration of a therapy for bladder cancer.wherein generating the classification further comprises generating the classification indicating an effect of the therapy associated with the first time, based on (i) applying a third model to at least one of the plurality of features or the plurality of metrics and (ii) a second classification at a second time.
18. The method of any one of claims 1-17, further comprising determining, by the one or more processors, a score indicating a reliability of the classification as a function of the classification, the plurality of features, and the plurality of metrics, andwherein storing the association further comprises storing the association of the subject with the classification and the score.-55- 4906-1384-1819.1Atty. Dkt. No.: 115872-345019. The method of any one of claims 1-18, further comprising providing, by the one or more processors, for presentation via a user interface, an output including information based on one or more of: the score, the classification, or at least one of the plurality of MRI images.
20. The method of any one of claims 1-19, wherein the plurality of imaging sequences for the plurality of MRI images further comprises at least one of T1 -weighted, T2-weighted, diffusion-weighted imaging (DWI), and dynamic contrast enhanced (DCE) MRI.
21. A system of classifying tumors in bladders using magnetic resonance imaging (MRI) data, comprising:one or more processors coupled with memory, the one or more processors configured to:receive for a subject at risk of or diagnosed with bladder cancer, a plurality of MRI images of a bladder having at least one tumor associated with the bladder cancer in the subject, the plurality of MRI images comprising (i) a first MRI image of the bladder acquired in accordance with a first imaging sequence of a plurality imaging sequences and (ii) a second MRI image of the bladder acquired in accordance with a second imaging sequence of the plurality of imaging sequences;identify (i) a first region of interest (ROI) defining the at least one tumor in the first MRI image and (ii) a second ROI defining the at least one tumor in the second MRI image;determine a plurality of features defining one or more visual characteristics of the at least one tumor based on applying a first model to the first MRI image and the first ROI; determine a plurality of metrics identifying one or more imaging biomarkers associated with the at least one tumor based on applying a second model to the second MRI and the second ROI;generate, using the plurality of features and the plurality of metrics, a classification corresponding to the at least one tumor associated with the bladder cancer in the subject; and store, using one or more data structures, an association between the subject and the classification.
22. The system of claim 21, the one or more processors configured to:-56- 4906-1384-1819.1Atty. Dkt. No.: 115872-3450determine for each of the plurality of features and the plurality of metrics, a corresponding significance metric as a function of at least one of the plurality of features or the plurality of metrics;select a first portion of the plurality of features and a second portion of the plurality of metrics based on the significance metric for each of the plurality of features and the plurality of metrics; andgenerate the classification using the first portion of the plurality of features and the second portion of the plurality of metrics.
23. The system of any one of claims 21 or 22, wherein the one or more processors are further configured to:receive, via a user interface, a selection of the first model from a plurality of first models;apply the first model selected from the plurality of first models on the first MRI image.
24. The system of any one of claims 21-23, wherein the one or more processors are further configured to:receive, via the user interface, a selection of a third model from a plurality of third models; andapply the third model selected from the plurality of third models to the plurality of features and the plurality of metric.
25. The system of any one of claims 21-24, wherein the one or more processors are further configured to generate a parametric map identifying the plurality of metrics for a plurality of voxels within the second ROI in the second MRI image.
26. The system of any one of claims 21-25, wherein the plurality of metrics for the second imaging sequence corresponding to a diffusion-weighted imaging (DWI) sequence comprises at least one of an apparent diffusion coefficient (ADC), true diffusion coefficient (D), a pseudo diffusion coefficient (D*), a perfusion fraction (f), or a kurtosis coefficient (K).-57- 4906-1384-1819.1Atty. Dkt. No.: 115872-345027. The system of any one of claims 21-26, wherein the plurality of metrics for the second imaging sequence corresponding to a dynamic contrast enhanced (DCE) MRI sequence comprises at least one of volume transfer constant (Ktrans), a blood plasma volume fraction (vP), or a volume fraction of extravascular extracellular space (ve).
28. The system of any one of claims 21-27, wherein the one or more processors are further configured to:receive, via a user interface, one or more inputs corresponding to the first ROI in the first MRI image and the second ROI in the second MRI image; orapply one or more image segmentation models to the first MRI image to identify the first ROI and the second MRI image to identify the second ROI.
29. The system of any one of claims 21-28, wherein the one or more processors configured to:receive a score indicating a severity of the bladder cancer in the subject; determine the plurality of features based on applying the first model to the score; and determine the plurality of metrics based on the applying the second model to the score.
30. The system of any one of claims 21-29, wherein the one or more processors configured to generate the classification identifying one of presence or absence of the bladder cancer in the subject based on applying a third model to at least one of the plurality of features or the plurality of metrics.
31. The system of any one of claims 21-30, wherein the one or more processors configured to generate the classification identifying a type of a plurality of types for the bladder cancer, based on applying a third model to at least one of the plurality of features or the plurality of metrics,wherein the plurality of types of the bladder cancer comprises at least one of urothelial carcinoma, squamous cell carcinoma, adenocarcinoma, or small cell carcinoma.-58- 4906-1384-1819.1Atty. Dkt. No.: 115872-345032. The system of any one of claims 21-31, wherein the one or more processors are further configured to generate the classification indicating a survival metric of the subject based on applying third model to at least one of the plurality of features or the plurality of metrics, wherein the survival metric comprises at least one of an overall survival (OS) metric, a progression free survival (PFS) metric, or a time to progression (TTP) metric.
33. The system of any one of claims 21-32, the one or more processors configured to generate the classification identifying the subject as one of a plurality of severity levels for the bladder cancer, based on applying a third model to at least one of the plurality of features or the plurality of metrics.
34. The system of any one of claims 21-33, wherein the one or more processors configured to generate the classification identifying the subject as a candidate or non-candidate for administration of a therapy for the bladder cancer.
35. The system of any one of claims 21-34, wherein the one or more processors configured to generate the classification identifying the subject as one of a predicted responder or a predicted non-responder to the therapy for the bladder cancer based on applying a third model to at least one of the plurality of features or the plurality of metrics.
36. The system of any one of claims 34 or 35, wherein the therapy for the bladder cancer comprises at least one of neoadjuvant chemotherapy, surgical resection, adjuvant chemotherapy, or radiotherapy, and wherein the subject is administered with the therapy when the classification identifies the subject as the candidate.
37. The system of any one of claims 21-36, wherein the one or more processors are further configured to:receive the plurality of MRI images of the bladder having the at least one tumor acquired at a first time prior to or subsequent to administration of a therapy for bladder cancer.generate the classification indicating an effect of the therapy associated with the first time, based on (i) applying a third model to at least one of the plurality of features or the plurality of metrics and (ii) a second classification at a second time.-59- 4906-1384-1819.1Atty. Dkt. No.: 115872-345038. The system of any one of claims 21-37, wherein the one or more processors configured to:determine a score indicating a reliability of the classification as a function of the classification, the plurality of features, and the plurality of metrics, andstore the association of the subject with the classification and the score.
39. The system of any one of claims 21-38, wherein the one or more processors configured to provide, for presentation via a user interface, an output including information based on one or more of: the score, the classification, or at least one of the plurality of MRI images.
40. The system of any one of claims 21-39, wherein the plurality of imaging sequences for the plurality of MRI images further comprises at least one of T1 -weighted, T2-weighted, diffusion-weighted imaging (DWI), dynamic contrast enhanced (DCE) MRI.-60- 4906-1384-1819.1