Systems and methods for patient outcome prediction in high grade gliomas with neuroimaging and machine learning

Abstract

A computer system for patient outcome prediction and analysis, including at least one processor programmed to: store one or more models trained to analyze multi-modal data, the multi-modal data including (i) resting state functional magnetic resource imaging (RS-fMRI) data, and (ii) patient attribute data; receive a plurality of RS-fMRI images of a patient; receive a plurality of attributes of the patient; generate a multi-modal dataset from the received data; execute the one or more models with the multi-modal dataset as an input to transform the multi-modal dataset into prediction data corresponding to one or more of (i) predicted functional outcomes, (ii) predicted survival outcomes; and (iii) predicted surgical outcomes; and output, from the one or more models, one or more prediction measures for the patient generated based on one or more of the predicted functional outcomes, the predicted survival outcomes, and the predicted surgical outcomes.

Description

CT SYSTEMS AND METHODS FOR PATIENT OUTCOME PREDICTION IN HIGH GRADE GLIOMAS WITH NEUROIMAGING AND MACHINE LEARNING CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to (i) U.S. Provisional Application Ser. No. 63 / 580,531 filed on September 5, 2023, and (ii) U.S. Provisional Application Ser. No.63 / 608,441 filed on December 11, 2023, each of which is incorporated herein by reference in their respective entireties.

[0002] U.S. Patent Application No. 17 / 763,421, filed on March 24, 2022 claims priority to U.S. Provisional Patent Application No. 62 / 905,884, filed on September 25, 2019. Both of these applications are hereby incorporated by reference in their respective entireties. STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH AND DEVELOPMENT

[0003] This invention was made with government support under CA203861 awarded by the National Institutes of Health. The government has certain rights in the invention. FIELD OF THE DISCLOSURE

[0004] The field of the disclosure relates generally to (i) pre-surgery biomarkers, devices and systems to obtain the biomarkers, and methods of using the biomarker to predict functional outcomes of a brain tumor patient before surgery, including using machine-learning algorithms, and (ii) methods and systems for predicting survival in patients suffering from glioblastoma based on multimodal neuroimaging before surgical or medical intervention using machine-learning algorithms. BACKGROUND OF THE DISCLOSURE

[0005] What constitutes brain function is an evolving concept. As techniques to map or image functional sites in the brain advance, the role of a given anatomic location changes in concert with technical capability. Most notable in a neurosurgical settingCT is the concept of “eloquent cortex.” These cortical regions are traditionally responsible for sensorimotor and language processing. Substantial effort is made to avoid eloquent cortex regions during surgical treatments because injury to these structures leads to overt neurological deficits with lasting impairment to the patient's quality and length of life. At present, cortical stimulation remains the gold-standard method of identifying these motor and speech areas during surgery. Often, the lack of stimulation-induced changes to motor movement or communication is interpreted as indicative of a cortical site of no functional consequence. This interpretation, however, neglects subtle cognitive processes such as attentional control, reasoning and planning, which are not easily assessed by standard clinical methods. These cognitive functions, however, are indeed relevant to patient outcomes and their under appreciation in pre-surgical planning may lead to a reduction in patient well- being and overall survival following treatment.

[0006] Glioblastoma (GBM) is a highly infiltrative and deadly form of brain cancer, and is the most common and aggressive malignant glioma, with an overall median survival of less than two years. The current standard of care includes surgical resection of the tumor, which can lead to functional and cognitive deficits. Given the heterogeneous nature of this disease, its course throughout the brain parenchyma is also extremely variable. For instance, there has been evidence of tumor growth resulting in death of surrounding tissue and local displacement of native cells, as well as the preservation of brain tissue within the tumor nidus maintaining functional responses. Limited studies have reported the presence of function within tumor boundaries, and this has been relegated to only brain regions where overt functional changes can be identified intraoperatively, e.g., eloquent cortex. Such regions constitute sensorimotor and language areas with their detection typically probed via direct electrical stimulation. However, direct electrical stimulation measurements typically neglect subtle cognitive processes such as attention, executive function, reasoning, and planning, which are not easily assessed in the operating room. Furthermore, these cognitive processes are relevant to patient outcomes and their underappreciation in pre-surgical planning may lead to a reduction in patient well-being and overall survival following treatment. While narrowly focused on eloquent cortex, past intratumoral function studies broadly investigated this phenomenon across tumor types. Moreover, these studies were limited by their sample size making population inferences about brain tumors in general, and glioblastomas in particular, are extremely challenging.CT Due to the variability in proliferative and infiltrative patterns across tumor grades, it is also difficult to interpret the prevalence of intratumoral function and its effects on patient outcomes following surgical resection.

[0007] In the context of glioblastoma, a central question in their management is how much functional tissue is retained within the tumor. To date, this has been limited both by the definition of eloquence and the mapping methodologies used for functional identification. A limited number of studies have identified preserved functional tissue despite the classical view of tumors as displacing and / or destroying normal brain tissue. For example, a subset of patients with tumors who had stimulation-identified functional tissue located within tumor boundaries were identified, and who experienced deficits when the tumors were resected. Magnetic source imaging has been performed on patients with cerebral lesions (primary tumors or otherwise), and approximately 24.5% possessed functional tissue within or at the lesion margins. Since previous research has been limited to classic eloquent cortex, the extent to which functionally viable tissue can be identified within tumors has not been thoroughly addressed.

[0008] High-grade gliomas (HGG) account for 60-70% of new cases of brain tumors. The efficacy of therapeutic interventions has been limited by various factors, such as genetic heterogeneity, accelerated cellular proliferation, and treatment-resistant cells. Consequently, the median survival rate for patients afflicted with HGGs remains low, with a median survival of merely 14 months. Standard clinical practice typically entails gross total resection of the tumor, followed by radiation and adjuvant chemoradiotherapy. However, surgical resection of the tumor carries the risk of causing or exacerbating functional impairment, a significant concern for the clinician, as postsurgical functional preservation (e.g., quality of life) is known to correlate with overall patient outcomes. Specifically, postoperative functional outcome scores have been shown to significantly outperform preoperative functional impairment scores in the context of survival prediction. To that end, the capability to forecast postsurgical functional outcomes from the initial diagnosis could prove advantageous in surgical planning and for better informing patients of their likely treatment outcomes (e.g., predicting functional outcomes before surgery could lead to improved disease management and informed patient care). Additionally, the ability to predict survival before treatment in GBM patients would lead to improved disease management, clinical trial enrollment, and patient care.CT

[0009] This background section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present disclosure, which are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art. SUMMARY

[0010] In one aspect, a computer system for patient outcome prediction and analysis, the computer system including at least one processor in communication with at least one memory device, the at least one processor programmed to: store one or more models trained to analyze multi-modal data, the multi-modal data including (i) resting state functional magnetic resource imaging (RS-fMRI) data, and (ii) patient attribute data; receive a plurality of RS-fMRI images of a patient; receive a plurality of attributes of the patient; generate a multi-modal dataset from the RS-fMRI images of a patient and the plurality of attributes of the patient; execute the one or more models with the multi-modal dataset as an input to transform the multi-modal dataset into prediction data corresponding to one or more of (i) predicted functional outcomes of the patient, (ii) predicted survival outcomes of the patient; and (iii) predicted surgical outcomes of the patient; and output, from the one or more models, one or more prediction measures for the patient, the one or more prediction measures being generated based on one or more of the predicted functional outcomes of the patient, the predicted survival outcomes of the patient, and the predicted surgical outcomes of the patient.

[0011] In another aspect, a computer-implemented method of predicting a patient outcome including: storing one or more models trained to analyze multi-modal data, the multi-modal data including (i) resting state functional magnetic resource imaging (RS- fMRI) data, and (ii) patient attribute data; receiving a plurality of RS-fMRI images of a patient; receiving a plurality of attributes of the patient; generating a multi-modal dataset from the RS-fMRI images of a patient and the plurality of attributes of the patient; executing the one or more models with the multi-modal dataset as an input to transform the multi- modal dataset into prediction data corresponding to one or more of (i) predicted functional outcomes of the patient, (ii) predicted survival outcomes of the patient; and (iii) predictedCT surgical outcomes of the patient; and outputting, from the one or more models, one or more prediction measures for the patient, the one or more prediction measures being generated based on one or more of the predicted functional outcomes of the patient, the predicted survival outcomes of the patient, and the predicted surgical outcomes of the patient.

[0012] Advantages will become more apparent to those skilled in the art from the following description of the preferred embodiments which have been shown and described by way of illustration. As will be realized, the present embodiments may be capable of other and different embodiments, and their details are capable of modification in various respects. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0014] The figures described below depict various aspects of the systems and methods disclosed therein. It should be understood that each figure depicts an embodiment of a particular aspect of the disclosed systems and methods, and that each of the figures is intended to accord with a possible embodiment thereof. Further, wherever possible, the following description refers to the reference numerals included in the following figures, in which features depicted in multiple figures are designated with consistent reference numerals.

[0015] There are shown in the drawings arrangements which are presently discussed, it being understood, however, that the present embodiments are not limited to the precise arrangements and are instrumentalities shown, wherein:

[0016] FIG. 1 illustrates a flowchart of a functional outcome prediction pipeline in accordance with at least one embodiment of the present disclosure.

[0017] FIG. 2A illustrates spatial tumor frequency maps for Karnofsky Performance Status (KPS), for KPS<70 and KPS≥70 according to embodiments as described in the present disclosure.CT

[0018] FIG. 2B illustrates a graph of average degree of tumor overlap per resting state network (RSN) for KPS<70 according to embodiments as described in the present disclosure.

[0019] FIG.2C illustrates a graph for average degree of tumor overlap for RSNs for KPS≥70 according to embodiments as described in the present disclosure.

[0020] FIG. 3A illustrates model results comparing a true class and a predicted class according to embodiments as described in the present disclosure.

[0021] FIG. 3B illustrates a receiver operating characteristic (ROC) curve for the model results according to embodiments as described in the present disclosure.

[0022] FIG.4A illustrates the strongest predictive features identified by the model according to embodiments as described in the present disclosure.

[0023] FIG. 4B illustrates the strongest resting state network connectivity features according to embodiments as described in the present disclosure.

[0024] FIG. 4C illustrates top connectivity patterns (>.75) associated with the strongest resting state network correlation features shown in FIG.4B.

[0025] FIG. 5A illustrates results of mapping the rank of the average network connectivity feature weights (rank of means of columns shown in FIG. 4B) onto functional connectivity (FC) probability maps.

[0026] FIG. 5B illustrates the mapping of the features that were the strongest spatial predictors in reference to the tumor's location in connection with FIGS.4B and 5A, according to embodiments as described in the present disclosure.

[0027] FIG. 6 illustrates a flow diagram of an exemplary computer- implemented process for outcome prediction and analysis, in accordance with at least one embodiment.

[0028] FIG.7 illustrates an exemplary computer system for performing the processes shown in FIGS.1 and 6, in accordance with at least one embodiment of the present disclosure.CT

[0029] FIG. 8 illustrates an exemplary configuration of a user computer device, in accordance with at least one embodiment of the present disclosure.

[0030] FIG. 9 illustrates an exemplary configuration of a server computer device, in accordance with at least one embodiment of the present disclosure.

[0031] FIG. 10 illustrates an exemplary model training data flow, in accordance with at least one embodiment of the present disclosure.

[0032] FIG. 11A illustrates a flowchart of a patient survival outcome prediction pipeline in accordance with at least one embodiment of the present disclosure.

[0033] FIG. 11B illustrates an example artificial neural network (ANN) model trained for classifying patients based on the survival prediction, according to some embodiments of the present disclosure.

[0034] FIG. 12 illustrates an example autoencoder according to some embodiments of the present disclosure.

[0035] FIG.13 illustrates an example survival model training and validation procedure according to embodiments described in the present disclosure.

[0036] FIG.14A illustrates a box plot and histograms for variables or inputs to the ANN model, according to embodiments as described in the present disclosure.

[0037] FIG. 14B illustrates histograms for ratios of variables based on survival category, according to embodiments as described in the present disclosure.

[0038] FIG.15A illustrates model results based on classifying patients into the appropriate survival group according to embodiments as described in the present disclosure.

[0039] FIG.15B illustrates model results based on classifying patients into the appropriate survival group according to embodiments as described in the present disclosure.CT

[0040] FIG.15C illustrates model results based on classifying patients into the appropriate survival group according to embodiments as described in the present disclosure.

[0041] FIG.15D illustrates model results based on classifying patients into the appropriate survival group according to embodiments as described in the present disclosure.

[0042] FIG. 15E illustrates model results based on extent of resection according to embodiments as described in the present disclosure.

[0043] FIG. 15F illustrates model results based on extent of resection according to embodiments as described in the present disclosure.

[0044] FIG. 15G illustrates spatial tumor frequency mapping according to embodiments as described in the present disclosure.

[0045] FIG.16A illustrates results of permutation feature importance based on classifying patients into the appropriate survival group according to embodiments as described in the present disclosure.

[0046] FIG. 16B illustrates mapping of strongest predictive cortical thickness features of survival according to embodiments as described in the present disclosure.

[0047] FIG. 16C illustrates a diagram of the top FC features derived from permutation feature importance of the top encoded FC features according to embodiments as described in the present disclosure.

[0048] FIG.16D illustrates a graph of mean and standard deviation (STD) of per-network feature weights calculated by averaging the within and between network feature weights for each network according to embodiments as described in the present disclosure.

[0049] FIG. 16E illustrates a graph of mean and STD of voxelwise tumor frequency segmented based on RSN segmentations according to embodiments as described in the present disclosure.CT

[0050] FIG.16F illustrates results of mapping per-network average feature weights onto FC probability maps according to embodiments as described in the present disclosure.

[0051] FIG.17 illustrates a block diagram of an example computing device according to some embodiments as described in the present disclosure.

[0052] FIG. 18 illustrates an example schematic diagram of a neural network model according to some embodiments as described in the present disclosure.

[0053] FIG.19 illustrates an example schematic diagram of a neuron in the neural network model shown in FIG.18. DETAILED DESCRIPTION

[0054] The present disclosure relates to, inter alia, (i) pre-surgery biomarkers, devices, and systems to obtain the biomarkers, and methods of using the biomarker to predict functional outcomes of a brain tumor patient before surgery, and (ii) methods and systems for predicting survival in patients suffering from glioblastoma based on multimodal neuroimaging and clinical data before surgical or medical intervention, each using machine-learning algorithms. I. Predicting Functional Outcomes of a Brain Tumor Patient Before Surgery

[0055] In one exemplary embodiment, the process may be performed by a functional outcome prediction and analysis (FOPA) computer device (also referred to as a survival and / or surgical outcome prediction and analysis (SOPA) computer device in some embodiments, wherein the FOPA computer device and the SOPA computer device may be the same devices / devices). In the exemplary embodiment, the FOPA computer device may be in communication with one or more client devices, one or more imaging systems, and one or more analysis models. As described below in further detail, the FOPA computer system includes one or more functional outcome prediction analysis models that analyze the images, patient attributes, and patient history of a patient with a tumor, such as a High-grade glioma (HGG), and use machine learning to determine postoperative functional outcomes for that patient.CT

[0056] The systems and methods described herein address a critical commercial problem in the domain of neuro-oncology. The problem is the lack of available predictive tools for patient outcomes following treatment for high-grade glioma (HGG), which leads to: A) Un-optimized Treatment Protocols: Without precise predictive capabilities, treatments may be generalized, potentially leading to suboptimal individual patient outcomes. B) Resource Misallocation: Medical resources, including time and finances, can be misused on treatments that may not be the most effective for individual patients. C) Increased Medical Costs: Inaccurate estimates can lead to extended hospital stays, more frequent post-treatment interventions, and additional diagnostic tests, escalating healthcare costs. D) Patient Quality of Life: Patients may undergo treatments without clarity on how it would affect their quality-of-life post-treatment. E) Clinical Trials Limitations: Without refined patient selection criteria based on precise predictions, clinical trials might not target the most suitable candidates, affecting the trial outcomes.

[0057] The disclosed systems and methods offer a solution to these challenges by providing a machine learning model that can accurately predict functional outcomes in High-grade gliomas (HGG) patients before undergoing any treatment, enabling tailored treatments that balance life expectancy and quality of life. Doing so facilitates better patient care, optimizes resource allocation, and can lead to cost savings in the healthcare industry.

[0058] The present disclosure describes the ability of machine learning (ML) to accurately classify postoperative functional outcomes in HGG patients prior to surgery. The systems and methods described herein use basic demographics and functional neuroimaging. The disclosure describes that both resting state functional connectivity and the tumor's location in relation to specific resting state networks can serve as reliableCT predictors of functional outcomes, leading to personalized therapeutic approaches tailored to individual patients.

[0059] Structural (e.g., T1-weighted pre- and post-contrast, T2-weighted, and DTI) and functional (e.g., blood oxygen level-dependent task or resting state fMRI) MRI (magnetic resource imaging) data are routinely acquired in the pre-surgical setting. The former assesses tumor size and location, while the latter helps evaluate a planned resection's functional and cognitive implications. Using the various MRI modalities as a guide, the surgeon faces the challenge of achieving maximal resection of infiltrative tissue while minimizing functional morbidity. These competing interests, long-term survival through maximum resection versus quality of life through functional preservation, significantly impact overall patient outcomes, and finding the optimal balance between the two is an ongoing challenge.

[0060] Task fMRI (T-fMRI) has traditionally been the conventional non- invasive approach for mapping functional brain networks before surgical intervention. However, T-fMRI has limitations that can constrain clinical utility. For example, the need for patients to engage in specific cognitive or motor tasks during the scan can pose challenges and may even be impractical for certain individuals. Alternatively, resting state fMRI (RS- fMRI) allows researchers to examine the intrinsic functional connectivity of the brain and the associated interactions between different networks without the confounding effects of task performance. Importantly, resting state functional mapping can be achieved without task-dependent patient participation, under sedation, and multiple functional networks can be mapped simultaneously, leading to a much lower failure rate than T-fMRI.

[0061] Machine learning (ML) is a field of artificial intelligence that extracts patterns from data to build models with wide-ranging applications in medical research, including brain tumors. Deep learning models have advanced image segmentation, enabling accurate classification of brain tissue categories (e.g., healthy, edema, necrotic / non- enhancing core, enhancing core) and resting state network mapping. MRI radiomic features have also proven valuable for predicting genetic features, identifying both pseudo and true tumor progression, and classifying transcriptome subtypes. Given this information, the systems and methods described herein use ML to extract and leverage pertinent features fromCT RS-fMRI data to preoperatively predict HGG patients' post-operative functional outcomes before surgical resection accurately.

[0062] The systems and methods described herein utilize ML and RS-fMRI to develop models capable of predicting functional outcomes in HGG patients. In one embodiment, the system trains random forest models to classify HGG patients into positive functional outcomes or negative functional outcomes. Input to the models includes, but is not limited to, age, measures of resting state functional network connectivity (FC), the tumor’s degree of overlap with each resting state network (RSN), and tumor volume. Autoencoders were used for dimensionality reduction, and permutation feature importance identified the strongest predictors of functional outcomes. With this information, the trained ML system can accurately predict postoperative functional outcomes at the time of initial diagnosis. Multi-modal neuroimaging data may be utilized and may include, without limitation, cortical thickness data, demographic information of patients, and data of genetic features of patient.

[0063] In the exemplary embodiment, the FOPA computer device trains the one or more models using historical data. In at least one embodiment, the models are trained with historical data for patients diagnosed with HGG on pathological examination of biopsy and resection-acquired brain samples. In these cases, a definitive diagnosis is achieved based on histomorphological and immunohistochemical characteristics supportive of HGG using the appropriate WHO guidelines. This data includes, but is not limited to, the finding of the presence of tumor cells with astrocytic-like appearance, microvascular proliferation, palisading necrosis, pleomorphic hyperchromatic nuclei, and frequent mitoses. Inclusion criteria for the data includes a new diagnosis of brain tumor (first occurrence), biopsy or surgical treatment, and the availability of pre-surgical structural and functional MRI.

[0064] Some potential clinical characteristics are summarized below in TABLE 1 and include survival duration (the difference between the first clinical visit and the date of death), the extent of resection (gross total resection (GTR), near total resection (NTR), or subtotal resection (STR)), O6-Methylguanine-DNA methyltransferase (MGMT) methylation status, epidermal growth factor receptor (EGFR) amplification status, telomerase reverse transcriptase (TERT) mutation status, isocitrate dehydrogenase 1 (IDH1) mutation status, and phosphatase and tensin homolog (PTEN) mutation status. History (Hx)15060-1836 (020692 / WO) PCT noted in TABLE 1 include alcohol use disorder, tobacco use, hypertension, hyperlipidemia, chronic kidney disease (CKD), cardiac issues, deep vein thrombosis or pulmonary embolism (DVT / PE), psychiatric disorders, visual deficits, stroke, weakness, and / or migraine / tension / cluster headaches. Presentation symptoms (Pw) noted included weakness, visual changes, hydrocephalus, confusion, headache, memory impairment, seizures, obesity (BMI>30), and diabetes.CTCTTABLE 1

[0065] In at least one embodiment, the neuroimaging for the models may be performed on a 3T scanner or other similar device. The sequences used may include MRPAGE, T2, EPI BOLD, and / or other sequences as desired.

[0066] The FOPA computer device preprocesses the structural data. The T1w and T2w images are visually inspected to ensure brain structures were free of blurring, ringing, striping, ghosting, etc., caused by head motion. For the training data, multiple raters may review the segmentations with an experienced neuroradiologist to ensure data quality. This may be performed as part of a supervised machine learning scheme. This is to ensure the proper categorization of the training data. Preprocessing of fMRI data follows previously described methods, including compensation for slice-dependent time shifts, elimination of systematic odd-even slice intensities, and rigid body correction of head movement. The data is then resampled and registered in 3mm3atlas space using affine transformations to the T1w structural image. Additional processing steps encompass the elimination of linear trends on a voxel-wise basis, application of temporal low-pass filtering to preserve frequencies below 0.1Hz, regression of nuisance waveforms, and spatial smoothing using 6mm full width at half maximum Gaussian blurring in all directions. Frame censoring is implemented using the DVARS measure as previously described.

[0067] The FOPA computer device performs automated tumor segmentation with a pre-trained convolutional neural network (CNN) architecture using post-contrast T1w, T2w, and Fluid Attenuated Inversion Recovery (FLAIR) scans. The algorithm segments tissue into vasogenic edema, necrotic / non-enhancing core, and enhancing core. Tumor segmentations undergo visual inspection for quality assurance, and any segmentations deemed inadequate may be excluded. A whole tumor mask is used for masking during the registration of the structural and functional images. The automated tumor segmentation maps are also used to create maps showing the relevant frequency of a voxel overlapping with the tumor. For each map, regions not contained in the segmentation are assigned a value of zero, regions segmented as edema are assigned a one, and the non-CT enhancing / enhancing core are assigned a value of two. Subsequently, the individual maps are aggregated to generate heat maps illustrating the extent to which the presence of a tumor influences each voxel within the atlas.

[0068] The analysis includes two measures of RS-fMRI, connectivity and spatial overlap of the tumor with each RSN. Connectivity measures can be calculated as follows. Regions of interest (ROI) derived from past studies may be used to generate similarity maps for multiple (e.g., 15) resting state networks (RSN). ROIs are generated by taking each network's top 200 voxels with the highest probabilities. The networks include dorsal somatomotor (SMD), inferior somatomotor (SMI), cinguloopercular (CON), auditory (AUD), default mode (DMN), parietal memory (PMN), visual (VIS), frontoparietal (FPN), salience (SAL), ventral attention (VAN), dorsal attention (DAN), medial temporal (MET), reward (REW), thalamus (THA), and basal ganglia (BGA). Network similarity is calculated by computing the distance correlation between the network-specific ROIs, resulting in a plurality (e.g., 120) within and between similarity measures. Between-network correlations are calculated by comparing all 200 voxels for the two given networks in a single calculation. Within-network connections are calculated by computing the distance correlation between each voxel and the other 199 voxels and averaging all measures.

[0069] The FOPA computer device calculates spatial features as follows. First, tumor segmentation maps are generated for each individual. Then RSN probability maps are acquired. These maps may be generated from >2000 subjects and reflect the probability of each voxel belonging to each of the 15 RSNs described above. Then, the FOPA computer device uses the dot product of the tumor segmentation maps with the RSN probability maps over voxels with which the argmax (maximum probability) of the probability maps corresponds to the network in question (e.g., if calculating the spatial overlap for SMD, only consider voxels that would be classified as SMD based on the argmax of the probabilities for all 15 networks). The FOPA computer device normalizes the measures by the number of voxels in the given calculation to account for networks of different sizes.

[0070] The FOPA computer device then activates machine learning and statistical analysis. The FOPA computer device classifies the Functional status utilizing Random Forest models, an ensemble methodology comprising multiple decision trees. TheCT FOPA computer device trains each of the trees independently on a distinct, randomly selected subset of the data, and the collective output of the trees forms the basis of the final model predictions. Before training, dimensionality reduction may be performed via an autoencoder with a single hidden layer to reduce the FC feature space to 11 components labeled as FC1, FC2…FC11. The model inputs included age, the encoded FC features, the spatial relationship of the tumor with respect to the RSNs, and tumor volume. The model has been trained to classify patients into two groups based on their Karnofsky Performance Status (KPS) score: KPS<70 (indicating a negative functional outcome) and KPS≥70 (indicating a positive functional outcome). The models are trained with 10-fold nested cross- validation (CV) stratified based on functional outcome, age, and tumor volume. Each outer fold reserved approximately ten samples for external validation, and the remaining samples for the corresponding outer fold were used to train the model with 10-fold cross-validation. In at least one embodiment, the autoencoder is trained on 80% of the FC data, with 20% reserved for validation termination.

[0071] The FOPA computer device uses permutation feature importance to identify the strongest predictive features of functional outcomes. Permutation feature importance involves randomly permuting each feature in the original dataset and evaluating the model's performance after each permutation. The FOPA computer device compares the model’s accuracy with permuted features to the accuracy without permuted features which allows the identification of features that significantly affect model performance and are, therefore, considered strong predictors. This iterative process is repeated 100 times for each model corresponding to a specific cross-validation fold. Results are computed using the validation data reserved for the respective fold and aggregated and averaged to obtain a single weight for each feature. This process is used to identify the strongest predictive features of functional outcomes and which functional networks were most associated with each encoded FC feature. Average network feature weights are generated by averaging all within and between-network feature weights for each network. Then, voxel-wise feature maps were generated by taking the dot product of the average feature weights with publicly available FC probability maps and normalized by the network size. The functional outcomes prediction pipeline and methods are summarized in FIG.1.

[0072] FIG. 1 illustrates a flowchart of the functional outcome prediction pipeline process 100 in accordance with at least one embodiment of the present disclosure.CT FIG. 1 illustrates at portion 102 (e.g., the “A: Diagnosis” portion shown in FIG. 1) an anatomic figure 104 that shows a tumor 106 (e.g., as shown in an MRI image). The connectivity between network ROIs is calculated and passed through an autoencoder for dimensionality reduction and is shown in portion 108 (e.g., the “B: RSN ROIs” portion shown in FIG. 1), portion 110 (e.g., the “C: ROI Connectivity” portion shown in FIG. 1), and portion 112 (e.g., the “D: Autoencoder” portion shown in FIG.1). Portion 114 (e.g., the “E: Segmentation” portion shown in FIG. 1) illustrates the tumor (e.g., 106) being segmented. Portion 114, portion 116 (e.g., the “F: RSN Maps” portion shown in FIG. 1), and portion 118 (e.g., the “G: RSN / Tumor overlap” portion shown in FIG.1) illustrate the segmentation being used to calculate the degree of overlap with the tumor and each RSN. Portion 120 (e.g., the “H: Train / Validate Random Forest” portion shown in FIG. 1) and portion 122 (e.g., the “I: KPS Prediction” portion shown in FIG.1) illustrate the connectivity and spatial features being used to train a random forest to classify patients based on postoperative functional outcome status (KPS<70, KPS≥70). The chart in portion 122 includes True Positive Rate incremented from 0, 0.2, 0.4, 0.6, 0.8 to 1 on the y-axis and False Positive Rate incremented from 0, 0.2, 0.4, 0.6, 0.8 to 1 on the x-axis. Portion 124 (e.g., the “J: Connectivity Features” portion shown in FIG. 1) and portion 126 (e.g., the “K: Spatial Features” portion shown in FIG. 1) illustrate feature selection being used to identify the relevant RSN in the context of both connectivity and spatial location of the tumor.

[0073] TABLE 1 describes patient characteristics, encompassing demographics, survival rates, classifications of surgical resections, identified genetic mutations, medical histories, and the symptoms at the time of diagnosis. Age at diagnosis and survival duration showed significant group differences (p<.02) between the positive and negative functional outcome groups, such that older age and lower survival duration were associated with poor functional outcomes. Similarly, a history of elevated incidence rates of hypertension and hyperlipidemia was associated with poorer functional outcomes (p<.01). Regarding presentation symptoms, higher rates of seizures were observed with poor functional outcomes (p=.02). The incident rate of presenting with headaches was approximately two-fold greater in the positive functional outcome group (p=.06).

[0074] FIG. 2A illustrates spatial tumor frequency maps 200 for KPS<70 (maps 202) and KPS≥70 (maps 204). FIG.2B illustrates a graph 206 of average degree ofCT tumor overlap per RSNs for KPS<70. FIG.2C illustrates a graph 208 for average degree of tumor overlap for RSNs for KPS≥70.

[0075] More specifically, FIG. 2A shows the regions exhibiting the most significant tumor presence, determined by averaging the tumor segmentation maps for all data and partitioned based on functional outcome group. FIG.2B shows the average degree of tumor overlap per RSN calculated by taking the dot product of the tumor frequency maps with published RSN probability maps and normalized by the size of each RSN for KPS<70. The most significant overlap was in AUD, BGA, and SMI networks. Similarly, FIG. 2C shows the overlap with KPS≥70. Here, BGA was much more involved and AUD slightly less involved compared to KPS<70.

[0076] FIGS. 3A and 3B illustrate the model results. FIG.3A, at chart 300, illustrates that the final cross-validation accuracy in classifying KPS was 94.1% (true negative rate (TNR)=95.2%, true positive rate (TPR)=93.8%). FIG. 3B, at graph 302, illustrates a receiver operating characteristic (ROC) curve with an AUC of .97. In FIGS.3A and 3B, the model results illustrate that the model achieved a nest cross-validation accuracy of 94.1%, with a minimum class specific accuracy of 93.8%.

[0077] FIG.4A illustrates the strongest predictive features identified by the model in graph 400. While tumor volume is only a moderate predictor, age was among the top predictors. In the context of encoded network connectivity features, FC1, FC2, and FC11 are the strongest predictors. Based on location, tumors in dorsal attention, cingulo-opercular, and basal ganglia are strong predictors of KPS.

[0078] FIG. 4B illustrates, at diagram 402, the strongest resting state network connectivity features (rank) derived from permutation feature importance from FC1, FC2, and FC11 and grouped based on network functionality (sensory / motor, subcortical, and association). Overall, connectivity involving sensory and motor networks is the strongest predictor of KPS. Additionally, permutation feature importance on the top encoded FC features (FC1, FC2, and FC11) shows primary involvement with networks associated with sensory and motor processes (SMD, SMI, VIS, and AUD).

[0079] FIG. 4C, at diagram 404, illustrates the top connectivity patterns (>.75) associated with the strongest resting state network correlation features shown in FIG.CT 4B. Some association networks (DMN, PMN, and DAN) show moderate involvement, but these were primarily inter-RSN connectivity with sensory and motor networks. Connectivity involving VAN, THA, and BGA are also not strong predictors. Based on the tumor's location in reference to a given RSN, DAN, CON, and BGA are strong predictors of KPS.

[0080] FIG. 5A, at map 500, illustrates the results of mapping the rank of the average network connectivity feature weights (rank of means of columns in FIG. 4B) onto the published FC probability maps. The primary networks involved in the context of connectivity were motor, vision, auditory, and reward. FIG.5B, at map 502, illustrates the mapping of the features that were the strongest spatial predictors in reference to the tumor's location (e.g., rank of spatial features in FIG.4A). These regions are associated with DAN, CON, BGA, and SAL networks.

[0081] FIG. 6 illustrates a flow diagram of an exemplary computer- implemented process 600 for functional outcome prediction and analysis, in accordance with at least one embodiment. In the exemplary embodiment, process 600 is performed by a functional outcome prediction and analysis (FOPA) computer device 702 (shown in FIG.7). In the exemplary embodiment, the FOPA computer device 702 is in communication with at least one client device 704 and at least one imaging device 706 (both shown in FIG.7).

[0082] In the exemplary embodiment, the FOPA computer device 702 stores 602 one or more models trained to analyze resting state functional magnetic resource imaging (RS-fMRI). In the exemplary embodiment, the FOPA computer device 702 receives 604 a plurality of RS-fMRI images of a patient. In the exemplary embodiment, the FOPA computer device 702 receives 606 a plurality of attributes of the patient. In the exemplary embodiment, the FOPA computer device 702 executes 608 the one or more models with the plurality of RS-fMRI images of the patient and the plurality of other data of attributes of the patient as inputs. The one or more models output(s) 610 prediction measures associated with (i) a functional outcome prediction, which may include a functional state probability for the patient, (ii) a survival outcome prediction, and / or (iii) a surgical outcome prediction of the patient. The prediction measure(s) may include information relating to functionality, survival, and / or surgery as described herein and be usable by a surgeon or other professional (e.g., human reviewer) to diagnose and / or generate / update a treatment plan for a patient. For example, prediction measures may be numerical data, comparisons with historical data, aCT text-based summary of the (e.g., numerical) predication data as generated by an LLM model, or any other human readable format to aid the professional.

[0083] FIG.7 illustrates an exemplary computer system 700 for performing the processes 100 and 600 (shown in FIGS. 1 and 6). In the exemplary embodiment, the system 700 is used for functional outcome prediction and analysis.

[0084] As described below in more detail, a functional outcome prediction and analysis (FOPA) computer system 702 may be programmed for functional outcome prediction and analysis. In addition, the FOPA computer system 702 may be programmed to train models to be used to predict patient functional outcomes after surgery. In some embodiments, the FOPA computer system 702 may be programmed to (1) store one or more models trained to analyze resting state functional magnetic resource imaging (RS-fMRI); (2) receive a plurality of RS-fMRI images of a patient; (3) receive a plurality of attributes of the patient; and / or (4) execute the one or more models with the plurality of RS-fMRI images of the patient and the plurality of images of the patient as inputs, wherein the one or more models output a functional state probability for the patient based upon a future surgery. In some embodiments, the FOPA computer device 702 receives updates to a surgical plan for the future surgery based upon the analysis of the functional state probability from the surgeon.

[0085] In the example embodiment, client computer devices 704 (e.g., client devices) are computers that include a web browser or a software application, which enables client devices 704 to communicate with FOPA computer system 702 using the Internet, a local area network (LAN), or a wide area network (WAN). In some embodiments, the client devices 704 are communicatively coupled to the Internet through many interfaces including, but not limited to, at least one of a network, such as the Internet, a LAN, a WAN, or an integrated services digital network (ISDN), a dial-up-connection, a digital subscriber line (DSL), a cellular phone connection, a satellite connection, and a cable modem. Client devices 704 can be any device capable of accessing a network, such as the Internet, including, but not limited to, a desktop computer, a laptop computer, a personal digital assistant (PDA), a cellular phone, a smartphone, a tablet, a phablet, wearable electronics, smart watch, virtual headsets or glasses (e.g., AR (augmented reality), VR (virtual reality), MR (mixed reality),CT or XR (extended reality) headsets or glasses), chat bots, voice bots, LLM bots or LLM-based bots, or other web-based connectable equipment or mobile devices.

[0086] In the example embodiment, the FOPA computer system 702 (also known as FOPA server 702) is a computer that include a web browser or a software application, which enables FOPA computer system 702 to communicate with client devices 704 using the Internet, a local area network (LAN), or a wide area network (WAN). In some embodiments, the FOPA computer system 702 is communicatively coupled to the Internet through many interfaces including, but not limited to, at least one of a network, such as the Internet, a LAN, a WAN, or an integrated services digital network (ISDN), a dial-up- connection, a digital subscriber line (DSL), a cellular phone connection, a satellite connection, and a cable modem. FOPA computer system 702 can be any device capable of accessing a network, such as the Internet, including, but not limited to, a desktop computer, a laptop computer, a personal digital assistant (PDA), a cellular phone, a smartphone, a tablet, a phablet, wearable electronics, smart watch, virtual headsets or glasses (e.g., AR (augmented reality), VR (virtual reality), MR (mixed reality), or XR (extended reality) headsets or glasses), chat bots, voice bots, LLM bots or LLM-based bots, or other web-based connectable equipment or mobile devices.

[0087] A database server 708 is communicatively coupled to a database 710 that stores data. In one embodiment, the database 710 is a database that includes one or more models, patient imagery, and / or patient information. In some embodiments, the database 710 is stored remotely from the FOPA computer system 702. In some embodiments, the database 710 is decentralized. In the example embodiment, a person can access the database 710 via the client devices 704 by logging onto the FOPA computer system 702.

[0088] Imaging device 706 may be any device capable of proving imaging data to the FOPA computer system 702. This may include MRI devices, as well as servers for storing said imaging data. In the example embodiment, imaging device 706 are computers that include a web browser or a software application, which enables imaging devices 706 to communicate with FOPA computer system 702 using the Internet, a local area network (LAN), or a wide area network (WAN).

[0089] In some embodiments, the imaging devices 706 are communicatively coupled to the Internet through many interfaces including, but not limitedCT to, at least one of a network, such as the Internet, a LAN, a WAN, or an integrated services digital network (ISDN), a dial-up-connection, a digital subscriber line (DSL), a cellular phone connection, a satellite connection, and a cable modem. Imaging devices 706 can be any device capable of accessing a network, such as the Internet, including, but not limited to, a desktop computer, a laptop computer, a personal digital assistant (PDA), a cellular phone, a smartphone, a tablet, a phablet, wearable electronics, smart watch, virtual headsets or glasses (e.g., AR (augmented reality), VR (virtual reality), MR (mixed reality), or XR (extended reality) headsets or glasses), chat bots, voice bots, LLM bots or LLM-based bots, or other web-based connectable equipment or mobile devices. Network 712 represents the Internet, a LAN, a WAN, or an integrated services digital network (ISDN), a dial-up- connection, a digital subscriber line (DSL), a cellular phone connection, a satellite connection, and / or a cable modem as described herein.

[0090] FIG. 8 depicts an exemplary configuration 800 of a user computer device 802, in accordance with one embodiment of the present disclosure. In the exemplary embodiment, user computer device 802 may be similar to, or the same as, client device 704 and / or imaging devices 706 (each shown in FIG. 7). User computer device 802 may be operated by a user 804.

[0091] User computer device 802 may include a processor 806 for executing instructions. In some embodiments, executable instructions may be stored in a memory area 808. Processor 806 may include one or more processing units (e.g., in a multi- core configuration). Memory area 808 may be any device allowing information such as executable instructions and / or patient data to be stored and retrieved. Memory area 808 may include one or more computer readable media.

[0092] User computer device 802 may also include at least one media output component 810 for presenting information to user 804. Media output component 810 may be any component capable of conveying information to user 804. In some embodiments, media output component 810 may include an output adapter (not shown) such as a video adapter and / or an audio adapter. An output adapter may be operatively coupled to processor 806 and operatively couplable to an output device such as a display device (e.g., a cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED) display, or “electronic ink” display) or an audio output device (e.g., a speaker or headphones).CT

[0093] In some embodiments, media output component 810 may be configured to present a graphical user interface (e.g., a web browser and / or a client application) to user 804. A graphical user interface may include, for example, an interface for viewing items of information provided by the FOPA computer system 702 (shown in FIG.7). In some embodiments, user computer device 802 may include an input device 812 for receiving input from user 804. User 804 may use input device 812 to, without limitation, provide information either through speech or typing.

[0094] Input device 812 may include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad or a touch screen), a gyroscope, an accelerometer, a position detector, a biometric input device, and / or an audio input device. A single component such as a touch screen may function as both an output device of media output component 810 and input device 812.

[0095] User computer device 802 may also include a communication interface 814, communicatively coupled to a remote device such as FOPA computer system 702. Communication interface 814 may include, for example, a wired or wireless network adapter and / or a wireless data transceiver for use with a mobile telecommunications network.

[0096] Stored in memory area 808 are, for example, computer readable instructions for providing a user interface to user 804 via media output component 810 and, optionally, receiving and processing input from input device 812. A user interface may include, among other possibilities, a web browser and / or a client application. Web browsers enable users, such as user 804, to display and interact with media and other information typically embedded on a web page or a website from FOPA computer system 702. A client application may allow user 804 to interact with, for example, FOPA computer system 702. For example, instructions may be stored by a cloud service, and the output of the execution of the instructions sent to the media output component 810.

[0097] FIG.9 depicts an exemplary configuration 900 of a server computer device 902, in accordance with one embodiment of the present disclosure. In the exemplary embodiment, server computer device 902 may be similar to, or the same as, FOPA computer system 702, database server 708, and imaging devices 706 (all shown in FIG. 7). Server computer device 902 may also include a processor 904 for executing instructions.CT Instructions may be stored in a memory area 906. Processor 904 may include one or more processing units (e.g., in a multi-core configuration).

[0098] Processor 904 may be operatively coupled to a communication interface 908 such that server computer device 902 is capable of communicating with a remote device such as another server computer device 902, FOPA computer system 702, imaging devices 706, and client devices 704 (shown in FIG.7) (for example, using wireless communication or data transmission over one or more radio links or digital communication channels). For example, communication interface 908 may audio input from client devices 704 via the Internet, as illustrated in FIG.7.

[0099] Processor 904 may also be operatively coupled to a storage device 912. Storage device 912 may be any computer-operated hardware suitable for storing and / or retrieving data, such as, but not limited to, data associated with one or more models. In some embodiments, storage device 912 may be integrated in server computer device 902. For example, server computer device 902 may include one or more hard disk drives as storage device 912.

[0100] In other embodiments, storage device 912 may be external to server computer device 902 and may be accessed by a plurality of server computer devices 902. For example, storage device 912 may include a storage area network (SAN), a network attached storage (NAS) system, and / or multiple storage units such as hard disks and / or solid- state disks in a redundant array of inexpensive disks (RAID) configuration.

[0101] In some embodiments, processor 904 may be operatively coupled to storage device 912 via a storage interface 910. Storage interface 910 may be any component capable of providing processor 904 with access to storage device 912. Storage interface 910 may include, for example, an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and / or any component providing processor 904 with access to storage device 912.

[0102] Processor 904 may execute computer-executable instructions for implementing aspects of the disclosure. In some embodiments, the processor 904 may be transformed into a special purpose microprocessor by executing computer-executableCT instructions or by otherwise being programmed. For example, the processor 904 may be programmed with the instruction such as illustrated in FIGS. 1 and 6.

[0103] FIG. 10 is a schematic diagram 1000 illustrating further detail of exemplary server computing device 902 (shown in FIG. 9). Server computing device 902 may communicate with other components shown in FIGS.7 and 8, such as database server 708 (or a third-party server), client systems 704 via a network 1002, and / or user computer device 802, where network 1002 may be the same as or similar to network 712. Server computing device 902 may include and / or be in communication with a database 1004 that stores data 1006 including patient data (database 1004 may be the same as or similar to database 710 and / or storage device 912). Data 1006 received from network 1002 may be stored in database 1004. Server computing device 902 may be configured to use data 1006 to generate a functional outcome prediction model module 1008 for controlling operations of the server computing device 902 (e.g., in accessing third-party databases via a digital portal), predicting functional outcomes as described herein. Data 1006 for use in connection with functional outcome prediction model module 1008 may include data regarding age, measures of resting state functional network connectivity, the tumor’s degree of overlap with each RSN, and tumor volume, and other applicable data as described herein.

[0104] In exemplary embodiments, server computing device 902 includes a training set builder module 1010 configured to submit one or more queries 1012 to database 1004 to retrieve subsets 1014 of data 1006, and to use those subsets 1014 to build training data sets 1016 for generating model module 1008. For example, query 1012 may be configured to retrieve certain fields from data 1006 for historical information of characteristics for a patient, and the like.

[0105] In exemplary embodiments, training set builder module 1010 may be configured to derive training data sets 1016 from retrieved subsets 1014. Each training data set 1016 corresponds to a historical data 1006 (“historical” in this context means completed in the past, as opposed to completed in real-time with respect to the time of retrieval by training set builder module 1010). Each training data set 1016 may include “model input” data fields along with at least one “result” data field representing a historical outcome associated with the model input. The model input data fields represent factors that may be expected to, or unexpectedly be found during model training to, have someCT correlation. Data 1006 may include a variety of patient data and other data, including a multi- modal dataset including a plurality of RS-fMRI data as described herein.

[0106] In exemplary embodiments, the model input data fields in training data sets 1016 may be generated from data fields in subset 1014 corresponding to historical data 1006. In other words, a trained machine learning model 1018 produced by a model trainer module 1020 for use by model module 1008 is trained to make predictions based on input values that can be generated from the data fields in data 1006. Values in the model input data fields may include values copied directly from values in a corresponding data field in the retrieved subset 1014, and / or values generated by modifying, combining, or otherwise operating upon values in one or more data fields in the retrieved subset 1014. The use of such data fields as model input data fields facilitates the machine learning model in weighing these factors directly.

[0107] After training set builder module 1010 generates training data sets 1016, training set builder module 1010 passes the training data sets 1016 to model trainer module 1020. In example embodiments, model trainer module 1020 is configured to apply the model input data fields of each training data set 1016 as inputs to one or more machine learning models. Each of the one or more machine learning models is programmed to produce, for each training data set 1016, at least one output intended to correspond to, or “predict,” a value of the at least one result data field of the training data set 1016. “Machine learning” refers broadly to various algorithms that may be used to train the model to identify and recognize patterns in existing data in order to facilitate making predictions for subsequent new input data.

[0108] Model trainer module 1020 is configured to compare, for each training data set 1016, the at least one output of the model to the at least one result data field of the training data set 1016, and apply a machine learning algorithm to adjust parameters of the model in order to reduce the difference or “error” between the at least one output and the corresponding at least one result data field. In this way, model trainer module 1020 trains the machine learning model to accurately predict the value of the at least one result data field. In other words, model trainer module 1020 cycles the one or more machine learning models through the training data sets 1016, causing adjustments in the model parameters, until the error between the at least one output and the at least one result data field falls below a suitableCT threshold, and then uploads at least one trained machine learning model 1018 to model module 1008 for application to generating recommendations 1022. In exemplary embodiments, model trainer module 1020 may be configured to simultaneously train multiple candidate machine learning models and to select the best performing candidate for each result data field, as measured by the “error” between the at least one output and the corresponding result data field, to upload to model module 1008.

[0109] In certain embodiments, the one or more machine learning models may include one or more neural networks, such as a convolutional neural network, a deep learning neural network, or the like. The neural network may have one or more layers of nodes, and the model parameters adjusted during training may be respective weight values applied to one or more inputs to each node to produce a node output. In other words, the nodes in each layer may receive one or more inputs and apply a weight to each input to generate a node output. The node inputs to the first layer may correspond to the model input data fields, and the node outputs of the final layer may correspond to the at least one output of the model, intended to predict the at least one result data field. One or more intermediate layers of nodes may be connected between the nodes of the first layer and the nodes of the final layer.

[0110] As model trainer module 1020 cycles through the training data sets 1016, model trainer module 1020 applies a suitable backpropagation algorithm to adjust the weights in each node layer to minimize the error between the at least one output and the corresponding result data field. In this fashion, the machine learning model is trained to produce output that reliably predicts the corresponding result data field. Alternatively, the machine learning model may have any suitable structure.

[0111] In some embodiments, model trainer module 1020 provides an advantage by automatically discovering and properly weighting complex, second- or third- order, and / or otherwise nonlinear interconnections between the model input data fields and the at least one output. Absent the machine learning model, such connections are unexpected and / or undiscoverable by human analysts.

[0112] The server computing device 902 of the present disclosure is configured to operate on input data related to functional outcomes and other data / information as described herein. In one exemplary embodiment, the server computing device 902CT executes the model module 1008 programmed to learn, without limitation, outcomes based upon varying information, relevant data sources for evidence, the queries used to prompt a user to provide relevant information, features of functional outcomes.

[0113] To facilitate this learning, the server computing device 902 includes one or more databases 1004 at which the data, including features, outcomes, etc., is stored. This data becomes one or more input training sets used by the training set builder 1010. Model outputs can be formatted for presentation or review as visual representations of recommendations, as text-based or natural language recommendations, and the like. In exemplary embodiments, model module 1008 may compare feedback, and may route a comparison result 1024 generated by comparing recommendation 1022 to the feedback to a model updater module 1026 of the server computing device 902. Model updater module 1026 is configured to derive a correction signal 1028 from comparison results 1024 received for one or more recommendations and to provide correction signal 1028 to model trainer module 1020 to enable updating or “re-training” of the at least one machine learning model to improve performance. The retrained at least one machine learning model 1018 may be periodically re-uploaded to model module 1008. An output 1029 of model module 1008 may be output to an external device such as a user computer 802, or used as part of re-training in connection with trainer set builder module 1010.

[0114] As described in more detail below in connection with the embodiment shown and described in connection with FIGS. 11A-19, server computing device 902 may also be configured to use data 1006 to generate a survival outcome prediction model module 1030 for controlling operations of the server computing device 902 (e.g., in accessing third-party databases via a digital portal), predicting patient survival as described herein. Data 1006 may include various survival data as shown and / or described in connection with FIGS.11A-19.

[0115] In exemplary embodiments, server computing device 902 includes a training set builder module 1032 configured to submit one or more queries 1034 to database 1004 to retrieve subsets 1036 of data 1006, and to use those subsets 1036 to build training data sets 1038 for generating survival outcome prediction model module 1030. For example, query 1034 may be configured to retrieve certain fields from data 1006 for historical information of characteristics for a patient, and the like.CT

[0116] In exemplary embodiments, training set builder module 1032 may be configured to derive training data sets 1038 from retrieved subsets 1036. Each training data set 1038 corresponds to a historical data 1006 (“historical” in this context means completed in the past, as opposed to completed in real-time with respect to the time of retrieval by training set builder module 1032). Each training data set 1038 may include “model input” data fields along with at least one “result” data field representing a historical outcome associated with the model input. The model input data fields represent factors that may be expected to, or unexpectedly be found during model training to, have some correlation.

[0117] In exemplary embodiments, the model input data fields in training data sets 1038 may be generated from data fields in subset 1036 corresponding to historical data 1006. In other words, a trained machine learning model 1040 produced by a model trainer module 1042 for use by model module 1030 is trained to make predictions based on input values that can be generated from the data fields in data 1006. Values in the model input data fields may include values copied directly from values in a corresponding data field in the retrieved subset 1036, and / or values generated by modifying, combining, or otherwise operating upon values in one or more data fields in the retrieved subset 1036. The use of such data fields as model input data fields facilitates the machine learning model in weighing these factors directly.

[0118] After training set builder module 1032 generates training data sets 1038, training set builder module 1032 passes the training data sets 1038 to model trainer module 1042. In example embodiments, model trainer module 1042 is configured to apply the model input data fields of each training data set 1038 as inputs to one or more machine learning models. Each of the one or more machine learning models is programmed to produce, for each training data set 1038, at least one output intended to correspond to, or “predict,” a value of the at least one result data field of the training data set 1038. “Machine learning” refers broadly to various algorithms that may be used to train the model to identify and recognize patterns in existing data in order to facilitate making predictions for subsequent new input data.

[0119] Model trainer module 1042 is configured to compare, for each training data set 1038, the at least one output of the model to the at least one result data fieldCT of the training data set 1038, and apply a machine learning algorithm to adjust parameters of the model in order to reduce the difference or “error” between the at least one output and the corresponding at least one result data field. In this way, model trainer module 1042 trains the machine learning model to accurately predict the value of the at least one result data field. In other words, model trainer module 1042 cycles the one or more machine learning models through the training data sets 1038, causing adjustments in the model parameters, until the error between the at least one output and the at least one result data field falls below a suitable threshold, and then uploads at least one trained machine learning model 1040 to model module 1030 for application to generating recommendations 1044. In exemplary embodiments, model trainer module 1042 may be configured to simultaneously train multiple candidate machine learning models and to select the best performing candidate for each result data field, as measured by the “error” between the at least one output and the corresponding result data field, to upload to model module 1030.

[0120] In certain embodiments, the one or more machine learning models may include one or more neural networks, such as a convolutional neural network, a deep learning neural network, or the like. The neural network may have one or more layers of nodes, and the model parameters adjusted during training may be respective weight values applied to one or more inputs to each node to produce a node output. In other words, the nodes in each layer may receive one or more inputs and apply a weight to each input to generate a node output. The node inputs to the first layer may correspond to the model input data fields, and the node outputs of the final layer may correspond to the at least one output of the model, intended to predict the at least one result data field. One or more intermediate layers of nodes may be connected between the nodes of the first layer and the nodes of the final layer.

[0121] As model trainer module 1042 cycles through the training data sets 1038, model trainer module 1042 applies a suitable backpropagation algorithm to adjust the weights in each node layer to minimize the error between the at least one output and the corresponding result data field. In this fashion, the machine learning model is trained to produce output that reliably predicts the corresponding result data field. Alternatively, the machine learning model may have any suitable structure.CT

[0122] In some embodiments, model trainer module 1042 provides an advantage by automatically discovering and properly weighting complex, second- or third- order, and / or otherwise nonlinear interconnections between the model input data fields and the at least one output. Absent the machine learning model, such connections are unexpected and / or undiscoverable by human analysts.

[0123] The server computing device 902 of the present disclosure is configured to operate on input data related to survival outcomes and other data / information as described herein. In one exemplary embodiment, the server computing device 902 executes the model module 1008 programmed to learn, without limitation, outcomes based upon varying information, relevant data sources for evidence, the queries used to prompt a user to provide relevant information, features of survival outcomes.

[0124] To facilitate this learning, the server computing device 902 includes one or more databases 1004 at which the data, including requests, responses, features, outcomes, etc., is stored. This data becomes one or more input training sets used by the training set builder 1032. Model outputs can be formatted for presentation or review as visual representations of recommendations, as text-based or natural language recommendations, and the like. In exemplary embodiments, model module 1030 may compare feedback, and may route a comparison result 1046 generated by comparing recommendation 1044 to the feedback to a model updater module 1048 of the server computing device 902. Model updater module 1048 is configured to derive a correction signal 1050 from comparison results 1046 received for one or more recommendations and to provide correction signal 1050 to model trainer module 1042 to enable updating or “re- training” of the at least one machine learning model to improve performance. The retrained at least one machine learning model 1040 may be periodically re-uploaded to model module 1030. An output 1052 of model module 1030 may be output to an external device such as a user computer 802, or used as part of re-training in connection with trainer set builder module 1032. While not expressly shown in FIG.10, any data sets or output of the models generated by model modules 1008 and 1030 may be used in conjunction with one another to gain additional and / or deeper understanding between functional outcomes and survival, e.g., for having jointly trained models and / or a creating a composite model.CT

[0125] As described in further detail above, the systems and methods describe a machine learning model engineered to predict functional outcomes for high-grade glioma (HGG) patients prior to surgical interventions. Inputs to this model include, but are not limited to, age, tumor location, tumor volume, and resting state functional MRI (RS- fMRI) measures.

[0126] The systems and methods described herein include many unique features, such as, but not limited to: Multimodal Data Integration: The model processes a combination of patient demographic data, tumor metrics, and RS-fMRI measures; Dimensionality Reduction: The incorporation of autoencoders ensures efficient handling and processing of the input data (High Accuracy: A cross-validation accuracy of 94.1% was achieved in classifying the Karnofsky Performance Status (KPS); and Feature Analysis: Identifies and quantifies the significance of predictive variables).

[0127] The systems and methods described herein provide many advantages over existing technologies, including, but not limited to: Comprehensive Prediction: enables prediction of outcomes beyond just survival metrics; Visual Interpretation: provides insightful visual mappings of results for better understanding; and Holistic Approach: the model enables the assessment of post-treatment quality of life, which can be integrated with survival prediction, facilitating a more balanced treatment approach that weighs both survival and quality of life considerations.

[0128] The systems and methods described herein have multiple potential applications and impact, such as, but not limited to, Clinical Enhancement: The integration of this model into clinical practices could elevate HGG patient management and standard of care; Personalized Medicine: It promotes an individualized treatment approach, aligning treatment plans with predicted outcomes; and Translational Research: The IP stands to further the field of translational research in the domain of HGG patient care.

[0129] The systems and methods provide for the potential of machine learning in accurately classifying postsurgical functional outcomes among HGG patients. These systems provide for feasible classification accuracy exceeding 90% before surgical intervention, using basic demographics, tumor volume, and RS-fMRI measures. Further, these systems are successful despite substantial heterogeneity present in the patient data, encompassing diverse demographics, tumor characteristics, medical histories, and initialCT symptoms (TABLE 1). This underscores the potential of integrating machine learning into clinical settings, particularly for informed decision-making in HGG patient management.

[0130] Historically, neuroimaging predictors of functional outcomes, survival, recurrence, and other clinically relevant determinants have been predominantly structural measures. This preference for structural measures is partly due to the invasive nature of HGGs, which can be more readily visualized through structural imaging techniques. In contrast, task and resting state fMRI generally involve ROI-based analyses, which can have limited clinical utility. However, these systems have leveraged deep learning and other emerging techniques to generate whole-brain RS-fMRI functional maps at both the individual and group levels, with group averages spanning thousands of individuals. These techniques extend well beyond the simple mapping of language and motor functions. They account for many functional networks, offering a more nuanced evaluation of the cognitive, behavioral, and emotional impacts that a planned resection might have. This shift towards a functional approach provides a more comprehensive assessment of brain tumors, moving beyond the limitations of traditional structural anatomical location. Consequently, these systems and methods have opened new avenues for the research and clinical community to evaluate brain network connectivity in the context of brain tumor management.

[0131] From a neurosurgical standpoint, the long-term survival of patients with HGGs is a balancing act between the extent of surgical resection of the lesion and functional preservation. While substantial research exists regarding survival prediction in relation to the extent of resection, investigations focusing on functional outcome predictions remain relatively limited and pose significant challenges. Previous studies have identified factors such as tumor size, cranial nerve manipulation, major brain vessel manipulation, posterior fossa location, and eloquent cortex involvement as significant determinants of postoperative deterioration in KPS. The disclosed systems indicate tumor volume was only a moderate predictor (FIG. 4A) of KPS, which might initially appear counterintuitive. However, this finding underscores the critical point that tumor location in relation to functional networks could have a more significant influence on functional outcomes than the size of the tumor alone.

[0132] The systems and methods described herein demonstrates how machine learning can accurately classify functional outcomes in HGG patients prior toCT surgical, chemical, or radiotherapy treatments. More particularly, these systems rely on age, tumor location, RS-fMRI measures, and tumor size. These systems and methods of incorporating these models into clinical practice, enable personalized treatment plans that balance quality of life with survival. Such models can drive a more nuanced approach to HGG patient management, prioritizing both longevity and post-treatment quality of life.

[0133] The present systems and methods utilize ML and RS-fMRI to develop models capable of predicting functional outcomes in HGG patients. Specifically, random forest models are trained to classify HGG patients into positive functional outcomes (KPS≥70) or negative functional outcomes (KPS<70). Input to the models include age, measures of resting state functional network connectivity (FC), the tumor’s degree of overlap with each RSN, and tumor volume. Autoencoders are used for dimensionality reduction, and permutation feature importance identified the strongest predictors of functional outcomes. These models can accurately predict postoperative functional outcomes at the time of initial diagnosis.

[0134] The present systems and methods: receive a plurality of RS-fMRI images of a patient (see portion 102 in FIG.1, for example); receive a plurality of attributes (see, e.g., TABLE 1) of the patient; generate a multi-modal dataset (e.g., from data 1006, shown in FIG. 10) from the RS-fMRI images of a patient and the plurality of attributes of the patient; transform the multi-model dataset into prediction data (see FIGS. 3A, 3B, 4B, 4C, for example); and execute the one or more models (see FIG. 10) with the multi-modal dataset as an input to transform the prediction data into a functional state probability for the patient based upon a future surgery, wherein the one or more models output the functional state probability as a prediction measure; and can be used to generate or update a surgical plan for the future surgery based upon the functional state probability. II. Predicting Survival Outcome in Patients

[0135] GBM is the most common and deadly primary tumor of the central nervous system, accounting for approximately 70% of newly diagnosed tumor in patients. The standard first-line treatment for newly diagnosed GBM is surgical resection and radiotherapy with concomitant temozolomide, followed by adjuvant temozolomide. Despite advances in stratified treatment, only 43% of patients survive one-year post-diagnosis, and 6.9% survive five years post-diagnosis. Intertumor / intratumor heterogeneity, characterizedCT by distinct genetic alterations, rapid proliferation, aggressive infiltration, multiple activated signal transduction pathways, and the emergence of treatment-resistant cells soon after therapy onset, limits the effectiveness of current therapies, leading to inevitable tumor recurrence and death. Given the dismal survival statistics and the heterogeneity of the disease, a non-invasive method that reliably predicts a patient’s overall survival (OS) would benefit clinicians, patients, and their families, and would lead to improved disease management, clinical trial enrollment, and patient care. This could apply to GBM and other brain tumors such as gliomas, meningiomas, pituitary tumors, medulloblastomas, schwannomas, craniopharyngiomas, hemangioblastomas, chordomas, and brain metastases.

[0136] Preoperative MRI is necessary for determining the best approach to surgical resection. To optimize patient outcomes, the surgeon weighs the extent of resection against its impact on functional preservation. In other words, the optimal balance between the extent of resection, quality of life, and survival of the patient is determined. Structural MRI (e.g., T1-weighted pre- and post-contrast, T2-weighted, and diffusion tensor imaging) is routinely employed to evaluate the location and size of the tumor, as well as morphological changes in brain anatomy distant from the tumor. Structural MRI is crucial to planning the extent of resection. Functional MRI (e.g., blood oxygen level-dependent (BOLD) task or resting state fMRI (RS-fMRI)), is used to map functional networks before resection to maximize functional preservation.

[0137] In some embodiments, machine learning (ML) (or artificial intelligence (AI)) based algorithms or models are generated by extracting patterns from raw data. ML based algorithms has significant applicability in analysis of brain tumors from many different aspects including, but not limited to, image segmentation, and / or inferring clinically relevant information based on the radionic feature of MRI. The image segmentation is used for classifying brain tissue from MRI into multiple categories. Inferring clinically relevant information based on the radiomic features of MRI includes predicting the tumor’s genetic characteristics, differentiation between pseudo and true progression, and classification of transcriptome-based subtypes. Using the rich information embedded within radiomic features in the MRI data, ML algorithms are trained using the MRI data to predict survival in GBM patients.CT

[0138] In some embodiments, ML algorithms and multimodal neuroimaging to develop ML algorithms capable of classifying three groups of overall survival in GBM patients. Multimodal neuroimaging (MN) combines data obtained from a plurality of neuroimaging techniques including, but not limited to, electroencephalography, and / or functional magnetic resonance imaging, and yields more detailed information about brain dynamics. By way of a non-limiting example, ML algorithms may be neural network- based models, such as deep feed-forward artificial neural networks (ANN) models. Deep feed-forward ANN models are trained to classify GBM patients according to their survival post diagnosis (or post-surgical resection or medical intervention). By way of a non-limiting example, the GBM patients may be classified according to their survival, e.g., less than one year, one to two years, or greater than two years (<1y, 1-2y, >2y). In some embodiments, input to the ANN models include, but not limited to, demographics (age and sex), contralesional cortical thickness (CT), and / or resting-state network functional connectivity (FC) correlations derived from a plurality of resting-state networks, which are calculated based on preoperative data.

[0139] An advantage of the ANN model, as described herein, in accordance with some embodiments, is that reliable and accurate survival classification is generated without reference to other clinical variables (e.g., genetics and / or the extent of resection). Additionally, functional connectivity and cortical thickness are leveraged to infer clinically relevant information about a patient’s mortality risk for improving surgical planning, stratified therapy, and proper decision-making for GBM patients.

[0140] In some embodiments, and by way of a non-limiting example, other types of algorithms may be deployed to enhance medical analysis and / or decision making. Such algorithms may include, but are not limited to, (i) machine learning algorithms (for example, Random Forest, Support Vector Machines, Decision Trees, Nearest Neighbor methods, and / or Gradient Boosting) that can be harnessed or used to analyze medical imaging data related to brain tumors allowing for precise or accurate detection and / or classification of tumor types through the examination of radiomic features; (ii) natural language processing (NLP) techniques such as Named Entity Recognition and / or Sentiment Analysis, can be employed to extract information from medical records and clinical notes, providing insights into a patient’s medical history, details about treatment progression, identification of potential risk factors, and / or treatment outcomes; (iii) deep learning modelsCT or deep learning algorithms including, but are not limited to, Autoencoders, Feed Forward Neural Networks, Recurrent Neural Networks (RNNs), and / or Convolutional Neural Networks (CNNs), that can be utilized to process brain imaging data to accurately segment tumor regions, and precise tumor boundary delineation, analyze genomic data, identify genetic mutations associated with the brain tumors, and, thereby, aiding in personalized treatment strategies and / or radiation therapy planning based on the segmentation; and / or (iv) Genomic Analysis Algorithms such as Sequential Algorithms, Mutation Calling Algorithms, and Pathway Analysis Algorithms, that delve into the genetic makeup of brain tumors.

[0141] Furthermore, the machine learning algorithms, according to some embodiments, enable the classification of brain tumor images into distinct categories, thus automating the process of tumor diagnosis and facilitating patient stratification. The predictive capabilities of the machine learning algorithms, according to some embodiments, are based on imaging data and clinical features, and contribute to the creation of personalized treatment plans for patients diagnosed with Glioblastoma Multiforme (GBM). Moreover, NLP techniques can analyze patient narratives, providing deeper insights into emotional states and subjective experiences. This facet enhances the understanding of patients' psychological well-being during their course of treatment. RNNs may be used for analyzing sequences of patient vital signs and treatment responses, enabling real-time monitoring and predictions of potential adverse events. Additionally, deep learning models, such as autoencoders, may be used for extracting latent features from multi-modal data sources, which assists in comprehensive patient profiling through integrative analysis. Genomic Analysis Algorithms play a pivotal role in identifying genetic mutations associated with such tumors, thereby supporting the creation of tailored treatment strategies and targeted therapies aligned with the molecular characteristics of the tumor. Furthermore, these algorithms explore the intricate signaling pathways influenced by genetic mutations. Consequently, they guide the selection of drugs that specifically target these altered pathways.

[0142] In some embodiments, and by way of a non-limiting example, MRI data (e.g., pre-surgical structural and functional MRI data) may be collected from adult patients who are diagnosed with brain tumor (first occurrence), have undergone biopsy or surgical treatment. Additionally, or alternatively, adult patients with certain demographics and / or clinical characteristics may be selected to collect the MRI data used for training the ANN model. Certain demographics may include age and / or gender of the patients. ClinicalCT characteristics may include an overall survival of the patient between the first MRI and the date of death of the patient. Other clinical characteristics may include, but are not limited to, the extent of resection (gross total (GTR), subtotal resection (STR), and near total resection (NTR), tumor location (frontal, parietal, temporal, occipital, cingulate, and / or other), Karnofsky Performance Status (KPS>70), O6-Methylguanine-DNA methyltransferase (MGMT) promoter methylation status, epidermal growth factor receptor (EGFR) amplification status, telomerase reverse transcriptase (TERT) mutation status, isocitrate dehydrogenase 1 (IDH1) mutation status, and phosphatase and tensin homolog (PTEN) mutation status. Multi-modal neuroimaging data may be utilized and may include, without limitation, cortical thickness data, demographic information of patients, and data of genetic features of patient.

[0143] Additionally, or alternatively, the clinical characteristics may include the detailed history of medical comorbidities (Hx) that each patient presented with at the time of GBM diagnosis, including alcohol use disorder, tobacco use, hypertension, hyperlipidemia, chronic kidney disease (CKD), cardiac disease (myocardial infarction, arrhythmia, valvular dysfunction), deep vein thrombosis or pulmonary embolism (DVT / PE), psychiatric disorders, visual deficits, stroke, weakness, obesity (BMI>30), diabetes, and headaches (migraine, tension, or cluster). Furthermore, the clinical characteristics may also include the presenting symptoms (Pw) of each patient, including weakness, visual changes, aphasia, hydrocephalus, confusion, headache, memory impairment, and seizures. The clinical characteristics may also include treatment regimens including patients treated with the Stupp protocol (60 Gy radiotherapy plus concurrent and adjuvant temozolomide) and various clinical trials of radiotherapy, bevacizumab, CCNU, disulfiram, doxorubicin, and temozolomide. Example clinical characteristics are described in a table reproduced below. In the table below, continuous data used the Kruskal-Wallis test, and ratios used chi squared test. Further, Hx stands for “history of,” Pw stands for “presented with,” STD stands for “standard deviation,” and MAD stands for “median absolute deviation.”CTCT15060-1836 (020692 / WO) PCT

[0144] In some embodiments, and by way of a non-limiting example, neuroimaging may be performed on a Siemens Trio or Skyra 3T MRI scanner. Structural MRI images included T1-weighted (T1w) magnetization prepared rapid acquisition gradientCT echo (MPRAGE: TE =2.53 ms, TR =1900 ms, TI =900 ms, 256 ×256 acquisition matrix, 0.976 × 0.976 × 1 mm voxels), fluid-attenuated inversion recovery (FLAIR: 2D, slice thickness 5mm, gap 1mm, 256x256 matrix 0.9x0.9mm pixel size, TE=129 ms, TR=8500 ms, TI = 2440 ms, flip angle 130), and T2-weighted (T2w) fast spin-echo (FSE: TE = 93 ms, TR= 5600 ms, 256 × 256 acquisition matrix, 1.093 ×1.093 × 2 mm voxels). RS-fMRI may be acquired using a BOLD-sensitive EPI sequence (voxel size 3mm3isotropic; echo time = 27 ms; repetition time = 2.2-2.9s; field of view = 256mm; flip angle = 90). Additionally, or alternatively, two RS-fMRI runs may be obtained, for example, for each patient yielding approximately 320 frames. In some embodiments, scanning may be performed by a scanner manufactured by Phillips Healthcare, Toshiba Medical Systems, GE, and / or Hitachi Medical Systems.

[0145] In some embodiments, and by way of a non-limiting example, structural data preprocessing is performed with FreeSurfer (http: / / surfer.nmr.mgh.harvard.edu). Visual inspection of the segmentation results is performed, and FreeSurfer-defined cortical parcels based on the Desikan-Killiany atlas are used to define regional CT measures in the contralesional side of GBM patients. The fMRI data is preprocessed using generally known method for preprocessing for compensation for slice-dependent time shifts, elimination of systematic odd-even slice intensities, and rigid body correction of head movement. The data are then resampled in 3mm atlas space using affine transformations to the T1w structural image. Additionally, or alternatively, preprocessing of fMRI data also includes voxel-wise removal of linear trends, temporal low- pass filtering retaining frequencies below 0.1Hz, regression of nuisance waveforms, and spatial smoothing with 6mm full width at half maximum Gaussian blurring in each direction. In some embodiments, and by way of a non-limiting example, frame censoring is implemented using the DVARS measure.

[0146] In some embodiments, an automated tumor segmentation is performed with a pre-trained convolutional neural network architecture using post-contrast T1w, T2w, and FLAIR scans. The whole tumor mask is used for masking during atlas registration. The tumor segmentation maps are also used to create voxelwise heat maps showing the frequency of a voxel overlapping with the tumor segmentation. By way of a non-limiting example, using ANN model (or an ML algorithm), the fMRI data is segmented for vasogenic edema, necrotic / non-enhancing core, and enhancing core. In some examples,CT tumor segmentations are visually examined and edited using ITK-SNAP to ensure precise delineation. Active contour techniques may be utilized in ITK-SNAP, with reference to T1w, T2w, and FLAIR scans, to generate a semi-automatic tumor segmentation. Furthermore, manual segmentation may be used to approximate areas with interrupted contours. Inadequate segmentations may result in the exclusion of the sample. A whole tumor mask, including all three tissue classes, may be used for masking during the atlas registration of the structural and functional images. Further, the automated tumor segmentation maps may be used to create voxelwise heat maps, which may show the relevant frequency of a voxel overlapping with the tumor segmentation. For each map, regions not contained in the segmentation are assigned a value of zero, while areas segmented as edema are assigned a value of 1, and the non-enhancing / enhancing core is assigned a value of 2. These segmentations are then averaged to generate voxel-wise heat maps of tumor frequency in atlas space.

[0147] In some embodiments, regions of interest (ROI) are used or define to generate similarity maps for a plurality of resting state networks (RSN). ROIs are developed by taking each network’s top 200 probabilities (corresponding to the top 200 voxels). By way of a non-limiting example, the networks include dorsal somatomotor (SMD), inferior somatomotor (SMI), cinguloopercular (CON), auditory (AUD), default mode (DMN), parietal memory (PMN), visual (VIS), frontoparietal (FPN), salience (SAL), ventral attention (VAN), dorsal attention (DAN), medial temporal (MET), reward (REW), thalamus (THA), and basal ganglia (BGA). The similarity between networks may be calculated by computing the distance correlation between the network-specific ROIs, resulting in a plurality of network similarity measures within and between networks. In some examples, a total number of network similarity measurements may be 120. A similarity measure can be generated with a single calculation regardless of the spatial dimension of the two vectors using distance correlation based on the distance matrix for each set of vectors. Thus, between-network correlations may be calculated by comparing all 200 voxels for the two given networks in a single calculation and by averaging the distance correlation between each voxel and the other 199 voxels for within-network connections.

[0148] FIG. 11A illustrates a flowchart of a survival prediction pipeline process 1100 in accordance with at least one embodiment of the present disclosure. FIG. 11A illustrates at portion 1102 (e.g., the “a: Diagnosis” portion shown in FIG. 11A) anCT anatomic figure 1104 that shows a tumor 1106 (e.g., as shown in an MRI image). Portion 1108 (e.g., the “b: Segmentation” portion shown in FIG. 11A) illustrates the tumor (1106) being segmented. Portion 1110 (e.g., the “c: CT” portion shown in FIG.11A) illustrates a CT features. Portion 1112 (e.g., the “d: RSN” portion shown in FIG.11A) and portion 1114 (e.g., the “e: FC Correlations” portion shown in FIG.11A) illustrate functional connectivity. Portion 1116 (e.g., the “f: Autoencoder” portion shown in FIG. 11A) illustrates the autoencoder, such as shown in FIG. 12. The connectivity is calculated and passed through the autoencoder. Portion 1118 (e.g., the “g: Neural Network” portion shown in FIG.11A) illustrates the overall survival prediction model. Portion 1120 (e.g., the “h: Train / Validate” portion shown in FIG. 11A) and portion 1122 (e.g., the “i: Survival Prediction” portion shown in FIG.11A, where survival prediction may also referred to as a survival diagnosis) illustrate the training / validation to classify patient survival and survival predictions. Portion 1124 (e.g., the “j: Feature Identification” portion shown in FIG. 11A) illustrates feature selection being used to identify both connectivity and spatial location. For example, portion 1120 in FIG.11A may utilize the training shown in FIG.10 for survival outcome prediction model module 1030. In the exemplary embodiment, pipeline process 1100 is performed by a survival functional outcome prediction and analysis (SOPA) computer device 702 (shown in FIG.7), where SOPA computer device 702 also performs process 600 as shown in FIG.6 relative to survival processes (e.g., instead of or in addition to functional outcome processes).

[0149] FIG.11B illustrates a NN model 1126. The NN (e.g., ANN) model (e.g., shown at portion 1118 in FIG.11A) is used in the analysis of fMRI data and may be an ANN model 1126 with one or more input layers 1128, one or more hidden layers 1130, and one or more output layers 1132. The ANN model 1126 may include 3 hidden layers 1130 with a certain amount neurons (e.g., eight) in each layer as shown in FIG.11B. Each neuron utilizes a normalized radial basis transfer function. The ANN model 1126 may be trained to classify patients into less than one year, between one and two years, or greater than two years of survival, where output layers 1132 may output in correspondence with such duration groupings (e.g., an output layer 1132 for each of less than one year, between one and two years, and greater than two years of survival).

[0150] Initially, multiple network architectures may be trained and compared, with the smallest network containing a single hidden layer and five artificial neurons and the largest network containing five hidden layers with 20 artificial neurons inCT each layer. However, the network (or the ANN model 1126) with three hidden layers 1130 and eight neurons showed the best performance, and, therefore, may be the preferred ANN model.

[0151] In some embodiments, input to the ANN model 1126 may include age, sex, contralesional CT measures, and / or FC features. Due to many FC features (e.g., 120 FC features) versus the size of data set, dimensionality reduction may be performed using an autoencoder.

[0152] FIG.12 illustrates an example autoencoder 1200 according to some embodiments of the present disclosure, such as shown at portion 1116 in FIG. 11A. Autoencoders are a type of feedforward neural network that attempts to map an input back to itself. The autoencoder 1200 may be a single hidden layer 1202 in between an input layer 1204 and an output layer 1206 and encode the original input into a reduced dimension feature space (based on the number of neurons in the hidden layer), which is then decoded back to an approximation of the original data. By way of a non-limiting example, the autoencoder 1200 may be a single hidden layer with 20 neurons with normalized radial basis transfer functions. In one example, a total of 11 encoded FC features (after removing nine sparse features) may be used for training the ANN model and are hereafter referred to as FC1, FC2, …, FC11.

[0153] FIG. 13 illustrates an example survival model training and validation procedure 1300 according to embodiments described in the present disclosure (e.g., such as shown by portion 1120 in FIG. 11A). The survival model training and validation procedure 1300 may include nested stratified cross-validation and hold out. By way of a non-limiting example, the initial data set may include 133 samples as shown at initial data set procedure 1302 and 10 stratified samples may be held out (in other words, not used for training) for cross-validation procedure 1304 including 123 samples. Then, the data may be partitioned into, for example, 26 folds. By way of a non-limiting example, each fold from fold procedures 1306, 1308 may hold out approximately five stratified samples not used for training to serve as within-fold validation testing. Within each fold, a model may be trained using 70% of the training data of the given fold. The remaining 30% of training data may be used for model validation termination, such that the model stopped training once the performance on the 30% validation data did not improve after ten epochs. This is representedCT at train / validate procedure 1310. Once the models are finished training, the trained models 1312 may be tested on their corresponding within-fold validation data and the original ten held-out samples at test procedures 1314 and 1316. In some embodiments, and by way of a non-limiting example, the autoencoder, such as the autoencoder described herein with reference to FIG. 12, may be trained on 80% of the data, with 20% of data reserved for validation termination.

[0154] In some embodiments, and by way of a non-limiting example, permutation feature importance may be used to identify the strongest predictive features of survival based on the trained models. Permutation feature importance involves evaluating a trained model’s performance on a given data set. Once the model performance is established, a single feature is randomly permuted in the original data set, and the data is tested on the model. After the given feature is permuted, the model performance is noted, and the process is repeated for each successive feature. The influence of a given feature is based on the impact that permuted feature had on the accuracy of the model compared to the accuracy of the model with no permuted features. Thus, features that significantly impact model performance are considered strong predictors.

[0155] In some embodiments, permutation feature importance may be determined for different ANN models trained in the cross-validation folds and evaluated based on the global hold-out data and the data held out for the given fold. Additionally, or alternatively, the process described herein for the permutation feature importance may be repeated multiple times (e.g., 1,000 times) for each model, and all results may be combined and averaged to get a single weight for each feature. In some embodiments, and by way of a non-limiting example, the feature weights may be first identified for the CT and the autoencoded FC features based on the survival prediction models. Once the feature weights are established for the autoencoded FC features, the process may be repeated on the autoencoder. The performance measure may be evaluated based on the impact of permuting a given feature on the autoencoded FC features, which further allows identifying the strongest within and between network FC features without directly using those features in the model.

[0156] In some embodiments, an average per-network feature weight may be generated by averaging all within and between-network feature weights for each givenCT network. For example, the average feature weight for SMD may be calculated by averaging all feature weights associated with SMD (e.g., SMDxSMD, SMDxSMI, SMDxCON, and so on). Voxelwise FC feature maps may be calculated by taking the dot product of the average feature weights with publicly available FC probability maps generated using a deep learning algorithm, and results that are averaged over about 2000 participants. Each voxel in the probability maps may include a softmax probability distribution representing the probability of that voxel belonging to each of the given networks. Accordingly, summing the multiplication of the mean feature weights with the associated voxelwise softmax probabilities yields an overall feature weight for each voxel.

[0157] FIG.14A illustrates box plot 1400 and histograms 1402 for some variables shown in TABLE 2, and ratios of variables based on survival categories as shown in FIG.14B. Histograms 1402 include histogram 1404 relating to <1 year, histogram 1406 relating to 1-2 years, and histogram 1408 relating to >2 years (in correspondence with such durations as shown in box plot 1400). FIG.14B illustrates survival categories 1410 according to the same durations. TABLE 2 described above provides information regarding demographics, median overall survival, tumor location, Karnofsky Performance Scale (KPS), treatments, genetic alterations, medical history, and presentation symptoms at the time of diagnosis in total and based on the three survival groups. As seen from the TABLE 2, The majority of the cohort is male (58%), with a mean age at diagnosis of 60.8 years and median overall survival of 14.1 months. By way of a non-limiting example, of various measures considered, ratios of sex, KPS>70, gross total resection (GTR), full Stupp protocol, bevacizumab, CCNU, doxorubicin treatment, temozolomide treatment, MGMT promoter methylation status, and history of tobacco use are found to be significantly different amongst survival groups as shown in FIG. 14B at 1410. Furthermore, patients presenting with symptoms of visual changes, memory impairment, and seizures showed significant differences among the survival groups (Chi-squared test). As shown in FIGS.14A and 14B, age at diagnosis may also significantly differ between groups (Kruskal-Wallis test).

[0158] FIGS. 15A-15I illustrate model results based on classifying patients into the appropriate survival group according to embodiments as described in the present disclosure. As shown in FIG. 15A at chart 1500, the models achieved 87% cross- validation accuracy, and the total accuracy in testing the models on the ten held-out samples is found to be 92.3% as shown in FIG. 15B at chart 1502. When combining the cross-CT validation and hold-out results, the total accuracy is found to be 90.6% as shown in FIG.15C at chart 1504. When treating all models as an ensemble and averaging the results on the ten held-out samples, the model achieved 100% accuracy as shown in FIG.15D at chart 1506. Kaplan-Meier survival curves, shown in FIG. 15E at graph 1508, based on the model classifications of the cross-validation data (p < 0.01). Further, when patients are further segmented by the extent of resection and completion of Stupp protocol, the model still significantly separated patients’ survival. FIG.15F illustrates a variety of additional Kaplan- Meier survival curves 1510, generated based on model classification, partitioned by extent of resection (gross total (GTR) at graph 1512 or subtotal (STR)) at graph 1514, completion of full Stupp protocol at graph 1516 or partial Stupp protocol at graph 1518, and the intersection of these measures at graphs 1520 (GTR+Full Stupp), 1522 (GTR+Partial Stupp), 1524 (STR+Full Stupp), and 1526 (STR+Partial Stupp). P-values were significant (p<0.1) for all factors, with the exception of GTR and STR combined with a partial Stupp protocol. This means that the model can accurately classify survival in GBM patients prior to surgical, chemical, or radiotherapy treatments. FIG. 15G illustrates spatial tumor frequency. Brain regions most affected by the tumors were calculated by averaging the tumor segmentation maps for all data and partitioned based on the survival group. Figure 15G shows the regions most affected by the tumors, calculated by averaging the tumor segmentation maps 1528.

[0159] FIG. 16A illustrates results 1600 of the permutation feature importance based on classifying patients into the appropriate survival group according to embodiments as described in the present disclosure. Age at diagnosis and sex are both strong predictors of overall survival, as shown in FIG.16A. CT regions encompassing the superior temporal sulcus, parahippocampal gyrus, pericalcarine, pars triangularis, and middle temporal regions are also strong predictors of survival, as shown in FIG.16A. Additionally, or alternatively, the rank of the CT features based on the anatomical segmentation, and / or several encoded FC features are also found to be strong predictors. Permutation feature importance on the encoded features (FC10, FC11, and FC4) is found to be primary involved with SMDxCON, SMDxVIS, DMNxDMN, SMIxPMN, AUDxPMN, SMIxDMN, AUDxAUD, SMIxVIS, SMDxTHA, and VISxVIS. When the feature weights for each network are averaged, SMD, VIS, CON, and SMI are found to be the strongest average predictors of survival. The mean and STD of the voxelwise tumor frequency based on RSNCT segmentations suggest that the networks with the most substantial tumor frequency overlap may include AUD, BGA, CON, and SMI. Additionally, it is also found based on the results of mapping the per-network average feature weights onto the published FC probability maps that the associated areas include the motor, occipital, opercular, and anterior insular regions.

[0160] FIG.16B illustrates strongest predictive cortical thickness features 1602 of survival. The strongest predictors included banks of the superior temporal sulcus, parahippocampal gyrus, pericalcarine, pars triangularis, and middle temporal regions.

[0161] FIG. 16C illustrates the top 10 FC features 1604 derived from permutation feature importance of the top encoded FC features (FC10, FC11, and FC4 (shown in FIG. 16A)). SMDxCON, SMDxVIS, DMNxDMN, SMIxPMN, AUDxPMN, SMIxDMN, AUDxAUD, SMIxVIS, SMDxTHA, and VISxVIS were the strongest features.

[0162] FIGS.16D-16F illustrate additional aspects of per-network feature weights and segmentation. FIG. 16D illustrates a graph 1606 of mean and STD of per- network feature weights calculated by averaging the within and between network feature weights for each network. SMD, VIS, CON, and SMI were the strongest average predictors. FIG. 16E illustrates a graph 1608 of Mean and STD of the voxelwise tumor frequency segmented based on published RSN segmentations (as shown in FIG. 11A). The networks with the strongest tumor frequency overlap included AUD, BGA, CON, and SMI. c) Results of mapping the per-network average feature weights onto the published FC probability maps. The associated regions include the occipital lobe, opercula, and anterior insula. FIG. 16F illustrates results of mapping the per-network average feature weights onto the published FC probability maps. The associated regions in mapping 1610 include the occipital lobe, opercula, and anterior insula.

[0163] Using the ANN models according to embodiments described herein, prediction of overall survival in GBM patients may be made with an accuracy of above 90% at the time of diagnosis and before treatment using basic demographics (age, sex), cortical thickness, and RS-fMRI connectomics. Using the ANN models, as described herein, an association between the GBM and its effects on the brain structure and brain functions globally can be established, for example, cortical thickness and brain-wide BOLD spectra are found to be associated with overall survival and tumor epigenetics. As described herein, CT in the contralesional superior temporal sulcus, parahippocampal gyrus,CT pericalcarine, pars triangularis, and middle temporal regions are strong predictors of overall survival in GBM patients. Thus, the structural alterations GBMs have on the brain are not restricted to the tumor’s location. Overall cortical thickness can be determined as an important imaging metric in classifying overall survival.

[0164] Using the ANN models, as described herein, it may be determined that the global effects of GBM may manifest as a widespread perturbation of resting state networks. The vision network, for example, may be the second-strongest average feature weight, for example, based on the prognostic value of functional connectivity in the motor and visual networks. Further, using the ANN models, as described herein, regions associated with motor function are found to be particularly strong predictors of overall survival based on the FC analysis, with SMD and SMI showing strong feature weights. The CON, thought to be related to “tonic alertness” (the ability to maintain arousal levels), is also found to be a strong predictor of overall survival. Motor regions are significant brain structural features in the ANN models described in the present disclosure. The contralesional CT analysis, based on the ANN models, as described herein, revealed moderate predictor strength in the paracentral and precentral gyrus, both associated with motor function, and both the somatomotor and portions of the CON networks are found to be connected and highly involved in activities of daily living, hence, quality of life, and, therefore, reinforces the notion that postsurgical functional status is vital to overall survival.

[0165] In some embodiments, the ANN models, or machine learning models, described in the present disclosure may predict other outcome metrics including, but are not limited to the following: (i) Overall survival rate metric that measures the percentage of patients who are still alive after a certain period of time following their brain tumor diagnosis; (ii) Progression-free survival (PFS) metric that measures the length of time during or after treatment that a patient lives without their tumor growing or spreading; (iii) Quality of life (QoL) metric that measures the overall well-being and satisfaction of patients with brain tumors based on factors such as physical functioning, emotional well-being, social relationships, and overall satisfaction with life; (iv) Neurological function metric that evaluates the impact of brain tumors on various neurological functions, such as motor skills, speech, memory, and cognitive abilities based on assessment of the extent of impairment caused by the tumor and the effectiveness of treatments in preserving or improving neurological function; (v) Symptom management metric that measures the effectiveness ofCT treatments in managing and reducing symptoms such as headaches, seizures, nausea, and / or neurological deficits due to a tumor for improving the patient's comfort and daily functioning; (vi) Treatment-related side effects metric that evaluates occurrence and severity of treatment-related side effects, such as fatigue, hair loss, cognitive impairments, and / or other complications, from treatments for brain tumors, such as surgery, radiation therapy, and / or chemotherapy; (vii) Functional independence metric that assesses the patient's level of functional independence because brain tumors can impact a patient's ability to independently perform daily activities including activities of daily living (ADLs) and instrumental activities of daily living (IADLs), such as bathing, dressing, cooking, and managing finances; (viii) Patient satisfaction metric provides insights into the patient's experience throughout their brain tumor journey and measures the level of satisfaction and contentment patients have with their overall care, communication with healthcare providers, access to information, and involvement in treatment decision-making; (ix). Emotional well- being metric that assesses the patient’s emotional well-being, including their levels of anxiety, depression, and overall psychological adjustment because brain tumors can have a significant psychological impact on patients, leading to anxiety, depression, and emotional distress; and / or (x) Caregiver burden metric that evaluates the impact of caregiving responsibilities on the caregiver's physical, emotional, and social well-being, providing insights into the support and resources needed for caregivers since brain tumors not only affect patients but also place a significant burden on their caregivers.

[0166] FIG.17 is a block diagram of an example computing device 1700, which may be a configuration of user computer device 802 shown in FIG. 8, including common components such as a processor, memory, communication interface, and the like. In the example embodiment, the computing device 1700 includes a user interface 1702 that receives at least one input from a user (such as user 802). The user interface 1702 may include a keyboard 1704 that enables the user to input pertinent information (where keyboard 1704 may be a configuration of input 812 shown in FIG. 8). The user interface 1702 may also include, for example, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad and a touch screen), a gyroscope, an accelerometer, a position detector, and / or an audio input interface (e.g., including a microphone).

[0167] Moreover, in the example embodiment, computing device 1700 includes a display interface 1706 that presents information, such as input events and / orCT validation results, to the user. The display interface 1706 may also include a display adapter 1708 that is coupled to at least one display device 1710. More specifically, in the example embodiment, the display device 1710 may be a visual display device, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a light-emitting diode (LED) display, and / or an “electronic ink” display. Alternatively, the display interface 717 may include an audio output device (e.g., an audio adapter and / or a speaker) and / or a printer.

[0168] The computing device 1700 also includes a processor 1712 and a memory device 1714. The processor 1712 is coupled to the user interface 1702, the display interface 1706, and the memory device 1714 via a system bus 1716. In the example embodiment, the processor 1712 communicates with the user, such as by prompting the user via the display interface 1706 and / or by receiving user inputs via the user interface 1702. The term “processor” refers generally to any programmable system including systems and microcontrollers, reduced instruction set computers (RISC), complex instruction set computers (CISC), application specific integrated circuits (ASIC), programmable logic circuits (PLC), and any other circuit or processor capable of executing the functions described herein. The above examples are for illustration purposes only, and thus are not intended to limit in any way the definition and / or meaning of the term “processor.”

[0169] In the example embodiment, the memory device 1714 includes one or more devices that enable information, such as executable instructions and / or other data, to be stored and retrieved. Moreover, the memory device 1714 includes one or more computer readable media, such as, without limitation, dynamic random-access memory (DRAM), static random-access memory (SRAM), a solid-state disk, and / or a hard disk. In the example embodiment, the memory device 1714 stores, without limitation, application source code, application object code, configuration data, additional input events, application states, assertion statements, validation results, and / or any other type of data. The computing device 1700, in the example embodiment, may also include a communication interface 1718 that is coupled to the processor 1712 via the system bus 1716. Moreover, the communication interface 1718 is communicatively coupled to data acquisition devices.

[0170] In the example embodiment, the processor 1712 may be programmed by encoding an operation using one or more executable instructions and providing the executable instructions in the memory device 1714. In the exampleCT embodiment, the processor 1712 is programmed to select a plurality of measurements that are received from data acquisition devices.

[0171] FIG. 18 illustrates an example artificial neural network model 1800, such as shown in FIG.11B. The example neural network model 1800 includes layers of neurons 1802, 1804, 1806, and 1808, including an input layer 1810 (including neurons 1802, such as neuron 1816), one or more hidden layers 1812 (including neurons 1804, 1806), and an output layer 1814 (including neurons 1808). Each layer may include any number of neurons, e.g., q, r, and n in FIG. 18, which may be any positive integer. It should be understood that neural networks of a different structure and configuration from that depicted in FIG.18 may be used to achieve the methods and systems described herein.

[0172] In the example embodiment, the input layer 1810 may receive different input data. For example, the input layer 1810 includes a first input a1 representing features generated from structural MRI data, a second input a2 representing resting state functional MRI network correlations, a third input a3 representing patient demographics, and so on. The input layer 1810 may include thousands or more inputs. In some embodiments, the number of elements used by the neural network model 1800 changes during the training process, and some neurons are bypassed or ignored if, for example, during execution of the neural network, they are determined to be of less relevance.

[0173] In the example embodiment, each neuron in hidden layer(s) 1814 processes one or more inputs from the input layer 1810, and / or one or more outputs from neurons in one of the previous hidden layers, to generate a decision or output. The output layer 1814 includes one or more outputs each indicating a label, confidence factor, weight describing the inputs, and / or an output image. In some embodiments, however, outputs of the neural network model 1800 are obtained from a hidden layer(s) 1812 in addition to, or in place of, output(s) from the output layer(s) 1814.

[0174] In some embodiments, each layer of the neural network plays a specific role in processing the input data. For example, when considering an 'n' dimensional input, the neural network's first layer undertakes preliminary processing across all dimensions to extract foundational features. Subsequent layers build upon these foundational features, recognizing complex patterns and relationships within the input data. The final layer combines the information from earlier layers to generate the desired output.CT

[0175] In some embodiments, each layer has a discrete, recognizable function with respect to input data. For example, if n is equal to 3, a first layer analyzes the first dimension of the inputs, a second layer the second dimension, and the final layer the third dimension of the inputs. Dimensions may correspond to aspects considered strongly determinative, then those considered of intermediate importance, and finally those of less relevance.

[0176] In other embodiments, the layers are not clearly delineated in terms of the functionality they perform. For example, two or more of hidden layers 1812 may share decisions relating to labeling, with no single layer making an independent decision as to labeling.

[0177] FIG. 19 illustrates an example neuron 1900 that corresponds, for example, to the neuron 1816 labeled as “1,1” in hidden layer 1812 of FIG.18, according to one embodiment. Each of the inputs to the neuron 1900 (e.g., the inputs in the input layer 1810 in FIG.18) is weighted such that input a1through apcorresponds to weights w1through wpas determined during the training process of the neural network model 1800.

[0178] In some embodiments, some inputs lack an explicit weight, or have a weight below a threshold. The weights are applied to a function α (labeled by a reference numeral 1902), which may be a summation and may produce a value z1which is input to a function 1904, labeled as f1,1(z1). The function 1904 is any suitable linear or non-linear function. As depicted in FIG. 19, the function 1904 produces one or more outputs, which may be provided to neuron(s) of a subsequent layer or used as an output of the neural network model 1800. For example, the outputs may correspond to index values of a list of labels or may be calculated values used as inputs to subsequent functions.

[0179] It should be appreciated that the structure and function of the neural network model 1800 and the neuron 1816 (e.g., 1900) depicted are for illustration purposes only, and that other suitable configurations exist. For example, the output of any given neuron may depend not only on values determined by past neurons, but also on future neurons.

[0180] The neural network model 1800 may include a convolutional neural network (CNN), a deep learning neural network, a reinforced or reinforcementCT learning module or program, or a combined learning module or program that learns in two or more fields or areas of interest. Supervised and unsupervised machine learning techniques may be used. In supervised machine learning, a processing element may be provided with example inputs and their associated outputs and may seek to discover a general rule that maps inputs to outputs, so that when subsequent novel inputs are provided the processing element may, based upon the discovered rule, accurately predict the correct output. The neural network model 1800 may be trained using unsupervised machine learning programs. In unsupervised machine learning, the processing element may be required to find its own structure in unlabeled example inputs. Machine learning may involve identifying and recognizing patterns in existing data in order to facilitate making predictions for subsequent data. Models may be created based upon example inputs in order to make valid and reliable predictions for novel inputs.

[0181] Additionally, or alternatively, the machine learning programs may be trained by inputting sample data sets or certain data into the programs, such as images, object statistics, and information. The machine learning programs may use deep learning algorithms that may be primarily focused on pattern recognition and may be trained after processing multiple examples. The machine learning programs may include Bayesian Program Learning (BPL), voice recognition and synthesis, image or object recognition, optical character recognition, and / or natural language processing – either individually or in combination. The machine learning programs may also include natural language processing, semantic analysis, automatic reasoning, and / or machine learning.

[0182] Based upon these analyses, the neural network model 1800 may learn how to identify characteristics and patterns that may then be applied to analyzing image data, model data, and / or other data. For example, the model 1800 may learn to identify features in a series of data points.

[0183] The embodiment shown and described in connection with FIGS. 11A-19 may utilize a server computer device such as shown and described in connection with FIG.9. III. Predicting Surgical Outcomes in a PatientCT

[0184] While functional outcomes may generally refer to the patient’s ability to perform daily activities and maintain cognitive and motor functions post-surgery, surgical outcomes are more focused on (e.g., physical) aspects such as the extent of tumor resection and the immediate success of the surgery in terms of removing the tumor while minimizing damage to surrounding brain tissue. Actual surgical outcomes depend on various factors, such as the surgeon’s skill, the tumor’s location, and its relationship with critical brain structures.

[0185] As described herein, the current standard of care includes surgical resection of the tumor, which can lead to functional and cognitive deficits. More specifically, standard clinical practice typically entails gross total resection of the tumor, followed by radiation and adjuvant chemoradiotherapy. Even more specifically, the standard first-line treatment for newly diagnosed GBM is surgical resection and radiotherapy with concomitant temozolomide, followed by adjuvant temozolomide. However, surgical resection of the tumor carries the risk of causing or exacerbating functional impairment, a significant concern for the clinician, as postsurgical functional preservation (e.g., quality of life) is known to correlate with overall patient outcomes. Due to the variability in proliferative and infiltrative patterns across tumor grades, it is also difficult to interpret the prevalence of intratumoral function and its effects on patient outcomes following surgical resection. For example, patients with tumors who had stimulation-identified functional tissue located within tumor boundaries may experience deficits when their tumors are resected.

[0186] As described herein, structural (e.g., T1-weighted pre- and post- contrast, T2-weighted, and DTI) and functional (e.g., blood oxygen level-dependent task or resting state fMRI) MRI (magnetic resource imaging) data are routinely acquired in the pre- surgical setting. The former assesses tumor size and location, while the latter helps evaluate a planned resection's functional and cognitive implications. Using the various MRI modalities as a guide, the surgeon faces the challenge of achieving maximal resection of infiltrative tissue while minimizing functional morbidity. These competing interests, long- term survival through maximum resection versus quality of life through functional preservation, significantly impact overall patient outcomes, and finding the optimal balance between the two is an ongoing challenge.CT

[0187] Additionally, neuroimaging predictors of recurrence and other clinically relevant determinants have been predominantly structural measures. Intertumor / intratumor heterogeneity, characterized by distinct genetic alterations, rapid proliferation, aggressive infiltration, multiple activated signal transduction pathways, and the emergence of treatment-resistant cells soon after therapy onset, limits the effectiveness of current therapies, leading to inevitable tumor recurrence and death.

[0188] Given this information, the systems and methods described herein may also use ML to extract and leverage pertinent features from RS-fMRI and patient data to accurately predict surgical outcomes, such as surgical outcomes relating to surgical resection, tumor recurrence, and functional deficits. In this regard, a surgical outcome prediction pipeline similar to the pipelines shown and described in connection with FIGS.1 and 11A may be prepared and utilized in connection with predicting surgical outcomes. Due to the overlap between functional outcomes and surgical outcomes, the functional outcome model module 1008 may also perform the task of generating a surgical outcome prediction model as part of generating the functional outcome model. Alternatively, there may be a separate surgical outcome prediction model module (not shown) specifically tailored to aspects of surgical outcomes, much like the respective modules for each of the functional outcome model module 1008 and the survival outcome model module 1030 shown and described in connection with FIG.10.

[0189] In the exemplary embodiment, the FOPA / SOPA computer device trains the surgical outcome prediction model using historical data as described herein, such as historical surgical outcome data for patients diagnosed with HGG (e.g., using pathological examination of biopsy and resection-acquired brain samples), and the clinical characteristics shown in TABLES 1 and 2, such as the extent of resection (gross total resection (GTR), near total resection (NTR), or subtotal resection (STR)) and other classifications of surgical resections.

[0190] For example, in one embodiment of a surgical outcome prediction model, and with a focus on resection data, the pipelines and model training systems and methods described herein leverage deep learning to offer a more nuanced evaluation of the cognitive, behavioral, and emotional impacts that a planned resection might have in connection with predicting surgical outcomes. The training of the surgical outcomeCT prediction model may take into account aspects described herein, such as: (i) the extent of surgical resection of the lesion and functional preservation (e.g., extent of resection (gross total (GTR), subtotal resection (STR), and near total resection (NTR), as shown in FIG.14B at 1410); (ii) tumor size and location, cranial nerve manipulation, major brain vessel manipulation, and posterior fossa location; (iii) morphological changes in brain anatomy distant from the tumor; (iv) functional networks before / after resection; and (v) results such as those shown and described in connection with FIGS.15E and 15F. In this example, the deep feed-forward ANN models described herein may be trained to classify GBM patients according based on the above categories of data to predict future surgical outcomes. An advantage of the ANN model, as described herein, in accordance with some embodiments, is that reliable and accurate surgical outcome classification is generated, thereby improving surgical planning for GBM patients. The surgical outcome dataset may be transformed into surgical outcome prediction data to generate a prediction measure for a surgical outcome.

[0191] Accordingly, in addition to the functional outcome model and the survival outcome models described herein, a surgical outcome prediction model may also be generated and used to predict surgical outcomes using the systems and methods described herein. The outputs (e.g., 1029, 1052, and likewise an output of a surgical outcome prediction model) of the models may be referred to as prediction measures, such prediction measures being based on one or more of the predicted functional outcomes for the patient, the predicted survival outcomes of the patient, and / or the predicted surgical outcomes of the patient, and being usable by a surgeon or the like to generate or update a surgical plan for the patient (if in fact there will still be a surgery that is performed, based on a determination made from an analysis of the outputs of the model). Predictors may be utilized with the surgical outcome model in the same or similar manner as described herein for other models. The surgical outcome model is for all intents and purposes the same as or similar to the functional and survival models as described herein except for the particular type of data (e.g., surgical data) used with the surgical outcome model.

[0192] At least one technical effect of the systems and methods described herein includes (i) training datasets of a neural network model being different from datasets during inference, and (ii) making a plurality of predictions for a patient based on multi-modal data as inputs, the plurality of predictions including but not limited to functional outcome, survival outcome, surgical outcome, and the like. For example, predicting a surgical outcomeCT of a planned brain surgery for a patient using at least RS-fMRI data, where a focus may be placed on making a surgical outcome prediction relating to example tumor recurrence, functional deficits, and the like. Put another way, an outcome of the surgery is predicted based on analysis of at least the resting state data as described herein.

[0193] The present systems and methods: receive a plurality of RS-fMRI images of a patient (see FIG. 11A at portion 1102, for example); receive a plurality of attributes (see, e.g., TABLE 2) of the patient; generate a multi-modal dataset (e.g., data 1006, shown in FIG. 10) from the RS-fMRI images of a patient and the plurality of attributes of the patient; transform the multi-modal dataset into prediction data (see FIGS.15A-15F, 16A- 16F, for example); and execute the one or more models (see FIG.10) with the multi-modal dataset as an input to transform the prediction data into one or more prediction measures relating to various prediction which can be used to update a diagnosis of the patient. IV. Machine Learning and Other Matters

[0194] The computer-implemented methods discussed herein may include additional, less, or alternate actions, including those discussed elsewhere herein. The methods may be implemented via one or more local or remote processors, transceivers, servers, and / or sensors (such as processors, transceivers, servers, and / or sensors mounted on vehicles or mobile devices, or associated with smart infrastructure or remote servers), and / or via computer-executable instructions stored on non-transitory computer-readable media or medium.

[0195] In some embodiments, FOPA / SOPA computer system 702 is configured to implement machine learning, such that FOPA / SOPA computer system 702 “learns” to analyze, organize, and / or process data without being explicitly programmed. Machine learning may be implemented through machine learning methods and algorithms (“ML methods and algorithms”). In an exemplary embodiment, a machine learning module (“ML module”) is configured to implement ML methods and algorithms. In some embodiments, ML methods and algorithms are applied to data inputs and generate machine learning outputs (“ML outputs”). Data inputs may include but are not limited to images, text data, and / or other types of data. ML outputs may include, but are not limited to identified objects, items classifications, textual product, and / or other data extracted from the images or textual data. In some embodiments, data inputs may include certain ML outputs.CT

[0196] In some embodiments, at least one of a plurality of ML methods and algorithms may be applied, which may include but are not limited to: linear or logistic regression, instance-based algorithms, regularization algorithms, decision trees, Bayesian networks, cluster analysis, association rule learning, artificial neural networks, deep learning, ensemble learning, dimensionality reduction, and support vector machines. In various embodiments, the implemented ML methods and algorithms are directed toward at least one of a plurality of categorizations of machine learning, such as supervised learning, unsupervised learning, and reinforcement learning.

[0197] In one embodiment, the ML module employs supervised learning, which involves identifying patterns in existing data to make predictions about subsequently received data. Specifically, the ML module is “trained” using training data, which includes example inputs and associated example outputs. Based upon the training data, the ML module may generate a predictive function which maps outputs to inputs and may utilize the predictive function to generate ML outputs based upon data inputs. The example inputs and example outputs of the training data may include any of the data inputs or ML outputs described above. In the exemplary embodiment, a processing element may be trained by providing it with a large sample of text with known characteristics or features. Such information may include, for example, information associated with a plurality of text of a plurality of different vendors, objects, items, and / or property.

[0198] In another embodiment, a ML module may employ unsupervised learning, which involves finding meaningful relationships in unorganized data. Unlike supervised learning, unsupervised learning does not involve user-initiated training based upon example inputs with associated outputs. Rather, in unsupervised learning, the ML module may organize unlabeled data according to a relationship determined by at least one ML method / algorithm employed by the ML module. Unorganized data may include any combination of data inputs and / or ML outputs as described above.

[0199] In yet another embodiment, a ML module may employ reinforcement learning, which involves optimizing outputs based upon feedback from a reward signal. Specifically, the ML module may receive a user-defined reward signal definition, receive a data input, utilize a decision-making model to generate a ML output based upon the data input, receive a reward signal based upon the reward signal definitionCT and the ML output, and alter the decision-making model so as to receive a stronger reward signal for subsequently generated ML outputs. Other types of machine learning may also be employed, including deep or ensemble learning techniques.

[0200] In some embodiments, generative artificial intelligence (AI) models (also referred to as generative machine learning (ML) models) may be utilized with the present embodiments. The voice bots or chatbots discussed herein may be configured to utilize artificial intelligence and / or machine learning techniques. For instance, the voice or chatbot may be a LLM chatbot. The voice or chatbot may employ supervised or unsupervised machine learning techniques, which may be followed by, and / or used in conjunction with, reinforced or reinforcement learning techniques. The voice or chatbot may employ the techniques utilized for large language models (LLM). The voice bot, chatbot, LLM-based bot, LLM bot, and / or other bots may generate audible or verbal output, text or textual output, visual or graphical output, output for use with speakers and / or display screens, and / or other types of output for user and / or other computer or bot consumption.

[0201] In some embodiments, the design system is configured to implement machine learning, such that the neural network “learns” to analyze, organize, and / or process data without being explicitly programmed. Machine learning may be implemented through machine learning (ML) methods and algorithms. In an example embodiment, a machine learning (ML) module is configured to implement ML methods and algorithms. In some embodiments, ML methods and algorithms are applied to data inputs and generate machine learning (ML) outputs. Data inputs may include, but are not limited to, analog and digital signals, sensor data, image data, video data, patient data, and the like. ML outputs may include, but are not limited to, digital signals, medical diagnoses, segmented images, health care predictions and guidance, and the like. In some embodiments, data inputs may include certain ML outputs.

[0202] In some embodiments, at least one of a plurality of ML methods and algorithms may be applied, which may include but are not limited to: linear or logistic regression, instance-based algorithms, regularization algorithms, decision trees, random forest, Bayesian networks, cluster analysis, association rule learning, artificial neural networks, deep learning, recurrent neural networks, Monte Carlo search trees, generative adversarial networks, dimensionality reduction, and support vector machines. In variousCT embodiments, the implemented ML methods and algorithms are directed toward at least one of a plurality of categorizations of machine learning, such as supervised learning, unsupervised learning, and reinforcement learning.

[0203] In some embodiments, ML methods and algorithms are directed toward supervised learning, which involves identifying patterns in existing data to make predictions about subsequently received data. Specifically, ML methods and algorithms directed toward supervised learning are “trained” through training data, which includes example inputs and associated example outputs. Based on the training data, the ML methods and algorithms may generate a predictive function which maps outputs to inputs and utilize the predictive function to generate ML outputs based on data inputs. The example inputs and example outputs of the training data may include any of the data inputs or ML outputs described above. For example, a ML module may receive training data comprising data associated with different patients and their corresponding outcomes, generate a model which maps the patient data to the outcome data, and recognize potential future outcomes for patients.

[0204] In some embodiments, ML methods and algorithms are directed toward unsupervised learning, which involves finding meaningful relationships in unorganized data. Unlike supervised learning, unsupervised learning does not involve user- initiated training based on example inputs with associated outputs. Rather, in unsupervised learning, unlabeled data, which may be any combination of data inputs and / or ML outputs as described above, is organized according to an algorithm-determined relationship. In an example embodiment, a ML module coupled to or in communication with the design system or integrated as a component of the design system receives unlabeled data, and the ML module employs an unsupervised learning method such as “clustering” to identify patterns and organize the unlabeled data into meaningful groups. The newly organized data may be used, for example, to extract further information about the potential classifications.

[0205] In some embodiments, ML methods and algorithms are directed toward reinforcement learning, which involves optimizing outputs based on feedback from a reward signal. Specifically ML methods and algorithms directed toward reinforcement learning may receive a user-defined reward signal definition, receive a data input, utilize a decision-making model to generate a ML output based on the data input, receive a rewardCT signal based on the reward signal definition and the ML output, and alter the decision-making model so as to receive a stronger reward signal for subsequently generated ML outputs. The reward signal definition may be based on any of the data inputs or ML outputs described above. In an example embodiment, a ML module implements reinforcement learning in a user recommendation application. The ML module may utilize a decision-making model to generate a ranked list of options based on user information received from the user and may further receive selection data based on a user selection of one of the ranked options. A reward signal may be generated based on comparing the selection data to the ranking of the selected option. The ML module may update the decision-making model such that subsequently generated rankings more accurately predict optimal constraints.

[0206] Based upon these analyses, the processing element may learn how to identify characteristics and patterns that may then be applied to analyzing and classifying objects. The processing element may also learn how to identify attributes of different objects in different lighting. This information may be used to determine which classification models to use and which classifications to provide. V. Additional Considerations

[0207] As will be appreciated based upon the foregoing specification, the above-described embodiments of the disclosure may be implemented using computer programming or engineering techniques including computer software, firmware, hardware or any combination or subset thereof. Any such resulting program, having computer- readable code means, may be embodied or provided within one or more computer-readable media, thereby making a computer program product, e.g., an article of manufacture, according to the discussed embodiments of the disclosure. The computer-readable media may be, for example, but is not limited to, a fixed (hard) drive, diskette, optical disk, magnetic tape, semiconductor memory such as read-only memory (ROM), and / or any transmitting / receiving medium such as the Internet or other communication network or link. The article of manufacture containing the computer code may be made and / or used by executing the code directly from one medium, by copying the code from one medium to another medium, or by transmitting the code over a network.

[0208] These computer programs (also known as programs, software, software applications, “apps,” or code) include machine instructions for a programmableCT processor and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms “machine-readable medium” “computer-readable medium” refers to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The “machine-readable medium” and “computer-readable medium,” however, do not include transitory signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0209] As used herein, the term “database” can refer to either a body of data, a relational database management system (RDBMS), or to both. As used herein, a database can include any collection of data including hierarchical databases, relational databases, flat file databases, object-relational databases, object-oriented databases, and any other structured collection of records or data that is stored in a computer system. The above examples are example only, and thus are not intended to limit in any way the definition and / or meaning of the term database. Examples of RDBMS’ include, but are not limited to including, Oracle® Database, MySQL, IBM® DB2, Microsoft® SQL Server, Sybase®, NoSQL, and PostgreSQL. However, any database can be used that enables the systems and methods described herein. (Oracle is a registered trademark of Oracle Corporation, Redwood Shores, California; IBM is a registered trademark of International Business Machines Corporation, Armonk, New York; Microsoft is a registered trademark of Microsoft Corporation, Redmond, Washington; and Sybase is a registered trademark of Sybase, Dublin, California.)

[0210] As used herein, a processor may include any programmable system including systems using micro-controllers, reduced instruction set circuits (RISC), application specific integrated circuits (ASICs), logic circuits, and any other circuit or processor capable of executing the functions described herein. The above examples are example only and are thus not intended to limit in any way the definition and / or meaning of the term “processor.”CT

[0211] As used herein, the terms “software” and “firmware” are interchangeable and include any computer program stored in memory for execution by a processor, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The above memory types are example only and are thus not limiting as to the types of memory usable for storage of a computer program.

[0212] In another example, a computer program is provided, and the program is embodied on a computer-readable medium. In an example, the system is executed on a single computer system, without requiring a connection to a server computer. In a further example, the system is being run in a Windows® environment (Windows is a registered trademark of Microsoft Corporation, Redmond, Washington). In yet another example, the system is run on a mainframe environment and a UNIX® server environment (UNIX is a registered trademark of X / Open Company Limited located in Reading, Berkshire, United Kingdom). In a further example, the system is run on an iOS® environment (iOS is a registered trademark of Cisco Systems, Inc. located in San Jose, CA). In yet a further example, the system is run on a Mac OS® environment (Mac OS is a registered trademark of Apple Inc. located in Cupertino, CA). In still yet a further example, the system is run on Android® OS (Android is a registered trademark of Google, Inc. of Mountain View, CA). In another example, the system is run on Linux® OS (Linux is a registered trademark of Linus Torvalds of Boston, MA). The application is flexible and designed to run in various different environments without compromising any major functionality.

[0213] In some embodiments, the system includes multiple components distributed among a plurality of computing devices. One or more components may be in the form of computer-executable instructions embodied in a computer-readable medium. The systems and processes are not limited to the specific embodiments described herein. In addition, components of each system and each process can be practiced independent and separate from other components and processes described herein. Each component and process can also be used in combination with other assembly packages and processes.

[0214] As used herein, an element or step recited in the singular and proceeded with the word “a” or “an” should be understood as not excluding plural elements or steps, unless such exclusion is explicitly recited. Furthermore, references to “example” or “one example” of the present disclosure are not intended to be interpreted as excludingCT the existence of additional examples that also incorporate the recited features. Further, to the extent that terms “includes,” “including,” “has,” “contains,” and variants thereof are used herein, such terms are intended to be inclusive in a manner similar to the term “comprises” as an open transition word without precluding any additional or other elements.

[0215] Furthermore, as used herein, the term “real-time” refers to at least one of the time of occurrence of the associated events, the time of measurement and collection of predetermined data, the time to process the data, and the time of a system response to the events and the environment. In the examples described herein, these activities and events occur substantially instantaneously.

[0216] In operation, a computer executes computer-executable instructions embodied in one or more computer-executable components stored on one or more computer-readable media to implement aspects of the invention described and / or illustrated herein. The order of execution or performance of the operations in embodiments of the invention illustrated and described herein is not essential, unless otherwise specified. That is, the operations may be performed in any order, unless otherwise specified, and embodiments of the invention may include additional or fewer operations than those disclosed herein. For example, it is contemplated that executing or performing a particular operation before, contemporaneously with, or after another operation is within the scope of aspects of the invention.

[0217] The computer-implemented methods discussed herein may include additional, less, or alternate actions, including those discussed elsewhere herein. The methods may be implemented via one or more local or remote processors, transceivers, servers, and / or sensors, and / or via computer-executable instructions stored on non-transitory computer-readable media or medium.

[0218] Additionally, the computer systems discussed herein may include additional, less, or alternate functionality, including that discussed elsewhere herein. The computer systems discussed herein may include or be implemented via computer-executable instructions stored on non-transitory computer-readable media or medium.

[0219] Example embodiments of systems and methods are described above in detail are not limited to the specific embodiments described herein but, rather,CT components of the systems and / or operations of the methods may be utilized independently and separately from other components and / or operations described herein. Further, the described components and / or operations may also be defined in, or used in combination with, other systems, methods, and / or devices, and are not limited to practice with only the systems described herein.

[0220] The computer-implemented methods discussed herein may include additional, less, or alternate actions, including those discussed elsewhere herein. The methods may be implemented via one or more local or remote processors, transceivers, servers, and / or sensors (such as processors, transceivers, and / or servers), and / or via computer-executable instructions stored on non-transitory computer-readable media or medium. Additionally, the computer systems discussed herein may include additional, less, or alternate functionality, including that discussed elsewhere herein. The computer systems discussed herein may include or be implemented via computer-executable instructions stored on non-transitory computer-readable media or medium.

[0221] As used herein, the term “non-transitory computer-readable media” is intended to be representative of any tangible computer-based device implemented in any method or technology for short-term and long-term storage of information, such as, computer-readable instructions, data structures, program modules and sub-modules, or other data in any device. Therefore, the methods described herein may be encoded as executable instructions embodied in a tangible, non-transitory, computer readable medium, including, without limitation, a storage device and / or a memory device. Such instructions, when executed by a processor, cause the processor to perform at least a portion of the methods described herein. Moreover, as used herein, the term “non-transitory computer-readable media” includes all tangible, computer-readable media, including, without limitation, non- transitory computer storage devices, including, without limitation, volatile and nonvolatile media, and removable and non-removable media such as a firmware, physical and virtual storage, CD-ROMs, DVDs, and any other digital source such as a network or the Internet, as well as yet to be developed digital means, with the sole exception being a transitory, propagating signal.

[0222] Although specific features of various embodiments of the disclosure may be shown in some drawings and not in others, this is for convenience only.CT In accordance with the principles of the invention, any feature of a drawing may be referenced and / or claimed in combination with any feature of any other drawing.

[0223] The patent claims at the end of this document are not intended to be construed under 35 U.S.C. § 112(f) unless traditional means-plus-function language is expressly recited, such as “means for” or “step for” language being expressly recited in the claim(s).

[0224] This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal language of the claims.

Claims

CT WHAT IS CLAIMED IS:

1. A computer system for patient outcome prediction and analysis, the computer system including at least one processor in communication with at least one memory device, the at least one processor programmed to: store one or more models trained to analyze multi-modal data, the multi- modal data including (i) resting state functional magnetic resource imaging (RS-fMRI) data, and (ii) patient attribute data; receive a plurality of RS-fMRI images of a patient; receive a plurality of attributes of the patient; generate a multi-modal dataset from the RS-fMRI images of a patient and the plurality of attributes of the patient; execute the one or more models with the multi-modal dataset as an input to transform the multi-modal dataset into prediction data corresponding to one or more of (i) predicted functional outcomes of the patient, (ii) predicted survival outcomes of the patient; and (iii) predicted surgical outcomes of the patient; and output, from the one or more models, one or more prediction measures for the patient, the one or more prediction measures being generated based on one or more of the predicted functional outcomes of the patient, the predicted survival outcomes of the patient, and the predicted surgical outcomes of the patient.

2. The computer system of claim 1, wherein the multi-modal dataset includes one or more of: anatomical brain imaging data; demographic information of the patient; and genetic information of the patient.

3. The computer system of claim 2, wherein the anatomical brain imaging data includes neuroimaging data relating to resting-state functional connectivity.

4. The computer system of claim 2, wherein the anatomical brain imaging data includes cortical thickness data.CT 5. The computer system of claim 1, wherein the one or more models include a functional outcome prediction model, a survival outcome prediction model, and a surgical outcome prediction.

6. The computer system of claim 5, wherein the functional outcome prediction model classifies functional outcomes based on data including one or more of: age, tumor location, RS-fMRI measures, and tumor size.

7. The computer system of claim 5, wherein each of the functional outcome prediction model, the survival outcome prediction model, and the surgical outcome prediction model comprise a machine learning model.

8. The computer system of claim 7, wherein each machine learning model comprises a supervised machine learning model trained at least in part on prior patient data.

9. The computer system of claim 5, wherein the survival outcome prediction model predicts a survival outcome based on one or more predictors.

10. The computer system of claim 9, wherein the one or more predictors include encoded network connectivity features.

11. The computer system of claim 1, wherein the one or more prediction measures are analyzed by a human reviewer to generate or update (i) a surgical plan for a future surgery of the patient based upon the predicted functional outcomes of the patient and / or the predicted surgical outcomes of the patient, and / or (ii) a survival diagnosis of the patient based on the predicted survival outcomes of the patient.

12. A computer-implemented method of predicting a patient outcome, the method comprising: storing one or more models trained to analyze multi-modal data, the multi- modal data including (i) resting state functional magnetic resource imaging (RS-fMRI) data, and (ii) patient attribute data; receiving a plurality of RS-fMRI images of a patient;CT receiving a plurality of attributes of the patient; generating a multi-modal dataset from the RS-fMRI images of a patient and the plurality of attributes of the patient; executing the one or more models with the multi-modal dataset as an input to transform the multi-modal dataset into prediction data corresponding to one or more of (i) predicted functional outcomes of the patient, (ii) predicted survival outcomes of the patient; and (iii) predicted surgical outcomes of the patient; and outputting, from the one or more models, one or more prediction measures for the patient, the one or more prediction measures being generated based on one or more of the predicted functional outcomes of the patient, the predicted survival outcomes of the patient, and the predicted surgical outcomes of the patient.

13. The method of claim 12, wherein the multi-modal dataset includes one or more of: anatomical brain imaging data; demographic information of the patient; and genetic information of the patient.

14. The method of claim 13, wherein the anatomical brain imaging data includes cortical thickness data.

15. The method of claim 12, wherein the one or more models include a functional outcome prediction model, a survival outcome prediction model, and a surgical outcome prediction.

16. The method of claim 15, wherein each of the functional outcome prediction model, the survival outcome prediction model, and the surgical outcome prediction model comprise a machine learning model.

17. The method of claim 16, wherein each machine learning model comprises a supervised machine learning model trained at least in part on prior patient data.

18. The method of claim 15, wherein the survival outcome prediction model predicts a survival outcome based on one or more predictors.CT 19. The method of claim 18, wherein the one or more predictors include encoded network connectivity features.

20. The method of claim 12, further comprising analyzing, by a human reviewer, the one or more prediction measures to generate or update (i) a surgical plan for a future surgery of the patient based upon the predicted functional outcomes of the patient and / or the predicted surgical outcomes of the patient, and / or (ii) a survival diagnosis of the patient based on the predicted survival outcomes of the patient.

Images