Method and system for deep learning based digital cancer pathology assessment

Abstract

The present disclosure provides a method and system for classifying and / or monitoring a cancer in a subject. A method for assessing a cancer in a subject may include obtaining a dataset including image data and / or tabular data from the subject, and processing the data with one or more trained algorithms to classify the cancer in the subject. Based on the result of the classification, the cancer in the subject can be assessed.

Description

[Technical field]

[0001] cross reference

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 287,158, filed December 8, 2021, U.S. Provisional Patent Application No. 63 / 345,804, filed May 25, 2022, and U.S. Provisional Patent Application No. 63 / 418,125, filed October 21, 2022, each of which is incorporated by reference in its entirety into this specification. [Background technology]

[0002] Prostate cancer is the leading cause of cancer death in men. Nevertheless, international standards for prognostication of patient outcome rely on non-specific and insensitive tools that usually lead to overtreatment or undertreatment. Summary of the Invention [Problem to be solved by the invention]

[0003]

[0003] Determining a patient's optimal cancer treatment is a challenging task, and oncologists must select the treatment with the highest probability of success and the lowest potential for toxicity. The difficulty of treatment selection is rooted in the enormous molecular, phenotypic, and prognostic heterogeneity that cancers exhibit. Herein, the need for accurate, globally scalable tools to aid in the personalization of cancer treatment is recognized. [Means for solving the problem]

[0004]

[0004] The present disclosure provides methods and systems for identifying or monitoring a cancer-related condition by processing a biological sample obtained or derived from a subject, such as a cancer patient. The biological sample (e.g., a tissue sample) obtained from the subject can be analyzed to prognose clinical outcomes, which may include, for example, distant metastasis, biochemical recurrence, death, progression-free survival, and overall survival.

[0005]

[0005] In one aspect, the present disclosure provides a method for assessing cancer in a subject, the method including: (a) obtaining a dataset including image data and tabular data derived from the subject; (b) processing the dataset using a trained algorithm to classify the dataset into one of a plurality of categories, where the classifying includes applying an image processing algorithm to the image data; and (c) assessing the subject's cancer based at least in part on the one of the plurality of categories classified in (b).

[0006]

[0006] In some embodiments, the trained algorithm is trained using self-supervised learning. In some embodiments, the trained algorithm includes a deep learning algorithm. In some embodiments, the trained algorithm includes a first trained algorithm that processes image data and a second trained algorithm that processes tabulated data. In some embodiments, the trained algorithm further includes a third trained algorithm that processes an output of the first and second trained algorithms. In some embodiments, the cancer is bladder cancer, breast cancer, cervical cancer, colorectal cancer, gastric cancer, kidney cancer, liver cancer, ovarian cancer, pancreatic cancer, prostate cancer, or thyroid cancer. In some embodiments, the cancer is prostate cancer. In some embodiments, the tabulated data includes clinical data of the subject. In some embodiments, the clinical data of the subject includes laboratory data, therapeutic interventions, or long-term outcomes. In some embodiments, the image data includes digital histopathology data. In some embodiments, the histopathology data includes images derived from a biopsy sample of the subject. In some embodiments, the images are obtained by microscopic examination of the biopsy sample. In some embodiments, the digital histopathology data is obtained from the subject before the subject undergoes treatment. In some embodiments, the treatment comprises radiation therapy (RT). In some embodiments, the RT comprises a pre-configured use of short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), dose-escalated RT (DE-RT), or a combination thereof. In some embodiments, the digital histopathology data is obtained from the subject after the subject has undergone the treatment. In some embodiments, the treatment comprises radiation therapy (RT). In some embodiments, the RT comprises a pre-configured use of short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), dose-escalated RT (DE-RT), or a combination thereof. In some embodiments, the method further comprises processing the image data using image segmentation, image stitching, object detection algorithms, or a combination thereof. In some embodiments, the method further comprises extracting features from the image data.

[0007]

[0007] In another aspect, the present disclosure provides a method for assessing cancer in a subject, the method including: (a) obtaining a dataset including at least image data derived from the subject; (b) processing the dataset using a trained algorithm to classify the dataset into one of a plurality of categories, where the classifying step includes applying an image processing algorithm to the image data, and the trained algorithm is trained using self-supervised learning; and (c) assessing cancer in the subject based at least in part on the one of the plurality of categories classified in (b).

[0008]

[0008] In some embodiments, the trained algorithm comprises a deep learning algorithm. In some embodiments, the cancer is bladder cancer, breast cancer, cervical cancer, colorectal cancer, gastric cancer, kidney cancer, liver cancer, ovarian cancer, pancreatic cancer, prostate cancer, or thyroid cancer. In some embodiments, the cancer is prostate cancer. In some embodiments, the image data comprises digital histopathology data. In some embodiments, the histopathology data comprises images derived from a biopsy sample of the subject. In some embodiments, the images are obtained by microscopic examination of the biopsy sample. In some embodiments, the digital histopathology data is obtained from the subject before the subject undergoes a treatment. In some embodiments, the treatment comprises radiation therapy (RT). In some embodiments, the RT comprises a pre-configured use of short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), dose-escalated RT (DE-RT), or a combination thereof. In some embodiments, the digital histopathology data is obtained from the subject after the subject undergoes a treatment. In some embodiments, the treatment comprises radiation therapy (RT). In some embodiments, the RT comprises a pre-configured use of short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), dose-escalated RT (DE-RT), or a combination thereof. In some embodiments, the method further comprises processing the image data with an image segmentation, image stitching, or object detection algorithm. In some embodiments, the method further comprises extracting features from the image data. In some embodiments, the dataset comprises image data and tabulated data. In some embodiments, the trained algorithm comprises a first trained algorithm that processes the image data and a second trained algorithm that processes the tabulated data. In some embodiments, the trained algorithm further comprises a third trained algorithm that processes an output of the first and second trained algorithms. In some embodiments, the tabulated data comprises clinical data of the subject. In some embodiments, the clinical data comprises laboratory data, therapeutic interventions, or long-term outcomes.

[0009]

[0009] Another aspect of the present disclosure provides a non-transitory computer-readable medium containing machine-executable code that, when executed by one or more computer processors, performs any of the methods described above or elsewhere in this specification.

[0010] Another aspect of the present disclosure provides a system including one or more computer processors and a computer memory coupled thereto, the computer memory including machine-executable code that, when executed by the one or more computer processors, performs any of the methods described above or elsewhere herein.

[0011]

[0011] Further aspects and advantages of the present disclosure will become readily apparent to those skilled in the art from the following detailed description, in which only illustrative embodiments of the present disclosure are shown and described. The present disclosure is capable of other and different embodiments, and its several details are capable of modification in various obvious respects, all without departing from the present disclosure. Accordingly, the drawings and description should be regarded as illustrative in nature, and not as restrictive. Incorporation by Reference

[0012]

[0012] All publications, patents, and patent applications mentioned in this specification are incorporated by reference herein to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent that the publications and patents or patent applications incorporated by reference conflict with the disclosure contained herein, the specification is intended to supersede and / or take precedence over such conflicting material.

[0013]

[0013] The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings (also referred to herein as "drawings" and "figures") in which: [Brief description of the drawings]

[0014] [Figure 1]

[0014] FIG. 1 is a diagram of a computer system programmed or otherwise configured to implement the methods provided herein. [Figure 2A]

[0015] FIG. 1 is a diagram of an example multimodal deep learning system and dataset, showing that the multimodal architecture consists of three parts: a tower stack that analyzes tabular clinical data, a tower stack that analyzes a variable number of digital histopathology slides, and a third tower stack that merges the resulting features and predicts a binary outcome. [Figure 2B] FIG. 1 is a diagram of an example multi-modal deep learning system and dataset for training a self-supervised model of image tower stacks. [Figure 2C] Table of an example multimodal deep learning system and dataset. The first 5 columns of the table show statistics for each study. The "combined" column shows statistics for the final dataset using all 5 studies for training and validation. ***RTOG9413 will randomize patients in a 2x2 fashion to examine the impact of hormonal therapy timing (pre-RT vs. RT initiation) and field size (prostate-only vs. complete pelvic RT). New abbreviations are used: disease-free survival (DFS), progression-free survival (PFS), and prostate cancer-specific mortality (PCSM). [Figure 3A]

[0016] An example of a comparison of a deep learning system with established clinical guidelines across outcomes from a clinical trial is shown. Performance results reported using time-dependent receiver operating characteristics for area under the curve (AUC) for sensitivity and specificity for the MMAI model (blue bars) versus the NCCN model (grey bars). Comparisons are made at 5 and 10 years for binary outcomes: distant metastasis (DM), biochemical recurrence (BCR), prostate cancer-specific survival (PCaSS), and overall survival (OS). [Figure 3B]An example of a comparison of a deep learning system with established clinical guidelines is shown in the table across outcomes from clinical trials. A summary table of the relative improvement of the AI ​​model over the NCCN model in various outcomes, split by performance on data from each trial in the test set. The relative improvement is given as (PAI-PNCCN) / PNCCN, where P is the performance of the model. [Figure 3C] An example of a comparison of a deep learning system to established clinical guidelines across outcomes from a clinical trial is shown. Results from an ablation study showing model performance when trained on a decreasing set of data inputs. NCCN refers to three variables: combined Gleason, baseline psa, t stage. NCCN+3 refers to NCCN plus Gleason primary, Gleason secondary, and age. Path refers to digital histopathology images. [Figure 3D] 3A-3H show an example of a comparison of a deep learning system with established clinical guidelines across outcomes from clinical trials, and performance comparisons for individual clinical trial subsets of the test set, with Figs. 3D-3H including the entire test set shown in Fig. 3A. [Figure 3E] 3A-3H show an example of a comparison of a deep learning system with established clinical guidelines across outcomes from clinical trials, and performance comparisons for individual clinical trial subsets of the test set, with Figs. 3D-3H including the entire test set shown in Fig. 3A. [Figure 3F] 3A-3H show an example of a comparison of a deep learning system with established clinical guidelines across outcomes from clinical trials, and performance comparisons for individual clinical trial subsets of the test set, with Figs. 3D-3H including the entire test set shown in Fig. 3A. [Figure 3G]3A-3H show an example of a comparison of a deep learning system with established clinical guidelines across outcomes from clinical trials, and performance comparisons for individual clinical trial subsets of the test set, with Figs. 3D-3H including the entire test set shown in Fig. 3A. [Figure 3H] 3A-3H show an example of a comparison of a deep learning system with established clinical guidelines across outcomes from clinical trials, and performance comparisons for individual clinical trial subsets of the test set, with Figs. 3D-3H including the entire test set shown in Fig. 3A. [Figure 4]

[0017] Figure 1 shows an example of a pathologist's interpretation of SSL tissue clusters. A self-supervised model in a multimodal model is trained to identify whether an enhanced version of a small patch of tissue comes from the same original patch, without seeing the labels in the clinical data. Once trained, each image patch in a dataset of 10.05M image patches is passed through the model to extract a 128-dimensional feature vector, which is then clustered and visualized using the UMAP algorithm31. The pathologist is then asked to interpret the 20 image patches closest to each of the 25 cluster centers, with explanatory text provided beside the inset. For clarity, only 6 clusters (in color) are highlighted, while the remaining clusters are shown in gray. See Figure 7 for the complete pathologist annotation. [Diagram 5]

[0018] Figure 1 shows an example image quilt for four exemplary patients. The dataset includes patients with a variable number of histopathology slides. To standardize the image input to the model, we segment the tissue from each slide and stitch all tissues into a single square image of 51200x51200 pixels, divided into 200x200 patches, to represent all the histopathology data for one patient. Image quilts for four patients are shown here. [Figure 6]

[0019] Figure 1 shows an example of kernel density sampling of example image patches. The brown boxes indicate the detection of kernels used to calculate kernel density, and oversample the patches input to the self-supervised training protocol according to kernel density. Each patch is partitioned into deciles according to density, and each decile is oversampled so that the MMAI model sees the same total number of images from each decile. [Figure 7-1]

[0020] Figure 1 shows an example of pathologist-interpreted patch clusters. Using UMAP, 25 clusters have been generated from the SSL features of all pathological tissue patches in study RTOG-9202. Each row in the image corresponds to the 25 image patches closest to the cluster center point. These were then inspected by a pathologist to determine a human-interpretable description of the clusters listed in the table. [Figure 7-2] Same as above. [Figure 8]

[0021] FIG. 1 shows an example of an NCCN model algorithm. The rules-based algorithm is based on D'Amico risk groups and models the guidelines published annually by the National Cancer Center Network. [Figure 9]

[0022] FIG. 1 is a schematic representation of a multimodal AI system described herein. [Figure 10]

[0023] FIG. 1 is a flow diagram depicting clinical trial pooling for testing and development of the models described herein. [Figure 11]

[0024] 1 is a table summarizing patient characteristics of the data analyzed by the models described herein. [Figure 12]

[0025] Distribution of MMAI scores determined by the MMAI model described herein for distant metastasis (DM) and prostate cancer-specific mortality (PCSM) among racial subgroups in the testing cohort (upper panel) and development cohort (lower panel). [Figure 13]

[0026] Table showing MMAI scores by racial subgroup in the development and testing cohorts. [Figure 14]

[0027] FIG. 13 summarizes the MMAI model scores determined by the MMAI model described herein by racial subgroups in the training and testing cohorts. [Figure 15A]

[0028] Figure 1 shows subdistribution hazard ratio (HR) results from Fine & Gray regression models in racial subgroups for distant metastasis (DM) MMAI and prostate cancer-specific mortality (PCSM) MMAI in the development and testing cohorts. Figure 2 shows DM results for the testing cohort. [Figure 15B] Figure 1 shows subdistribution hazard ratio (HR) results from Fine & Gray regression models in racial subgroups for distant metastasis (DM) MMAI and prostate cancer-specific mortality (PCSM) MMAI in the development and testing cohorts. Figure 2 shows DM results for the development cohort. [Figure 15C] Figure 1 shows subdistribution hazard ratio (HR) results from Fine & Gray regression models in racial subgroups for distant metastasis (DM) MMAI and prostate cancer-specific mortality (PCSM) MMAI in the development and testing cohorts. Figure 2 shows PCSM results for the testing cohort. [Figure 15D] Figure 1 shows subdistribution hazard ratio (HR) results from Fine & Gray regression models in racial subgroups for distant metastasis (DM) MMAI and prostate cancer-specific mortality (PCSM) MMAI in the development and testing cohorts. Figure 2 shows DM results for the development cohort. [Figure 16]

[0029] Figure 1 shows the results of subdistribution hazard ratio (HR) from Fine & Gray regression model in the racial subgroups of the MMAI model described herein.The HRs of 5-year biochemical failure (BF5yr MMAI), 10-year BF (BF10yr MMAI), 5-year distant metastasis (DM5yr MMAI), 10-year DM (DM10yr MMAI), 10-year prostate cancer-specific mortality (PCSM10yr MMAI), and 10-year overall survival (OS10yr MMAI) are shown in the test cohort. [Figure 17]

[0030] FIG. 1 shows subdistribution hazard ratio (HR) results from Fine & Gray regression models in racial subgroups of the MMAI model described herein for DM5-yrMMAI (panel a) and PCSM10-yrMMAI (panel b) in tabular form for the test and training cohorts. [Figure 18A]

[0031] FIG. 1 shows estimated risk / cumulative incidence curves by racial subgroups for DM in the entire cohort. [Figure 18B] Figure 1. Estimated risk / cumulative incidence curves for PCSM by racial subgroup in the entire cohort. [Figure 19A]

[0032] FIG. 1 shows risk stratification of the MMAI model (DM MMAI) within racial subgroups in the development, testing, and total cohorts. [Figure 19B]

[0033] FIG. 1 shows risk stratification of the MMAI model (PCSM) MMAI within racial subgroups in the development cohort, testing cohort, and total cohort. [Figure 20A]

[0034] FIG. 1 shows risk stratification of the MMAI model (DM MMAI) within racial subgroups in the development, testing, and total cohorts. [Figure 20B]

[0035] FIG. 1 shows risk stratification of the MMAI model (PCSM) MMAI within racial subgroups in the development cohort, testing cohort, and total cohort. [Figure 21]

[0036] 1 shows the cumulative incidence curves of distant metastasis (DM) in a cohort of prostate cancer patients. [Figure 22]

[0037] 1 is a table summarizing patient characteristics of risk-stratified cohorts predicted by the artificial intelligence model described herein. [Figure 23]

[0038] Table showing differential risk stratification of the same patient cohort using National Comprehensive Cancer Network (NCCN) risk stratification and multimodal artificial intelligence risk stratification. [Figure 24]

[0039] FIG. 1 shows MMAI predicted risk of distant metastasis after 10 years (DM10-yr) for a cohort of patients compared with NCCN classification. [Figure 25A]

[0040] 1 is a graphical representation of differential stratification of patient cohorts by NCCN and methods disclosed herein. [Figure 25B] 1 is a graphical representation of differential stratification of patient cohorts by NCCN and methods disclosed herein. [Figure 26]

[0041] Patient flow diagram for study enrollment from the parent clinical trial NRG / RTOG9902. H&E=hematoxylin and eosin, MMAI=multimodal artificial intelligence, DPEP=digital pathology evaluable population, RT=radiotherapy, AS=androgen suppression, CT=chemotherapy. [Figure 27A]

[0042] FIG. 1 shows population characteristics of participants in the parent clinical trial NRG / RTOG9902. [Figure 27B] FIG. 1 shows MMAI scores between treatment groups for a population of individuals in NRG / RTOG9902. [Figure 28A]

[0043] FIG. 1 is a table showing univariate analysis of the association of the MMAI algorithm with DM and PCSM endpoints. [Figure 28B]1 is a table showing multivariate analysis of the association of the MMAI algorithm with DM and PCSM endpoints while adjusting for individual clinical risk factors. [Figure 29A]

[0044] 1 is a table showing the prognostic performance of MMAI for distant metastasis (DM). [Figure 29B] 1 is a table showing the prognostic performance of MMAI for prostate cancer-specific mortality (PCSM) within subgroup classifications. [Figure 30A]

[0045] 1 is a table showing the multivariate analysis of the MMAI algorithm for PM after adjusting for all clinical risk factors. [Figure 30B] 1 is a table showing the multivariate analysis of the MMAI algorithm for PCSM after adjusting for all clinical risk factors. [Figure 31A]

[0046] 1 is a table showing multivariate analysis of the DM prognostic MMAI algorithm for BF, CSM, and OS. [Figure 31B] 1 is a table showing multivariate analysis of the PCSM prognostic MMAI algorithm for BF, CSM, and OS. [Figure 32A]

[0047] Figure 1 shows cumulative incidence curves of estimated distant metastasis (DM) risk by quartile 4 vs. quartiles 1-3, as predicted by DM-optimized multimodal artificial intelligence (DM MMAI). [Figure 32B] Figure 1 shows cumulative incidence curves of estimated prostate cancer-specific mortality risk (PCSM) by quartile 4 vs. quartiles 1-3 predicted by PCSM-optimized multimodal artificial intelligence (PCSM MMAI). [Figure 33A]

[0048] FIG. 1 shows cumulative incidence curves of estimated distant metastasis (DM) risk by quartile 4 vs. quartiles 1-3 DM MMAI by treatment group. [Figure 33B] FIG. 1 shows cumulative incidence curves of estimated prostate cancer-specific mortality risk (PCSM) by quartile 4 vs. quartiles 1-3 PCSM MMAI by treatment group. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015]

[0049] While various embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions will occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed.

[0016]

[0050] As used in this specification and claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. For example, the term "nucleic acid" includes a plurality of nucleic acids, including mixtures thereof.

[0017]

[0051] As used herein, the term "subject" generally refers to an entity or medium having genetic information that can be tested or detected. The subject can be a human, an individual, or a patient. The subject can be a vertebrate, such as a mammal. Non-limiting examples of mammals include humans, monkeys, farm animals, sport animals, rodents, pets, and the like. The subject can be a male subject. The subject can be a female subject. The subject may exhibit symptoms indicative of the subject's health or physiological state or condition, such as the subject's cancer-related health or physiological state or condition. Alternatively, the subject may be asymptomatic with respect to such health or physiological state or condition. The subject may be suspected of having a health or physiological state or condition. The subject may be at risk for developing a health or physiological state or condition. The health or physiological condition may correspond to a disease (e.g., cancer). The subject may be an individual diagnosed with a disease. The subject may be an individual at risk for developing a disease.

[0018]

[0052] As used herein, "cancer diagnosis," "diagnosing cancer," and related or derived terms include identifying cancer in a subject, determining the aggressiveness of cancer, or determining the stage of cancer.

[0019]

[0053] As used herein, "cancer prognosis," "determining a cancer prognosis," and related or derived terms include predicting a patient's clinical outcome, assessing the risk of cancer recurrence, determining treatment modality, or determining treatment efficacy.

[0020]

[0054] As used herein, the term "nucleic acid" generally refers to a polymer of nucleotides of any length, deoxyribonucleotides (dNTPs) or ribonucleotides (rNTPs), or analogs thereof. Nucleic acids may have any three-dimensional structure and may perform any function, known or unknown. Non-limiting examples of nucleic acids include deoxyribonucleic acid (DNA), ribonucleic acid (RNA), coding or non-coding regions of a gene or gene fragment, loci defined from linkage analysis, exons, introns, messenger RNA (mRNA), transfer RNA, ribosomal RNA, short interfering RNA (siRNA), short hairpin RNA (shRNA), microRNA (miRNA), ribozymes, cDNA, recombinant nucleic acids, branched nucleic acids, plasmids, vectors, isolated DNA of any sequence, isolated RNA of any sequence, nucleic acid probes, and primers. Nucleic acids may contain one or more modified nucleotides, such as methylated nucleotides and nucleotide analogs. If present, modifications to the nucleotide structure may occur before or after assembly of the nucleic acid. The sequence of nucleotides of a nucleic acid may be interrupted by non-nucleotide components. Nucleic acids can be further modified after polymerization, such as by conjugation or conjugation with a reporter agent.

[0021]

[0055] As used herein, the term "target nucleic acid" generally refers to a nucleic acid molecule in a starting population of nucleic acid molecules having a nucleotide sequence, the presence, amount, and / or sequence, or one or more changes therein, of which it is desired to determine. The target nucleic acid may be any type of nucleic acid, including DNA, RNA, and analogs thereof. As used herein, "target ribonucleic acid (RNA)" generally refers to a target nucleic acid that is RNA. As used herein, "target deoxyribonucleic acid (DNA)" generally refers to a target nucleic acid that is DNA.

[0022]

[0056] As used herein, the terms "amplifying" and "amplification" generally refer to increasing the size or amount of a nucleic acid molecule. The nucleic acid molecule may be single-stranded or double-stranded. Amplification may include generating one or more copies of the nucleic acid molecule or "amplification products." Amplification may be performed, for example, by extension (e.g., primer extension) or ligation. Amplification may include performing a primer extension reaction to generate a strand complementary to the single-stranded nucleic acid molecule, and optionally generating one or more copies of the strand and / or the single-stranded nucleic acid molecule. The term "DNA amplification" generally refers to generating one or more copies of a DNA molecule or "amplified DNA product." The term "reverse transcription amplification" generally refers to generating deoxyribonucleic acid (DNA) from a ribonucleic acid (RNA) template by the action of reverse transcriptase.

[0023] Embodiments of the present disclosure

[0024]

[0057] Despite its high prevalence, accurate, sensitive and specific diagnosis of prostate cancer remains challenging. Although prostate cancer is often indolent and may be cured by treatment, the adverse effects of overtreatment and undertreatment make it the leading cause of cancer-related disability worldwide and remains one of the leading causes of cancer deaths in men. Determining the optimal course of treatment for a patient with prostate cancer is a challenging medical task that requires consideration of the patient's overall health, characteristics of the patient's cancer, the side-effect profile of many possible treatments, outcome data from clinical trials involving similarly diagnosed patients, and prognosticating the expected future outcome of the patient at hand. This difficulty is exacerbated by the lack of readily accessible prognostic tools to better risk stratify patients.

[0025]

[0058] Artificial intelligence (AI) is enabling insights to be gained from huge data sets that were previously difficult to interpret. Whereas standard risk stratification tools are fixed and based on a small number of variables, AI can learn from large amounts of minimally processed data across a range of modalities. AI systems are low-cost, massively scalable, and can improve over time through use.

[0026]

[0059] There is a strong need for accurate, globally scalable tools to aid in the personalization of treatment. The methods and systems disclosed herein demonstrate the personalization of prostate cancer treatment by predicting long-term, clinically relevant outcomes (e.g., distant metastasis, biochemical recurrence, partial response, complete response, death, relative survival, cancer-specific survival, progression-free survival, disease-free survival, 5-year survival, and overall survival) using novel multimodal deep learning models trained on digital histopathology and clinical data of prostate biopsies.

[0027]

[0060] The present disclosure provides methods, systems, and kits for identifying or monitoring cancer-related categories and / or conditions by processing a biological sample obtained from or derived from a subject (e.g., a male patient suffering from or suspected of suffering from prostate cancer). A biological sample obtained from a subject (e.g., a prostate biopsy sample) can be analyzed to identify a cancer-related category (which can include, for example, determining the presence or absence of a cancer-related category, or a quantitative assessment (e.g., risk, predicted outcome)). Such subjects may include subjects with one or more cancer-related categories and subjects without a cancer-related category. A cancer-related category or condition can include, for example, cancer positive, cancer negative, cancer stage, predicted response to cancer treatment, and / or predicted long-term outcome (e.g., disease metastasis, biochemical recurrence, partial response, complete response, relative survival, cancer-specific survival, progression-free survival, disease-free survival, 5-year survival, or overall survival).

[0028] Assay of biological samples

[0029]

[0061] Biological samples can be obtained from or derived from human subjects (e.g., male subjects). Biological samples can be stored before processing at various temperatures (e.g., room temperature, under refrigerated or freezer conditions, 25°C, 4°C, -18°C, -20°C, or -80°C), or in various storage conditions, such as various suspensions (e.g., formalin, EDTA collection tube, cell-free RNA collection tube, or cell-free DNA collection tube). Biological samples can be obtained from subjects with or suspected of having cancer (e.g., prostate cancer), or from subjects without or not suspected of having cancer.

[0030]

[0062] The biological sample can be used to diagnose, detect or identify a disease or health or physiological condition in a subject by analyzing the biological sample. The biological sample or a portion thereof can be analyzed to determine the likelihood that the sample is positive for a disease or health condition (e.g., prostate cancer). Alternatively, or in addition, the methods described herein can include diagnosing a subject with a disease or health condition, monitoring a disease or health condition in a subject, and / or determining a subject's propensity for a health disease / condition. In some embodiments, the biological sample can be used to classify the sample and / or the subject into a cancer-related category and / or to identify the subject as having a particular cancer-related condition. The cancer-related category or condition can include a diagnosis (e.g., positive or negative for cancer), a particular type of cancer (e.g., prostate cancer), a stage of cancer, a predicted outcome or prognosis, a predicted response to one or more treatments, or a combination thereof.

[0031]

[0063] The measurable substance can be a source of a sample. The substance can be a fluid, such as a biological fluid. Fluid substances can include blood (e.g., whole blood, plasma, serum), umbilical cord blood, saliva, urine, sweat, serum, semen, vaginal fluid, gastric and digestive fluids, cerebrospinal fluid, placental fluid, cavity fluid, ocular fluid, serum, breast milk, lymphatic fluid, or combinations thereof.

[0032]

[0064] The material may be a solid, for example a biological tissue. The material may include normal healthy tissue. The tissue may be associated with various types of organs. Non-limiting examples of organs may include the brain, breast, liver, lung, kidney, prostate, ovary, spleen, lymph nodes (including tonsils), thyroid, pancreas, heart, skeletal muscle, intestine, larynx, esophagus, stomach, or combinations thereof.

[0033]

[0065] The material may include a tumor. The tumor may be benign (non-cancerous), pre-malignant, or malignant (cancer), or a metastasis thereof. Non-limiting examples of tumors and associated cancers include acoustic neuroma, acute lymphoblastic leukemia, acute myeloid leukemia, adenocarcinoma, adrenal cortical carcinoma, AIDS-related cancer, AIDS-related lymphoma, anal cancer, angiosarcoma, appendix cancer, astrocytoma, basal cell carcinoma, bile duct cancer, bladder cancer, bone cancer, brain tumors, such as cerebellar astrocytoma, cerebral astrocytoma / malignant glioma, ependymoma, medulloblastoma, supratentorial primitive neuroectodermal tumor, visual pathway and hypothalamic glioma, breast cancer, bronchial adenoma, Burkitt's lymphoma, cancer of unknown primary, central nervous system lymphoma, bronchogenic Cancer, cerebellar astrocytoma, cervical cancer, childhood cancer, chondrosarcoma, chordoma, choriocarcinoma, chronic lymphocytic leukemia, chronic myeloid leukemia, chronic myeloproliferative disease, colon cancer, craniopharyngioma, cutaneous T-cell lymphoma, cystadenocarcinoma, detumorable small round cell tumor, germ cell cancer, endocrine system cancer, endometrial cancer, endothelial sarcoma, ependymoma, epithelial cancer, esophageal cancer, Ewing's sarcoma, fibrosarcoma, germ cell tumor, gallbladder cancer, stomach cancer, gastrointestinal carcinoid tumor, gastrointestinal stromal tumor, digestive system cancer, genitourinary system cancer, glioma, hairy cell leukemia, head and neck cancer, heart cancer, hemangioblastoma, hepatocellular (liver) cancer, Hodgkin's lymphoma, hypopharyngeal cancer, intraocular melanoma, pancreatic islet cell carcinoma, Kaposi's sarcoma, kidney cancer, laryngeal cancer, leiomyosarcoma, lip and oral cavity cancer, liposarcoma, liver cancer, lung cancer, including non-small cell lung cancer and small cell lung cancer, carcinoma, lymphangiosarcoma, lymphangioendothelioma, lymphoma, leukemia, macroglobulinemia, bone malignant fibrous histiocytoma / osteosarcoma, medulloblastoma, medullary carcinoma, melanoma, meningioma, mesothelioma, metastatic squamous cell neck cancer of unknown primary, oral cancer, multiple endometrial cancer Myeloid tumor syndrome, myelodysplastic syndrome, myeloid leukemia, myxosarcoma, nasal and paranasal sinus cancer, nasopharyngeal cancer, neuroblastoma, non-Hodgkin's lymphoma, non-small cell lung cancer, oligodendroglioma, oral cavity cancer, oropharyngeal cancer, osteosarcoma / malignant fibrous histiocytoma of bone, ovarian cancer, ovarian epithelial cancer, ovarian germ cell tumor, pancreatic cancer, pancreatic islet cell cancer, papillary adenocarcinoma, papillary carcinoma, paranasal sinus and nasal cancer, parathyroid cancer, penile cancer, pharyngeal cancer, pheochromocytoma, pineal astrocytoma, pineal germinoma, pituitary adenoma, pleuropulmonary blastoma, plasma cell neoplasm, primary central nervous system lymphoma, prostate cancer,Rectal cancer, renal cell carcinoma, renal pelvis and ureter transitional cell carcinoma, retinoblastoma, rhabdomyosarcoma, salivary gland cancer, sarcoma, sebaceous gland carcinoma, seminoma, skin cancer, cutaneous Merkel cell carcinoma, small intestine cancer, soft tissue sarcoma, squamous cell carcinoma, gastric cancer, sweat gland carcinoma, synovium, T-cell lymphoma, testicular tumor, pharyngeal cancer, thymoma, thymic carcinoma, thyroid cancer, trophoblastic tumor (gestational), cancer of unknown primary site, urethral cancer, uterine sarcoma, vaginal cancer, vulvar cancer, Waldenstrom's macroglobulinemia, Wilms' tumor, or combinations thereof. The tumors may be associated with various types of organs. Non-limiting examples of organs may include brain, breast, liver, lung, kidney, prostate, ovary, spleen, lymph nodes (including tonsils), thyroid, pancreas, heart, skeletal muscle, intestine, larynx, esophagus, stomach, or combinations thereof.

[0034]

[0066] The material may include a mixture of normal healthy tissue or tumor tissue. The tissue may be associated with various types of organs. Non-limiting examples of organs may include brain, breast, liver, lung, kidney, prostate, ovary, spleen, lymph nodes (including tonsils), thyroid, pancreas, heart, skeletal muscle, intestine, larynx, esophagus, stomach, or combinations thereof. In some embodiments, the tissue is associated with the prostate of a subject. In the case of biological samples (e.g., biopsy samples) that include cells and / or tissues, the biological samples may be further analyzed or assayed. In some embodiments, the biopsy samples may be fixed, processed (e.g., dehydrated), embedded, frozen, stained, and / or examined under a microscope. In some embodiments, digital slides are generated from the processed samples.

[0035]

[0067] In some embodiments, the material may include a variety of cells, including eukaryotic cells, prokaryotic cells, fungal cells, cardiac cells, lung cells, kidney cells, liver cells, pancreatic cells, germ cells, stem cells, induced pluripotent stem cells, gastrointestinal cells, blood cells, cancer cells, bacterial cells, bacterial cells isolated from a human microbiome sample, and cells circulating in human blood. In some embodiments, the material may include the contents of a cell, e.g., the contents of a single cell, or the contents of multiple cells.

[0036]

[0068] In some embodiments, a substance may include one or more markers, the presence or absence of which is indicative of some phenomenon, such as a disease, disorder, infection, or environmental exposure. A marker may be, for example, a cell, a small molecule, a macromolecule, a protein, a glycoprotein, a carbohydrate, a sugar, a polypeptide, a nucleic acid (e.g., deoxyribonucleic acid (DNA), ribonucleic acid (RNA)), a cell-free nucleic acid (e.g., cf-DNA, cf-RNA), a lipid, a cellular component, or a combination thereof.

[0037]

[0069] Biological samples can be taken before and / or after treatment of a subject with cancer. Biological samples can be obtained from a subject during treatment or treatment regimen. Multiple biological samples may be taken from a subject to monitor the effectiveness of treatment over time. Biological samples can be taken from subjects known to have or suspected to have cancer (e.g., prostate cancer). Biological samples can be taken from subjects with unexplained symptoms, such as fatigue, nausea, weight loss, aches and pains, weakness, or bleeding. Biological samples can be taken from subjects with explainable symptoms. Biological samples can be taken from subjects at risk for developing cancer due to factors such as family history, age, hypertension or prehypertension, diabetes or prediabetes, overweight or obesity, environmental exposures, lifestyle risk factors (e.g., smoking, alcohol, or drug use), or the presence of other risk factors.

[0038]

[0070] After obtaining a biological sample from a subject, the biological sample can be processed to generate a data set indicative of a disease, condition, cancer-related category, or health status of the subject. For example, a tissue sample can be subjected to a histopathological assay (e.g., microscopy including digital image acquisition such as whole slide imaging) to generate image data based on the biological sample. Alternatively, a liquid sample or markers isolated from the sample can be subjected to testing (e.g., clinical laboratory tests) to generate tabular data. In some embodiments, the sample is assayed for the presence, absence, or amount of one or more metabolites (e.g., prostate specific antigen (PSA)).

[0039] Data Type

[0040]

[0071] The methods and systems described herein take as input one or more datasets. The one or more datasets may include tabular data and / or image data. The tabular data and / or image data may be obtained from a biological sample of a subject. In some embodiments, the data is not derived from a biological sample.

[0041]

[0072] The data may include images of tissue samples taken from a biopsy of a subject. The image data may be obtained by microscopy of the biopsy sample. The microscopy may consist of optical microscopy, virtual or digital microscopy (such as whole slide imaging (WSI)), or any suitable microscopy technique known in the art. The microscopy images may be subjected to one or more processing steps, such as filtering, segmentation, concatenation, or object detection.

[0042]

[0073] The tabular data described herein may include any non-image data related to the subject's health or condition (e.g., disease). Tabular data may include clinical data such as: laboratory data (e.g., prostate serum antigen (PSA) level), qualitative measures of cytopathology (e.g., Gleason grade, Gleason score), structured or unstructured health data (e.g., digital rectal exam results), medical image data or results (e.g., X-ray, computed tomography (CT) scan, magnetic resonance imaging (MRI) scan, positron emission tomography (PET) scan, or ultrasound results, e.g., transrectal ultrasound results), age, medical history, previous or current cancer status (e.g., remission, metastasis) or stage, current or previous therapeutic interventions, long-term outcome, and / or National Comprehensive Cancer Network (NCCN) classification or components thereof (e.g., combined Gleason score, t-stage, baseline PSA) at one or more time points.

[0043]

[0074] In some embodiments, the therapeutic intervention may include radiation therapy (RT). In some embodiments, the therapeutic intervention may include chemotherapy. In some embodiments, the therapeutic intervention may include surgical intervention. In some embodiments, the therapeutic intervention may include immunotherapy. In some embodiments, the therapeutic intervention may include hormonal therapy. In some embodiments, the RT may include RT with a preset use of short-term androgen deprivation therapy (ST-ADT). In some embodiments, the RT may include RT with a preset use of long-term ADT (LT-ADT). In some embodiments, the RT may include RT with a preset use of dose-escalated RT (DE-RT). In some embodiments, the surgical intervention may include radical prostatectomy (RP). In some embodiments, the therapeutic intervention may include any combination of the therapeutic interventions disclosed herein. In some embodiments, the long-term outcome may include distant metastasis (DM). In some embodiments, the long-term outcome may include biochemical recurrence (BR). In some embodiments, the long-term outcome may include partial response. In some embodiments, the long-term outcome may include complete response. In some embodiments, the long term outcome may include death. In some embodiments, the long term outcome may include relative survival. In some embodiments, the long term outcome may include cancer specific survival. In some embodiments, the cancer specific survival may include prostate cancer specific survival (PCaSS). In some embodiments, the long term outcome may include progression free survival. In some embodiments, the long term outcome may include disease free survival. In some embodiments, the long term outcome may include 5 year survival. In some embodiments, the long term outcome may include overall survival (OS). In some embodiments, the long term outcome may include any combination of the long term outcomes disclosed herein.

[0044]

[0075] Data used in the methods and systems described herein may be subjected to one or more processing steps. In some embodiments, the data (e.g., image data) is subjected to image processing, image segmentation, and / or object detection processes, coded as image processing, image segmentation, or object detection algorithms. Image processing procedures may filter, translate, scale, rotate, mirror, shear, combine, compress, split, concatenate, extract features from images, and / or smooth images before downstream processing. In some embodiments, multiple images (e.g., histopathology slides) are combined to form an image quilt. The image quilt may be converted to a representation (e.g., tensor) useful for downstream processing of the image data. Image segmentation processing may partition an image into one or more segments that contain a factor or region of interest. For example, an image segmentation algorithm may process a digital histopathology slide to determine regions of tissue that are distinct from regions of blanks or artifacts. In some embodiments, the image segmentation algorithm may include machine learning or artificial intelligence algorithms. In some embodiments, image segmentation may precede image processing. In some embodiments, image processing may precede image segmentation. The object detection process may include detecting the presence or absence of a target object (e.g., a cell or a cell part, such as a nucleus). In some embodiments, object detection may precede image processing and / or image segmentation. For example, images found by an image detection algorithm to contain one or more objects of interest may be concatenated in a subsequent image processing step. Alternatively, or in addition, image processing may precede object detection and / or image segmentation. For example, raw image data may be processed (e.g., filtered) and the processed image data may be subjected to an object detection algorithm. The image data may be subjected to multiple image processing, image segmentation, and / or object detection steps in any suitable order. In one example, the image data is optionally subjected to one or more image processing steps to improve image quality.The processed images are then run through an image segmentation algorithm to detect regions of interest (e.g., regions of tissue in a set of histopathology slides). The regions of interest are then run through an object detection algorithm (e.g., an algorithm to detect nuclei in an image of tissue) and regions found to have at least one target object are linked to produce processed image data for downstream use.

[0045]

[0076] In some embodiments, the data (e.g., tabular data) may be subjected to one or more processing steps. Processing steps may include, but are not limited to, standardization, or normalization. The one or more processing steps may, for example, discard data that contains non-true values ​​or that contains very few observations. The one or more processing steps may additionally or alternatively standardize the encoding of the data values. Different input data sets may have identical parameter values ​​that are encoded differently depending on the source of the data set. For example, "900", "900.0", "904", "904.0", "-1", "-1.0", "None", "NaN" may all encode "missing" parameter values. The one or more processing steps may recognize variations in encoding for the same value and standardize the data sets to have a uniform encoding for a given parameter value. Thus, the processing steps may reduce irregularities in the input data for downstream use. The one or more data sets may normalize the parameter values. For example, numerical data may be scaled, whitened, colored, embellished, or standardized. For example, the data may be scaled or shifted to fall into a particular interval (e.g., [0,1] or [-1,1]) and / or correlated. In some embodiments, categorical data may be encoded as one-hot vectors. In some embodiments, one or more different types of tabular (e.g., numerical, categorical) data may be concatenated. In some embodiments, the data is not subjected to processing steps.

[0046]

[0077] Data may be obtained at one or more time points. In some embodiments, data is obtained at an initial time point and a later time point. The initial time point and the later time point may be separated by any suitable length of time, such as 1 hour, 1 day, 1 week, 2 weeks, 3 weeks, 4 weeks, 6 weeks, 12 weeks, 4 months, 5 months, 6 months, 7 months, 8 months, 9 months, 10 months, 11 months, 1 year, 2 years, 3 years, 4 years, 5 years, 6 years, 7 years, 8 years, 9 years, 10 years, or more. In some embodiments, data is from three or more time points. In some embodiments, data is from 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or more time points.

[0047] Pre-Trained Algorithms

[0048]

[0078] After using one or more assays to process one or more biological samples from a subject to generate one or more datasets indicative of the subject's cancer status (e.g., one or more cancer-related categories), a trained algorithm can be used to process one or more of the datasets (e.g., visual data and / or tabular data) to determine the subject's cancer status. For example, a trained algorithm can be used to determine the presence or absence of (e.g., prostate) cancer in a subject based on imaging data and / or laboratory data. The trained algorithm can be configured to identify the cancer status with an accuracy of at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 85%, at least about 90%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99%, or greater than 99% for at least about 25, at least about 50, at least about 100, at least about 150, at least about 200, at least about 250, at least about 300, at least about 350, at least about 400, at least about 450, at least about 500, or greater than about 500 independent samples.

[0049]

[0079] The trained algorithm may include an unsupervised machine learning algorithm. The trained algorithm may include a supervised machine learning algorithm. The trained algorithm may include a classification and regression tree (CART) algorithm. The supervised machine learning algorithm may include, for example, a random forest, a support vector machine (SVM), a neural network, or a deep learning algorithm. The trained algorithm may include a self-supervised machine learning algorithm.

[0050]

[0080] In some embodiments, the machine learning algorithm of the method or system described herein utilizes one or more neural networks. In some cases, a neural network is a type of computational system that can learn relationships between input data sets and target data sets. A neural network may be a software representation of a human nervous system (e.g., a cognitive system) and is intended to capture the ability of "learning" and "generalization" as used by humans. In some embodiments, the machine learning algorithm includes a neural network consisting of a CNN. Non-limiting examples of structural components of the machine learning algorithm described herein include the following: CNN, recurrent neural network, dilated CNN, fully connected neural network, deep generative model, Boltzmann machine.

[0051]

[0081] In some embodiments, a neural network is composed of a series of layers called "neurons." In some embodiments, a neural network is composed of an input layer, where data is presented, one or more internal and / or "hidden" layers, and an output layer. The neurons are connected to neurons in other layers via connections with weights, which are parameters that control the strength of the connections. The number of neurons in each layer may be related to the complexity of the problem to be solved. The minimum number of neurons required in a layer may be determined by the complexity of the problem, and the maximum number may be limited by the neural network's ability to generalize. The input neurons may receive the data presented to them and send the data to a first hidden layer through connection weights that are modified during training. The first hidden layer may process the data and send the results to the next layer through a second set of weighted connections. Each subsequent layer may "pool" the results from the previous layer into more complex relationships. In addition, whereas traditional software programs require specific instructions to be written to perform a function, a neural network is programmed by training itself with a known set of examples and being able to modify itself during (and after) training to provide a desired output, such as an output value. After training, when a neural network is presented with new input data, it is configured to generalize what it "learned" during training and apply what it learned in training to new, previously unseen input data in order to generate an output that is related to that input.

[0052]

[0082] In some embodiments, the neural network is comprised of an artificial neural network (ANN). An ANN may be a machine learning algorithm that can be trained to map an input data set to an output data set, and includes an interconnected group of nodes organized into multiple layers of nodes. For example, an ANN architecture may be comprised of at least one input layer, one or more hidden layers, and one output layer. An ANN may include any total number of layers, and any number of hidden layers, where the hidden layers act as trainable feature extractors that allow for mapping a set of input data to an output value or a set of output values. As used herein, a deep learning algorithm (such as a deep neural network (DNN)) is an ANN that includes multiple hidden layers, e.g., two or more hidden layers. Each layer of a neural network may include a number of nodes (i.e., "neurons"). A node receives inputs directly from the input data or from the output of a node of a previous layer, and performs a particular operation, e.g., a summation operation. The connections from the inputs to the nodes have weights (or weighting coefficients) associated with them. A node may add up the pairwise products of all inputs with their associated weights. The weighted sum may be offset with a bias. The output of a node or neuron may be gated using a threshold or activation function. The activation function may be a linear or nonlinear function. The activation function may be, for example, a rectified linear unit (ReLU) activation function, a leaky ReLU activation function, or other functions such as a saturated hyperbolic tangent, identity, binary step, logistic, arctan, softsign, parametric rectified linear unit, exponential linear unit, softplus, bending identity, softexponential, sinusoid, sinc, Gaussian, or sigmoid function, or a combination thereof.

[0053]

[0083] The weighting coefficients, bias values, and thresholds, or other computational parameters of a neural network may be "taught" or "learned" in a training phase that uses one or more sets of training data. For example, the parameters may be trained using input data from a training data set and gradient descent or backpropagation techniques such that the output values ​​that the ANN computes are consistent with the examples contained in the training data set.

[0054]

[0084] The number of nodes used in the input layer of an ANN or DNN may be at least about 10, 50, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10,000, 20,000, 30,000, 40,000, 50,000, 60,000, 70,000, 80,000, 90,000, 100,000, or more. In other examples, the number of nodes used in the input layer may be at most about 100,000, 90,000, 80,000, 70,000, 60,000, 50,000, 40,000, 30,000, 20,000, 10,000, 9000, 8000, 7000, 6000, 5000, 4000, 3000, 2000, 1000, 900, 800, 700, 600, 500, 400, 300, 200, 100, 50, 10, or fewer. In some examples, the total number of layers (including the input layer and the output layer) used in an ANN or DNN may be at least about 3, 4, 5, 10, 15, 20, or more. In other examples, the total number of layers may be at most about 20, 15, 10, 5, 4, 3, or less.

[0055]

[0085] In some examples, the total number of learnable or trainable parameters, e.g., weighting coefficients, biases, or thresholds used in an ANN or DNN may be at least about 10, 50, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10,000, 20,000, 30,000, 40,000, 50,000, 60,000, 70,000, 80,000, 90,000, 100,000, or more. In other examples, the number of learnable parameters may be at most about 100,000, 90,000, 80,000, 70,000, 60,000, 50,000, 40,000, 30,000, 20,000, 10,000, 9000, 8000, 7000, 6000, 5000, 4000, 3000, 2000, 1000, 900, 800, 700, 600, 500, 400, 300, 200, 100, 50, 10, or fewer.

[0056]

[0086] In some embodiments of the machine learning algorithms described herein, the machine learning algorithm includes a neural network, such as a deep CNN. In some embodiments where a CNN is used, the network is constructed with any number of convolutional layers, dilation layers, or fully connected layers. In some embodiments, the number of convolutional layers is between 1 and 10, and the number of dilation layers is between 0 and 10. The total number of convolutional layers (including input and output layers) may be at least about 1, 2, 3, 4, 5, 10, 15, 20, or more, and the total number of dilation layers may be at least about 1, 2, 3, 4, 5, 10, 15, 20, or more. The total number of convolutional layers may be at most about 20, 15, 10, 5, 4, 3, or less, and the total number of dilation layers may be at most about 20, 15, 10, 5, 4, 3, or less. In some embodiments, the number of convolutional layers is between 1 and 10, and the fully connected layers are between 0 and 10. The total number of convolutional layers (including input and output layers) may be at least about 1, 2, 3, 4, 5, 10, 15, 20, or more, and the total number of fully connected layers may be at least about 1, 2, 3, 4, 5, 10, 15, 20, or more. The total number of convolutional layers may be at most about 20, 15, 10, 5, 4, 3, 2, 1, or less, and the total number of fully connected layers may be at most about 20, 15, 10, 5, 4, 3, 2, 1, or less.

[0057]

[0087] In some embodiments, the machine learning algorithm comprises a neural network including a CNN, an RNN, an augmented CNN, a fully connected neural network, a deep generative model, and / or a deep restricted Boltzmann machine.

[0058]

[0088] In some embodiments, the machine learning algorithm is composed of one or more CNNs. The CNNs may be deep ANNs and feed-forward ANNs. CNNs may also be applied to the analysis of visual images. The CNNs may include an input layer, an output layer, and multiple hidden layers. The hidden layers of the CNNs may include convolutional layers, pooling layers, fully connected layers, and normalization layers. The layers may be organized in three dimensions: width, height, and depth.

[0059]

[0089] A convolutional layer applies a convolution operation to the input and can pass the result of the convolution operation to the next layer. When processing images, the convolution operation can reduce the number of free parameters, allowing the network to be deeper with fewer parameters. In a neural network, each neuron may receive input from several locations in the previous layer. In a convolutional layer, neurons may receive input only from limited subregions of the previous layer. The parameters of a convolutional layer may consist of a set of learnable filters (or kernels). The learnable filters may have small receptive fields and span the full depth of the input volume. During the forward pass, each filter may be convolved across the width and height of the input volume to compute the dot product between the entries of the filter and the input, generating a two-dimensional activation map for that filter. As a result, the network can learn filters that activate when it detects a certain type of feature at some spatial location in the input.

[0060]

[0090] In some embodiments, the machine learning algorithm is comprised of an RNN. An RNN is a neural network with periodic connections that can encode and process sequential data. The RNN can include an input layer configured to receive a sequence of inputs. The RNN can additionally include one or more hidden recurrent layers that maintain a state. At each step, the hidden recurrent layer can compute an output and a next state of the layer. The next state may depend on the previous state and the current input. The state can be maintained across steps, capturing dependencies in the input sequence.

[0061]

[0091] An RNN can be a long short-term memory (LSTM) network. An LSTM network can be made up of LSTM units. An LSTM unit can contain cells, input gates, output gates, and forget gates. A cell can be responsible for tracking dependencies between elements in an input sequence. An input gate can control the degree to which new values ​​flow into a cell, a forget gate can control the degree to which values ​​remain in a cell, and an output gate can control the degree to which values ​​in a cell are used to compute the output activation of the LSTM unit.

[0062]

[0092] Or an attention mechanism (e.g. a transformer). An attention mechanism focuses or "pays attention" to certain input regions while ignoring other regions. This can potentially improve the model's performance by making certain input regions less relevant. At each step, the attention unit can compute, among other operations, the dot product of a context vector with the input for that step. The output of the attention unit can define where the most relevant information is in the input sequence.

[0063]

[0093] In some embodiments, the pooling layer includes a global pooling layer, which can combine the outputs of neuron clusters in one layer to a single neuron in the next layer. For example, a max pooling layer can use the maximum value from each of the neuron clusters in the previous layer. An average pooling layer can use the average value from each of the neuron clusters in the previous layer.

[0064]

[0094] In some embodiments, a fully connected layer connects every neuron in one layer to every neuron in another layer. In a neural network, each neuron may receive input from several places in the previous layer. In a fully connected layer, each neuron can receive input from every element in the previous layer.

[0065]

[0095] In some embodiments, the normalization layer is a batch normalization layer. A batch normalization layer can improve the performance and stability of a neural network. A batch normalization layer can provide zero mean / unit variance inputs to any layer of a neural network. Advantages of using a batch normalization layer can include faster networks to train, faster learning rates, easier initialization of weights, more feasible activation functions, and a simplified process of creating deep networks.

[0066]

[0096] The trained algorithm can be configured to accept a plurality of input variables and generate one or more output values ​​based on the plurality of input variables. The plurality of input variables can consist of one or more data sets indicative of cancer-related categories. For example, the input variables can include microscopy images of a biopsy sample of the subject. The plurality of input variables can also include clinical health data of the subject.

[0067]

[0097] The trained algorithm may include a classifier such that each of the one or more output values ​​includes one of a fixed number of possible values ​​(e.g., a linear classifier, a logistic regression classifier, etc.) indicating the classification of the biological sample and / or subject by the classifier. The trained algorithm may include a binary classifier such that each of the one or more output values ​​includes one of two values ​​(e.g., {0,1}, {positive, negative}, or {high risk, low risk}) indicating the classification of the biological sample and / or subject by the classifier. The trained algorithm may be another type of classifier such that each of the one or more output values ​​includes one of three or more values ​​(e.g., {0,1,2}, {positive, negative, or indeterminate}, or {high risk, intermediate risk, or low risk}) indicating the classification of the biological sample and / or subject by the classifier. The output values ​​may include descriptive labels, numerical values, or combinations thereof. Some of the output values ​​may include descriptive labels. Such descriptive labels can provide an identification or indication of the subject's disease or disorder status, and may include, for example, positive, negative, high risk, medium risk, low risk, or indeterminate. Such descriptive labels can provide an identification of a treatment for the subject's cancer-related category, and may include, for example, a therapeutic intervention, a duration of the therapeutic intervention, and / or a dosage of the therapeutic intervention suitable for treating a subject falling into a particular cancer-related category.

[0068]

[0098] Some of the output values ​​may include numerical values, such as binary, integer, or continuous values. Such binary output values ​​may include, for example, {0,1}, {positive,negative}, or {high risk,low risk}. Such integer output values ​​may include, for example, {0,1,2}. Such continuous output values ​​may include, for example, probability values ​​at least 0 and less than or equal to 1. Such continuous output values ​​may include, for example, an unnormalized probability value of at least 0. Such continuous output values ​​may indicate a cancer-related category prognosis for the subject. Some of the numerical values ​​may be mapped to descriptive labels, for example, by mapping 1 to "positive" and 0 to "negative."

[0069]

[0099] A portion of the output values ​​may be assigned based on one or more cutoff values. For example, a binary classification of a sample may assign an output value of "positive" or 1 if the sample indicates that the subject has at least a 50% chance of having a cancer-associated condition (e.g., a type or stage of cancer) or belonging to a cancer-associated category. For example, a binary classification of a sample may assign an output value of "negative" or 0 if the sample indicates that the subject has less than a 50% chance of belonging to a cancer-associated category. In this instance, a single cutoff value of 50% is used to classify the sample into one of two possible binary output values. Examples of single cutoff values ​​include about 1%, about 2%, about 5%, about 10%, about 15%, about 20%, about 25%, about 30%, about 35%, about 40%, about 45%, about 50%, about 55%, about 60%, about 65%, about 70%, about 75%, about 80%, about 85%, about 90%, about 91%, about 92%, about 93%, about 94%, about 95%, about 96%, about 97%, about 98%, and about 99%.

[0070]

[0100] As another example, a classification of a sample can be assigned an output value of "positive" or 1 if the sample indicates that the subject is at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 85%, at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99% or more in a cancer-related category (e.g., cancer diagnosis or prognosis). Classification of the sample can be assigned an output value of "positive" or 1 if the sample indicates that the subject is greater than about 50%, greater than about 55%, greater than about 65%, greater than about 65%, greater than about 70%, greater than about 75%, greater than about 80%, greater than about 85%, greater than about 90%, greater than about 91%, greater than about 92%, greater than about 93%, greater than about 94%, greater than about 95%, greater than about 96%, greater than about 97%, greater than about 98%, or greater than about 99% likely to belong to a cancer-related category (e.g., long-term outcome).

[0071]

[0101] Classification of the sample can be assigned an output value of "negative" or 0 if the sample indicates that the subject has less than about 50%, less than about 45%, less than about 40%, less than about 35%, less than about 30%, less than about 25%, less than about 20%, less than about 15%, less than about 10%, less than about 9%, less than about 8%, less than about 7%, less than about 6%, less than about 5%, less than about 4%, less than about 3%, less than about 2%, or less than about 1% chance of having a cancer-related condition or belonging to a cancer-related category (e.g., positive for prostate cancer). A classification of a sample can be assigned an output value of "negative" or 0 if the sample indicates that the subject has about 50% or less, about 45% or less, about 40% or less, about 35% or less, about 30% or less, about 25% or less, about 20% or less, about 15% or less, about 10% or less, about 9% or less, about 8% or less, about 7% or less, about 6% or less, about 5% or less, about 4% or less, about 3% or less, about 2% or less, or about 1% or less of the probability that the subject has a cancer-related condition (e.g., prostate cancer).

[0072]

[0102] Classification of a sample can be done by assigning an output value of "indeterminate" or 2 if the sample is not classified as "positive", "negative", 1 or 0. In this case, a set of two cutoff values ​​is used to classify the sample into one of three possible output values. Example sets of cutoff values ​​include {1%, 99%}, {2%, 98%}, {5%, 95%}, {10%, 90%}, {15%, 85%}, {20%, 80%}, {25%, 75%}, {30%, 70%}, {35%, 65%}, {40%, 60%}, and {45%, 55%}. Similarly, n sets of cutoff values ​​can be used to classify a sample into one of n+1 possible output values, where n is any positive integer.

[0073]

[0103] The trained algorithm can be trained using multiple independent training samples. Each of the independent training samples can include a biological sample from a subject, an associated dataset obtained by assaying the biological sample (as described elsewhere herein), clinical data from the subject, and one or more known output values ​​corresponding to the biological sample and / or the subject (e.g., clinical diagnosis, prognosis, absence, or treatment efficacy of the subject's cancer-related condition). The independent training samples can include biological samples and associated datasets and outputs obtained or derived from multiple different subjects. The independent training samples can include biological samples and associated datasets and outputs obtained from the same subject at multiple different time points (e.g., periodically, such as weekly, biweekly, monthly, yearly, etc.). The independent training samples can be associated with the presence of a cancer-related condition (e.g., a training sample including biological samples and associated datasets and outputs obtained or derived from multiple subjects known to have a cancer-related condition). Independent training samples may be associated with the absence of a cancer-associated condition (e.g., training samples including biological samples and associated datasets and output obtained from or derived from multiple subjects known to have not been previously diagnosed with a cancer-associated condition or multiple subjects who have received a negative test result for a cancer-associated condition).

[0074]

[0104] The trained algorithm can be trained using at least about 5, at least about 10, at least about 15, at least about 20, at least about 25, at least about 30, at least about 35, at least about 40, at least about 45, at least about 50, at least about 100, at least about 150, at least about 200, at least about 250, at least about 300, at least about 350, at least about 400, at least about 450, or at least about 500 independent training samples. The independent training samples may include acellular biological samples and clinical data related to the presence of cancer-related categories and / or acellular biological samples and clinical data related to the absence of cancer-related categories. The trained algorithm can be trained using about 500 or less, about 450 or less, about 400 or less, about 350 or less, about 300 or less, about 250 or less, about 200 or less, about 150 or less, about 100 or less, or about 50 or less independent training samples related to the presence of cancer-related categories. In some embodiments, the biological sample and / or clinical data is independent of the sample used to train the trained algorithm.

[0075]

[0105] The trained algorithm may be trained using a first number of independent training samples associated with the presence of a cancer-associated category and a second number of independent training samples associated with the absence of a cancer-associated category. The first number of independent training samples associated with the presence of a cancer-associated category may be less than or equal to the second number of independent training samples associated with the absence of a cancer-associated category. The first number of independent training samples associated with the presence of a cancer-associated category may be equal to the second number of independent training samples associated with the absence of a cancer-associated category. The first number of independent training samples associated with the presence of a cancer-associated category may be greater than the second number of independent training samples associated with the absence of a cancer-associated category.

[0076]

[0106] The trained algorithm may identify at least about 50%, at least about 55%, at least about 60%, or at least about 70% of the cancer-related categories for at least about 5, at least about 10, at least about 15, at least about 20, at least about 25, at least about 30, at least about 35, at least about 40, at least about 45, at least about 50, at least about 100, at least about 150, at least about 200, at least about 250, at least about 300, at least about 350, at least about 400, at least about 450, or at least about 500 independent training samples. The accuracy of the trained algorithm may be calculated as the percentage of independent test samples (e.g., subjects known to be in the cancer-associated category or subjects with a negative clinical test result for the cancer-associated category) that are correctly identified or classified as belonging or not belonging to the cancer-associated category.

[0077]

[0107] The trained algorithm may be configured to identify a cancer-related category with a positive predictive value (PPV) of at least about 5%, at least about 10%, at least about 15%, at least about 20%, at least about 25%, at least about 30%, at least about 35%, at least about 40%, at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99% or more. The PPV of identifying a cancer-associated category using the trained algorithm can be calculated as the proportion of acellular biological samples identified or classified as having a cancer-associated category that correspond to subjects that truly belong to that cancer-associated category.

[0078]

[0108] The trained algorithm may be configured to identify a cancer-related category with a negative predictive value (NPV) of at least about 5%, at least about 10%, at least about 15%, at least about 20%, at least about 25%, at least about 30%, at least about 35%, at least about 40%, at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99% or more. The NPV of identifying a cancer-related category using a trained algorithm can be calculated as the proportion of a subject dataset identified or classified as not having a cancer-related category that corresponds to subjects who truly do not belong to that cancer-related category.

[0079]

[0109] The trained algorithm may be at least about 5%, at least about 10%, at least about 15%, at least about 20%, at least about 25%, at least about 30%, at least about 35%, at least about 40%, at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, The cancer-associated category may be configured to identify a cancer-associated category with a clinical sensitivity of at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99%, at least about 99.1%, at least about 99.2%, at least about 99.3%, at least about 99.4%, at least about 99.5%, at least about 99.6%, at least about 99.7%, at least about 99.8%, at least about 99.9%, at least about 99.99%, at least about 99.999%, or more. The clinical sensitivity of identifying a cancer-associated category using the trained algorithm may be calculated as the proportion of independent test samples associated with the cancer-associated category (e.g., subjects known to belong to the cancer-associated category) that are correctly identified or classified as having the cancer-associated category.

[0080]

[0110] The trained algorithm may be at least about 5%, at least about 10%, at least about 15%, at least about 20%, at least about 25%, at least about 30%, at least about 35%, at least about 40%, at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99%, at least about 100%, at least about 101%, at least about 102%, at least about 103%, at least about 104%, at least about 105%, at least about 106%, at least about 107%, at least about 108%, at least about 109%, at least about 110%, at least about 111%, at least about 112%, at least about 113%, at least about 114%, at least about 115%, at least about 116%, at least about 117%, at least about 118%, at least about 119%, at least about 120%, at least about 121%, at least about 122%, at least about 123%, at least about 124%, at least about 125%, at least about 126%, at least about 127%, at least about 128%, at least about 129%, at least about 130%, at least about 131%, at least about 132%, at least about The cancer-associated category may be configured to identify a cancer-associated category with a clinical specificity of at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99%, at least about 99.1%, at least about 99.2%, at least about 99.3%, at least about 99.4%, at least about 99.5%, at least about 99.6%, at least about 99.7%, at least about 99.8%, at least about 99.9%, at least about 99.99%, at least about 99.999%, or more. Clinical specificity for identifying a cancer-associated category using a trained algorithm may be calculated as the proportion of independent test samples associated with the absence of a cancer-associated category (e.g., subjects with a negative clinical test result for the cancer-associated category) that are correctly identified or classified as not belonging to the cancer-associated category.

[0081]

[0111] The trained algorithm may be configured to distinguish a cancer-related category with an area under the curve (AUC) of at least about 0.50, at least about 0.55, at least about 0.60, at least about 0.65, at least about 0.70, at least about 0.75, at least about 0.80, at least about 0.81, at least about 0.82, at least about 0.83, at least about 0.84, at least about 0.85, at least about 0.86, at least about 0.87, at least about 0.88, at least about 0.89, at least about 0.90, at least about 0.91, at least about 0.92, at least about 0.93, at least about 0.94, at least about 0.95, at least about 0.96, at least about 0.97, at least about 0.98, at least about 0.99, or more. The AUC can be calculated as the integral of the receiver operating characteristic (ROC) curve (e.g., the area under the ROC curve) associated with a trained algorithm in classifying a dataset derived from subjects into or out of a cancer-related category.

[0082]

[0112] The trained algorithm can be adjusted or tuned to improve one or more of the following performance, accuracy, PPV, NPV, clinical sensitivity, clinical specificity, or AUC of identifying cancer-related categories. The trained algorithm can be adjusted or tuned by adjusting the parameters of the trained algorithm (e.g., a set of cutoff values ​​used to classify biological samples as described elsewhere herein, or the weights of a neural network). The trained algorithm can be adjusted or tuned continuously during the training process or after the training process is completed.

[0083]

[0113] After the trained algorithm is initially trained, a subset of inputs can be identified as the most influential or most important to include to make a high-quality classification. For example, a subset of clinical data can be identified as the most influential or most important to include to make a high-quality classification or discrimination of a cancer-related category (or a subtype of a cancer-related category). The clinical data or a subset thereof may be ranked based on a classification metric that indicates the influence or importance of each parameter in making a high-quality classification or discrimination of a cancer-related category (or a subtype of a cancer-related category). Such metrics may be used in some embodiments to meaningfully reduce the number of input variables (e.g., predictor variables) that can be used to train the trained algorithm to a desired performance level (e.g., based on a desired minimum accuracy, PPV, NPV, clinical sensitivity, clinical specificity, AUC, or a combination thereof). For example, if training a trained algorithm with a large number of input variables, including tens to hundreds in the trained algorithm, results in greater than 99% classification accuracy, instead training the trained algorithm using only a selected subset of the large number of input variables, such as about 5 or less, about 10 or less, about 15 or less, about 20 or less, about 25 or less, about 30 or less, about 35 or less, about 40 or less, about 45 or less, about 50 or less, or about 100 or less of the most influential or most significant input variables, may result in reduced classification accuracy but still provide acceptable accuracy (e.g., at least about 50 %, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, or at least about 99%.The subset can be selected by rank-ranking the entire number of input variables and selecting a predetermined number of input variables (e.g., about 5 or less, about 10 or less, about 15 or less, about 20 or less, about 25 or less, about 30 or less, about 35 or less, about 40 or less, about 45 or less, about 50 or less, or about 100 or less) having the best classification metric.

[0084]

[0114] The systems and methods described herein can use multiple trained algorithms to determine an output (e.g., a cancer-related category of a subject). The systems and methods can include 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or more trained algorithms. A trained algorithm of the multiple trained algorithms can be trained on a particular type of data (e.g., image data or tabular data). Alternatively, a trained algorithm can be trained on multiple types of data. The input of a trained algorithm may include the output of one or more other trained algorithms. In addition, a trained algorithm can receive the output of one or more trained algorithms as input.

[0085] Identifying or monitoring cancer-related categories or conditions

[0086]

[0115] After processing the dataset using the trained algorithm, a cancer-associated category can be identified or monitored in the subject. Identification may be based, at least in part, on quantitative or qualitative measures of the biological sample (e.g., of a histopathology slide of a biopsy sample), proteomic data including quantitative measures of proteins of the dataset in a panel of cancer-associated proteins, and / or metabolomic data including quantitative measures of a panel of cancer-associated metabolites.

[0087]

[0116] Cancer-related category can characterize the cancer-related status of the subject.As a non-limiting example, the cancer-related status can include the status that the subject has or does not have cancer (e.g., prostate cancer), the status that the subject is at risk of cancer or has a certain risk level (e.g., high risk, low risk), the predicted long-term outcome of cancer (e.g., distant metastasis, biochemical recurrence, partial response, complete response, overall survival, cancer-specific survival, progression-free survival, disease-free survival, 5-year survival, death), response or acceptance to therapeutic intervention, or combinations thereof.

[0088]

[0117] A subject may be identified as belonging to a cancer-related category with an accuracy of at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99%, or more. The accuracy of identifying an individual's cancer-related category by the trained algorithm may be calculated as the proportion of independent test samples (e.g., subjects known to belong to a cancer-related category or subjects with a negative clinical test result corresponding to a cancer-related category) that are correctly identified or classified as belonging to or not belonging to a cancer-related category.

[0089]

[0118] A subject may be determined to be in a cancer-associated category with a positive predictive value (PPV) of at least about 5%, at least about 10%, at least about 15%, at least about 20%, at least about 25%, at least about 30%, at least about 35%, at least about 40%, at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99% or more. The PPV of identifying a cancer-associated category using a trained algorithm can be calculated as the proportion of biological samples identified or classified as belonging to a cancer-associated category that correspond to subjects that truly belong to that cancer-associated category.

[0090]

[0119] A cancer-related category may be identified in a subject with a negative predictive value (NPV) of at least about 5%, at least about 10%, at least about 15%, at least about 20%, at least about 25%, at least about 30%, at least about 35%, at least about 40%, at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99% or more. The NPV of identifying a cancer-associated category using a trained algorithm can be calculated as the proportion of biological samples identified or classified as not belonging to the cancer-associated category that correspond to subjects that truly do not belong to that cancer-associated category.

[0091]

[0120] Subjects are at least about 5%, at least about 10%, at least about 15%, at least about 20%, at least about 25%, at least about 30%, at least about 35%, at least about 40%, at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, at least about 90%, at least about 100%, at least about 101%, at least about 102%, at least about 103%, at least about 104%, at least about 105%, at least about 106%, at least about 107%, at least about 108%, at least about 109%, at least about 200%, at least about 201%, at least about 202%, at least about 203%, at least about 204%, at least about 205%, at least about 206%, at least about 207%, at least about 208%, at least about 209%, at least about 300%, at least about 301%, at least about 302%, at least about 303, at least about 304, at least about 305, at least about 306, at least about 307, at least about 308, at least about 309, at least about 310, at least about 311, at least about 312, at least about 313, at least about 314, at least about 315, at least about 316, at least about 317, at least about 318. The cancer-associated category may be identified with a clinical sensitivity of at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99%, at least about 99.1%, at least about 99.2%, at least about 99.3%, at least about 99.4%, at least about 99.5%, at least about 99.6%, at least about 99.7%, at least about 99.8%, at least about 99.9%, at least about 99.99%, at least about 99.999%, or more. The clinical sensitivity of identifying a cancer-associated category using a trained algorithm may be calculated as the proportion of independent test samples associated with belonging to the cancer-associated category (e.g., subjects known to belong to the cancer-associated category) that are correctly identified or classified as belonging to the cancer-associated category.

[0092]

[0121] The cancer-related category is at least about 5%, at least about 10%, at least about 15%, at least about 20%, at least about 25%, at least about 30%, at least about 35%, at least about 40%, at least about 50%, at least about 55%, at least about 60%, at least about 65%, at least about 70%, at least about 75%, at least about 80%, at least about 81%, at least about 82%, at least about 83%, at least about 84%, at least about 85%, at least about 86%, at least about 87%, at least about 88%, at least about 89%, The cancer-associated category may be identified with a clinical specificity of at least about 90%, at least about 91%, at least about 92%, at least about 93%, at least about 94%, at least about 95%, at least about 96%, at least about 97%, at least about 98%, at least about 99%, at least about 99.1%, at least about 99.2%, at least about 99.3%, at least about 99.4%, at least about 99.5%, at least about 99.6%, at least about 99.7%, at least about 99.8%, at least about 99.9%, at least about 99.99%, at least about 99.999%, or more. Clinical specificity for identifying a cancer-associated category using a trained algorithm may be calculated as the proportion of independent test samples associated with not belonging to the cancer-associated category (e.g., subjects with a negative clinical test result for the cancer-associated category) that are correctly identified or classified as not belonging to the cancer-associated category.

[0093]

[0122] After a cancer-related category is identified in a subject, a subtype of the cancer-related category (e.g., selected from among a plurality of subtypes of the cancer-related category) may be further identified. The subtype of the cancer-related category may be determined based at least in part on quantitative or qualitative measures of a biological sample (e.g., of a histopathology slide of a biopsy sample), proteomic data including quantitative measures of proteins of a dataset in a panel of cancer-related proteins, and / or metabolomic data including quantitative measures of a panel of cancer-related metabolites. For example, a subject may be identified as being at risk for a subtype of prostate cancer (e.g., among several subtypes of prostate cancer). After identifying a subject as being at risk for a subtype of prostate cancer, a clinical intervention for the subject may be selected based at least in part on the subtype of prostate cancer for which the subject is identified as being at risk. In some embodiments, the clinical intervention is selected from a plurality of clinical interventions (e.g., clinically indicated for different subtypes of prostate cancer).

[0094]

[0123] Once a subject is identified as belonging to a cancer-related category, the subject may optionally be provided with a therapeutic intervention (e.g., prescribing an appropriate course of treatment to treat the subject's cancer type, subtype, or condition). The therapeutic intervention may include prescribing an effective amount of a drug or other therapy (e.g., radiation therapy, chemotherapy), a surgical intervention (e.g., radical prostatectomy), further testing or evaluation of the cancer-related category, further monitoring of the cancer-related category, or a combination thereof. If the subject is currently being treated with a course of treatment for the cancer-related category, the therapeutic intervention may include a subsequent different course of treatment (e.g., to increase the effectiveness of the treatment because the current course of treatment is not effective).

[0095]

[0124] The therapeutic intervention may include recommending a secondary laboratory test for the subject to confirm the diagnosis of the cancer-related category, which may include imaging tests, blood tests, computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, ultrasound scans, x-rays, positron emission tomography (PET) scans, PET-CT scans, bone scans, lymph node biopsies, or combinations thereof.

[0096]

[0125] Analysis of biopsy samples (e.g., analysis of microscopic images of prostate tissue), proteomic data including quantitative measures of proteins of a dataset in a panel of cancer-related category-related proteins, and / or metabolomic data including quantitative measures of a panel of cancer-related category-related metabolites can be evaluated over a period of time to monitor a patient (e.g., a subject with or at risk of cancer, or a subject undergoing treatment for cancer). In such cases, the patient's dataset measures may change over the course of treatment. For example, a dataset measure of a patient whose risk of a cancer-related category has decreased due to effective treatment may shift toward a profile or distribution of a healthy subject (e.g., a subject who is cancer-free or in remission from cancer). Conversely, a dataset measure of a patient whose risk of a cancer-related category has increased due to ineffective treatment may shift toward a profile or distribution of a subject who is at higher risk of a cancer-related category or a subject whose cancer-related category is more advanced.

[0097]

[0126] The cancer-related category of the subject can be monitored by monitoring a course of treatment for treating the cancer or cancer-related condition of the subject. The monitoring may include evaluating the cancer-related category or condition of the subject at two or more time points. The evaluation may be based on at least a quantitative or qualitative measure of the biological sample (e.g., of a histopathology slide of a biopsy sample), proteomic data including a quantitative measure of the proteins of a dataset in a panel of cancer-related proteins, and / or metabolomic data including a quantitative measure of a panel of cancer-related metabolites, determined at each of the two or more time points.

[0098]

[0127] In some embodiments, differences in quantitative or qualitative measures of a biological sample (e.g., of a histopathology slide of a biopsy sample), proteomic data including quantitative measures of proteins of a dataset in a panel of cancer-associated proteins, and / or metabolomic data including quantitative measures of a panel of cancer-associated metabolites determined between two or more time points may be indicative of one or more clinical indications, such as (i) a diagnosis of a cancer-associated condition in a subject, (ii) a prognosis of a cancer-associated condition in a subject, (iii) an increased risk of a cancer-associated condition in a subject, (iv) a decreased risk of a cancer-associated condition in a subject, (v) the effectiveness of a course of treatment for treating a cancer-associated condition in a subject, and (vi) the ineffectiveness of a course of treatment for treating a cancer-associated condition in a subject.

[0099]

[0128] In some embodiments, a difference in a quantitative or qualitative measure of a biological sample (e.g., of a histopathology slide of a biopsy sample), a proteomic data including a quantitative measure of a protein of a dataset in a panel of cancer-related category-related proteins, and / or a metabolomic data including a quantitative measure of a panel of cancer-related metabolites determined between two or more time points may indicate a cancer-related condition or a diagnosis of a cancer-related category in a subject. For example, if a cancer-related condition is not detected in a subject at an earlier time point, but is detected in the subject at a later time point, the difference indicates that the cancer-related condition in the subject is a diagnosis. Based on this indication of a diagnosis of the cancer-related condition in the subject, a clinical action or decision may be made, such as, for example, prescribing a new therapeutic intervention for the subject. The clinical action or decision may include recommending a secondary laboratory test for the subject to confirm the diagnosis of the cancer-related category. The secondary laboratory tests may include imaging tests, blood tests, computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, ultrasound scans, x-rays, positron emission tomography (PET) scans, PET-CT scans, bone scans, lymph node biopsies, or combinations thereof.

[0100]

[0129] In some embodiments, differences in quantitative or qualitative measures of a biological sample (e.g., of a histopathology slide of a biopsy sample), proteomic data including quantitative measures of proteins of a dataset in a panel of cancer-associated proteins, and / or metabolomic data including quantitative measures of a panel of cancer-associated metabolites determined between two or more time points may be indicative of a subject's prognosis in a cancer-associated category.

[0101]

[0130] In some embodiments, a difference in the quantitative or qualitative measure of the biological sample (e.g., of a histopathology slide of a biopsy sample), the proteomic data including a quantitative measure of proteins of a dataset in a panel of proteins associated with a cancer-related category, and / or the metabolomic data including a quantitative measure of a panel of metabolites associated with a cancer-related category determined between two or more time points may indicate that the subject has an elevated risk of a cancer-related condition. For example, if a cancer-related condition is detected in a subject at both an early time point and a later time point and the difference is a negative difference (e.g., if the quantitative or qualitative measure of the biological sample (e.g., of a histopathology slide of a biopsy sample), the proteomic data including a quantitative measure of proteins of a dataset in a panel of proteins associated with a cancer-related category, and / or the metabolomic data including a quantitative measure of a panel of metabolites associated with a cancer-related category increases from an early time point to a later time point), the difference may indicate that the subject has an elevated risk of a cancer-related condition. Based on this indication of an increased risk of a cancer-related condition, a clinical action or decision can be made, such as prescribing a new therapeutic intervention for the subject or switching therapeutic interventions (e.g., terminating the current treatment and prescribing a new treatment). The clinical action or decision can include recommending a secondary laboratory test for the subject to confirm the increased risk of the cancer-related category. The secondary laboratory test can include imaging tests, blood tests, computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, ultrasound scans, x-rays, positron emission tomography (PET) scans, PET-CT scans, bone scans, lymph node biopsies, or combinations thereof.

[0102]

[0131] In some embodiments, a difference in the quantitative or qualitative measure of the biological sample (e.g., of a histopathology slide of a biopsy sample), the proteomic data including a quantitative measure of a protein of a dataset in a panel of cancer-associated proteins, and / or the metabolomic data including a quantitative measure of a panel of cancer-associated metabolites determined between two or more time points may indicate that the subject has a reduced risk of a cancer-associated condition. For example, if a cancer-associated condition is detected in a subject at both an earlier time point and a later time point and the difference is a positive difference (e.g., if the quantitative or qualitative measure of the biological sample (e.g., of a histopathology slide of a biopsy sample), the proteomic data including a quantitative measure of a protein of a dataset in a panel of cancer-associated proteins, and / or the metabolomic data including a quantitative measure of a panel of cancer-associated metabolites decreases from an earlier time point to a later time point), the difference may indicate that the subject has a reduced risk of the cancer-associated condition. Based on this indication of a reduced risk of the cancer-associated condition, a clinical action or decision (e.g., continuation or termination of a current therapeutic intervention) can be made for the subject. The clinical action or decision may include recommending a secondary laboratory test for the subject to confirm the reduction in risk of a cancer-related category, which may include an imaging test, a blood test, a computed tomography (CT) scan, a magnetic resonance imaging (MRI) scan, an ultrasound scan, an x-ray, a positron emission tomography (PET) scan, a PET-CT scan, a bone scan, a lymph node biopsy, or a combination thereof.

[0103]

[0132] In some embodiments, a difference in a quantitative or qualitative measure of a biological sample (e.g., of a histopathology slide of a biopsy sample), a proteomic data including a quantitative measure of a protein of a dataset in a panel of cancer-associated proteins, and / or a metabolomic data including a quantitative measure of a panel of cancer-associated metabolites determined between two or more time points may indicate the effectiveness of a course of treatment for treating a cancer-associated condition in a subject. For example, if a cancer-associated condition is detected in a subject at an earlier time point but not in the subject at a later time point, the difference may indicate that a course of treatment for treating a cancer-associated condition in a subject is effective. Based on this indication that a course of treatment for treating a cancer-associated condition in a subject is effective, a clinical action or decision may be made, such as to continue or terminate a current therapeutic intervention for the subject. The clinical action or decision may include recommending a secondary laboratory test for the subject to determine the effectiveness of a course of treatment for treating a cancer-associated category. The secondary laboratory tests may include imaging tests, blood tests, computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, ultrasound scans, x-rays, positron emission tomography (PET) scans, PET-CT scans, bone scans, lymph node biopsies, or combinations thereof.

[0104]

[0133] In some embodiments, a difference in the quantitative or qualitative measure of the biological sample (e.g., of a histopathology slide of a biopsy sample), the proteomic data including a quantitative measure of a protein of a dataset in a panel of cancer-associated proteins, and / or the metabolomic data including a quantitative measure of a panel of cancer-associated metabolites determined between two or more time points may indicate the ineffectiveness of a course of treatment for treating the cancer-associated category of the subject. For example, if a cancer-associated condition is detected in a subject at both an early time point and a later time point and the difference is negative or is a zero difference (e.g., if the quantitative or qualitative measure of the biological sample (e.g., of a histopathology slide of a biopsy sample), the proteomic data including a quantitative measure of a protein of a dataset in a panel of cancer-associated proteins, and / or the metabolomic data including a quantitative measure of a panel of cancer-associated metabolites increases or remains at a constant level from the early time point to the later time point), and if an effective treatment was indicated at the earlier time point, the difference may indicate that a course of treatment for treating the cancer-associated condition of the subject is not effective. Based on this indication that the course of treatment for treating the subject's cancer-related condition is ineffective, a clinical action or decision can be made, such as terminating the current therapeutic intervention for the subject and / or switching to (e.g., prescribing) a different new therapeutic intervention. The clinical action or decision may include recommending a secondary laboratory test for the subject to determine the ineffectiveness of the course of treatment for treating the cancer-related condition. The secondary laboratory test may include imaging tests, blood tests, computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, ultrasound scans, x-rays, positron emission tomography (PET) scans, PET-CT scans, bone scans, lymph node biopsies, or combinations thereof.

[0105] Reporting of cancer-related conditions

[0106]

[0134] After a cancer-related condition has been identified or an elevated risk of a cancer-related condition has been monitored in the subject, a report may be electronically outputted that is indicative of (e.g., identifies or provides an indication of) the subject's cancer-related condition. The subject may not exhibit a cancer-related condition (e.g., is asymptomatic of a cancer-related condition, such as the presence or risk of prostate cancer). The report is presented on a graphical user interface (GUI) of the user's electronic device. The user may be the subject, a caregiver, a physician, a nurse, or other medical professional.

[0107]

[0135] The report may include one or more clinical indicators, such as (i) a diagnosis of the subject's cancer-related condition, (ii) a prognosis of the subject's cancer-related category, (iii) an increased risk of the subject's cancer-related category, (iv) a decreased risk of the subject's cancer-related category, (v) an efficacy of a course of treatment for treating the subject's cancer-related category, (vi) an ineffectiveness of a course of treatment for treating the subject's cancer-related category, and (vii) a long-term outcome of the cancer-related category. The report may include one or more clinical actions or decisions made based on these one or more clinical indicators. Such clinical actions or decisions may be directed to a therapeutic intervention of the subject's cancer-related condition, or further clinical evaluation or testing.

[0108]

[0136] For example, a clinical indication of a diagnosis of a cancer-related condition in a subject may involve the clinical action of prescribing a new therapeutic intervention for the subject. As another example, a clinical indication of an increased risk of a cancer-related condition in a subject may involve the clinical action of prescribing a new therapeutic intervention for the subject or switching therapeutic interventions (e.g., terminating a current treatment and prescribing a new treatment). As another example, a clinical indication of a decreased risk of a cancer-related condition in a subject may involve the clinical action of continuing or terminating a current therapeutic intervention for the subject. As another example, a clinical indication of the effectiveness of a course of treatment to treat a cancer-related condition in a subject may involve the clinical action of continuing or terminating a current therapeutic intervention for the subject. As another example, a clinical indication of the ineffectiveness of a course of treatment to treat a cancer-related condition in a subject may involve the clinical action of terminating a current therapeutic intervention for the subject and / or switching (e.g., prescribing) a new, different therapeutic intervention.

[0109]

[0137] In some embodiments, the therapeutic intervention may include radiation therapy (RT), chemotherapy, surgical intervention (e.g., radical prostatectomy), further testing or evaluation of the cancer-related category, further monitoring of the cancer-related category, or a combination thereof. If the subject is currently undergoing treatment with a course of treatment for the cancer-related category, the therapeutic intervention may include a subsequent different course of treatment (e.g., to increase the effectiveness of the treatment because the current course of treatment is not effective). The therapeutic intervention may include recommending a secondary laboratory test to the subject to confirm the diagnosis of the cancer-related category. The secondary laboratory test may include imaging tests, blood tests, computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, ultrasound scans, x-rays, positron emission tomography (PET) scans, PET-CT scans, bone scans, lymph node biopsies, or a combination thereof.

[0110] Computer Systems

[0111]

[0138] The present disclosure provides computer systems programmed to implement the methods of the present disclosure. Figure 1 illustrates a computer system 101 programmed or otherwise configured to, for example, (i) train and test a trained algorithm, (ii) use the trained algorithm to process image data and / or tabular data to determine a cancer-related category or cancer-related status of a subject, (iii) assess the cancer of the subject based on the classified category, (iv) identify or monitor the cancer-related category or cancer-related status of the subject, and (v) electronically output a report indicating the cancer-related category or cancer-related status of the subject.

[0112]

[0139] The computer system 101 can coordinate various aspects of the analysis, calculation, and generation of the present disclosure, such as, for example, (i) training and testing the trained algorithm, (ii) processing the image data and / or tabular data using the trained algorithm to determine the cancer-related category or cancer-related status of the subject, (iii) assessing the cancer of the subject based on the classified category, (iv) identifying or monitoring the cancer-related category or cancer-related status of the subject, and (v) electronically outputting a report indicating the cancer-related category or cancer-related status of the subject. The computer system 101 can be a user's electronic device or a computer system located remotely with respect to the electronic device. The electronic device can be a mobile electronic device.

[0113]

[0140] The computer system 101 includes a central processing unit (CPU, also referred to herein as "processor" and "computer processor") 105, which may be a single-core or multi-core processor, or multiple processors for parallel processing. The computer system 101 also includes memory or memory locations 110 (e.g., random access memory, read-only memory, flash memory), an electronic storage unit 115 (e.g., hard disk), a communication interface 120 (e.g., network adapter) for communicating with one or more other systems, and peripherals 125, such as cache, other memory, data storage, and / or electronic display adapters. The memory 110, the storage unit 115, the interface 120, and the peripherals 125 can communicate with the CPU 105 via a communication bus (solid lines), such as a motherboard. The storage unit 115 can be a data storage unit (or data repository) for storing data. The computer system 101 can be operatively coupled to a computer network ("network") 130 with the aid of the communication interface 120. The network 130 may be the Internet, the Internet and / or an extranet, or an intranet and / or an extranet in communication with the Internet.

[0114]

[0141] In some embodiments, the network 130 is a telecommunications network and / or a data network. The network 130 may include one or more computer servers that enable distributed computing, such as cloud computing. For example, the one or more computer servers may enable cloud computing via the network 130 ("cloud") to perform various aspects of the analysis, calculation, and generation of the present disclosure, such as, for example, (i) training and testing the trained algorithms, (ii) using the trained algorithms to process data and determine the cancer-related category of the subject, (iii) determining a quantitative measure indicative of the cancer-related category of the subject, (iv) identifying or monitoring the cancer-related category of the subject, and (v) electronically outputting a report indicative of the cancer-related category of the subject. Such cloud computing may be provided, for example, by a cloud computing platform such as Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform, IBM cloud, etc. In some embodiments, the network 130 may implement a peer-to-peer network with the aid of the computer system 101, whereby devices coupled to the computer system 101 may act as clients or servers.

[0115]

[0142] CPU 105 may include one or more computer processors and / or one or more graphic processing units (GPUs). CPU 105 may execute a series of machine-readable instructions, which may be embodied in a program or software. The instructions may be stored in a memory location, such as memory 110. The instructions may be directed to CPU 105, which may then be programmed or otherwise configured to implement the methods of the present disclosure. Examples of operations performed by CPU 105 may include fetch, decode, execute, and writeback.

[0116]

[0143] The CPU 105 may be part of a circuit, such as an integrated circuit. One or more other components of the system 101 may also be included in the circuit. In some embodiments, the circuit is an application specific integrated circuit (ASIC).

[0117]

[0144] The storage unit 115 can store files such as drivers, libraries, saved programs, etc. The storage unit 115 can store user data, e.g., user preferences and user programs. In some embodiments, the computer system 101 can include one or more additional data storage units that are external to the computer system 101, such as on a remote server in communication with the computer system 101 through an intranet or the Internet.

[0118]

[0145] The computer system 101 can communicate with one or more remote computer systems via the network 130. For example, the computer system 101 can communicate with a remote computer system of a user. Examples of remote computer systems include a personal computer (e.g., a portable PC), a slate or tablet PC (e.g., an Apple® iPad®, a Samsung® Galaxy Tab), a phone, a smartphone (e.g., an Apple® iPhone®, an Android-enabled device, a Blackberry®), or a personal digital assistant. A user can access the computer system 101 via the network 130.

[0119]

[0146] The methods described herein may be implemented by machine (e.g., computer processor) executable code stored in electronic storage locations of computer system 101, such as, for example, on memory 110 or electronic storage unit 115. Machine executable or machine readable code may be provided in the form of software. In use, the code may be executed by processor 105. In some embodiments, the code may be retrieved from storage unit 115 and stored in memory 110 for ready access by processor 105. In some circumstances, electronic storage unit 115 may not be used and machine executable instructions may be stored in memory 110.

[0120]

[0147] The code may be precompiled and configured for use on a machine having a processor that executes the code, or it may be compiled at run time. The code may be provided in a programming language that can be selected to allow the code to be executed in a precompiled or as-compiled fashion.

[0121]

[0148] The embodiments of the systems and methods provided herein, such as the computer system 101, may be embodied in programming. Various aspects of the technology may be considered as a "product" or "article of manufacture" in the form of machine (or processor) executable code and / or associated data typically carried or embodied in some type of machine-readable medium. The machine executable code may be stored in an electronic storage unit such as a memory (e.g., read-only memory, random access memory, flash memory, etc.) or a hard disk. A "storage" type medium may include tangible memory of a computer, a processor, etc., or its associated modules, such as various semiconductor memories, tape drives, or disk drives, which may provide non-transitory storage at any time for software programming. All or a portion of the software may be communicated at times over the Internet or various other telecommunications networks. Such communication allows loading of the software from one computer or processor to another, such as, for example, from a management server or host computer to a computer platform of an application server. Thus, other types of media that may carry software elements include light waves, radio waves, electromagnetic waves, such as those used across physical interfaces between local devices, through wired and optical fixed line networks, and on various air links. The physical elements that carry such waves, such as wired or wireless links, optical links, etc., may also be considered media that carry software. As used herein, unless limited to non-transitory tangible "storage" media, terms such as computer or machine "readable storage medium" refer to any medium that participates in providing instructions to a processor for execution.

[0122]

[0149] Thus, a machine-readable medium such as a computer executable code can take many forms, such as a tangible storage medium, a carrier wave medium, or a physical transmission medium. Non-volatile storage media include optical or magnetic disks, such as any of the storage devices in any computer, such as may be used to implement the databases, etc., shown in the figures. Volatile storage media include dynamic memory, such as the main memory of such a computer platform. Tangible transmission media include coaxial cables, copper wire, optical fibers, and the like, including the wires that make up a bus in a computer system. Carrier wave transmission media can take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Thus, common forms of computer readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, other magnetic media, CD-ROMs, DVDs or DVD-ROMs, other optical media, punch cards paper tape, other physical storage media with patterns of holes, RAM, ROM, PROMs and EPROMs, FLASH-EPROMs, other memory chips or cartridges, carrier waves carrying data or instructions, cables or links carrying such carrier waves, or other media from which a computer can read programming code and / or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.

[0123]

[0150] The computer system 101 can include or communicate with an electronic display 135 that includes a user interface (UI) 140 to provide, for example: (i) a visual display indicating the training and testing of a trained algorithm, (ii) a visual display of data indicating the subject's cancer-related category, (iii) a quantitative measure of the subject's cancer-related category, (iv) an identification of the subject as having a cancer-related category, or (v) an electronic report indicating the subject's cancer-related category. Examples of UIs include, but are not limited to, graphical user interfaces (GUIs) and web-based user interfaces.

[0124]

[0151] The methods and systems of the present disclosure may be implemented by one or more algorithms. The algorithms may be implemented by software when executed by the central processing unit 205. The algorithms may, for example, (i) train and test a trained algorithm, (ii) use the trained algorithm to process image data and / or tabular data to determine a cancer-related category or cancer-related status of a subject, (iii) assess the cancer of the subject based on the classified category, (iv) identify or monitor the cancer-related category or cancer-related status of the subject, and (v) electronically output a report indicating the cancer-related category or cancer-related status of the subject.

[0125]

[0152] Example 1: Personalizing prostate cancer treatment with multimodal deep learning

[0126]

[0153] The methods and systems disclosed herein demonstrate personalization of prostate cancer treatment by predicting long-term, clinically relevant outcomes (distant metastasis, biochemical recurrence, death from prostate cancer, and overall survival) using a novel multimodal deep learning model trained on digital histopathology and clinical data of prostate biopsies. The exemplary system of the present disclosure includes a trained algorithm that was trained and validated using a dataset from five randomized Phase III multinational trials conducted across hundreds of clinical sites. Clinical and histopathology data were available for 5,654 of 7,957 patients (71.1%), resulting in 16.1 terabytes of histopathology images with 10-20 years of patient follow-up. When compared to the National Cancer Center Network (NCCN) risk groups, the most commonly used risk stratification tool, the deep learning model showed superior prognostic and discrimination performance across all outcomes examined. This artificial intelligence system may enable oncologists to computationally model the most likely outcomes for a given patient and determine the patient's optimal treatment. Equipped with a digital histopathology scanner and internet access, any clinic can offer such capabilities, allowing low-cost, universal access to vital personalization of care.

[0127]

[0154] NCCN risk groups are based on an international standard for risk stratification developed in the late 1990s and referred to as D'Amico risk groups. This system is based on tumor grade assessed by digital rectal examination, serum prostate-specific antigen (PSA) measurements, and histopathology. Although this three-tiered system continues to form the backbone of treatment recommendations worldwide, its prognostic and discriminatory performance for risk stratifying patients has been suboptimal. This is in part because the core variables of these models are highly subjective and nonspecific in nature. For example, Gleason grading was developed in the 1960s but remains highly subjective and has unacceptable interobserver reproducibility, even among expert urologic pathologists. More recently, tissue-based genomic biomarkers have demonstrated improved prognostic performance. However, almost all of these tests lack validation in prospective randomized clinical trials in the intended use population, and international adoption has been limited due to cost and turnaround time issues. Thus, there remains an unmet clinical need for improved tools to personalize treatment for prostate cancer.

[0128]

[0155] Artificial intelligence (AI) has demonstrated impressive capabilities in many use cases in healthcare, from physician-level diagnosis to workflow optimization, and has the potential to support cancer treatment as clinical adoption of digital histopathology continues. AI is beginning to make progress in histopathology-based prognosis, for example by predicting short-term patient outcomes and improving the accuracy of Gleason-based cancer grading in post-operative surgical specimens. Whereas standard risk stratification tools are fixed and based on a small number of variables, AI can learn from large amounts of minimally processed data across various modalities. In contrast to genomic biomarkers, AI systems are low-cost, massively scalable, and incrementally improved through use. Furthermore, a key challenge for any biomarker is having optimal data to train and validate the relevant endpoints, and some commercially available prognostic biomarkers in oncology are trained on retrospective convenience sampling.

[0129]

[0156] The methods and systems disclosed herein can include a multimodal artificial intelligence (MMAI) system that meaningfully overcomes the unmet need for outcome prognostication in localized prostate cancer and can generate generalizable biomarkers with potential for global adoption. By utilizing multimodal deep learning with digital histopathology, localized prostate cancer prognostic biomarkers from five Phase III randomized clinical trials were used to train the algorithms described herein.

[0130]

[0157] A unique dataset was generated from five large multinational randomized phase III clinical trials (NRG / RTOG9202, 9408, 9413, 9910, 0126) in men with localized prostate cancer. All patients received short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), definitive radiation therapy (RT) with pre-specified use of rituximab, and / or dose-escalated RT (DE-RT) (Figure 2C). Of the 7,957 patients enrolled in these five trials, 7,752 patients had complete baseline clinical data and 5,654 patients had complete baseline and digital histopathology data. This represents 16.1TB of histopathology images from 16,204 histopathology slides of pre-treatment biopsy samples.

[0131]

[0158] The MMAI architecture can ingest both tabular (clinical) and image-based (histopathology) data, making it uniquely suited for randomized clinical trial data. The complete architecture is shown in Figure 2A. Each patient in the dataset is represented by clinical variables (including laboratory data, pathology data, therapeutic interventions, and long-term outcomes), and digital histopathology slides (median 3.5 slides). Joint learning across both data streams is complex and involved the construction of three separate deep learning pipelines: one for images, one for the tabular data, and a third to integrate them. Data was standardized across trials for consistency.

[0132]

[0159] Effectively learning relevant features from a variable number of digital histopathology slides involved several preprocessing steps to standardize the images, followed by self-supervised training. For each patient, all tissue sections in the patient's biopsy slide were segmented and composited into a single large image of fixed width and height across all patients, called the image quilt (Figure 5). A H×W grid was overlaid on top of the image quilt and sliced ​​into 256×256 pixel patches across the RGB channels. These patches were then used to train a self-supervised (SSL) model to learn histopathological features useful for downstream AI tasks. Figure 2B illustrates this part of the pipeline. Once trained, the SSL model can then ingest patches of the image quilt and output a 128-dimensional vector representation for each patch. Concatenating all these vectors in the same spatial orientation as the original patches gives us a H×W×128 tensor (the feature quilt) that compresses the initially huge image quilt into a compact representation useful for further downstream learning.

[0133]

[0160] SSL is a method that can be used to learn from unannotated datasets. A typical ML setup leverages supervised learning, where a dataset consists of data points (e.g., images) and data labels (e.g., object classes). In contrast, in SSL, synthetic data labels are extracted from the original data and used to train a general feature representation that can be used for downstream tasks. Momentum contrast is a technique that takes a set of image patches, generates augmented copies of each patch, and then trains a model to predict whether any two augmented copies are derived from the same original patch, which can be useful in learning features for medical tasks. The structural setup is shown in Figure 2B and is described in further detail elsewhere in this document.

[0134]

[0161] To guide the SSL process to patch regions that are more likely to be clinically useful, the patches in the dataset were oversampled based on nucleus density. An object detection model trained to detect nuclei was used to estimate the number of nuclei in each patch. The patches were divided into deciles based on this count, and each decile was oversampled so that the net number of images seen during one epoch of training was the same for each decile. An example image is shown in Figure 6.

[0135]

[0162] The system described herein can learn from patient-level annotations without the need for annotations in histopathology slides, and furthermore, the self-supervised learning of the image model allows it to learn from new image data without the need for annotations.

[0136]

[0163] Learning from tabular data involves two steps. First, the clinical data was standardized across trials and used to pre-train the TabNet architecture through self-supervision by masking parts of the data and training the model to learn them. Each patient's data was then passed through TabNet to extract a feature vector, which was concatenated with the output of the image pipeline. The concatenated vector was then passed through further neural layers and the model output a binary outcome for the task at hand.

[0137]

[0164] The internal data representation of the SSL model is shown in Figure 4. Image patches from the entire dataset were passed through the SSL model, and model features (128-dimensional vectors output by the model) were extracted for each patch. The Uniform Manifold Approximation and Projection (UMAP) algorithm was then applied to these features, projecting the features from 128 dimensions to 2 dimensions and plotting each patch as an individual point. Adjacent data points represent image patches that the model considered similar. UMAP grouped the feature vectors into 25 clusters, some of which are shown in different colors. The inset shows example image patches that are close to the cluster center points in the feature space. The 20 image patches closest to the cluster center points were then interpreted by a pathologist. An example interpretation is shown in Figure 4, and the full interpretation in Figure 7. Despite never being trained on clinical annotations, the SSL model learned human-interpretable image features that indicate complex aspects of cancer, such as Gleason grade or tissue type.

[0138]

[0165] Six different MMAI models were trained and tested across four endpoints (DM, BCR, PCaSS, OS) and two time frames: 5 and 10 years. The performance of these models was measured by the time-dependent area under the receiver operating characteristic curve (AUC) of sensitivity and specificity taking into account competing events. Sensitivity is defined as the proportion of correct positive predictions for indicated positive events (sensitivity = predicted_positive / num_positive), and specificity is defined as the proportion of correct negative predictions for indicated negative events (specificity = predicted_negative / num_negative). In this metric, 0.5 represents chance accuracy and 1.0 represents perfect accuracy.

[0139]

[0166] The NCCN model served as the baseline comparator, as shown in Figure 8. Patients were grouped into low-, intermediate-, and high-risk groups using three variables: Gleason score, tumor t stage, and baseline PSA.

[0140]

[0167] The results are shown in Figures 3A-H. For each outcome and time point, a separate model was trained. In Figure 3A and Figures 2D-H, blue bars represent the performance of the MMAI model trained on a particular task, while grey bars represent the performance of the corresponding NCCN model. Figure 2B shows the relative improvement of MMAI over NCCN on a subset of outcomes and test sets from five studies. The MMAI model consistently outperformed the NCCN model on all outcomes tested. The relative improvement in AUC varied from 11.45% to 19.72%. Furthermore, the subsets of studies all uniformly showed relative improvement over NCCN.

[0141]

[0168] An ablation study was performed to evaluate the effect of various data components specific to the MMAI model. Additional MMAI models were trained using the following data setups: NCCN variables only, pathology images only, pathology images + NCCN variables (combined Gleason score, t-stage, baseline PSA), pathology images + NCCN variables + three additional variables used in the model (age, Gleason primary, Gleason secondary). Each additional data component improved performance, with the full setup (pathology, six clinical variables) producing the best results (Figure 2C).

[0142]

[0169] The MMAI system significantly outperformed the model-encoded NCCN risk stratification tool in predicting four important future patient outcomes: distant metastasis, biochemical recurrence, prostate cancer-specific survival, and overall survival. By creating a deep learning architecture that simultaneously incorporates multiple data types (of varying sizes) from patients, as well as clinical data, a deep learning system was built that was able to predict long-term patient outcomes with significantly greater accuracy than established clinical models.

[0143]

[0170] The methods and systems described herein can leverage robust, large-scale clinical data from five different prospective, randomized, multinational trials with 10-20 years of patient follow-up on 5,654 patients across diverse populations. Validating these prognostic classifiers with large amounts of clinical trial data (in the intended use population) uniquely positions these tools as therapeutic decision-making aids. Similarly, significant drawbacks of genomics-based assays are their high cost and long test turnaround times. AI tools do not bear such limitations, making the barriers to large-scale international adoption significantly lower. Despite nearly 60% of the world having access to the internet, only about 4% (the US population) has easy access to genomics-based assays. The ever-increasing adoption of digital histopathology, coupled with internet connectivity, may support the global proliferation of AI-based prognostic and predictive testing, enabling access to critical treatment personalization at a lower cost.

[0144]

[0171] method

[0145]

[0172] Tabular Pipeline. Tabular clinical data were split into numerical and categorical variables. Numerical variables were whitened (mean subtraction + max normalization) to the range of [-1,1]. Categorical variables were treated as one-hot vectors that were embedded into 2-3 dimensional vectors according to traditional word-to-vec techniques, with the number of dimensions being determined by the formula, D = Round(1.6 · num c attributes 0.56 We used a TabNet model with input being a concatenation of categorical and numerical variables (parameters: learning rate 0.2, Adam optimizer with step learning rate scheduler, batch size 1024, up to 50 epochs with early stopping with patience 10 epochs).

[0146]

[0173] Image Pipeline. The ResNet50 model was used in conjunction with the MoCo-v2 training protocol (parameters: learning rate = 0.03 with cosine learning rate schedule, moco-t = 0.2, multi-layer perceptron head, batch size 256, MoCo-v2 data augmentation, 200 epochs) to train the SSL model used in the system architecture of Figure 2B. For each held-out test set in Figure 3A, the corresponding SSL model was pre-trained using only the images from the training data. Certain image patches were oversampled using kernel density sampling as described elsewhere herein. Once the SSL pre-training was complete, all W × H patches were fed into the SSL pre-trained ResNet50 model, generating a W × H × 128 feature quilt for each image quilt. The final image model used for prediction was a two-layer CNN model with batch norm and dropout, taking the feature tensors as input. The final CNN model was trained with the Adam optimizer with a step learning rate scheduler, a batch size of 32, a maximum of 150 epochs, and a learning rate of 0.01.

[0147]

[0174] Downstream Pipeline. A joint fusion approach was used to leverage information from both modalities (image and tabular features). Images were featurized into feature tensors and fed into the final image model to generate a feature vector, and the tabular features were fed separately into the TabNet model to generate another feature vector. Two fully connected layers processed the concatenated feature vectors of each pipeline and output a prediction probability. For patients with missing histopathology data, the image-based feature vectors were zeroed before concatenation.

[0148]

[0175] Dataset preparation. All patient-level baseline clinical data, digital histopathology slides of prostate biopsies, and longitudinal outcomes from five landmark, large-scale, prospective, randomized, international clinical trials involving 5,654 patients, 16,204 histopathology slides, and a median patient follow-up of 10 to 20 years were used. These trials were RTOG 9202, 9408, 9910, 0126, and 9413 (Figure 2C). These trials randomized patients with various combinations of radiation therapy (RT) and androgen deprivation therapy (ADT): RT+short-term ADT (RTS), RT+intermediate ADT (RTM), RT+long-term ADT (RTL), and RT dose and volume levels (RT+). Slides were digitized by NRG Oncology using a Leica Biosystems Aperio AT2 digital pathology scanner at 20x resolution over a 2-year period. Histopathology images were manually reviewed for image quality and clarity. Digital slides were converted into a single image quilt of 200 × 200 patches for each unique patient prior to model training. The clinical variables collected in each trial were subtly different. Six clinical variables that were available in all trials (combined Gleason, Gleason primary, Gleason secondary, t-stage, baseline PSA, age) and digital histopathology were used to train and validate the models.

[0149]

[0176] Tissue segmentation. After slicing slides into 256x256 pixel patches at 10x zoom, an artifact classifier was developed by training a ResNet-18 to classify whether a patch showed usable tissue or was blank or showed artifacts. The artifact classifier was trained for 25 epochs and optimized using SGD with a learning rate of 0.001. The learning rate was decreased by 10% every 7 epochs. 3661 patches (tissue vs. non-tissue) were manually annotated, of which the classifier was trained on 3366, achieving a validation accuracy of 97.6% on the remaining 295. This artifact classifier was then used to segment tissue sections during image quilting.

[0150]

[0177] Nuclear density sampling. Due to the large variability in staining intensity and stain degradation, readily available pre-trained models for nuclei detection and segmentation were unable to accurately detect nuclei in the majority of slides. To overcome this, we trained a nuclear detector using the object detection method in YOLOv5 (github.com / ultralytics / yolov5).

[0151]

[0178] To train the YOLOv5 model, a selection of representative 34 sample slides were manually labeled using the QuPath image analysis platform. First, tissues were segmented using the "Simple tissue detection" module. Next, cells were segmented using the "Watershed cell detection" module, with parameters manually tuned for each slide. A YOLOv5-Large model was then trained on annotations from 29 slides and evaluated on the remaining 5. The model was trained using 256 × 256 patches at 10x zoom.

[0152]

[0179] Model performance metrics (AUC). For each model and each outcome, we estimated a time-dependent receiver operating characteristic that takes into account competing events using the R-package timeROC, which sweeps a threshold t in the interval [0,1] and compares the model predictions with

number

[0153]

[0180] NCCN Model. The NCCN model was coded according to the algorithm in Figure 8 and stratified patients into low-, intermediate-, and high-risk groups using three clinical variables: Gleason, t-stage, and baseline PSA.

[0154]

[0181] Example 2: Evaluating the algorithmic fairness of the MMAI algorithm

[0155]

[0182] In this example, we describe the algorithmic fairness, i.e., performance of a multimodal AI (MMAI) model utilizing clinical and digital histopathology data in AA and non-AA prostate cancer patients treated on the NRG / RTOG prostate cancer trial.

[0156]

[0183] Prostate cancer (PCa) is the second leading cause of cancer-related deaths in men, and it is well known that African American (AA) men experience a greater burden of disease due to their more advanced disease course and younger age at diagnosis.

[0157]

[0184] There are limitations to using population-based datasets and retrospective studies to study prognostic outcomes and associated disparities in AA men with PCa because these data often do not adequately represent AA men in the cohort and lack the long-term follow-up outcomes necessary to develop prognostic risk models. Ideally, studies including large numbers of AA men enrolled in prospective randomized controlled trials (RCTs) would allow for the assessment of long-term prognostic outcomes with a representative AA sample while minimizing the risk of selection bias and other confounding factors. The Radiation Oncology Group (RTOG) and NRG Oncology cooperative groups prioritize recruiting a representative proportion of AA patients to prostate cancer clinical trials. Large-scale RCTs provide an opportunity to model differences in long-term prognostic outcomes and risk, which will lead to more nuanced clinical decision-making for treatment selection and treatment optimization in AA men with PCa.

[0158]

[0185] method

[0159]

[0186] Explanation of multi-cohort data

[0160]

[0187] With permission from NRG Oncology, a National Clinical Trials Network (NCTN) group funded by the National Cancer Institute (NCI), we collected a unique dataset from five large, multinational, randomized phase III clinical trials (NRG / RTOG-9202, 9408, 9413, 9910, 0126) in men with localized prostate cancer. All patients received definitive radiation therapy (RT) with or without predefined use of androgen deprivation therapy (ADT). The duration of RT combined with short-term ADT was 4 months, the duration of intermediate ADT was 36 weeks, and the duration of long-term ADT was 28 months. In total, 7,752 eligible participants were randomized to these five trials.

[0161]

[0188] Multimodal AI (MMAI) model explained

[0162]

[0189] Four MMAI models described herein were trained and deployed on the dataset. The MMAI models jointly learned relevant features from each patient's digital histopathology slides and clinical data. Image vector representations were learned and extracted from tissue sections of biopsy slides through self-supervised pre-training. The combination of these image feature vectors and feature vectors derived from clinical data were fed into a multimodal fusion pipeline to output risk scores for desired clinical endpoints including distant metastasis (DM) and prostate cancer specific mortality (PCSM). The cohort was split 80 / 20 into development and validation datasets, and the MMAI models were trained and optimized on the development set and then validated on the remaining validation set. The first MMAI to predict risk of DM and PCSM was as described in Example 1. The second MMAI model to predict risk of DM and PCSM included multimodal learning based on a multi-instance learning-based neural network with an attention mechanism using the time to event of the desired clinical endpoint as a label, as detailed herein below. A schematic of the second set of MMAI models is shown in Figure 9. A comparison of study findings based on the MMAI model described in Example 1 is also discussed below.

[0163]

[0190] Methods for developing multimodal deep learning models

[0164]

[0191] Model Development Overview

[0165]

[0192] The five studies were stratified by 1) study type, 2) distant metastasis status, and 3) patient clinical risk and randomly split into a development set (80%) and a validation set (20%) for model development and validation, respectively. Each MMAI model was trained and optimized on the development set by a 5-fold cross-validation scheme, and the development set was further split into a training subset and a tuning subset at each fold. The training subset was used to update the learnable model parameters, while the tuning subset was used to monitor unbiased performance during training and tune the hyperparameters. As this training process produced five separate models, an ensemble model was then constructed by averaging over the five model outputs to form a single risk score for each patient.

[0166]

[0193] Clinical Data Preprocessing

[0167]

[0194] All clinical variables (T stage, Gleason score, primary / secondary Gleason pattern) were treated as numerical variables and standardized based on the mean and standard deviation of the training data. Missing clinical data were imputed using the k-Nearest Neighbors method, and missing values ​​were imputed using the mean of the five nearest neighbors found in the training set.

[0168]

[0195] Image feature extraction model development

[0169]

[0196] Effective learning of relevant features from a variable number of digital histopathology slides involves both image standardization and self-supervised pre-training. For each patient, all pre-treatment tissue sections in the patient's biopsy slide were segmented into patches of 256 × 256 pixels across each RGB channel. A tissue classifier was developed by training ResNet-18 to classify whether a patch showed usable tissue or blank or artifact. The artifact classifier was trained for 25 epochs and optimized using stochastic gradient descent with a learning rate of 0.001. The learning rate was decreased by 10% every 7 epochs. 3661 patches (tissue vs. non-tissue) were manually annotated, of which the classifier was trained on 3366, achieving a validation accuracy of 97.6% on the remaining patches. The artifact classifier was used to segment tissue sections and to remove low-quality images during image feature generation.

[0170]

[0197] The patches filtered by the artifact classifier were then used to train a self-supervised model to learn tissue morphological features useful for downstream tasks. The ResNet-50 model was used in conjunction with the MoCo-v2 training protocol (parameters: learning rate = 0.03 with cosine learning rate schedule for 200 epochs, moco-t = 0.2, multi-layer perceptron head, batch size 256, default MoCo-v2 parameters for augmentation) to train the self-supervised model. Images of patients with Gleason primary ≥ 4 were used to pre-train the corresponding self-supervised model to effectively learn the relevant tissue morphological features. Once the self-supervised pre-training was completed, all patches containing available tissues from the whole slide images were fed into the self-supervised pre-trained ResNet-50 model to generate a 128-dimensional vector representation for each patch.

[0171]

[0198] Downstream multimodal prognostic model development

[0172]

[0199] The downstream prognostic model took the image feature tensor, which is the concatenation of feature vectors from all patches of each patient, and the preprocessed clinical data as inputs for each patient. In the second model, an attention multi-instance learning network was employed to learn weights for each image feature vector from each patch. A single 128-dimensional image vector was generated from each patient's image feature tensor by taking the weighted sum of the image vectors of all patches from the same patient. The weights were then learned by an attention mechanism. All preprocessed clinical data were considered as numerical variables and processed through a single linear layer to learn a 6-dimensional clinical vector representation. The concatenation of the 128-dimensional image vector and the 6-dimensional clinical vector was further processed in a neural network-based joint fusion pipeline to effectively learn from both clinical and image data and output a risk score for the outcome of interest (Figure 9).

[0173]

[0200] For training purposes, we used the negative log partial likelihood, and the model prediction score was the estimated relative log hazard. Binary indicators of the event of interest and the corresponding time to event were used as labels for model development. The negative log partial likelihood loss was parameterized by the model weights θ and formulated as follows:

[0174]

number

[0175]

[0201] Model performance metrics across subgroups

[0176]

[0202] Distant metastasis (DM), prostate cancer-specific mortality (PCSM), biochemical failure (BF), and overall survival (OS) were evaluated. DM and PCSM were selected because they are strongly correlated with prostate cancer incidence and mortality, and are more likely to represent clinically useful measures, reflecting the greater burden of prostate cancer in the AA population compared with non-AA populations.

[0177]

[0203] Model performance between AA and non-AA PCa patients was assessed for each MMAI model for DM MMAI versus DM endpoints and PCSM MMAI versus PCSM endpoints. Patients with unknown or missing racial status were excluded from the analysis cohort. All assessed endpoints were time-to-event outcomes, patients lost to follow-up were censored, and death before experiencing the event of interest was considered a competing event. Racial subgroup analysis was performed by comparing the distribution of clinical variables and MMAI scores (median and interquartile range (IQR) for continuous variables, percentage for reported categorical variables) and assessing the prognostic ability of the MMAI models in AA and non-AA men. Both the MMAI continuous score (every 0.05 score increase) and categorized risk groups were used to assess the fairness of the algorithm. For MMAI categorical groups, model scores were ranked by deciles, and then deciles with similar prognosis were grouped into three groups based on the corresponding endpoints on which the MMAI models were originally trained. For example, the DM MMAI model was grouped into deciles 1–4, 5–9, and 10, and the PCSM MMAI model into deciles 1–5, 6–9, and 10. Model performance was compared with Fine-Gray or Cox Proportional Hazard models using DM and PCSM as primary endpoints and BF and OS as secondary endpoints. Kaplan-Meier or cumulative incidence estimates were calculated and compared using log-rank or Gray tests. p values ​​were then post hoc adjusted using the Bonferroni method for pairwise cumulative incidence comparisons between subgroups.

[0178]

[0204] result

[0179]

[0205] A schematic of the pooling of eligible clinical trial participants is shown in Figure 10. There were a total of 948 AA patients (16.6%), 4,731 non-AA patients (82.9%), and 29 patients (0.5%) with unknown or missing racial status; these 29 patients were excluded from all analyses.

[0180]

[0206] In both the development and testing cohorts, the median ages of AA and non-AA patients were 69 and 71 years, respectively. In the development cohort, AA patients had higher baseline median PSA (13 vs. 10 ng / mL), more T1-T2a (61 vs. 55%), more Gleason 8-10 (17 vs. 13%), and more National Comprehensive Cancer Network (NCCN) high risk (42 vs. 35%) compared to non-AA patients. With the exception of T stage, similar findings were observed in the testing cohort (Figure 11). For all MMAI models in both the development and testing cohorts, the distributions overlapped between AA and non-AA subgroups (Figure 12). The median (IQR) scores for the DM optimized model (DM MMAI) were 0.36 (0.26-0.47) for AAs and 0.36 (0.26-0.49) for non-AAs in the development cohort and 0.38 (0.29-0.47) vs. 0.37 (0.27-0.48) for the testing cohort. The median DM MMAI scores were 0.38 (0.30-0.38) for AAs and 0.40 (0.32-0.50) for non-AAs in the development cohort and 0.40 (0.32-0.49) vs. 0.40 (0.32-0.50) for the testing cohort (Figure 13). Findings for the first MMAI model are reported in Figure 14.

[0181]

[0207] Performance of Multi-Modal AI (MMAI) Models in Subgroups

[0182]

[0208] In the test cohort, the DM MMAI model score showed a strong prognostic signal for DM in both AA (hazard ratio [HR] per 0.05 score increase for DM: 1.2, p = 0.007) and non-AA subgroups (1.4 for DM, p < 0.001) (Figure 15A). Similarly, the PCSM MMAI score showed a strong prognostic signal for PCSM in both AA (HR per 0.05 score increase for PCSM: 1.2, p = 0.01) and non-AA subgroups (1.5 for PCSM, p < 0.001) (Figure 15B). All original models showed similar results in both AA and non-AA subgroups (Figures 16 and 17).

[0183]

[0209] The cumulative incidence rates of the racial subgroups were compared in the entire cohort. The estimated DM rates after 10 years were 5% (3%-6%) in the AA subgroup and 7% (6%-8%) in the non-AA subgroup (Figure 18A). Both MMAI models were able to risk stratify patients within the AA subgroup and within the non-AA subgroup (Figure 19A and Figure 19B). In the test cohort, for the DM MMAI models, the estimated 5-year DM rates in the AA subgroup were 3% (95%CI:0%-6%), 8% (95%CI:3%-14%), and 20% (95%CI:2%-38%), and in the non-AA subgroup were 1% (95%CI:0%-1%), 5% (95%CI:3%-7%), and 23% (95%CI:14%-32%). Adjusted pairwise comparisons between AA and non-AA for different risk groups were not statistically significant (p-values ​​= 0.36, 1.00, 1.00, respectively). Similarly, for the PCSM MMAI models, the estimated 10-year PCSM rates were 5% (95%CI: 0%-10%), 8% (95%CI: 2%-14%), and 30% (95%CI: 9%-51%) for the AA subgroup, and 1% (95%CI: 0%-3%), 8% (95%CI: 5%-11%), and 19% (95%CI: 11%-28%) for the non-AA subgroup (Figure 18B). Adjusted pairwise comparisons between AA and non-AA for different risk groups were not statistically significant (p-values ​​= 1.00, 1.00, 1.00, respectively). The original MMAI models showed similar results for both models, for both AA and non-AA subgroups (Figures 20A and 20B).

[0184]

[0210] Consideration

[0185]

[0211] AI-based biomarkers can help physicians tailor treatment suggestions for patients with prostate cancer. However, AA men may be underrepresented in population data used to develop novel biomarkers. Previous biomarker studies have questioned the value of biomarkers developed primarily in non-AA cohorts when applied to AA men. This paucity of genomic data including the AA population may exacerbate known health disparities experienced by this population by algorithmically encoding such inequities. These observations highlight the need to use more clinically relevant endpoints and apply rigorous methods to control for selection bias to examine biomarker performance across racial boundaries.

[0186]

[0212] Sufficient data was used to train the MMAI model, and the prognostic performance of the AI ​​model was found to be comparable between AA and non-AA subgroups. The DM MMAI and PCSM MMAI models performed similarly in both AA and non-AA patient populations, demonstrating algorithmic fairness in the application of the tool. This approach supports the use of these AI biomarkers to personalize treatment selection for men with prostate cancer across racial groups. In addition, this analysis provides a methodology for integrating the principles of algorithmic fairness in routine biomarker discovery and validation.

[0187]

[0213] Example 3: Risk stratification of prostate cancer patients using the MMAI

[0188]

[0214] In this example, the ability of the multimodal artificial intelligence (MMAI) model described herein to stratify patients into risk groups was compared to that of the National Comprehensive Cancer Network (NCCN) risk stratification schema.

[0189]

[0215] From the dataset described in Example 1, the 5,569 individuals for whom a definitive NCCN risk classification could be made were sorted into one of ten deciles based on their 10-year risk of distant metastasis (DM10-yr) according to their corresponding MMAI scores predicted by the MMAI model described in Example 2. Each decile was then stratified into one of three MMAI prognostic risk groups: "MMAI low", "MMAI intermediate", and "MMAI high" based on their MMAI DM10-yr scores (<10%, 10%-25%, and >25%, respectively) (Figure 21). The baseline characteristics of each MMAI prognostic risk group are shown in Figure 22. Figure 23 shows, for each MMAI prognostic risk group (rows), the number of individuals classified as "low", "intermediate" (favorable and unfavorable), or "high" (NCCN high or very high) according to the NCCN risk schema.

[0190]

[0216] To determine whether MMAI could better prognosticate distant metastasis at 10 years than NCCN risk classification, the probability of DM10-yr was calculated again for each individual using the MMAI model. Figure 24 shows the average 10-year risk of DM10-yr (confidence interval in brackets) for individuals with a given NCCN and MMAI classification. As shown in Figure 24, the risk of DM10-yr is roughly the same for individuals classified as low risk by NCCN and MMAI. However, the MMAI model can better determine which individuals are actually at high risk of DM among those classified as intermediate or high risk by the NCCN scheme. As shown in Figure 24, the subset of individuals with NCCN "intermediate" classification classified as MMAI "high" had an MMAI predicted probability of DM10-yr of 60%, whereas the subset of individuals with NCCN "high" classification classified as MMAI "high" had an MMAI predicted probability of DM10-yr of 36%. Thus, MMAI-based risk classification was able to stratify NCCN "medium" individuals with risk of metastasis. As shown in Figure 25A, the MMAI model identified 6 times more patients with the lowest risk of metastasis than NCCN. Approximately 83% of the overall NCCN "medium" risk individuals had MMAI low scores and therefore low risk of metastasis, whereas approximately 13.2% of the overall NCCN "high" risk individuals had MMAI low scores. Approximately 28% of the overall individuals classified as "high" risk by NCCN had MMAI DM10-yr scores of "high" (risk ≧30%) (Figure 25B).

[0191]

[0217] Thus, compared to the NCCN classification, the MMAI system disclosed herein can better stratify individuals at risk of prostate cancer metastasis.

[0192]

[0218] Example 4: External validation of the MMAI model

[0193]

[0219] This example describes the validation of the multimodal artificial intelligence (MMAI) model for prognosticating prostate cancer risk described herein.

[0194]

[0220] patient

[0195]

[0221] NRG / RTOG-9902 enrolled 397 men with high-risk localized prostate cancer (PCa) randomized to receive long-term androgen suppression (AS) with radiotherapy (RT) alone (AS+RT) or with adjuvant chemotherapy (CT) (AS+RT+CT) between January 2000 and October 2004. CT consisted of paclitaxel, estramustine, and oral etoposide administered for 21 days starting 28 days after 70.2 Gy of RT. The AS regimen consisted of luteinizing hormone-releasing hormone (LHRH) for 24 months starting 2 months before RT, plus an oral antiandrogen for 4 months before and during RT. Men with PSA 20-100 and Gleason score ≥7 or clinical stage ≥T2 and Gleason score ≥8 were enrolled. The 10-year results were not statistically significant (p>0.05 for all listed endpoints) between the two treatment groups for overall survival (OS), biochemical failure (BF), local progression (LP), distant metastasis (DM), or disease-free survival (DFS). Therefore, in this example, all men were pooled into one cohort regardless of treatment group.

[0196]

[0222] Sample processing and scanning

[0197]

[0223] RTOG-9902 pre-treatment biopsy slides were digitized at 20x resolution by NRG Oncology using a Leica Biosystems Aperio AT2 digital pathology scanner. Histopathology images were reviewed for quality and clarity by NRG Biobank operators and the artificial intelligence data capture team. Low-quality images were removed using a pre-built artifact classifier.

[0198]

[0224] Explaining the Multimodal AI (MMAI) model

[0199]

[0225] An MMAI architecture with six MMAI algorithms was developed and validated using five Phase III NRG trials (RTOG9202, 9408, 9413, 9910, 0126) utilizing both digital histopathology slides and clinical data from each patient, as described in Example 2 and shown in Figure 9. This MMAI architecture resulted in risk scores for two locked MMAI algorithms optimized for the desired clinical endpoints of distant metastasis (DM) and prostate cancer-specific mortality (PCSM).

[0200]

[0226] Endpoints

[0201]

[0227] The primary endpoints of the study were (1) time to DM, defined as the number of days from randomization to the date of distant metastasis, and (2) time to PCSM, defined as the number of days from randomization to death from prostate cancer.

[0202]

[0228] Secondary endpoints were time to biochemical failure (BF), defined as the number of days from randomization to biochemical failure (prostate-specific antigen (PSA) failure or initiation of salvage hormone therapy, whichever came first), and overall survival (OS), defined as the number of days from randomization to death.

[0203]

[0229] Subjects lost to follow-up before experiencing the event of interest were censored at last follow-up. For DM, PCSM, and BF, death before experiencing the event of interest was considered a competing event.

[0204]

[0230] Statistical analysis

[0205]

[0231] The digital pathology evaluable population (DPEP) was defined as subjects randomized to RTOG-9902 with high-quality histopathology data and no missing baseline clinical variables, including age, clinical T stage, primary and secondary Gleason grades, and baseline PSA, to generate an MMAI algorithm score for analysis.

[0206]

[0232] Baseline demographic and clinical characteristics were summarized descriptively for the DPEP and intention to treat (ITT) populations and compared between the DPEP population and a subgroup of patients from the ITT population without high-quality histopathology data. Descriptive summaries were prepared using counts and portions (%) for categorical variables and medians and interquartile ranges (IQR) for continuous variables. P values ​​were calculated using the Wilcoxon rank-sum test for continuous variables and Pearson's chi-square test or Fisher's exact test for categorical variables.

[0207]

[0233] The prognostic performance of the MMAI algorithm was evaluated by univariate and multivariate analyses. Fine and Gray regression was used to estimate subdistribution hazard ratios (sHRs) and 95% confidence intervals (CIs) for DM, PCSM, and BF endpoints. Cox Proportional Hazard regression was used to estimate HRs and 95% CIs for the OS endpoint. MMAI algorithm scores were divided by quartiles and summarized using cumulative incidence curves to provide 5- and 10-year estimated DM and PCSM rates and corresponding two-sided 95% CIs. Tests for MMAI-treatment interactions were also performed as exploratory analyses.

[0208]

[0234] All statistical analyses were performed using R version 4.1.2 (R Foundation for Statistical Computing, Vienna, Austria). All statistical tests were two-sided, with a significance level of 0.05. Findings from prognostic model validation were reported using the TRIPOD reporting standards.

[0209]

[0235] result

[0210]

[0236] participants

[0211]

[0237] MMAI algorithm scores were generated for 318 of the original 397 clinical trial patients enrolled in NRG / RTOG-9902 (slides were available for 85% of the total study cohort; of these, 5.6% could not be included due to poor image quality). Figure 26 shows the flow of patients from the NRG / RTOG 9902 clinical trial to the DPEPs included for model validation. Baseline characteristics of the study DPEPs are shown in Figure 27. The evaluable population included men with a median baseline PSA of 23.0 ng / mL, of whom 32% had cT3–4 disease, 67% had Gleason Grade group 4 or 5 disease, and 54% had NCCN high-risk features >1. At a median follow-up of 10.1 years, 42 men experienced DM and 29 experienced PCSM. Baseline characteristics among patients for whom MMAI algorithm scores could be obtained were not statistically significantly different compared to the 62 patients excluded from this validation study. Similarly, within the DPEP, there were no differences in baseline characteristics between the two treatment arms. The median (IQR) scores were 0.54 (0.44-0.62) for the DM-optimized algorithm (DM MMAI) and 0.53 (0.47-0.60) for the PCSM-optimized algorithm (PCSM MMAI), and both scores were similar between the two NRG / RTOG9902 treatment groups (Figure 27B).

[0212]

[0238] Model performance

[0213]

[0239] Compared with clinical and pathological factors, the MMAI algorithm was significantly prognostic across outcome measures. In univariate analysis, the DM MMAI algorithm continuous score was statistically associated with DM endpoints (sHR 2.33, 95% CI 1.60-3.38, p<0.001), and the PCSM MMAI algorithm was statistically associated with PCSM endpoints (HR 2.63, 95% CI 1.70-4.08, p<0.001) (Figure 28A). When evaluating non-optimized secondary endpoints, the DM MMAI was statistically significantly associated with the risk of BF, PCSM, and OS. Similarly, the PCSM MMAI was statistically significantly associated with the risk of DM and OS (Figure 28B).

[0214]

[0240] For DM endpoints, the DM MMAI was prognostic in most clinical subgroups, including both treatment groups, age, non-African American, both PSA groups, Gleason 8-10, clinical T stage, and patients with one NCCN high-risk factor (Figure 29A). Similarly, for PCSM endpoints, the PCSM MMAI was prognostic in most subgroups, including both treatment groups, age, both racial subgroups, PSA < 20 ng / mL, Gleason 8-10, clinical T stage, and patients with any NCCN high-risk factor (Figure 29B). In multivariate analyses (Figures 30A-B and 31A-B), controlling for individual clinical factors such as age, baseline PSA, Gleason, T stage, and number of NCCN high-risk factors was consistently significantly prognostic for both the DM MMAI and the PCSM MMAI.

[0215]

[0241] Using quartile splits for the DM MMAI, the estimated 5-yr and 10-yr DM rates for patients in the bottom 75% (Q1-Q3) were 4% (95%CI 1%-6%) and 7% (95%CI 4%-10%), while the estimated 5-yr and 10-yr DM rates for those in the highest quartile (Q4) were 19% (95%CI 10%-28%) and 32% (95%CI 21%-43%), with sHRs of 5.1 (95%CI 2.7-9.3, p<0.001) (Figure 32A). Similar results were observed for the MMAI for PCSM (sHR 4.1, 95%CI 2.0-8.4, p<0.001) (Figure 32B).

[0216]

[0242] There were no statistically significant interactions between DM MMAI quartile group (Q4 vs. Q1-Q3) and CT treatment effect (interaction p=0.08) or between PCSM MMAI quartile group and CT treatment effect (interaction p=0.79) (Figures 33A and 33B). Among patients in the top 25% ranked by DM MMAI, the 5-year estimated absolute benefit with additional CT use was 14% and the 10-year estimated absolute benefit was 18% (Figure 33B).

[0217]

[0243] Consideration

[0218]

[0244] The prognostic ability of the MMAI classifier previously developed with men from five phase III PCa trials (NRG / RTOG9202, 9408, 9413, 9910, 0126) was further validated with an external validation set of men from NRG / RTOG9902, which enrolled men at high risk of disease progression. The parent trial, NRG / RTOG9902, did not provide statistically significant clinical outcomes when compared with the treatment arms, so the validation sample was treated as one single cohort (study arms were well balanced and mean MMAI classifier scores were similar between treatment arms). The differences between low and high risk groups by the MMAI classifier for DM and PCSM were large and statistically significant even in the NCCN high and very high risk populations. In multivariate analysis, the MMAI score was independently prognostic, even after controlling for variables known to be associated with prognostic risk (patient age, Gleason score, T stage). Associations of the MMAI classifier with DM and PCSM within subgroups suggested additional discriminatory and prognostic power of the MMAI across the continuum of high- and very-high-risk disease.

[0219]

[0245] While preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. The present invention is not intended to be limited by the specific examples provided herein. Although the present invention has been described with reference to the foregoing specification, the description and illustration of the embodiments herein should not be construed in a limiting sense. Numerous variations, changes, and substitutions will occur to those skilled in the art without departing from the present invention. Furthermore, it should be understood that all aspects of the present invention are not limited to the specific depictions, configurations, or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the present invention described herein may be employed in the practical application of the present invention. Accordingly, the present invention is intended to cover such alternatives, modifications, variations, or equivalents. That is, it is intended that the following claims define the scope of the present invention, and that methods and structures within the scope of these claims and their equivalents are hereby covered.

Claims

1. 1. A method for assessing cancer in a subject, comprising: (a) obtaining a dataset comprising image data and tabular data from the subject; (b) processing the dataset with a trained algorithm to classify the dataset into one of a plurality of categories, wherein the classifying step comprises applying an image processing algorithm to the image data; (c) assessing the cancer in the subject based at least in part on the one of the plurality of categories classified in (b); and A method comprising:

2. The method of claim 1 , wherein the trained algorithm is trained using self-supervised learning.

3. The method of claim 1 , wherein the trained algorithm comprises a deep learning algorithm.

4. The method of claim 1 , wherein the trained algorithms include a first trained algorithm that processes the image data and a second trained algorithm that processes the tabular data.

5. The method of claim 4 , wherein the trained algorithms further include a third trained algorithm that processes outputs of the first and second trained algorithms.

6. 4. The method of any one of claims 1 to 3, wherein the cancer is bladder cancer, breast cancer, cervical cancer, colorectal cancer, gastric cancer, kidney cancer, liver cancer, ovarian cancer, pancreatic cancer, prostate cancer, or thyroid cancer.

7. The method of claim 6, wherein the cancer is prostate cancer.

8. The method of claim 1 , wherein the tabulated data comprises clinical data of the subject.

9. 10. The method of claim 8, wherein the clinical data for the subject comprises laboratory data, therapeutic intervention, or long-term outcome.

10. The method of claim 1 , wherein the image data comprises digital histopathology data.

11. The method of claim 10 , wherein the histopathology data comprises images derived from a biopsy sample of the subject.

12. The method of claim 11 , wherein the image is obtained by microscopic examination of the biopsy sample.

13. 13. The method of any one of claims 10 to 12, wherein the digital histopathology data is obtained from the subject before the subject undergoes treatment.

14. 14. The method of claim 13, wherein the treatment comprises radiation therapy (RT).

15. 15. The method of claim 14, wherein the RT comprises a preset use of short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), dose-escalated RT (DE-RT), or a combination thereof.

16. 13. The method of any one of claims 10 to 12, wherein the digital histopathology data is obtained from the subject after the subject has undergone a treatment.

17. 17. The method of claim 16, wherein the treatment comprises radiation therapy (RT).

18. 18. The method of claim 17, wherein the RT comprises a preset use of short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), dose-escalated RT (DE-RT), or a combination thereof.

19. The method of any one of claims 1 to 3 and 10 to 12, further comprising processing the image data using an image segmentation, image stitching, object detection algorithm, or a combination thereof.

20. The method of any one of claims 1 to 3 and 10 to 12, further comprising extracting features from the image data.

21. 1. A method for assessing cancer in a subject, comprising: (a) acquiring a dataset comprising at least image data from the subject; (b) processing the dataset with a trained algorithm to classify the dataset into one of a plurality of categories, the classifying comprising applying an image processing algorithm to the image data, the trained algorithm being trained using self-supervised learning; (c) assessing the cancer in the subject based at least in part on the one of the plurality of categories classified in (b); A method comprising:

22. 22. The method of claim 21 , wherein the trained algorithm comprises a deep learning algorithm.

23. 22. The method of claim 21, wherein the cancer is bladder cancer, breast cancer, cervical cancer, colorectal cancer, gastric cancer, kidney cancer, liver cancer, ovarian cancer, pancreatic cancer, prostate cancer, or thyroid cancer.

24. 24. The method of claim 23, wherein the cancer is prostate cancer.

25. 22. The method of claim 21, wherein the image data comprises digital histopathology data.

26. 26. The method of claim 25, wherein the histopathology data comprises images derived from a biopsy sample of the subject.

27. 27. The method of claim 26, wherein the image is obtained by microscopic examination of the biopsy sample.

28. 28. The method of any one of claims 25 to 27, wherein the digital histopathology data is obtained from the subject before the subject undergoes a treatment.

29. 29. The method of claim 28, wherein the treatment comprises radiation therapy (RT).

30. 30. The method of claim 29, wherein the RT comprises a preset use of short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), dose-escalated RT (DE-RT), or a combination thereof.

31. 28. The method of any one of claims 25 to 27, wherein the digital histopathology data is obtained from the subject after the subject has undergone a treatment.

32. 32. The method of claim 31 , wherein the treatment comprises radiation therapy (RT).

33. 33. The method of claim 32, wherein the RT comprises a preset use of short-term androgen deprivation therapy (ST-ADT), long-term ADT (LT-ADT), dose-escalated RT (DE-RT), or a combination thereof.

34. 28. The method of any one of claims 21 to 27, further comprising processing the image data using an image segmentation, image stitching, or object detection algorithm.

35. 28. The method of any one of claims 21 to 27, further comprising extracting features from the image data.

36. 28. The method of any one of claims 21 to 27, wherein the dataset comprises image data and tabular data.

37. 37. The method of claim 36, wherein the trained algorithms include a first trained algorithm that processes the image data and a second trained algorithm that processes the tabulated data.

38. 38. The method of claim 37, wherein the trained algorithms further include a third trained algorithm that processes outputs of the first and second trained algorithms.

39. 37. The method of claim 36, wherein the tabulated data comprises clinical data for the subject.

40. 40. The method of claim 39, wherein the clinical data comprises laboratory data, therapeutic intervention, or long-term outcome.

Images