Method and system for AI-based immune profiling for cancer patient stratification

Abstract

Systems and methods for predicting a patient's response to a therapeutic agent are disclosed. The systems and methods may include a data store for storing images of a tissue sample from a patient, the images including one or more features of the tissue sample, a computing device including a first neural network model, a second neural network model, and / or a third neural network model, and a display system configured to display results indicative of the patient's response status.

Description

[Technical Field]

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 403,652, filed September 2, 2022, which is incorporated herein by reference in its entirety. [Background technology]

[0002] Cancer immunotherapy is an emerging field of cancer treatment whose primary goal is to utilize the patient's own immune system to recognize and destroy tumor cells. Various forms of immunotherapy have been developed and are in various stages of preclinical and clinical development. Forms of immunotherapy include, but are not limited to, monoclonal antibodies and adoptive cell transfer.

[0003] Checkpoint blockade, an aspect of cancer immunotherapy, has shown efficacy in both clinical and preclinical studies as an adjuvant and alternative to conventional cancer therapy. The effectiveness of checkpoint blockade arises from relieving T cells from the inhibitory effects of checkpoint molecules. T cells in the tumor microenvironment (TME) increase the expression of checkpoint molecules, such as programmed cell death 1 (PD-1) and programmed cell death 1 ligand (PD-L1), in response to multiple TME-derived factors.

[0004] Cancer therapeutics, including checkpoint inhibitors (including monoclonal antibodies), present a problem when the therapeutics lack effectiveness in patients with either primary or acquired resistance, which can occur when a cancer does not respond to immunotherapy strategies, or, in the case of checkpoint inhibitors, when cancer cells fail to express the checkpoint inhibitor and thereby do not provide a target for the therapeutic. Summary of the Invention

[0005] An embodiment of the present disclosure is a system for predicting patient response to a cancer immunotherapy therapeutic, the system comprising: a data store for storing images of tissue samples from patients, the images including one or more features of the tissue samples; and a computing device communicatively coupled to the data store, the computing device comprising: a first neural network model configured to generate one or more tumor mutation burden (TMB) indicators from the one or more features; a second neural network model configured to generate one or more programmed death-ligand 1 (PD-L1) expression level indicators from the one or more features and / or to generate one or more programmed death-1 (PD-1) expression level indicators from the one or more features; the computing device includes a third neural network model configured to calculate a tumor necrosis factor (TMB) indicator, one or more PD-L1 expression level indicators and / or one or more PD-1 expression level indicators, and / or one or more TIL-associated metrics; or a fourth neural network model configured to predict a patient's response status to a cancer immunotherapy therapeutic according to one or more TMB indicators, one or more PD-L1 expression level indicators and / or one or more PD-1 expression level indicators, and / or one or more TIL-associated metrics; and a display system communicatively connected to the computing device and configured to display results indicative of the patient's response status.

[0006] In a specific embodiment, the prediction of a patient's response status to a cancer immunotherapy therapeutic is made according to one or more medical history features of the patient. The third neural network model may include a segmentation algorithm configured to generate an identification of a tumor region in the tissue sample based on the one or more features, and the identification is used to further generate one or more TIL-associated metrics. In a specific embodiment, the segmentation algorithm is configured to predict a TIL mask, which is used in combination with the tumor mask to calculate one or more TIL-associated metrics. In particular embodiments, the one or more TIL-related metrics include a predicted number of TIL clusters in the identified tumor region, a predicted size of TIL clusters in the identified tumor region, a predicted TIL cluster spread in the identified tumor region, a total number of TILs in the identified tumor region, the distance between TIL cells compared to the distance between tumor cells, the distance of TIL cells from tumor cells, the average size of TIL cells, stromal TIL% (sTIL%) (which may be calculated as the percent of stroma within the identified tumor region in the tissue sample that is infiltrated by TILs), and / or intra-tumor TIL% (iTIL%), which may be calculated as the percent of the identified tumor region in the tissue sample that is infiltrated by TILs. In various embodiments, the first neural network model includes a classification algorithm, and the one or more TMB indicators include a TMB indicator for the tissue sample that is either a TMB-positive classification or a TMB-negative classification. If the tissue sample has a TMB value above a predetermined TMB threshold, the classification algorithm may generate a TMB-positive classification. In some cases, the first neural network model comprises a regression algorithm and the one or more TMB indicators comprise a TMB indicator that is a quantitative TMB score. In various embodiments, the second neural network model comprises a classification algorithm and the one or more PD-L1 expression level indicators comprise a PD-L1 expression level indicator that is either a PD-L1 expression positive classification or a PD-L1 expression level negative classification.The second neural network model may comprise a classification algorithm, and the one or more PD-1 expression level indicators comprise a PD-1 expression level indicator that is either a PD-1 expression positive classification or a PD-1 expression level negative classification. In certain embodiments, the classification algorithm generates a PD-L1 expression positive classification when the tissue sample has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value. In some embodiments, the classification algorithm generates a PD-1 expression positive classification when the tissue sample has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value. The second neural network model may comprise a regression algorithm, and the one or more PD-L1 expression level indicators comprise a quantitative PD-L1 score. In certain embodiments, the second neural network model comprises a regression algorithm, and the one or more PD-1 expression level indicators comprise a quantitative PD-1 score. The quantitative PD-L1 score and / or the quantitative PD-1 score may be a tumor proportion score (TPS). The quantitative PD-L1 score and / or the quantitative PD-1 score may be a combined positive score (CPS).

[0007] In some embodiments, the third neural network model is further configured to generate one or more TIL masks from the one or more features, and the fourth neural network model is further configured to predict a patient's response status to a cancer immunotherapy therapeutic using the one or more TIL masks.

[0008] In various embodiments, the cancer immunotherapy therapeutic is a checkpoint inhibitor, and the checkpoint inhibitor can be a PD-L1 inhibitor or a PD-1 inhibitor. The optional tissue sample can be from the patient's bladder, and in some embodiments, the patient response to the cancer immunotherapy therapeutic is related to bladder cancer. The tissue sample can be from a patient known to have a PD-L1-positive cancer. The tissue sample can be from a patient known to have a PD-1-positive cancer. In specific embodiments, the tissue sample is from a patient known to have any one or more of non-small cell lung cancer, melanoma, Hodgkin's lymphoma, bladder cancer, renal cancer, and breast cancer. The cancer immunotherapy therapeutic, in specific embodiments, targets PD-L1 or PD-1. In some embodiments, the cancer immunotherapy therapeutic comprises one or more antibodies, one or more adoptive cell therapies, or a combination thereof. The one or more antibodies can comprise at least one of a monoclonal antibody, a bispecific antibody, or a trispecific antibody. In some embodiments, the one or more adoptive cell therapy therapies comprise immune cells expressing one or more engineered antigen receptors. In some embodiments, the one or more engineered antigen receptors comprise at least one of a chimeric antigen receptor, a non-native T cell receptor, or a combination thereof. In certain embodiments, the one or more adoptive cell therapy therapies comprise at least one of a T cell, a natural killer cell, a natural killer T cell, or a combination thereof.

[0009] In particular embodiments, the third neural network model includes a region / TIL segmentation algorithm and a cell segmentation algorithm. The cell segmentation algorithm may calculate one or more cell-based features. In particular embodiments, the one or more cell-based features include morphology, color (nuclear), color (extranuclear), texture (energy, correlation, contrast, homogeneity, dissimilarity, or angular second moment). In some embodiments, the cell segmentation algorithm generates a number of TIL cells or an average number of TIL cells. The cell segmentation algorithm may, in particular embodiments, generate an average tumor cell size, an average TIL cell size, an average distance between TIL cells, an average color of TIL cells, and / or an average distance between TIL cells compared to tumor cells.

[0010] Embodiments of the present disclosure include a method for predicting a patient's response to a cancer immunotherapy therapeutic, the method including: obtaining, by one or more processors, an image of a tissue sample from the patient, the image including one or more features of the tissue sample; generating, by the one or more processors via a first neural network model, one or more tumor mutational burden (TMB) indicators from the one or more features; generating, by the one or more processors via a second neural network model, one or more programmed death-ligand 1 (PD-L1) expression level indicators from the one or more features and / or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features; generating, by the one or more processors via a third neural network model, one or more tumor infiltrating lymphocyte (TIL)-associated metrics from the one or more features; and predicting, by the one or more processors via a fourth neural network model, the patient's response status to the cancer immunotherapy therapeutic using the one or more TMB indicators, the one or more PD-L1 expression level indicators, or the one or more PD-1 level indicators, and one or more associated metrics. In some embodiments, the method further comprises selecting for inclusion of the patient in a cohort of patients participating in the clinical trial according to the patient's response status, hi some embodiments, the method further comprises displaying on a display screen a graphical representation indicating the patient's response status.

[0011] In certain embodiments, the prediction of a patient's response status to a cancer immunotherapy therapeutic further uses one or more medical history features of the patient. In specific embodiments, the third neural network is a segmentation algorithm that identifies tumor regions in a tissue sample. In some embodiments, the segmentation algorithm predicts one or more TIL mask structure features in the identified tumor region to generate one or more TIL-related metrics. The predicted TIL mask structure features are a predicted number of TIL clusters in the identified tumor region, a predicted size of TIL clusters in the identified tumor region, or a predicted TIL cluster spread in the identified tumor region. In some embodiments, the one or more TIL mask structure features include a total number of TILs in the identified tumor region, a distance between TIL cells compared to the distance between tumor cells, a distance of TIL cells from tumor cells, an average color of TIL cells, and / or an average size of TIL cells. In some embodiments, the TIL% value is intratumoral TIL% (iTIL%), where iTIL% is calculated as the percent of the identified tumor region in the tissue sample that is infiltrated by TILs. In certain embodiments, the TIL% value is stromal TIL% (sTIL%), where the sTIL% value is calculated as the percentage of stroma within an identified tumor area in a tissue sample that is infiltrated by TILs.

[0012] In particular embodiments, the first neural network model is a classification algorithm that generates a TMB indicator, which is either a TMB-positive or TMB-negative classification for the tissue sample. In certain aspects, in specific embodiments, the classification algorithm generates a TMB-positive classification if the tissue sample has a TMB value above a predetermined TMB threshold. The first neural network model may be a regression algorithm that generates a TMB indicator, which is, in various embodiments, a quantitative TMB score, and the second neural network model is, in specific embodiments, a classification algorithm that generates a PD-L1 expression level indicator, which is either a PD-L1 expression-positive classification or a PD-L1 expression-negative classification. The second neural network model is, in specific embodiments, a classification algorithm that generates a PD-1 expression level indicator, which is either a PD-1 expression-positive classification or a PD-1 expression-negative classification. In some embodiments, the classification algorithm generates a PD-L1 expression-positive classification if the tissue sample has a PD-L1 expression level value above a predetermined PD-L1 expression level value. In specific embodiments, if the tissue sample has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value, the classification algorithm generates a PD-1 expression positive classification. In some embodiments, the second neural network model is a regression algorithm that generates a PD-L1 expression level indicator that is a quantitative PD-L1 score. In various embodiments, the second neural network model is a regression algorithm that generates a PD-1 expression level indicator that is a quantitative PD-1 score. In some embodiments, the quantitative PD-L1 score and / or PD-1 score is a tumor proportion score (TPS). In some embodiments, the quantitative PD-L1 score and / or PD-1 score is a combined positive score (CPS).

[0013] In some embodiments, the cancer immunotherapy therapeutic is a checkpoint inhibitor, e.g., a PD-L1 inhibitor or a PD-1 inhibitor.

[0014] In various embodiments, the method further includes generating, by the one or more processors, one or more TIL masks from the one or more features via a third neural network model, and predicting the patient's response status to the cancer immunotherapy therapeutic further includes using the one or more TIL masks.

[0015] In some embodiments, the tissue sample is from the bladder, and the patient response to the cancer immunotherapy therapeutic is associated with bladder cancer. In various embodiments, a determination is made that the patient is or will be a responder based on the response status, and in accordance with the determination, the patient is administered a therapeutically effective amount of the cancer immunotherapy therapeutic. In some embodiments, a determination is made that the patient is not or will not be a responder based on the response status, and in accordance with the determination, the patient is not administered the cancer immunotherapy therapeutic. In specific embodiments, the tissue sample is from a patient known to have a PD-L1-positive cancer or a PD-1-positive cancer. The tissue sample is from a patient known to have any of non-small cell lung cancer, melanoma, Hodgkin's lymphoma, bladder cancer, kidney cancer, or breast cancer.

[0016] In particular embodiments, the cancer immunotherapy therapeutic targets PD-L1 or PD-1. The cancer immunotherapy therapeutic, in specific embodiments, comprises one or more antibodies, adoptive cell therapy, immunomodulators, or combinations thereof, where the antibodies may be monoclonal antibodies, bispecific antibodies, or trispecific antibodies. In specific embodiments, the adoptive cell therapy comprises immune cells expressing one or more engineered antigen receptors. In some embodiments, the engineered antigen receptor is a chimeric antigen receptor, a non-natural T cell receptor, or a combination thereof. The adoptive cell therapy, in various embodiments, comprises T cells, natural killer cells, natural killer T cells, or a combination thereof.

[0017] The method may further include obtaining a sample from the patient and / or diagnosing the patient as having cancer.

[0018] In a specific embodiment, there is a method that includes obtaining, by one or more processors from a data source, an image of a tissue sample from a patient and one or more features of the tissue sample; generating, by the one or more processors via one or more neural network models, one or more of: (a) one or more tumor mutational burden (TMB) indicators from the one or more features; (b) one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features, or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features; and predicting, by the one or more processors via the one or more neural network models, one or more of the one or more TMB indicators, one or more PD-L1 expression level indicators, one or more PD-1 expression level indicators, and / or one or more TIL % values, a patient's response status to a cancer immunotherapy therapeutic; and determining whether to include the patient in a cohort of patients participating in a clinical trial based on the patient's response status.

[0019] In certain embodiments, there is a method that includes obtaining, by one or more processors from a data source, an image of a tissue sample from a patient and one or more features of the tissue sample; and generating, by the one or more processors via one or more neural network models, a prediction of the patient's response status to a cancer immunotherapy therapeutic using the image.

[0020] In a particular embodiment, a system for predicting a patient response to a cancer immunotherapy therapeutic comprises: a data store for storing images of a tissue sample from the patient, the images including one or more features of the tissue sample; and a computing device communicatively coupled to the data store, the computing device comprising: a first neural network model configured to generate one or more tumor mutation burden (TMB) indicators from the one or more features; a second neural network model configured to generate one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features and / or a second neural network model configured to generate one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features; a third neural network model configured to calculate one or more tumor-infiltrating lymphocyte (TIL)-associated metrics from the one or more features; a fourth neural network model configured to calculate one or more cell-based features from the one or more features; and a fifth neural network model configured to predict a patient's response status to the cancer immunotherapy therapeutic according to the one or more TMB indicators, one or more PD-L1 expression level indicators or one or more PD-1 expression level indicators, respectively, the one or more TIL-associated metrics, and the one or more cell-based features; and a display system communicatively coupled to the computing device and configured to display results indicative of the patient's response status.

[0021] In some embodiments, there is a method that includes obtaining, by one or more processors from a data source, images of a tissue sample from a patient and one or more features of the tissue sample; and training, by the one or more processors, one or more neural network models to predict the patient's response status to a cancer immunotherapy therapeutic using the images.

[0022] In a particular embodiment, there is a method for predicting a patient response to a cancer immunotherapy therapeutic, the method comprising: obtaining, by one or more processors, an image of a tissue sample from the patient, the image including one or more features of the tissue sample; generating, by the one or more processors, one or more tumor mutational burden (TMB) indicators from the one or more features; (optionally), generating, by the one or more processors, one or more programmed death-ligand 1 (PD-L1) expression level indicators or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features; generating, by the one or more processors, one or more tumor infiltrating lymphocyte (TIL)-associated metrics from the one or more features; and training, by the one or more processors, a fourth neural network model to predict the patient's response status to the cancer immunotherapy therapeutic based on one or more of the one or more TMB indicators (if utilized), one or more PD-L1 expression level indicators, one or more PD-1 level indicators, and the one or more associated metrics. [Brief explanation of the drawings]

[0023] For a more complete understanding of the principles disclosed herein and their advantages, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which: [Figure 1A] 1 illustrates a system for predicting patient or treatment response to a therapeutic agent, according to various embodiments. [Figure 1B] 1 illustrates another system for predicting patient or treatment response to a therapeutic agent, according to various embodiments. [Figure 1C] 1 illustrates another system for predicting patient or treatment response to a therapeutic agent, according to various embodiments. [Figure 2A] 1 illustrates a method for predicting patient / treatment response to a therapeutic agent, according to various embodiments. [Figure 2B]1 illustrates another method for predicting patient / treatment response to a therapeutic agent, according to various embodiments. [Figure 2C] 1 illustrates another method for predicting patient / treatment response to a therapeutic agent, according to various embodiments. [Figure 2D] 1 illustrates a method for predicting patient / treatment response to a therapeutic agent, according to various embodiments. [Figure 2E] 1 illustrates another method for predicting patient / treatment response to a therapeutic agent, according to various embodiments. [Figure 2F] 1 illustrates another method for predicting patient / treatment response to a therapeutic agent, according to various embodiments. [Figure 3] FIG. 1 is a block diagram of a computing system configured to perform a method for predicting patient / treatment response to a therapeutic agent, according to various embodiments.

[0024] It should be understood that the drawings are not necessarily drawn to scale, and that objects in the drawings are not necessarily drawn to scale relative to each other. The drawings are depictions intended to clarify and understand various embodiments of the devices, systems, and methods disclosed herein. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or similar parts. Furthermore, it should be understood that the drawings are not intended to limit the scope of the present teachings in any way. DETAILED DESCRIPTION OF THE INVENTION

[0025] I. Overview Embodiments of the present disclosure include methods for predicting whether an individual will respond to one or more specific therapies. In specific embodiments, the one or more specific therapies include any type of one or more cancer immunotherapy therapeutics. In certain cases, the one or more cancer immunotherapy therapeutics target one or more checkpoint inhibitors. In particular embodiments, the one or more checkpoint inhibitors include PD-L1 or PD-1, or a mixture thereof. The type of cancer immunotherapy therapeutic may be of any type, including therapeutics that utilize, at least in part, any type of antibody. In specific embodiments, the antibody includes a monoclonal antibody, which may or may not be humanized. The antibody may be used as a cancer immunotherapy therapeutic by itself, or the antibody may be configured as part of any type of chimeric molecule, such as an engineered antigen receptor. The antibody may be configured as part of any type of fusion protein.

[0026] In specific embodiments, an individual (e.g., a patient) is subjected to a method in which multiple parameters are measured to determine whether the individual will respond to a particular therapy. In particular cases, the parameters include one or more tumor mutation burden indicators, one or more PD-L1 expression level indicators and / or one or more PD-1 expression level indicators, and one or more TIL metric values. In some embodiments, patient characteristics such as biological sex, age, personal or family medical history (such as whether the individual and / or one or more family members have previously had any disease, including cancer, including PD-L1-positive cancer and / or PD-1-positive cancer) may also be included as a parameter or parameters to consider to determine whether the individual will respond to a particular therapy.

[0027] In certain embodiments, an individual (e.g., a patient) is predicted to be or determined to be a responder to one or more cancer immunotherapy therapeutics. In some cases, the individual is predicted to be or determined to be more likely to respond to one or more cancer immunotherapy therapeutics when compared to a particular population, such as the general population, a population of cancer patients, or a population lacking expression of one or more checkpoints, such as PD-L1 or PD-1.

[0028] The present disclosure is not limited to these exemplary embodiments and applications or to the manner in which the exemplary embodiments and applications operate or are described herein. Further, the figures may show simplified or partial views and the dimensions of elements in the figures may be exaggerated or otherwise not to precise scale.

[0029] Unless otherwise defined, scientific and technical terms used in connection with the teachings set forth herein shall have the meanings commonly understood by those of ordinary skill in the art. Further, unless otherwise required by context, singular terms shall include pluralities and plural terms shall include the singular. Generally, the nomenclature utilized in connection with, and techniques of, chemistry, biochemistry, molecular biology, pharmacology, and toxicology are described herein and are well known and commonly used in the art.

[0030] While the present teachings will be described in conjunction with various embodiments, it is not intended that the present teachings be limited to such various embodiments. On the contrary, the present teachings encompass various alternatives, modifications, and equivalents, as will be appreciated by those skilled in the art.

[0031] In describing various embodiments, the specification may present a method and / or process as a particular series of steps. However, to the extent that the method or process does not depend on the particular order of steps described herein, the method or process should not be limited to the particular sequence of steps described, and one of ordinary skill in the art will readily appreciate that the sequence may be varied and still be within the spirit and scope of the various embodiments.

[0032] II. SYSTEMS AND WORKFLOWS FOR PREDICTING PATIENT OR THERAPEUTIC RESPONSE TO THERAPEUTIC TREATMENTS FIG. 1A illustrates a system 100 for predicting patient or treatment response to a therapeutic agent, according to various embodiments. While system 100 may be applicable to any suitable therapy, for example, therapy for any type of cancer, system 100 depicted in FIG. 1A is shown for predicting patient response to a cancer immunotherapy therapeutic agent. As shown in FIG. 1A, system 100 includes one or more neural networks 110 that can be used to analyze images 105 to make a prediction of patient response 190 (also referred to herein as "response 190") based on the analysis of images 105. In various embodiments, one or more patient features 108 can also be used in analyzing and predicting patient response 190 to the therapeutic agent.

[0033] According to various embodiments, the image 105 may be an image of a tissue sample from a patient. In various embodiments, the image 105 may include one or more features of the tissue sample (e.g., cell shape, size, distribution, and / or density, etc.), which may be used for analysis by one or more neural networks 110 to predict patient response 190 to a therapeutic agent. In various embodiments, the image 105 may include a hematoxylin and eosin (H&E) stained tissue sample from a patient, or any other sample, which may be used for routine histopathology techniques to diagnose disease (e.g., cancer, including bladder cancer). In various embodiments, the image 105 may be obtained from a laboratory or clinical department, such as a histology department, prior to analysis, or may be stored in a data store, database, or data storage device for retrieval prior to analysis. The image 105 may be an image of fresh or frozen tissue.

[0034] According to various embodiments, the one or more patient features 108 can include any features associated with and / or related to the patient that can aid in analyzing or predicting the patient's response to a therapeutic agent. Such features 108 can include, for example, but are not limited to, age, biological sex, weight, smoking history, one or more symptoms, previous treatment options, previous treatment regimens, time since diagnosis, number of previous diagnoses for the same disease or any other disease that may affect the relevant organ or tissue, medical information for the patient's family, the presence or absence of one or more biomarkers for the disease, or any anatomical location of the patient. In various embodiments, the one or more patient features 108 can include one or more medical history features of the patient. In various embodiments, the one or more patient features 108 can be extracted from a patient health profile, for example, from an electronic health record system.

[0035] 1A, system 100 includes one or more neural networks 110 for analyzing an input image (e.g., image 105) and predicting a patient response 190 based on the analysis. In various embodiments, one or more neural networks 110 include a first neural network model 120 (also referred to herein as “model 120”), a second neural network model 140 (also referred to herein as “model 140”), and a third neural network model 160 (also referred to herein as “model 160”). Each of models 120, 140, and / or 160 may include a machine learning algorithm that can input image 105 along with one or more features of a tissue sample to generate respective models 130, 150, and / or 170, as shown in FIG. 1A.

[0036] In various embodiments, the one or more neural networks 110 include a fourth neural network model 180 (also referred to herein as “model 180”), which may include a machine learning algorithm that can take one or more of the outputs 130, 150, and 170 as inputs to the model 180 and generate a patient response 190. In various embodiments, the model 180 can input an image 105 and / or one or more features of a tissue sample, as shown in FIG. 1A. In various embodiments, the model 180 can also input one or more patient features 108 as part of the analysis and generate a patient response 190, as shown in FIG. 1A.

[0037] In various embodiments, one or more of the neural networks 110, and thus by extension the models 120, 140, 160, and 180, may include at least one of, or any combination of, for example, but not limited to, a Convolutional Neural Network (CNN), a Feedforward Neural Network (FNN), a Recurrent Neural Network (RNN), a Modular Neural Network (MNN), a Residual Neural Network (ResNet), an Ordinary Differential Equation Neural Network (Neural ODE), or another type of neural network. Each of the one or more of the neural networks 110, and thus by extension the models 120, 140, 160, and 180, may include one or more algorithms, one or more mathematical techniques, one or more machine learning algorithms, or a combination thereof.

[0038] In various embodiments, one or more neural networks 110, and thus by extension, models 120, 140, 160, and 180, may be trained using large datasets. In various embodiments, the training of one or more neural networks 110, and thus by extension, models 120, 140, 160, and 180 may be self-supervised learning (SSL) or supervised. In various embodiments, models 120, 140, 160, and 180 may be pre-trained using SSL and then trained for one or more specific tasks (e.g., various indicators or metrics listed below) via supervised learning. Thus, the dataset for training the models may include images annotated by clinicians / pathologists. In various embodiments, each of the trained models 120, 140, 160, and / or 180 may then be configured to analyze one or more features of the tissue sample from the image 105, for example, to extract necessary data, learn from the necessary data, and then make a decision or prediction, such as an output 130, 150, 170, and / or patient response 190, respectively.

[0039] According to various embodiments, for example, to predict patient response to a cancer immunotherapy therapeutic, model 120 may be configured to generate one or more tumor mutational burden (TMB) indicators as output 130 based on one or more features of image 105. Similarly, model 140, according to various embodiments, may be configured to generate one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features, or to generate one or more programmed cell death 1 (PD-1) expression level indicators from one or more features of image 105 as output 150. In various embodiments, the cancer immunotherapy therapeutic is a checkpoint inhibitor, and the checkpoint inhibitor may be a PD-L1 inhibitor or a PD-1 inhibitor. Similarly, model 160, according to various embodiments, may be configured to calculate one or more tumor infiltrating lymphocyte (TIL)-related metrics from one or more features of image 105 as output 170.

[0040] In various embodiments, model 120 includes a classification algorithm or a regression algorithm, both of which are trained using the annotated dataset as described above. To generate one or more TMB indicators as output 130, the classification algorithm of model 120 may incorporate one or more features about the tissue sample from image 105, for example, and generate output 130 as either a TMB-positive or TMB-negative classification. In various embodiments, a TMB-positive classification is generated as output 130 if the TMB value exceeds a predetermined TMB threshold. In various embodiments, the regression algorithm of model 120 may generate a TMB value or a quantitative TMB score as the TMB indicator for output 130.

[0041] In various embodiments, model 140 includes a classification algorithm or a regression algorithm, both of which are trained using the annotated dataset as described above. The analysis can be performed using either a classification algorithm or a regression algorithm to generate one or more PD-L1 expression level indicators or one or more PD-1 expression level indicators as output 150. In various embodiments, the classification algorithm of model 140 may generate a PD-L1 expression level indicator as output 150 that is either a PD-L1 expression positive classification or a PD-L1 expression negative classification. In various embodiments, if the tissue sample in image 105 has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value, the classification algorithm of model 140 may generate a PD-L1 expression positive classification. In various embodiments, the regression algorithm of model 140 may generate one or more PD-L1 expression level indicators as output 150 that includes a quantitative PD-L1 score. In various embodiments, the quantitative PD-L1 score may be a tumor proportion score (TPS) or a combined positive score (CPS).

[0042] In various embodiments, the classification algorithm of model 140 may generate as output 150 a PD-1 expression level indicator that is either a PD-1 expression positive classification or a PD-1 expression level negative classification. In various embodiments, if the tissue sample in image 105 has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value, the classification algorithm of model 140 may generate a PD-1 expression positive classification. In various embodiments, the regression algorithm of model 140 may generate one or more PD-1 expression level indicators as output 150, including a quantitative PD-1 score. In various embodiments, the quantitative PD-1 score may be a tumor proportion score (TPS) or a combined positive score (CPS).

[0043] In various embodiments, model 160 may include one or more segmentation algorithms, each of which may be configured to generate one or more TIL masks, one or more tumor masks, one or more stromal masks, one or more necrosis masks, or one or more cell nuclei masks as output 170 from one or more features of image 105. In various embodiments, model 160 may include a region segmentation algorithm. In various embodiments, model 160 may include a cell segmentation algorithm. In various embodiments, model 160 may calculate a TIL-related metric (such as, but not limited to, TIL%, sTIL%, or number of TIL clusters) by using one or more TIL masks, one or more tumor masks, one or more stromal masks, one or more necrosis masks, or one or more cell nuclei masks as output 170.

[0044] In various embodiments, model 160 may include a segmentation algorithm and may be trained using the annotated dataset as described above. In various embodiments, the segmentation algorithm may be used to generate an identification of a tumor region in a tissue sample based on one or more features of image 105, according to various embodiments. This identification may be used to further generate or calculate one or more TIL-related metrics from the one or more features of image 105 as output 170. In various embodiments, the segmentation algorithm may be referred to as a region segmentation algorithm or a TIL region segmentation algorithm. In various embodiments, the segmentation algorithm may be referred to as a tumor region segmentation algorithm. In various embodiments, the segmentation algorithm may be configured to predict a TIL mask, which is used in combination with the tumor mask to calculate one or more TIL-related metrics as output 170. In various embodiments, the one or more TIL-related metrics may include a predicted number of TIL clusters in the identified tumor region as output 170. In various embodiments, the one or more TIL-related metrics may include a predicted size of a TIL cluster in the identified tumor region as an output 170. In various embodiments, the one or more TIL-related metrics may include a predicted TIL cluster spread in the identified tumor region as an output 170. In various embodiments, the one or more TIL-related metrics may include intratumoral TIL% (iTIL%) as an output 170. In various embodiments, iTIL% may be calculated as the percent of the tumor region identified in the tissue sample that is infiltrated by TILs as an output 170. In various embodiments, the one or more TIL-related metrics may include stromal TIL% (sTIL%) as an output 170. In various embodiments, an sTIL% value may be calculated as the percent of stroma within the tumor region identified in the tissue sample that is infiltrated by TILs as an output 170.

[0045] In various embodiments, the model 160 may include a segmentation algorithm for cell segmentation, which may be referred to as a cell segmentation algorithm. In various embodiments, the cell segmentation algorithm may input the image 105 and provide cell segmentation as output 170. In various embodiments, the segmented cells may be labeled based on the region segmentation output. For example, a segmented cell may be labeled as a TIL cell if the cell is within a TIL region. In various embodiments, the cell segmentation algorithm is configured to calculate one or more features associated with cells (also referred to herein as cell-based features) contained in the image 105 of the tissue sample. In various embodiments, the calculated cell-based features may include morphology, color (nuclear), color (extranuclear), texture (energy, correlation, contrast, uniformity, dissimilarity, angular second moment), chromatin backbone morphology, and color, among many other features that may be extracted. In various embodiments, the cell segmentation algorithm of model 160 may be configured to generate a number of TIL cells or an average number of TIL cells as output 170. In various embodiments, the cell segmentation algorithm of model 160 may be configured to generate an average tumor cell size as output 170. In various embodiments, the cell segmentation algorithm of model 160 may be configured to generate an average TIL cell size as output 170. In various embodiments, the cell segmentation algorithm of model 160 may be configured to generate an average distance between TIL cells as output 170. In various embodiments, the cell segmentation algorithm of model 160 may be configured to generate an average color of the TIL cells as output 170. In various embodiments, the cell segmentation algorithm of model 160 may be configured to generate an average distance between TIL cells compared to tumor cells, or the like, as output 170. In various embodiments, the one or more TIL-related metrics may include a total number of TILs in the identified tumor region as output 170.In various embodiments, the one or more TIL-related metrics may include as an output 170 the distance between TIL cells compared to the distance between tumor cells. In various embodiments, the one or more TIL-related metrics may include as an output 170 the distance of TIL cells from tumor cells. In various embodiments, the one or more TIL-related metrics may include as an output 170 the average color of the TIL cells. In various embodiments, the one or more TIL-related metrics may include as an output 170 the average size of the TIL cells.

[0046] In various embodiments, model 180 may be configured to generate a response status of a patient or treatment response 190 to, for example, a cancer immunotherapy therapeutic by applying one or more of outputs 130, 150, and 170 as inputs to model 180. In various embodiments, one or more TMB indicators of output 130, one or more PD-L1 expression level indicators or one or more PD-1 expression level indicators, respectively, of output 150, and one or more TIL-associated metrics of output 170 may be used for analysis by model 180, according to various embodiments, to predict a patient's response status to a cancer immunotherapy therapeutic. As described above, model 180 may include a machine learning algorithm that is trained using patient cancer immunotherapy response data, and response 190 may include annotated labels of responder, partial responder, or non-responder as ground truth for training. In various embodiments, model 180 may be configured such that each of outputs 130, 150, and 170 is weighted relative to one another in analysis by the machine learning algorithm of model 180. In various embodiments, different weightings applied to each of outputs 130, 150, and 170 may affect and / or significantly change output 190 of the patient response to a therapeutic agent (e.g., a cancer immunotherapy therapeutic agent).

[0047] In various embodiments, model 180 may be trained using a set of H&E images from patients with available response data to a desired treatment or therapeutic agent (e.g., a cancer immunotherapy therapeutic agent).

[0048] For example, the model 180 may be trained using the following four patients: -Patient 1, H&E image, responder - Patient 2, H&E image, non-responder -Patient 3, H&E image, responder -Patient 4, H&E image, responder

[0049] For each patient, each of the models 120, 140, and 160 may be run to generate the following datasets from the images 105, outputs 130 (TMB score), 150 (PD-L1 score), and 170 (TIL mask and / or TIL features), respectively: - Patient 1, H&E image, TMB score, PD-L1 score, TIL mask, TIL features, responder - Patient 2, H&E image, TMB score, PD-L1 score, TIL mask, TIL features, non-responder - Patient 3, H&E image, TMB score, PD-L1 score, TIL mask, TIL features, responder - Patient 4, H&E image, TMB score, PD-L1 score, TIL mask, TIL features, responder

[0050] Using the above dataset of inputs 130, 150, and 170, model 180 may be trained as follows: "H&E image, TMB score, PD-L1 score, TIL mask, TIL features" along with ground truth or labels as either "responders" or "non-responders," or in some cases, as "partial responders." Once model 180 is trained, it may be used to generate patient responses 190 based on a single image similar to image 105 and the output generated by models 120, 140, and 160 trained using one or more features of the single image.

[0051] 1B illustrates a system 102 for predicting a patient or treatment response to a therapeutic agent, according to various embodiments. System 102 is similar to system 100 described with respect to FIG. 1A in that, to generate response 190, neural network 110 having image 105, models 120, and 140 generates outputs 130 and 150, respectively, and model 180 incorporates outputs 130 and 150, and / or image 105 and / or patient features 108. System 102 differs from system 100 in that, instead of having model 160 (as in system 100) that generates output 170, which can be incorporated by model 180 to generate response 190, system 102 includes two models 160a and 160b, which generate outputs 170a and 170b, respectively, which can be incorporated by model 180 to generate response 190. In other words, models 160a and 160b represent two separate segmentation algorithms. In various embodiments, model 160a may include a region / TIL segmentation algorithm. In various embodiments, model 160b may include a cell segmentation algorithm.

[0052] In various embodiments, the region / TIL segmentation algorithm of model 160a may be used to generate an identification of a tumor region in the tissue sample based on one or more features of image 105. In various embodiments, the identification may be used to further generate or calculate one or more TIL-related metrics as output 170a from one or more features of image 105. In various embodiments, the region / TIL segmentation algorithm of model 160a may be configured to predict a TIL mask, which is used in combination with the tumor mask to calculate one or more TIL-related metrics as output 170a. In various embodiments, the one or more TIL-related metrics may include a predicted number of TIL clusters in the identified tumor region as output 170a. In various embodiments, the one or more TIL-related metrics may include a predicted size of TIL clusters in the identified tumor region as output 170a. In various embodiments, the one or more TIL-related metrics may include a predicted TIL cluster spread in the identified tumor region as output 170a. In various embodiments, the one or more TIL-related metrics may include intratumoral TIL% (iTIL%) as output 170a. In various embodiments, iTIL% may be calculated as output 170a as the percentage of tumor area identified in the tissue sample that is infiltrated by TILs. In various embodiments, the one or more TIL-related metrics may include stromal TIL% (sTIL%) as output 170a. In various embodiments, an sTIL% value may be calculated as the percentage of stroma within tumor area identified in the tissue sample that is infiltrated by TILs as output 170a.

[0053] In various embodiments, the cell segmentation algorithm of model 160b may input image 105 and perform cell segmentation as output 170b. In various embodiments, the cell segmentation algorithm of model 160b may input the region / TIL mask generated by model 160a and perform cell segmentation as output 170b. In various embodiments, the segmented cells may be labeled based on the region segmentation output. For example, a segmented cell may be labeled as a TIL cell if the cell is within a TIL region. In various embodiments, the cell segmentation algorithm is configured to calculate one or more features associated with cells (also referred to herein as cell-based features) contained in image 105 of the tissue sample. In various embodiments, the calculated cell-based features may include morphology, color (nuclear), color (extranuclear), texture (energy, correlation, contrast, uniformity, dissimilarity, angular second moment), chromatin backbone morphology, and color, among many other features that may be extracted. In various embodiments, the cell segmentation algorithm of model 160b may be configured to generate a number of TIL cells or an average number of TIL cells as output 170b. In various embodiments, the cell segmentation algorithm of model 160b may be configured to generate an average tumor cell size as output 170b. In various embodiments, the cell segmentation algorithm of model 160b may be configured to generate an average TIL cell size as output 170b. In various embodiments, the cell segmentation algorithm of model 160b may be configured to generate an average distance between TIL cells as output 170b. In various embodiments, the cell segmentation algorithm of model 160b may be configured to generate an average color of the TIL cells as output 170b. In various embodiments, the cell segmentation algorithm of model 160b may be configured to generate an average distance between TIL cells compared to tumor cells, or the like, as output 170b.In various embodiments, the one or more TIL-related metrics may include as output 170b the total number of TILs in the identified tumor region. In various embodiments, the one or more TIL-related metrics may include as output 170b the distance between TIL cells compared to the distance between tumor cells. In various embodiments, the one or more TIL-related metrics may include as output 170b the distance of TIL cells from tumor cells. In various embodiments, the one or more TIL-related metrics may include as output 170b the average color of TIL cells. In various embodiments, the one or more TIL-related metrics may include as output 170b the average size of TIL cells.

[0054] In various embodiments, model 180 may be configured to generate a response status of a patient or treatment response 190 to, for example, a cancer immunotherapy therapeutic by applying one or more of outputs 130, 150, 170a, and 170b as inputs to model 180. In various embodiments, one or more TMB indicators of output 130, one or more PD-L1 expression level indicators or one or more PD-1 expression level indicators, respectively, of output 150, and one or more TIL-associated metrics of output 170 may be used for analysis by model 180, according to various embodiments, to predict a patient's response status to a cancer immunotherapy therapeutic. As described above, model 180 may include a machine learning algorithm that is trained using patient cancer immunotherapy response data, and response 190 may include annotated labels of responder, partial responder, or non-responder as ground truth for training. In various embodiments, model 180 may be configured such that each of outputs 130, 150, 170a, and 170b is weighted relative to one another in analysis by the machine learning algorithm of model 180. In various embodiments, different weightings applied to each of outputs 130, 150, 170a, and 170b may affect and / or significantly change output 190 of the patient response to a therapeutic agent (e.g., a cancer immunotherapy therapeutic agent).

[0055] For each patient, each of models 120, 140, 160a, and 160b may be run to generate the following datasets from images 105, outputs 130 (TMB score), 150 (PD-L1 score), 170a (TIL mask and / or TIL features), and 170b (cell-based features), respectively. - Patient 1, H&E image, TMB score, PD-L1 score, TIL mask, TIL features, cell-based features, responder - Patient 2, H&E image, TMB score, PD-L1 score, TIL mask, TIL features, cell-based features, non-responder - Patient 3, H&E image, TMB score, PD-L1 score, TIL mask, TIL features, cell-based features, responder - Patient 4, H&E image, TMB score, PD-L1 score, TIL mask, TIL features, cell-based features, responder

[0056] Using the above dataset of inputs 130, 150, 170a, and 170b, model 180 may be trained as follows: H&E image, TMB score, PD-L1 score, TIL mask, TIL features, cell-based features, along with ground truth or labels as either a "responder" or a "non-responder," or in some cases, as a "partial responder." Once model 180 is trained, it may be used to generate a patient response 190 based on a single image similar to image 105 and the output generated by models 120, 140, 160a, and 160b trained using one or more features of the single image.

[0057] FIG. 1C illustrates another system 200 for predicting patient or treatment response to a therapeutic agent, according to various embodiments. Similar to the system 100 described with respect to FIG. 1A, the system 200 may be applicable to any suitable therapy, for example, a therapy for any type of cancer. While the system 200 is depicted in FIG. 1C for predicting patient response to a cancer immunotherapy therapeutic agent, the system 200 is suitable for generating patient responses to any other therapeutic agent. As shown in FIG. 1C, the system 200 includes one or more neural networks 210 that can be used to analyze an image 205 to generate a prediction of a patient response 290 (also referred to herein as "response 290") based on the analysis of the image 205. In various embodiments, one or more patient features 208 can also be used in analyzing and predicting the patient response 290 to the therapeutic agent. The image 205 and one or more patient features 208 can be substantially similar to or identical to the image 105 and one or more patient features 108, respectively, described with respect to FIG. 1A, and therefore will not be described in further detail.

[0058] As shown in FIG. 1C , the system 200 includes one or more neural networks 210 for analyzing an input image (e.g., image 205) and predicting a patient response 290 based on the analysis. In various embodiments, the one or more neural networks 210 include one or more classification algorithms, one or more regression algorithms, and / or one or more segmentation algorithms for analyzing and generating one or more final and intermediate outputs. In various embodiments, an intermediate output may be generated by one or more of the classification algorithms, regression algorithms, and / or segmentation algorithms, which may then be used as an input in one or more other of the classification algorithms, regression algorithms, and / or segmentation algorithms to generate a final output. Each of the classification algorithms, regression algorithms, and / or segmentation algorithms of the one or more neural networks 210 is a machine learning algorithm that can input the image 205 along with one or more features of the tissue sample to generate a response 290, as shown in FIG. 1C .

[0059] In various embodiments, the one or more neural networks 210 may include, for example, without limitation, at least one of, or any combination of, a convolutional neural network (CNN), a feedforward neural network (FNN), a recurrent neural network (RNN), a modular neural network (MNN), a residual neural network (ResNet), an ordinary differential equation neural network (neural ODE), or another type of neural network.

[0060] In various embodiments, one or more neural networks 210 may be trained using a large dataset. In various embodiments, training of one or more neural networks 210 may be self-supervised (SSL) or supervised. Thus, the dataset for training the model may include images annotated by a clinician / pathologist. In various embodiments, each of the classification, regression, and / or segmentation algorithms of one or more neural networks 210 may then be configured to analyze one or more features of the tissue sample from the image 105, e.g., to extract necessary data, learn from the necessary data, and then make a determination or prediction, such as patient response 190, respectively.

[0061] 1A, 1B, or 1C, one or more neural network models 110 or 210 may be configured to generate a prediction of a patient's response status to a cancer immunotherapy therapeutic using image 105 or image 205. According to various embodiments, to predict patient response to a cancer immunotherapy therapeutic, for example, one or more neural networks 210 may be configured to generate one or more tumor mutational burden (TMB) indicators based on one or more features of image 205. In various embodiments, one or more neural networks 210 may be configured to generate one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features, or one or more programmed cell death 1 (PD-1) expression level indicators from one or more features of image 205, according to various embodiments. In various embodiments, one or more neural networks 210 may be configured to calculate one or more tumor infiltrating lymphocyte (TIL)-related metrics from one or more features of image 205, according to various embodiments.

[0062] In various embodiments, one or more neural networks 210 are trained using the annotated dataset as described above. The classification algorithm of the one or more neural networks 210 may, for example, incorporate one or more features about the tissue sample from the image 205 and generate a TMB-positive or TMB-negative classification. These features may be used to generate one or more TMB indicators. In various embodiments, a TMB-positive classification is generated as an intermediate output if the TMB value exceeds a predetermined TMB threshold. In various embodiments, the regression algorithm of the one or more neural networks 210 may generate a TMB value or a quantitative TMB score as the TMB indicator for output 130.

[0063] In various embodiments, the one or more neural networks 210 may generate one or more PD-L1 expression level indicators or one or more PD-1 expression level indicators. In various embodiments, the one or more neural networks 210 may include a classification algorithm that may generate as output 150 a PD-L1 expression level indicator that is either a PD-L1 expression positive classification or a PD-L1 expression level negative classification. In various embodiments, the classification algorithm may generate a PD-L1 expression positive classification if the tissue sample in image 105 has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value. In various embodiments, the one or more neural networks 210 may include a regression algorithm for generating one or more PD-L1 expression level indicators as output 150 that includes a quantitative PD-L1 score. In various embodiments, the quantitative PD-L1 score may be a tumor proportion score (TPS) or a combined positive score (CPS).

[0064] In various embodiments, the classification algorithm may generate as an intermediate output a PD-1 expression level indicator that is either a PD-1 expression positive classification or a PD-1 expression level negative classification. In various embodiments, if the tissue sample in image 205 has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value, the classification algorithm may generate a PD-1 expression positive classification. In various embodiments, the regression algorithm may generate one or more PD-1 expression level indicators that include a quantitative PD-1 score. In various embodiments, the quantitative PD-1 score may be a tumor proportion score (TPS) or a combined positive score (CPS).

[0065] In various embodiments, the one or more neural networks 210 may include a segmentation algorithm trained using the annotated dataset as described above. The segmentation algorithm, according to various embodiments, is used to generate an identification of tumor regions in the tissue sample based on one or more features of the image 205. These metrics may be used to generate or calculate one or more TIL-related metrics from the one or more features of the image 205. In various embodiments, the segmentation algorithm uses one or more TIL mask structure features in the identified tumor region to generate one or more TIL-related metrics as intermediate outputs. In various embodiments, the one or more TIL mask structure features may include, as intermediate outputs, a predicted number of TIL clusters in the identified tumor region. In various embodiments, the one or more TIL mask structure features may include, as intermediate outputs, a predicted size of TIL clusters in the identified tumor region. In various embodiments, the one or more TIL mask structure features may include, as intermediate outputs, a predicted TIL cluster spread in the identified tumor region. In various embodiments, the one or more TIL mask structure features may include as an intermediate output the total number of TILs in the identified tumor region. In various embodiments, the one or more TIL mask structure features may include as an intermediate output the distance between TIL cells compared to the distance between tumor cells. In various embodiments, the one or more TIL mask structure features may include as an intermediate output the distance of TIL cells from tumor cells. In various embodiments, the one or more TIL mask structure features may include as an intermediate output the average color of TIL cells. In various embodiments, the one or more TIL mask structure features may include as an intermediate output the average size of TIL cells. In various embodiments, the one or more TIL-related metrics may include intratumoral TIL% (iTIL%) as an intermediate output. In various embodiments, iTIL% may be calculated as the percentage of the identified tumor region in the tissue sample that is infiltrated by TILs. In various embodiments, the one or more TIL-related metrics may include stromal TIL% (sTIL%) as an intermediate output.In various embodiments, an sTIL% value may be calculated as an intermediate output as the percentage of stroma within the tumor area identified in the tissue sample that is infiltrated by TILs.

[0066] In various embodiments, the one or more neural networks 210 may include one or more segmentation algorithms, each of which may be configured to generate one or more TIL masks, one or more tumor masks, one or more stromal masks, one or more necrosis masks, or one or more nucleus masks as intermediate outputs. In various embodiments, the one or more segmentation algorithms of the one or more neural networks 210 may calculate a TIL-related metric (such as, but not limited to, TIL%, sTIL%, or number of TIL clusters) by using the one or more TIL masks, one or more tumor masks, one or more stromal masks, one or more necrosis masks, or one or more nucleus masks.

[0067] In various embodiments, one or more neural networks 210 may be configured to generate, for example, a response status of a patient or treatment response 290 to a cancer immunotherapy therapeutic by applying one or more of the above-described intermediate outputs as inputs to the one or more neural networks 210. In various embodiments, one or more TMB indicators, one or more PD-L1 expression level indicators, or one or more PD-1 expression level indicators, and one or more TIL-associated metrics may be used in an analysis to predict a patient's response status to a cancer immunotherapy therapeutic, according to various embodiments. As described above, the one or more neural networks 210 may include a machine learning algorithm trained using the patient's cancer immunotherapy response data, and the response 290 may include annotated labels of responder, partial responder, or non-responder as ground truth for training. In various embodiments, the one or more neural networks 210 may be configured such that each of the intermediate outputs is weighted relative to one another in analysis by the machine learning algorithm. In various embodiments, different weightings applied to various intermediate outputs can influence and / or significantly change the patient response 290 to a therapeutic agent (eg, a cancer immunotherapy therapeutic agent).

[0068] In various embodiments, no biomarkers are generated. In various embodiments, one or more neural networks 210 predict patient response directly from images 205.

[0069] 2A illustrates a method S100 for predicting patient / treatment response to a therapeutic agent, according to various embodiments. Method S100 may be implemented, for example, via a computing system 300, as described with respect to FIG. 3 below. As shown in FIG. 2A, method S100 includes, in step S110, obtaining or receiving, by one or more processors of computing system 300, an image of a tissue sample from a patient (e.g., image 105); in step S120, generating, by one or more processors, one or more tumor mutation burden (TMB) indicators from the one or more features (e.g., as output 130) via a first neural network model (e.g., model 120); and in step S130, generating, by one or more processors, one or more programmed death-ligand 1 (PD-L1) expression level indicators from the one or more features or one or more programmed death-ligand 1 (PD-L1) expression level indicators from the one or more features via a second neural network model (e.g., model 140). and generating, by the one or more processors, one or more tumor infiltrating lymphocyte (TIL)-associated metrics from the one or more features via a third neural network model (e.g., model 160) in step S140. And predicting, by the one or more processors, a patient's response status (e.g., response 190) to the cancer immunotherapy therapeutic using the one or more TMB indicators, the one or more PD-L1 expression level indicators, or the one or more PD-1 level indicators, respectively, and the one or more associated metrics via a fourth neural network model (e.g., model 180) in step S150.

[0070] In various embodiments, method S100 may optionally include displaying a graphical representation indicating the patient's response status on a display screen. For example, the individual's predicted response status may be displayed to a clinician or other healthcare provider, clinical trial administrator, researcher, or other entity via a graphical user interface, e.g., as part of clinical trial management software or other software. In various embodiments, method S100 may optionally include selecting whether to include the patient in a cohort of patients participating in the clinical trial according to the patient's response status. In specific embodiments, it may be desirable to determine whether a clinical trial candidate may be able to respond to a cancer immunotherapy therapeutic. Certain embodiments include treating an individual with a therapeutically effective amount of a cancer immunotherapy therapeutic, where the individual is a responder as determined by the systems or methods disclosed herein, and the individual may or may not be a participant in the clinical trial. In particular embodiments, an individual predicted to be a responder based on measuring, assaying, and / or generating multiple neural network models encompassed herein is administered a therapeutically effective amount of the cancer immunotherapy therapeutic.

[0071] In various embodiments of method S100, predicting a patient's response status to a cancer immunotherapy therapeutic further uses one or more medical history features of the patient. In various embodiments of method S100, the third neural network is a segmentation algorithm that identifies tumor regions in the tissue sample. In various embodiments of method S100, the segmentation algorithm predicts a TIL mask and uses this in combination with the tumor mask to calculate one or more TIL-related metrics. In various embodiments of method S100, the one or more TIL-related metrics include a predicted number of TIL clusters in the identified tumor region. In various embodiments, the one or more TIL-related metrics include a predicted size of TIL clusters in the identified tumor region.

[0072] In various embodiments of method S100, the TIL-related metric includes intratumoral TIL% (iTIL%). In various embodiments, iTIL% is calculated as the percent of tumor area identified in the tissue sample that is infiltrated by TILs. In various embodiments, the TIL% value is stromal TIL% (sTIL%). In various embodiments, the sTIL% value is calculated as the percent of stroma within the tumor area identified in the tissue sample that is infiltrated by TILs.

[0073] In various embodiments of method S100, the third neural network is a cell segmentation algorithm, and the cell segmentation algorithm calculates one or more cell-based features. In various embodiments, the one or more cell-based features include morphology, color (nuclear), color (extranuclear), texture (energy, correlation, contrast, uniformity, dissimilarity, angular second moment), chromatin skeletal morphology, and color. In various embodiments, the cell segmentation algorithm generates a number of TIL cells or a mean number of TIL cells. In various embodiments, the cell segmentation algorithm generates a mean tumor cell size. In various embodiments, the cell segmentation algorithm generates a mean TIL cell size. In various embodiments, the cell segmentation algorithm generates a mean distance between TIL cells. In various embodiments, the cell segmentation algorithm generates a mean color of TIL cells. In various embodiments, the cell segmentation algorithm generates a mean distance between TIL cells compared to tumor cells. In various embodiments, the one or more TIL-related metrics include a predicted TIL cluster spread in the identified tumor region. In various embodiments, the one or more TIL-related metrics include a total number of TILs in the identified tumor region. In various embodiments, the one or more TIL-related metrics include a distance between TIL cells compared to a distance between tumor cells. In various embodiments, the one or more TIL-related metrics include a distance of TIL cells from tumor cells. In various embodiments, the one or more TIL-related metrics include an average color of TIL cells. In various embodiments, the one or more TIL-related metrics include an average size of TIL cells.

[0074] In various embodiments of method S100, the first neural network model is a classification algorithm that generates a TMB indicator that is either a TMB-positive or a TMB-negative classification for the tissue sample. In various embodiments, the classification algorithm generates a TMB-positive classification if the tissue sample has a TMB value above a predetermined TMB threshold. In various embodiments, the first neural network model is a regression algorithm that generates a TMB indicator that is a quantitative TMB score.

[0075] In various embodiments of method S100, the second neural network model is a classification algorithm that generates a PD-L1 expression level indicator that is either a PD-L1 expression positive classification or a PD-L1 expression negative classification. In various embodiments, the second neural network model is a classification algorithm that generates a PD-1 expression level indicator that is either a PD-L1 expression positive classification or a PD-1 expression negative classification. In various embodiments, if the tissue sample has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value, the classification algorithm generates a PD-L1 expression positive classification. In various embodiments, if the tissue sample has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value, the classification algorithm generates a PD-1 expression positive classification. In various embodiments, the second neural network model is a regression algorithm that generates a PD-L1 expression level indicator that is a quantitative PD-L1 score. In various embodiments, the second neural network model is a regression algorithm that generates a PD-1 expression level indicator that is a quantitative PD-1 score. In various embodiments, the quantitative PD-L1 score is a tumor proportion score (TPS) or a combined positive score (CPS). In various embodiments, the quantitative PD-1 score is a tumor proportion score (TPS) or a combined positive score (CPS). In various embodiments, the cancer immunotherapy therapeutic is a checkpoint inhibitor. In various embodiments, the checkpoint inhibitor is a PD-L1 inhibitor or a PD-1 inhibitor.

[0076] In various embodiments, the tissue sample is from the bladder and the patient response to the cancer immunotherapy therapeutic is associated with bladder cancer. In various embodiments, a determination is made that the patient is or will be a responder based on the response status. In various embodiments, following the determination, the patient is administered a therapeutically effective amount of the cancer immunotherapy therapeutic.

[0077] In various embodiments, a determination is made that the patient is not or will not be a responder based on the response status, and in various embodiments, following the determination, the patient is not administered the cancer immunotherapy treatment.

[0078] In various embodiments, the tissue sample is from a patient known to have or suspected of having PD-L1-positive or PD-1-positive cancer. In various embodiments, the tissue sample is from a patient known to have or suspected of having non-small cell lung cancer, melanoma, Hodgkin's lymphoma, bladder cancer, kidney cancer, or breast cancer. In various embodiments, the cancer immunotherapy therapeutic targets PD-L1 or PD-1. In various embodiments, the cancer immunotherapy therapeutic comprises one or more antibodies, adoptive cell therapy, immunomodulators, or combinations thereof. In various embodiments, the antibody is a monoclonal antibody, a bispecific antibody, or a trispecific antibody. In various embodiments, the adoptive cell therapy comprises immune cells expressing one or more engineered antigen receptors. In various embodiments, the engineered antigen receptor is a chimeric antigen receptor, a non-natural T cell receptor, or a combination thereof. In various embodiments, the adoptive cell therapy comprises T cells, natural killer cells, natural killer T cells, or a combination thereof. In various embodiments, method S100 may optionally include obtaining a sample from a patient. In various embodiments, method S100 may include diagnosing the patient as having cancer.

[0079] 2B illustrates another method S200 for predicting patient / treatment response to a therapeutic agent, according to various embodiments. Method S200 may be implemented, for example, via a computing system 300 as described with respect to FIG. 3 below. As shown in FIG. 2B , method S200 includes, in step S210, obtaining or receiving, by one or more processors from a data source, an image of a tissue sample from a patient and one or more features of the tissue sample; and, in step S220, computing, by one or more processors of computing system 300 via a first neural network model, (a) one or more tumor mutation burden (TMB) indicators from the one or more features, (b) one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features, or (c) one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features. and (c) generating one or more tumor infiltrating lymphocyte (TIL)-associated metrics from the one or more features, and in step S230, predicting, by one or more processors via one or more neural network models, a patient's response status to the cancer immunotherapy therapeutic using the one or more TMB indicators, the one or more PD-L1 expression level indicators or the one or more PD-1 expression level indicators, and the one or more TIL-associated metrics, and in step S240, selecting whether to include the patient in a cohort of patients participating in a clinical trial based on the patient's response status. In various embodiments, method S200 may optionally include displaying, on a display screen, a graphical representation indicating the patient's response status.

[0080] In an alternative embodiment, step S220 can be omitted and step S230 can be directed to inferring the patient's response status to a cancer immunotherapy therapeutic from the image itself, without the use of calculated / generated biomarkers.

[0081] In various embodiments of method S200, predicting a patient's response status to a cancer immunotherapy treatment further uses one or more medical history features of the patient. In various embodiments of method S200, the one or more neural network models may include a segmentation algorithm configured to predict a TIL mask and use this in combination with the tumor mask to calculate one or more TIL-related metrics. In various embodiments of method S200, the TIL-related metric is a predicted number of TIL clusters in the identified tumor region. In various embodiments, the TIL-related metric is a predicted size of TIL clusters in the identified tumor region.

[0082] In various embodiments of method S200, the TIL-related metric includes intratumoral TIL% (iTIL%). In various embodiments, iTIL% is calculated as the percent of tumor area identified in the tissue sample that is infiltrated by TILs. In various embodiments, the TIL% value is stromal TIL% (sTIL%). In various embodiments, the sTIL% value is calculated as the percent of stroma within the tumor area identified in the tissue sample that is infiltrated by TILs.

[0083] In various embodiments of method S200, the one or more neural network models may include a segmentation algorithm for cell segmentation, which may be referred to as a cell segmentation algorithm. In various embodiments, the cell segmentation algorithm may input an image and perform cell segmentation. In various embodiments, the segmented cells may be labeled based on the region segmentation output. For example, a segmented cell may be labeled as a TIL cell if the cell is within a TIL region. In various embodiments, the cell segmentation algorithm is configured to calculate one or more features associated with cells contained in the image (also referred to herein as cell-based features). In various embodiments, the calculated cell-based features may include morphology, color (nuclear), color (extranuclear), texture (energy, correlation, contrast, uniformity, dissimilarity, angular second moment), chromatin backbone morphology, and color, among many other features that may be extracted. In various embodiments, the cell segmentation algorithm may be configured to generate a TIL cell count or an average number of TIL cells. In various embodiments, the cell segmentation algorithm may be configured to generate an average tumor cell size. In various embodiments, the cell segmentation algorithm may be configured to generate an average TIL cell size. In various embodiments, the cell segmentation algorithm may be configured to generate an average distance between TIL cells. In various embodiments, the cell segmentation algorithm may be configured to generate an average color of the TIL cells. In various embodiments, the cell segmentation algorithm may be configured to generate an average distance between TIL cells compared to tumor cells. In various embodiments, the TIL-related metric is a predicted TIL cluster spread in the identified tumor region. In various embodiments, the TIL-related metric includes a total number of TILs in the identified tumor region. In various embodiments, the TIL-related metric may include the distance between TIL cells compared to the distance between tumor cells.In various embodiments, the TIL-related metric may include the distance of the TIL cells from the tumor cells. In various embodiments, the TIL-related metric may include the average color of the TIL cells. In various embodiments, the TIL-related metric may include the average size of the TIL cells.

[0084] In various embodiments of method S200, the one or more neural network models may include a classification algorithm that generates a TMB indicator that is either a TMB-positive or a TMB-negative classification for the tissue sample. In various embodiments, if the tissue sample has a TMB value above a predetermined TMB threshold, the classification algorithm generates a TMB-positive classification.

[0085] In various embodiments, the one or more neural network models may include a regression algorithm that generates a TMB indicator that is a quantitative TMB score.

[0086] In various embodiments of method S200, the one or more neural network models may include another classification algorithm that generates a PD-L1 expression level indicator that is either a PD-L1 expression positive classification or a PD-L1 expression negative classification. In various embodiments, the classification algorithm generates a PD-1 expression level indicator that is either a PD-1 expression positive classification or a PD-1 expression negative classification. In various embodiments, if the tissue sample has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value, the classification algorithm generates a PD-L1 expression positive classification. In various embodiments, if the tissue sample has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value, the classification algorithm generates a PD-1 expression positive classification.

[0087] In various embodiments, the one or more neural network models may include another regression algorithm that generates a PD-L1 expression level indicator that is a quantitative PD-L1 score. In various embodiments, the another regression algorithm generates a PD-1 expression level indicator that is a quantitative PD-1 score. In various embodiments, the quantitative PD-L1 score is a tumor proportion score (TPS) or a combined positive score (CPS). In various embodiments, the quantitative PD-1 score is a tumor proportion score (TPS) or a combined positive score (CPS). In various embodiments, the cancer immunotherapy therapeutic is a checkpoint inhibitor. In various embodiments, the checkpoint inhibitor is a PD-L1 inhibitor or a PD-1 inhibitor.

[0088] In various embodiments, the tissue sample is from the bladder and the patient response to the cancer immunotherapy therapeutic is associated with bladder cancer. In various embodiments, a determination is made that the patient is or will be a responder based on the response status. In various embodiments, following the determination, the patient is administered a therapeutically effective amount of the cancer immunotherapy therapeutic.

[0089] In various embodiments, a determination is made that the patient is not or will not be a responder based on the response status, and in various embodiments, following the determination, the patient is not administered the cancer immunotherapy treatment.

[0090] In various embodiments, the tissue sample is from a patient known to have or suspected of having PD-L1-positive or PD-1-positive cancer. In various embodiments, the tissue sample is from a patient known to have or suspected of having non-small cell lung cancer, melanoma, Hodgkin's lymphoma, bladder cancer, kidney cancer, or breast cancer. In various embodiments, the cancer immunotherapy therapeutic targets PD-L1 or PD-1. In various embodiments, the cancer immunotherapy therapeutic comprises one or more antibodies, adoptive cell therapy, immunomodulators, or combinations thereof. In various embodiments, the antibody is a monoclonal antibody, a bispecific antibody, or a trispecific antibody. In various embodiments, the adoptive cell therapy comprises immune cells expressing one or more engineered antigen receptors. In various embodiments, the engineered antigen receptor is a chimeric antigen receptor, a non-natural T cell receptor, or a combination thereof. In various embodiments, the adoptive cell therapy comprises T cells, natural killer cells, natural killer T cells, or a combination thereof. In various embodiments, method S100 may optionally include obtaining a sample from a patient. In various embodiments, method S100 may include diagnosing the patient as having cancer.

[0091] 2C illustrates a method S300 for predicting patient / treatment response to a therapeutic agent, according to various embodiments. Method S300 may be implemented, for example, via a computing system 300 as described with respect to FIG. 3 below. As shown in FIG. 2C , method S300 may include, in step S310, obtaining or receiving, by one or more processors of computing system 300, an image of a tissue sample from a patient (e.g., image 105); in step S320, generating, by one or more processors, one or more tumor mutation burden (TMB) indicators from the one or more features (e.g., as output 130) via a first neural network model (e.g., model 120); in step S330, generating, by one or more processors, one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features (e.g., as output 150) via a second neural network model (e.g., model 140); and in step S340, generating, by one or more processors, one or more programmed cell death ligand 1 (PD-L1) expression level indicators from the one or more features (e.g., as output 150). Thus, the method includes generating one or more tumor infiltrating lymphocyte (TIL)-associated metrics from the one or more features (e.g., as output 170a) via a third neural network model (e.g., model 160a); in step S350, generating, by one or more processors, one or more cell-based features from the one or more features (e.g., as output 170b) via a fourth neural network model (e.g., model 160b); and in step S360, predicting, by one or more processors, a patient's response status (e.g., response 190) to the cancer immunotherapy therapeutic using the one or more TMB indicators, respectively one or more PD-L1 expression level indicators or one or more PD-1 level indicators, the one or more TIL-associated metrics, and the one or more cell-based features via the fourth neural network model (e.g., model 180).

[0092] In any of Figures 2A-2C, the method may lack one or more tumor mutation burden (TMB) related neural network models, as shown in Figures 2D-2F.

[0093] In Figure 2D, for method S105, S110-2 corresponds to S110 in Figure 2A, in Figure 2D, S120-2 corresponds to S130 in Figure 2A, in Figure 2D, S130-2 corresponds to S140 in Figure 2A, and in Figure 2D, S140-2 corresponds to S150 in Figure 2A.

[0094] In Figure 2E of method S205, S210-2 corresponds to S210 in Figure 2B. In Figure 2E, S220-2 corresponds to S220 in Figure 2B (except that S220-2 lacks a step related to the TMB indicator). In Figure 2E, S230-2 corresponds to S230 in Figure 2B (except that S230-2 lacks a step related to the TMB indicator). In Figure 2E, S240-2 corresponds to S240 in Figure 2B.

[0095] In Figure 2F of method 305, S310-2 corresponds to S310 in Figure 2C. In Figure 2F, S320-2 corresponds to S330 in Figure 2C. In Figure 2F, S330-2 corresponds to S340 in Figure 2C. In Figure 2F, S340-2 corresponds to S350 in Figure 2C. In Figure 2F, S350-2 corresponds to S360 in Figure 2C (except that S350-2 lacks the step related to the TMB indicator).

[0096] III. Computer-Implemented Systems 3 is a block diagram illustrating a computer system 300 configured to perform a method for predicting patient / treatment response to a therapeutic agent according to various embodiments, and in which embodiments of the disclosed systems and methods, or portions thereof, may be implemented. For example, the illustrated computer system may be a local computer system or a remote computer system operably connected to a control system for controlling or monitoring the systems and methods of various embodiments herein. In various embodiments of the present teachings, the computer system 300 may include a bus 302 or other communication mechanism for communicating information and a processor 304 coupled to the bus 302 for processing information. In various embodiments, the computer system 300 may also include memory, which may be a random-access memory (RAM) 306 or other dynamic storage device coupled to the bus 302 for determining instructions to be executed by the processor 304. The memory may also be used to store temporary variables or other intermediate information during execution of instructions by the processor 304. In various embodiments, computer system 300 may further include a read only memory (ROM) 308 or other static storage device coupled to bus 302 for storing static information and instructions for processor 304. A storage device 310, such as a magnetic disk or optical disk, may be provided and coupled to bus 302 for storing information and instructions.

[0097] In various embodiments, computer system 300 may be coupled via bus 302 to a display 312 (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD)) for displaying information to a computer user. An input device 314, including alphanumeric and other keys, may be coupled to bus 302 for communicating information and command selections to processor 304. Another type of user input device is a cursor control 316, such as a mouse, trackball, or cursor direction keys, for communicating directional information and command selections to processor 304 and for controlling cursor movement on display 312. This input device 314 typically has two degrees of freedom in two axes, a first axis (i.e., x) and a second axis (i.e., y), allowing the device to specify a position in a plane. However, it should be understood that input devices 314 that allow three-dimensional (x, y, and z) cursor movement are also contemplated herein. According to various embodiments, components 312 / 314 / 316, together or individually, may comprise a control system that connects the remaining components of a computer system to the systems herein and methods performed on such systems, and controls the execution of the methods and related system operations.

[0098] Consistent with particular implementations of the present teachings, results may be provided by computer system 300 in response to processor 304 executing one or more sequences of one or more instructions contained in memory 306. Such instructions may be read into memory 306 from another computer-readable medium or computer-readable storage medium, such as storage device 310. Execution of the sequences of instructions contained in memory 306 may cause processor 304 to perform the processes described herein. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement the present teachings. Thus, the techniques of this disclosure are not limited to any specific combination of hardware circuitry and software.

[0099] The terms "computer-readable medium" (e.g., data store, data storage device, etc.) or "computer-readable storage medium," as used herein, refer to any medium that participates in providing instructions to processor 304 for execution. Such media may take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Examples of non-volatile media may include, but are not limited to, dynamic memory, such as memory 306. Examples of transmission media may include, but are not limited to, coaxial cables, copper wire, and fiber optics, including the wires that comprise bus 302.

[0100] Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, RAM, PROM, and EPROM, FLASH-EPROM, another memory chip or cartridge, or any other tangible medium from which a computer can read.

[0101] In addition to computer-readable media, instructions or data may be provided as signals on a transmission medium included in a communication device or system to provide sequences of one or more instructions to the processor 304 of the computer system 300 for execution. For example, a communication device may include a transceiver having signals indicative of instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in this disclosure. Representative examples of data communication transmission connections may include, but are not limited to, telephone modem connections, wide area networks (WANs), local area networks (LANs), infrared data connections, NFC connections, etc.

[0102] It should be understood that the methodologies, flowcharts, diagrams, and accompanying disclosure described herein may be implemented using computer system 300 as a standalone device, or on a distributed network or shared computer processing resources, such as a cloud computing network. In one or more embodiments, computer system 300 may include a single computer (or computer system) or multiple computers communicating with each other in a distributed implementation.

[0103] The methodologies described herein may be implemented by various means, depending on the application. For example, these methodologies may be implemented in hardware, firmware, software, or any combination thereof. In the case of a hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or any combination thereof.

[0104] In various embodiments, the methods of the present teachings may be implemented as firmware and / or software programs and applications written in conventional programming languages ​​such as C, C++, Python, etc. When implemented as firmware and / or software, the embodiments described herein may be implemented on a non-transitory computer-readable medium having stored thereon a program for causing a computer to perform the above-described methods. It should be understood that the various engines described herein may be provided on a computer system such as computer system 300, whereby processor 304 performs the analyses and decisions provided by these engines according to instructions provided by any one or combination of memory components 306 / 308 / 310 and user input provided via input device 314.

[0105] While the present teachings will be described in conjunction with various embodiments, it is not intended that the present teachings be limited to such various embodiments. On the contrary, the present teachings encompass various alternatives, modifications, and equivalents, as will be appreciated by those skilled in the art.

[0106] In describing various embodiments, the specification may present a method and / or process as a particular series of steps. However, to the extent that the method or process does not depend on the particular order of steps described herein, the method or process should not be limited to the particular sequence of steps described, and one of ordinary skill in the art will readily appreciate that the sequence may be varied and still be within the spirit and scope of the various embodiments.

[0107] V. Tumor Mutation Burden (TMB) In certain embodiments, the disclosed systems, compositions, and methods utilize tumor mutational burden (TMB) as at least one parameter or metric to determine whether a patient is likely to respond to a particular type of therapy. In various embodiments, tumor mutational burden may include the total number of mutations (alterations) present in the DNA of cancer cells. In particular cases, cancers (including solid tumors) with a large number of mutations may be more likely to respond to certain types of immunotherapy, including cancer immunotherapy therapeutics. In specific embodiments, TMB is defined as the number of non-genetic mutations per million bases (Mb) of interrogated genomic sequence (Merino, et al. (2020). J Immunother Cancer. 8(1):e000147, incorporated herein by reference in its entirety). Measurement of TMB may be performed by any suitable method, including targeted panel sequencing, gene-targeted sequencing, or whole-exome sequencing, and in specific embodiments, is performed by, for example, next-generation sequencing (NGS). The quantitative value of TMB for a sample can be considered a TMB indicator, and in specific cases, the amount is measured as the number of mutations per megabase (Mb). The systems, compositions, and methods of the present disclosure can generate (by one or more processors via a neural network model) one or more tumor mutation burden (TMB) indicators from one or more features of an image of a tissue sample from an individual having or suspected of having bladder cancer. In specific embodiments, one or more processors can obtain an image of a tissue sample (from a patient), including an image including one or more features of the tissue sample, and then one or more processors can generate one or more TMB indicators from the one or more features via a first neural network model.

[0108] In particular embodiments, the neural network model is configured to generate one or more TMB indicators from one or more features of a patient tissue sample. In specific embodiments, the neural network model includes a classification algorithm, and the one or more TMB indicators include a TMB indicator for a tissue sample with either a TMB-positive classification or a TMB-negative classification. In some embodiments, the neural network model includes a classification algorithm, and if the tissue sample has a TMB value above a predetermined TMB threshold, the one or more TMB indicators include a TMB indicator for a tissue sample with a positive classification. In such cases utilizing a classification algorithm, in certain embodiments, the output may be considered TMB High or TMB Low. In certain embodiments, TMB High may be 10 or greater, while TMB Low is less than 10.

[0109] However, in certain embodiments, the associated neural network model includes a regression algorithm and the one or more TMB indicators include a TMB indicator that is a quantitative TMB score. In specific cases, the TMB score may range from 0 to 100, and the predetermined TMB threshold may be any value therein (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 11 , 91, 92, 93, 94, 95, 96, 97, 98, 99, or 100).

[0110] In particular cases, the TMB prediction parameters for the model are trained using hematoxylin and eosin (H&E) images from, for example, muscle invasive bladder cancer (MIBC) patients. In various embodiments, in the generation of the model, and at the time of its use, TMB is calculated from an NGS molecular panel applied to the tissue.

[0111] VI. Checkpoint Expression Levels In embodiments of the present disclosure, systems, methods, and compositions utilize checkpoint expression levels as at least one parameter or metric to determine whether a patient is likely to be a responder to a particular type of therapy. In some cases, the checkpoint is any type of checkpoint, but in specific embodiments, the checkpoint is an inhibitory checkpoint. In certain embodiments, the checkpoint is programmed cell death 1 (PD-1) or programmed cell death ligand 1 (PD-L1). In particular embodiments, checkpoint expression is quantified based on images of tissue samples from patients. In specific embodiments, checkpoint expression is quantified based on H&E images of tissue samples from patients. In specific embodiments, a neural network model can be trained based on one or more features from H&E images (e.g., from bladder cancer patients, including muscle-invasive bladder cancer patients) for one or more PD-L1 expression level indicators or PD-1 expression level indicators.

[0112] In certain embodiments, the neural network model comprises a classification algorithm, and the one or more PD-L1 expression level indicators comprise a PD-L1 expression level indicator that is either a PD-L1 expression positive classification or a PD-L1 expression level negative classification. In other embodiments, the neural network model comprises a classification algorithm, and the one or more PD-1 expression level indicators comprise a PD-1 expression level indicator that is either a PD-1 expression positive classification or a PD-1 expression level negative classification. In some embodiments, if the tissue sample has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value, the classification algorithm generates a PD-L1 expression positive classification. In such cases, the output may be PD-L1 High or PD-L1 Low. In some embodiments, if the tissue sample has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value, the classification algorithm generates a PD-1 expression positive classification. In such cases, the output may be PD-1 High or PD-1 Low. The thresholds for PD-L1 High and PD-L1 Low, or PD-1 High and PD-1 Low, may vary depending on the cancer immunotherapy therapeutic and / or cancer, and in certain embodiments, there are different thresholds that apply based on the drug and cancer.

[0113] In certain embodiments, the neural network model comprises a regression algorithm and the one or more PD-L1 expression level indicators comprise a quantitative PD-L1 score, which may be a tumor proportion score (TPS) or a combined positive score (CPS). In certain embodiments, the neural network model comprises a regression algorithm and the one or more PD-1 expression level indicators comprise a quantitative PD-1 score, which may be a tumor proportion score (TPS) or a combined positive score (CPS). For the regression algorithms for both PD-L1 and PD-1, the TPS score ranges from 0 to 100, e.g., 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44 , 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, or 100. For the regression algorithms for both PD-L1 and PD-1, the CPS score ranges from 0 to 100, e.g., 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44 , 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, or 100.In a specific embodiment, PD-L1 High or PD-1 High is 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54 , 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, or 100 or more TPSs or CPSs.

[0114] In some cases, the model directly predicts TPS and / or CPS scores (i.e., a regression model). In some embodiments, the model directly predicts PD-L1 High vs. PD-L1 Low for different thresholds, or PD-1 High vs. PD-1 Low for different thresholds. Specific cases may utilize, for example, TPS≧1, TPS≧10, or TPS≧50.

[0115] VII. Tumor-infiltrating lymphocyte (TIL)-related metrics Tumor-infiltrating lymphocytes are a type of white blood cell that migrate from the blood toward tumors. Tumor-infiltrating lymphocytes may include both T cells and B cells and recognize and kill cancer cells. In particular embodiments, the presence of lymphocytes in tumors is often associated with better clinical outcomes. The present disclosure provides systems, methods, and compositions that utilize a TIL-related metric as at least one parameter to determine whether a patient is likely to respond to a particular type of therapy. In specific embodiments, the presence of TILs in tumor samples is measured by any suitable method, including immunohistochemical staining, H&E staining, or both. In specific embodiments, the presence of TILs in tumors is determined by a pathologist, who may utilize IHC staining on tissue samples to visualize TIL cells, or they may be visualized on H&E slides. TILs have distinctive characteristics that allow pathologists to distinguish them from other cells, such as being more purple, darker, and rounder than tumor cells. In specific embodiments, the algorithm is trained using a pathologist's annotations of TILs on H&E images. In specific embodiments, the pathologist evaluates TILs in the stromal compartment of the tumor. In specific embodiments, the pathologist evaluates TILs at the border of an invasive tumor, for example. TILs outside the tumor border, within the necrotic zone, within the fibrotic zone, and / or associated with abscesses are excluded in specific cases. The pathologist may evaluate one or more H&E-stained sections of the tumor.

[0116] In particular embodiments, the neural network model is configured to calculate one or more TIL-related metrics from one or more features of the tissue sample. In particular embodiments, the TIL-related metrics can be calculated based on a TIL mask, a tumor mask, a necrosis mask, a stroma mask, and a cell nucleus segmentation mask. Several masks can be used to calculate "TIL-related" metrics (e.g., iTIL%, sTIL%, TIL clusters, etc.). The neural network can predict a segmentation mask (i.e., a TIL mask on top of the tissue image along with other masks such as tumor, stroma, necrosis, and similar cell nucleus segmentation). These segmentation masks are then used to calculate metrics such as the following features: tumor-infiltrating lymphocyte (TIL) percentage (%), total number of TILs, number of TIL clusters, distance between TIL cells compared to the distance between tumor cells, distance of TIL cells from tumor cells, average color of TIL cells, average size of TIL cells, etc.

[0117] In various embodiments, the neural network model may be a cell segmentation algorithm that inputs an image and performs cell segmentation. In various embodiments, the segmented cells may be labeled based on the region segmentation output. For example, a segmented cell may be labeled as a TIL cell if the cell is within a TIL region. In various embodiments, the cell segmentation algorithm may calculate one or more cell-based features. In various embodiments, the cell-based features may include morphology, color (nuclear), color (extranuclear), texture (energy, correlation, contrast, homogeneity, dissimilarity, angular second moment), chromatin skeletal morphology, and color, among many other features that may be extracted. In various embodiments, the cell segmentation algorithm may be configured to generate the number or average number of TIL cells, average tumor cell size, average TIL cell size, average distance between TIL cells, average color of TIL cells, and average distance between TIL cells compared to tumor cells. In various embodiments, the TIL-related metric is the predicted TIL cluster spread in the identified tumor region, the total number of TILs in the identified tumor region, the distance between TIL cells compared to the distance between tumor cells, the distance of TIL cells from tumor cells, the average color of TIL cells, and / or the average size of TIL cells.

[0118] VIII. Cancer Immunotherapy Drugs The present disclosure provides embodiments in which an individual in need of cancer immunotherapy is evaluated for responder status to cancer immunotherapy. Embodiments of the present disclosure include predicting patient response to a cancer immunotherapy therapeutic. In specific embodiments, the cancer immunotherapy therapeutic is an immune checkpoint inhibitor. The cancer immunotherapy therapeutic may be an antibody or adoptive cell therapy using immune cells, or both may be utilized, for example. In specific embodiments, the antibody is a monoclonal antibody, and the monoclonal antibody may target any checkpoint. In specific cases, the antibody may be monospecific or bispecific. The antibody may target, for example, PD-L1 or PD-1. When adoptive cell therapy is utilized, the cell may be an immune cell, such as, for example, a T cell, a natural killer (NK) cell, or an NK T cell. In specific embodiments, the cell may be engineered to express a non-native protein, such as an antigen receptor, which may include an extracellular domain that is an antibody that targets an immune checkpoint. In other cases, the antigen receptor includes an extracellular domain that binds to an immune checkpoint. In specific embodiments, the cancer immunotherapy therapeutic is a checkpoint inhibitor, such as a PD-L1 inhibitor or a PD-1 inhibitor. In particular embodiments, the cancer immunotherapy therapeutic can be combined with one or more other types of cancer therapy, such as surgery, radiation, hormone therapy, other immunotherapy, or chemotherapy. **********

[0119] In particular embodiments, the method includes utilizing an image of a tissue sample from the patient, the tissue sample including one or more features. In specific embodiments, the method includes obtaining, by one or more processors from a data source, the image of the tissue sample from the patient and the one or more features of the tissue sample. In either case, the method may include generating, e.g., by one or more processors via one or more neural network models, (a) one or more tumor mutational burden (TMB) indicators from the one or more features, (b) one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features, or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features, and (c) one or more tumor infiltrating lymphocyte (TIL) metric values ​​from the one or more features; predicting, by the one or more processors via one or more neural network models, the one or more TMB indicators, the one or more PD-L1 expression level indicators, or the one or more PD-1 expression level indicators, and the one or more TIL metrics, for the patient's response status to the cancer immunotherapy therapeutic; and selecting, based on the patient's response status, whether to include the patient in a cohort of patients participating in a clinical trial or whether to administer a therapeutically effective amount of the one or more cancer immunotherapy therapeutics to the patient.

[0120] In particular embodiments, the method includes utilizing an image of a tissue sample from a patient, the tissue sample including one or more features; obtaining, by one or more processors from a data source, the image of the tissue sample from the patient and the one or more features of the tissue sample; and computing, by the one or more processors via one or more neural network models, (a) one or more tumor mutational burden (TMB) indicators from the one or more features; (b) one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features; or (c) one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features. and (c) generating one or more tumor infiltrating lymphocyte (TIL) percentage (%) values ​​(as an example of TIL metric values) from the one or more features; and predicting, by one or more processors via one or more neural network models, a patient's response status to the cancer immunotherapy therapeutic using the one or more TMB indicators, the one or more PD-L1 expression level indicators or the one or more PD-1 expression level indicators, and the one or more TIL% values; and selecting, based on the patient's response status, whether to include the patient in a cohort of patients participating in a clinical trial.

[0121] Various embodiments of the method include obtaining, by one or more processors, an image of a tissue sample from a patient, the image including one or more features of the tissue sample. In some cases, the party obtaining the image is the same party that handles the processing and / or production steps. In other cases, the party obtaining the image is not the same party that handles the processing and / or production steps. The tissue sample may be obtained from an archive or may be evaluated immediately after collection. The tissue sample may or may not be derived from a biopsy of an individual who has been diagnosed with cancer. The individual may be known to have or suspected of having cancer based on one or more symptoms.

[0122] In specific embodiments, the disclosed systems, methods, and compositions can be used for any type of cancer, including hematological and solid tumors. The cancer can be of any stage or grade, can be refractory or non-refractory, and can be metastatic or non-metastatic. The cancer can be of the bladder, brain, breast, lung, colon, pancreas, prostate, liver, spleen, kidney, stomach, cervix, uterus, ovary, testis, gallbladder, bone, thyroid, skin, endometrium, rectum, etc.

[0123] In some embodiments, the subject has one or more symptoms of bladder cancer, such as blood in the urine; the need to urinate more frequently than usual; pain or burning during urination; feeling the need to urinate immediately even when the bladder is not full; difficulty urinating or a weak urine stream; having to get up to urinate multiple times at night; being unable to urinate; unilateral lower back pain; loss of appetite and weight loss; feeling tired or sluggish; swelling in the legs; bone pain; or a combination thereof. In such cases, the subject may be suspected of having bladder cancer or may be known to have bladder cancer. Diagnosis of bladder cancer can be performed using cystoscopy, biopsy, blood test, urine cytology, computerized tomography (CT) urography, pyelography, or a combination thereof. Other tests, such as magnetic resonance imaging (MRI), positron emission tomography (PET), bone scan, and / or chest X-ray, may also be performed to determine the extent of cancer. Bladder cancer can be urothelial carcinoma, squamous cell carcinoma, or adenocarcinoma. Bladder cancer can be non-invasive, non-muscle invasive, or muscle invasive. Bladder cancer can be of any stage or grade, including papillary carcinoma in situ, carcinoma in situ, T1, T2, T3, or T4.

[0124] Embodiments of the present disclosure include treatment methods for individuals determined to be responders based on the systems, methods, and compositions encompassed herein. In specific embodiments, responder individuals are administered a therapeutically effective amount of one or more cancer immunotherapy therapeutics based on measurements made by the systems, methods, and compositions encompassed herein. In specific embodiments, non-responder individuals are not administered one or more cancer immunotherapy therapeutics based on measurements made by the systems, methods, and compositions encompassed herein. In various embodiments, a therapeutically effective amount of one or more cancer immunotherapy therapeutics is administered to an individual determined to have or be predicted to have a response to one or more cancer immunotherapy therapeutics based on having suitable one or more TMB indicators, one or more PD-L1 and / or PD-1 expression level indicators, and one or more TIL-associated metrics (all from one or more features of an image of a tissue sample). In a particular embodiment, a therapeutically effective amount of one or more cancer immunotherapy therapeutics is administered to an individual determined to have or predicted to have a response to the one or more cancer immunotherapy therapeutics based on providing an image of a tissue sample from the patient, the tissue sample including one or more features of the tissue sample; generating, by one or more processors via a first neural network model, one or more TMB masks from the one or more features; generating, by one or more processors via a second neural network model, one or more PD-L1 expression level indicators from the one or more features; generating, by one or more processors via a third neural network model, one or more TIL-associated metrics from the one or more features; and predicting, by one or more processors via a fourth neural network model, that the individual will likely respond to the cancer immunotherapy therapeutic.In certain embodiments, a therapeutically effective amount of one or more cancer immunotherapy therapeutics is administered to an individual determined to have or predicted to have a response to the one or more cancer immunotherapy therapeutics based on: obtaining, by one or more processors from a data source, an image of a tissue sample from the patient and one or more features of the tissue sample; generating, by the one or more processors, via one or more neural network models, (a) one or more tumor mutational burden (TMB) indicators from the one or more features, (b) one or more programmed death-ligand 1 (PD-L1) expression level indicators from the one or more features, and (c) one or more tumor infiltrating lymphocyte (TIL) percentage (%) values ​​from the one or more features; and determining or predicting, by the one or more processors, via the one or more neural network models, that the patient is likely to respond to the cancer immunotherapy therapeutics using the one or more TMB indicators, the one or more PD-L1 expression level indicators, and the one or more TIL-associated metric values.

[0125] IX. Citation of Embodiments Embodiment 1. A system for predicting a patient response to a cancer immunotherapy therapeutic, the system comprising: a data store for storing images of a tissue sample from the patient, the images including one or more features of the tissue sample; and a computing device communicatively coupled to the data store, the computing device configured to generate one or more tumor mutation burden (TMB) indicators from the one or more features, one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features, or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features. a computing device including a second neural network model configured to generate a tumor-infiltrating lymphocyte (TIL)-related metric from the one or more features; a third neural network model configured to calculate one or more tumor-infiltrating lymphocyte (TIL)-related metrics from the one or more features; or a fourth neural network model configured to predict a patient's response status to a cancer immunotherapy therapeutic according to one or more TMB indicators, one or more PD-L1 expression level indicators, or one or more PD-1 expression level indicators, and / or one or more TIL-related metrics; and a display system communicatively connected to the computing device and configured to display results indicative of the patient's response status.

[0126] Embodiment 2. The system of embodiment 1, wherein the prediction of a patient's response status to a cancer immunotherapy therapeutic is made according to one or more medical history features of the patient.

[0127] Embodiment 3. The system of embodiment 1 or 2, wherein the third neural network model includes a segmentation algorithm configured to generate an identification of tumor regions in the tissue sample based on one or more features, and the identification is used to further generate one or more TIL-associated metrics.

[0128] Embodiment 4. The system of embodiment 3, wherein the segmentation algorithm is configured to predict a TIL mask, which is used in combination with the tumor mask to calculate one or more TIL-related metrics.

[0129] Embodiment 5. The system of embodiment 4, wherein the one or more TIL-associated metrics include a predicted number of TIL clusters in the identified tumor region.

[0130] Embodiment 6. The system of embodiment 4, wherein the one or more TIL-related metrics include a predicted size of a TIL cluster in the identified tumor region.

[0131] Embodiment 7. The system of embodiment 4, wherein the one or more TIL-related metrics include a predicted TIL cluster spread in the identified tumor region.

[0132] Embodiment 8. The system of embodiment 4, wherein the one or more TIL-related metrics include the total number of TILs in the identified tumor region.

[0133] Embodiment 9. The system of embodiment 4, wherein the one or more TIL-related metrics include the distance between TIL cells compared to the distance between tumor cells.

[0134] Embodiment 10. The system of embodiment 4, wherein the one or more TIL-associated metrics include the distance of TIL cells from tumor cells.

[0135] Embodiment 11. The system of embodiment 4, wherein the one or more TIL-related metrics include the mean color of the TIL cells.

[0136] Embodiment 12. The system of embodiment 4, wherein the one or more TIL-related metrics include the average size of TIL cells.

[0137] Embodiment 13. A system described in any one of embodiments 1 to 12, wherein the one or more TIL-related metrics include intratumoral TIL% (iTIL%).

[0138] Embodiment 14. The system of embodiment 13, wherein iTIL% is calculated as the percentage of tumor area identified in the tissue sample that is infiltrated by TILs.

[0139] Embodiment 15. The system of embodiment 1, wherein the one or more TIL-related metrics include stromal TIL% (sTIL%).

[0140] Embodiment 16. The system of embodiment 15, wherein the sTIL% value is calculated as the percentage of stroma within the tumor area identified in the tissue sample that is infiltrated by TILs.

[0141] Embodiment 17. The system of any one of embodiments 1 to 16, wherein the first neural network model includes a classification algorithm and the one or more TMB indicators include TMB indicators for tissue samples that are either TMB-positive or TMB-negative classified.

[0142] Embodiment 18. The system of embodiment 17, wherein the classification algorithm generates a TMB positive classification if the tissue sample has a TMB value above a predetermined TMB threshold.

[0143] Embodiment 19. The system of any one of embodiments 1 to 16, wherein the first neural network model includes a regression algorithm and the one or more TMB indicators include a TMB indicator that is a quantitative TMB score.

[0144] Embodiment 20. The system of any one of embodiments 1 to 19, wherein the second neural network model comprises a classification algorithm, and the one or more PD-L1 expression level indicators comprise a PD-L1 expression level indicator that is either a PD-L1 expression positive classification or a PD-L1 expression level negative classification.

[0145] Embodiment 21. The system of any one of embodiments 1 to 19, wherein the second neural network model comprises a classification algorithm and the one or more PD-1 expression level indicators comprise a PD-1 expression level indicator that is either a PD-1 expression positive classification or a PD-1 expression level negative classification.

[0146] Embodiment 22. The system of embodiment 20, wherein the classification algorithm generates a PD-L1 expression positive classification if the tissue sample has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value.

[0147] Embodiment 23. The system of embodiment 21, wherein the classification algorithm generates a PD-1 expression positive classification if the tissue sample has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value.

[0148] Embodiment 24. The system of any one of embodiments 1 to 19, wherein the second neural network model comprises a regression algorithm and the one or more PD-L1 expression level indicators comprise a quantitative PD-L1 score.

[0149] Embodiment 25. The system of embodiment 1, wherein the second neural network model comprises a regression algorithm and the one or more PD-1 expression level indicators comprise a quantitative PD-1 score.

[0150] Embodiment 26. The system of embodiment 24, wherein the quantitative PD-L1 score is a tumor proportion score (TPS).

[0151] Embodiment 27. The system of embodiment 24, wherein the quantitative PD-L1 score is a combined positive score (CPS).

[0152] Embodiment 28. The system of embodiment 25, wherein the quantitative PD-1 score is a tumor proportion score (TPS).

[0153] Embodiment 29. The system of embodiment 25, wherein the quantitative PD-1 score is a combined positive score (CPS).

[0154] Embodiment 30. A system described in any one of embodiments 1 to 29, wherein the cancer immunotherapy therapeutic agent is a checkpoint inhibitor.

[0155] Embodiment 31. The system of embodiment 30, wherein the checkpoint inhibitor is a PD-L1 inhibitor or a PD-1 inhibitor.

[0156] Embodiment 32. A system described in any one of embodiments 1 to 31, wherein the third neural network model is further configured to generate one or more TIL masks from the one or more features, and the fourth neural network model is further configured to predict a patient's response status to a cancer immunotherapy therapeutic using the one or more TIL masks.

[0157] Embodiment 33. The system of any one of embodiments 1, wherein the tissue sample is from the patient's bladder and the patient response to the cancer immunotherapy therapeutic is related to bladder cancer.

[0158] Embodiment 34. The system of any one of embodiments 1 to 33, wherein the tissue sample is from a patient known to have a PD-L1-positive cancer.

[0159] Embodiment 35. A system described in any one of embodiments 1 to 33, wherein the tissue sample is from a patient known to have a PD-1 positive cancer.

[0160] Embodiment 36. A system described in any one of embodiments 1 to 35, wherein the tissue sample is from a patient known to have any one or more of non-small cell lung cancer, melanoma, Hodgkin's lymphoma, bladder cancer, kidney cancer, and breast cancer.

[0161] Embodiment 37. A system described in any one of embodiments 1 to 36, wherein the cancer immunotherapy therapeutic agent targets PD-L1 or PD-1.

[0162] Embodiment 38. The system of any one of embodiments 1 to 37, wherein the cancer immunotherapy therapeutic comprises one or more antibodies, one or more adoptive cell therapy agents, or a combination thereof.

[0163] Embodiment 39. The system of embodiment 38, wherein the one or more antibodies comprise at least one of a monoclonal antibody, a bispecific antibody, or a trispecific antibody.

[0164] Embodiment 40. The system of embodiment 38, wherein the one or more adoptive cell therapies comprise immune cells expressing one or more engineered antigen receptors.

[0165] Embodiment 41. The system of embodiment 40, wherein the one or more engineered antigen receptors comprise at least one of a chimeric antigen receptor, a non-natural T cell receptor, or a combination thereof.

[0166] Embodiment 42. A system described in any one of embodiments 38 to 41, wherein the one or more adoptive cell therapies include at least one of T cells, natural killer cells, natural killer T cells, or a combination thereof.

[0167] Embodiment 43. The system of embodiment 1 or 2, wherein the third neural network model includes a region / TIL segmentation algorithm and a cell segmentation algorithm.

[0168] Embodiment 44. The system of embodiment 43, wherein the cell segmentation algorithm calculates one or more cell-based features.

[0169] Embodiment 45. The system of embodiment 44, wherein the one or more cell-based features include morphology, color (nuclear), color (extranuclear), texture (energy, correlation, contrast, uniformity, dissimilarity, or angular second moment).

[0170] Embodiment 46. The system of embodiment 43, wherein the cell segmentation algorithm generates a number of TIL cells or a mean number of TIL cells.

[0171] Embodiment 47. The system of embodiment 43, wherein the cell segmentation algorithm generates an average tumor cell size.

[0172] Embodiment 48. The system of embodiment 43, wherein the cell segmentation algorithm generates an average TIL cell size.

[0173] Embodiment 49. The system of embodiment 43, wherein the cell segmentation algorithm generates an average distance between TIL cells.

[0174] Embodiment 50. The system of embodiment 43, wherein the cell segmentation algorithm generates an average color of the TIL cells.

[0175] Embodiment 51. The system of embodiment 43, wherein the cell segmentation algorithm generates an average distance between TIL cells compared to tumor cells.

[0176] Embodiment 52. A method for predicting patient response to a cancer immunotherapy therapeutic, the method comprising: obtaining, by one or more processors, an image of a tissue sample from the patient, the image including one or more features of the tissue sample; generating, by the one or more processors via a first neural network model, one or more tumor mutational burden (TMB) indicators from the one or more features; generating, by the one or more processors via a second neural network model, one or more programmed death-ligand 1 (PD-L1) expression level indicators from the one or more features or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features; generating, by the one or more processors via a third neural network model, one or more tumor infiltrating lymphocyte (TIL)-associated metrics from the one or more features; and predicting, by the one or more processors via a fourth neural network model, the patient's response status to the cancer immunotherapy therapeutic using the one or more TMB indicators, the one or more PD-L1 expression level indicators, or the one or more PD-1 level indicators, and one or more associated metrics.

[0177] Embodiment 53. 53. The method of embodiment 52, further comprising selecting whether or not the patient is included in a cohort of patients participating in a clinical trial according to the patient's response status.

[0178] Embodiment 54. The method of embodiment 52 or 53, further comprising displaying on a display screen a graphical representation indicating the patient's response status.

[0179] Embodiment 55. The method of any one of embodiments 52 to 54, wherein predicting a patient's response status to a cancer immunotherapy therapeutic further uses one or more medical history features of the patient.

[0180] Embodiment 56. The method of any one of embodiments 52 to 55, wherein the third neural network is a segmentation algorithm that identifies tumor regions in the tissue sample.

[0181] Embodiment 57. The method of embodiment 56, wherein the segmentation algorithm predicts one or more TIL mask structure features in the identified tumor region to generate one or more TIL-related metrics.

[0182] Embodiment 58. The method of embodiment 57, wherein the predicted TIL mask structure feature is a predicted number of TIL clusters in the identified tumor region.

[0183] Embodiment 59. The method of embodiment 57, wherein the predicted TIL mask structure feature is the predicted size of the TIL cluster in the identified tumor region.

[0184] Embodiment 60. The method of embodiment 57, wherein the predicted TIL mask structure feature is a predicted TIL cluster spread in the identified tumor region.

[0185] Embodiment 61. The method of embodiment 57, wherein the one or more TIL mask structure features include the total number of TILs in the identified tumor region.

[0186] Embodiment 62. The method of embodiment 57, wherein the one or more TIL mask structure features comprise the distance between TIL cells compared to the distance between tumor cells.

[0187] Embodiment 63. The method of embodiment 57, wherein one or more TIL mask structure features include a distance of TIL cells from tumor cells.

[0188] Embodiment 64. The method of embodiment 57, wherein one or more TIL mask structure features comprise an average color of the TIL cells.

[0189] Embodiment 65. The method of embodiment 57, wherein one or more TIL mask structure features comprise an average size of TIL cells.

[0190] Embodiment 66. The method of any one of embodiments 52 to 65, wherein the TIL% value is intratumoral TIL% (iTIL%).

[0191] Embodiment 67. The method of embodiment 66, wherein iTIL% is calculated as the percentage of tumor area identified in the tissue sample that is infiltrated by TILs.

[0192] Embodiment 68. The method of any one of embodiments 52 to 65, wherein the TIL% value is stromal TIL% (sTIL%).

[0193] Embodiment 69. The method of embodiment 68, wherein the sTIL% value is calculated as the percentage of stroma within the tumor area identified in the tissue sample that is infiltrated by TILs.

[0194] Embodiment 70. The method of any one of embodiments 52 to 69, wherein the first neural network model is a classification algorithm that generates a TMB indicator that is either a TMB-positive or a TMB-negative classification for the tissue sample.

[0195] Embodiment 71. The method of embodiment 70, wherein the classification algorithm generates a TMB-positive classification if the tissue sample has a TMB value above a predetermined TMB threshold.

[0196] Embodiment 72. The method of any one of embodiments 52 to 69, wherein the first neural network model is a regression algorithm that generates a TMB indicator that is a quantitative TMB score.

[0197] Embodiment 73. The method of any one of embodiments 52 to 72, wherein the second neural network model is a classification algorithm that generates a PD-L1 expression level indicator that is either a PD-L1 expression positive classification or a PD-L1 expression level negative classification.

[0198] Embodiment 74. The method of any one of embodiments 52 to 72, wherein the second neural network model is a classification algorithm that generates a PD-1 expression level indicator that is either a PD-1 expression positive classification or a PD-1 expression level negative classification.

[0199] Embodiment 75. The method of embodiment 73, wherein the classification algorithm generates a positive PD-L1 expression classification if the tissue sample has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value.

[0200] Embodiment 76. The method of embodiment 74, wherein the classification algorithm generates a positive PD-1 expression classification if the tissue sample has a PD-1 expression level value that exceeds a predetermined PD-1 expression level value.

[0201] Embodiment 77. The method of any one of embodiments 52 to 72, wherein the second neural network model is a regression algorithm that generates a PD-L1 expression level indicator that is a quantitative PD-L1 score.

[0202] Embodiment 78. The method of any one of embodiments 52 to 72, wherein the second neural network model is a regression algorithm that generates a PD-1 expression level indicator that is a quantitative PD-1 score.

[0203] Embodiment 79. The method of embodiment 77, wherein the quantitative PD-L1 score is a tumor proportion score (TPS).

[0204] Embodiment 80. The method of embodiment 78, wherein the quantitative PD-1 score is a tumor proportion score (TPS).

[0205] Embodiment 81. The method of embodiment 77, wherein the quantitative PD-L1 score is a combined positive score (CPS).

[0206] Embodiment 82. The method of embodiment 78, wherein the quantitative PD-1 score is a combined positive score (CPS).

[0207] Embodiment 83. The method of any one of embodiments 52 to 82, wherein the cancer immunotherapy therapeutic is a checkpoint inhibitor.

[0208] Embodiment 84. The method of embodiment 83, wherein the checkpoint inhibitor is a PD-L1 inhibitor or a PD-1 inhibitor.

[0209] Embodiment 85. The method of any one of embodiments 52 to 84, further comprising generating, by one or more processors, one or more TIL masks from the one or more features via a third neural network model, and predicting the patient's response status to the cancer immunotherapy therapeutic further comprises using the one or more TIL masks.

[0210] Embodiment 86. The method of any one of embodiments 52 to 85, wherein the tissue sample is from the bladder and the patient response to the cancer immunotherapy therapeutic is associated with bladder cancer.

[0211] Embodiment 87. The method of any one of embodiments 52 to 86, wherein a determination is made that the patient is or will be a responder based on response status, and in accordance with the determination, the patient is administered a therapeutically effective amount of a cancer immunotherapy therapeutic.

[0212] Embodiment 88. The method of any one of embodiments 52 to 86, wherein a determination is made that the patient is not or will not be a responder based on response status, and in accordance with the determination, the patient is not administered a cancer immunotherapy therapeutic agent.

[0213] Embodiment 89. The method of any one of embodiments 52 to 88, wherein the tissue sample is from a patient known to have a PD-L1 positive cancer or a PD-1 positive cancer.

[0214] Embodiment 90. The method of any one of embodiments 52 to 89, wherein the tissue sample is from a patient known to have any of non-small cell lung cancer, melanoma, Hodgkin's lymphoma, bladder cancer, kidney cancer, or breast cancer.

[0215] Embodiment 91. The method of any one of embodiments 52 to 90, wherein the cancer immunotherapy therapeutic targets PD-L1 or PD-1.

[0216] Embodiment 92. The method of any one of embodiments 52-91, wherein the cancer immunotherapy therapeutic comprises one or more antibodies, adoptive cellular therapies, immunomodulators, or combinations thereof.

[0217] Embodiment 93. The method of embodiment 92, wherein the antibody is a monoclonal antibody, a bispecific antibody, or a trispecific antibody.

[0218] Embodiment 94. The method of embodiment 92, wherein the adoptive cell therapy comprises immune cells expressing one or more engineered antigen receptors.

[0219] Embodiment 95. The method of embodiment 94, wherein the engineered antigen receptor is a chimeric antigen receptor, a non-natural T cell receptor, or a combination thereof.

[0220] Embodiment 96. The method of any one of embodiments 92-94, wherein the adoptive cell therapy comprises T cells, natural killer cells, natural killer T cells, or a combination thereof.

[0221] Embodiment 97. The method of any one of embodiments 52 to 96, further comprising obtaining a sample from a patient.

[0222] Embodiment 98. The method of any one of embodiments 52 to 97, further comprising diagnosing the patient as having cancer.

[0223] Embodiment 99. A method comprising: obtaining, by one or more processors from a data source, an image of a tissue sample from a patient and one or more features of the tissue sample; generating, by the one or more processors via one or more neural network models, one or more of: (a) one or more tumor mutational burden (TMB) indicators from the one or more features; (b) one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features, or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features; and (c) one or more tumor infiltrating lymphocyte (TIL) percentage (%) values ​​from the one or more features; predicting, by the one or more processors via the one or more neural network models, a patient's response status to a cancer immunotherapy therapeutic using one or more of the one or more TMB indicators, one or more PD-L1 expression level indicators, one or more PD-1 expression level indicators, and / or one or more TIL% values; and determining whether to include the patient in a cohort of patients participating in a clinical trial based on the patient's response status.

[0224] Embodiment 100. A method comprising: obtaining, by one or more processors from a data source, an image of a tissue sample from a patient and one or more features of the tissue sample; and generating, by the one or more processors, via one or more neural network models, a prediction of the patient's response status to a cancer immunotherapy therapeutic using the image.

[0225] Embodiment 101. A system for predicting patient response to a cancer immunotherapy therapeutic, the system comprising: a data store for storing images of tissue samples from patients, the images including one or more features of the tissue samples; and a computing device communicatively connected to the data store, the computing device comprising: a first neural network model configured to generate one or more tumor mutation burden (TMB) indicators from the one or more features; a second neural network model configured to generate one or more programmed cell death-ligand 1 (PD-L1) expression level indicators from the one or more features, or configured to generate one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features; a computing device including a third neural network model configured to calculate one or more tumor-infiltrating lymphocyte (TIL)-associated metrics from the one or more features; a fourth neural network model configured to calculate one or more cell-based features from the one or more features; and a fifth neural network model configured to predict a patient's response status to a cancer immunotherapy therapeutic according to the one or more TMB indicators, one or more PD-L1 expression level indicators or one or more PD-1 expression level indicators, respectively, the one or more TIL-associated metrics, and the one or more cell-based features; and a display system communicatively connected to the computing device and configured to display results indicative of the patient's response status.

[0226] Embodiment 102. A method comprising: obtaining, by one or more processors from a data source, images of a tissue sample from a patient and one or more features of the tissue sample; and training, by the one or more processors, one or more neural network models to predict the patient's response status to a cancer immunotherapy therapeutic using the images.

[0227] Embodiment 103. A method for predicting patient response to a cancer immunotherapy therapeutic, the method comprising: obtaining, by one or more processors, an image of a tissue sample from the patient, the image including one or more features of the tissue sample; generating, by the one or more processors, one or more tumor mutational burden (TMB) indicators from the one or more features; generating, by the one or more processors, one or more programmed death-ligand 1 (PD-L1) expression level indicators or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features; generating, by the one or more processors, one or more tumor infiltrating lymphocyte (TIL)-associated metrics from the one or more features; and training, by the one or more processors, a fourth neural network model to predict the patient's response status to the cancer immunotherapy therapeutic based on one or more of the one or more TMB indicators, one or more PD-L1 expression level indicators, one or more PD-1 level indicators, and the one or more associated metrics.

[0228] Embodiment 104. A method of treating a subject suffering from bladder cancer, the method comprising administering to the subject a cancer immunotherapy therapeutic, wherein the subject has been determined to be responsive to the cancer immunotherapy therapeutic via a trained machine learning classifier that distinguishes between responsive and non-responsive subjects that have received the cancer immunotherapy therapeutic based, at least in part, on an analysis of one or more of PD-L1 expression level, PD-1 expression level, and one or more tumor-infiltrating lymphocyte (TIL)-associated metrics in the subject.

Claims

1. A system for evaluating patient response to cancer immunotherapy drugs, wherein the system A data store for storing images of tissue samples from a patient, wherein the images include one or more features of the tissue sample. A computing device that is communicably connected to the data store, wherein the computing device is A first neural network model configured to generate one or more programmed cell death ligand 1 (PD-L1) expression level indicators from one or more features, and / or to generate one or more programmed cell death 1 (PD-1) expression level indicators from one or more features, A second neural network model configured to calculate one or more tumor-infiltrating lymphocyte (TIL) related metrics from one or more of the aforementioned features, and A computing device comprising (a) one or more PD-L1 expression level indicators and / or one or more PD-1 expression level indicators, and (b) a third neural network model configured to calculate the patient's response status to the cancer immunotherapy drug according to one or more TIL-related metrics, A display system, which is communicatively connected to the computing device and configured to display results indicating the patient's response status, A system equipped with these features.

2. A method for evaluating patient response to cancer immunotherapy drugs, wherein the method is Obtaining an image of a tissue sample from a patient using one or more processors, wherein the image includes one or more features of the tissue sample. The one or more processors generate one or more programmed cell death ligand 1 (PD-L1) expression level indicators and / or one or more programmed cell death 1 (PD-1) expression level indicators from the one or more features via a first neural network model, The one or more processors generate one or more tumor-infiltrating lymphocyte (TIL) related metrics from the one or more features via a second neural network model, The one or more processors calculate the patient's response status to the cancer immunotherapy drug via a third neural network model, using the one or more PD-L1 expression level indicators and / or the one or more PD-1 level indicators and the one or more TIL-related metrics. Methods that include...

3. The system according to claim 1, wherein the calculation of the patient's response status to the cancer immunotherapy drug is performed according to one or more medical history features of the patient.

4. The second neural network model includes a segmentation algorithm configured to generate identification of tumor regions in the tissue sample based on one or more features, and further generates one or more TIL-related metrics using the identification. The segmentation algorithm is configured to compute a TIL mask, which is used in combination with a tumor mask to compute one or more TIL-related metrics. The system according to claim 1, wherein the one or more TIL-related metrics include an estimated number of TIL clusters in the identified tumor region, an estimated size of TIL clusters in the identified tumor region, an estimated TIL cluster spread in the identified tumor region, the total number of TILs in the identified tumor region, the distance between TIL cells compared to the distance between tumor cells, the distance of TIL cells from tumor cells, the average color of TIL cells, and / or the average size of TIL cells.

5. The one or more TIL-related metrics mentioned above (i) an intratumor TIL% (iTIL%), wherein the iTIL% is calculated as the percentage of the identified tumor area in the tissue sample infiltrated by the TIL, and / or (ii) Stromal TIL% (sTIL%), wherein the sTIL% value is calculated as the percentage of stroma within the identified tumor region in the tissue sample infiltrated by TIL. The system according to claim 1, including the following:

6. The system further includes a fourth neural network model configured to generate one or more tumor mutation load (TMB) indicators from one or more of the aforementioned features, The system according to claim 1, wherein the third neural network model is further configured to calculate the patient's response status to the cancer immunotherapy drug according to the one or more TMB indicators.

7. The fourth neural network model described above is (i) A classification algorithm comprising a TMB indicator for the tissue sample in which one or more TMB indicators are either TMB positive or TMB negative, wherein if the tissue sample has a TMB value exceeding a predetermined TMB threshold, the classification algorithm generates the TMB positive classification and / or (ii) A regression algorithm comprising one or more TMB indicators, wherein the TMB indicator is a quantitative TMB score, The system according to claim 6.

8. The first neural network model described above, (i) A classification algorithm comprising one or more PD-L1 expression level indicators, wherein the PD-L1 expression level indicator is either a PD-L1 expression positive classification or a PD-L1 expression level negative classification, and the classification algorithm generates the PD-L1 expression positive classification when the tissue sample has a PD-L1 expression level value exceeding a predetermined PD-L1 expression level value. (ii) A classification algorithm comprising one or more PD-1 expression level indicators which are either PD-1 expression positive classification or PD-1 expression level negative classification, wherein if the tissue sample has a PD-L1 expression level value that exceeds a predetermined PD-L1 expression level value, the classification algorithm generates the PD-L1 expression positive classification. (iii) comprising a regression algorithm, wherein the one or more PD-L1 expression level indicators include a quantitative PD-L1 score such as a tumor percentage score (TPS) or a composite positivity score (CPS), and / or (iv) A regression algorithm comprising one or more PD-1 expression level indicators comprising a quantitative PD-1 score such as a tumor percentage score (TPS) or a composite positivity score (CPS), The system according to claim 1.

9. The system according to claim 1, wherein the cancer immunotherapy drug is a checkpoint inhibitor such as a PD-L1 inhibitor or a PD-1 inhibitor.

10. The second neural network model is further configured to generate one or more TIL masks from the one or more features, The system according to claim 1, wherein the third neural network model is further configured to calculate the patient's response status to the cancer immunotherapy drug using the one or more TIL masks.

11. (i) The tissue sample is from the patient's bladder, and the patient's response to the cancer immunotherapy drug is related to bladder cancer, (ii) The tissue sample is from a patient who is known to have PD-L1 positive cancer, (iii) The tissue sample is from a patient who is known to have PD-1 positive cancer, (iv) The tissue sample is from a patient who is known to have one or more of the following: non-small cell lung cancer, melanoma, Hodgkin lymphoma, bladder cancer, kidney cancer, and breast cancer. (v) The cancer immunotherapy drug targets PD-L1 or PD-1, and / or (vi) The cancer immunotherapy drug comprises one or more antibodies, one or more adoptive cell therapies, or a combination thereof, wherein the one or more antibodies comprises at least one of monoclonal antibodies, bispecific antibodies, or trispecific antibodies, and the one or more adoptive cell therapies comprises immune cells expressing one or more engineered antigen receptors, such as one or more engineered antigen receptors including at least one of chimeric antigen receptors, non-natural T cell receptors, or a combination thereof, and / or the one or more adoptive cell therapies comprises at least one of T cells, natural killer cells, natural killer T cells, or a combination thereof. The system according to claim 1.

12. The system according to claim 1, wherein the second neural network model includes a region / TIL segmentation algorithm and a cell segmentation algorithm.

13. (i) The cell segmentation algorithm calculates one or more cell-based features, the one or more cell-based features include morphology, color (nuclear), color (extranuclear), and texture (energy, correlation, contrast, uniformity, difference, or angular moment of inertia), (ii) The cell segmentation algorithm generates the number of TIL cells or the average number of TIL cells, (iii) The cell segmentation algorithm generates the average tumor cell size, (iv) The cell segmentation algorithm generates the average TIL cell size, (v) The cell segmentation algorithm generates the average distance between TIL cells, (vi) The cell segmentation algorithm generates the average color of the TIL cells, and / or (vii) The cell segmentation algorithm generates the average distance between TIL cells compared to tumor cells. The system according to claim 1.

14. (i) Select whether or not to include the patient in the cohort of patients participating in the clinical trial, according to the patient's response status. (ii) Displaying a graphical representation on a display screen indicating the patient's response status, and / or (iii) Diagnosing the patient as having cancer. The method according to claim 2, further comprising:

15. (i) A determination is made, based on the response status, that the patient is a responder or will become a responder, and in accordance with the determination, the patient is administered a therapeutically effective dose of the cancer immunotherapy drug, or (ii) A determination is made, based on the response status, that the patient is not a responder or is unlikely to become a responder, and in accordance with the determination, the patient is not administered the cancer immunotherapy drug. The method according to claim 2.

Images