Machine learning enabled treatment response analysis using a cell density based computational pathology signature

Abstract

An image depicting a tissue sample is received. One or more tissue compartments present in the image of the tissue sample are identified including by generating a compartment map identifying pixels in the image depicting the tissue compartments. One or more cells present in the image of the tissue sample are identified including by generating a cell map identifying pixels in the image depicting the cells. A density-based pathology signature for the image of the tissue sample is determined based on the compartment map and the cell map. The density-based pathology signature includes a density of each cell type present in the one or more tissue compartments. A response computation model is applied to determine, based on the density-based pathology signature, a predictive output indicative of a likelihood of a patient associated with the tissue sample responding to an anticancer therapy such as immunotherapy.

Description

MACHINE LEARNING ENABLED TREATMENT RESPONSE ANALYSIS USING A CELL DENSITY BASED COMPUTATIONAL PATHOLOGY SIGNATURECROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Application No. 63 / 717,152, entitled “MACHINE LEARNING ENABLED RESPONSE PREDICTION USING A COMPUTATIONAL PATHOLOGY SIGNATURE” and fded on November 6, 2024, the disclosure of which is incorporated herein by reference in its entirety.TECHNICAL FIELD

[0002] The subject matter described herein relates generally to machine learning and more specifically to machine learning based techniques for analyzing response to one or more treatments using cell density based computational pathology signatures.INTRODUCTION

[0003] Disease heterogeneity, or the variability of causes and symptoms across patients, can have a profound effect on how patients respond to treatments. Complex diseases attributable to a combination of genetic and environmental factors may be especially prone to exhibiting a high level of genetic, epigenetic, and / or phenotypic heterogeneity. In cancer, for example, disease heterogeneity may include variability in driver mutations. This means that two different patients with the same broad type of cancer can carry different sets of mutations, which can further accumulate as the cancer progresses. Consequently, a patient may exhibit markedly different responses to different treatments targeting the same broad type of cancer. Moreover, a treatment that elicits a desirable clinical endpoint in one patient may be far less effective for another patient with the same broad type of cancer.SUMMARY

[0004] Systems, methods, and articles of manufacture, including computer program products, are provided for treatment response prediction using a machine learning derived cell density based pathology signature. For example, in some cases, the cell density based pathology signature of a patient may include a density of a variety of different types of cells (or cell types) in one or more different tissue compartments of a tissue sample depicted in an image (e.g., a whole slide image (WSI) and / or the like). In instances where the tissue sample is an oncological tissue sample (e.g., a tumor sample obtained through biopsy), the cell density based pathology signature of the patient may include a density of tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells in each of a tumor stroma and a tumor epithelium. In some cases, a response computation model may be applied to determine, based at least on the cell density based pathology signature of the patient, a predictive output indicative a responder status of the patient for one or more treatments. For instance, in some cases, the predictive output may include a cellular composition metric indicative of the likelihood of the patient responding to (or benefitting from) an anticancer therapy, such as an immunotherapy (e.g., an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). Alternatively, the predictive output may include a patient subgroup classification (e.g., patients expected to respond to (or responders of) or patients not expected to respond to (or non-responders of) the anticancer therapy) determined by at least evaluating the cellular composition metric of the patient against one or more thresholds.

[0005] In one aspect, there is provided a system for treatment response prediction using a machine learning derived cell density based pathology signature. The system may include at least one data processor and at least one memory. The at least one memory may store instructions that result in operations when executed by the at least one data processor. The operations may include:receiving an image depicting a tissue sample; identifying one or more tissue compartments present in the image of the tissue sample, wherein the identifying the one or more tissue compartments includes generating a compartment map identifying one or more pixels in the image depicting the one or more tissue compartments; identifying one or more cells present in the image of the tissue sample, wherein the identifying the one or more cells includes generating a cell map identifying one or more pixels in the image depicting the one or more cells; determining, based at least on the compartment map and the cell map, a density-based pathology signature for the image of the tissue sample, wherein the density-based pathology signature includes a density of each cell type of a plurality of cell types present in the one or more tissue compartments; and applying a response computation model to determine, based at least on the density-based pathology signature, a predictive output indicative of a likelihood of a patient associated with the tissue sample responding to an anticancer therapy.

[0006] In another aspect, there is provided a computer-implemented method for treatment response prediction using a machine learning derived cell density based pathology signature. The method may include: receiving an image depicting a tissue sample; identifying one or more tissue compartments present in the image of the tissue sample, wherein the identifying the one or more tissue compartments includes generating a compartment map identifying one or more pixels in the image depicting the one or more tissue compartments; identifying one or more cells present in the image of the tissue sample, wherein the identifying the one or more cells includes generating a cell map identifying one or more pixels in the image depicting the one or more cells; determining, based at least on the compartment map and the cell map, a density-based pathology signature for the image of the tissue sample, wherein the density-based pathology signature includes a density of each cell type of a plurality of cell types present in the one or more tissue compartments; andapplying a response computation model to determine, based at least on the density -based pathology signature, a predictive output indicative of a likelihood of a patient associated with the tissue sample responding to an anticancer therapy.

[0007] In another aspect, there is provided a computer program product for treatment response prediction using a machine learning derived cell density based pathology signature. The computer program product may include a non-transitory computer readable medium storing instructions that result in operations when executed by at least one data processor. The operations may include: receiving an image depicting a tissue sample; identifying one or more tissue compartments present in the image of the tissue sample, wherein the identifying the one or more tissue compartments includes generating a compartment map identifying one or more pixels in the image depicting the one or more tissue compartments; identifying one or more cells present in the image of the tissue sample, wherein the identifying the one or more cells includes generating a cell map identifying one or more pixels in the image depicting the one or more cells; determining, based at least on the compartment map and the cell map, a density-based pathology signature for the image of the tissue sample, wherein the density-based pathology signature includes a density of each cell type of a plurality of cell types present in the one or more tissue compartments; and applying a response computation model to determine, based at least on the density-based pathology signature, a predictive output indicative of a likelihood of a patient associated with the tissue sample responding to an anticancer therapy.

[0008] In some variations, one or more features disclosed herein including the following features can optionally be included in any feasible combination.

[0009] In some variations, a tissue segmentation model is applied to identify the one or more tissue compartments present in the image of the tissue sample.

[0010] In some variations, the tissue segmentation model identifies the one or more tissue compartments by at least assigning, to each pixel in the compartment map, a label identifying a type of tissue compartment depicted by the pixel.

[0011] In some variations, a cell segmentation model is applied to identify the one or more cells present in the image of the tissue sample.

[0012] In some variations, the cell segmentation model identifies the one or more cells by at least identifying one or more pixels depicting a nucleus of each cell.

[0013] In some variations, the density of a cell type present in a tissue compartment comprises a ratio of a quantity of cells of the cell type present in the tissue compartment relative to a geometric area of the tissue compartment.

[0014] In some variations, the geometric area of the tissue compartment is determined based at least on the compartment map. The quantity of the cells of the cell type present in the tissue compartment is determined based at least on a superimposition of the compartment map and the cell map.

[0015] In some variations, the predictive output includes a cellular composition metric comprising a continuous value indicative of the likelihood of the patient responding to the anticancer therapy.

[0016] In some variations, the patient as a responder or a non-responder of the anticancer therapy based at least on whether the cellular composition metric of the patient satisfies one or more thresholds.

[0017] In some variations, the density-based pathology signature comprises a vector populated by a plurality of values corresponding to the density of each cell type of the plurality of cell types present in the one or more tissue components.

[0018] In some variations, the density-based pathology signature includes one or more of lymphocyte density in epithelium, macrophage density in stroma, fibroblast density in stroma, mitotic cell density in epithelium, tumor cell density in epithelium, lymphocyte density in stroma, macrophage density in epithelium, or endothelial cell density in stroma.

[0019] In some variations, the one or more tissue compartments include tumor stroma and tumor epithelium.

[0020] In some variations, the plurality of cell types include tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells.

[0021] In some variations, the image includes at least a portion of a stain-treated whole slide image (WSI).

[0022] In some variations, the response prediction computation model is trained based at least on one or more sample images of tissue samples.

[0023] In some variations, the training the response computation model includes generating a sample compartment map identifying one or more tissue compartments present in each sample image, generating a sample cell map identifying one or more cells present in each sample image, determining, based at least on the sample compartment map and the sample cell map, a sample density-based pathology signature for each sample image; applying the response computation model to determine, based at least on the sample density-based pathology signature included with each training sample, a sample predictive output, and adjusting one or more parameters of the response computation model to reduce a discrepancy between the sample predictive output generated by the response computation model for each training sample and a corresponding ground-truth label included with the training sample.

[0024] In some variations, the training the response computation model includes applying a tissue segmentation model to generate a sample compartment map identifying one or more tissue compartments present in each sample image, and applying a cell segmentation model generating a sample cell map identifying one or more cells present in each sample image.

[0025] In some variations, the sample predictive output of the response computation model includes a sample cellular composition metric.

[0026] In some variations, one or more thresholds for differentiating, based at least on the sample cellular composition metric, between two or more patient subgroups are determined.

[0027] In some variations, the two or more patient subgroups include a high-metric patient subgroup for patients expected to benefit from the anticancer therapy and a low-metric patient subgroup for patients not expected to benefit from the anticancer therapy.

[0028] In some variations, the one or more thresholds are determined to increase a hazard ratio (HR) between the high-metric patient subgroup for responders of the anticancer therapy and a high-metric patient subgroup for responders of an alternate therapy.

[0029] In some variations, the one or more sample images are associated with patients treated with the anticancer therapy, and the one or more thresholds are determined based at least on response data for patients treated with an alternate anti cancer therapy.

[0030] In some variations, the anti cancer therapy includes an immunotherapy and the alternate anti cancer therapy includes chemotherapy.

[0031] In some variations, the anticancer therapy includes at least one of an immunotherapy, surgery, radiation therapy, chemotherapy, hormone therapy, and targeted therapy.

[0032] In some variations, the anticancer therapy includes atezolizumab.

[0033] Implementations of the current subject matter can include, but are not limited to, methods consistent with the descriptions provided herein as well as articles that comprise a tangibly embodied machine-readable medium operable to cause one or more machines (e.g., computers, etc.) to result in operations implementing one or more of the described features. Similarly, computer systems are also described that may include one or more processors and one or more memories coupled to the one or more processors. A memory, which can include a non- transitory computer-readable or machine-readable storage medium, may include, encode, store, or the like one or more programs that cause one or more processors to perform one or more of the operations described herein. Computer implemented methods consistent with one or more implementations of the current subject matter can be implemented by one or more data processors residing in a single computing system or multiple computing systems. Such multiple computing systems can be connected and can exchange data and / or commands or other instructions or the like via one or more connections, including, for example, to a connection over a network (e.g. the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, or the like), via a direct connection between one or more of the multiple computing systems, etc.

[0034] The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, and from the claims. While certain features of the currently disclosed subject matter are described for illustrative purposes in relation to the use of immunotherapies, such as atezolizumab and other anti-PD-(L)l therapies, as oncological treatments (e.g., advanced non-small cell lung cancer (NSCLC)), it should be readily understood that such features are not intended to be limiting. The claims that follow this disclosure are intended to define the scope of the protected subject matter.DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, show certain aspects of the subject matter disclosed herein and, together with the description, help explain some of the principles associated with the disclosed implementations. In the drawings,

[0036] FIG. 1 depicts a system diagram illustrating an example of a digital pathology system, in accordance with some example embodiments;

[0037] FIG. 2A depicts a flowchart illustrating an example of a process for machine learning enabled treatment response analysis using a cell density based pathology signature, in accordance with some example embodiments;

[0038] FIG. 2B depicts a flowchart illustrating an example of a process for training a response computation model to perform treatment response analysis using cell density based pathology signatures, in accordance with some example embodiments;

[0039] FIG. 3 depicts a whole slide image (WSI) depicting a tumor sample that has undergone an example of lesion segmentation, tissue compartment segmentation, and cell labeling, in accordance with some example embodiments;

[0040] FIG. 4A depicts a graph illustrating the Kaplan-Meier curve for the overall survival (OS) probability predictions of patients treated with atezolizumab and patients treated with docetaxel, in accordance with some example embodiments;

[0041] FIG. 4B depicts a graph illustrating the Kaplan-Meier curve for the overall survival (OS) probability predictions of patients with a high cellular composition metric treated with atezolizumab, patients with a high cellular composition metric treated with docetaxel, patients witha low cellular composition metric treated with atezolizumab, patients with a low cellular composition metric treated with docetaxel, in accordance with some example embodiments;

[0042] FIG. 5 depicts a bar graph illustrating the relative importance of different cell types in the derivation of cellular composition metric by an example of a response computation model, in accordance with some example embodiments;

[0043] FIG. 6 depicts a heatmap illustrating the correlation between cell densities and bulk ribonucleic acid (RNA) sequencing (RNAseq) signatures, in accordance with some example embodiments;

[0044] FIG. 7(a) depicts a scatterplot illustrating the relative distributions of the epithelial densities of lymphocytes and mitotic cells in patients exhibiting a high cellular composition metric and patients exhibiting a low cellular composition metric, in accordance with some example embodiments;

[0045] FIG. 7(b) depicts a scatterplot illustrating the relative distributions of the stromal densities of macrophages and fibroblasts in patients exhibiting a high cellular composition metric and patients exhibiting a low cellular composition metric, in accordance with some example embodiments; and

[0046] FIG. 8 depicts a block diagram illustrating an example of a computing system, in accordance with some example embodiments.

[0047] When practical, similar reference numbers denote similar structures, features, or elements.DETAILED DESCRIPTION

[0048] Complex diseases with a high level of genetic, epigenetic, and phenotypic heterogeneity may be difficult to treat effectively when the same treatment elicits differentresponses from patients with the same broad disease type. For example, immunotherapies, such as anti-PD-(L)l therapies, show significant promise in treating cancers. However, not every patient with the same type of cancer, such as advanced non-small cell lung cancer (NSCLC), benefit from immunotherapies. Instead, for some patients, immunotherapies do not provide an adequate improvement to clinical outcome, such as longer overall survival (OS), to warrant the potential side effects. Such suboptimal disease responses motivate the development of new biomarkers to identify patients more likely to benefit from immunotherapies. For instance, transcriptome biomarkers for immunotherapies are molecular profiles of gene expression predicative of patient response to immunotherapy. Nevertheless, state-of-the-art transcriptomebased approaches, which relies on molecular genetic tests such as fluorescence in situ hybridization (FISH) testing, are cost prohibitive and require patient transcriptome data. Accordingly, in some example embodiments, a patient’s responder status, including a probability of the patient responding to an immunotherapy, may be determined based on a pathological analysis of one or more microscopy images (e.g., whole slide images (WSIs)) depicting the patient’s tissue samples. In some cases, pathological analysis may be preferable to transcriptome based diagnostics (e.g., RNA sequence based molecular subtyping), which may not always be suitable due to prohibitive cost and the uncertain availability of patient transcriptome data.

[0049] In some example embodiments, an image of a tissue sample may undergo pathological analysis to determine the presence (or absence) of one or more features indicative of the responder status of a patient associated with the tissue sample. For example, in some cases, the image of the tissue sample may undergo pathological analysis to determine the probability of a cancer patient (e.g., a non-small cell lung cancer (NSCLC) patient) responding to (or benefitting from) an anticancer therapy. Immunotherapies, such as anti-PD-Ll or PD-L1 inhibiters (e.g.,atezolizumab and / or the like), are one example of anticancer therapies. Other options for anticancer therapies include surgery, radiation therapy, chemotherapy, hormone therapy, and targeted therapy. It should be appreciated that the choice of anticancer therapies may impose tradeoffs in clinical benefits (e.g., increased survival, improved quality of life, and / or the like) and cost (e g., side effects, drug prices, and / or the like). In some cases, the clinical benefits of different anticancer therapies may be determined by a pathological analysis of patient tissue samples (e.g., tumor samples).

[0050] Despite significant advancements in both equipment and technique, conventional pathological analysis suffers from several shortcomings. For example, pathologist interpretations of the features present in the image of the tissue sample may be prone to intra-and inter-reader variability, particularly where the features exhibit visual ambiguity, thus leading to inconsistent and unreliable patient diagnoses. Efforts to digitize pathological analysis are thwarted by the obscurity of features having sufficiently meaningful nexus to the responder status of the patient. Even state-of-the-art computer vision models, when deployed to detect irrelevant features, will still yield inaccurate responder status. Accordingly, in some example embodiments, a cell-density based pathology signature of the patient may be determined based at least on a density of one or more types of cells (or cell types) in one or more tissue compartments of the tissue sample depicted in the image. As described in more detail below, the cell-density based pathology signature may be determined by applying one or more segmentation models to identify the one or more tissue compartments as well as cells of the one or more cell types present in the image. Moreover, in some cases, a response computation model may be trained to determine, based at least on the cell density based pathology signature of the patient, a responder status of the patient.

[0051] In some example embodiments, the one or more segmentation models may include a tissue compartment segmentation model trained to segment the image of the tissue sample by at least identifying pixels in the image depicting different tissue compartments, such as tumor stroma, tumor epithelium, and / or the like. For example, in some cases, the tissue compartment segmentation model may segment the image of the tissue sample by at least assigning, to each pixel in the image of the tissue sample, a label (e.g., a numerical value) identifying the pixel as depicting background or, alternatively, the type of tissue compartment depicted by the pixel. In some cases, the tissue compartment segmentation model may generate a compartment map in which each pixel is assigned a label (e.g., a numerical value) identifying the type of tissue compartment depicted by the corresponding pixel in the image of the tissue sample.

[0052] In some example embodiments, the one or more segmentation models may further include a cell segmentation model trained to segment the image of the tissue by at least identifying pixels in the image depicting different types of cells (or cell types), such as tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, endothelial cells, and / or the like. For example, in some cases, the cell segmentation model may segment the image of the tissue sample by at least assigning, to each pixel in the image of the tissue sample, a label (e.g., a numerical value) identifying the pixel as depicting background or, alternatively, the type of the cell depicted by the pixel. In some cases, instead ofidentifying the pixels depicting an entire cell, the cell segmentation model may identify one or more pixels depicting the nucleus of the cell, such as the pixel at the center of the nucleus of the cell. In some cases, the cell segmentation model may generate a cell map in which each pixel is assigned a label (e.g., a numerical value) identifying the type of cell depicted by the corresponding pixel in the image of the tissue sample.

[0053] In some example embodiments, the cell-density based pathology signature of the patient may be determined based at least on the compartment map generated by the tissue compartment segmentation model and the cell map generated by the cell segmentation model. For example, in some cases, the cell-density based computation pathology signature of the patient may be determined by at least determining, based at least on the compartment map, a geometric area of each tissue compartment present in the image of the tissue sample. Moreover, in some cases, the compartment map and the cell map may be superimposed (overlayed) in order to determine the quantity of each cell type present in each tissue compartment present in the image of the tissue sample. In some cases, the density of a cell type present in a tissue compartment may correspond to a ratio of the quantity of cells of the cell type present in that tissue compartment and a geometric area of the tissue compartment. In some cases, the cell-density based pathology signature may include the density of each cell type across the different tissue compartments present in the image of the tissue sample. For instance, where the tissue sample is an oncological tissue sample (e.g., tumor sample), the cell density based pathology signature of the patient may include a vector (or another data structure) populated by values corresponding to the densities of tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells in each of a tumor stroma and a tumor epithelium.

[0054] In some example embodiments, the response computation model may determine, based at least on the cell density based pathology signature of the patient, a predictive output indicative of a responder status of the patient for one or more treatments. For example, in some cases, the predictive output may include a cellular composition metric determined based at least on the cell density based pathology signature of the patient. In some cases, the cellular composition metric may be a continuous value indicative of the likelihood of the patient responding to (orbenefitting from) one or more immunotherapies (e.g., an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). In some cases, the cellular composition metric of the patient may correspond to an overall survival (OS) of the patient. Alternatively, in some cases, the cellular composition metric of the patient may undergo further evaluation against one or more thresholds. In some cases, the one or more thresholds may differentiate between different patient subgroups, such as responders and non-responders of an anticancer therapy (e.g., an immunotherapy an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). Accordingly, in some cases, the predictive output of the response computation model may include one or more patient subgroups associated with the patient. In some cases, the one or more thresholds may be determined to increase (or maximize) the differentiation between different patient subgroups such as, for example, a high-metric patient subgroup for responders of one anticancer therapy (e.g., an immunotherapy such as atezolizumab) and a high-metric patient subgroup for responders of an alternate anticancer therapy (e.g., a chemotherapy such as docetaxel).

[0055] FIG. 1 depicts a system diagram illustrating an example of a digital pathology system 100, in accordance with some example embodiments. Referring to FIG. 1, the digital pathology system 100 may include a digital pathology platform 110, an imaging system 120, and a client device 130. In the example of the digital pathology system 100 shown in FIG. 1, the digital pathology platform 110, the imaging system 120, and the client device 130 may be communicatively coupled via a network 140. In some cases, the imaging system 120 may include one or more imaging devices including, for example, a microscope 122 (e.g., an optical microscope, a bright-field microscope, a compound microscope, an inverted microscope, a fluorescence microscope, and / or the like), a camera 124 (e.g., a digital camera), and a scanner 126(e.g., a whole slide scanner). The client device 130 may be a processor-based device including,for example, a workstation, a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable apparatus, and / or the like. The network 140 may be a wired network and / or a wireless network including, for example, a local area network (LAN), a virtual local area network (VLAN), a wide area network (WAN), a public land mobile network (PLMN), the Internet, and / or the like.

[0056] Referring again to FIG. 1, in some example embodiments, the digital pathology platform 110 may include a segmentation engine 120, a cellular composition engine 125, and a response computation model 127. In some cases, the digital pathology platform 110 may receive, from the imaging system 120, an image 101. In some cases, the image 101 may depict a tissue sample, such as an oncological tissue sample. In some cases, the image 101 may include at least a portion of a whole slide image (WSI) generated, for example, by digitally scanning a tissue slide containing the tissue sample. In some cases, the image 101 may be treated with a stain to enhance the visualization of certain features of interest present in the tissue sample depicted in the image 101. For example, in some cases, the image 101 of the tissue sample may be treated with a stain (e.g., an immunohistochemistry (IHC) stain) sensitive to specific proteins (e.g., a tumor associated antigen (TAA) targeted by the immunotherapy) such that the stain- treated image may be analyzed to detect the presence (or absence) of cells expressing these proteins within the tissue sample. Alternatively and / or additionally, the image 101 of the tissue sample may be treated with a stain (e.g., hematoxylin and eosin (H&E) stain) to enable visualization of one or more cells (or subcellular compartments) present in the tissue sample depicted in the image 101 including, for example, cell nuclei (stained blue by the hematoxylin dye) as well as cytoplasm and extracellular matrix (stained various shades of pink by the eosin dye).

[0057] In some example embodiments, the segmentation engine 120 may apply, to the image 101, a tissue segmentation model 121 to generate a compartment map 123 localizing one or more tissue compartments (e.g., tumor stroma, tumor epithelium, and / or the like) present in the image 101. In some cases, the segmentation engine 120 may also apply, to the image 101, a cell segmentation model 122 to generate a cell map 124 localizing cells (or cell nuclei) of various cell types (e.g., tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, endothelial cells, and / or the like) present in the image 101. In some cases, the cellular composition engine 125 may determine, based at least on the compartment map 123 and the cell map 124, a densitybased pathology signature 126 for a patient associated with the tissue sample depicted in the image 101. Furthermore, in some cases, the response computation model 127 may determine, based at least on the density -based pathology signature 126, a predictive output 128 indicative of a likelihood of the patient responding to (or benefitting from) an anticancer therapy (e g., an immunotherapy such as an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab).

[0058] FIG. 2A depicts a flowchart illustrating an example of a process 200 for machine learning enabled treatment response analysis using a cell density based pathology signature, in accordance with some example embodiments. Referring to FIGS. 1 and 2A, the process 200 may be performed by the pathology platform 110 shown in FIG. 1. For example, in some cases, the pathology platform 110 may perform the process 200 to determine, based at least on the tissue sample depicted in the image 101, the predictive output 128 indicative of a likelihood of the patient associated with the tissue sample responding to (or benefitting from) an anticancer therapy (e.g., an immunotherapy such as an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). As described in more detail below, in some cases, the predictive output 128 may be determined based on the density -based pathology signature 126, which may include the density of one or more types ofcells (or cell types) across different tissue compartments in the tissue sample depicted in the image101.

[0059] At 202, an image depicting a tissue sample is received. In some example embodiments, a digital pathology platform may receive the image of the tissue sample from an imaging system that includes, for example, one or more microscopes, cameras, or scanners. In some cases, the image of the tissue sample may be treated with a stain (e.g., hematoxylin and eosin (H&E) stain) to enable visualization of one or more cells (or subcellular compartments) present in the tissue sample including, for example, cell nuclei (stained blue by the hematoxylin dye) as well as cytoplasm and extracellular matrix (stained various shades of pink by the eosin dye). In some cases, the image of the tissue sample may include at least a portion of a whole slide image (WSI). In some cases, the image of the tissue sample may be generated by the imaging system, for example, digitally scanning a tissue slide containing the tissue sample. In some cases, the tissue sample depicted in the image may be an oncological tissue sample (e.g., a tumor sample obtained through biopsy).

[0060] At 204, one or more tissue compartments present in the image of the tissue sample are identified. In some example embodiments, a tissue segmentation model is applied to generate a compartment map localizing one or more tissue compartments present in the image of the tissue sample. In some cases, the tissue segmentation model may localize the one or more tissue compartments by at least identifying, in the image of the tissue sample, one or more pixels depicting each type of tissue compartment. For example, where the tissue sample is an oncological tissue sample, the tissue segmentation model may identify pixels depicting tumor stroma and pixels depicting tumor epithelium. In some cases, the tissue segmentation model may localize each type of tissue compartment by at least assigning, to each pixel in the image ofthe tissue sample, a label (e.g., a numerical value) identifying the pixel as depicting background or, alternatively, the type of tissue compartment depicted by the pixel. In some cases, each pixel in the resulting compartment map be associated with a label (e.g., a numerical value) identifying the type of tissue compartment depicted by the corresponding pixel in the image of the tissue sample.

[0061] At 206, one or more cells present in the image of the tissue sample are identified. In some example embodiments, a cell segmentation model is applied to generate a cell map localizing one or more cells present in the image of the tissue sample. In some cases, the cell segmentation model may localize the one or more cells by at least identifying, in the image of the tissue sample, one or more pixels depicting each cell. For example, in some cases, the cell segmentation model may identify one or more pixels depicting the nucleus of each cell, such as the pixel at the center of the nucleus of the cell. In some cases, in addition to identifying the one or more pixels depicting each cell, the cell segmentation model may further identify the type of the cell. For instance, where the tissue sample is an oncological tissue sample, the cell segmentation model may identify pixels depicting tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, endothelial cells, and / or the like. In some cases, the cell segmentation model may localize each cell by at least assigning, to each pixel in the image of the tissue sample, a label (e.g., a numerical value) identifying the pixel as depicting background or, alternatively, the type of cell (or cell type) depicted by the pixel. In some cases, each pixel in the resulting cell map be associated with a label (e.g., a numerical value) identifying the type of cell depicted by the corresponding pixel in the image of the tissue sample.

[0062] At 208, a density-based pathology signature is determined for the image of the tissue sample. In some example embodiments, the density-based pathology signature isdetermined based at least on the compartment map and the cell map. In some cases, the densitybased pathology signature may be determined by at least determining, based at least on the compartment map and the cell map, a quantity of each type of cell (or cell type) in the different tissue compartments present in the image of the tissue sample. For example, where the tissue sample is an oncological tissue sample, the density-based pathology signature may include the quantity of tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells in each of a tumor stroma and a tumor epithelium. In some cases, the quantity of each cell type present in a tissue compartment may be determined by at least superimposing the cell map and the compartment map and determining a count of each cell type present in the tissue compartment. In some example embodiments, the density-based pathology signature may include a vector (or another data structure) populated by values corresponding to the densities of each cell type (e.g., tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, endothelial cells, and / or the like) in different tissue compartments (e.g., tumor stroma, tumor epithelium, and / or the like).

[0063] At 210, a response computation model is applied to determine, based at least on the density-based pathology signature, a predictive output indicative of a likelihood of a patient associated with the tissue sample responding to an anticancer therapy. In some example embodiments, the predictive output may include a cellular composition metric determined based at least on the density-based pathology signature, which includes the densities of different types of cells (or cell types) in one or more tissue compartments. For example, where the tissue sample is an oncological tissue sample, the cellular composition metric may be determined based at least on the densities of tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells in tumor stroma and tumor epithelium. In some cases, the cellular compositionmetric may be a continuous value indicative of the likelihood of the patient responding to (or benefitting from) an anticancer therapy, such as one or more immunotherapies (e.g., an anti-PD- L1 or PD-L1 inhibitor such as atezolizumab). Other examples of anticancer therapies include surgery, radiation therapy, chemotherapy, hormone therapy, targeted therapy, and / or the like. In this context, the term “response” or “responding to” may refer to a range of responses including positive responses as well as negative responses. For instance, where the tissue sample is an oncological tissue sample, patient response may include a positive response in the form of a reduction in tumor size or a negative response in the form of an increase in tumor size or the metastasis of tumor cells.

[0064] In some cases, the cellular composition metric of the patient may correspond to an overall survival (OS) of the patient. Alternatively, in some cases, the cellular composition metric of the patient may undergo further evaluation against one or more thresholds. In some cases, the one or more thresholds may differentiate between different patient subgroups, such as responders and non-responders of an anticancer therapy (e.g., an immunotherapy such as an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). Accordingly, in some cases, the predictive output of the response computation model may include one or more patient subgroups associated with the patient. As described in more detail below, in some cases, the response computation model may be trained on sample images of patients treated with one anticancer therapy (e.g., an immunotherapy such as atezolizumab) while the determination of the one or more thresholds may leverage data from an alternate anticancer therapy (e.g., a chemotherapy such as docetaxel).

[0065] In some cases, the one or more thresholds may be determined to increase (or maximize) the differentiation between different patient subgroups such as, for example, a high- metric patient subgroup for responders of one anticancer therapy (e.g., an immunotherapy suchas atezolizumab) and a high-metric patient subgroup for responders of an alternate anti cancer therapy (e.g., a chemotherapy such as docetaxel). For example, in some cases, the one or more thresholds may be determined to increase (or maximize) the hazard ratio between different patient subgroups such as, for example, the high-metric patient subgroup for responders of one anti cancer therapy (e g., an immunotherapy such as atezolizumab) and the high-metric patient subgroup for responders of an alternate anticancer therapy (e.g., a chemotherapy such as docetaxel). It should be appreciated that hazard ratio (HR) measures how often a particular event occurs in one group (e.g., the high-metric patient subgroup for responders of one anticancer therapy) compared to another group (e g., the high-metric patient subgroup for responders of an alternate anticancer therapy) over time. In some cases, hazard ratio may be the result of survival analysis and may be used to compare the hazard rates between two groups. In some cases, a hazard ratio of 1 means the event rates are the same, a hazard ratio greater than 1 indicates a higher risk in the first group, and a hazard ratio than 1 indicates a lower risk.

[0066] FIG. 2B depicts a flowchart illustrating an example of a process 250 for training a response computation model to perform treatment response analysis using cell density based pathology signatures, in accordance with some example embodiments. Referring to FIGS. 1 and 2B, the process 250 may be performed by the pathology platform 110 shown in FIG. 1. For example, in some cases, the pathology platform 1 10 may perform the process 250 to train the response computation model 127 to determine, based at least on the density-based pathology signature 126 associated with the image 101, the predictive output 128 indicative of a likelihood of the patient associated with the tissue sample responding to (or benefitting from) an immunotherapy (e.g., an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). In some cases, the density-based pathology signature 126 may include the density of one or more types of cells(or cell types) across different tissue compartments in the tissue sample depicted in the image 101. As described in more detail below, the response prediction model 127 may be trained to determine the predictive output 128 based at least on the density of one or more types of cells (or cell types) across different tissue compartments in the tissue sample depicted in the image 101.

[0067] At 252, one or more sample images of tissue samples are received. In some example embodiments, a digital pathology platform may receive the one or more sample image from an imaging system that includes, for example, one or more microscopes, cameras, or scanners. In some cases, each sample image of a tissue sample may be treated with a stain (e g., hematoxylin and eosin (H&E) stain) to enable visualization of one or more cells (or subcellular compartments) present in the tissue sample including, for example, cell nuclei (stained blue by the hematoxylin dye) as well as cytoplasm and extracellular matrix (stained various shades of pink by the eosin dye). In some cases, each sample image may include at least a portion of a whole slide image (WSI). In some cases, each sample image may be generated by the imaging system, for example, digitally scanning a tissue slide containing the tissue sample. In some cases, the tissue sample depicted in each sample image may be an oncological tissue sample (e.g., a tumor sample obtained through biopsy).

[0068] At 254, one or more tissue compartments present in each sample image are identified. In some example embodiments, a tissue segmentation model is applied to generate, for each sample image of the one or more sample images, a sample compartment map localizing one or more tissue compartments present in the sample image. In some cases, the tissue segmentation model may localize the one or more tissue compartments by at least identifying, in each sample image, one or more pixels depicting each type of tissue compartment. For example, where the tissue sample is an oncological tissue sample, the tissue segmentation model mayidentify pixels depicting tumor stroma and pixels depicting tumor epithelium. In some cases, the tissue segmentation model may localize each type of tissue compartment by at least assigning, to each pixel in a sample image, a label (e.g., a numerical value) identifying the pixel as depicting background or, alternatively, the type of tissue compartment depicted by the pixel. In some cases, each pixel in the resulting sample compartment map of the sample image may be associated with a label (e.g., a numerical value) identifying the type of tissue compartment depicted by the corresponding pixel in the sample image.

[0069] At 256, one or more cells present in each sample image are identified. In some example embodiments, a cell segmentation model is applied to generate, for each sample image of the one or more sample images, a cell map localizing one or more cells present in the sample image. In some cases, the cell segmentation model may localize the one or more cells by at least identifying, in each sample image, one or more pixels depicting each cell. For example, in some cases, the cell segmentation model may identify one or more pixels depicting the nucleus of each cell, such as the pixel at the center of the nucleus of the cell. In some cases, in addition to identifying the one or more pixels depicting each cell, the cell segmentation model may further identify the type of the cell. For instance, where the tissue sample is an oncological tissue sample, the cell segmentation model may identify pixels depicting tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, endothelial cells, and / or the like. In some cases, the cell segmentation model may localize each cell by at least assigning, to each pixel in a sample image, a label (e.g., a numerical value) identifying the pixel as depicting background or, alternatively, the type of cell (or cell type) depicted by the pixel. In some cases, each pixel in the resulting sample cell map of the sample image may be associated with a label (e.g., a numerical value) identifying the type of cell depicted by the corresponding pixel in the sample image.

[0070] At 258, a sample density-based pathology signature is determined for each sample image. In some example embodiments, the sample density-based pathology signature of each sample image is determined based at least on the sample compartment map and the sample cell map of the sample image. In some cases, the sample density-based pathology signature of each sample image may be determined by at least determining, based at least on the sample compartment map and the sample cell map of the sample image, a quantity of each type of cell (or cell type) in the different tissue compartments present in the sample image. For example, where the tissue sample is an oncological tissue sample, the sample density-based pathology signature may include the quantity of tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells in each of a tumor stroma and a tumor epithelium. In some cases, the quantity of each cell type present in a tissue compartment may be determined by at least superimposing the sample cell map and the sample compartment map and determining a count of each cell type present in the tissue compartment. In some example embodiments, the sample density -based pathology signature may include a vector (or another data structure) populated by values corresponding to the densities of each cell type (e.g., tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, endothelial cells, and / or the like) in different tissue compartments (e.g., tumor stroma, tumor epithelium, and / or the like).

[0071] At 260, one or more training samples are generated in which each training sample includes the sample density-based pathology signature of a sample image and a ground-truth label. In some example embodiments, the ground-truth label associated with the sample densitybased pathology signature of a sample image may correspond to a likelihood of a patient whose tissue sample exhibits the density-based pathology signature responding to (or benefitting from) an immunotherapy (e.g., an anti-PD-Ll or PD-Ll inhibitor such as atezolizumab). Alternativelyand / or additionally, the ground-truth label associated with the sample density-based pathology signature of a sample image may include a patient subgroup classification (e.g., a responder or non-responder of the one or more immunotherapies) of a patient whose tissue sample exhibits the density-based pathology signature. In some cases, the one or more training samples may include a threshold quantity of training samples. For example, in some cases, the one or more training samples may include a threshold quantity (e.g., two or more) of training samples from patients treated with an immunotherapy (e.g., an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). Furthermore, in some cases, the one or more training samples may also include a threshold quantity (e.g., two or more) of training samples from patients treated with an alternate therapy (e.g., a chemotherapy such as docetaxel).

[0072] At 262, a response computation model is applied to determine, based at least on the sample density-based pathology signature included with each training sample, a sample predictive output. In some example embodiments, the sample predictive output may include a cellular composition metric determined based at least on the sample density -based pathology signature, which includes the densities of different types of cells (or cell types) in one or more tissue compartments. For example, where the tissue sample is an oncological tissue sample, the cellular composition metric may be determined based at least on the densities of tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells in tumor stroma and tumor epithelium. In some cases, the cellular composition metric may be a continuous value indicative of the likelihood of a patient whose tissue sample exhibits the corresponding densitybased pathology signature responding to (or benefitting from) one or more immunotherapies (e g., an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). In some cases, the cellular composition metric of the patient may correspond to an overall survival (OS) of the patient.Alternatively, in some cases, the cellular composition metric of the patient may undergo further evaluation against one or more thresholds. In some cases, the one or more thresholds may differentiate between different patient subgroups, such as responders and non-responders of an immunotherapy (e.g., an anti-PD-Ll or PD-L1 inhibitor such as atezolizumab). Accordingly, in some cases, the sample predictive output of the response computation model may include one or more patient subgroups associated with the patient.

[0073] At 264, one or more parameters of the response computation model are adjusted to reduce a discrepancy between the predictive output generated by the response computation model for each training sample and a corresponding ground-truth label included with the training sample. In some example embodiments, the training of the response computation model may include adjusting the one or more parameters (e.g., weights, biases, and / or the like) of the response computation model to reduce a loss function quantifying a difference between the ground-truth label of the training sample and the predictive output, such as a sample cellular composition metric, generated by the response computation model for the sample image included with the training sample.

[0074] In some cases, the response computation model may be trained on sample images from patients treated with one anticancer therapy (e.g., an immunotherapy such as atezolizumab) to determine a likelihood of a patient responding to (or benefitting from) the anti cancer therapy. For example, in some cases, the response computation model may be trained to determine, based at least on a density -based pathology signature of an image of a tissue sample associated with the patient, a cellular composition metric (or risk metric) indicative of the likelihood of the patient responding to (or benefitting from) the anti cancer therapy (e.g., an immunotherapy such as atezolizumab). In some cases, the response computation model may further determine, based atleast one or more thresholds, a classification of the patient as belonging to one or more patient subgroups (e.g., responders or non-responders of the anticancer therapy (e.g., an immunotherapy such as atezolizumab). For instance, in some cases, the response computation model may classify the patient as a high-metric patient subgroup for responders of the anti cancer therapy if the cellular composition metric (or risk metric) of the patient satisfies the one or more thresholds. Alternatively, in some cases, the response computation model may classify the patient as a low- metric patient subgroup for non-responders of the anticancer therapy if the cellular composition metric (or risk metric) of the patient fails to satisfy the one or more thresholds.

[0075] In some cases, the one or more thresholds may be determined to increase (or maximize) the differentiation between different patient subgroups. For example, in some cases, the determination of the one or more thresholds may leverage response data for an alternate therapy (e g., a chemotherapy such as docetaxel). Furthermore, in some cases, the one or more thresholds may be determined to increase (or maximize) the differentiation (e.g., hazard ratio (HR)) between a high-metric patient subgroup for responders of one therapy (e.g., an immunotherapy such as atezolizumab) and a high-metric patient subgroup for responders of the alternate therapy (e.g., a chemotherapy such as docetaxel).

[0076] FIG. 3 depicts examples of images of tissue samples undergoing lesion segmentation, tissue compartment segmentation, and cell segmentation, in accordance with some example embodiments. FIG. 3(A) depicts an example of a whole slide image (WSI) (shown at a low power or a large field of view) depicting a tissue sample. The whole slide image (WSI) in FIG. 3(A) undergoes lesion segmentation to identify pixels depicting tumor lesion, thus generating the image shown in FIG. 3(B) in which the portion of the tissue sample corresponding to tumor lesion is annotated in a different color than the non-tumor lesion portion of the tissuesample to enable differentiation therebetween. FIG. 3(C) depicts a portion of an example of a whole slide image (WSI) (shown at a higher power or a small field of view) that undergoes compartment segmentation to identify pixels depicting tumor epithelium and pixels depicting tumor stroma. FIG. 3(D) depicts the results of compartment segmentation in which regions corresponding to tumor epithelium and regions corresponding tumor stroma are annotated in different colors to enable differentiation therebetween. FIG. 3(E) depicts a portion of another example of a whole slide image (WSI) (shown at a resolution of 100 microns) that undergoes cell segmentation to localize and label the different types of cells present therein. FIG. 3(F) depicts the results of the localizing and labeling in which regions corresponding to tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, endothelial cells, and other cells are annotated in different colors to enable differentiation therebetween.

[0077] Experimental Examples T

[0078] A tissue segmentation model and a cell segmentation model were applied to 1,243 digitized hematoxylin and eosin (H&E) slide images from two trials of atezolizumab versus docetaxel in second-line advanced non-small cell lung cancer (NSCLC). The tissue segmentation model and the cell segmentation model were implemented using Lunit Scope IO, a deep learning model trained for tumor microenvironment (TME) analysis. POPLAR, a phase 11 trial with digitized hematoxylin and eosin (H&E) slides from 162 patients, was used as the training set while OAK, the phase III follow-on with 1081 evaluable patients, was used as the validation set. The tissue segmentation model was applied to each slide image to identify and label tissue compartments. The cell segmentation model was applied to each slide image to identify and label cells. From the output of the tissue segmentation model and cell segmentation model, the density of tumor cells, lymphocytes, fibroblasts, macrophages, endothelial cells, andmitotic figures was calculated separately for tumor epithelium and intra-tumoral stroma. A response computation mode, implemented using an elastic-net regularized Cox regression model in this example, was applied to these density values to be fitted on the atezolizumab-treated patients of the training set to predict overall survival (OS). The response computation model was applied to all patients to produce a cellular composition metric (or risk metric) for each patient. The patients were divided into model-low and model-high groups based on a threshold that was identified, based on the training set, to maximize between-arm median overall survival (OS) difference in model-high patients. The performance of the response computation model in identifying patients likely to benefit from atezolizumab was evaluated by an overall survival (OS) concordance index (c-index) in each treatment arm, by hazard ratio (HR) [95% confidence interval] between arms in the model-low and -high groups, and by the interaction term in a Cox regression model containing treatment arm, model low / high categorization, and their interaction in the validation set.

[0079] The response computation model was trained to operate on eight features: lymphocyte density in epithelium, macrophage density in stroma, fibroblast density in stroma, mitotic cell density in epithelium, tumor cell density in epithelium, lymphocyte density in stroma, macrophage density in epithelium, and endothelial cell density in stroma. Of those features, in tumor epithelium, lymphocytes and tumor cell densities were positively correlated with overall survival (OS) in atezolizumab-treated patients, while mitotic cell and macrophage densities were negatively correlated. In the intra-tumoral stroma, the densities of fibroblasts, macrophages, lymphocytes, and endothelial cells were positively correlated with overall survival (OS). The cellular composition metric (or risk metric) output by the model was more strongly associated with overall survival (OS) in the atezolizumab arm (c-index=0.58, 95% confidenceinterval: [0.55, 0.61]) than in the docetaxel arm (c-indcx~0.54 [0.51, 0.57]). Among the n=464 model-high validation set patients, atezolizumab-treated patients had significantly longer overall survival (OS) (HR=0.65, [0.53, 0.80]) with no significant difference between arms in model-low patients (n=617, HR=0.92 [0.78,1.10]). Model low / high category had a significant interaction (p=0.02) with treatment in survival prediction.

[0080] In the validation set, the n = 464 model-high patients achieved longer overall survival (OS) with atezolizumab treatment (HR.~0.65 [0.53, 0.80]) while no effect was seen in the n — 617 model-low patients (HR=0.92 [0.78,1.10]). Among atezolizumab-treated patients, the model risk score was associated with overall survival (OS) (c-index=0.58 [0.55, 0.61]) more strongly than in docetaxel-treated patients (c-index=0.54 [0.51, 0.57]). Model low / high category had a significant interaction (p=0.02) with treatment arm in survival prediction. FIG. 4A depicts a comparison of the overall survival (OS) probabilities of patients treated with atezolizumab and those treated with docetaxel. FIG. 4B depicts a comparison of the overall survival (OS) probabilities of the atezolizumab-treated patients and the docetaxel-treated patients, further differentiated by the cellular composition metric (or risk metric) output by the response computation model. As shown in FIGS. 4A-B, the response computation model successfully differentiated between patients who are atezolizumab responders and patients who are atezolizumab non-responders. Furthermore, the cellular composition metric (or risk metric) output by the response computation model enables the identifications of patients who do not respond to atezolizumab but who can benefit from docetaxel. Table 1 below summarizes the C- index (95% confidence interval) of the cellular composition metric (or risk metric) in the validation set.

[0081] Table 1

[0082] FIG. 5 depicts a bar graph illustrating the relative importance of the eight features: lymphocyte density in epithelium, macrophage density in stroma, fibroblast density in stroma, mitotic cell density in epithelium, tumor cell density in epithelium, lymphocyte density in stroma, macrophage density in epithelium, and endothelial cell density in stroma. FIG. 5 shows that that lymphocyte density in epithelium, fibroblast density in stroma, tumor cell density in epithelium, lymphocyte density in stroma, and macrophage density in epithelium are positively correlated with the likelihood of a patient responding to (or benefitting) from atezolizumab. Conversely, macrophage density in stroma, mitotic cell density in epithelium, and endothelial cell density in stroma are negatively correlated with the likelihood of a patient responding to (or benefitting) from atezolizumab. The distributions of the highest-importance features are shown in the graphs depicted in FIG. 6. Finally, FIG. 7 depicts a heatmap illustrating the correlation between the densities of tumor cels, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells, and the corresponding bulk ribonucleic acid (RNA) sequencing (RNAseq) signatures for tumor proliferation, T cells, B cells, macrophages, cancer associated fibroblasts (CAFs), and endothelial cells.

[0083] FIG. 8 depicts a block diagram illustrating an example of a computing system 800 consistent with implementations of the current subject matter. Referring to FIGS. 1-8, the computing system 800 can be used to implement the digital pathology platform 110, the imaging system 120, the client device 130, and / or any components therein.

[0084] As shown in FIG. 8, the computing system 800 can include a processor 810, a memory 820, a storage device 830, and an input / output device 840. The processor 810, the memory 820, the storage device 830, and the input / output device 840 can be interconnected via a system bus 850. The processor 810 is capable of processing instructions for execution within the computing system 800. Such executed instructions can implement one or more components of, for example, the digital pathology platform 110, the imaging system 120, and the client device 130. In some example embodiments, the processor 810 can be a single-threaded processor. Alternately, the processor 810 can be a multi -threaded processor. The processor 810 is capable of processing instructions stored in the memory 820 and / or on the storage device 830 to display graphical information for a user interface provided via the input / output device 840.

[0085] The memory 820 is a computer readable medium such as volatile or non-volatile that stores information within the computing system 800. The memory 820 can store data structures representing configuration object databases, for example. The storage device 830 is capable of providing persistent storage for the computing system 800. The storage device 830 can be a solid state drive, a floppy disk device, a hard disk device, an optical disk device, or a tape device, or other suitable persistent storage means. The input / output device 840 provides input / output operations for the computing system 800. In some example embodiments, the input / output device 840 includes a keyboard and / or pointing device. In various implementations, the input / output device 840 includes a display unit for displaying graphical user interfaces.

[0086] According to some example embodiments, the input / output device 840 can provide input / output operations for a network device. For example, the input / output device 840 can include Ethernet ports or other networking ports to communicate with one or more wiredand / or wireless networks (e.g., a local area network (LAN), a wide area network (WAN), theInternet).

[0087] In some example embodiments, the computing system 800 can be used to execute various interactive computer software applications that can be used for organization, analysis and / or storage of data in various formats. Alternatively, the computing system 800 can be used to execute any type of software applications. These applications can be used to perform various functionalities, e.g., planning functionalities (e.g., generating, managing, editing of spreadsheet documents, word processing documents, and / or any other objects, etc ), computing functionalities, communications functionalities, etc. The applications can include various add-in functionalities or can be standalone computing products and / or functionalities. Upon activation within the applications, the functionalities can be used to generate the user interface provided via the input / output device 840. The user interface can be generated and presented to a user by the computing system 800 (e.g., on a computer screen monitor, etc.).

[0088] One or more aspects or features of the subject matter described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs, field programmable gate arrays (FPGAs) computer hardware, firmware, software, and / or combinations thereof. These various aspects or features can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship ofclient and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0089] These computer programs, which can also be referred to as programs, software, software applications, applications, components, or code, include machine instructions for a programmable processor, and can be implemented in a high-level procedural and / or object- oriented programming language, and / or in assembly / machine language. As used herein, the term “machine-readable medium” refers to any computer program product, apparatus and / or device, such as for example magnetic discs, optical disks, memory, and Programmable Logic Devices (PLDs), used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor. The machine-readable medium can store such machine instructions non-transitorily, such as for example as would a non-transient solid- state memory or a magnetic hard drive or any equivalent storage medium. The machine-readable medium can alternatively or additionally store such machine instructions in a transient manner, such as for example, as would a processor cache or other random query memory associated with one or more physical processor cores.

[0090] To provide for interaction with a user, one or more aspects or features of the subject matter described herein can be implemented on a computer having a display device, such as for example a cathode ray tube (CRT) or a liquid crystal display (LCD) or a light emitting diode (LED) monitor for displaying information to the user and a keyboard and a pointing device, such as for example a mouse or a trackball, by which the user may provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. Forexample, recurrent provided to the user can be any form of sensory recurrent, such as for example visual recurrent, auditory recurrent, or tactile recurrent; and input from the user may be received in any form, including acoustic, speech, or tactile input. Other possible input devices include touch screens or other touch-sensitive devices such as single or multi-point resistive or capacitive track pads, voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices and associated interpretation software, and the like.

[0091] In the descriptions above and in the claims, phrases such as “at least one of’ or “one or more of’ may occur followed by a conjunctive list of elements or features. The term “and / or” may also occur in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradicted by the context in which it used, such a phrase is intended to mean any of the listed elements or features individually or any of the recited elements or features in combination with any of the other recited elements or features. For example, the phrases “at least one of A and B;” “one or more of A and B;” and “A and / or B” are each intended to mean “A alone, B alone, or A and B together.” A similar interpretation is also intended for lists including three or more items. For example, the phrases “at least one of A, B, and C;” “one or more of A, B, and C;” and “A, B, and / or C” are each intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together.” Use of the term “based on,” above and in the claims is intended to mean, “based at least in part on,” such that an unrecited feature or element is also permissible.

[0092] The subject matter described herein can be embodied in systems, apparatus, methods, and / or articles depending on the desired configuration. The implementations set forth in the foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects relatedto the described subject matter. Although a few variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations can be provided in addition to those set forth herein. For example, the implementations described above can be directed to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of several further features disclosed above. In addition, the logic flows depicted in the accompanying figures and / or described herein do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Other implementations may be within the scope of the following claims.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method, comprising: receiving an image depicting a tissue sample; identifying one or more tissue compartments present in the image of the tissue sample, wherein the identifying the one or more tissue compartments includes generating a compartment map identifying one or more pixels in the image depicting the one or more tissue compartments; identifying one or more cells present in the image of the tissue sample, wherein the identifying the one or more cells includes generating a cell map identifying one or more pixels in the image depicting the one or more cells; determining, based at least on the compartment map and the cell map, a density -based pathology signature for the image of the tissue sample, wherein the density -based pathology signature includes a density of each cell type of a plurality of cell types present in the one or more tissue compartments; and applying a response computation model to determine, based at least on the density-based pathology signature, a predictive output indicative of a likelihood of a patient associated with the tissue sample responding to an anticancer therapy.

2. The method of claim 1, further comprising: applying a tissue segmentation model to identify the one or more tissue compartments present in the image of the tissue sample.

3. The method of claim 2, wherein the tissue segmentation model identifies the one or more tissue compartments by at least assigning, to each pixel in the compartment map, a label identifying a type of tissue compartment depicted by the pixel.

4. The method of any of claims 1 to 3, further comprising: applying a cell segmentation model to identify the one or more cells present in the image of the tissue sample.

5. The method of claim 4, wherein the cell segmentation model identifies the one or more cells by at least identifying one or more pixels depicting a nucleus of each cell.

6. The method of any of claims 1 to 5, wherein the density of a cell type present in a tissue compartment comprises a ratio of a quantity of cells of the cell type present in the tissue compartment relative to a geometric area of the tissue compartment.

7. The method of claim 6, further comprising: determining, based at least on the compartment map, the geometric area of the tissue compartment; and determining, based at least on a superimposition of the compartment map and the cell map, the quantity of the cells of the cell type present in the tissue compartment.

8. The method of any of claims 1 to 7, wherein the predictive output includes a cellular composition metric, and wherein the cellular composition metric comprise a continuous value indicative of the likelihood of the patient responding to the anticancer therapy.

9. The method of claim 8, further comprising: classifying, based at least on whether the cellular composition metric of the patient satisfies one or more thresholds, the patient as a responder or a non-responder of the anti cancer therapy.

10. The method of any of claims 1 to 9, wherein the density -based pathology signature comprises a vector populated by a plurality of values corresponding to the density of each cell type of the plurality of cell types present in the one or more tissue components.

11. The method of any of claims 1 to 10, wherein the density -based pathology signature includes one or more of lymphocyte density in epithelium, macrophage density in stroma, fibroblast density in stroma, mitotic cell density in epithelium, tumor cell density in epithelium, lymphocyte density in stroma, macrophage density in epithelium, or endothelial cell density in stroma.

12. The method of any of claims 1 to 11, wherein the one or more tissue compartments include tumor stroma and tumor epithelium.

13. The method of any of claims 1 to 12, wherein the plurality of cell types include tumor cells, mitotic cells, lymphocytes, macrophages, fibroblasts, and endothelial cells.

14. The method of any of claims 1 to 13, wherein the image comprises at least a portion of a stain-treated whole slide image (WSI).

15. The method of any of claims 1 to 14, further comprising: training, based at least on one or more sample images of tissue samples, the response computation model.

16. The method of claim 15, wherein the training the response computation model includes generating a sample compartment map identifying one or more tissue compartments present in each sample image, generating a sample cell map identifying one or more cells present in each sample image, determining, based at least on the sample compartment map and the sample cell map, a sample density-based pathology signature for each sample image, applying the response computation model to determine, based at least on the sample density-based pathology signature included with each training sample, a sample predictiveoutput, and adjusting one or more parameters of the response computation model to reduce a discrepancy between the sample predictive output generated by the response computation model for each training sample and a corresponding ground-truth label included with the training sample.

17. The method of claim 16, wherein the training the response computation model includes applying a tissue segmentation model to generate a sample compartment map identifying one or more tissue compartments present in each sample image, and applying a cell segmentation model generating a sample cell map identifying one or more cells present in each sample image.

18. The method of any of claims 16 to 17, wherein the sample predictive output of the response computation model includes a sample cellular composition metric.

19. The method of claim 18, further comprising: determining one or more thresholds for differentiating, based at least on the sample cellular composition metric, between two or more patient subgroups.

20. The method of claim 19, wherein the two or more patient subgroups include a high-metric patient subgroup for patients expected to benefit from the anti cancer therapy and a low-metric patient subgroup for patients not expected to benefit from the anticancer therapy.

21. The method of claim 20, wherein the one or more thresholds are determined to maximize a hazard ratio (HR) between the high-metric patient subgroup for responders of the anticancer therapy and a high-metric patient subgroup for responders of an alternate therapy.

22. The method of any of claims 20 to 21, wherein the one or more sample images areassociated with patients treated with the anticancer therapy, and wherein the one or more thresholds are determined based at least on response data for patients treated with an alternate anti cancer therapy.

23. The method of claim 22, wherein the anticancer therapy comprises an immunotherapy and the alternate anti cancer therapy comprises chemotherapy.

24. The method of any of claims 1 to 23, wherein the anticancer therapy includes at least one of an immunotherapy, surgery, radiation therapy, chemotherapy, hormone therapy, and targeted therapy.

25. The method of any of claims 1 to 24, wherein the anticancer therapy comprises atezolizumab.

26. A system, comprising: at least one data processor; and at least one memory storing instructions, which when executed by the at least one data processor, result in operations comprising the method of any of claims 1 to 25.

27. A non-transitory computer readable medium storing instructions, which when executed by at least one data processor, result in operations comprising the method of any of claims 1 to 25.

Images